The confusion here stems from the fact that "Celsius" is not the name of the unit, rather it is a classifier for the unit "degree," of which there is also "Fahrenheit." Because "Celsius" and "Fahrenheit" are names (Anders Celsius and Daniel Fahrenheit), they remain capitalized.
However, a "kelvin" is a unit just like "meter" or "gram," which, despite being named after someone (Lord Kelvin), is spelled lowercase, just like all SI units named after someone (c.f. "watt," "hertz").
Wikipedia's entry on SI units explains this clearly:
Names of units follow the grammatical rules associated with common nouns: in English and in French they start with a lowercase letter (e.g., newton, hertz, pascal), even when the symbol for the unit begins with a capital letter. This also applies to "degrees Celsius", since "degree" is the unit.
(It goes on to explain that they are capitalized in German, because that's how German works. But your question isn't about German.)
Note that "degrees Kelvin" is not correct. Celsius and Fahrenheit are "degrees" because 100° C is not twice 50° C, nor is 100° F twice 50° F (in fact, that hardly makes sense).
But because 0 K is absolute zero, 100 K is twice 50 K, just like how 100 m is twice 50 m, which means the Kelvin scale is not a degree scale. (Oddly, note that "Kelvin" is capitalized here because it is functioning as an adjective modifying "scale," so is capitalized just like "American" would be. It's just the unit that isn't capitalized. Whew.)
(I'll confess, I don't entirely understand this myself, but that's how it was explained to me.)
Even though "kelvin" is not capitalized, its abbreviation is, to distinguish it from the SI prefix "kilo-," which is k: "kg," "km."
See also Wikipedia's entry on "Kelvin."