Although perhaps not as authoritative as Tom Lehrer (See terdon's comment under OP ), here are some examples from text books:
" Place value - [...] a particular position in a place-value notation, for example, units, tens, hundreds [...] " 1)
And, though referring to the right side of the decimal place:
" Decimal - [...] The first position to the right of the point (representing tenths) is [...] A decimal fraction is [...] a number of tenths, plus a number of hundredths, plus a number of thousands, etc. [...]" 2)
And from a much older dictionary my father had (for which he paid a whopping USD $0.60 [ah, that'd be six tenths of a dollar :-P ] ), we have:
"Decimal System - [...] 20,349 = 9 units + 4 tens + 3 hundreds + 0 thousands + 2 tenthousands. [...]" 3)
[Interesting, the editor parser flags "tenthousands" and offers tries to pull it apart to 'ten thousand' or hyphenate to ten-thousands. I guess our dear SE. programmers have not read reference #3. ]
[EDIT - clarification based on side discussions]
The problem, or confusion here is that these units (tens, hundreds, etc.) can be singular or plural units (similar to Fish), but also can be like adjectives. Perhaps a linguist can chime in here with the correct term (quantifier adjective, a "naming noun" (see below) ?).
So, consider that
2 in the hundreds place
is exactly the same structure as
2 in the red place
We'd never say "2 in the red's place" - red is not a noun, neither is "hundreds" here. The units are labels modifying "place". What place? The hundreds place. The hundreds go in that place, but they don't possess it.
Consider it another but similar way. I have two boxes, one red, one blue. And I have a bunch of marbles. I'd say "I put 2 marbles in the red box and 4 in the blue [box]" I'd never say "marbles in the box belonging to red, " likewise not "marbles in red's box."
It's the same with hundreds - we put 2 units of something into the place holder which has these unit labels attached. Part of the confusion is that they all end in "s" ... so I would not be surprised if an author were tempted to think them as nouns, but I have not seen it yet.
Does that make sense?
[EDIT 16 SEPT 13 Found another reference. Though just a blog, it gives a nice simple answer.]
First and foremost, the apostrophe should NEVER EVER EVER be used to
show a plural (that is, any naming word [a noun] with –s on the end).
For example, noodles, chocolates, flowers. This is also true of
numbers and dates – tens, hundreds, thousands (10s, 100s, 1000s; 60s,
The –s is added to show that there is more than one of that particular
thing, end of. No apostrophe needed, thanks.
The ONLY time you add an apostrophe to a plural noun is if you’re
showing possession. [...] 4)
Note that the so-called debate ( which I was called out on :)) stemmed from thinking the units, tens, etc were possessive nouns modifying "place", i.e., "the ten's (or tens') place." This makes as much sense as "the red's box" or stranger, "the reds' box". Clearly, the problem arrises because of the words ending in S.
One final phrase which I hope will make this answer canonical enough:
The places where the units, tens, and hundreds go.
(They don't own those places :))
1) Harper Collins Dictionary of Mathematics, New York, 1991. pg. 450
2) Penguin Dictionary of Mathematics, London, 1989. pg. 104
3) The Universal Encyclopedia of Mathematics, New American Library of Literature, New York, 1964, pg. 163