If you're typesetting SI units, it seems logical to follow the conventions of the Bureau international des poids et mesures. From the SI Brochure, §5.3.3:
The numerical value always precedes the unit, and a space is always used to separate the unit from the number. (…) The only exceptions to this rule are for the unit symbols for degree, minute, and second for plane angle,
″, respectively, for which no space is left between the numerical value and the unit symbol.
§5.3.7 goes on to say that “When it is used, a space separates the number and the symbol
In practice, it is quite common to see non-alphabetic units such as
°C typeset without an intervening space. I've never seen a rule that distinguished between single-letter units and longer units.
Note that the rule doesn't specify how wide the space should be. Some references recommend a normal inter-word space, while others recommend a thin space. In any case, the space is nonbreakable.
These rules need not apply to currencies, especially when they are written before the number. Specifying that single-character currencies don't take a space ($42, £42, €42) while multiple-character currencies do (AUD 42, A$ 42) doesn't feel completely outlandish, maybe that's what you remember?