By that I refer to the sound people make when they're thinking. Most people write "hm" nowadays, so they may not know of this, but traditionally, people wrote it as "h'm". The apostrophe can't indicate elision, since "h'm" is an onomatopoeia, so why is it there?
Yes, as FF suggests, a spelling like "h'm" might be thought proper by some people, if they believe -- like all proper spelling, this is a matter of faith, though not morals -- that "h'm", a breathy voiced nasal resonant with voiceless onset used to mark a brief pause for ostensible thought, is actually short for hum, a word describing resonance in general, including breathy nasal resonance.
Of course, why not? Surely such people exist, and for them that's a proper spelling. Certainly hum is a part of ho, hum, another phatic phrase mimicking the headshake or eyeroll of boredom -- though that is close to the opposite of what "h'm" means. Still, everybody makes up their own language, and that's the way they made theirs up.
For others, who see no more connection between "h'm" and hum than they do between bug and humbug or pole and polar, the apostrophe (being silent in any event and therefore superfluous in onomatopoeia, as the OQ points out) may be dropped without penalty.
Punctuation, especially of apostrophe's, is very chaotic and cannot be depended on in general; in onomatopoeia it is even less valuable. In fact, since search engines ignore punctuation, chaotic apostrophization seems to incur no penalty any more.