Oh my!
In the above example, to me, "oh" seems to suggest one should pronounce "o" as a short vowel, whereas "o", seems to suggest one should pronounce "o" as a long vowel. In other words, I would expect it to be spelled as:
O my!
I would expect "oh" to be used in this example, meaning to express "oops":
Oh o!
... or used an an expression of excited amazement, when watching beautiful fireworks for instance, like this:
Oh!
So why is "Oh my!" spelled this way?
There is no difference between the pronunciation of oh and o; in both the cases, the pronunciation is /oʊ/.
Oh is used to express surprise, anger, disappointment, or joy; it is also used when reacting to something just said.
Oh, shut up.
O is also an archaic spelling of oh; it was also used before a name in direct address.
Give peace in our time, O Lord.
I think the other answers miss mentioning that the spelling O is most often used in poetic language before a name/noun in direct address, not necessarily just in a religious context.
O CAPTAIN! my Captain! our fearful trip is done;
The ship has weather’d every rack, the prize we sought is won;
The port is near, the bells I hear, the people all exulting,
While follow eyes the steady keel, the vessel grim and daring:
But O heart! heart! heart!
O the bleeding drops of red,
Where on the deck my Captain lies,
Fallen cold and dead.
— Walt Whitman
It is more common to see the spelling Oh as an interjection or expression of surprise, although the spelling O can be used and is not incorrect, even if it is falling out of favour. Note that there are some phrases where the spelling Oh is nearly always preferred.
Oh yeah? You want to make something of it?
'Why' questions are hard in spelling. The answer could be something like 'Noah Webster said so' or 'in the primary dialect of Wessex in AD XXXX, all words ending with a certain pronunciation were spelled a certain way (and those are the only source of such spellings in Modern English)', or 'somebody just made it up one day, and everybody started using it from then on' (oh, that last example is too much like the first).
Unfortunately nothing like that seems to be the case for 'oh'. It looks like it just is. There is a history to it (as others have noted) where 'oh' does alternate with 'o'. But it's just that 'oh' seems to be more popular nowadays. And there doesn't seem to be any traceable evidence that shows an event or historical trend explaining it.
According Wikipedia (and other sources on the web) the word "O" is an indicator of a vocative phrase. Some languages, including the Celtic languages, have a true vocative case but English does not. The Wikipedia article includes this paragraph:
Historically, and in poetic or rhetorical speech, vocative phrases in English were prefaced by the word O, as is often seen in the King James Version of the Bible: "O ye of little faith" (in Matthew 8:26). Another example is the recurrent use of the phrase "O (my) Best Beloved" by Rudyard Kipling in his Just So Stories. The use of O may be considered a form of clitic and should not be confused with the interjection Oh (The Chicago Manual of Style, 15th edition, section 5.197). However, as the Oxford English Dictionary points out, "O" and "Oh" were originally used interchangeably. With the advent of "Oh" as a written interjection, however, "O" is the preferred modern spelling in vocative phrases.
I think this explains why "oh" is spelt with an "h"
I can't agree that o and oh are the same in meaning. 'Praise the Lord, o my soul' is a vocative command to my soul, 'oh' would be most out of place as it would indicate surprise.
O Lord! Why indeed? In fact it's not always spelt with an "h"...
...but in general the presence of the "h" does tend to make it more readable in other contexts.
"Oh" is an interjection expressing any number of emotions, including surprise, exasperation, desire, and gratitude.
"O" begins an address to a person or to God (thus it performs the same function that the vocative case performs in certain languages, for example in Latin; hence the term "Vocative 'O'").
A cute twist: in Othello's lament, "O Desdemona!", the "O" may be read in whichever sense one wishes, either as an address to his slain wife or as a lament for her.
"O'" with an apostrophe can either serve as a contraction of the word "of" (e.g. "8 o'clock", "cat o' nine tails") or serve to create a patronymic in English renditions of Irish Gaelic (O'Dwyer = Ó Dubhuir = Son of Dubhuir).
Finally, "O" may stand for the letter itself or for the number "0" (007 is pronounced, "Double O Seven"; in this last capacity, "Oh" is often substituted: "a rendezvous at 0800" may be read, "a rendezvous at zero eight hundred hours" or, informally, as, "a rendezvous at Oh eight hundred hours").
The choice to add the "h" would seem to be either an attempt to distinguish between the various meanings of the word or a simple fluke in orthography. I can't detect any difference in the pronunciation.
I feel that O and Oh are the same both in meaning and pronunciation so "Oh no" and "O no" are the same. The only case you will have to use Oh is that if the next word starts with vowel as in "Oh o". "O o" is confusing and doesn't please your eyes.
