In the above example, to me, "oh" seems to suggest one should pronounce "o" as a short vowel, whereas "o", seems to suggest one should pronounce "o" as a long vowel. In other words, I would expect it to be spelled as:
I would expect "oh" to be used in this example, meaning to express "oops":
... or used an an expression of excited amazement, when watching beautiful fireworks for instance, like this:
So why is "Oh my!" spelled this way?