"Oh" itself is an exclamation and can be followed by a comma, which could also be an exclamation mark; besides, you can make other exclamations using "Oh" as a starter ("Oh yeah!" could exemplify this). I would like to share some examples taken from the Oxford Advanced Learner's Dictionary:
1) Used when you are reacting to something that has been said, especially if you did not know it before:
‘I saw Ben yesterday.’ ‘Oh yes, how is he?’
Oh dear! What's happened now?
You can't come tonight? Oh well, see you next week then.
‘Emma has a new job.’ ‘Oh, has she?’
2) Used to express surprise, fear, joy, etc.:
- Oh, how wonderful!
- Oh no, I've broken it!
3) Used to attract somebody's attention:
- Oh, Sue! Could you help me a moment?
The position of the punctuation marks must vary as the intention (emphasis) of the speaker/writer varies; thus, "Oh, no! I've broken it!" and "Oh no, I've broken it!" are both correct.
Thank you for reading.