It all depends upon the context and intonation.
Neither phrase is inherently conceited or pedantic.
A delicious smell was wafting through an open door. Naturally, I stepped inside to find the source of the heavenly aroma.
There is nothing inherently wrong with that statement. It doesn't carry any sort of negative connotation toward the listener.
And, you can always say of course without being haughty.
I always stop on my way home from work to buy fresh bread. Of course I usually feel guilty afterwards for the extra calories and carbs, but it has never stopped me.
It is the usage where you are adding it in front of a piece of information to barb your listener that sounds more obnoxious.
Naturally you would say that!
Of course Bill thought you should be the one to pay for the damages.
Naturally, people who buy red cars are just trying to be ostentatious.
Of course, if you eat tuna you are a dolphin killer.
So, of course you can. Just make sure you use it naturally to avoid alienating your listener.