Having been a non native speaker, several times I had been asked to write my name IN VERNACULAR or IN MOTHER TONGUE during my college days. I couldn't tell apart the subtle difference and thought of them to be synonymous those days.
According to the wikipedia article, VERNACULAR is the language spoken by a group of people inhabiting a particular region/ country which is regarded as of low - status. https://en.m.wikipedia.org/wiki/Vernacular
Mother tonge is in some countries the language/ dialect of one's ethnic group rather than the first language- the one exposed to from his birth.
https://en.m.wikipedia.org/wiki/First_language
Thus, to reach a conclusion,
1.The definitions of the both overlaps to a certain degree.
- Definition of mother tongue seems little VAGUE.
Which out of the two is MORE REFINED & FORMAL to denote one's first language? I ask this on the grounds:-
1.The term vernacular has derogatory connotation.
- on the contrary, I think that mother tongue is a SUBSET of vernacular and not vice versa.( Mostly ones mother tongue is the product of his vernacular.)
Also, I personally of the opinion that using the word vernacular indicates imperialistic approach of the user as witnessed by the excerpts given below. (I m taken by the retro- noir- word "ANGLO- VERNACULAR").