If you call someone a grammar nazi, there's a decent chance that they will be offended. This is because you are accusing them of being unreasonable. If you called them the grammar police, I would expect a nearly identical reaction.
If someone hears you call someone else a grammar nazi, it would be very strange for them to be offended at the mere use of the term.
It sounds like there are some regional differences in how this term may be perceived, especially in Europe. To be clear, I'm an American and I'm referring to American usage. I do not however think that the criticism that this usage must be do to ignorance is at all fair. I've been to Germany, where I visited plenty of historical Nazi sites including the Nuremberg Documentation Center and Dachau. I'm familiar with what went on over there. The usage of nazi in the sense of someone who is simply authoritarian does indeed trivialize the word, which should be insulting to the Nazis, not their victims.