The best way to ask this would be to give a set of examples.
You use a word not common in everyday usage, and someone asks you "who says that" in a negative fashion.
you work obliges you to move frequently between indoors and outside. You wear sunglasses. You don't bother to take them off when you move inside; a coworker tells you people who wear sunglasses inside are insert demeaning pejorative here
You wear a piece of clothing and someone asks or tells you something like "jesus who wears that anymore?"
Someone who is overly critical? the closest word or Idiom I have seen for these is either Cavil, or "hater". No one really knows the word Cavil, and haters is far too MTV for me to use in conversation without throwing up.
It doesn't necessarily need to be a single word, or idiom, but rather a way to respond to those overly critical people. the ones who try to put you down to make themselves feel better and/or who are overly concerned with behaviors of others that do not impact themselves at all.
