Is there a single word or a phrase for conspiratorially exchanging glances when someone does something inappropriate?
They exchanged looks.
It doesn't specifically convey judgment of something inappropriate, just a silent exchange between people. But is a common use of the phrase that two people "exchange looks" to keep a mutual judgment silent.
"complicit glances", not an idiom but may describe what you are looking for.
complicit (adj) "Involved with others in an activity that is unlawful or morally wrong" ODO
"Every now and then they would exchange complicit glances."
You could also use the more contemporary expression "side eye".
Although I agree with Mitch's "a knowing glance", I think I prefer the lesser-syllabic "a knowing look".