... so I just let them continue
without causing a "fuss",
but it still "bugs me" inside.
Your situation seems like a good time to bite your tongue.
bite your/one's tongue (phrase)
Make a desperate effort to avoid saying something.
I often felt like I had to bite my tongue and take deep breaths to avoid berating them for their complete lack of common sense or
perspective.
It is certainly not the kind of situation where confrontation is advisable - and that is why our seething guide bites his tongue and
suppresses the urge to curse at their thoughtless conduct.
Lexico
To stop yourself from saying something that you would really like to
say:
I wanted to tell him exactly what I thought of him, but I had to bite my tongue. Cambridge
He chuckled, hauled out the ironwork, and set to work hooking up the
printer to the laptop. He was doing it all wrong, but I bit my tongue.
I'd realized long ago the futility of telling Michael how to do
computer tasks, so I left him to figure it out and want back to
selling my dusty ironwork. Donna Andrews; Revenge of the
Wrought-Iron Flamingos (2006)
I had to bite my tongue. I would just have said there was a dead
body in the woods. It was far too complicated to explain what was
actually happening. For now, it just needed something to grab their
attention, to make them respond. Jon Wilmot; The Trail (2014)
As a teacher, it is hard to bite your tongue and let a reader
continue reading when he or she makes a mistake, but it is the only
way struggling readers will develop the monitoring system good readers
already have. When a reader makes a mistake, wait until he or she
reaches the end of the page, and then intervene. Timothy Rasinski;
Rebuilding the Foundation (2011)