I think it may depend on the education/reading level of the speaker. In English, there is nothing wrong with saying "We're having dinner chez the Bakers tonight," or even "... dinner chez Baker", and those usages are the one I come across the most. It wouldn't make sense to say "We're having dinner at chez the Bakers' tonight," when the chez would be redundant.
I've never heard anyone use it with a preposition at all, in fact, though I'll take your word for the fact that some do.
If they use the preposition "to", then they don't understand what "chez" is supposed to be doing in the sentence. "We're going to chez the Baker's tonight," again makes chez redundant.
I suppose some could say, "We're having dinner chez the Baker's house," when house would be redundant. Again, I've never heard it misused that way, but I grant that it probably happens.
If your friends happen to have the surname "Panisse", then there would be nothing wrong with saying "We're going chez Panisse tonight." In my other examples, I avoid using the word "go" because it's ambiguous: it refers both to a future tense construction, as in "I'm going to stay right here", and to travel, as in "Where are you going?" "We're going to the bowling alley."
If you'd like to edit your question to avoid the verb "to go" and proper nouns with the word "Chez" in them, it might result in fewer sidebars. On the other hand, it might result in the question becoming moot, also.
If you have other examples of cases you've observed where the speaker clearly meant "the house of...", as in (for example), "I just watched chez Bakers burn down," meaning, "I watched the Bakers' house burn down", that might help. But in my experience, the dictionaries are correct, and the speaker (in the case of the usage in this last example) would be mistaken.