We have: "it takes two to tango", "two is company; three is a crowd", etc... Are there any similar sayings that refer to four people?
No. Not a very common one, at least.
The reason there are expressions for one, two, and three is because of the special status accorded to a couple in the culture where the expression originated.
So: there are lots of "one" expressions indicating that one person is not enough to have a one-on-one relationship. "Table for one." "One is the loneliest number."
There are "two" expressions, like "It takes two to tango," indicating that "two" is enough to have a one-on-one relationship.
There are "three" expressions, like "third wheel" or "three's a crowd," indicating that those people not in the one-on-one, couples, relationship, are extraneous. Three in this case stands for "more than two," and no special "four" or "five" expressions are needed.
The only think I can think of that even remotely qualifies is "fourth wheel," probably a back-formation from "third wheel." If you have three people, a couple and a single, at an event together, the couple may try to set up the single with a "fourth wheel" to keep him or her from being a "third wheel."
People talk about trying to find a "fourth" for bridge or golf, but that doesn't sound like what you're talking about. Similarly, the "Fourth Estate" is a group of people, but again, that's more specific than your examples.
To summarize: when it comes to idioms, any person more than two is a third, no matter how many of them there are.