Playing musical people in this context means playing someone you are not to adjust to the social requirements.
Musical from wiktionary:
by extension, any situation where people repeatedly change positions
The answer comes from the previous paragraph.
"Well, I do hope you manage to loosen up a litte before too much longer," he said. "I mean, the whole idea of being here is to enjoy oneself. It would be such a shame if one person allowed her inhibitions to ruin the atmosphere for everyone else, wouldn't it"
Basically he is saying to Lynn to not be a party pooper, and play along with the rest of the guests.
To which she responds:
"You never said anything about playing musical people."
The follow-up paragraph in which he responds confirms that it's about adjusting to the different kind of people at the party:
"Oh dear, I do hope you're not going to start preaching any middle-class morals."