There's nothing grammatically wrong with your in OP's example (but the comparison should be to all your base are belong to us! :). Consider...
We look forward to your arrival
We look forward to you arriving
...where obviously both are fine. In the first case we've got a noun, and in the second it's a gerund. We can substitute spending time with us in either of those roles, so both your and you are grammatically valid. The reason OP often sees your on invitations, etc., is simply that it's rather more formal in tone, and thus seems more appropriate to that context.