If you had gone to the football game yesterday, you should have invited me to go with you.
As you say, this sounds weird. The only way I can read the should is obligation, which doesn't fit with the counterfactual If you have gone.
If you had gone to the football game yesterday, you would have invited me to go with you.
This is grammatically fine, though perhaps an odd thing to say.
If you were going to the football game yesterday, you should have invited me to go with you.
This works as a non-counter-factual conditional (I think that means that it is not what you call a third conditional: this terminology of first, second, third conditionals appears to be taught only to ESL students), with, again, the "obligation" meaning of should. But in that reading you were going is not a past continuous: it is the past of the "intended future" form you are going.