Does one need to use the article in this case?
No. "To have dinner" is the idiom.
While "to have a dinner" is not ungrammatical, I have been struggling to concoct a circumstance in which I might say that, and it's hard. The only one I have been able to come up with is with a different meaning of the word "dinner", viz a formal event at which one eats (i.e. a banquet). Then we might say "Our society has a dinner every year on the occasion of ... ".
But I can't come up with a likely use with the ordinary meaning of "dinner".