Despite the advice of style guides (they're just an opinion)
Well, they're not just an opinion; they're an informed opinion.
Now, that informed opinion can range from restating a basic rule of standard English, through picking between several equally valid choices in the interests of consistency (which is to be valued in itself), to repeating zombie rules with absolutely no justification.
Even in the last case though, it's still an informed opinion, even if I might argue also a misinformed one.
And if you are going to disagree, fine, but it should likewise be an informed disagreement.
In order to make an informed disagreement you need to have a very clear idea why you are choosing otherwise, which ideally includes a clear idea about why they on their part were making the recommendation they did.
This goes whether one is going against the guideline constantly, or just in a particular case.
(Consider, Philip Roth and Ernest Hemingway used sentences that are respectively much longer or much terser than a most guides would recommend, and each produced masterful work in doing so, but do you not think they would know why guides would recommend they do things differently and by extension precisely why they chose to write as they did).
One of your comments on another answer makes me think you may be barking up the wrong tree:
Plus, the state and year are essential information. Why would we set them off with commas?
This suggests you might think that the use of commas here is parenthetical. If so, you are incorrect on two counts.
The first is that parenthetical clauses are not for "non-essential" information. They are for information that is additional to the main clause syntactically, but that doesn't mean it's not essential (the thing to do with non-essential information is to delete it). It could even be the most important piece of information:
My 40-year-old cousin, who I despise, is visiting me tomorrow.
Here "40-year-old" is almost certainly less vital as information than "who I despise". It's syntactically an addition, but it's essential to understanding why I'm bothering to comment on the matter.
And we very often use parenthetical clauses precisely because without them either our reason for caring about the statement will be unknown, something will be unclear to the reader, or for other reasons where the parenthetical clause is essential to understanding.
The second, is that these are not parenthetical clauses. (One can reasonably also read the "New Jersey" as parenthetical, since it clarifies the statement that we would have it if was just "…Elizabeth rock band…").
Rather, in each of these the second comma exists to clarify the use of the first comma.
The use of commas with place names ("Elizabeth, New Jersey") and dates ("June 6, 1999") is a specific use of commas that differs from most of the ways commas are used within sentences.
And as such, we can confuse them with those more common uses, particularly as separating clauses. The second comma after the rest of the comma-divided term sets it off precisely so we can see the comma-divided term (place-name or date) and read it as such.
Indeed, I certainly read your first example as such—that "The Elizabeth," was one clause, and the rest another—and the form jarred. I fared better with your later examples, but then I already knew how you were playing hide-the-comma there, so I could just mentally add them back in.
It would be nice if we'd a weaker punctuation symbol in regular use with English to use here, but we don't.
So I'd certainly put in those second commas, in each case (or to be honest, I'd likely miss them some of the time and hopefully catch them on review, but that's a slip not a decision).
By all means, if you're using a style-guide for advice rather than writing for publication, break whatever rule you desire, up to and beyond colouring every instance of the word house blue like Mark Z. Danielewski did, but do try to know why the guide is saying what it is saying: Your decision to forgo the second comma is a decision to forgo the means by which we clarify the type of comma the first comma was; once you understand that, then the final decision is up to you.
If it is for publication, and it's the set style, then your editor will just hate you every time they add a comma back in, and hate you more if you go to print and they missed one. Breaking rules in this case requires much stronger justification.