The word "Gentleman" from that era refers to a man who does not himself have to work for a living. So "ladies and gentlemen" is a way of flattering your audience by implying that they are of a high social class (most likely higher than most of them actually are).
"Gentlewoman" doesn't quite have the same cachet, as it was far more common for a woman to not have to work than a man ("work" in this case meaning seeking paid employment from sources outside the family home). It takes far less wealth to pull this trick off than it does to have the ability for noone in the household to work. The best way to imply that a woman is of this particular social strata is not to say that she doesn't work, but rather than she is married to a man who doesn't work. Hence, "Lady".