I looked in 19th century slang dictionaries, and found different origins from those in other answers.
This entry, published 1890 in Slang and its analogues past and present. A dictionary, historical and comparative of the heterodox speech of all classes of society for more than three hundred years. With synonyms in English, French, German, Italian, etc.
GEEZER, subs. (popular). — An appellation, sometimes, but not necessarily, of derision and contempt; applied to both sexes, but generally to women. Usually, OLD GEEZER. For synonym, see WITCH.
I also found this entry, published 1889, in A dictionary of slang, jargon & cant, embracing English, American, and Anglo-Indian slang, pidgin English, tinker's jargon, and other irregular phraseology
Geezer (popular), wife, old woman. Dutch slang, geeze or geese, a girl, a mistress, vide GANDER. Also a man derisively.
He'd flirt and boat, but never wrote A note to his old geezer. — J.F. Mitchell: Jimmy Johnson's Holiday.
So those sources seem to indicate that the word originally came from Dutch slang for a mistress, and started to be applied to the wife and other old women, and was derisively applied to men.