In this context you would use the word 'late'.
The late Elvis Presley...
Mrs Smith's late husband used to...
This means the person is deceased.
However, it's normally used when the person hasn't remarried. So my wife's late husband could be confusing, since if her husband's dead who are you?!
You could still use this phrase but for clarity I would recommend either
my wife's late former husband
my wife's late first husband
(Note the adjective order. Former late husband sounds like he's no longer dead and first late husband sounds like she's got more than one dead husband.)
Ex-husband is used so often as a phrase that I think if you said late ex-husband people would think they were divorced before he died. For this reason, late first husband might be better than late former husband. I think semantically late first husband is better as well, because he didn't technically stop being her husband when he died (otherwise the phrase late husband wouldn't be used to begin with).