Armed to the teeth is an expression where the idea behind it is, you are carrying so many weapons all over, that in order to fit them all in you had to carry one in your mouth, i.e. clenched between your teeth. (X)ed to the teeth would carry the idea of 'extremely prepared for' some event where you need (X).
For (X)ed to the hilt, the idea is comparing the (X) or whatever thing allows you to do or use (X) to a sword, where the area from the tip to the hilt is the entire usable, functional area, so you have made the absolute maximum usage possible of the thing in question.
Being mortgaged to the hilt means that you have gotten the largest mortgage possible, you are at the absolute limit of your creditworthiness, and you couldn't possibly carry any more debt without sinking.
Being mortgaged to the teeth would mean that you are prepared with more mortgages than any person would ever consider reasonable. I would have to assume that the one who is "mortgaged to the teeth" is prepared to issue mortgages, but honestly the "to the teeth" metaphor is such a bad fit for the idea of mortgages that it's hard to make sense of it.
More likely the intent behind the expression was something like mortgaged up to the eyebrows, which, much like to the hilt, would mean basically to the maximum extent possible or perhaps even a bit more, where the ability to pay off the debt is actually in question.