It has always seemed obvious to me: Three-teen and five-teen are harder to say than thirteen and fifteen.
People are lazy.
laziness really doesn't make the pronunciation easier metathesis does not seem as natural.
(And, really, it would be the same with third and fifth. Try saying, once, tooth, threeth, fourth, fiveth.)
Some people will note the middle English "fifthe", and object, but I'll point out that the cardinal is voiced as far back as middle English. You have to go back to old English to get an unvoiced "fif" as the cardinal number, and then you have to ask why middle English became voiced only in the cardinal.
It is true that the obvious always assumes the current context, but the current context is derived from the old context, and the principle of laziness applies in the old context at least as much as it applies now.
I am doing too much reading between the lines in the above.
Here is the entry for thirteen in the Online Etymology Dictionary:
Summary: it was spelled "thrittene" in Middle English, and shifted (metathesis) in the late 14th century. While the metathesis is recorded, the cause or reason is not. (How can we travel back in time and take a survey?)
Asserting laziness is probably oversimplifying. The impulse to change things for the sake of change is also a probable cause, and there are other known causes as well, such as linguistic fashion. Different people would have had different reasons, and the generalization of the metathesis is a statistical event, a "summation of many reasons.
So I'll apologize for asserting laziness.
fifteen and five in the Online Etymology Dictionary,
and five in Wiktionary
Note that five was five in Middle English, but fif in Old English. This is not metathesis, but the causes of the pronunciation change are not clearly stated in the above or anything else I have handy today. (I should check the Online Oxford, but it's one in the morning. I have other things I should be doing.)
(end second afterthought)