The difference between the who/whom debate and the preposition-ending debate is that the former has its root in a long tradition of English grammar, while the latter arose from the editorial labours of an extremely picky generation of classicists.
The use of who/whom as distinct subject/object pronouns (like thou/thee) has largely atrophied, but until barely a century ago it was prevalent and common. It was second nature to most native speakers to use them appropriately. The same distinction was made in the Saxon and Norman (Germanic and Romance) languages that contributed to English, and some modern Germanic and Romance languages still employ distinct subject and object relative pronouns. However, since their use is understood but largely ignored in modern English, they are now considered an optional nicety.
As for avoiding ending a sentence with a preposition, there was no such rule in medieval English. It was only the belief of those linguists who were educated in Latin and convinced that Latin is the true root of English (and therefore English grammar should conform with Latin) that caused this phenomenon to arise, in spite of there being no evidence of such a rule in earlier English writings. However, the myth prevailed to the point where it was accepted as "correct" grammar, and the resultant torturous grammatical constructions that have been perpetrated in the name of correctness make one feel ill.
So, in summary:
- Who are you talking to?
This is unambiguous and commonly used, and therefore mostly acceptable.
- Whom are you talking to?
This is a pleasant nicety, but not necessary. Certainly acceptable.
- To whom are you talking?
This is "correct", but will get you a filthy look and possibly a kick in the nuts in all but the most snobbish company.
For a really damning and amusing exposition of the whole issue, I strongly recommend Bill Bryson's excellent book, The Mother Tongue.