In the case of /'wɛlθi/ vs. /'wɚði/, we're looking at more than one hypothetical rule, and the more I look into it, the more it screams ?exception. Of course, much like OP says, English phonetic rules are kind of made to be broken and/or drive ESL learners insane.
Judging from the excellent data set collected by @John, I believe it is a combination of two phonetic rules, one based on manner of [l_] vs. [ɹ_], and one based on voicing of the fricatives when followed by /i/ (I won't go into other vowels, but there is a pattern).
First rule: left over from OE, unvoiced fricatives to voiced when followed by a (high) vowel.
- breath vs. breathing
- grief vs. grieving
- worth vs. worthy
Second rule: R-vocalization and r-colored vowels influence voicing in /'wɚði/. The influence of /l/ on the interdental fricative, however, is not sufficient to ease into a comfortable voiced pronunciation.
The slight difference in /l/v and /ɹ_/v -- liquid vs. rhotic -- may be enough to trump the First rule based on the influence of the rhotic /ɹ_/ on the fricative. Therefore we see a voiceless "th" when preceded by /l/.
That there are so few -orthy words (c.f. all the words that can add "-worthy" to make an adjective), but with the limited data set and with the very relevant OED citation from @Alex with "worthy" making an appearance long before "-lthy," later to be followed by "wealthy..." @John, I think I see where you're going....
Both /l/ and /r/ are alveolar and located very close in the mouth, but with the slight difference in pronunciation when it comes to approximates. Interesting to note that in languages like Japanese, which does not differentiate these phonemes, many native speakers do not notice the difference between the pronunciation of /l/ and /ɹ/ in English.
Could this be the case of the Great Vowel Shift?
Since the meanings of the two words are so closely related, I checked the etymology of wealth:
From Middle English welth, welthe, weolthe ("happiness, prosperity"), alteration (due to similar words in -th: compare helth ("health"), derth ("dearth")) of wele ("wealth, well-being, weal"), from Old English wela ("wealth, prosperity"), from Proto-Germanic *welô (“well-being, prosperity”), from Proto-Indo-European *wel- (“good, best”), equivalent to weal + -th. Cognate with Dutch weelde ("wealth"), Low German weelde ("wealth"), Old High German welida, welitha ("wealth") Related also to German Wohl ("welfare, well-being, weal"), Danish vel ("weal, welfare"), Swedish väl ("well-being, weal"). from here
From worth or wurth, from Old English weorþ, from Proto-Germanic werþaz (“towards, opposite”) (the noun developing from the adjective). Cognate with German wert/Wert, Dutch waard ("adjective"), Swedish värd.
Keep in mind that "þ" (thorn) evolved into "th." One might just toss the potential rules at this point.
Always handy: IPA chart