There might be exceptions to the statement in the title of your question, but I'm not going to quibble. The simplest reason for the lack or scarcity of word-initial ð in English (for words of all grammatical classes, whether nouns, adjectives or verbs, outside of a small closed set of "function words") is because there is no regular historical source for it. In Old English, the unvoiced sounds [s f θ] are believed to have been in complementary distribution with their voiced counterparts [z v ð]; the voiced consonants only occurred when they were both preceded and followed by other voiced sounds (such as between vowels, or after a voiced consonant and before a vowel) and the voiceless consonants occurred elsewhere (before and after voiceless consonants, at the end of words, and also at the start of words). The distribution in Old English may have been different depending on the dialect, and some dialects of Middle English are known to have had voiced initial [v] and [z] at least instead of [f] and [s]. However, while a handful of words with initial /v/ in Modern English come from this source (vat and vixen) I can't find any comparable words with /z/ or /ð/.
In the development up to Modern English, the Old English phonetic values of these fricatives were mostly preserved, but there was irregular voicing of some fricatives at the start or end of some commonly used words and suffixes. That's why we have /ð/ in this, that and them (as well as /v/ in of, and z in the plural suffix -(e)s). In fact, this change is still in progress to some extent; one word where it is incomplete is with (which is sometimes pronounced with voiceless /θ/, and sometime with voiced /ð/.)
In Modern English, the classes of nouns that start with /v/ and /z/ have been expanded due to loanwords from sources like French and Greek. But none of the main source languages for English loanwords have /ð/. (Modern Greek does, and Spanish has [ð] as a non-phonemic allophone of /d/ (though not utterance-initially), but I don't know of any loanwords to English where this is reflected).
There are a few other sources of words that start with /v/ and /z/: newly coined words that aren't from earlier roots, like brand names or exclamatory, imitative or "expressive" words. In the case of brand names and the like, the lack of any clear way to write word-initial /ð/ ("th" will generally be taken as /θ/) and the pre-existing rarity of other words that start with this sound probably prevent it from being used.
In the case of expressions and imitations, I think it reflects an overall marginal status of /ð/ as a phoneme in English. It's often noted that even in other contexts than the start of a word, /θ/ and /ð/ are almost never contrastive in English. The pair /s/ and /z/ are more often contrastive (in pairs like lice, lies or mace, maze). The sounds /θ/ and /ð/ are also rarer than /s/ and /z/ overall. This may be why word-initial /z/ is found fairly often in imitative words and exclamations like zap, zip, zowie, but word-intial /ð/ is not.
Here is a quote about it from the paper Dental fricatives and stops in Germanic: deriving diachronic processes from synchronic variation, by Bridget Smith 2007 (She has also written a paper covering acoustic analysis of dental fricatives in Modern English, available from her "selected publications" web page):
Historically, the dental fricative was one voiceless phoneme in Old
English, with a voiced allophone between voiced sounds (much as the
voicing can be generalized today). It could be represented
orthographically with thorn <þ> or edh <ð>, which could be used
interchangeably to represent either the voiced or voiceless variant.
At this time, the alveolar and labio-dental fricatives were also
subject to voicing assimilation, but were written with only the
voiceless graphemes /s/ and /f/ (Mitchell & Robinson 2001:15). After
the Norman Conquest brought unprecedented numbers of loanwords into
English, /s/ and /f/ became contrastive with /z/ and /v/,
respectively, due to loanwords that contained these sounds in
contrastive positions. Borrowing between dialects that had different
distributions, such as the initial voiced fricatives in dialects in
the Southwest of England, may also have contributed to the
phonologization, creating opposing forms such as fox and vixen. The
sounds in assimilatory voicing patterns, and in the new borrowed
lexemes became phonemic by around 1250. French did not have a
word-initial voiced dental fricative, however, so it is more difficult
to ascertain when the phonologization of /θ/ and /ð/ occurred.
If we take a broader look at languages, /ð/ is fairly rare as a distinct phoneme, and it is easily changed into other sounds, but there are definitely languages where it occurs at the start of more words than it does in English (for example, the aforementioned Modern Greek).
