English is essentially a gender-free language (which is odd for a Germanic language; most have three genders). We still have some vestiges of a time when gender was important, and it is an unfortunate fact that the "genderless" versions of many words have defaulted to the something that looks masculine, but for the most part anything that was indicated using affixes (prefixes or suffixes/endings) has vanished, leaving us with only a few words that were completely different. (Yes, you can still find some usage of feminine terms ending in "ess", and much more rarely "ix", but they are the exception rather than the rule today.)
That said, the only difference between the word for male cousin and female cousin in older versions of English was the vowel in what is now the unstressed final syllable and an almost imperceptible final "e" (which has gone on to become silent in most English words, leaving only the remnants of its umlaut behind — with allowances for the Great Vowel Shift, of course). In the French pronunciation of the cognate words cousin and cousine, the final syllable is stressed, which makes the difference between them durable in speech (even if we ignore writing). Unstressed vowel changes, on the other hand, are difficult to perceive at all in speech (there are many things that will have a greater effect on the subtle nuances of a minimal schwa than conscious use of gender, such as talking with your mouth full or being a little yawny), and are naturally headed for deletion. When you are hardly pronouncing the final syllable, the isn't a whole lot of difference between ku'zun and ku'zin. This is not the case, though, with distinctly different words; you have to mess up the pronunciation of aunt pretty badly before it starts to sound like uncle.
English has a strong resistance to gender anyway, largely because it has been an exoteric language for most of its existence. Apart from the distinct dialects one can find in written Old English (resulting from various mixtures of Frisian, Anglian, Saxon, Jutish, and a few borrowings from the native Celts), half of the country was occupied by Norse people (the Danelaw) speaking a similar but different language with its nouns classified into slightly different gender categories (once obvious sex correspondence is taken out of consideration), and there was a significant population of people who were either native Celtic language speakers or Celtic/Englisc bilingual — and that's before English became a language of conquest. As Tim pointed out in a comment, it's easier just to drop the whole gender thing than it is to worry about whether your spoon is a man, a woman or something else entirely (in this part of the country, on a Tuesday in May). Combine that with the Principle of Least Effort (that'd be a fancy term for "laziness of the mouth" in linguistics), and you wind up with complicated things like gender only showing up where there is no way to avoid it easily.