What makes these two words so different that 'man' is changed to 'men', but 'German' is changed to 'Germans'?
German is from the Latin word germani; unlike the similar-looking demonyms Englishman and Frenchman, it is etymologically unrelated to the word man and does not form a plural the same way.
Words like human and German are not from man and do not contain the (Germanic) morpheme man, as you say. Only the morpheme man is properly pluralised as men. See the list of words in Tchrist's answer for an overview of which words are from man and which aren't. In human and German, we're dealing with the Latin suffix -anus, which means something vague like "having to do with x", shortened to -an in English.
Cf. Republican (same Latin suffix), Qur'an (an Arabic morpheme): we don't say two Republicen were reading their Qur'en.
The true etymology is very important, because folk etymology is not very common in general and hardly ever accepted by the "writing classes". So it does not often become popular enough to displace the original.
Further, there would be little difference in pronunciation between human and humen, because the last syllable is unstressed; folk etymology normally originates in speech, not writing, which means that this plural would be much less useful to speakers. (In women, a trick was performed by having a differently pronounced first syllable to distinguish between woman (/wʊ-/) and women (/wɪ-/), which words would otherwise be pronounced (almost) the same.)
It’s because they are not, or are not thought of as being, composed of a base noun which the word man has been pasted on to the end of. Without the word man to start with, you won’t get men out of the plural.
Here’s a longer list of such things:
In other words, there is no man morpheme present to undergo that word’s (now irregular) i-mutation the way there is in these:
See the difference?
Be aware that one on rare occasion still sees this pattern:
- foramen > foramina
- gravamen > gravamina
- legumen > legumina
- molimen > molimina
- nomen > nomina
- stamen > stamina
- specimen > specimina
- tegmen > tegmina
- tentamen > tentamina
- velamen > velamina
Most of those Latin inflections have now been assimilated into regular English plurals, except sometimes in scientific literature.
As tchrist pointed out in his answer, it hinges on whether the word is considered to be base descriptive word + man. Some of your examples, like German, were never formed that way. However, the OED gives an example of Normen as a plural in Old English:
OE Anglo-Saxon Chron. (Tiber. B.i) anno 1066 Þær wæs Harold cyning of Norwegan & Tostig eorl ofslagen, & gerim folces mid heom, ægðer ge Normana ge Englisca, & þa Normen [flugon þa Englis[c]a].
OE Anglo-Saxon Chron. (Tiber. B.iv) anno 1049 Harold for to Norwegum, Magnus fædera, syððan Magnus dead wæs, & Normen hine underfengon.
Compare to Englisman, Englishmen, which are still both acceptable. I'd guess that as Norman became seen more as the name of the people, its plural became more regular. It stopped being a compound word and became a simple word.
Another way of saying it- a man from France is called a Frenchman, and many of them are referred to collectively as FrenchMEN. A man from Germany (we see here with the "y" that "man" is included as a coincidence- you can't take it as "many from Ger" either...) would be called a Germanman and many would be Germanmen. I think they might have said it back in WWII, but I could be wrong.