On the simplest level, a man of the earth is someone who works the soil — a farmer or gardener. It is an epithet that usually conveys dignity and respect, implying that the person so called has a special relationship to the soil and growing things.
In the original Hebrew of Gen 9.20, Noah is termed an ish ha-adama, ‘man of the earth/soil’, but translations of the verse are rarely literal. Thus earlier English speakers would have learned the expression from hearing about the verse rather than the verse itself:
Noah is called in the Hebrew (in Gen. ix. 20.) Vir terrae, a Man of the Earth, that is, a Husbandman; according to an usual Phrase of Scripture, which calls a Soldier, a Man of War; a strong Man, a Man of the Arms, Vir brachii, (Iob. xxii. 8.) a Murderer, a Man of Blood; an Orator, a Man of Words; and a Shepherd, a Man of Cattel.— Andrew Tooke, trans., François Pomey, The Pantheon, 1698, 162.
Noah was a man of the earth, a husbandman, or one who tilled the ground. — Alexander Cruden, A Complete Concordance to the Holy Scriptures, 1839.
From its biblical use it essentially became an honoric for a farmer or gardener, including all the positive qualities of one “close to the soil”:
Denis was in his late 90s and enjoyed marvellous health up to recently. Denis was a man of the earth and loved people calling to purchase flowers, honey and anything else that Dinny would produce at his farm. — Tipperary Live, 4 Feb. 2018.
Williamson is a man of the earth, soft spoken, not much for crowds or awards, like the Elite Farmer Award he and wife Che Che recently received from Farm Bureau Montgomery. — Montgomery Herald, 28 Nov. 2012.
The man of the earth, the man of the soil
In his lonely allotment he labours and toils.
There's not much to do since he turned sixty-five,
So he took to his garden to keep him alive,
And I think it's his joy and his pride. — Bernie Perry, “The Man of the Earth” (song), 1977.
In more metaphorical or figurative contexts, the meaning of man of the earth depends on the value assigned to earth: for instance, a man bound to material things as opposed to heavenly, lofty, or “higher,” intellectual pursuits or a “man of earth” as opposed to one “with his head in the clouds.”