I think that the word "whereon" in the next-to-last line of the four quoted by the poster resolves the question of who (or what) is feeding and on what food. Translated into nonpoetical English, the lines come out something like this:
There was a shepherd that did live = A shepherd lived [or] There once was a shepherd.
And held his thoughts as high as were the mounts = His thoughts were unusually elevated—as high as the mountains, you might say.
the mounts whereon his flocks did hourly feed = His sheep fed every hour on [the grasses of] the mountain slopes ...
him by = ... while he stood or sat nearby [or] near him.
So, removed from the rarefied heights of Thoreau's poesy to the pedestrian flatland of coherent speech, the four lines of verse work out to something like this:
There once was a shepherd whose thoughts were as elevated as the mountains on whose grassy slopes his sheep browsed every hour, while he stood or sat nearby and watched over them.
or this:
There once was a shepherd whose thoughts were as elevated as the mountains on whose grassy slopes his sheep browsed every hour, near him.
but probably not this:
There once was a shepherd of Walden
Whose thoughts were so hot they were scaldin'
His flock they would feed
On the heath and the weed
While he mused on "The Battle of Maldon"
Whether you read "him by" in the fourth line of Thoreau's poem as meaning "while he stood or sat nearby" or as meaning "near him" (that is, reading "him by" as "by him"), the image is essentially the same.
Richard Lanham, A Handbook of Rhetorical Terms, second edition (1991), lists several rhetorical terms that might apply to using "him by" to mean "by him":
anastrophe: unusual arrangement of words or clauses within a sentence, often for poetic effect.
anoiconometon: improper arrangement of words.
hysteron proteron: syntax or sense out of normal logical or temporal order.
synchisis: the word order of a sentence is confused.
Since Thoreau is operating as a poet, all of these rhetorical devices are available him to.
What is not happening in the poem is the sheep feeding the shepherd—not with the grasses of the mountain slopes, nor with the milk of the ewes, nor again with the chops of the innocent lambs, neither hourly nor ever, as his thoughts run to the fleecy clouds girdling the mountain heights like a silken band round the shapely thigh of some immense Hippolyta. Nay, indeed—as sheep never say.
In fact the most puzzling thing to me about this verse is why Thoreau thought it necessary for the sheep to feed hourly. Were they on some sort of small-meals diet that called for frequent snacking at regular intervals? Or did Thoreau mean to suggest that they ate for many hours every day? I suppose we should count ourselves lucky that he didn't have the sheep feeding "minutely"—that would have been really confusing.