Is it correct to say "Many's the hour have I spent listening to his fatuous ideas"? I'm not quite sure if inversion is possible with "many's the..." My gut feeling says "why not?!", but my left hemisphere is dubious about it.
With this exact wording, no. It would be ungrammatical. "I have spent..." is a relative clause to "Many's the hour," which is interestingly already inverted itself:
Many is the hour is like saying Fair is my beloved!
And the relative clause should follow as:
Many is the hour (that) I've spent listening to his fatuous ideas.
With its adverbial synonym "Many an hour," though, it would be perfectly fine:
Many an hour have I spent by the big fish pond at Rohaine.
Many an hour have I and my playmates pelted that tree with sticks and stones.