Calling 'to wait' an adverb (or adverbial, in either the nounal or adjectival sense of this word) is really stretching the definition of 'adverb/ial' too far. It is a sad fact that 'adverb' was the class that 'doesn't appear to be one of the other 7 parts of speech' became.
'I am waiting happily' is very different (though they may be said in the same situation) in meaning from 'I am happy to wait'. It's better to regard 'to wait' as the 'completer' of the adjective 'happy' here; it even really modifies ('changes to a certain degree' rather than the 'attaches additional meaning' sense) what 'happy' means in the context ('I am happy' means 'I am almost joyful', but 'happy to wait' means 'not unwilling to wait'; imagine 'joyful to wait' or 'unhappy to wait'). It is 'necessary' in the sense that it modifies meaning rather than just adds additional information (contrast 'I am waiting quietly).