In written English, the phrase will probably is 87 times more likely to appear than may probably, including the month of May, and a preponderance of references from the 19th and early 20th centuries. The construction may probably is certainly acceptable grammatically and it doesn't even sound terribly awkward, but it is not at all common.
Generally, the modal may expresses four ideas, but only two of them comfortably apply to He may probably forget his umbrella. I offer the options from most comfortable to least:
1.0 Expressing possibility:
that may be true
Since possibility and probability are logically intertwined, one might consider it a redundancy, but it could also be a matter of emphasis or even a subtle distinction between the possibility of may and the probability of probably.
He will probably forget his umbrella, gets the job done colloquially.
He may perhaps forget his umbrella, is even better, creating a similar redundant emphasis.
He may probably forget his umbrella, might capture a level of attention by its uniqueness.
1.1 Used when admitting that something is so before making another, more important point:
they may have been old-fashioned, but they were excellent teachers
Hypothetically, the next sentence in the larger context might be: He will certainly remember his wallet. If that were the case, may in the first sentence would draw a comparison in parallel with will in the second sentence. The words probably and certainly would then emphasize the likelihood of each possibility: He may probably forget his umbrella. He will certainly remember his wallet.
The other two common meanings of may would almost universally demand a different syntax:
2.0 Expressing permission:
you may use a sling if you wish
For permission, theoretically, one might say, He probably may forget his umbrella, but it is hard to imagine giving someone permission to forget. It is even harder to imagine a situation where offering a person permission to forget his umbrella is a matter of probability, unless it were somehow tied irrationally to the probability of the weather.
3.0 Expressing a wish or hope:
may she rest in peace
May he probably forget his umbrella, seems like a strange wish. I would go for the whole nine yards and say, May he forget his umbrella!