- James is Max's supposedly Irish ancestor.
- James is Max's supposed Irish ancestor.
In the first sentence, supposedly modifies Irish. The sentence implies that James is Max's ancestor and that he is supposedly Irish.
In the second one, the adjective supposed modifies the noun ancestor. And so, the sentence implies that James may or may not be Max's ancestor, but James is certainly Irish.
As long as you cannot offer any context to determine whether James is actually Irish, or whether he may or may not be Max's ancestor, I'd say that both sentences are correct.