If I say something like
He is big compared to the other guy.
He is relatively big compared to the other guy.
is the adverb relatively wrong because of redundancy?
I'd likely use "big relative to the other guy", or "a little bigger than", depending on what you're trying to convey.
It seems to me that "relatively big compared to" is trying to use the word relatively to mean "slightly", or "not by much". When you get rid of it, you have "big compared to", which can mean that he's thousands of times larger, or only a little larger.
Depending on context, it could also be a double comparison. The door is 8 feet tall. Guy-A is 8.5 feet tall. Guy-B is 6 feet tall. Therefore, while it is obvious that Guy-A is bigger than Guy-B, what may not be obvious is that Guy-A is big relative to the door, as compared to Guy-B, who is small relative to the door. So, with regard to the door, "Guy-A is relatively big compared to Guy-B."