A: How do I know if my professor is good?
B: Do you understand what he says?
A: Yes, but that might be because I'm a natural genius, and not necessarily the fact that he is good at teaching.
B: Oh, you said "the fact that he is good at teaching" - so you are already taking for granted that he is a good professor!
A: You're an idiot.
This scenario was just made up - this never actually happened to me. However, I thought it is possible nonetheless (very rare though).
Can saying "the fact that X" imply "X is a fact"? What could A say instead to prevent B from using that logic?