A) No, I didn't miss you.

B) Yes, I didn't miss you.

C) No, I did miss you.

D) Yes, I did miss you.

According to my common sense perfect answers can be C) and B) only, and reason behind it is- "didn't" is used/asked in the question, so if the person answering the question agrees with the person asking the question he will probably say "Yes, I didn't miss you," and if he does not agree he should answer "No, I did miss you."

I have checked online everywhere and all four are treated as correct. So, just asking to the experts present here for their views on this, in any context of English language or usage.