A) No, I didn't miss you.

B) Yes, I didn't miss you.

C) No, I did miss you.

D) Yes, I did miss you.

According to my common sense perfect answers can be C) and B) only, and reason behind it is- "didn't" is used/asked in the question, so if the person answering the question agrees with the person asking the question he will probably say "Yes, I didn't miss you," and if he does not agree he should answer "No, I did miss you."

I have checked online everywhere and all four are treated as correct. So, just asking to the experts present here for their views on this, in any context of English language or usage.

EDIT1 : Actually i was hoping that the SPEAKER Of QUESTION can tweak the question to get the desired answers only( only (B), and (C) in my case).But, the English seems to be weak to me after reading all the answers and responses from the experts,as the SPEAKER OF ANSWER totally has the control of the responses and he can speak any of the all four answers A,B,C,D.

What if the question asked was :

Do you miss me a little? OR Did you miss me a little? OR SIMPLY Did you miss me?

I guess again all the previous answers A,B,C,D are correct. ;-)