In context, when would it be appropriate to use "answer" or "response"? I always tend to use "answer" personally, but I have always this nagging feeling I could be wrong.
Response is a very broad term, and includes all reactions to a stimulus.
Answer is more specifically a response to a direct question.
One can always respond to anything (an event, an injury, a letter, a speech, a question...) but one can only answer a question. As Emanuil said, it's also possible to respond to a question without actually answering it; listen to politicians being interviewed.