It is honestly contextually dependent.
Nerd can refer to someone who is completely absorbed by something, be it science related or otherwise. You can be a history nerd, or an English nerd if you are really into that particular activity beyond a "normal" level.
Secondarily, "nerd" also conveys a certain culture, centered around, yes, video games, comic books, braces, unfashionable attire, and certain voice and speech patters.
The stereotypical "nerd" might use highly technical language instead of usual slang and parlance. Often, the "math-sci" nerd you may picture coincides with this separate "nerd" culture. The tech-savvy nerd you knew in high school, the one interested in math and science, sort of gave birth to the comic-book reading nerd you observe in XKCD.
But today, "Nerd culture" far extends beyond math and science. You can nerd it out in almost anything, and furthermore, the gamer-comic book culture I talked about above, is now firmly independent of the math-science geek crowd. But of course, there is still some overlap.