I keep writing dataset. Is that correct, or should I write data set?
As @mmyers notes, dataset does not appear in any dictionaries. However, there are 172 incidences in the Corpus of Contemporary American English, and all but a handful are in the “academic” section, representing formal academic writing. Its lack of appearance in dictionaries is probably because it is a fairly new coinage, the two examples from the Corpus of Historical American English are from 2001. Nothing from before then. Interestingly, the British National Corpus has 51 incidences, dating from the 1980s to the mid 1990s.
Given that information, I guess I would classify dataset as technical jargon, but it's really not much of a jargon term. Any technical audience would have no problem with it; a non-technical audience should still easily understand its meaning.
The APA Style Blog comes down firmly on the data set spelling. Although dataset is understandable, two words still seems to be preferred even in academic settings.