On the NOAD reports spelling checker, but it simply describes it as another term for spell-checker; spell checker is reported in the description of spell-checker as possible variant.
To spell-check (or to spell check) means to check the spelling of a text using a spell-checker; the meaning of spell-checker is reported to be a computer program that checks the spelling of words in files of text, typically by comparison with a stored list of words.
To check the spelling would have a more generic meaning that to spell check, as the latter verb implicates the use of a computer program. It is possible that, as nowadays the computer is used also as text editor, to spell check is used even when the operation has not be done with a computer (e.g., the professor has spell-checked my essay).
Searching for [check] the spelling and [spellcheck] on the Corpus of Contemporary American ([check] matches check, checks, checked) I get the following data (the values are the frequencies per million; the images can also be seen here, and here):
The data evidences that spellcheck is used more frequently in the last 10 years.