Spook was actually used by black people to refer to white people, presumably on the notion of “white” ghosts.
[SE spook, a ghost]
- (US black) a white person.
- 1939 [US] P.E. Miller Down Beat’s Yearbook of Swing n.p.: spook: a white musician.
- 1944 [US] D. Burley Orig. Hbk of Harlem Jive 19: Us young homes, and lanes and hipstuds, gray and fay, and spook and spade.
(Green’s Dictionary of slang)
As for its usage about black people, Etymonline suggests that:
The derogatory racial sense of "black person" is attested from 1945, perhaps from the notion of dark skin being difficult to see at night.
The following article from the Newsweek.com has an interesting story on how the term spook was initially used to refer to black people:
According to Merriam-Webster, the word "spooky" is defined as, "relating to, resembling or suggesting spooks." A further break-down of "spook" gives way to the meaning, "ghost, specter" or "an undercover agent: spy." But the Dutch word describing apparitions, which first came into use around the 19th century, took on a more sinister meaning around World War II, when white American soldiers started referring to their Black counterparts as "spooks."
Originally, pilots of the Tuskegee Institute—derived of the first Black military aviators in the U.S. Army Air Corps—were called the "Spookwaffe"—a play on the German term "waffe," which means weapon or gun. When airmen returned from their posts with the nickname, white Americans caught wind of the name and began linking the term "spook" to blackness, thus resulting in the word transitioning into a racial slur and its derogatory use.