The lady's line is (probably*) doubly wrong.
First, that's not what this construction means (as pretty much any native speaker knows). The presence of two negative elements in the man's sentence is due to negative concord, not the presence of two separate semantic negations. Here are some resources describing negative concord in modern-day varieties of English: grammar.about.com, Yale Grammatical Diversity Project: English in North America.
Second, as you point out, even if we do ignore negative concord when interpreting the sentence, we cannot logically infer that the man "is afraid of ghosts."
If the sentence has two semantic negations, it should be logically equivalent to
"It is false that I am afraid of no ghost."
I can only think of two logically valid interpretations for this:
The one you mention, where "no ghost" is taken as meaning "the absence of a ghost." In that case, the sentence doesn't say anything about whether the man is afraid of the presence of ghosts. He might be, or he might not be: the sentence doesn't say.
The one where "I am afraid of no ghost" is taken to mean "There is no ghost of which I am afraid." Under this interpretation, what the lady said is almost correct, but still not logically justified. When we're dealing with countable nouns like "ghost," the logical negation of "There is no" is "There is at least one." Therefore, "It is false that there is no ghost of which I am afraid" logically implies "It is true that there is at least one ghost of which I am afraid." The lady would have been correct under this interpretation if she had said "That means you ARE afraid of at least one ghost."
*The lady's statement could be considered "technically" true (so only wrong once) if it were possible to interpret the original sentence as meaning "There is no ghost of which I am not afraid," but I don't think that's grammatically possible. I guess "I love no ghost" means "There is no ghost that I love," so maybe if you consider "ain't afraid of" to constitute a single idea equivalent to a verb like "love", you could interpret "I ain't afraid of no ghost" as "[I ain't afraid of] [no ghost]" = "There is no ghost that [I ain't afraid of]", but that interpretation seems very strained to me.