Jim posted a good compound adjective in the comments: phobia-inducing.
Example of use:
Many phobia-inducing stimuli (such as blood, snakes, and heights) evoke fear in most people, and the decision as to when this becomes unreasonable or excessive depends to a degree on the investigators' judgment (and perhaps own fears)
("Specific Phobia", by Bavanisha Vythilingum and Dan J. Stein, in Clinical Manual of Anxiety Disorders, edited by Dan J. Stein, p. 46)
If you are feeling more adventurous, you could use a compound based on Greek roots like 0xFEE1DEAD's suggestion of "phobogenic" (adj.). Google Books provides a fair number of examples, all from pretty specialized contexts. A typical educated English speaker would not be expected to know this word, but might reasonably be expected to to be able to figure out the (approximate) meaning.
The expected equivalent noun to "phobogenic" would be "phobogen" (n.) (compare carcinogenic adj. and carcinogen n.), used as a count noun (e.g. "Spiders are a phobogen for me") but I haven't been able to find any examples of this hypothetical word being used at all. There is only one apparent true positive that I found via Google search ("In phobias, subjects are preoccupied by their fears only when confronted to the phobogen stimuli...") and here it seems to be used as an adjective, not as a noun (possibly as an awkward translation from an adjective in another language, like Spanish fobógeno/French phobogène, that would be more naturally translated as "phobogenic").
who causes a phobia
but, the answer could be both :)