In standard Modern English, negation is achieved by adding not after an auxiliary verb. If no such verb is present then the dummy auxiliary do (does, did) is introduced. For example:
I have gone → I have not gone (have is the auxiliary)
He goes → He does not go (no auxiliary in the original sentence)
Different rules apply in subjunctive, imperative and non-finite clauses.
So, here in your question, only the first sentence is correct, the second can be fine, depending on what you mean (but without the comma), and the third is incorrect.
I'm NOT going to Spain to have fun.
The correct form for the second sentence:
I'm going not to have fun, but to look for a job.
or with a slightly different meaning:
I'm not going to have fun, but to look for a job.
And the correct form for the third sentence:
I'm not going to have fun, but study instead.
I'm going not to have fun, but study instead.
You can keep this in your mind that not must always appear after the auxiliary and before the main verb.
See the Wikipedia article about negation in English for more information.