I am pretty certain that either (a) you misheard/misread this, or (b) the speaker/writer was mistaken or joking.
Or double-joking, really. 'What in tarnation?' is (to the best of my knowledge) a now-outdated exclamation meaning something close to 'What the hell?' I have only ever encountered it in Western narratives, where it seems only to be used light-heartedly to mark the speaker as an old-timer.
'Tarnation' seems to be euphemistic in the first place, a derivation of 'damnation' (possibly by way of 'darnation', as in 'darn it!').
So, saying 'What in carnation?' seems likely to be a play on a play on a euphemism. In any case, I am fully convinced that it has no literal meaning in its own right. Its status in your encounter with it rather depends on intention and reception.
Accounts of tarnation's general origin all seem to agree. Merriam-Webster (for example) is particularly succinct on the matter, and also mentions one source in James Joyce, who was Irish by birth (and fiendishly well-read), so my broad association of this expression with the Old West might be a little narrow.
I can find no grounds anywhere for regarding 'carnation' in this context as anything but a joke (by the film's writers) or a mistake (by either the writers, or more likely the transcriber).