While I personally think the second example is stylistically strongest, there's nothing really wrong with the first. It just makes the reader parse "X on Y" as a prepositional phrase before getting to the copula, and "the effect of X" is easier to digest as a unit than "the effect of X on Y."
Moreover, although the third example may have an odd feel to it, there is still nothing wrong with that word order either. I can imagine the formulation being used to emphasize the word strength. (I'm not sure what XZY is supposed to mean; did you intend to write "X on Y" there as well?)
Imagine an expressive voice reading the line:
We investigate how strong is the effect of X on Y.
And consider a parallel construction:
We wonder how foolish is the man who thinks otherwise.
A little bit poetic, perhaps a bit dramatic — but certainly no glaring solecism to be worried about.
On the surface, this obviously conflicts with John Lawler's view about the third example. He is answering as a grammarian and for most uses his view on that example would be considered standard. I only mentioned my point about No. 3 because English is really more flexible than "standard grammar" would allow.
The "poetic" or "dramatic" effect I refer to is called anastrophe. It refers to an inversion of word order used for effect as a rhetorical device. From Ward Farnsworth's Classical English Rhetoric:
Some standard purposes of [anastrophe]:
a. The unexpected placement of words calls attention to them. Pushing a word into an especially early or late position often creates emphasis in itself; then the emphasis is still greater because the ordering mildly violates the reader's expectations.
b. Inversion may put words in an order that creates an attractive rhythm.
c. Inversion may compress a meaning into fewer words.
d. Inversion sometimes causes the full meaning of a sentence to become clear only late in its progress; this bit of suspense makes the finish more climactic when it arrives
That said, in most cases it would be safer to avoid anastrophe if you don't really know how to use it or intend to do so — to use it, as @AndrewLeach suggests, you first need to understand that you are deviating from a standard or "normal" word order.