Some style guides recommend refraining from using colons after verbs, so if there is going to a pedant reading your text, it might be better to say “My question is [ the following | thus | this ]: why is ice so slippery?”. That actually sounds better to me anyways, but there's nothing wrong with what you had.
No comma doesn't work because of the inverted subject-verb order.
The comma one is technically okay, but I personally rather dislike it. The Handbook of Good English calls it a sort of “soft colon” (and grants a capital letter to the question if you want further emphasis on the question), but no matter how many times I see it, it just looks wrong to my eyes since you don't write “My question is, not important” or “My question is, really a statement”. But I digress.
That said, you have a few other options that work too depending on your taste:
- My question is( , ) “Why is ice so slippery?”
- My question is why ice is so slippery.
- I question why ice is so slippery.