Some parents will be offended no matter how you ask and some parents won't be offended no matter how you ask.

In general, it's definitely not a good idea to use the word "it" to refer to a human being, as others have mentioned already. 

1. Is the baby a boy or a girl? (most commonly used and accepted)
2. Is the baby a he or a she? (doesn't quite sound right, the usage of "he/she" seems to be used mostly in reference to animals)
3. Is the baby a male or a female? (sounds very formal but would generally be acceptable if asked by a professional working with the baby, such as a doctor, nurse, social worker, etc.)