Actually the idiom goes, "to break a (butter)fly on the wheel". The wheel in question being a torture device, for humans rather than flies. From Wikipedia:
The breaking wheel, also known as the Catherine wheel or simply the wheel, was a torture device used for capital punishment in the Middle Ages and early modern times for public execution by bludgeoning to death.
The sledgehammer mentioned in the top answer is from a different idiom entirely, "cracking a nut with a sledgehammer". Same idea, of course, but the idioms are not to be mixed and matched at will except for comedic effect.
In German and Russian, there's also "to shoot sparrows with cannons" (mit Kanonen auf Spatzen schießen/стрелять из пушки по воробьям). That one hasn't caught on in English, though.