This has always puzzled me.
It looked nothing like the dessert.
If they thought it was in some way bomb-like, why the extra 'e'?
There are two different machines here, the British bombe and the Polish bomba.
According to the caption of this photo of Bletchley Bombe Wiring:
Jack Copeland's The Essential Turing:
It goes on to give another story from declassified American documents, that the machine must have a stopping mechanism where part of the machine woud drop onto the machine and make a loud noise. But this explanation seems unlikely.
It's important to realize that bombe, the decryption device designed to decipher Enigma-encrypted signals, was designed by a Polish cryptologist Marian Rejewski in 1938. The device was named by him, bomba kryptologiczna, while an alternative to honor the creator was also coined: bomba Rejewskiego.
As you properly noted, it isn't a bomb. Probably the author gave it this name because it resembled a bomb. We can only guess and that won't really do anything.
In 1939, a year later, a similar device inspired by "bomba Rejewskiego" was produced at Bletchley Park by Alan Turing and later improved in 1940 by Gordon Welchman. During this time, the people obviously realized the same dilemma you're having about the device not being a bomb, but to keep the naming as consistent as possible, they named it a bombe - both to refer to the original name by Rejewski and to diverge from the "bomb" implication.
I think this is as close as you're going to get, unless you can get your hands on some historic documents containing the exact process of Rejewski's work, which I don't think you'll be able to get on the internet, but it's still important to realize that the Polish name was the initial one, so that's what you must be coming from.