Here: hyperdictionary
I find this explanation:
Admiral grace hopper (an early computing pioneer better known for inventing cobol) liked to tell a story in which a technician solved a glitch in the harvard mark ii machine by pulling an actual insect out from between the contacts of one of its relays, and she subsequently promulgated bug in its hackish sense as a joke about the incident (though, as she was careful to admit, she was not there when it happened). For many years the logbook associated with the incident and the actual bug in question (a moth) sat in a display case at the Naval Surface Warfare Center (NSWC). The entire story, with a picture of the logbook and the moth taped into it, is recorded in the "Annals of the History of Computing", Vol. 3, No. 3 (July 1981), pp. 285--286.
The text of the log entry (from September 9, 1947), reads "1545 Relay #70 Panel F (moth) in relay. First actual case of bug being found". This wording establishes that the term was already in use at the time in its current specific sense - and Hopper herself reports that the term "bug" was regularly applied to problems in radar electronics during WWII.
Indeed, the use of "bug" to mean an industrial defect was already established in Thomas Edison's time, and a more specific and rather modern use can be found in an electrical handbook from 1896 ("Hawkin's New Catechism of Electricity", Theo. Audel & Co.) which says: "The term "bug" is used to a limited extent to designate any fault or trouble in the connections or working of electric apparatus." It further notes that the term is "said to have originated in quadruplex telegraphy and have been transferred to all electric apparatus."
The latter observation may explain a common folk etymology of the term; that it came from telephone company usage, in which "bugs in a telephone cable" were blamed for noisy lines. Though this derivation seems to be mistaken, it may well be a distorted memory of a joke first current among telegraph operators more than a century ago!
Actually, use of "bug" in the general sense of a disruptive event goes back to Shakespeare! In the first edition of Samuel Johnson's dictionary one meaning of "bug" is "A frightful object; a walking spectre"; this is traced to "bugbear", a Welsh term for a variety of mythological monster which (to complete the circle) has recently been reintroduced into the popular lexicon through fantasy role-playing games.
As for "debugging", wikipedia notes that the Oxford English Dictionary entry for "debug" contains a use of "debugging" in the context of air-plane engines in 1945.
For the use of "bug" in software, then, at first in the history of computers there was no real software, the program was hardcoded with hardware parts. So it seems sensible to say that the term transitioned from hardware to software when the latter started to be independant from the former..
EDIT
I found in Robert Leckie's book "Strong Men Armed: The United States Marines Against Japan" that he refers to a "faulty" airplane as "a plane full of bugs". I think it was then a Navy/Marine expression at that time (the book was written in 1962 though).