Heavy music is a somewhat dated term now. Once upon a time, you could have used it to distinguish between, say, Schoenberg (heavy, serious, intellectual) and Gershwin (light, entertaining). The heavy music demands more of the listener.
Heavy can also mean somber. The funeral march from Beethoven's Heroic Symphony (the 3rd) is heavy. A nocturne by Chopin is light and delicate in comparison.
Heavy can also mean loud and oppressive, as the OP points out. With present day amplification, however, almost anything can be made heavy. Attributing "heaviness" to a particular style or genre of music is mostly a matter of taste.
On the other hand, any of these could drive a person crazy.