Perhaps strangely, I'd say the most idiomatic word for cause of the problem is actually problem itself:
2 b : a source of perplexity, distress, or vexation
I found the problem!
Since this sense of the word means source, it already has the implied meaning of source of the issue or, in other words, cause of the problem. It's somewhat recursive, but it's still natural usage.
Another word, which seems to have been suggested in a now-deleted comment under the question, is culprit:
3 : the source or cause of a problem
// Lack of exercise and poor diet are the main culprits in heart disease.
In the example sentence used in its definition, culprit seems a better choice than problem would be.
However, in the sentence I found the ___!, Google Books Ngram Viewer suggests that problem is the more common word, at least in print.