Using the construction requested in the question, the expression you're looking for is the following:
to know what buttons to push
Depending on how you look at it, this isn't quite a full-blown idiom, but just a common use of language.
3 c : a hidden sensitivity that can be manipulated to produce a desired response
// knows how to push my buttons
Note that what I provide here uses the precise construction asked for:
to know what (noun) to (verb)
→ to know what buttons to push
The phrasing in the definition is slightly different, but the sense is the same.
However, based on some comments received, if you want to change the construction from what was remembered into something that is more explicit, you could instead say something like this:
You're making somebody mad by pushing all their buttons that you know to push.
Other variations are certainly possible, but it's the use of buttons and push in a grammatical way that is the essential part of the expression.