The confusing part here is that you are using "press" to "depress" and then also using "press" to "undo" the cause of the same action. You then switch to the sound a button makes ("click") and use that to refer the the "un-pressing" action. All of which is acceptable and illustrate why this is not straightforward.
"Toggle" means to "switch between states" so a "press" could perform the action of "returning to the previous state" regardless of the present state. So you could say "depress" and then "when we click/press the button a second time, we toggle it back."
In our shop we refer to the states as "pressed" and "unpressed" (which, oddly, is defined but not for this use, but we use them anyway). Then frequently refer to actions on the button as a "toggle" ("click to toggle" or "press to toggle") especially in cases where we do not want to or cannot accurately specify it's current state.