There's a Hungarian phrase that can be literally translated as something like "fall off the other side of the horse". (The literal implication is either that instead of falling off this side of the horse, you fell off the other side; or that in your zeal to get on the horse, you overshot the target and fell off the other side.) It means that you are at one extreme of a situation, and you want to change it so hard that you fall into the opposite extreme. Is there an equivalent of this phrase/idiom in English?
The idiom go overboard (“To go to extremes, especially as a result of enthusiasm”) implies extremity, although not necessarily the opposite extremity.
Answering the ‘It means that you are at one extreme of a situation, and you want to change it so hard that you fall into the opposite extreme’ part:
In Old Tales Retold from Grecian Mythology in Talks Around the Fire (1876) by Augusta Larned (as at google books), it is stated that
To escape Scylla only to fall on Charybdis
‘has become a proverbial expression’.
I recently came across an old British proverb that refers to the very situation that your Hungarian saying does. From James Kelly, Complete Collection of Scottish Proverbs (1721):
[The proverb:] You breed of Lady Mary, when you're good you're o'er good.
[Explanation:] A drunken Man beg'd Lady Mary [the Virgin Mary, presumably] to help him on his Horse, and having made many Attempts to no purpose, he always reiterated the same Petition ; at length he jump'd quite over. O Lady Mary, (said he) when thou art good, thou art o'er good.
Unfortunately, as awareness of the difficulty of mounting a horse while drunk (or otherwise incapacitated) became less commonplace in the English-speaking world, this particular proverb fell into disuse. But it vividly captures the idea of finally succeeding in a difficult climb, only to go sliding down the other side (of the horse).
Not exactly what you're asking for, but throw out the baby with the bath water might be suitable in many contexts.