That is the translation (provided by Wikiquote) of the Dutch proverb "Vertrouwen komt te voet en vertrekt te paard." I don't like this translation very much for conversational use. It doesn't "feel" right. Neither does "Trust comes on foot, but leaves on horseback."
The actual, somewhat lengthy, meaning of the proverb is that a single stupidity can ruin trust or reputation that took years to build.
I tentatively prefer "Trust is hard to gain but easy to lose." However, I'm not native-English speaking.
What is the "best" way to succinctly express the idea behind the proverb in English?