I have faced such situation twice or thrice in my Uni. and work life. Due to competition your peer or ill-wisher tries to you put in dark light intentionally. So, it gives wrong impression of you to other people. If your ill-wisher succeed, then your image in spoiled in front of others without you actually doing any wrong.
So, what you call a person who spoils your image or malign your image in public.
For example, He is a ____. He spoiled my image in my office.
In real, he did not spoiled my image, but he tried and my co-workers supported me. One more thing, I use to use ill-wisher until now but I feel I need one perfect word. Using ill-wisher sounds stupid.