I see this periodically when someone is arguing about a group of people, such as defending a certain group of individuals, or even just trying to explain why something is the way it is. They are accused of being "one of them", or being "with them" to try and invalidate their argument.
Is there a name for this specific type of fallacy, other than an ad-hominem?