There always seem to be different, often unrelated points being made when it comes to the word impact:
- That it should not be used to mean "affect" or "an effect" because it means to forcefully come into contact with / strike something.
- That it cannot be used as a verb because it was traditionally a noun.
While #2 is along the lines of argumentation that a lot of people seem to be immersed in, it's not the actual issue with the word (i.e., the process of making nouns into verbs has always happened and this word has undergone that process as well quite a long time ago); #1 is.
And even regarding that, people who claim that it cannot be used figuratively are probably being a little too conservative/traditionalist, seeing as words taking on figurative meanings is also a common and natural process in language. Even this is not the problem, however.
The main problem lies with people who use the words impact and affect (as a verb) or effect (as a noun) interchangeably. They mean different things. Affect simply means to have an effect or influence on; i.e., to cause something to change in some way. Impact, on the other hand, means to collide with or crash into something or to pack something in firmly when used literally.
Now, this can be extended to its figurative meaning, which can be used to mean "to affect something in a shocking, violent way" (when used as a verb) or "the shocking, violent effect of something on something else" (when used as a noun).
That is, affect means to have any kind of effect on something, while impact means to have a strong, often violent effect on something. Here is an example that should make it clear: let's say that I had a car accident.
- Our car was impacted in the accident, destroying it completely and resulting in broken legs for me. (Literal sense)
- After the accident, my life was impacted since I couldn't walk for 6 months. (Figurative sense)
As you can see, there's nothing wrong with either of these usages. Since losing your ability to walk is incredibly disruptive/painful, it would be safe to say the accident had an impact on your life.
The problem, of course, is when people use impact for any and every thing that wouldn't necessarily have been affected violently/shockingly. It was a trend that started in the business world, as with many things, to hyperbolize usage in order to make their statements "stand out". This is why such misuse is still within the realm of jargon. This is the problem most people have with the word.
What's more, there is another class of people who promote the use of impact over affect/effect simply because many people confuse the latter two words. This is not only poor advice, since it encourages people to avoid learning about the difference rather than teaching them it, but it also opened the Pandora's box we are dealing with today.
The problem with such use becomes especially evident in negative constructions. Your proposed example is a good one to illustrate the point:
Certain type checks allow reordering without impacting semantics.
Saying "without impacting semantics" means that "reordering" would not have a shocking/violent effect on "semantics". The problem with this, of course, is that it implies that it could still have a not-so-shocking effect on them, so you now have an unclear sentence. This is not to mention that, judging from the context, "reordering" would never have a shocking/violent effect on "semantics", so it would be incorrect usage even in the positive sense.
This is why, except in specific cases where an effect is incredible, shocking, or violent, the word impact should be avoided. You want affect when using the verb and effect when using the noun.