If I add a new feature (called "feature X") to existing software, what do I say about the action?
I added support for feature X
I added support of feature X
Or something different?
Neither one. You're not adding support of the feature; you're adding the feature itself.
First, let's address your premise:
should be either:
(Either eliminate the "an", or change "software" to its adjective form. "An existing software" sounds off.)
Now, to answer your question, simply say:
As for when to use the word support: if feature X had already been added, and support for the feature was added some months later, you might say:
(I'm assuming that feature refers to some new capability, and support refers to some support for that capability, such as built-in help menus.)
They mean different things. I added support for feature X means that your contribution improved feature X in some way. I added support of feature X means that you added feature X to something else which in some way supported it. It sounds as if it's the second one that you want if you're adding it to existing software. If so, it would be preferable to precede 'support' with the definite article: I added the support of feature X.
In the software context, support for feature X is used to mean that feature X can now be implemented. For example, in the LaTeX editor TeXworks, support for synctex has been added. This means that TeXworks is now built so that it works with synctex.