You could rephrase it a bit to use "weight", or even "priority", like so:
Given two states in the diagram, each state has a value for [priority | weight | other?]. That sets up the following alternatives for the next sentence:
- If state 1 > state 2, we choose state 1.
- If state 1 has the greatest priority, we choose it.
- We choose the state with the greatest priority.
- If state 1 priority > state 2 priority, we choose state 1.
I would personally opt for the third alternative. So I would say, "Given the two states in the diagram, each has a value for priority. We choose the state with the greatest priority."
It may be important for you to use the term "priority" in this case. Stop and ask yourself, what does the value really represent? Is it really priority? In that case, you should definitely use that term. If you get into the game of searching for an adequate synonym because it makes the writing sound better, then you're entering into euphemism. Technical writing is always clearer if you use the real terms, and stick to them consistently. So... Clearly set up the term usage in your passage, and it will make more sense in the long run.
Now, is "priority" better than "dominant"? Sorry, I can't answer that because I don't know the system you're describing. If it's your system, you have to sit down and work out what the value really does, and assign a term to it. If you're describing somebody else's system, you had better ask. In my experience, usage of terms like this can be very important. Again, describe the "reality* of the system, and don't look for euphemism.