The phrases "higher priority", "lower priority", and "lowering the priority" are all independent of how priority is encoded. Higher priority doesn't mean encoded with higher numbers. It means this will be done before that will be done. I could encode priority with letters of the alphabet, and in any order.
The problem is some people might not realize that when explaining how to configure your networking devices. They may assume everyone "knows" that 1 is lower than 2 both numerically and in encoded priority. If they never make that clear, not even with an example, then you're just going to have to run a test and find out. Which you probably should do in any case. One good test is worth 1000 expert opinions.
I know with the route command on my windows 10 that the "Metric" field is used for this. It's defined as "METRIC specifies the metric, ie. cost for the destination."
That wording makes it clear that larger numbers indicate lower prioritylarger numbers indicate lower priority. At least in this system.