I have seen both usages, and want to learn how to choose the appropriate option.
The usage I have most often seen in quantitative technical fields is that there is an upper bound on a variable. And if you use "an upper bound of..." you are referring to that limit, rather than the variable that is being limited.
For example, "there is an upper bound of 50 miles per hour on my car's velocity."
See Mathwords.com for "upper bound of" and this article ("A Simple Upper Bound on the Redundancy of Huffman Codes") for many, many examples of "upper bound on" usage.