Consider the phrase:
You have come one (if not many) times to my house ...
I am not sure if the noun "times" should be plural or not. It sounds better with plural, but this answer states that parenthesis are used for things which can be safely removing without affecting the meaning of the sentence. In that sense, if you get rid of the info in parenthesis the singular should be used.
Is there a canonical answer here? This related question has just one answer, which to be honest is not very helpful (and has just one vote). Also, my question has the extra issue of having a parenthesis, which surely makes it distinctive from the one linked above.