First, I am assuming there is a typo, and the sentence is meant to be I can't tell if . . .
Generally speaking, if there is conflict between the verb form used, it should be the one that agrees with the noun that is closest to it.
In this case, that would be Amanda:
I can't tell if you or Amanda is the best.
If are were used, it could lead to confusion with the following type of interpretation:
I can't tell if (you and Amanda) are the best. (Or if it's two other people who are the best.)
The problem here is not only the second-person conjugation versus the third-person conjugation, but also confusion between singularly and plurality.
However, there is no definitive rule for this. If, no matter which verb form is used, there is confusion or the wording looks unnatural, then the sentence should be rephrased.
If it can be argued that neither verb form is correct, then it can also be argued that the sentence simply can't be written so as to use either verb form.
Perhaps the simplest way of rephrasing it is to reverse the position of the nouns and the adjective:
I can't tell if the best is you or Amanda.
With this version, where the verb comes before the nouns, the conjugation of the verb is the same, regardless of the noun being in the second person or third person.