The answer is yes, to be absolutely correct, the sentence should use the subjunctive, but not in the way you suggest. It should read "If his participation in the Kardashian circus weren't already a sign that he was mentally ill..." The "weren't" is in subjunctive mood because the clause is a counterfactual conditional, proposing something that is not true, namely that his participation in the circus is not a sign he is mentally ill. The "was" in "he was mentally ill" just describes the counterfactual condition, so it doesn't take the subjunctive. That "was" could also be "is," depending on what the sentence is intended to mean. Does it mean that he was mentally ill at some time in the past? Then "was." Does it mean that he is mentally ill now? Then "is."
An analogous sentence would be, "If his text weren't [subjunctive] indication enough that he was totally into her [not subjunctive], then the picture was."