Person A says to B: "I thought you were smarter/stronger/better/more trustworthy than that. I should have known better."
I was thinking it would be termed passive aggressive, but after reading up on it, that doesn't seem right. I'm tripped up on how focused on the Person A the two sentences are. Typically an insult would be "You're not very smart" and that would be that. But this is "I'm disappointed in myself for thinking better of you."
Is there a good term for this sort of insult?