In my mind there are four errors:
The use of "him" in places of "he" in "him... went to Virginia" is clearly an error (as pointed out).
The other issue making this odd sentence read so badly is the disagreement between "places he vacationed" and "the time when".
In addition, there is a tense/time disagreement between "vacationed" (past) and "is the time when" (present).
So, if rewritten as follows, it is improved:
Of all the times he vacationed, Ian's favorite was the time when he and his best friend went to Virginia.
Of all the places he vacationed, Ian's favorite was Virginia where he went with his best friend.
- More controversially though, from my perspective as an Australian English speaker, it's the use of the verb "vacationed" that grates on my ear as a fourth error. Do you guys really say, "I vacationed New York last week." ?
Even if I substitute "holidayed" (the equivalent verb in UK and Aus English for "vacationed") then I still feel like a preposition is missing. For example, in my version of English, it must be, "I holidayed in Singapore before starting the job in London." and could never be, "I holidayed Singapore before starting the job in London". "Holidaying" is something that is done at or in a location, not something that is done to the location. I am assuming this would be the same with "vacationed" in US usage, but you are welcome to correct me if I am wrong.
So this is where I end up:
Of all the places he had vacationed in, Ian's favorite was Virginia where he went with his best friend.