The expression is making an analogy to a competition. When referring to winners and rankings after a competition, we say that the winner "came in first place," meaning that they were ranked first out of all competitors. Likewise, the second best performance (who did not perform as well as first place, but outperformed all others) "came in second place."
In general, to say that something "came in a close second" suggests that while it wasn't as good as first place, it was almost as good and still very worthy of attention.
It can be contrasted with saying that someone won "by a mile" or "by a landslide", phrases which imply that the second place was not nearly as good as first place.