How should one end - ie, sign-off - a letter that begins with "To whom it may concern"?
According to Miss Manners, the answer is "Yours faithfully". That is what I use.
"Sincerely" is a fairly common formal sign off.
You may also want to check out the answers to this question.
It partly depends on the contents of the letter. If I'm submitting, say, a letter of reference, that might warrant something a little different than a general inquiry, a request for a favor, or a formal complaint.
As was stated previously, Sincerely is perfectly acceptable, and almost always a safe option. But if you wanted to soften a complaint, then Respectfully might be a better choice.
By default, in the To-Whom-It-May-Concern situation, I don't know who will be reading the letter (otherwise, I would have begun with something less generic). With that in mind, I'll sometimes want to end with my contact information, in case the matter needs further discussion to resolve:
If you have any questions, do not hesitate to contact me. Thank you, John R. Doe (555) 867-5309 jrdoe18220@ELU.com