Who is for a subject, whom is for a direct or indirect object).
A trick for remembering this is to replace who with he or she, and whom with him or her.
Subject: He gave you flowers? -> Who gave you flowers?
Direct Object: You called him? -> Whom did you call?
Indirect Object: You gave flowers to him? -> To whom did you give flowers?