I agree that the answer is a non sequitur for cases where the response is truly unrelated, and that this tactic may be employed as a dodge, as in the example. But you can also answer with a seeming non sequitur as a way of answering obliquely.
This description would fit the example if B was late because he stopped to buy the bagel, and could express B's valuation of gustatory pleasure over punctuality.
There's a great example of this in the movie LA Confidential, when Kevin Spacey lies to a struggling boxer about getting his brother's prison term commuted. After the boxer has shared his information, fulfilling his part of the deal, he asks, "I'll hear from you about my brother?"
By way of reply, Spacey pantomimes a boxer's defensive stance and says: "Keep it up, Lenny. Up."