What's the difference between using "should" and "must" when making guesses?

Question

I will clarify my question first by giving you some examples of the usage I am specifically asking about:

The dish should be ready by now.
You must be exhausted, come in.

Most online teaching materials distinguish the usage between should and must by level of certainty. Using must implies greater certainty. However, it seems to me other factors also come into play. For example, the following sentence is a bit unnatural to me (please let me know if you think otherwise).

Tom isn't home yet. He should have been stuck in traffic.

It seems that when we use should, we are making guesses about a reasonable (often desirable) outcome if things've gone according to plan. Hence, I think it's okay to say:

Tom isn't home yet, but he should have been back 2 hours ago.

My conclusion is we don't use should when we are making guesses that an undesirable event has happened. In other cases, both can be used and imply different levels of certainty.

Ask Tom. He must know.
Ask Tom. He should know.

Could you tell me what you think? Thank you very much.

Answer 1

There's no rule here, but generally speaking it's about the speaker's confidence level. Which is to say, how sure are they that their statement is factual?

The dish should be ready by now.

Means that you think it's ready, whereas

The dish must be ready by now.

Means that you're pretty certain it's ready.

Similarly when Tom must know, you're stating that you're almost certain that he does. Whereas when Tom should know, you're suggesting that he's likely to, but there's no guarantee.

In most usage, that's all the choice of word here means. You could easily say either without significantly changing the context, other than your confidence in what you're saying.

However it's also used for emphasis, when you are equally confident about both: "You must be tired" implies a confidence that you don't actually have, because you don't know how they feel, but you're really just emphasising how long their journey has been, or how busy/stressful their day has been.

It seems that when we use should, we are making guesses about a reasonable (often desirable) outcome if things've gone according to plan. Hence, I think it's okay to say:

In short, no. There may be a slight tendency to use either word in positive/negative scenarios, but I've never encountered it and a listener certainly wouldn't assume it based on word choice. It's possible that "must" is used more often when worried, because we're getting ourselves worked up and stressed and are assuming the worst: but if that is the case, it's just a side effect of your psychology in the moment, not part of the linguistics of the situation.