The ambiguity of pronoun.

Question

And now, a little detail bothered him. Not about the case. No, that was fine. It was something else.

Something about Allie.

But damn, he couldn’t put his finger on it. He was fine when she’d left this morning. At least he thought he was. But sometime after her call, maybe an hour or so, something clicked in his mind. The little detail.

Detail. Something insignificant? Something important? Think . . . think . . . Damn, what was it?

His mind clicked.

Something . . . something . . . something said? Something had been said? Yes, that was it. He knew it. But what was it? Had Allie said anything on the phone? That had been when it started, and he ran through the conversation again. No, nothing out of the ordinary.

---

This clip is from The notebook by Nicholas Sparks. I asked question to pinpoint many pronouns. It's so ambiguous that I can't be sure about them.

1)that was it.

That indicates "a little detail" and it means "what Allie said on the phone"?

2)That had been when it started,

If I revive the left-out part on the above sentence, It will be like "That had been on the phone when it started", is it right?

Then, in this sentence, what is "that" indicating and what is "it" pointing to?

Answer 1

The narrator is musing on something that "bothered him," but he can't decide what the bothersome thing is. All he knows is that the bother is a minor thing -- a detail, a little detail, which might be important or might not. He considers whether the bother is something about the case, and he rejects the possibility; he considers whether the bother was something mentioned in a phone conversation. And he decides

Yes, that [something said on the phone] is it [the bothersome thing].

But he's still not close to figuring out "it," the bothersome thing actually is. Something said on the phone, but something Allie said? No, he rejects that consideration as he runs through (i.e., reviews in his mind) what Allie said and decides her words were "ordinary." But

That [the phone conversation] had been when it [ the bothersome thing] started [to bother him].