Clarissa Dalloway (the Mrs. Dalloway of the title) is listening to a former suitor, Peter Walsh, detail his plans for helping a married woman obtain a divorce so he can marry her. Clarissa thinks that Peter has never had any idea ("the ghost of a notion") about the feelings of other people. She's phrased this very generally -- that Peter has no understanding of any of the feelings that anyone else might have. But she's really talking about Peter's inability to understand her and her feelings for him.
Clarissa was ambivalent about Peter when she broke with him years earlier, and she's ambivalent about that decision now. Yet Peter has no inkling about those feelings, so much so that he can casually talk to her about his plans to marry another and (as she notes to herself) younger woman.