What does harsh mistress mean in this sentence?
Nostalgia can be a harsh mistress.
It deliberately has two meanings, using the ambiguity for poetic effect.
One meaning of mistress is as the feminine form of master, and hence it is saying nostalgia must be obeyed, (and is harsh).
Another meaning of mistress is as a woman one (traditionally based on heterosexist assumptions, a man) has a romantic and/or sexual affair with. Hence it is saying that nostalgia is "courted" as something beloved, but responds harshly (in many ways a more fraught position than having a harsh mistress in the other sense).
The second is probably the predominant meaning, but the first is deliberately still in there, which affects its poetic impact.
Just where the balance lies depends on the subject:
In the case of nostalgia, it's hard to see the first meaning as applying much as there's always a voluntary aspect to engaging in nostalgia.
In the case of the sea (a common subject of the idiom), the balance is greater - those who work on the sea typically chose that career out of a love for it, but must also continue to work it to make their living even if they are no longer as fond.
In the case of The Moon in the famous Heinlein novel, both meanings are definitely present, but given that the story is about a revolt against an undemocratic leadership, the author more deliberately wants the dichotomy of mistress/slave brought to mind.
According to Google Ngram, the 1830s is where we see the turn of phrase "harsh mistress" take root to describe the world, Russia, and actual females. Its usage there invokes the female equivalent of a "harsh master".
The phrase is used to describe someone (or something) with exacting standards that are hard, if not impossible, to satisfy.
Heavenly bodies like the earth and moon, as well as countries, are generally considered feminine in English (and explicitly so in languages like Spanish: la luna, la tierra), but this doesn't necessarily confer a woman's status (or historical lack thereof) to the object itself. For instance, "France deployed her troops" does not imply France's characteristics are "feminine" (as defined by patriarchal society?).
Without context, it's hard to discern the meaning(s) in the sentence. I lean toward "woman master" rather than "kept woman." Insofar as women are stereotyped as "love objects", the sentence could mean that nostalgia exerts some kind of lure that is ultimately unrewarding. However, it's not necessary to read in a woman's status, as the sentence means that nostalgia has some form of dominance despite nostalgia's lack of merit, regardless of "her" role.
Edit: removed non-existent Biblical reference, lazy citing on my part!
Mistress - something you fall dearly in love with.
The term X "is a harsh mistress" is an aphorism. "Mistress" is the traditional title for a head "Teacher" in a boarding school... As in "Head Mistress."
The meaning should be obvious to all but the least intelligent.
It means X "is a hard teacher."
For example; Reality is a harsh mistress, means "Reality is a hard teacher, lesson, etc. et al words synonymous with learning and/or experience.
Early literal use of 'a harsh mistress'
The earliest instances of "a harsh mistress" in English refer to the female counterpart of a master— that is, either an employer (of a servant or employee) or an owner (of a slave). From a 1713 (fourth edition) translation of Terence's The Eunuch:
From Sarah Scott, The History of Sir George Ellison (1766):
From Emma Roberts, "The White Wolf," in The Mirror of Literature, Amusement and Instruction (December 2–9, 1826):
From George Sargent, The English Peasant Girl (1830s[?]):
And from Charlotte Brontë, Villette, volume 1 (1853):
Early figurative use of 'a harsh mistress'
Early figurative use of the phrase follows in the same vein. From a letter from Michael Faraday to the biographer, dated December 23, 1829, in John Paris, The Life of Sir Humphrey Davy (1831):
From Robert Willmott, Conversations at Cambridge (1836):
From "Duty," in The Young Woman's Journal (June 1913):
From Cyrill Hall, Triumphs of Invention (1920) [combined snippets]:
Burgeoning figurative usage of 'a harsh mistress'
More recently, writers have extended the figurative use of "a harsh mistress" considerably. From James Cooper, Sheila of Big Wreck Cove: A Story of Cape Cod (1922):
From Paule Henry-Bordeaux, The Circe of the Deserts (1925):
From Young Men, volume 52 (1926):
From Otto Lütken, Congo Gods (1929):
From Robert Frothingham, Trails Through the Golden West (1932):
From Betty Jacobsen, A Girl Before the Mast (1934):
From Geoge Slocombe, The Heart of France: Parisian, Provincial & Peasant (1934):
From The Spectator: Property Insurance Review, volume 2 (1936):
And that takes us through 1936.
The double meaning of mistress that Jon Hanna discusses in his answer to this question is undoubtedly a factor in the sense of the wording as it is used today. Nevertheless, during the first century of its use, the phrase "X is a harsh mistress" seems to have used mistress primarily in its "female counterpart to master" sense, not in its "illicit girlfriend" sense.