I am looking for a word that refers to the state of extreme, deep melancholy mixed with pain, anxiety, and wistfulness; the state in which one feels that their hearts are going to burst with sadness.
Anguish is a suitable word to express the type of pain, a mix of anxiety and grief, that arises when a relationship breaks up or a loved one dies.
Alternatively, another expression which often accompanies grief is sorrow, a feeling which is more intense than sadness.
a feeling of deep distress caused by loss, disappointment, or other misfortune suffered by oneself or others. an event or circumstance that causes sorrow
source: Oxford Dictionaries
I've noticed no one has yet suggested Despair
1. loss of hope; hopelessness.
2. someone or something that causes hopelessness: He is the despair of his mother. verb (used without object)
3. to lose, give up, or be without hope (often followed by of ): to despair of humanity. verb (used with object)
4. Obsolete . to give up hope of.
Though the definition doesn't quite meet the description you've presented, it has often been a single word I've used to try and describe what (I believe) you're referring to.
"Melancholy" is an appropriate umbrella term for all the emotions you identify. In casual usage, "melancholy" appears to be less overwhelming than its classical definition, but you might emphasize its degree (ie, "all-consuming melancholy").
I refer you to Robert Burton's magisterial Anatomy of Melancholy, in which he defines his subject as :
(T)hat ... melancholy which goes and comes upon every small occasion of sorrow, need, sickness, trouble, fear, grief, passion, or perturbation of the mind, any manner of care, discontent, or thought, which causeth anguish, dullness, heaviness and vexation of spirit, any ways opposite to pleasure, mirth, joy, delight ... Melancholy in this sense is the character of mortality.
Burton identifies melancholy as a confluence of many mental and emotional states. These states include the "pain, anxiety, and wistfulness" you mention. Among them, Burton also numbers (in the beginning of Section 3 of the Anatomy) :
Fear and sorrow without a just cause, suspicion, jealousy, discontent, solitariness, irksomeness, continual cogitations, restless thoughts, vain imaginations ...
He also ties melancholy to physical symptoms, including (again, in Section 3):
convulsions, cold sweat, heaviness of heart, palpitation, cardiaca, fearful dreams, much waking, prodigious fantasies ...
How about woe?
woe /woʊ/ noun
- grievous distress, affliction, or trouble: His woe was almost beyond description.
- an affliction: She suffered a fall, among her other woes. interjection
- an exclamation of grief, distress, or lamentation.
One relevant option is despondency. From The American Heritage Dictionary of the English Language, Fourth Edition (2000):
despondency n. Depression of spirits from loss of hope, confidence, or courage; dejection.
Merriam-Webster's Eleventh Collegiate Dictionary (2003) defines despondent as follows:
despondent adj. feeling or showing extreme discouragement, dejection, or depression.
A usage note beneath that definition clarifies how several related terms differ:
DESPONDENT, DESPAIRING, DESPERATE, HOPELESS mean having lost all or nearly all hope. DESPONDENT implies a deep dejection arising from a conviction of the uselessness of further effort. DESPAIRING suggests the slipping away of all hope and often despondency. DESPERATE implies despair that prompts reckless action or violence in the face of defeat or frustration. HOPELESS suggests despair and the cessation of effort or resistance and often implies acceptance or resignation. [examples omitted]
One vivid metaphor for despondency is John Bunyan's "slough of despond" in The Pilgrim's Progress, which a Wikipedia article describes as "a deep bog ... into which the character Christian sinks under the weight of his sins and his sense of guilt for them."
Another term with something of the same sense of existential pointlessness is anomie, which the Eleventh Collegiate defines, on the personal level, as follows:
personal unrest, alienation, and uncertainty that comes from a lack of purpose or ideals
Ennui. Among others, wikitionary says:
A gripping listlessness or melancholia caused by boredom; depression
SAUDADE n. A Nostalgic Longing to Be Near Again to SomeThing -or- Someone That Is Quite Distant or That Has Been Loved and Then Lost. ^the love that remains" ------+-------------------
DISTRAUGHT Has Been Described As Being the Most Helpless and Pained a Person Can Feel Emotionally.
Priyah Claims It Is the Sinking Feeling in Your Heart and Is Accompanied by That Pain That Is Felt in the Chest Because You Know the Reason Behind It, Yet It Is When You Can't Do Anything About It.
best Described as: A Situation In Which No One Except the Person Who Is Responsible Can Access You. You Are Completely Closed Up to the Rest Of the World.
It Is Only That Person That You Show Your Pain To. And No Matter What He/She Does, It Won't Make the Pain Any Better
Priyah Continues on By Stating That Even When the Other Person Is All Concerned and Worried about You, It Doesn't Make a Difference, Because You Are Do Far Deep Into That Emotion, That Everything Else Seems Secondary.