“Our unfinished symphony” lacks the notion of absolute finality that you seek (after all, until one of the parties to the relationship/composers of the symphony dies, they could theoretically get back together and finish their symphony), ...
... but to the extent that one really wanted to, they could add to or paraphrase it to remove all doubt, perhaps as follows:
Grieve not for what might have been. Rather please try to think of it fondly as “our forever unfinished symphony.”;
Grieve not for what might have been. Rather please try to think of it fondly as “our never-/ne'er-to-be-finished symphony”;
Grieve not for what might have been. Rather please try to think of it fondly as "the never-/ne'er-to-be-composed notes/measures/lines/finale of our unfinished symphony.
(‘Unfinished Symphony’ by Richard Schweitzer from musicxray[dot]com)
(example of “never-to-be-composed” used in a sentence with “what might have happened”; from ‘Opera in History: From Monteverdi to Cage’ by Herbert Lindenberger, via Google Books)
(example of “unfinished symphony used in a sentence with “what might have been”; from Suzanne Strutman’s Introduction to Thomas Wolfe’s unfinished novel, ‘The Good Child's River’; via Google Books)
With the added tag "poetry" in mind:
Along the general musical lines of the above variations on the theme of “unfinished symphony” (but admittedly, more comparable to @CandiedOrange’s "The road not taken"), you could consider the idea of “the/our unsung song” or preferably, as inverted and used in the title of a book by Monique Ritter: “The Song Unsung.”
The provided link is to the website of the novel’s publisher, The Strategic Publishing Group, where a synopsis uses the notion of “what might have happened on another path” (in direct reference to the title of Mr. Frost’s poem) in its first paragraph, as well as both CandiedOrange’s “Regrets” and "The road not taken" together in paragraph 3.
(I fear that this edit is reading more and more like a reason for giving favorable consideration to CandiedOrange’s good answer than for doing so to my answer!)
Anyway, in response to your addition of the tag “poetry” (not “famous poetry,” mind you), in addition to the book mentioned above, I’ve also found a nice poem with the same title:
“The Song Unsung” by Abdul Wahab
So many people have sung so many songs
On and from earth to heaven
but they have not sung me
I am the song
Though I have the best beat
I move with a great rhythm
Yet I have not been put on music
The reason I know not
Nor I want to know
As it could be anything
But I am sure that it is not their ability
Neither the quality which I carry
If any one sings me
Instantly he or she will be
On the cloud nine
As I am the song
None has ever touched me
Oh singers of the world
If you sing me
I shall make your fame spread so wide
That your name can not hold
As I am the song unique
And totally unsung.
Although this poem, as most poems, is subject to different interpretations, as I interpret it, the poet is saying that the song that we leave unsung in our lives would be/have been far better than the one we chose to sing ... the one we are now singing (at least in our own minds, in retrospect from our own present lives).
If my pessimistic interpretation is correct (and I hope it's not, even if that would lessen the poem's relevance to your question), the poet’s message is sad and very sobering.