ie. Barry couldn’t go to such and such bar anymore because it reminded him of where he met his first love that didn’t work out and he had to much emotional baggage
This is what Wikipedia says about emotional baggage:
Emotional baggage is an everyday expression that correlates with many varied but similar concepts within social sciences, self-help movements, and other fields: its general concern is with unresolved issues of an emotional nature, often with an implication that the emotional baggage is detrimental.
As a metaphorical image, it is that of carrying all the disappointments, wrongs, and trauma of the past around with one in a heavy load.
Different context can have it mean different things. However, given the above, I think the single work most likely to represent it in a lot of cases would be trauma:
1 a : an injury (such as a wound) to living tissue caused by an extrinsic agent
1 b : a disordered psychic or behavioral state resulting from severe mental or emotional stress or physical injury
1 c : an emotional upset
// the personal trauma of an executive who is not living up to his own expectations
— Karen W. Arenson
Baggage is something you have to carry around as an unwanted chore; it's not something that you look forward to dealing with.
Figuratively speaking, it would be uncommon for somebody to use the term emotional baggage to refer to something positive or happy. If they aren't talking about something emotionally scarring (traumatic), then they would normally either not use the term at all or they would qualify it in some way.
To paraphrase the example sentence slightly:
It reminded him of where he met his first love that didn’t work out, and he had too much trauma from that experience.