According to the OED (and you can see this page for free), this usage is "originally and chiefly N. American". The first quote it cites is in Domestic Manners Of The Americans (1832), which I think refers to this:
What is it, man? say.
It can also start a sentence, as OED lists in subsequent entries:
Say—d'you run with our machine?
— Lantern (N.Y., 1852)
Here's another example from 1857:
"Say! What are you laughing at? I only did it for fun."
— The Bay-path: A Tale of New England Colonial Life
However, I think the gangster association comes from the 1920s, based on this Huffington Post article:
In the 1920s, gangsters like Jack McGurn - Al Capone’s main assassin and general of his troops - would begin many sentences with “Say.” For example: “Say, what’s the beef?” Or, “Say, I wasn’t anywhere near the place. See?” Say and see were like bookends to the street comment.
— Hipster Language: How To Talk Like A 20s Gangster
As you can see, you didn't mishear; gangsters also say "see". For example, in the 1931 movie Little Caesar: "You're hangin' around with me, see?"