I'm not a native speaker, but at least here in California, it seems that "sweetheart" refers predominantly to women.
How customary is for a woman to call man "sweetheart"?
English Language & Usage Stack Exchange is a question and answer site for linguists, etymologists, and serious English language enthusiasts. It only takes a minute to sign up.
Sign up to join this communityI'm not a native speaker, but at least here in California, it seems that "sweetheart" refers predominantly to women.
How customary is for a woman to call man "sweetheart"?
It isn't gender specific, at least not in Australia.
It's very common to hear males being called "sweetheart" or "sweetie" by their partners.
You also often hear the term "high school sweetheart", used in situations such as
"his parents were high school sweethearts"
or
"he was her high school sweetheart"
and is used equally to describe both males and females.
The use of "sweetheart" is somewhat gendered in the US, but on the namer's side, rather than the name-ee's. Specifically, women can call anyone "sweetheart" but straight men usually only call girls and women "sweetheart".
In general, a man who casually calls women "sweetheart" (usually older men in my part of the US) is going to have some other term for men, whereas women can use it for both. Similarly, mothers are more likely to call all members of the family "sweetheart" whereas fathers are more likely to call wives and daughters that but choose something like "buddy" for sons. The same holds true for similar endearments like "sweetie" and "honey" or "hon".
It's hard to find written evidence for this kind of mostly-spoken phenomenon. However, a Google Books search for the phrase "sweetheart her father said" turns up 23 actual examples, almost all of which are actually a father calling his daughter "sweetheart"; in contrast, there are only nine viewable instances of "sweetheart his father said" and only two of those are a father actually calling his son "sweetheart"—in the other seven, "his" father is speaking to some other, female character. I suspect the disparity would be even greater for, say, storekeepers and bartenders talking to patrons or hard-bitten detectives talking to clients, if I could find a way to search for that.
Obviously this is a broad generalization, not an absolute rule. There are undoubtedly men who defy that gender-stereotype, either as an intentional subversion or just as a personal quirk. However, the upshot of the general tendency is that you are probably statistically more likely to hear "sweetheart" applied to women than to men, which may account for your sense that it is used more often for women.
As a term of endearment between two people who are already close, there is no gender rule. It can be used between couples for example. The more complex case is when it's used as a diminutive, such as from parent to child. There is a dynamic there that is ok in families but not ok between strangers. I would be fine, for example, calling my teenage son sweetheart, but I would never use that term with (say) a server in a restaurant.
Sometimes the "stranger" rule is inverted when age reverses the power dynamic a little. For example, I am not remotely bothered when a sixty year old server in a diner calls me "hon". But I would never do it to them.