I've heard this quote in the past, but have never really found any reference to the meaning of it. "They say! What say they? Let them say." was the personal motto of reclusive millionaire Edward F. Searles.
This would be in response to a reproach grounded in folk wisdom or popular opinion—perhaps interrupting the reproach after just two words, the two spat back at the beginning of this “motto.” To paraphrase:
You throw popular opinion in my face? What is that popular opinion? No, never mind—let the great mass of fools say whatever they will.
SUPPLEMENTAL to Brian Donovan's ANSWER:
The slogan was well known in the later 19th and early 20th century: Lily Langtry, for instance, is said to have had it either carved over her mantle or displayed in stained glass in the house the Prince of Wales built for her, and Bernard Shaw found it carved over the mantle of the house he bought in Adelphi Terrace.
The unusual syntax may be attributed to an origin in Renaissance Scots; it appears to have been originally the motto of Marischal College in Aberdeen, founded by the Earl Marischal of Scotland George Keith in 1593. A number of internet sources cite the oldest version as THAI HAIF SAID : QUHAT SAY THAY : LAT THAME SAY. But the earliest actual citation I have found is from 1824, in one of John MacCulloch's 'letters' to Walter Scott in The Highlands and Western Isles....
There are several posts in Notes & Queries claiming classical precedents, Latin (Aiunt. Quid aiunt? Aiant.) and Greek (λέγουσιν ἄ θέγουσιν λεγέτωσαν), and Keith was said to be a fine scholar in these languages.
"They Say"-The people will always have opinions, advise and suggestions for you.
"What Say They?"- In a way it means, how does it matter what they say? (its your life, your decision you are the actor) and also it could imply what say do they have in your life(authority to speak about someone else's life)
"Let them say"- They will always say...you cannot stop them. So let them say. To take or leave is entirely your decision.
It's origins are Scottish -- Thay Haif Said: Quhat Say Thay? Lat Thame Say -- and date back to Clan Keith in the 16th Century
If you want to learn more, there is an interesting article here: http://rdhardesty.blogspot.com/2014/03/say.html