Statistical significance is a sophisticated concept, and arriving at statistical significance with raw data involves even more sophistication. Perhaps one person in a thousand knows what "statistically significant" means. I do, but I'll be darned if I could explain it effectively to a person who'd like to know, and that's NOT because I grasp the concept so well that I'd have difficulty dumbing it down!
I say, "When in Rome, do as the Romans." In other words, feel free to use the term without explanation whenever you are in front of your fellow dataphiles, but steer clear of using the term in the presence of laypersons.
In a speech before a mixed audience (dataphiles and laypersons), the occasional use of the term "statistical significance" to reach the former is fine, as long as you break it down and explain it simply to reach the latter. In other words, throw the former a bone; give the latter some milk--it's easier to digest. Even the brilliant apostle Paul said, "I have become all things to all men that by all means I may win some" (I Corinthians 9:22b).
As a rhetorician I have been trained to think pragmatically when attempting to communicate with a mass audience. A good rhetor needs to adapt his or her message to each and every audience, keeping in mind that the audience, the purpose of the speech, and the nature of the occasion determine in large part what one says and how one says it.
And so it is with writing. First answer the following questions: What is my message? Who is my audience? What is my purpose? What is the occasion? Only after answering those questions should you move on to the traditional canons of invention, organization, delivery, and style.
"Statistical significance" is a stylistic choice, as is "significant," but if you have doubts about using either, perhaps you'd be better off using a different term. My goodness, the English language has more words than any other language on earth! Consult a good thesaurus or two; I'll bet you'll find at least a dozen or two equivalent terms (more-or-less) for "significant" and its cognates.