11

In Do We Really Need the S-word? in 'American Scientist', the author Megan D. Higgs writes

Did the people who introduced the word’s use in statistics intend for it to be interpreted according to its current everyday meaning? The answer is not simple. In his 2001 book The Lady Tasting Tea: How Statistics Revolutionized Science in the Twentieth Century, David Salsburg contends the word carried much less weight in the late 19th century, when it meant only that the result showed, or signified, something. Then, in the 20th century, significance began to gather the connotation it carries today, of not only signifying something but signifying something of importance. The coinciding of this change in meaning with a steady increase in its use by more scientists with less statistical training has had a big impact on the interpretation of scientific results. My sentiments echo Salsburg’s: “Unfortunately,” he writes, “those who use statistical analysis often treat a significant test statistic as implying something much closer to the modern meaning of the word.”

I did some searching on Google, but unfortunately, most results are about the term "Historical significance", and I failed to find a definitive reference about the change in the meaning of "significant".

Can you tell me about more about the meaning change, like when and why did it happen?

  • 2
    I would imagine that this had to do with the parallel developments of statistics and experimental science. You should try googling again but focus on the history of statistics — p value, student t-test and the like. – David Jul 28 at 19:15
  • I think „importance“ is not the right Attribute. It can be argued that results which are not significant are not „unimportant“ but simply show no conclusive confirmation at all. (And not all significant results are important either) – eckes Jul 29 at 8:42
  • 1
    Compare this with the dozens of different technical meanings of the term "normal" in different branches of mathematics. – alephzero Jul 29 at 9:50
  • I remember a lawyer explaining to me that he'd seen it argued in court that "significant" merely meant "not insignificant" (i.e. something more than hardly anything). And was therefore to be avoided if you wanted it to mean what it is used for in general usage. – abligh Jul 29 at 10:10
  • I disagree with the quoted text completely. I've never seen anyone interpret "statistically significant" to mean important. It has always meant something like "result which is not product of randomness in data and certainly means something". That something could be completely and utterly unimportant. – Davor Jul 29 at 10:22
8

Salsburg appears to be wrong.

The OED gives meaning 2: "That has or conveys a particular meaning; that signifies or indicates something." from 1573; and meaning 4a: " Sufficiently great or important to be worthy of attention; noteworthy; consequential, influential." from 1642.

He might of course be right that meaning 4a was less common until the 20th century: the OED does not tell us that. But it is not true that it didn't exist until then.

The semantic shift seems very natural and unsurprising to me.

The OED dates the statistical sense from 1885.

6

This link explains why influential statisticians are concerned about the use of the term statistical significance. Editorial from Nature

Their concerns are not merely linguistic, but if we just focus on those, we see that the fundamental problem is that 'significance' in statistics is a technical term that emphatically does not have the same meaning as in non-technical language.

In statistics we say a result is significant if we believe that it is not just a quirk or aberration but reflects a meaningful association (paraphrased from Whelan, naked Statistics (2013)). But a result can be significant in that sense without being significant in any other way whatever, and certainly without being " Sufficiently great or important to be worthy of attention; noteworthy; consequential, influential." In practice very few experimental results realistically can be so described, however compelling the statistical evidence for them.

The ambiguity in the word 'significant' can lead to misunderstanding about the importance of any particular scientific announcement, and ambiguity in language always offers scope for the unscrupulous to deceive people. It is improbable that the statisticians who introduced the technical meaning of significance into their subject wished for anyone to be deceived.

  • 1
    While I agree that the two meanings are different, I'm not sure there's really any ambiguity. In lay usage, people nearly always say "statistically significant" for the statistics meaning and just "significant" to mean important. Suppose that a statistically valid trial of a new drug indicates that it increases survival rate by 1%. Nobody is going to call that result "significant", even though it is "statistically significant." – David Richerby Jul 29 at 9:50
  • All very well (well not quite) but in no way an answer to the question. And from memory one of the concerns of the Nature editorial (always worrying about areas of science they don’t publish) was that the results of clinical trials with too few data points that showed no statistically significant difference were being used to say that a treatment was harmless, when in showed a sizeable difference, that might well be real and would have been shown to be so with more subjects in the study. (The converse of the statistically significant but quantitatively unimportant effect.) – David Jul 29 at 17:38
  • @David maybe I did not express myself clearly enough. To spell it out: Colin Fine showed that the meaning of the word has not changed. My final sentences is my direct answer to the OP's question. – JeremyC Jul 29 at 21:09
3

In the scientific community, the use of significant can largely be attributed to the work of RA Fisher (who is among the subjects of The Lady Tasting Tea, and the source of the title of the book).

In particular, scientific results are often deemed significant, and therefore publishable in a peer-reviewed journal, if the p value is 0.05 or less.

Fisher describes his significance test and p values in Statistical Methods for Research Workers (1925). Of note:

The value for which P = .05, or 1 in 20, is 1.96 or nearly 2; it is convenient to take this point as a limit in judging whether a deviation is to be considered significant or not. [Emphasis added]

The article P Value and the Theory of Hypothesis Testing: An Explanation for New Researchers contains a good summary of p values, hypothesis and significance testing, Fisher's role in this development, and some of the common misconceptions around p values.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.