English Language & Usage Stack Exchange is a question and answer site for linguists, etymologists, and serious English language enthusiasts. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Context: I have an automated test that sometimes fails for unknown reason. It's a sort of false positive: the feature it tests is not broken. Most of the time, either it succeeds or fails for right reason (the feature is broken).

How would you describe this phenomenon of doing the expected job most of the time, but failing a few times for (apparently) no reason?

share|improve this question
up vote 1 down vote accepted

In software engineering, we commonly call these flaky tests.

share|improve this answer

Speaking from a perspective of statistics, your null hypothesis would be that your system has an existing error. The fun begins when you believe that test 123 detects error 123.

There is a good explanation at http://en.wikipedia.org/wiki/Type_I_and_type_II_errors, which has several phrases of various registers.

Type I error - error of the first kind - false positive - crying wolf

Type II error - error of the second kind - false negative - wolf present but failed to see

The latter set is from "The Boy Who Cried Wolf," one of Aesop's Fables.

You correctly applied the term false positive, which means: although the test says there's a problem, there really isn't one.

I think that if you said test 123 was occasionally "crying wolf," then even pointed-haired bosses would understand (perhaps better than "Type I error" or "false positive").

From a system integration point of view, it would be best to fix the test so that it reliably detects the problem.

share|improve this answer
+1 for literature and pointy-haired bosses and being correct, because a +2-3 isn't available. – BrianDHall Apr 24 '13 at 17:38

I'd say something like "the test is breaking intermittently". It's hard to express this idea concisely though because, given the nature of testing, describing a test as "breaking" will often carry the connotation that what it is testing has failed, rather than the test itself.

share|improve this answer

You could describe the false positives as an anomaly - an irregular, unexpected event.

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.