Take the 2-minute tour ×
English Language & Usage Stack Exchange is a question and answer site for linguists, etymologists, and serious English language enthusiasts. It's 100% free, no registration required.

Context: I have an automated test that sometimes fails for unknown reason. It's a sort of false positive: the feature it tests is not broken. Most of the time, either it succeeds or fails for right reason (the feature is broken).

How would you describe this phenomenon of doing the expected job most of the time, but failing a few times for (apparently) no reason?

share|improve this question
add comment

4 Answers

up vote 1 down vote accepted

In software engineering, we commonly call these flaky tests.

share|improve this answer
add comment

Speaking from a perspective of statistics, your null hypothesis would be that your system has an existing error. The fun begins when you believe that test 123 detects error 123.

There is a good explanation at http://en.wikipedia.org/wiki/Type_I_and_type_II_errors, which has several phrases of various registers.

Type I error - error of the first kind - false positive - crying wolf

Type II error - error of the second kind - false negative - wolf present but failed to see

The latter set is from "The Boy Who Cried Wolf," one of Aesop's Fables.

You correctly applied the term false positive, which means: although the test says there's a problem, there really isn't one.

I think that if you said test 123 was occasionally "crying wolf," then even pointed-haired bosses would understand (perhaps better than "Type I error" or "false positive").

From a system integration point of view, it would be best to fix the test so that it reliably detects the problem.

share|improve this answer
    
+1 for literature and pointy-haired bosses and being correct, because a +2-3 isn't available. –  BrianDHall Apr 24 '13 at 17:38
add comment

I'd say something like "the test is breaking intermittently". It's hard to express this idea concisely though because, given the nature of testing, describing a test as "breaking" will often carry the connotation that what it is testing has failed, rather than the test itself.

share|improve this answer
add comment

You could describe the false positives as an anomaly - an irregular, unexpected event.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.