For example, the car mechanic can't replicate the problem you are having every day, but when you drive it off the service dept, there it is again. Or, when seeing the dentist, the tooth ache goes away, and comes back when you leave.
Around my office, such a problem is called a Heisenbug, a pun on the name of the great physicist Werner Heisenberg, who first described the observer effect (the rule that observing any phenomenon will change it) and the uncertainty principle (the rule that you can know either where something is or how fast it is going, but not both).
The frustrating thing about a Heisenbug is that there is no way to know for certain you have fixed it. You may believe you understand the cause of a particular Heisenbug, you may address that cause, and the Heisenbug may appear to go away, but you never know..
I would call it an "elusive" problem.
1: tending to evade grasp or pursuit
2: hard to comprehend or define
3: hard to isolate or identify
An intermittent problem.
- "My new car has been having an intermittent battery problem."
- "The forecast is for intermittent rain."
- "The patient was having intermittent pains."
Sounds like a gremlin
An imaginary mischievous sprite regarded as responsible for an unexplained problem or fault, especially a mechanical or electronic one: a gremlin in my computer omitted a line
Oxford Dictionaries Online
For a longer discussion, see this article in Wikipedia
Also, there is a wonderful dramatization of the gremlin effect in aviation in Nightmare at 20,00 Feet, a Twilight Zone episode with William Shatner (Galloping around the cosmos is a game for the young. - Captain Kirk).
I happen to like Repairman's Syndrome, where the presence of someone in the know makes the thing work.
Not an exact match to the definition posed in the question, but it is closely related:
When you are using a device or system and it malfunctions, and then you try to cause that malfunction again to show someone, but it does not malfunction this time, you would say the malfunction is not reproducible.
"not reproducible" does not necessarily mean the malfunction will or will not happen later; it just means it does not happen when you try to cause it to happen.
The first thing that comes to mind is:
ˈwakəˌmōl/ noun NORTH AMERICAN
- an arcade game in which players use a mallet to hit toy moles, which appear at random, back into their holes.
"next time you are near a kiddie amusement park, go in and play a round of whack-a-mole"
- used with reference to a situation in which attempts to solve a problem are piecemeal or superficial, resulting only in temporary or minor improvement.
"the site's security team has an ongoing battle against spammers, but it's a game of whack-a-mole"
tending to slip from the hold or grasp or from position: a slippery rope.
likely to slip away or escape: slippery prospects.
- elusive or evanescent.
Cat and mouse
b : a contrived action involving constant pursuit, near captures, and repeated escapes
- a puzzling or inexplicable occurrence or situation
- a person, thing, or situation that is mysterious, puzzling, or ambiguous
- someone or something that is difficult to understand or explain
"This is a complete enigma. I've been to the mechanic three times and they hear nothing. But whenever I'm alone in the car the rattling returns." ~ 20-30 million people a year
It's a "Shy" Problem or symptom. Like when you went to your doctor and the symptoms mysteriously subside or completely go away (usually temporarily) or when you get called in to solve a computer, technical, or mechanical problem, and the problem or symptoms cannot be brought forth. Also called "white coat shy", "technician shy", or simply "a gremlin". However, especially in software, it's "an undocumented feature".