It is such a common colloquialism that discussions of its origin is hard to find. But I assume that its a reference to how when boxing there is a tendency to protect the face leaving one's abdomen exposed. A gut check would be to throw a punch where someone isn't prepared to defend.
gut check definition TFD
a pause to assess
The notion of the intestines as a seat of emotions is ancient (see bowel) and probably explains expressions such as gut reaction (1963), gut feeling (by 1970), and compare guts. Gut check attested by 1976. etymonline
And Urban Dictionary alludes to U.S. Army and Tennis (racquet strings of cat gut)
My research finds no boxing origin.
Green's Dictionary of Slang suggest its origin is from sports, but doesn’t a specific one:
Gut-check (n.) [orig. sporting use] [1970s+] (US)
- a quick reassessment of strategy and stiffening of morale.
The New Partridge Dictionary of Slang and Unconventional English attests its usage a bit earlier, from the late ‘60s.
a test of courage or determination US — Fred Hester, Slang on the 40 Acres, p. 15, 1968 “
The UD says that it was originally an expression from tennis:
Gut check was originally a tennis term to describe a player pausing a moment to think about what just happened by closely examining and possibly adjusting his racquet strings. Back in the day strings were made of cat gut, commonly refered to as gut. This usually occurred after a great shot by an opponent. As with so many terms it has become popular to describe something very different.