The term derives from mathematics. Linear processes are ones that progress in a simple additive way. For example a linear series 1, 2, 3 … may be reasonably confidently predicted to develop as … 4, 5, 6 …
A non-linear process is one in which the progression is not so simple. It may be predictable but increasingly strong, as with 1, 2, 4, 8, 16… or it maybe unpredictable, as with 3, 23, 142, 28… ?
By analogy, if someone goes non-linear, it is suggested that their behaviour becomes increasingly intense, or becomes difficult or impossible to predict.
See for example Wikipedia:
In mathematics and science, a nonlinear system is a system in which the change of the output is not proportional to the change of the input. Nonlinear problems are of interest to engineers, biologists, physicists, mathematicians, and many other scientists because most systems are inherently nonlinear in nature. Nonlinear dynamical systems, describing changes in variables over time, may appear chaotic, unpredictable, or counterintuitive, contrasting with much simpler linear systems.
Examples of the application of the linear/nonlinear concept to emotions are easily found. For instance:
“In this chapter, the authors define what nonlinear dynamic systems are and show how emotions posses the characteristics of such systems. The authors then discuss the shortcomings of linear models and how linear dynamic models are better equipped to answer basic questions about the nature of emotion.”