I suspect that the person who said "We're going off tangent here" may have had in mind not "off track" but "off topic." Going "off track" may indicate either jumping the track (as a derailed train might) or leaving the marked trail (and being in danger of getting lost). Both of those possibilities are fairly strong negative descriptions of going wrong.
In contrast, "going off topic" or "getting off topic" simply means being diverted into an area that isn't the focus of present business. In this respect, it has much the same meaning as "going off on a tangent."
In geometry a "tangent" is a straight line that touches a circle or sphere at exactly one point and then runs infinitely away from that point in both directions. Going off topic is likewise moving steadily away from the point of the meeting or discussion. The problem in both cases isn't that the conversation is utterly derailed or lost in the wilderness; it's that starting from a point of contact with the relevant topic, the conversation is moving farther away from it.
A search using the Ngram Viewer indicates that the phrase "go off tangent" goes back at least 59 years and that it is growing in popularity rather than declining. The earliest possible instance I found in the search results is a somewhat ambiguous snippet from a novel by Edward Dahlberg, Those Who Perish (1934):
...asked the Secretary in a tall-ish voice which went off tangent. "I make it a motion, Mr. Chairman," said the manufacturer of broadcloth. "Now I think we're getting somewhere," said the man with the adenoidal boom. ...
The oldest clear-cut instance I found is from Hearings of the U.S. Senate Committee on Small Business, volume 3 (1954), which involves testimony from a man who evidently was held in one of the infamous Japanese-American internment camps of the 1940s:
Mr. Shibata. And in the camp I became active in different work. Am I going off tangent?
Mr. Purcell. They will stop you if you go off tangent.
Mr. Jonas. Quite all right.
Mr. Shibata. I was Secretary of the city council in Tule Lake Camp and the camp was self-governed, and one of the major issues that came up ...
From the same year, in Edmund Bergler, The Writer and Psychoanalysis, we have this specimen:
The analyst, misunderstanding what is going on, interprets the oedipal repetition, with the result that facts and interpretation are at odds and the whole analysis goes off tangent.
The next oldest is also from political discourse, in this case the Nigeria House of Representatives, Parliamentary Debates, volume 18, part 2 (1965):
[Unidentified speaker]: The Member for Gaya North (Alhaji Yunusa) made some very important points though he was going off tangent, but there are arithmetical and geometrical tangents which are admissible, even in trade. This his off tangent is admissible even in trade. He said that we should—
Mr. A. T. Mbegbu (Bende West): I think that when we refer to tangents we talk of things falling off. Is the Minister of Trade now trying to include the points "off tangent" or is he trying to exclude them?
Representative of the numerous examples that the Ngram Viewer found from recent years is this one from Anton Manning, The Military's Relationship with the Media (2007):
Another officer narrated the experience he had with media requests to interview U.S. Army Chief of Staff Eric Shinseki. When the cameras were off, the media members briefed Shinseki on the types of questions they had and what they were looking for. “When it came down to it with the cameras rolling, they would go completely off tangent.”
In none of these examples does "going off tangent" express the intended idea as clearly as would "going off on a tangent," "going off topic," "going off track, " or "going off target."