Assuming that this question is not specific to the mathematical domain, and that answering the question as it relates to general English applies, then answering this relies on a combination of style and the intention of the author. (Which means that it's open to interpretation and there isn't a single universal answer.)
For the sake of providing some kind of answer, I will quote the Chicago article referred to in a comment under the question.
From The Chicago Manual of Style (17th ed.), 6.25 (it's behind a paywall, so won't be accessible without a subscription):
A dependent clause that follows a main, independent clause should not be preceded by a comma if it is restrictive—that is, essential to fully understanding the meaning of the main clause. For instance, in the first example below, it is not necessarily true that “we will agree to the proposal”; the dependent if clause adds essential information.
We will agree to the proposal if you accept our conditions.
Paul sighed when he heard the news.
He wasn’t running because he was afraid; he was running because he was late.
If the dependent clause is merely supplementary or parenthetical (i.e., nonrestrictive, or not essential to the meaning of the main clause), it should be preceded by a comma. Such distinctions are occasionally tenuous. In the fourth example below, the meaning—and whether the subject is running or not—depends almost entirely on the presence of the comma (compare with the third example above). If in doubt, rephrase.
I’d like the tom yum, if you don’t mind.
At last she arrived, when the food was cold.
She has a point, whether you agree with her or not.
He wasn’t running, because he was afraid of the dark.
Because he was afraid of the dark, he wasn’t running.
This explanation indicates if you should use a comma in the sentence in the question or not.
However, it's not the exact wording of the sentence that determines if there should be a comma. What matters is what the author is trying to convey. If the author means for the information to be essential, then there should be no comma; if the author means it to be supplementary, then there should be a comma.
Consider the sentence in the question:
？ We say that a foo admits a bar [,] if baz is quz.
Try rephrasing it in two ways: one where the information is essential and the other where it's supplemntary. Then, determine which of those versions best conveys the meaning that is intended.
I will starve to death only if I don't eat.
→ I will starve to death if I don't eat.
We say that a foo admits a bar only if baz is quz.
→ We say that foo admits a bar if baz is quz.
I will starve to death. (By the way, I hope you don't mind.)
→ I will starve to death, if you don't mind.
We say that foo admits a bar. (By the way, baz can be quz.)
→ We say that foo admits a bar, if baz is quz.
Whichever version best represents the intention of the author is the version to use. (According to Chicago anyway.) Of course, it's much better if you actually are the author so you don't need to make assumptions about intentions.
Take one of the example sentences from Chicago that had a comma and remove it:
I'd like tom yum if you don't mind.
In the original version, there was a comma and if you don't mind was supplementary; here if you don't mind is essential (in terms of communicating grammatical information). In short, without the comma, it's now saying this:
I'd like tom yum only if you don't mind.
In other words, if you do mind, then I won't like it.
This sort of makes sense in one interpretation, but it also fails to make sense if considered in light of whether or not you like the taste of the soup. Whether or not somebody else minds shouldn't logically affect your taste buds. However, idiomatically, the sentence would probably be taken to mean that you won't enjoy eating the soup if the other person is bothered by you doing so.
Chicago is not the only style guide, and other style guides might give different guidance in terms of punctuation and grammar (and even author intention). But since Chicago was mentioned, this is how Chicago would interpret the question.