The sentence is grammatically correct as written. The primary source of the confusion is probably because of the "phrase" x = 2.
ORIGINAL: The maximum number of solutions is given by the formula, x * x. For example, consider the case, x = 2, when this maximum is 4.
The first sentence introduces the idea that we are referring to a maximum. This is important only because it helps us understand that we are talking about a maximum, which helps us to understand the use of "this" in the subsequent sentence.
So let's look at the second sentence:
ORIGINAL: For example, consider the case, x = 2, when this maximum is 4.
SIMPLIFIED: Consider the case when this maximum is 4.
This is semantically essentially the same as "Consider the case when the maximum is 4"; the use of this just helps tie it back to the first sentence.
This sentence has a similar structure as the following (grammatical) sentences:
The scenario where that happens is unlikely.
I am interested in the time when you were away.
Tell me about the scenario when that happens.
Your sentence is correct because you have an adjectival clause (when this maximum is 4) that modifies the noun case. The word when is needed to introduce the clause "this maximum is 4."