Which one is correct?
a- We deal with the problem x, where the input domain is vectorial.
b- We deal with the problem x where the input domain is vectorial.
I want to notify that particularly for the problem x the input domain is vectorial.
Each has a different meaning.
For sentence a, we deal with problem x, where the input domain is vectorial, consider this sentence: instead of deal as the verb, we are with x, where [something is happening]. In this case, it's easier to see that 'where something is happening' is about x. So in sentence a, the input domain is vectorial in problem x.
For sentence b, consider we are with x where something is happening. Because 'with x' is not necessary for the sentence (cf. "we are with Bob where the food is") we can take it out to understand our sentence better. This gives us "we are where something is happening". This shows us that 'where something is happening' works as an adverbial phrase modifying 'are'. When you don't have a comma, you essentially have the sentence "We deal where the input domain is vectorial" - this makes sense grammatically, but I don't think it really makes sense semantically - how can you deal where this is happening?
The grammar in both is fine, but for the exact meaning you've got, you need a comma.