The expression in the sense that, implies that from the existing senses, the speaker is concerned with a particular one. So it's restrictive. And it can be negated without ambiguity. But I agree that if another negation follows in the sense that, the sentence can become intricate.
Your funny example,
She is not cute in the sense of being a cat.
means that
She is cute, but not in the sense of being a cat [i.e. not because she is a cat, not due to her "catness", it is not her being a cat that makes her cute - maybe she has a certain affinity to emotions or gestures that seem human, or something else, we don't know]
So the sentence asserts a quality, but denies a particular reason for that quality.
A is not B in the sense that C (A is B, but not because of C).
"Not" negates the verb "is" but not in an absolute way, as the NP "in the sense of being a cat" comes to limit this negation. The negation is true only in this sense...
Your other sentence
In fact, the model is not identified in the sense that data cannot distinguish between model A and B.
means that the identity of the model does not lie in the inability of the data to act in a certain way. The model is not identified by a certain fact [that of the data being unable to distinguish between A and B], and that fact happens to be expressed by a negative clause. So the model can be identified, but not by means of that negative fact.
The sentence is correct, but intricate, and might prompt the reader to re-read it in order to process the information because of the two negations. Normally, I would use an affirmative verb, if you ask me, and use but not in the restrictive clause. It is less prone to ambiguities:
She was a lady, but not in the sense that she was not modern.
But your sentence would look awkward, without stating by what the model is actually identified:
*In fact, the model is identified, but not in the sense that data cannot distinguish between model A and B.