This is a matter of style, and there is no single correct answer.
What you are dealing with is hyphenation that includes an open compound.
According to The Chicago Manual of Style (17th ed.), 6.80:
The en dash can be used in place of a hyphen in a compound adjective when one of its elements consists of an open compound or when both elements consist of hyphenated compounds. Whereas a hyphen joins exactly two words, the en dash is intended to signal a link across more than two. Because this editorial nicety will almost certainly go unnoticed by the majority of readers, it should be used sparingly, when a more elegant solution is unavailable. As the first two examples illustrate, the distinction is most helpful with proper compounds, whose limits are made clear within the larger context by capitalization. The relationship in the third example depends to some small degree on an en dash that many readers will perceive as a hyphen connecting music and influenced. The relationships in the fourth example are less awkwardly conveyed with a comma.
the post–World War II years
Chuck Berry–style lyrics
country music–influenced lyrics (or lyrics influenced by country music)
a quasi-public–quasi-judicial body (or, better, a quasi-public, quasi-judicial body)
Note that you are not using any capitalized words, so the guidance here is less applicable that it might be otherwise. Even Chicago says that many people might not notice the subtlety of using an en dash over a hyphen.
However, compare these two:
use case-specific
use case–specific
Does the en dash make a noticeable enough impact that it no longer looks so strange?
Of course, even though use case is an open noun, you could choose to use a hyphen in order to indicate where the division of the three words lies:
use-case specific
That, too, clearly indicates the meaning being expressed.
It also matches how some people hyphenate ice-cream sandwich, despite the fact that ice cream is normally an open noun.
If the three words are being used to adjectivally modify something else, then you could use three hyphens in a row:
use-case-specific software
Since people are used to seeing use case, this triple hyphenation shouldn't cause any problem in terms of comprehension.
Other alternatives to hyphenation involve using quotations marks or italics (or other forms of representational styling) instead of hyphens:
"use case" specific
use case specific
However, since there is nothing special about the use of this particular phrase, it might look even stranger when set off this way than when using some form of hyphenation.
Compare those alternatives to hyphens used in a different context:
It's one of those "don't bother dressing up" invitations.
It's one of those don't bother dressing up invitations.
Unlike use case, this phrase seems more normal when set apart from the surrounding text.
If it's only the three words being used, and they don't together modify something else, you could choose to omit hyphenation altogether, if you think there wouldn't be any risk of misunderstanding and it looked better to you:
use case specific
This matches how some other people don't hyphenate ice cream sandwich.
Of course, you could simply rephrase:
The software is use case specific.
→ The software is specific to each use case.
How you choose to style this is up to you. Since there is no universal rule, barring other guidance, go with the method that looks best with the rest of the text in your document.