If it is license rather than licence, defense rather than defence, offense rather than offence, then why not finanse?
I think it depends on the etymology of each term. During centuries of separation from Britain, American English retained the original -se ending in certain words borrowed from French, while British English modified it to -ce.
finance (n.): was originally spelt with the suffix '-ce '.
- c.1400, "an end, settlement, retribution," from Old French finance ".
- c.1300, "forbidding, prohibition," also "action of guarding or protecting," from Old French defense.
- ," (12c.), from Latin licentia "freedom, liberty, license," "
- late 14c., "hurt, harm, injury, pain," from Old French ofense
There is no reason to expect consistency in the use of the spelling -nse vs. the spelling -nce. The use of -nce vs -nse is partly, but not entirely based on etymology.
- License comes from Latin licentia, and finance ends in the suffix -ance which comes from Latin -antia (and partly from -entia). Defense and offense come from Latin defensa and offensa/offensus.
Here are the etymologically regular developments:
Latin -tia underwent patalatalization and regularly developed to French -ce.
Latin -sa(m) regularly developed to French -se.
Latin -sus/-sum regularly developed to French -s.
According to Wikipedia, after the "Late Old French (c. 1250–1300)" time period, -nce and -nse would have been pronounced identically in French. This seems to have led to non-etymological use of "c" vs. "s" in this context in Middle French.
The OED says that defense probably comes from "Anglo-Norman defens, deffens, diffence, Anglo-Norman and Middle French defence, deffence, defense, deffense". It says that for some reason, the word could be masculine in Anglo-Norman; that presumably is the reason for the Anglo-Norman forms without a final -e. You can see that variation between -nse and -nce spellings existed in the French etymon of the English word.
Similarly, the OED says offense is partly from "Anglo-Norman and Middle French offense, offence" (and their Latin etymon offensa), and partly from "Anglo-Norman and Middle French offens" (and their Latin etymon offensus).
For license, the -se spelling seems to have arisen partly from analogy in English. Some noun-verb pairs show an alternation between -ce and -se, and this probably contributed to the use of the spelling license for the English verb. And the use of the -se spelling for the verb presumably contributed to or reinforced the use of the -se spelling for the noun.
Finance can also be a verb, but for some reason, -se has not become a recognized variant spelling of this word.