It's accident, specific to each individual use of the country name in English. Let's take Austria and Hungary as an example. You might want there to be a deep phonological or etymological reason that the two have different endings. No. It appears to be accident.
Austria
In German, it's Österreich, and other Germanic languages use names ending in -reich. English instead borrowed from a Latinization of Österreich, Austria. There is an accidental shift in this Latinization, as Ost means east in German but Auster means south in Latin. The Latin usage is first recorded in 1147 (Wikipedia). The word enters English by the 16th century, apparently with minimal interference from an intermediary language like French.
William Caxton, in his French to English dictionary of 1480, notes the older Germanic use alongside a Francophone version ending in -e:
De duc daustrice // Of the duke of ostryche.
However, the French usage never comes over to English. Instead, by 1537 (in the Sex Linguarum of Latin, French, Spanish, Italian, English, and German) and 1538 (in Sir Thomas Elyot's dictionary) the English usage is Austria:
austria oostenryck austriche austria austria easterick
Vienna, a cite in Austria, an other in Dolpheni
Hungary
Hungaria is one medieval Latinization, and others include Ungri, Ungari, Avari, and so on (Wikipedia). Hungary is already in use in the late Middle English period, as demonstrated in the 15th century Chronicle of John Harding:
Kyng Edwarde sente then into Hungary
For his cousyn, the sonne of Emond Ironesyde.
Similarly, Thomas Elyot (1538) calls the country Hungary:
Pannonia, the royalme of Hungary.
Since the administrative language in the late Middle English period was still largely French, that suggests a possible French influence for the country name. Note, however, that Hungaria is also a form that exists in Middle and Early Modern English. Here's Ranulf Hidgen in the 14th century:
Out of þe more Pannonia Hunni went an huntynge, and passed long by marys and wateres, and folwed þe trace of hertes, ut dicit Herodotus, and so at þe laste þei founde þe lasse Pannonia, and torned home aȝen, and fette to hem grete strengþe and com eft in to þe lasse Pannonia, and put out þe men þat were þerynne, and cleped þe lond Hungaria.
And here's Thomas Elyot (now in 1542) using both Hungaria and Hungry:
Pannonia, the countreye nowe called Hungaria whiche toke that name of an other Hungaria nowe named Iulira, a countreye in the northe parte of the world not ferre from the riuer called Tanais, and is trybutarye to the Moscouites. The boundes of Hungry is now moche larger than it was of olde tyme
The usage of Hungary didn't fully standardize until the 18th century; Samuel Johnson refers exclusively to "Hungary" in his dictionary of 1755 (Lexicons of Early Modern English search). It's unclear why Hungary won out over Hungaria, when authors once used both regularly.
So it's messy
Even from those two country's narratives, it's hard to know why Hungary took prominence (despite the presence of Hungaria as a form), or why Austria took hold instead of something like Ostrich. I can trace through the decades which names are used, and make some guesses for these specific words, but they defy generalization to other country names.
For instance, I can say that Saxony and Saxonia both appear in Middle English, but Bavaria was Bauer or Baueres in Middle English before, somehow, the Latin spelling Bavaria took hold (Middle English Corpus). I can't say why Saxony took precedence, let alone whether Hungary and Saxony could be grouped together as following one deterministic rule while Bavaria and Austria together follow another deterministic rule. It feels more arbitrary than that.
