Yes, that is correct, from both my experience and what I can find in encyclopaedias.
For example, from Britannica's entry for Queen Elizabeth II of Great Britain:
Scenes from Queen Elizabeth II’s youth, United Kingdom, 1930s and ’40s.
Or from the entry on Charles I
Charles, accompanied by the duke of Buckingham, King James I’s favourite...
Similarly, on the Wikipedia page for Frederick II, Holy Roman Emperor:
Arguably the liveliest cultural innovation in the 13th century was Mediterranean, centered on Frederick II's polyglot court...
(The above is actually a quote from a 2012 book on history.)
Frederick II's troops paid with leather coins...
The only benefit from Innocent III's guardianship...
It also agrees with how would would the possessive of a regnal name in speech.
