Since we're not primarily looking at constructs such as governor general where there are no names mentioned, let's ignore these.
For the form (Title) (Name), taking personal preferences seems to lead to a rather polarising discussion. Let's instead look first at accepted conventions with other titles, specifically Mr and the null title.
Before looking at two individuals called Mr X, consider two people, Mr X and Mr Y.
The plural of Mr is Messrs according to the Merriam Webster online dictionary. See also the more extensive article on the same subject in their Learner's Dictionary website. In both places, the form used to refer collectively to Mr X and Mr Y is Messrs X and Y. (The female version of Messrs is Mesdames, with more discussion here.)
This might suggest that the form to use is Presidents Roosevelt, but we need to look more carefully. Where two people are both called Mr Smith, for example, they are both named using the form Messrs Smith and Smith. Here's an example (emphasis mine; emphases in original dropped):
In his monthly report to the Church Envoy in February 1915, Rev JL Mortimer commented that the window was executed by Messrs Smith and Smith, of Dunedin.
- Holy Cross, St Kilda
This is therefore poor support for using the two Presidents Roosevelt form, as supported by the ngram analysis below.
Looking now at what we might call the null title, we find that if we have two people, both called Smith, it is customary to call them the Smiths. See also the post Family name pluralisation. Extrapolating this to your question's context produces the form two President Roosevelts.
Nevertheless, these are both somewhat awkward extrapolations, so let's check them against actual usage, using the 41st and 43rd US Presidents, each referred to as President Bush.
Looking at these forms in ngram, we find that Presidents Bush is significantly more popular. When we dig deeper, though, we find that many references are to phrases like "Presidents Bush and Obama", which isn't relevant here. There is only one relevant instance in the first two pages of listings, and that is a piece of fiction.
The relevant references using the form President Bushes are more numerous and in more serious writing. Here's an example (emphasis mine):
In the two most recent Gulf War conflicts, the weight of popular opinion led both President Bushes to seek congressional authorization for the invasions of Iraq in 1991 and 2003.
- Kenneth Dautrich and David A Yalof, American Government
You ask:
If you have two people with the same name, affixed with the same honorific, do you pluralize the honorific or the name?
Based on the above, the standard convention is to pluralise the name.