Since there are multiple vice presidents, the noun would be vice presidents.
If you are saying then that that group of people, the vice presidents, own something, then yes it would be, perhaps as your hunch:
vice presidents' offices
However this is more like a literal possessive, as if you were saying:
The 5 vice presidents' offices were burned in an arson attack.
If instead you mean to say that you are filling a position/s (or an "office"), then you might consider restructuring the sentence, such as:
In filling the vacancies for the offices of our 5 vice presidents, [we reviewed many qualified applicantions...]