With modifier: We were going on an air-to-air photo mission.
Without: We were going on our first air to air.
We were going on our first air-to-air.
It should stay hyphenated, I think. It's still a phrasal adjective, modifying a noun that is now implied; it just turns into a nominalized adjective when the noun is removed.
Note that it's the adjectives that modify the noun here, not vice versa. Unless I misunderstood what you mean by modifier.