When employing ordinal numbers you are explaining a relationship between items in a set. The term you use to describe those numbers will largely depend upon the items in the set.
For example: Primary care vs. Tertiary care. You would describe these levels as tiers. Hence, if there were quaternary care, etc. you would describe this as higher-tier.
If you are speaking of classes: First class, second class, etc. You would call these higher classed (and of course these would typically go in reverse order first being highest).
If you are speaking of orders: Tertiary, Quaternary, etc. You would call these higher-ordered.
And, in any case, if you wish to describe a portion of a set, you would typically define your terms: e.g. Higher-ordered thingamajigs (Quaternary and beyond).
Of course, given that order is within the base definition and etymology of ordinal, I would say high(er)-order is always superficially correct.