The phrases "in brief" and "in short" function as adverbs, but as what part of speech do "brief" and "short" function in these phrases? "Brief" stands alone as both a noun and adjective and could be seen as either in the phrase (i.e. preposition "in" + noun/adjective "brief"), but "short" in the meaning used by the phrase only occurs as an adjective. Are these two phrases exceptional in that an adjective is the object of the preposition (discounting constructions such as, "Do you have this in blue?")? Are they simply "fixed" phrases that can't be broken down into their constituents?
Well, I think grammarians are going to have various opinions, but the Oxford English Dictionary thinks these are both nouns. Or, more precisely, they are adjectives used as "absolute" constructions which omit the noun they reference implicitly, somewhat similarly to a process where we use an adjective as a substantive, like referring to the weak when we mean the weak [people].
There are all sorts of uses in English (and other languages) where we just use AdjectiveX to mean AdjectiveX [things, people, etc.], and when such uses become idiomatic, the adjective is effectively converted into a noun. Combining such an adjective with a preposition isn't an exception in English either, though these idioms are a little unusual in that such absolute constructions often have the before the adjective to mark it as a noun.
Anyhow, here are some excerpts from the definitions and related senses in the OED.
For in brief:
a. in brief: in few words, shortly, concisely. With ellipsis of ‘to speak’: In short, to sum up.
†b. the brief, used absol. like the short. Obs.
The (a) usage has an earliest citation of 1423, which is roughly the same time when "brief" first came to be used as an adjective in English. (Notably, the use of "brief" as a noun in the sense of a formal letter, dispatch, or note has citations at least a century earlier; whether there's a connection between this idiom and the slightly older noun is a question I leave to the professional etymologists. The OED seems to think "no.")
For in short:
I. The neuter adj. used absol.
With prepositions, forming adv. phrases.
a. in short (also Sc. †at short): briefly, concisely. From the 18th c. onwards used only as parenthetical phrase, introducing or accompanying a summary statement of what has been previously said. †in short and plain: briefly and plainly.
†b. in short (? also Sc. at short): in a short time, quickly. Obs.
c. for short: as an abbreviation.
the short: the total, the result, upshot; a brief summing up of something which has been previously explained in full. Now only dial. (Cf. the long and the short of (it, etc.) ...) †short is: ‘to speak briefly’, ‘the short of the matter is’.
Phr. to draw short and long : to draw lots by means of straws, etc. of different lengths.
The phrase in short and plain dates at least back to Chaucer, but the OED has the first citations of at short and in short in the first sense in the 1500s.
I believe that these are abbreviated phrases, e.g. in brief is short for something like in brief terms. So brief is an adjective, but the word it modifies has been elided.