Adverb versus Preposition
It is definitely not a preposition. A rule of thumb to tell and adverb from a preposition, is that the preposition requires an object:
Preposition:
I walked down the hill.
Adverb:
I walked down.
Here is a case where above would be a preposition:
If you have questions, you can look at my notes above this sentence.
If you omit the object ("this sentence"), then that's an adverb.
Note that these are basically the same words. They are used in two different ways.
Preposition versus Adjective
As for thinking that it's an adjective, that's another matter. A case where you can validly argue for the adjective against the adverb is this:
If you have questions, you can look at my above notes.
In English, the attributive adjective generally goes before the name, so it makes sense. The conventional answer is therefore that it is an adjective (though one could also argue for an adverb, or rather that it is an adverb that came to be used an adjective). Note, however, that it this adjective use is rather formal language.
But if you put the word "above" after the noun, the perception is definitely that of an adverb -- an answer to the question "where".
Again, if one really wanted to look at "above" as a postpositive adjective, perhaps one could build a case.
Arbitration
There is another consideration: every child could understand (albeit with some effort) that "above" can be either a preposition or an adverb. Throw in the fact that it could also be an adjective, and that might be too much!
So (except for the rarer case when "above" is before the noun) why not settle for the simple answer and say it's an adverb when it's not a preposition? In my humble opinion, grammar has to be as simple as possible, so as to be taught to the largest number of people. Between two ways of explaining something to others, one should go for the simplest and most intuitive.
I am not saying anything special here, since that seems to be the conventional position of dictionaries. Indeed, MacMillan is explicit about these three categories (preposition, adverb and adjective), specifically for the case at hand.