I need to understand, is 'drool' also used to express how you are in awe of something. Like, is this sentence right?
I am all drools for this library. It has a great collection of books.
Correct me if I am wrong.
It would probably be understandable—but the noun drool isn't normally used that way. And anybody hearing it would find it strange.
Also, while I'd like to say that adrool is a word, I can't. Because it isn't.
However, something that is a word, and which fits your meaning, is agog:
: full of intense interest or excitement : EAGER
// kids all agog over new toys
So, in your sentence, the closest expression that uses an entirely normal phrase is:
I'm all agog over this library. It has a great collection of books.