Look has two related but different senses.
On the one hand it aligns with verbs like gaze and stare, intransitive verbs denoting the action of an animate subject.
Jack stared nervously at the girl.
Jack looked nervously at the girl.
In these cases nervous is an Adjunct which modifies the verb stare and thus must be cast in adverbial form. If you to transform the predicate into an adjunct modifying Jack it retains this form:
Looking nervously at the girl, Jack shuffled his feet.
On the other hand, look aligns with verbs like be and appear, copulas which attribute a particular character to the subject.
The girl was nervous.
The girl looked nervous.
In these cases nervous is an Argument of the sentence; it is attributed to the subject, the girl, and is therefore cast in adjectival form. When you transform this predicate you employ a different structure but retain the adjectival form:
The nervous-looking girl licked her lips.
Other verbs of sensation have similar but not always identical properties. Taste for instance is a transitive verb in its agentive use:
Amy tasted the candy. — The candy tasted sweet. — sweet-tasting candy
With hear, the distinction between agentive and copular is expressed lexically, with a different word — and that word has its own copular and agentive senses:
Bob heard the bell. — The bell sounded loud. — the loud-sounding bell ... BUT ALSO
Frank heaved on the bellrope and the bell sounded loudly, waking the entire quarter. — the loudly sounding bell.
So the rule governing adjective vs adverb depends on how the verb is used; but how any given verb may be used is a matter of historical contingency.