The adjective wary and adverb warily each have gingerly as a synonym, instead of the adjectival form being ginger. If it were ginger, I could also imagine the adverb being the flat ginger. I hesitate before calling the adverb gingerly flat because, while it is indeed equivalent to its associated adjective rather than gaining a suffix, the "non-flat" suffix is already built-in. Is there a term for adjectives such as gingerly that already look like adverbs? Possibly not, because it seems rare.
I'm also baffled by how it happened, because the noun ginger gives us an adjective ginger with a meaning unlike any discussed above, and the -ly prefix is largely an invention of recent centuries to make adverbs stand out from any associated adjectives, rather than something that adjectives tended to use to distinguish them from associated nouns.