I know that a gerund is a noun, so it should be modified by an adjective. However, it is also a verb form. Can I modify it by using an adverb?
If you modify a gerund "from the outside", you treat it as a noun, and so you use an adjective:
That's quick thinking! (= that is a quick act of thinking)
I heard a faint rustling of feathers or clothes.
When emphasising the nominal aspect of a gerund as above, this normally corresponds with expressing the agent of the action with of. When you use a or an, this forces you to emphasise the nominal aspect.
But you can also modify a gerund from within the gerundial construction, where it functions as a verb, so you use an adverb:
She left by quickly crossing the street and hailing a cab.
I don't like speaking softly when there is no need.
Emphasising the verbal aspect of a gerund as above usually corresponds with not expressing the agent at all within the gerundial construction (but rather outside of it, e.g as the subject of the main clause, or the object, or whatever).