The infinite, in my understanding, means huge/countless. So in what sense can we call a verb huge/countless?
According to Etymonline, the derivation is:
"simple, uninflected form of a verb," 1510s, from L.L. infinitivus "unlimited, indefinite," from L. infinitus (see infinite). "Indefinite" because not having definite person or number.
So it is used more of the sense of indefinite. It doesn't mean something that approaches the infinite in scope, at least not in the mathematical sense.
Maybe it'd also help to consider the etymology of 'finite':
Why? Because http://grammar.ccc.commnet.edu/grammar/verbs.htm suggests that finite verbs can be considered as 'finished verbs' and similarly for nonfinite verbs:
Non-finite verbs (think "unfinished") ...
1 Latin grammar distinguishes between finite verb forms and infinite verb forms. There are verb forms that can only be used after a subject (noun or personal pronoun). They have personal endings for the first/second/third person, singular and plural).
2 There are other verb forms which are not connected with a subject: infinitives, participles, gerunds, gerundive, supin.
Verb forms in (1) are called finite verb forms because their use is limited to the use with a subject. "Finite" here means there is a limit to the use of these verb forms.
Verb forms in (2) don't have this limit. They are used in a lot of structures. One might also say finite verb forms have personal endings, infinite verb forms have no personal endings.
The two terms are standard terms in general grammar rather than helpful for English grammar as English has almost no personal endings. Nevertheless there is a difference between "go" in "I go" and "You can go" or "I want to go".