Why are some verbs only usable in the infinitive? The one example I can think of is "to spite" (see "to wit in the accepted answer). While wiktionary claims that spited is a word, that doesn't match other ite words, like bite or smite. OED disagrees, saying there is no past tense.
There is at least one verb in modern English that is only used in the infinitive; namely to wit. What happened there was that wit used to be a normal verb (conjugated I wot, I wist, I have wist), but all uses except the infinitive to wit dropped out of use.
The same thing seems to be happening to the verb spite. For the verb spite, all uses except the infinitive are quite rare in modern English, and it is conceivable they will vanish completely relatively soon. It appears that in Shakespeare's time, the use of spite was not restricted to the infinitive:
And that which spites me more than all these wants,
He does it under name of perfect love; — Taming of the Shrew, Shakespeare.
Searching Shakespeare, it looks like he uses spite in the infinitive 4 times, and in other conjugations 3 times.
My OED (2nd ed, 1989) does not say spite as a verb can only be used in the infinitive, nor does it say it has no past tense. When the past tense isn't specifically listed, it is regular. Many example quotes give spited as the past tense and part participle.
There is one sense of the verb spite that's limited to the phrase to spite (one), i.e. in the infinitive. Here, it has a specific meaning; it is transitive, taking a person as object. However, there are other senses too. The infinitive isn't the only possible usage.
Learner's dictionaries like OALD and LDOCE tend to have limited senses or usages for certain words, showing learners only the most common usages, just like they only list a subset of the vocabulary as headwords and limit the definition words to 2000.
Unfortunately, rhyming words are not a reliable test. See Wikipedia: English irregular verbs
And, for that matter, if its bite/bit and smite/smote, then what exactly does that say about "spite" anyway? That we should pick a random vowel? :)