###Latinate agent nouns end in *-tor* (or *-sor*)

As RegDwigнt's answer mentions, verbs that form agent nouns in *-or* usually have the form of Latin perfect passive participle stems. 

Latin perfect passive participle stems characteristically end in **t** or **s**. So in practice, verbs that form agent nouns in *-or* almost always  end in *-t(e)* or *-s(e),* and form agent nouns in *-tor* or *-sor* specifically. The Latin suffix is often given as *-tor* specifically.

###Other *-or* words tend to be from French

There are agent nouns in English that end in *-or* preceded by a letter other than *t* or *s*. The main origin of this type of *-or* agent noun is French: in Old French, the Latin ending *-atorem* (as well as some other similar endings) was phonetically reduced to *-or/-our/-ur* (Modern French *-eur*), which was taken into English as an agent-noun suffix *-or.*

- A large proportion of the exceptions only occur in legal English, where *-or* is used more than in regular English (often as a counterpart to *-ee*). For example, "deliveror" and "settlor" are basically only found in legal contexts; in other contexts, the spellings "deliverer" and "settler" are used.

- Exceptions that are not restricted to legal English include *conquer-or, purvey-or, survey-or, counsel-(l)-or,* and *vend-or* (the last has a less common but accepted variant *vend-er*).

- *sail-or* is an example of the French suffix being used on an English base, the verb *sail.*

There are other *-or* words that don't end in *-tor* or *-sor,* but most of the rest aren't built directly on an English verb as a base. For example, *donor, emperor, tailor, juror* don't refer to people who "done", "emper", "tail" or "jure".

###It's hard to rule out the *-er* suffix as a potential candidate

Ending in *-t(e)* or *-s(e)* is (mostly) necessary for a verb to form an agent noun in *-or,* but it certainly isn't sufficient. [Sven Yargs left a comment mentioning "disrupter"](https://english.stackexchange.com/questions/4733/what-s-the-rule-for-adding-er-vs-or-when-forming-an-agent-noun-from-a-ver/486896#comment803421_137523) as an example of an agent noun that is commonly spelled with *-ter.*