The final consonant of the verb is relevant, indirectly, because of the etymology of the two endings. However, there is no simple rule for the spellings, because there are some complications due to both etymology and analogy.

## Latin agent nouns end in *-tor* or *-sor*

As RegDwigнt's answer mentions, verbs that form agent nouns in *-or* usually have the form of Latin perfect passive participle stems. 

Latin perfect passive participle stems characteristically end in **t** or **s**. So in practice, verbs that form agent nouns in *-or* almost always  end in *-t(e)* or *-s(e),* and form agent nouns in *-tor* or *-sor* specifically. In the context of Latin, the suffix is often given as *-tor* rather than as *-or.*

Examples:

- *actor* (Latin and English), from the Latin verb *ago* ("do, act"), perfect passive participle stem *act-*
* *possessor* (Latin and English), from the Latin verb *possideo* ("have, hold, possess"), perfect passive participle stem *possess-*

You can see that these two words have the same spelling in modern English as in Latin. For complicated reasons, the corresponding English verbs have the form of the Latin perfect passive particle stems: *act* and *possess.*

## *-or* preceded by any other consonant comes from French

There are agent nouns in English that end in *-or* preceded by a letter other than *t* or *s*. The main origin of this type of *-or* agent noun is French. In Old French, the *t* in the Latin ending *-atorem* (as well as some other similar endings) was lost by regular sound change, and the syllables merged, causing it to become *-or/-our/-ur* (Modern French *-eur*). This was taken into Middle English as an agent-noun suffix *-our*. 

In Modern English, this ending came to be unstressed and its vowel became reduced, making it sound identical in English pronunciation to the *-or* ending found in Latin words, and its spelling has therefore been simplified in most cases to *-or.*

- A large proportion of *-or* agent nouns only occur in legal English, where *-or* is used more than in regular English (often as a counterpart to *-ee*). For example, "deliveror" and "settlor" are basically only found in legal contexts; in other contexts, the spellings "deliverer" and "settler" are used.

- Some *-or* agent nouns that are not restricted to legal English are *conquer-or, purvey-or, survey-or, counsel-(l)-or,* and *vend-or* (the last has a less common but accepted variant *vend-er*).

- *sail-or* is an example of the French suffix being used on an English base, the verb *sail.*

There are other *-or* words that don't end in *-tor* or *-sor,* but most of the rest aren't built directly on an English verb as a base. For example, *donor, emperor, tailor, juror* don't refer to people who "done", "emper", "tail" or "jure".

There seems to be sporadic use of the form *-or* instead of *-er* simply as a means of forming words with a specialized meaning, sometimes as part of technical terminology or jargon. 

- The Oxford English Dictionary entry for the word "sailor" describes it as "An altered spelling of sailer n., probably assimilated to tailor, in order to distinguish the designation of a regular calling from the unspecialized agent-noun. The differentiation, however, does not appear in our early examples, and was not fully established before the 19th cent."

- In electronics, the form "[expandor](https://www.thefreedictionary.com/expandor)", as well as a derived term "compandor" (from *compressor* and *expandor*), has apparently been used.


## It's hard to rule out *-er* as a potential spelling for an agent noun

Ending in *-t(e)* or *-s(e)* is (mostly) necessary for a verb to form an agent noun in *-or,* but it certainly isn't sufficient. [Sven Yargs left a comment mentioning "disrupter"](https://english.stackexchange.com/questions/4733/what-s-the-rule-for-adding-er-vs-or-when-forming-an-agent-noun-from-a-ver/486896#comment803421_137523) as an example of an agent noun that is commonly spelled with *-ter.*

The suffix *-er* is highly productive and is used to form nouns with a range of meanings. I don't think any rule based on meaning will be reliable as a guide to using *-er* vs. *-or.* To give counterexamples to some alleged rules that I've sometimes seen:

- Words ending in *-er* can be animate (*runner, worker, speaker, reader*) or inanimate (*washer, dryer, circuit breaker, holder*). Words ending in *-or* also can be animate (*administrator, director*) or inanimate (*elevator, insulator, ventilator*).

- Words ending in *-er* can be names of professions (*teacher, writer, baker, publisher, typographer, undertaker*), as can words ending in *-or* (*sculptor, legislator, conductor*).