If I want the user to revert their operation of selecting an item, should I say: "Unselect the option" or "Deselect the option"?
Dictionaries (Merriam-Webster and New Oxford American Dictionary) have deselect but not unselect. The NOAD defines deselect as “turn off (a selected feature) on a list of options on a computer menu”, which is what you want.
Unselected, on the other hand, is used to qualify something that has not been selected, not something that was selected and isn't anymore.
Until quite recently, the "un-" prefix for verbs was pretty much limited to what Whorf called the "cryptotype" of verbs to do with fastening, wrapping, enclosing: "unlock, unwind, unwrap, unstrap, unfasten, untie, unlatch, unroll" - there were a very few exceptions like "unsay" (which is highly literary).
In recent years, particularly in IT, there are far more, and the prefix has become somewhat productive. Perhaps this is because so much is virtual in the computer world, so that many more operations can be reversed than in the physical world.
So while the dictionaries may only list "deselect", "unselect" is certainly in use, and I would have no qualms about using it.
The adjectival prefix "un-" is a different matter, and has long been very widely applicable.
To follow up on Colin Fine's answer, unselect is primarily used in the context of computer-related instructions, and I suspect that deselect is, too, these days. A third verb that performs much the same function in computer contexts (and that, like unselect, doesn't appear in the most recent Merriam-Webster's Collegiate Dictionary) is uncheck.
At the online home of the computer magazine where I work, authors use deselect and unselect to mean the same thing—"to remove from selected status"—though deselect is about five times as common (200 unique matches in a Google search for deselect versus 42 unique matches for unselect). Thus, one author writes, "Deselect the check box labeled Startup Dialog and then deselect the New Slide Dialog check box." Another writes, "If the photos have different aspect ratios, make sure you unselect Maintain Aspect Ratio at the bottom of the dialog box."
Uncheck is considerably more common on our website than either deselect or unselect (586 unique matches in a Google search), even though it has a somewhat more precise meaning: "to remove the mark from a check box in a list of options offered by a software program." Thus: "To remove an app from the list, simply uncheck the box next to that application."
The Eleventh Collegiate (2003) dates deselect to 1965, and gives it the definition "dismiss, reject." But this definition first appeared in the Tenth Collegiate (1993). In both the Eighth Collegiate (1973), where deselect debuted, and the Ninth Collegiate (1983), the only definition provided for deselect is "to dismiss (a trainee) from a training program," which I've never heard deselect used to mean.
An Ngram graph of deselect (blue line), unselect (red line), and uncheck (green line) for the years 1880 through 2008 makes clear that none of the three terms was commonly used 35 years ago:
It seems to me that viewing deselect as being fundamentally more legitimate that unselect or uncheck requires a lot of squinting. I would expect the next Webster's to include entries for all three terms, though unselect may appear only in the series of undefined words listed beneath the entry for the prefix un-.
Quotation from https://blogs.oracle.com/wxd/entry/select_deselect :
Actually, I do know that its 'Deselect All', but I only know that because somebody told me. I'm sure someone here who can quote the style and editorial guides complete with page references and footnotes off the top of their head would have been able to point out to me the grammatical and semantic reasoning behind that decision, notwithstanding the fact that unselect isn't actually a word, even though I thought it might be, because my vocabulary necessarily contains a mixture of English, US English, and web terms, which means I'm never quite sure these days when I write an email or comp a blurb that I'm making any sense at all.
Other good discussions: