4

Traditionally (and roughly) dictionaries are written by teams of editors who gather large corpora, identify new words or new word usages, examine how those words are used, and then assign parts of speech and write definitions based on their examinations.

With no ill will meant toward dictionary editors, this seems a task well suited for computers. (As an interesting aside, see this anti-capitalist critique of automation.) Google and Apple, especially, have troves of written material, both old and current. It seems to me (and I confess I have no computer background) this could easily be used to largely automate dictionary writing.

Computers can identify which words are used, in books, emails, texts, etc, which are yet undefined. By comparison with other words used in similar contexts (its location in sentence structure, the meanings and parts of speech of other words which appear in the same place, etc.) the computer could create a list of likely synonyms and assign a part of speech. A human editor would probably then still need to write the definition, but the bulk of the work is done.

Are dictionaries doing this? In some ways this is simply glorified text-prediction. I have looked online but found nothing. The problem with searching for information regarding dictionaries is that every result is a definition.

  • Are you referring to the following for instance, from Collins Dictionary: Word usage trends for “meaning” : collinsdictionary.com/dictionary/english/meaning - or this: Popularity: Top 10% of words from M-W : merriam-webster.com/dictionary/meaning – user66974 Jun 26 '16 at 20:00
  • @Josh61 Ooh I didn't know that I was, but I very well have been. Looks interesting – Unrelated Jun 26 '16 at 20:04
  • 1
    Don't forget Amazon, Facebook and LinkedIn (Microsoft)... – Drew Jun 26 '16 at 20:51
  • 1
    New usages that haven't yet made it into dictionaries might never do so, but you can always find them on Urban Dictionary or the like. Dictionary editors probably tend to hang back because they know that in many cases, the neologism won't survive anyway, rather than because they don't have the right computer tools to help them keep up with emerging usage trends. It's not like English needs to maximise the number of new words being introduced. To some extent, it's quite the opposite. – FumbleFingers Jun 26 '16 at 21:05
  • @FumbleFingers Here is an interesting article about a 'superdictionary' and the argument that every word should be defined nytimes.com/2014/01/22/books/… – Unrelated Jun 26 '16 at 21:10
3

The short answer is yes, dictionaries do use corpora and electronic searches on those corpora to identify words and phrases, as well as their grammatical categories and semantic relationships.

Here is a quote from Macmillan:

"Using intelligent software... we can find every example in the corpus of a particular word, phrase, grammatical pattern, or collocation. It is this information which forms the basis for everything we say about words in the dictionary."

Macmillan goes on to describe how their software not only finds every occurrence of a word and it's variations, it also outputs one page summaries of the important grammatical and semantic relationships of the word. They give an example:

"The program first collects all the examples of the word being investigated.... Then it applies a second stage of analysis. This time, the software looks at particular grammatical relationships. In the case of evidence, it finds all the sentences where evidence is the object of a verb, then identifies the most frequent verbs used in this pattern.... [P]eople often talk (or write) about giving evidence, finding evidence, presenting evidence, or gathering evidence. Similarly, the... [software outputs] a list of the adjectives that most frequently modify this noun: we may say there is little evidence for something, or talk about clear evidence, strong evidence, or scientific evidence."

They also use the corpora to generate the labels for certain words, for example archaic, informal, American, journalism, etc. They can output these labels based on which documents in the corpora (for example, old texts or audio recordings) the words primarily appear in. Here is their example:

"When we look at all the examples of eatery in the corpus we find that a majority come from newspapers and magazines, and most of these newspapers and magazines are from the U.S. So in the dictionary, the word eatery has two ‘labels’: mainly american and mainly journalism."

One thing Macmillan does not mention, however, is finding synonyms for novel words. They suggests that the primary data for new words is citations. The examples they give are (1) using green as a transitive verb to mean "to make something more environmentally friendly" and (2) using handbags as an adjective. But still, if these uses occur in the corpus, they are obviously subject to the same kind of analysis as other words.

These are just example quotes from Macmillan, but certainly other big dictionaries like Oxford are doing the same thing. If you search around, you are likely to find information about their processes.

  • Publish or perish still holds. – Stan Jun 30 '16 at 22:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.