I am looking to identify list of words that I can add to my search algorithm for a database.

I would like to know if there is a list of words that I can identify that serve little purpose. I would like to search a database for exact matches without the use of certain words. For example, the, an, a, to, and. I am not sure what you call these type of words.

  • 2
    Welcome to EL&U. I think most of the contributors here would take issue with the notion that words like the or and "serve little purpose"; an evening with the Queen would probably be rather different from one with Queen — or with a queen for that matter. Are you simply referring to very common words, for example, to ignore when indexing text? – choster Dec 2 '14 at 17:24
  • 3
    The list of words to be excluded from a text search is typically called a "stopword list". As an example, see the MySQL Full Text Search Stopword List http://dev.mysql.com/doc/refman/5.5/en/fulltext-stopwords.html. (it's not that these words "serve little purpose", they do serve an important purpose. The problem with these words is that matches based on these words often yield results which tend not to be relevant the actual topic being searched.) – spencer7593 Dec 2 '14 at 17:36
  • 1
    See also en.wikipedia.org/wiki/Stop_words – John Lawler Dec 2 '14 at 18:29

According to the ones you shared in your example:

  • the, an,a = articles.
  • and = additive coordinating conjunction.
  • to = preposition.

I guess you could use just "Common Words" or "Stop Words", that's usually among the technical jargons that are stated while reading about indexing operations in Search Engines.

e.g., http://lucene.apache.org/core/3_0_3/api/core/org/apache/lucene/analysis/package-summary.html#Tokenization


| improve this answer | |
  • (1) ELU being aimed at proficient linguists, giving the POS's of 'and' etc is over-simplistic. (2) There doesn't seem to be any user-friendly supporting evidence. – Edwin Ashworth Aug 19 '19 at 18:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.