I'm working on a program that essentially removes all duplicates from a list, i.e. reducing the list to its unique components. Is there a single word or small compound word that conveys this transformation?

  • 2
    The word that I would say is most commonly used for this purpose in the programming community, at least, is 'uniq', from the Unix command of the same name.... – Hellion Nov 12 '11 at 1:29
  • 1
    @Hellion: I really wouldn't look toward Unix when trying to build software used by humans. ux.stackexchange.com would have a fit. – Stefan Kendall Nov 12 '11 at 1:46

Deduplication is the standard term for removing extra duplicate copies from a data set, such that remaining items appear once each.

Much of the usage of deduplication before about 1980, as shown by ngrams, corresponds to its meaning as a biological term. (E.g., 1847: "This is regarded as a deduplication of the original organ ... this is regarded as a collateral deduplication of three staminal organs, ...").

Regarding its meaning as applied to data files, I believe deduplication entered common use in connection with Z39.50 efforts during the late 1970's or early 1980's.

| improve this answer | |
  • A company I formerly worked for had an extensive deduplication process for culling repetitive documents from hard drives. The word was also frequently shortened to dedupe or deduping when working on the documents. – saritonin Nov 12 '11 at 3:40
  • As an aside, nowadays, some filesystems (e.g. ZFS) do that for you. Just think how much easier that would have been... – calum_b Nov 12 '11 at 13:02

Even if you find such a word, who are your users? Clarity is more important than brevity.

"Removing duplicates" seems to be an adequate statement for what you describe.

| improve this answer | |
  • Microsoft Excel does indeed call this feature "Remove Duplicates." See office.microsoft.com/en-us/excel-help/… – saritonin Nov 12 '11 at 3:44
  • Agreed - but my use of the word will be in a technical context, and having a shortcut term (such as "deduplicate") will make the discussion easier. – Nathan Fig Nov 13 '11 at 2:09

If you don’t mind its novel application to computer science, one possibility is exemplify. In the literal sense of providing a single example to represent a class of items, your list of distinct elements is set of such examples.

It sounds vaguely jargony, too. “First we obtain our list from memory, then exemplify it by reducing with function f.”

| improve this answer | |
  • I like this use of the word exemplify - might not use it in this case, but will bear it in mind. – Nathan Fig Nov 13 '11 at 2:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.