I'm working on a program that essentially removes all duplicates from a list, i.e. reducing the list to its unique components. Is there a single word or small compound word that conveys this transformation?
2
-
2The word that I would say is most commonly used for this purpose in the programming community, at least, is 'uniq', from the Unix command of the same name....– HellionCommented Nov 12, 2011 at 1:29
-
1@Hellion: I really wouldn't look toward Unix when trying to build software used by humans. ux.stackexchange.com would have a fit.– Stefan KendallCommented Nov 12, 2011 at 1:46
Add a comment
|
3 Answers
Deduplication is the standard term for removing extra duplicate copies from a data set, such that remaining items appear once each.
Much of the usage of deduplication before about 1980, as shown by ngrams, corresponds to its meaning as a biological term. (E.g., 1847: "This is regarded as a deduplication of the original organ ... this is regarded as a collateral deduplication of three staminal organs, ...").
Regarding its meaning as applied to data files, I believe deduplication entered common use in connection with Z39.50 efforts during the late 1970's or early 1980's.
Verb uniquify, "To eliminate duplicates from a list, especially when using the *nix command/function uniq" (Wiktionary, sense 1) and noun uniquification also might be used. I favor the alternate spelling uniqify vs uniquify because I see uniqify used more frequently in Python and Julia programming contexts. Neither form appears in ngrams, however.
answered Nov 12, 2011 at 3:35
-
A company I formerly worked for had an extensive deduplication process for culling repetitive documents from hard drives. The word was also frequently shortened to dedupe or deduping when working on the documents. Commented Nov 12, 2011 at 3:40
-
As an aside, nowadays, some filesystems (e.g. ZFS) do that for you. Just think how much easier that would have been...– calum_bCommented Nov 12, 2011 at 13:02
Even if you find such a word, who are your users? Clarity is more important than brevity.
"Removing duplicates" seems to be an adequate statement for what you describe.
-
Microsoft Excel does indeed call this feature "Remove Duplicates." See office.microsoft.com/en-us/excel-help/… Commented Nov 12, 2011 at 3:44
-
Agreed - but my use of the word will be in a technical context, and having a shortcut term (such as "deduplicate") will make the discussion easier. Commented Nov 13, 2011 at 2:09
If you don’t mind its novel application to computer science, one possibility is exemplify. In the literal sense of providing a single example to represent a class of items, your list of distinct elements is set of such examples.
It sounds vaguely jargony, too. “First we obtain our list from memory, then exemplify it by reducing with function f.”
-
I like this use of the word exemplify - might not use it in this case, but will bear it in mind. Commented Nov 13, 2011 at 2:09