2

Suppose that there exists a dataset consisting 1.5m scientific papers. I have done a lot of processing on the table to mitigate the noises in it, handling null values, etc. My extensive work resulted in a much cleaner dataset (of 650k papers). What word can I use:

We ... a dataset consisting 650k papers.

Possible candidates:

  • made
  • compiled
  • built
Improve this question
5
  • 1
    If the result of your work is a cleaner data set, then you cleaned up the existing data set and produced a new data set consisting of 650,000 papers.
    – michael.hor257k
    Sep 18, 2016 at 5:48
  • 1
    If you had created a dataset from scratch, then your words seem OK; I would add constructed to the list. In the case you describe, I would say that you refined the existing dataset.
    – Scott - Слава Україні
    Sep 18, 2016 at 6:58
  • The point is that the cleaning is an important part and the result is valuable. In other words, the dataset can itself be published as a new dataset with a meaningful added value. So, e.g., using only cleaned up is somehow weak in this context.
    – Shayan
    Sep 18, 2016 at 9:32
  • What's wrong with processed ?
    – David Handelman
    Sep 18, 2016 at 16:12
  • I sense that processing is more related to a process not the product. In other words, it does not emphasize that the product is a special and useful result. For example, I sense that a word like compile (without considering whether natives use that in this context), means we built something and is focused on the result not the process: e.g., in "We compiled a dataset by performing extensive data cleaning on dataset X"
    – Shayan
    Sep 18, 2016 at 18:23

6 Answers 6

Reset to default
4

Another possibility is curate. "We curated a dataset consisting of 650k papers."

Select, organize, and present (online content, merchandise, information, etc.), typically using professional or expert knowledge.

An example from "Genomics Needs A Killer App":

Traditionally, much of this information has been distributed through academic publications. Many companies curate papers to extract valuable information for clinical genomic and R&D applications: Ingenuity, Biobase, Thomson Reuters, and others.

Improve this answer
2

I like @michael.hor257k's suggestion, but you could also use munged (sounds like monger in fishmonger). From Wikipedia:

Mung or munge is computer jargon for a series of potentially destructive or irrevocable changes to a piece of data or a file. It is sometimes used for vague data transformation steps that are not yet clear to the speaker. Common munging operations include removing punctuation or html tags, data parsing, filtering, and transformation. … Munging can also describe the processing or filtering of raw data into another form.

I often say I've munged some data, or cleaned it up. There are also several books on how to data mung, so it's a well known term (among people who mung:) for this sort of thing.

Improve this answer
1
  • 1
    Huh, never saw that one before—very useful word!
    – Janus Bahs Jacquet
    Sep 18, 2016 at 9:02
1

There are domain-specific technical terms (verbs) to denote the actions mentioned by the OP like cleansing, scrubbing, wrangling and munging (this one is already mentioned in another answer) but I am not sure if they fit in the example sentence as it is. Perhaps it can be adapted like:

We cleansed/scrubbed/wrangled/munged the dataset of 1.5m scientific papers into a smaller one of 650k papers.

Also, distilled connotes reduction (1.5m to 650k) and improvement (processing on the table to mitigate the noises in it, handling null values, etc):

We distilled a dataset consisting of 650k papers.

M-W:

distill verb

: to take the most important parts of something and put them in a different and usually improved form

He has perfectly distilled the meaning of the holiday into a poem.

A widely accepted and understood term would be prepared (as in data preparation).

We prepared a dataset consisting of 650k papers.

M-W:

prepare verb

: to make (someone or something) ready for some activity, purpose, use, etc.

: to make or create (something) so that it is ready for use

The pharmacist prepared the prescription.

Another generic(nontechnical) term would be extracted.

We extracted a dataset consisting of 650k papers.

M-W:

extract verb

: to get (something, such as information) from something

Investigators were able to extractuseful information from the company's financial records.

They are hoping to extract new insights from the test results

.

Improve this answer
0

"You condensed the data set."

Condense — Cambridge

verb To reduce something, such as a speech or piece of writing, in length
"I condensed ten pages of comments into/to two."

Improve this answer
0

If you are speaking of choosing 650K papers from 1.5M papers, then you have culled the papers:

cull, transitive verb

to select from a group : choose culled the best passages from the poet's work

to reduce or control the size of (as a herd) by removal (as by hunting) of especially weaker animals; also : to hunt or kill (animals) as a means of population control

I'm a little unclear as to whether you are controlling the papers (which themselves comprise your data) or the data contained within the papers. I might use cull for controlling papers, or cleanse the data within the papers. For example:

We culled the papers that were not peer reviewed or that had a p-value of greater than .01.

We cleansed the data of the papers that did not take patient's age or socioeconomic status into account.

Improve this answer
-1

We assembled a dataset consisting 650k papers.

Improve this answer
2
  • Assemble sounds more like building something up, rather than condensing it down to the essential bits. Your answer could use a reference as well, maybe I am mistaken.
    – Helmar
    Sep 19, 2016 at 6:50
  • 1
    @Helmar to me, assemble is similar to "making a collection". If I assemble all my ingredients on the kitchen table before starting to cook, I'll have a calmer process than if I scramble to look for them as I go. (You are right, I should have provided a reference.)
    – aparente001
    Sep 19, 2016 at 6:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.