1

How many words would be required for a comprehensive English learners dictionary and what level of effort would be required to create such a dictionary from scratch?

"Comprehensive" means it has enough of the most commonly used words to support someone who uses it until they reach native-level fluency.

"Level of effort" means the approximate person hours for creation of the dictionary content (headwords, pronunciations, meanings) but excluding anything technical like creation of the database/system used to do the work.

"From scratch" means, it could be done in a way that the resulting dictionary would have no legal obligations to any other party, i.e. definitions get created from the creators' own knowledge and research and not from other copyrighted dictionaries.

For example, one might answer "12,000 words and 10,000 person hours."

EDIT 3/2/2021: The original question, left as background above, was closed as "opinion-based" and because questions must be "answered with facts and citations" so I'd like to clarify the question in a way that can be answered with facts and citations: Are there any published dictionary works that have published the number of entries contained in the work as well as the level of effort that was required to create them? I actually found one for which I can provide an answer if this question is opened back up, and perhaps there would be more examples from others with the question posed in this way.

The Motivation:

As a software developer who has created multiple language-related websites, I have frequently found myself in a position of needing programmatic access to a free dictionary. There are tons of web-based dictionaries that say they are free but they are not really free... especially if you want to use them for a commercial project, such as an English language learning website. Even for academic research, non-profit use, or similar projects, many dictionary providers will require you to pay them, sometimes for each word you want to look up programmatically, and of course there are all kinds of restrictions on how you use their material (like you can't make copies of anything to be stored on your computer), as well as things you have to do (copyright notices, links to their site, etc). I've even found some that are free, but they're illegally giving access to another party's copyrighted material.

I wish there was a dictionary out there that was so free the creator would literally put a link on their homepage to download their database, including all dictionary content, with no login required and no strings attached. Literally anyone could download it and do whatever they wanted with it for free and with no obligations.

I don't think such a dictionary exists. If it does I'd be happy to know where to find it. Otherwise, I'm thinking of trying to get one created. I've got the technical ability to create such a database and make it available online, but I have zero ability to actually create the content. So, I'm trying to figure out the level of effort. If I'm told 10,000 hours would do it, it doesn't seem impossible that I could find 100 qualified people to donate 100 hours each (over the course of a year or two) to do the work... I'd donate my time to do the tech work, to publish it in real time as it is being built, and to support the type of truly free access described above.

8
  • 1
    See en.m.wikipedia.org/wiki/Wiktionary. It is apparently based on out-of-copyright dictionaries. Also has entries created by bots.
    – Xanne
    Commented Feb 12, 2021 at 4:44
  • Thanks for the comment. I've seen Wiktionary but it comes with a dual license, both of which have some of the undesirable requirements described above. I'm also not wild about the bots and dated source material - I'd want human creation and up to date language.
    – Kairei
    Commented Feb 12, 2021 at 6:20
  • What about WordNet as mentioned in english.stackexchange.com/q/8233/191178 ? I guess you can argue that it comes with “strings attached” but if that string is that you have to cite them, is that really a dealbreaker?
    – Laurel
    Commented Feb 12, 2021 at 16:46
  • Thanks for the suggestion. Yes, I reviewed WordNet as well and yes I consider any requirement to include any notice a deal breaker as it means the end user has to worry about if some original content owner (or some future new owner of that copyright) might make changes or otherwise cause issues. The WordNet license page even has what I'd consider a "warning" about making sure you have an attorney review the license based on the planned use.
    – Kairei
    Commented Feb 12, 2021 at 17:05
  • While I'm pretty convinced there isn't one, I certainly appreciate more potential "free" dictionaries... but getting some opinions on the main question of the level of effort to create one would be very much appreciated. I've continued to do my own research and found something called COMLEX which has about 35,000 entries. According to nlp.cs.nyu.edu/comlex/index.html, it was created "by a team of four linguistics graduate students, working half-time for approximately one year." That's about 4,000 person hours for 35,000 entries (about 8 or 9 per person per hour). Sound reasonable?
    – Kairei
    Commented Feb 13, 2021 at 2:11

0

Browse other questions tagged .