Tell me more ×
English Language & Usage Stack Exchange is a question and answer site for linguists, etymologists, and serious English language enthusiasts. It's 100% free, no registration required.
up vote 1 down vote favorite

I need your help to set all rules for extracting names/concepts from a phrase.

for example, in the phrase "in toshka, it's always sunny" the name/concept here is toshka, which is a place in Egypt. i found it because it's a Preposition followed by a Noun

What are other rules for finding entities/names?

share|improve this question
-all rules-? That's quite a big order for any language. – Mitch Sep 26 '12 at 15:04
At the risk of offering a trivial answer in comparison to @tchrist's daunting suggestion, I'd suggest that a dictionary based approach would have reasonable but incomplete success with modest effort. – Russell McMahon Sep 26 '12 at 15:10
You need to be much more specific about what kind of named entity you want to find. Geographical locations? Cities, nations, areas? People, prefessions, titles, first or last names? etc. etc. etc. – Mitch Sep 26 '12 at 15:20
You mean a noun following a preposition. Aren't names/concepts always going to be described by nouns? If you can identify a noun following a preposition, why can't you just identify the noun? Do you really mean: 'What are some accepted frames for identifying nouns?'? Here's a reasonably well-known (but not totally accurate) one: The ____ is / are (not) nice. ____ is nice would pick up singular proper nouns (and nominative pronouns). Of course, you'd need different adjectives that would be appropriate for nouns like proton, DNA, vacuum, momentum ... – Edwin Ashworth Sep 26 '12 at 15:35
yes Edwin,I'm looking for accepted frames for identifying nouns, but not all frames, for example:"danny the dog is fierce", the name/concept here is "danny" – Omar Gamil Sep 26 '12 at 15:49

closed as not constructive by Carlo_R., Andrew Leach, Mitch, simchona♦ Sep 26 '12 at 15:22

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or specific expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, see the FAQ for guidance.

1 Answer

up vote 3 down vote accepted

This sounds like computational linguistics to me, but it isn’t clear which particular NLP task you actually mean here.

If you are just looking for nouns, you probably need a part-of-speech tagger. Note that POS taggers that are not integrated into syntactic analysers do not perform at all as well as those that are. And you are going to need some sort of lemmatizer to reduce to citation forms using inflectional morphology.

Because you mention entities, it may also be a matter named-entity recognition.

But when you say “concept”, I wonder whether you don’t need something more like the Stanford Event Parser. Now you may also have to work with derivational morphology.

Any way you slice it, this is much too complex, and complicated, a topic to be succinctly and definitively answered here. You will need to look into the current state-of-the-art research in CL and NLP.

share|improve this answer
you are absolutely correct tchrist, this IS an NLP research, the problem with NER is that it cannot recognize outside what it's taught, the example above failed with every NER engine i tried. i'm not looking for a complete solution, just a bunch of rules like the one i mentioned, preposition after a noun, Verb, 3rd ps. sing. present followed by noun, etc... – Omar Gamil Sep 26 '12 at 15:17
@OmarGamil You will find that biomedical NER engines need to be able to recognize hitherto-unseen entities, because researchers are forever coining new names for things. They typically use statistical approaches. Try running Jabberwocky through them and see what they find. :) I don’t know which ones you’ve used, but it is possible that SO or Linguistics.SE might be a better fit for your question, provided you edit it a bit for focus. – tchrist Sep 26 '12 at 15:22
Thank you tchrist ;) – Omar Gamil Sep 26 '12 at 15:46

Not the answer you're looking for? Browse other questions tagged or ask your own question.