7

I am trying to make a script that can dissect an English sentence.

The problem is, I have no idea how to dissect an English sentence when the words are not familiar. I know what the nouns, verbs, etc are, because I recognize the words. I can give the script a large set of words so it can recognize them as well, but surely there are certain "patterns" that one can check to see what part of a sentence the noun or verb is, even though the noun or verb is not known at the time.

Are there such patterns for the English language? Can one dissect a sentence by only "recognizing" a few words?

Understanding the whole language is not necessary, as nobody really understands everything of a natural language.

Having simple patterns and "rules" should be enough. I understand that, because English is a natural language, every rule or pattern would have some kind of exception and mapping each one of those would probably take forever. But ignoring those exceptions (apart from the most common), I could make the script understand quite a lot.

If necessary, I could theoretically then add exceptions until done (read: forever).

Here are a few examples of patterns. (I'm not sure if these are exactly what I need, but as an example they should be fine.)

  • Subject + Verb (S-V)
  • Verb + Subject (V-S)
  • Subject + Verb + Direct Object (S-V-DO)
  • Subject + Verb + Complement (S-V-SC)
  • Subject + Verb + Indirect Object + Direct Object (S-V-IO-DO)
  • Subject + Verb + Direct Object + Object Complement (S-V-DO-OC)

Recognizing in what pattern a sentence falls, is probably first thing I should aim for. Of course, this is much easier said then done, also, for every pattern I find, questions seem to be missing for it, but that is another issue I can deal with later.

From what I can tell, the best course of action is to find the verb in the sentence, then check if the words before it match as the subject, then check the words behind it. Etc.

While finding the verb should not be that big an issue, matching a subject seems quite hard: it can't come from a database since, thanks to names, there are infinite possibilities. Still.. shouldn't a subject follow a pattern just as well?

Also, if I were to go this route, wouldn't it end up being a maze of patterns(not that thats necesarily a bad thing)?

Improve this question
7
  • 3
    You seem to be asking for a formal language definition for an informal/natural language that is not restricted to formal rules. Consequently, yours is an extremely broad question. Here’s a related web page I found that discusses formal language and natural English: goo.gl/aYYxt
    – danorton
    Commented Sep 14, 2011 at 13:14
  • Thanks for the link, from scanning through it, I think it should allow the script to recognize simple sentences.
    – Johan
    Commented Sep 14, 2011 at 13:41
  • What do you mean by simple sentences? The patterns in a natural language are never all standard, and so making a script is not going to be easy. You'd have to classify every pattern possible in a given language... not something you can do in a weekend. :)
    – Alenanno
    Commented Sep 14, 2011 at 13:43
  • You mention making a distinction between nouns and verbs, but there are quite a few cases where a word could be either if taken out of context, e.g., use, run, slice, die, table. "Green" is usually an adjective but sometimes a noun.
    – nnnnnn
    Commented Sep 14, 2011 at 13:44
  • Thanks for the link @danarton, that should help me get started, everybody else, thanks for your comments, I updated the question to give better description
    – Johan
    Commented Sep 14, 2011 at 14:05

2 Answers 2

Reset to default
6

There is an entire field looking at this question called "Computational Linguistics". Looking at any online translation tool you can see that they still have a ways to go, but there has been a lot done on parsing English.

Stanford provides a robust English parser here with the homepage here.

I recommend using an established library for your script rather than writing your own. There is no reason to reinvent the wheel, especially a wheel that contains the complications of the English language.

Improve this answer
7
  • Thanks, this should help me quite alot! However, if you don't reinvent the wheel, you can not be sure that it is round ;)
    – Johan
    Commented Sep 14, 2011 at 16:56
  • @Johan I disagree, having invented the wheel does not give you any special knowledge of its roundness. The same holds true for more complicated problems, such as parsing English. This is a very common and dangerous fallacy in software development. Avoid the "it wasn't built here" mentality.
    – Jordan Bentley
    Commented Sep 14, 2011 at 17:28
  • @Johan I should also note that in college I took an entire class on formal languages/context free grammars/regular expressions/etc and it barely scratched the surface of this problem. Unless you have an original contribution to the field, take a page from Newton and stand on the shoulders of giants :)
    – Jordan Bentley
    Commented Sep 14, 2011 at 17:31
  • For now, I'm just exploring possibilities, having made alot that was 'already out there' has made me a better programmer, so I'm not gonna give up that philosophy, then again, once I understand what makes up a good english-sentence parser, what makes the wheel 'round' there is nothing against using a wheel thats much rounder then mine. But at least then I know why it's round. (sorry if I just stretched the metaphore too far :P )
    – Johan
    Commented Sep 14, 2011 at 19:38
  • Is this something for work or are you just doing this for fun?
    – Jordan Bentley
    Commented Sep 14, 2011 at 19:46
0

Only to a very limited degree.

For example there are a number of words which must be followed by a noun phrase ("the", "my", "some"), so you can be pretty sure that one of the words following these will be a noun. But you can't necessarily determine which.

In general, no, you can't do this without knowing a lot of words.

Improve this answer
3
  • So the script should know words like "the", "my", "some", etc. (and other words that have specific rules attached), that shouldn't be more then a few hundred, right? Compared to knowing all nouns / verbs / etc that is not that much :)
    – Johan
    Commented Sep 14, 2011 at 14:10
  • @Johan: I think it's probably fair to say collectively humanity has spent at least billions of pounds trying to crack computerised parsing of language in recent decades. And they haven't really gotten very far, to be honest. Maybe you shouldn't set your sights too high when it comes to how well your script performs.
    – FumbleFingers
    Commented Sep 14, 2011 at 15:26
  • While the ultimate goal would obviously indeed be a far future thing, on short notice, I only want to recognize simple sentences, I have no doubt that it will be easy to confuse the script.
    – Johan
    Commented Sep 14, 2011 at 16:52

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .