Is It Possible To Get Better Results For Parsing Imperative Sentences With Stanfordnlp?

I want to find patterns in sentence structure. Therefore I'm trying to get the parse tree as preprocessing. Until now I used the Stanford CoreNLPParser. Many of my sentences are i

Solution 1:

I looked at your examples, and it seems that the parser is classifying the first word in your imperatives as noun (NN) or proper noun (NNP). I think that is because a capital letter is present in the beginning of the first word (and proper nouns start with a capital letter). Essentially, the parser hasn't "learned" that every sentence in English starts with a capital letter, and that the capital letter in the beginning provides no information about the part-of-speech.

Of course, the long-term solution to this is that the parser should be trained on other texts with imperatives, but there is also a short-run hack to this problem: since the problem lies with capital letters, we can use the .lower() String function in Python (or any other corresponding function in other programming languages) to preprocess the string to lowercase before parsing it.

I myself tried this with your examples and now the parser is correctly classifying the beginning of your sentences as Verb Phrases.

Note:: The above hack may lead to incorrect classification of Proper Nouns because they will no longer have a capital letter. As long as you are not using the parser to detect Proper Nouns alongside imperatives, you should be okay

Solution 2:

Unfortunately the part-of-speech tagger is trained on the Wall Street Journal from years ago. So there are issues where imperative statements aren't in the training data. So it's going to guess wrong at times. But on some imperative statements it does the right thing as well. I think if the first word is a clear verb like "Call" you will get better performance.

Another issue I saw is the verb "text" (as in send a text message) is not being handled well.

I think we would be excited to add some contemporary data and add some imperative training data to help out.