Natural Language Processing

My position as a high school math teacher reinforced in me the importance of breaking down language into smaller units with power to convey meaning. Though I am still pretty verbose, working with immigrants and refugees (and trying to teach them Algebra) made me more conscious of the words that I chose when explaining concepts or giving instructions.

As a mathematician and grammar geek, I want to insist on the importance of proper syntax and maintain that it is integral to conveying meaning, but the truth is more nuanced than that. Syntax and context can change the meaning of the words that we use, but that is why the terminology is “sentiment analysis” not “sentiment decree” – there is a margin of error to assigning sentiment based on collections of key words. However, there is still much to learn from reducing sentences down to their most meaningful items.

A common practice in sentiment analysis is to reduce responses down to the stopwords level, which removes articles and other words deemed to be non-essential. This reduction allows a computer algorithm to more neatly discover relationships between words (as features) and sentiments (as a dependent variable).

This slideshow requires JavaScript.



Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: