Sentiment analysis has become a very
popular big data application in the recent
years.
Consider for example, the hundreds of
millions of tweets everyday.
As a result, organizations can listen to
the voice of their consumers like never
before.
Manufacturers of consumer goods from
electronics to food products are able to
measure the popularity of their brands
versus the those of their competitors by
simply counting the number of positive
versus the number of negative comments
that they find on forums like Twitter,
Facebook, emails, wherever.
They even convert the voice that they
listen to in a call center to text and
then measure the number of positive
sentiment versus the negative one.
Well, most often the sentiment is negative
because people don't talk too many
positive things about brands, but when
they do, that means the brand is probably
very positive.
So you can build that bias in, but this
has become an extremely popular
application.
In the big data world and let's see how it
works using Beijian machine learning.
Think about a few comments that all of you
might have been posting on the forum.
While I haven't used real comments.
I made this up, obviously.
There aren't that many comments on the
forum; I wish there were.
Be that as it may, think about comments
like, like this, it's a positive comment.
There may be lots of these.
But there may be some which are very
negative.
And so on and so forth.
Suppose we were able to manually label
comments as being positive or negative.
In this obviously we use human intuition,
and not some automated technique.
This is called the training phase.
After that, could we figure out using
Naive Bayes.
Whether a new comment is positive or
negative.
Let's see how to do that.
First, we need to compute the a priori
probability, that is the over all chance
that a comment is positive which is simply
the number of positive comments divided by
the total number of comments which is
8000, in this case 6000 of them being
positive.
And similarly, the probability of a
comment being negative a prioiri, which is
the number of negative comments divided by
the total.
And then needing to compute the
likelihoods.
For example, the probability that the word
like occurs within the positive comments.
So of the 6,000 positive comments, like
occurs in only 2,000 of them.
So, we get this likelihood.
Similarly the probability that enjoy
occurs amongst the positive comments can
be computed similarly.
Now, I notice that there are no comments
which are positive, but include the word,
hate.
We can't put zero for this because that
would make the formula go completely out
of whack.
So we replace those by a low number, like
one.
This is called smoothing, in the Naive
Bayes classifier.
And this is important because we have to
include all likelihood probabilities.
Now we do the same thing for the negative
comments.
The probability that hate occurs amongst
negative comments is 800 out of the 2000
negative comments because this comment
contains hate.
The probability that war occurs is one,
like occurs again by smoothing is 1/2000
and so on.
Not also that the probability that enjoy
which might be thought of as a, as a
positive comment.
Amongst the negative comments is not that
small.
I will come to this phenomena later.
The reason is that the enjoy is occurring
along with a negative term, similarly with
LUT.
For the moment, we need to factor in all
the likelihood probabilities simply by
looking at the words and their
occurrences.
And we'll worry about things like, not
next week, or rather the week afterwards.
We're only going to consider the words
marked bold.
And for these words the likelihood
probabilities look like this.
So we can compute the probability that
like occurs amongst all the positive
comments easy and so on and simply the
same thing is done for the negative
comments.
You can work this out for yourself.
It's a good exercise to do.
Now faced with a new tweet, say, "I really
liked this simple course a lot; something
we haven't seen before." We can compute
the likelihood ratio, which is simply the
probability of like occurring, because
like happens to occur in this tweet, given
that is positive and all.
Everything in the enumerator is given it
is positive.
And for hate, which does not occur we
compute the probability that hate does not
occur, given that the sentiment is
positive by taking one minus the
likelihood ratio.
Clearly the hate can be there or not
there.
So if it's not there, the probably of not
hate given positive is one minus the
probably of hate given positive.
We include every possible word amongst the
bold words that we have considered and
even those which don't occur in this
tweet, we include their probabilities by
taking one minus.
Lastly, we multiply it by the A prior
probability.
Similarly for the denominator.
And we get a likelihood ratio of .026 over
a very small number .00005, which is very
much larger than one.
So, the system can easily label this tweet
as being positive without ever having seen
it before.
This is an example of a machine having
learned to identify which tweets are
positive and which are negative, based on
historical data using the naive Bayes
classifier.