Hey. In this video,
we will briefly discuss what will be covered during the next weeks.
So, during this week we will discuss text classification tasks.
So these are the tasks that are very popular in any applications.
For example, you need to predict sentiment for some reviews.
You need to know whether the review is positive or negative.
Or, you need to filter spam in your emails or something else.
So, what you do is actually you represent your text as a bag of words,
you compute some nice features and you apply some machine
learning algorithm to predict the class of this text.
There are actually lots of practical tips that you need to know to succeed in this task.
So, during this week my colleague will tell you about it.
Now, the next week will be about
representing text not as a bag of words but as a sequence.
So, what can you do when you represent a text as a sequence of words?
One task would be language modeling.
So language models are about predicting
the probabilities of the next words given some previous words.
So, this can be used to do text generation.
And this is useful in many applications.
For example, if you do machine translation,
you are given some sequence of words,
some sentence on English and then you need to translate it,
let's say to Russian,
so you need to generate some Russian text and that is where you'll need language model.
Now, another important task is called sequence tagging.
So this is the task when you have a sequence of words and you
need to predict text for each of the words in this sequence.
For example, it could be part-of-speech texts
so you need to know that some words are nouns,
some words are verbs and so on.
Another task would be to find named entities and this is really useful.
For example, you can find some names of the cities and use them
as features for your previous task for text classification.
Now, another task which is called semantic
slot filling has been just covered in our previous video.
So this is about some slots.
For example, you need to pass a query and you need to know that
the person wants to book a table for some specific time in some specific place.
And those time and place would be the slots for you.
Now, we can do something even more complicated
and try to understand the meaning of words or some pieces of text.
How do we represent the meaning?
Well, one easy way to do this would be to use vectors.
So, you map all the words to some vectors.
Let's say 300 dimensional vectors of
some float numbers and these vectors will have really nice properties.
So, similar words will have similar vectors.
For example, this nice picture tells you that Cappuccino and Espresso are the same thing
just because the cosine similarity between the vectors is really high.
Now, we will also discuss topic models.
Topic models deal with documents as a whole and they also represent them by
some vectors that can tell you what are the topics in these certain documents.
This is really useful when you need to, for example,
describe the topics of a big data set like Wikipedia or
some news flows or social networks or any other text data that you are interested in.
Now, this is just another example of how those methods can be used.
So, let's say that we could represent our words by vectors
just by three dimensional vectors and we have them depicted here in this space.
And we know the similarity between them.
So, we know the distance between those blue dots.
Once we know these distances,
we can create a similarity graph for words.
So, in the middle picture,
the nodes are words,
and the edges have the similarities between the nodes.
Now, this graph is actually very useful.
Why? Because when you have some labels for some nodes of this graph, for example,
if you know that "Laugh" has the label
"Funny," you can try to propagate these labels through the graph.
So those words that are similar will get the same labels.
For example, the word "Haha" there will also get the label
"Funny" because it is similar to the word "Laugh."
Okay. This can be used in many different applications and we'll cover this in week three.
Now, the next one will be more advanced
and this week will be about sequence to sequence tasks.
Actually, nearly any task in NLP can be somehow stated as a sequence to sequence tasks.
Just to give you a few examples,
it would be about machine translation.
So there, obviously, you have
one sentence and you need to translate it to the other sentence.
So, these are the two sequences.
But, for example, in summarization,
you have the big document as an input.
This is some long sequence and you need to produce some short summary,
and this is also a sequence.
You get this task in speech recognition or in
conversational chat-bot where you have some questions and answers.
Right? All these tasks can be nicely solved with
so-called encoder-decoder architecture in neural networks.
Let us see just the idea.
So, we're given a sentence and we have an encoder.
Now we feed this sentence to the encoder.
What we get is some hidden representation of the input sentence.
After that, the decoder generates the output sentence.
So, this is how we get our final translation,
or summary, or something else.
Now, during the last week of our course,
we will combine all the knowledge that we have to build a dialogue system.
So, dialogue systems can be different and there are at least two important types.
One type would be goal-oriented agents that try to solve some particular task.
For example, they can assist you in a bank,
or help you with online shopping, or something like that.
On the contrary, there are also conversational, entertaining,
chat-bots that just wants to somehow hold a conversation with you.
So, there are different types of methods to be used in
these types of tasks and we will cover them in details during the last week.
And the project will be about
stack overflow chat-bot that tries to assist with the search.
So, stay with us and we will discuss everything in many details.