Hello
and welcome back to the course on Deep Natural Language Processing.
And today we're talking about classical versus deep learning models.
So let's have a look.
as we discussed we've got this diagram.
And on the left we've got the natural language processing models overall.
And on the right we have the deep learning models overall.
So there overlap is a deep natural language processing.
Those are the very advanced types of natural language processing models
which use deep learning.
And then finally at the bottom here
we've got the sequence to sequence models which will be interested in at the end.
And so I thought that this tutorial would be fun.
if we had a look at the comparison between a couple.
So we'll look at a couple of examples from here.
we'll look at an example from here.
And then finally we'll mention the sequence to sequence just to kind of,
lay out the playing field so we know what we're dealing with.
and what types exist.
Because we could definitely
just jump into sequence to sequence and look at the most powerful things,
but then it would still be a mystery what our natural language processing
models, how are they different to deep natural language processing models
and how are they different
to sequence, to sequence, what are the applications and so on.
So let's have a look at a couple of examples.
All right.
Some examples.
We're going to have our diagram on the right.
And we're going to start going through these examples one by one.
So number one is the if else rules.
and this is a way
that we used to create chatbots back in the day.
So if else rules, are located over here on our diagram in the, just the NLP part.
And what they entail is
a huge list of possible questions and answers to those questions.
And so once somebody in the chat asks a question or we can identify
that part of the sentence is the question that we have prerecorded,
then we will give them
the correct answer, the answer that is associated with that question.
But as you can imagine, such mechanical, such a mechanical approach
to answering questions or chatting with people
does not result in, anything,
human like anything realistic.
And the results in something like this rather,
where you have questions and then you have the predetermined answers,
but people want something tailored to them, something specific, like
they're asking about something else but doesn't fit into that,
list of questions and answers.
And, it just very quickly becomes a mess.
But that's how, we used to create chatbots,
the next, type or the next, natural language processing model
we will look at is called, an audio
frequency components analysis.
And it's used for speech recognition and sits over here.
So, we're specifically looking like we're purposefully
looking at different examples.
So not just chat bots.
We're looking at different applications.
So you can see that NLP can be applied to speech recognition.
And there used to be or still are algorithms
that use non deep learning methods for speech recognition.
And one of them is audio frequency components analysis.
So in essence what happens is we look at the sound wave of you know,
somebody talking some prerecorded or real time
audio of a human speech.
And then we try to identify what waveforms exist in there.
So in a very kind of simple, way of explaining it,
let's have a look at, for example, here we've got a Fourier transformation.
and we've got the there's our frequency over there.
So it's like one of these it's going over there.
of course this is, this doesn't
even look like human speech to be, frank, but the concept stage.
So, like, we look at what frequency is, can it consists of
and then we compare them to
like the pre recorded frequencies.
So we know that, for instance, certain certain combination of frequencies
means, this type of word or this type of work.
And it's not just about the frequencies, of course.
It's it's much more complex than that.
But this is a very, general overview of what happens.
We look at the frequencies, certain mathematical operations.
This is a key point that we're not doing any, neural, computations.
We're not creating neural networks, which just doing mathematical,
calculations around the frequencies that we can observe of comparing them
to the mathematical calculations we have in a library of,
and pre analyzed frequencies.
And then we're matching it up with finding what word a person is saying,
what question they're asking, or, what the sentence is meaning.
And then that is how we recognize speech.
So there's another version of natural language processing
which is in the green area.
Next one is
the bag of Words model, which is used for classification, a very,
very popular approach for,
text analysis or natural language processing.
And what it does is
basically a while it also sits in the NLP, part and just the green area.
although as we will see further on, it can sit
both in the green area and in the purple area over here.
But that's, something for a later discussion.
What's the bag of words model does is there's our bag of words,
and the way it works is a you might have, like, lots of, text.
For instance, here we've got,
tutors or lecturers or examiners
who are marked, grades.
So who gave a pass or fail grade to, different,
papers or different assignments that were submitted.
And then they also left a comment.
So somebody said great job and left a comment of one meaning a pass.
Somebody said,
amazing work left comment of one well done, one, very well written one.
And then poor effort
zero could have that better, zero try harder than exam zero and so on.
And so from this what the bag of words model will do is it
will put all these words, into a bag and it will remember.
So how often does the word grace come up with a one.
And how often does the word grade come up with a zero?
This is again a very,
general explanation, very high level explanation of what happens.
But in essence, this is, the concept that it will look at the words
and it'll look it will try to classify these words.
It will I will try to, associate these words
with either positive results or negative results in our case.
And so in this case like amazing would be most likely associated with positive.
You don't often see amazing when you're trying to say
when when the sentiment is negative when you try it,
when the person was trying to say something like the work wasn't
well done or well would be associated with one.
But like the word work could be associated with both.
But then these other words, they would be mostly associated past, some like words
like poor or harder or, could have maybe it would be associated with zeros.
And so then it'll remember that will keep these words in a bag.
And next time something comes up, for instance, somebody says,
good, good job.
keep keep it up or something like that.
It will analyze the words that are in that new sentence,
by pulling them out of the bag and looking at them and understanding,
are they mostly associated ones or zeros and then will be able to predict,
it'll be able to classify the new, new comment,
even without knowing what the mark was passive will be able to say
based on the comment, will be able to say, is it a pass or fail?
so that's like a very simplistic application of the bag of words model.
But there's another one, an NLP model.
okay.
So let's look at a deep natural language processing model.
and this one is going to be able to get over here.
It's called convolutional neural networks for text recognition.
So they like the model we're going to be looking at
further down is indeed a deep natural language processing model.
But I was I wanted to give an example of a different model.
So not not just the one that we look at, but like also a deep
natural language processing model, but a different one and one.
I came across is, when we use convolutional neural networks
for, text recognition and then for the, for classification.
And so if, if you're familiar with convolutional networks
or even if you're not, what they're used for is it's a neural network
that is used for mostly image recognition for like videos,
self-driving cars use them to detect obstacles on roads, people and so on.
So mostly it's used for image processing or video processing.
And so it is very interesting to see how convolutional
neural networks can actually be used for text processing.
And the way it works is,
it's these words are, transformed into a matrix,
and, that's done through an operation called embedding of words.
And then once they're in a matrix,
the same principles as convolution,
as we would apply for images in convolutional neural networks are applied.
So there's a convolution operation going through these images.
then they're,
they're pulled Max pull a min pulled or sampled.
and then they're flattened.
and then, you know, then we have the prediction.
So we won't go into detail right now of how convolution
convolutional, operations work.
there's lots, of, tutorials out there and so on, and it's
just not in the scope of this course, because in this course, we'll be looking
at mostly recurrent neural networks, which is RNN and a different type
of neural networks.
But I just wanted to put it out there that you could technically use
convolutional neural networks for text recognition,
just as you would do, with images, a very interesting concept.
That was one of the earlier ones that was explored.
And then we moved on to, recurrent neural networks, as we'll see down the track.
So there we go. That's another one.
And then finally, the main model that we'll be looking for,
working with sequence to sequence has many, many, many different applications.
As we'll see further down.
And that's the one that's over there.
So I'll give you a short preview of what's coming up.
Looks something like this.
And don't worry, it might look very complex right now, but
in a couple of tutorials from here will be, very,
very comfortable with what's going on here and how this whole construct is working
and why it's called sequence to sequence and what exactly it allows us to do.
So there we go.
I hope you enjoyed this quick overview and comparison
of, the different types of models and different,
applications that they have, and we'll continue next time.
I look forward to seeing you there.
And until then, enjoy deep NLP.