What Allison should know what is in it anyway?
Internet is, that massive computer network.
The one that's becoming really big now.
What do you mean, that's big? How does one not.
What do you write to? What? Like mail?
No, a lot of people use it and communicate.
It's. I guess I can communicate with NBC writers and producers.
Allison, can you explain what internet is?
How amazing is that?
Just over 20 years ago, people didn't even know what the internet was.
And today we can't even imagine our lives are flooded.
Welcome to the deep Learning A to Z course.
My name is Killer Meiko and along with the co instructor huddle under Pontus.
We're super excited to have you on board.
And today we're going to give you a quick overview of what deep learning it is
and why it's picking up right now.
So let's get started.
Why did we have a look at that clip and what is this photo over here.
Well that clip was from 1994.
This is a photo of a computer from 1980.
And the reason
why we kind of delving into history a little bit is because neural networks,
along with deep learning, have been around for quite some time,
and they've only started picking up now and impacting the world right now.
But if you look back at the 80s, you'll see that
even though they were invented in the 60s and 70s, they really caught on
to a trend or got caught wind in the 80s.
So people started talking about them a lot.
there was a lot of research in that area, and everybody thought that deep
learning or neural networks were this new thing
that is going to impact the world, is going to change
everything, is going to solve all the world problems.
And then it kind of slowly died off over the next decade.
And so what happened?
Why did why did the neural networks not survive and not change the world?
It wasn't the reason for that, that it was just not good enough
that they're, you know, not that good at predicting things
that are not that good at modeling and basically just not a good, invention.
Or is there another reason?
Well, actually, there is another reason.
And the reason is in front of us.
It's the fact that technology back then was not up to the right
standard to facilitate neural networks.
In order for neural networks and deep learning to work properly,
you need two things.
You need data, and you need a lot of data.
And you need processing power.
You need strong computers to process that data and facilitate the neural networks.
So let's have a look at how,
as a data or storage of data has evolved over the years.
And then we'll look at how technology has evolved.
So here we've got three years 1956 1980 2017.
how did storage look back in 1956?
Well, there's a hard drive, and that hard drive is only a five.
Wait for it, Meg.
Hard drive.
That's five megabytes right there.
on the forklift, the size of a small room.
That's a hard drive.
Being, transported to another, location on a plane.
And, that is what storage looked like in the in 1956.
You had to pay a company.
Had to pay 2500 dollars of those days, dollars
to rent that hard drive to rent it, not buy a to rent it for one month.
in 1980, the situation improved a little bit.
So here we've got a ten megabyte hard drive for 3500 dollars.
It's still very expensive and only ten megabytes.
So that's like one photo these days.
And, today in 2017, we've got a 256 gigabyte
SSD card for $150, which can fit on your finger.
And, if you're watching this video a year later or like in 2019 or 2025,
you probably laughing to yourself because by then you have even stronger,
storage capacity.
But nevertheless, the point stands.
So if we compare these across the board and without even taking price and size
into consideration, just the capacity of whatever was trending at the time.
So, from 1956 to 1980, capacity increased about double.
And then it increased about 25,600 times.
And the, you know, the length of the periods is not that different.
So from 1956 to 19, aged 24 years, from 1980 to 2017, 37 years,
so not that much of an increase in time, but a huge jump in technological progress.
And that stands to show that this is not a linear trend.
This is an exponential growth in technology.
And if we add into it, take into account price and size,
it'll be in the millions of, increase.
And here we actually have a chart on a logarithmic scale.
So if we plot the, hard drive cost per gigabyte,
you'll see that looks something like this.
We're very quickly approaching zero.
right now you can get storage on Dropbox and Google Drive,
which doesn't cost you anything. Cloud storage.
and, that's going to continue.
And in fact, over the years, this is going to, go even further.
Right now, scientists are really looking into using DNA for storage.
And right now it's quite expensive.
It costs $7,000 to synthesize, two megabytes of data.
and then another $2,000 to read it.
But that kind of reminds you
this whole situation of the harddrive and the plane, you know,
that this is going to be mitigated very, very quickly with this exponential curve.
10 to 10 years from now, 20 years from now,
everybody's going to be using DNA storage if we go down this direction.
And here are some stats around that.
So yeah, you can explore it further.
Maybe pause this, pause the video.
If you want to read a bit more about this.
This is from nature.com.
And basically you can store all of the world's data
in just one kilo, one kilogram of DNA storage.
Or you can store about 1,000,000,000TB of data in one gram of DNA storage.
So that's just something to to show how quickly we're progressing.
And that this is why deep learning is picking up.
Now that we are finally at the stage where we have enough data to train super,
cool, super sophisticated models back then, in the 80s,
when it was first initially invented, it was just wasn't the case.
And, the second thing we talked about is, processing capacity.
So here we've got an exponential curve again, on a log scale.
It's on ideally portrayed here, but on the right you can see it's a log scale.
And, this is how computers have been evolving.
So again, feel free to pause this slide.
This is called Moore's Law.
You've probably heard of it, how quickly
the processing capacity of computers has, been evolving.
right now we're somewhere over here where an average computer you can buy
for a thousand bucks, thinks the at the speed of,
the a brain over at,
by 2025 will be the speed of a human or 2023,
and then, by 2050 or 2045, it'll surpass
all of the, humans combined.
So basically, we're entering the era of computers that are extremely powerful,
that can process things way faster than, than we than we can imagine.
And that is what is facilitating deep learning.
So all of this brings us to the question, what is deep learning?
What what is this whole neural network situation?
What what is going on? What are we even talking about here?
And you've probably seen a picture something like this.
So let's dive into it. What is deep learning?
This gentleman over here, Geoffrey
Hinton, is known as the godfather of deep learning.
And, he, did research on deep learning in the 80s.
And, he's done lots and lots of work, lots of research papers.
he's published, in, deep learning right now, he works at Google.
So a lot of the things
that we're going to be talking about actually come from Geoffrey Hinton.
And, you can see a lot.
He's got quite a few YouTube videos.
He explains things really well.
So I highly recommend checking them out.
And so the idea behind deep learning is to, look at the human brain.
And this is quite there's going to be quite
a bit of neuroscience coming up and in these tutorials.
And what are we trying to do here is to mimic how the human brain operates.
As you know, we don't know that much.
You don't know everything about the human brain.
But that little model, we know we want to mimic it and recreate it.
And why is that?
Well, because the human brain seems to be one of the most powerful tools
on this planet for learning, for learning, adapting skills and then applying them.
And if computers could copy that, then we could just
leverage what natural selection has already decided for us.
All of those kind of algorithms that it has decided are the best.
We just going to leverage that. Why reinvent the bicycle? Right.
So let's see how this works here.
We've got some neurons.
So these are neurons which have been smeared on to gloss
and then have been, looked at under a microscope with some coloring.
And this is you can see what they look like.
So they have like a body,
they have these branches and they have like tails and so on.
And so you can see them, they have like a nucleus inside in the middle.
And that's, that's basically what a neuron looks like in,
the human brain, there's approximately 100 billion neurons all together.
So these are individual neurons.
These are actually motor neurons because they're bigger, they're easier to see.
But nevertheless, there's 100 billion neurons in the human brain.
And each neuron is connected to as many as about a thousand of its neighbors.
So, to give you a picture, this is what it looks like.
This is an actual, section of, the human brain.
And this is the cerebellum,
which is this part of your brain at the back.
It's, is responsible for, like,
motor tricks and for, you know, keeping your balance and a
some language capabilities and stuff like that.
So, this is just to show how vast,
how many neurons there are there, like billions
and billions and billions of neurons all connect to your brain.
It's not like when you're talking about 5 or 500 or 1000 or 1 million.
This is billions of neurons in there.
And, yeah.
So that's that's what we're going to be trying to recreate.
So how do we recreate this in a computer?
Well, we create an artificial structure called an artificial neural net,
where we have nodes or neurons,
and we're going to have some neurons for input values.
So these are values that you, that you know about a certain situation.
So for instance you're modeling something you want to predict something.
You always going to have some input, something to start your, predictions off.
And then that's called the input layer.
Then you have the output so that, value that you want to predict or the surprise,
whether it's, is somebody going to leave, the bank or stay in the bank?
Is is this a fraudulent transaction?
It's a real transaction and so on.
so that's going to be your output layer.
And in between we're going to have a hidden layer.
So, as you can see in your brain, you have so many neurons.
So some information is coming in through your eyes, ears, nose.
So you basically your senses.
And then it's, it's not just going right away to the output
where you have
the result is going through
all of these billions and billions and billions of neurons
before it gets to the output.
And this is the whole concept behind it that we're going to model the brain.
So we need these hidden layers that are there before the output.
So the input layers, neurons connected the hidden
layer neurons that hidden layer, neurons that connect to the output value.
And so this is, this is pretty cool, but what is this all about?
Where is the deep learning here?
Why is it called deep learning is nothing deep in here?
Well, this is kind of like an option, which one might call shallow learning.
There isn't much indeed going on.
But why is it called deep learning?
Well, because then we take this to the next level.
We separate it even further.
And we have not just one hidden layer.
We have lots and lots and lots of hidden layers.
And then we connect to everything.
Just like in the human brain, we connect everything, interconnected everything.
And that's how the input values
are processed through all these hidden layers, just like in the human brain.
Then we have an output value, and now we're talking deep learning.
So that's what deep learning is all about on a very abstract level.
In the further tutorials, we're going to dissect and dive deep into deep learning.
And by the end of it, you will know what deep learning is all about,
and you will know how to apply it in your projects.
Super excited about this!
Can't wait to get started and I look forward
to seeing you in the next tutorial and till then, enjoy deep learning!