So now let's talk about Amazon Redshift,
which is a database, but also an analytics engine.
So Redshift is based on the PostgreSQL technology,
but instead of Postgres,
it's not used for online transaction processing.
It's actually an all app type of database,
which means online analytical processing.
And it's used for analytics and data warehousing.
So it has a 10 X better performance
than any other data warehouses out there,
and it scales to petabytes and petabytes of data.
So the idea is that you would load all your data
into Redshift and then very quickly
you can randomize it from within Redshift.
So the Redshift has good performance improvements
because it's actually using a columnar storage of data,
so instead of row based,
and it has a parallel query engine.
You pay as you go for all the instances you provision
in your Redshift cluster.
And to perform your queries,
you can use just directly some SQL statements.
So any business intelligence tools,
such as Amazon Quicksights
or other ones such as Tableau integrate with Redshift.
And so if you had to compare Redshift and Athena,
Redshift, first you have to load the data,
sometimes from Amazon S3,
you have to load all the data into Redshift.
And then Redshift to Athena,
well, if it's loaded in Redshift,
Redshift is going to have much faster queries.
Also Redshift can do much faster joins
and much faster integration
because Redshift actually has something
that Athena does not have.
Redshift has indexes, and it builds indexes
to have this very high performance for a data warehouse.
So if it's just an ad hoc query on Amazon S3,
then Athena is going to be a great use case.
But if it's like intense data warehousing
with many queries and they're complicated,
there are joins, aggregations and so on,
then a Redshift is going to be a better candidate.
So your Redshift cluster has two things.
It has leader nodes,
and they do query planning and results aggregation.
And then it has compute notes,
and they actually perform the queries
and they send back the results to the leader.
And because it's a Redshift cluster,
you have to provision the node size in advance.
And if you wanted to do therefore cost saving,
you could use reserved instances.
So your Redshift cluster has a leader node
and then some compute nodes,
and then you would submit a query
in the SQL form to the leader node
and the query would happen in the backend.
So this is it for an overview of Redshift.
So let's talk about snapshots
and disaster recovery in Redshift.
So Redshift has no multi-AZ mode, okay?
All your cluster is in one availability zone.
And so for this, if you want
to have a disaster recovery strategy for Redshift,
you need to take snapshots.
So snapshots are point in time backups of a cluster,
and they will be stored internally in Amazon S3.
And they are incremental,
only what that has changed will be saved.
And that will save you a lot of space, obviously.
You can restore a snapshot into a new Redshift cluster,
and you have two modes for cluster for snapshots.
You can take them manually,
or you can take them in an automated way.
So if you automate it,
every eight hours is going to be a snapshot
or every five gigabytes on the schedule.
And then you can set the retention
for your automated snapshots,
or if you wanna do a manual snapshot,
the snapshot is going to be retained
until you delete it manually.
And then a really, really cool feature of Redshift
is that you can configure Redshift
to automatically copy snapshots,
whether they're automated or manual
of the cluster into another AWS region,
hence giving you a disaster recovery strategy.
So let's take this Redshift cluster,
your original cluster,
and then we have another region.
So we're going to take snapshots.
They're going to be automatically copied
into your new region.
And from there you can restore a new Redshift cluster
from that copied snapshots.
Now let's talk about how we ingest data into Redshift.
So we have three ways I'm going to describe to you.
The first one is to use Amazon Kinesis Data Firehose.
So we have Firehose that will be receiving data
from different sources,
and then it will send it into Redshift.
And to do so, it will first write the data
into an Amazon S3 bucket,
and then Kinesis Data Firehose will issue automatically
an S3 copy command to load the data into Redshift.
Now, how does this copy command work?
We can also use it manually.
So you would load data into S3,
and then you would issue a copy command directly
from Redshift to copy data from S3 bucket,
using an iam_role into your Amazon Redshift cluster.
And as we've seen, there are two ways of doing so.
You might go through the internet
because your S3 buckets are public through the internet.
I mean, they're not public,
but they're connected to the internet.
And so therefore you're going to do that flow
through the internet back into your Redshift cluster,
and this is without enhanced VPC routing.
But if you wanted all your network to remain private
into your virtual private cloud,
then you can enable enhance VPC routing
to have all the data flow through the VPC entirely.
And finally, if you wanted to insert data
using the JDBC driver into the Redshift cluster,
you could do so as well.
So for example, if you have an application,
an EC2 instance that needs to write data
into your Redshift cluster,
you would need to use this method.
And in that case, it is much better
to write large batches of data into Amazon Redshift,
instead of one row at a time,
which would be truly inefficient for this type of database.
So a cool feature of Redshift is Redshift Spectrum.
The idea is that you would have data in Amazon S3,
and you want to analyze it using Redshift,
but you don't want to load it into Redshift first.
And on top of it,
you want to use a lot more processing power.
So you use Redshift Spectrum,
and you must have a Redshift cluster already available
to start the query.
And then once you start the query,
the query will then be submitted
to thousands of Redshift Spectrum nodes
that will perform the query onto your data in S3.
So let's go for an example.
You have your Redshift cluster with a leader node,
and a bunch of compute notes as we've seen.
And the data you want to analyze is in Amazon S3.
So in this case, we're going to run a query
onto our Redshift cluster,
as we can see what we want to do is
that the table we want to query is living in S3.
So we have from S3 dots and whatever you want.
In that case, Spectrum will launch automatically.
And so the query is going to be submitted
to thousands of Redshift Spectrum nodes,
who are going to read the data from Amazon S3
and perform some aggregations.
And when they're done,
they're going to send back the results
into your own Amazon Redshift cluster,
and then you will get back into whoever initiated the query.
But with this feature,
we can leverage a lot more processing power from Redshift
than the one we have provisioned into our cluster
without in the first place loading the data
from Amazon S3 into Redshift.
So that's it for Amazon Redshift.
I hope you liked it,
and I will see you in the next lecture.