So now let's talk about Amazon Athena.
So Athena is a serverless query service
to help you analyze the data
stored in Amazon S3 buckets.
And to analyze this data
you're going to use the standard SQL language
to query the files.
Behind the scenes, Athena is built on the Presto engine
which uses the SQL language.
So the idea is that users
are going to load data into your S3 buckets
or you are going to load data into your S3 buckets
and then you would use the Athena service to query
and analyze this data in Amazon S3 without moving it.
So Athena is serverless
and it analyzes directly your data
living in your S3 buckets.
So it supports different formats,
such as CSV, JSON, ORC,
Avro,
and Parquet
and possibly others.
And the pricing is very simple.
You're just going to pay a fixed amount
per terabytes of data scans.
You don't need to provision any database again
because the whole service is serverless.
So Athena is commonly used with another tool
called Amazon Quicksight to create reports
and dashboards.
So we'll see Quicksight later in depth.
But Amazon Quicksight connects to Athena
which connects to your S3 buckets.
Now the use cases for Amazon Athena
are to do adhoc queries, business intelligence,
analytics, reporting
and to analyze end query, any kind of logs
that originates from your AWS services.
So it could be your VPC flow logs,
your load balancer logs,
your CloudTrail trails,
and so on.
So as an exam tip, anytime you need to analyze data
in Amazon S3 using a serverless SQL engine.
You can think about Athena.
Now I've talked about performance improvements
and you can actually improve Athena performance.
And this is the exam will test you on this as well.
So, first of all, because you pay for the amount
of data scanned per terabytes,
you need to use a type of data,
where you're going to scan less data.
And for this, you can use a columnar data type
for cost savings because you only scan the columns you need.
So therefore, the recommended formats for Amazon Athena
are going to be Apache Parquet
and
ORC.
And it's going to give you a huge performance improvement
and to get your files into the Apache Parquet
or ORC formats, you must use a service
that will see as well in the section.
For example, Glue, glue can be very helpful
to convert your data as an ETL job between,
for example, CSV
and Parquet.
Now also because we want to scan less data,
we need to compress data for smaller retrieval.
So there are different compression mechanism
you can use that I have listed it right here.
Next, if you know, you're going to query,
all the time on some specific columns,
you can partition your data sets,
and partitioning data sets means that in your S3 buckets
you're going to have the full path with slashes
and each slash will be a different column name
with a specific value.
And so you're organizing,
you're partitioning your data in Amazon S3,
so that when you query it you know exactly,
in which folder, at which path in Amazon S3,
you need to scan for data.
So here's an example of data partitions.
So we have flight data in Parquet formats
and then we do slash year equals 1991.
So we partition by year
and we'll have one folder for each year.
Then within each year, we'll have month.
So month equals one
and within each month, we'll have days.
Days equals one.
And so when I do a query on Athena
and I filter for a specific year,
a specific month
and a specific day, then it will know exactly,
to which folder in Amazon S3 to get the data from.
And therefore will only recover a subset of the data
therefore will have really, really good partitioning.
Finally, the last performance improvement
you need to do is to use bigger files,
to minimize the overhead.
So if you have many, many, many small files in Amazon S3
Athena is not going to be as performance
as if you had larger files for example, 128 megabytes
and over.
Because larger files are easier to scan
and easier to retrieve.
Okay, that's it for this lecture.
I hope you liked it.
And I will see you in the next lecture.