Instructor: So now let's talk about Amazon FSX.
So FSX is a service, a managed service on AWS,
which allows you to launch third party
high-performance file systems on AWS.
It's fully managed, and the idea is that,
maybe sometimes you don't wanna use Amazon S3,
you don't want to use EFS, you wanna use something else
and you may want to use for example Lustre,
or Windows File Server or NetApp ONTAP.
All these things are third party file systems
that you can launch directly on AWS
because they are managed by FSX.
So from an exam perspective,
you should think of FSX as a way to launch
third party high-performance file systems,
and the exam will test on FSX for Lustre
and FSX for Windows File Server.
For NetApp ONTAP it does not cover the exam.
And it is very likely in the future that FSX
will add more file systems to the FSX service.
Now, just so you know, I will only cover them in this slide
or in other slides if they appear at the exam.
So until then, this lecture and the next lectures
will only cover FSX for Lustre,
and FSX of Windows File Server.
So now let's talk about Amazon FSX for Windows
and the problem statements.
So let's explore first Amazon FSX for Windows
and this Windows File Server.
So you remember EFS, EFS was to give a shared POSIX system
for Linux machines and FSX for Windows is similar
is going to give you a fully managed
Windows file system shared drive,
accessible for your Windows computers
and also Linux computers.
So, it supports the SMB protocol as well as Windows NTFS,
and it has Microsoft Active Directory integration,
ACLs and user quotas.
So the idea that it gives you a file system that is shared
for all your Windows machines easily.
But you can also mount it on Linux EC2 instance
so it works both for Windows and Linux
but remember this is a Windows file system.
Now it has massive scale,
it can scale up to tens of gigabytes per second,
millions of IOPS and hundreds of petabytes of data.
Storage options for your FSX for Windows
are either to use an SSD and this is when you want to have,
latency sensitive workloads, think databases,
media processing data analytics,
or HDD this is a hard drive for another kind of workload
such as a home directory or a CMS and so on
but latency is less important.
SSD is of course going to be more expensive than HDD.
Now FSX for Windows can be also accessed
from your on-premises infrastructure,
using a VPN or the Direct Connect service.
And you can configure it to be Multi-AZ,
and provide you high availability.
You can also back up your data daily to Amazon S3.
Now let's talk about the second kind of Amazon FSX
which is Amazon FSX for Lustre.
And Lustre is used to do a distributed file system
that is going to be used for large-scale computing.
So once I explain you the word Lustre
is going to make sense.
So Lustre is the derived from Linux and cluster.
It is used for Machine Learning
and high performance computing or HPC,
and this is a keyword you need to look for
to know that you need FSX for Lustre.
So you can have applications such as Video Processing,
Financial Modeling, Electronic Design Automation,
you have massive scale, so you can scale
up to hundreds of gigabytes of data per second,
millions of IOPS and sub milliseconds latency.
And for storage, two options, either you want an SSD
for very low latency, IOPS intensive workload
as well as small and random file operations or HDD
if you want throughput intensive workloads,
for large and sequential file operations
and SDD is going to be more expensive than HDD.
You have seamless integration with Amazon S3,
that means that you can read S3 as a file system
through FSX, and you can write the output
of the computations from FSX back to Amazon S3,
and that is something that the exam may ask you about.
Finally, Amazon FSX for Lustre,
can be used from on-premises servers,
through VPN or Direct Connect.
For FSX you also need to know
the File System Deployment Options
and there's two, you need to know
there is scratch file system and persistent file system.
So scratch file system is going to be temporary storage
and the data will not be replicated.
That means that you have a file
and you will lose it if the underlying server fails.
But thanks to this optimization, we get really high burst
so we get six times the performance
of a persistent file system, and you get, for example,
200 megabytes per second per terabyte of throughput
so it's actually really, really big.
So the use case of a scratch file system
is going to do short term processing of data
and you want to optimize your cost
by not having data being replicated.
So that means that you have FSX, your compute instances,
are going to connect on AZ one, AZ two,
and then the FSX for Lustre scratch file system
only has one copy of your data and as is shown
on this diagram right here, just one copy.
Finally, you can also have optional S3 bucket
underlying for the data repository.
For Persistent File System,
it's going to be for long term storage,
the data is going to be replicated
within the same availability zone,
so not across AZ, but within the same AZ.
But that means that if you have a failure
of a underlying server, then the files
will be replaced transparently within minutes.
So the use case for a Persistent File System
is as the name indicates long term processing
and storage of sensitive data.
So the idea is exactly the same
in terms of the architecture
remember FSX only lives for Lustre only within one single AZ
and the FSX for Lustre file system in persistent mode
will have two copies of the data
so you can see, there is some replication right now
from one data volume to the next data volume.
So that's it, the exam will ask you scenario question
on how to choose within scratch or persistence,
so you should know enough now.
I hope you like this lecture
and I will see you in the next lecture.