So now let's talk about
the Amazon OpenSearch Service.
And Amazon OpenSearch is a successor
to something you may have heard before
called Amazon ElasticSearch.
So the name change was due to some licensing issues.
So in DynamoDB, just to do a comparison,
you can only query the data by primary key,
or if you have indexes on your database.
But with OpenSearch,
you can actually, as the name indicate,
search any fields, even for partial matches.
So it's very common to use OpenSearch
to provide search to your application.
And so you would use OpenSearch
as a compliment to another database.
So OpenSearch can be used for search
but also, as the name doesn't indicate,
you can also do analytic queries on top of OpenSearch.
So to create OpenSearch and to use it,
you need to create a cluster of instances.
So it's not a serverless service.
And OpenSearch has its own query language.
It does not support SQL.
So you can ingest data from different places,
such as Kinesis Data Firehose,
IoT, CloudWatch Logs,
or any of your custom-built application.
You have security
provided through integration with Cognito, IAM.
You get at-risk encryption and in-flight encryption.
And as I said, you can do analytics
on top of the OpenSearch service.
So you can use something called OpenSearch Dashboards
to create visualizations on top of your OpenSearch data.
So here are some common patterns to use OpenSearch.
So you would have DynamoDB and it will contain your data.
This is where your users will insert,
and delete, and update data.
And then you send all the streams in a DynamoDB stream,
which is then picked up by Lambda Function.
And that Lambda Function will insert the data
into Amazon OpenSearch in real time.
And through this process,
what happens that your application
now has the ability to search for a specific item,
for example, to do a partial search with the item name
and then find the item ID out of it.
And then once the item ID is obtained,
then it will call DynamoDB
to actually retrieve the full item
from your DynamoDB table.
So that's a common pattern
in which OpenSearch provides the search capability.
Whereas your main source of data
still remains your DynamoDB table.
There's also other ways.
So you can ingest CloudWatch Logs into OpenSearch.
So the first one is to use
what's called a CloudWatch Log subscription filter,
sending data in real time to Lambda Function
that is managed by AWS.
And then the Lambda Function
in real time sends all the data into Amazon OpenSearch.
Or you can also use a CloudWatch Logs
and then Subscription Filter.
But this time, Kinesis Data Firehose
can read from the subscription filter.
And then near real time,
because this is Data Firehose,
data will be inserted in Amazon OpenSearch.
Other pattern around Kinesis.
So to send Kinesis data streams into Amazon OpenSearch,
you have two strategies.
The first one is to use Kinesis Data Firehose.
This is a near real time, again, type of service.
You can optionally do some data transformation
using a Lambda Function
and then send data into Amazon OpenSearch.
Or you can use Kinesis Data Streams again.
But this time you would create a Lambda Function
that would read the data stream in real time,
and then you would write custom code
to have the Lambda Function write
to Amazon OpenSearch in real time.
So all these patterns are valid.
And now, you know pretty much all the possible architectures
for using Amazon OpenSearch.
So that's it for this lecture.
I hope you liked it.
And I will see you in the next lecture.