1 00:00:00,570 --> 00:00:03,030 ‫Okay, so now let's go one step deeper 2 00:00:03,030 --> 00:00:06,170 ‫and talk about the different caching strategies 3 00:00:06,170 --> 00:00:09,110 ‫you can implement and the considerations you have 4 00:00:09,110 --> 00:00:10,130 ‫coming with it. 5 00:00:10,130 --> 00:00:12,690 ‫So if you want to have a read, I recommend this link. 6 00:00:12,690 --> 00:00:14,340 ‫All the information is from this link 7 00:00:14,340 --> 00:00:15,710 ‫with a little bit of bonus, 8 00:00:15,710 --> 00:00:17,890 ‫but I'm going to try to explain in summarize everything 9 00:00:17,890 --> 00:00:19,960 ‫to you in this lecture. 10 00:00:19,960 --> 00:00:22,340 ‫So, is it safe to cache data? 11 00:00:22,340 --> 00:00:25,970 ‫In general yes, but sometimes your data may be out of date. 12 00:00:25,970 --> 00:00:27,800 ‫And so you may have eventually consistency. 13 00:00:27,800 --> 00:00:30,840 ‫So it's not for every type of data sets, Okay. 14 00:00:30,840 --> 00:00:34,150 ‫Some data, you want to make sure that you only cache it, 15 00:00:34,150 --> 00:00:36,320 ‫if it's okay to cache it. 16 00:00:36,320 --> 00:00:38,170 ‫Is caching effective for the data? 17 00:00:38,170 --> 00:00:40,120 ‫Is also another question you need to ask yourself. 18 00:00:40,120 --> 00:00:43,360 ‫So for example, if your data is changing slowly, 19 00:00:43,360 --> 00:00:45,720 ‫and few keys are frequently needed, 20 00:00:45,720 --> 00:00:47,300 ‫then that could be great, Okay. 21 00:00:47,300 --> 00:00:49,000 ‫But for an anti pattern for example, 22 00:00:49,000 --> 00:00:51,430 ‫if your data is changing very, very quickly, 23 00:00:51,430 --> 00:00:54,750 ‫and you need all the key space in your data set, 24 00:00:54,750 --> 00:00:57,510 ‫then maybe caching will not be as effective. 25 00:00:57,510 --> 00:00:58,870 ‫Then you need to ask yourself the question 26 00:00:58,870 --> 00:01:02,100 ‫is the data structured correctly for caching? 27 00:01:02,100 --> 00:01:04,100 ‫For example, if you have key value, 28 00:01:04,100 --> 00:01:07,290 ‫or if you want to store your aggregations results, 29 00:01:07,290 --> 00:01:08,600 ‫caching is great. 30 00:01:08,600 --> 00:01:10,490 ‫But you need to make sure that data 31 00:01:10,490 --> 00:01:13,527 ‫is structured accordingly before caching it, 32 00:01:13,527 --> 00:01:15,840 ‫Okay, because caching is about saving time, 33 00:01:15,840 --> 00:01:16,880 ‫it's about optimizing. 34 00:01:16,880 --> 00:01:18,510 ‫It's about getting there faster. 35 00:01:18,510 --> 00:01:20,290 ‫So you wanna make sure the data is structured 36 00:01:20,290 --> 00:01:22,370 ‫the correct way for your queries, Okay? 37 00:01:22,370 --> 00:01:24,130 ‫And then the final question, 38 00:01:24,130 --> 00:01:25,590 ‫which is, the more important one is, 39 00:01:25,590 --> 00:01:27,460 ‫which caching design pattern 40 00:01:27,460 --> 00:01:30,390 ‫is going to be the most appropriate for us? 41 00:01:30,390 --> 00:01:33,140 ‫And this is the discussion we're going into right now. 42 00:01:33,140 --> 00:01:36,650 ‫So there is the first one called Lazy Loading, 43 00:01:36,650 --> 00:01:38,800 ‫but it may come up at the exam 44 00:01:38,800 --> 00:01:42,147 ‫under Cache-Aside or Lazy Population. 45 00:01:42,147 --> 00:01:44,430 ‫The three things mean the same thing. 46 00:01:44,430 --> 00:01:45,830 ‫So let's have a look. 47 00:01:45,830 --> 00:01:47,340 ‫So we have our application. 48 00:01:47,340 --> 00:01:50,410 ‫We have Amazon ElastiCache and Amazon RDS, 49 00:01:50,410 --> 00:01:52,310 ‫and there are three distinct components. 50 00:01:52,310 --> 00:01:56,660 ‫So if your application goes and wants to read something, 51 00:01:56,660 --> 00:01:58,860 ‫first, it's going to ask the cache, 52 00:01:58,860 --> 00:02:02,920 ‫and so ElastiCache it would be Redis or Memcached, 53 00:02:02,920 --> 00:02:04,250 ‫would say, yes, I have something 54 00:02:04,250 --> 00:02:06,380 ‫and it's called a Cache hit. 55 00:02:06,380 --> 00:02:07,790 ‫That's the greatest case. 56 00:02:07,790 --> 00:02:10,690 ‫Now, if there is no Cache hit, it's called a Cache miss. 57 00:02:10,690 --> 00:02:13,980 ‫So the application makes a request to ElastiCache. 58 00:02:13,980 --> 00:02:15,650 ‫ElastiCache does not have the data, 59 00:02:15,650 --> 00:02:16,870 ‫so we get a Cache miss. 60 00:02:16,870 --> 00:02:20,230 ‫So we need to be able to find the data where is. 61 00:02:20,230 --> 00:02:23,380 ‫So we go and read the data from your database, 62 00:02:23,380 --> 00:02:25,780 ‫in this example, Amazon RDS, 63 00:02:25,780 --> 00:02:28,540 ‫then the application has the correct data. 64 00:02:28,540 --> 00:02:30,880 ‫And what it's going to do is that it's going to write 65 00:02:30,880 --> 00:02:32,980 ‫that data to the cache to make sure 66 00:02:32,980 --> 00:02:35,470 ‫that any other application who requests 67 00:02:35,470 --> 00:02:39,220 ‫the same data will go directly into Cache hit. 68 00:02:39,220 --> 00:02:41,350 ‫So this architecture is great, 69 00:02:41,350 --> 00:02:45,740 ‫because only the requested data is going to be cached, Okay? 70 00:02:45,740 --> 00:02:48,430 ‫If there is no data that is requested from RDS, 71 00:02:48,430 --> 00:02:49,900 ‫it will not end up in the cache. 72 00:02:49,900 --> 00:02:52,060 ‫So it's very efficient. 73 00:02:52,060 --> 00:02:55,210 ‫And in case, somehow your cache gets wiped 74 00:02:55,210 --> 00:02:57,840 ‫or you have a node failure, then it's not fatal. 75 00:02:57,840 --> 00:02:59,700 ‫It will increase latency because the case needs 76 00:02:59,700 --> 00:03:00,533 ‫to be one warmed 77 00:03:00,533 --> 00:03:04,400 ‫and by warm, it means that all the reads have to go to RDS 78 00:03:04,400 --> 00:03:05,233 ‫and then be cached. 79 00:03:05,233 --> 00:03:07,300 ‫So that's what the warm up is called. 80 00:03:07,300 --> 00:03:08,850 ‫But there's some cons. 81 00:03:08,850 --> 00:03:11,880 ‫First of all, if we get a Cache hit is great. 82 00:03:11,880 --> 00:03:13,320 ‫But if we get a Cache miss, 83 00:03:13,320 --> 00:03:16,920 ‫that means there's three network calls being done 84 00:03:16,920 --> 00:03:18,620 ‫from your application to ElastiCache, 85 00:03:18,620 --> 00:03:21,330 ‫which is a Cache miss from your application to RDS, 86 00:03:21,330 --> 00:03:23,020 ‫which is read from Data Base and finally, 87 00:03:23,020 --> 00:03:24,520 ‫another write to the cache. 88 00:03:24,520 --> 00:03:25,970 ‫And so for your users, 89 00:03:25,970 --> 00:03:28,240 ‫when they are reading something, 90 00:03:28,240 --> 00:03:29,960 ‫and they're seeing some latency, 91 00:03:29,960 --> 00:03:31,080 ‫they're not used to it. 92 00:03:31,080 --> 00:03:35,880 ‫And so, that three round trips may be a bad user experience. 93 00:03:35,880 --> 00:03:37,760 ‫And finally, stale data. 94 00:03:37,760 --> 00:03:40,880 ‫So if your data is updated in RDS, 95 00:03:40,880 --> 00:03:44,000 ‫then it will not be necessarily updating in ElastiCache. 96 00:03:44,000 --> 00:03:47,760 ‫And so it's possible to have outdated data in the cache. 97 00:03:47,760 --> 00:03:51,430 ‫And that's what you need to ask yourself is my data Okay 98 00:03:51,430 --> 00:03:54,300 ‫to be out of date and eventually consistent? 99 00:03:54,300 --> 00:03:56,170 ‫So that's the first case, 100 00:03:56,170 --> 00:03:59,240 ‫and so the exam does expect you to understand how 101 00:03:59,240 --> 00:04:04,150 ‫to read a Cache-Aside or Lazy Loading type of structure. 102 00:04:04,150 --> 00:04:05,900 ‫So I'm using Python, which is I think, 103 00:04:05,900 --> 00:04:09,000 ‫an easy language to read, to have a look. 104 00:04:09,000 --> 00:04:12,100 ‫So we need to look at this code and understand 105 00:04:12,100 --> 00:04:15,410 ‫that it is indeed a Lazy Loading strategy. 106 00:04:15,410 --> 00:04:18,830 ‫So let's have a look, and this is very easy to read. 107 00:04:18,830 --> 00:04:20,130 ‫So def means definition 108 00:04:20,130 --> 00:04:23,200 ‫and it's a function definition to get a user 109 00:04:23,200 --> 00:04:25,420 ‫and the argument is the user ID. 110 00:04:25,420 --> 00:04:26,990 ‫And so we call it on line 17. 111 00:04:26,990 --> 00:04:30,580 ‫We want the user equals get_user id 17, Okay. 112 00:04:30,580 --> 00:04:32,970 ‫And now how does the user get his get? 113 00:04:32,970 --> 00:04:36,920 ‫So first, would you record equals cache.get user_id. 114 00:04:36,920 --> 00:04:39,660 ‫So here, we're going to check in ElastiCache, 115 00:04:39,660 --> 00:04:42,900 ‫if somehow that user ID is already cached. 116 00:04:42,900 --> 00:04:46,550 ‫And so if we get a Cache miss, if record is none, 117 00:04:46,550 --> 00:04:49,470 ‫we go in this loop, else, that'd be a Cache hit, 118 00:04:49,470 --> 00:04:51,030 ‫and we just return the record. 119 00:04:51,030 --> 00:04:52,900 ‫So we see that if we get a Cache hit, 120 00:04:52,900 --> 00:04:55,890 ‫here, we go directly into the response. 121 00:04:55,890 --> 00:04:57,480 ‫Now if you get a Cache miss, 122 00:04:57,480 --> 00:05:00,430 ‫then we run a database query to find that user 123 00:05:00,430 --> 00:05:01,263 ‫in our database, 124 00:05:01,263 --> 00:05:03,020 ‫so we do db.query 125 00:05:03,020 --> 00:05:06,510 ‫and we have a SQL query that goes past to RDS and say, 126 00:05:06,510 --> 00:05:08,070 ‫Okay, we get the record, 127 00:05:08,070 --> 00:05:10,590 ‫then we populate the cache with the results. 128 00:05:10,590 --> 00:05:13,750 ‫So we'll do cache.set user_id, record 129 00:05:13,750 --> 00:05:16,600 ‫so that the next time someone does cache.get 130 00:05:16,600 --> 00:05:18,470 ‫it will be a result, it will be a Cache hit. 131 00:05:18,470 --> 00:05:20,430 ‫And then finally, we return the record, 132 00:05:20,430 --> 00:05:22,250 ‫which is exactly the same strategy 133 00:05:22,250 --> 00:05:24,040 ‫as what I showed you in the diagram. 134 00:05:24,040 --> 00:05:26,860 ‫So from this code, it appears to be indeed, 135 00:05:26,860 --> 00:05:28,950 ‫Lazy Loading, Okay. 136 00:05:28,950 --> 00:05:31,070 ‫Again, you don't need to know how to write code, 137 00:05:31,070 --> 00:05:33,460 ‫but reading code and understand what the code does. 138 00:05:33,460 --> 00:05:34,810 ‫It's very little of the exam, 139 00:05:34,810 --> 00:05:38,230 ‫but for ElastiCache, it is necessary, Okay. 140 00:05:38,230 --> 00:05:43,050 ‫The second type of strategy is called Write Through. 141 00:05:43,050 --> 00:05:46,430 ‫And Write Trough, is when you add or update the cache 142 00:05:46,430 --> 00:05:49,280 ‫when your database is updated. 143 00:05:49,280 --> 00:05:50,113 ‫So let's have a look. 144 00:05:50,113 --> 00:05:53,140 ‫Same thing application ElastiCache and RDS. 145 00:05:53,140 --> 00:05:55,080 ‫And when our application talks to ElastiCache 146 00:05:55,080 --> 00:05:57,980 ‫we get a Cache hit which is great, 147 00:05:57,980 --> 00:06:01,290 ‫and when there is a write happening to RDS, 148 00:06:01,290 --> 00:06:06,140 ‫so when our application modifies the Amazon RDS database, 149 00:06:06,140 --> 00:06:09,590 ‫then on top of it, it's going to write to the cache. 150 00:06:09,590 --> 00:06:11,160 ‫So this is why it's called the Write Through, 151 00:06:11,160 --> 00:06:16,160 ‫because we write through ElastiCache to RDS, Okay. 152 00:06:16,270 --> 00:06:19,380 ‫So what do we get out of this architecture? 153 00:06:19,380 --> 00:06:23,000 ‫Well, the data in the cache is never stale, Okay. 154 00:06:23,000 --> 00:06:25,680 ‫Whenever there is a change in Amazon RDS, 155 00:06:25,680 --> 00:06:29,123 ‫then automatically, there will be a change in your cache. 156 00:06:30,010 --> 00:06:31,540 ‫And this time, 157 00:06:31,540 --> 00:06:34,310 ‫you get a Write penalty versus a Read penalty. 158 00:06:34,310 --> 00:06:36,360 ‫So before when we had a Cache miss, 159 00:06:36,360 --> 00:06:38,900 ‫we had to do three network calls in total, 160 00:06:38,900 --> 00:06:41,090 ‫and that was a Read penalty. 161 00:06:41,090 --> 00:06:43,680 ‫But now when we write to the database, 162 00:06:43,680 --> 00:06:45,280 ‫we have a Write penalty. 163 00:06:45,280 --> 00:06:47,690 ‫Each write now requires two calls 164 00:06:47,690 --> 00:06:50,060 ‫one to the database and one to the cache. 165 00:06:50,060 --> 00:06:51,900 ‫And from the user perspective, 166 00:06:51,900 --> 00:06:56,610 ‫a user is expecting more a write to take longer than a read. 167 00:06:56,610 --> 00:06:58,590 ‫For example, if you were on the social media 168 00:06:58,590 --> 00:07:01,630 ‫and you post something then you expect that post 169 00:07:01,630 --> 00:07:03,250 ‫to last maybe a second or two. 170 00:07:03,250 --> 00:07:04,920 ‫But if you fetch a profile, 171 00:07:04,920 --> 00:07:07,040 ‫you expected to take a split second. 172 00:07:07,040 --> 00:07:09,870 ‫So here, the users understand that any write, 173 00:07:09,870 --> 00:07:12,800 ‫any changes to the database may take a little bit longer. 174 00:07:12,800 --> 00:07:15,780 ‫So there may be better from a user perspective. 175 00:07:15,780 --> 00:07:17,300 ‫Now, what about the cons? 176 00:07:17,300 --> 00:07:19,000 ‫Well, there is missing data 177 00:07:19,000 --> 00:07:24,000 ‫until the Amazon RDS the database is updated or added. 178 00:07:24,400 --> 00:07:27,180 ‫That means that your ElastiCache or your cache in general, 179 00:07:27,180 --> 00:07:29,890 ‫will not have all the data it needs 180 00:07:29,890 --> 00:07:31,890 ‫until the data is written to RDS 181 00:07:31,890 --> 00:07:33,250 ‫and so there may be a problem. 182 00:07:33,250 --> 00:07:36,900 ‫And so for this, you can combine the Write Through strategy 183 00:07:36,900 --> 00:07:39,750 ‫with Lazy Loading such as if your application 184 00:07:39,750 --> 00:07:42,170 ‫does not find the data in ElastiCache 185 00:07:42,170 --> 00:07:45,550 ‫then it also does Lazy Loading into RDS. 186 00:07:45,550 --> 00:07:48,700 ‫And so we can combine the two strategies together. 187 00:07:48,700 --> 00:07:50,720 ‫And here we get a Cache Churn. 188 00:07:50,720 --> 00:07:54,320 ‫That means that as we add a lot and a lot of data in RDS, 189 00:07:54,320 --> 00:07:56,860 ‫there will be a lot and a lot of data in ElastiCache, 190 00:07:56,860 --> 00:07:59,460 ‫but there is a chance that this data will never be read. 191 00:07:59,460 --> 00:08:00,700 ‫So that could be a problem 192 00:08:00,700 --> 00:08:03,880 ‫if your cache is very small, Okay. 193 00:08:03,880 --> 00:08:06,100 ‫Likewise, we'll look at some Pseudocode. 194 00:08:06,100 --> 00:08:09,610 ‫So this time, we have the function called save_user 195 00:08:09,610 --> 00:08:11,090 ‫so before it was get_user 196 00:08:11,090 --> 00:08:13,400 ‫because it was a read optimization on the cache, 197 00:08:13,400 --> 00:08:15,560 ‫but now it's a write optimization on the cache. 198 00:08:15,560 --> 00:08:17,320 ‫So it's save_user 199 00:08:17,320 --> 00:08:20,670 ‫and in here, we just look at this function save_user 200 00:08:20,670 --> 00:08:23,090 ‫and the first thing we do is to save to the database. 201 00:08:23,090 --> 00:08:26,130 ‫So we do record equals db.query update users 202 00:08:26,130 --> 00:08:28,470 ‫and we update the user with the values. 203 00:08:28,470 --> 00:08:30,260 ‫And then we push it into the cache. 204 00:08:30,260 --> 00:08:33,250 ‫So cache.set user_id, record 205 00:08:33,250 --> 00:08:34,750 ‫and then we return the record. 206 00:08:34,750 --> 00:08:36,690 ‫So this is the Write Through strategy 207 00:08:36,690 --> 00:08:39,240 ‫that's been defined and Pseudocode. 208 00:08:39,240 --> 00:08:41,020 ‫And you can as you see, 209 00:08:41,020 --> 00:08:43,700 ‫combine the Write Through and the Lazy Loading 210 00:08:43,700 --> 00:08:45,920 ‫because this is a function called save_user 211 00:08:45,920 --> 00:08:47,840 ‫and the other one was a function called get_user 212 00:08:47,840 --> 00:08:50,200 ‫so these two can be used together. 213 00:08:50,200 --> 00:08:53,820 ‫Then there is Cache Evictions and Time-to-live or TTL. 214 00:08:53,820 --> 00:08:56,120 ‫So your cache has a limited size. 215 00:08:56,120 --> 00:08:58,720 ‫And so there can be something called Cache Eviction, 216 00:08:58,720 --> 00:09:00,690 ‫to remove data out of your cache. 217 00:09:00,690 --> 00:09:03,290 ‫So for example, it can happen if you delete an item 218 00:09:03,290 --> 00:09:05,270 ‫explicitly from the cache, 219 00:09:05,270 --> 00:09:07,940 ‫or if the memory of the cache is full. 220 00:09:07,940 --> 00:09:10,340 ‫And then your item has not been used recently, 221 00:09:10,340 --> 00:09:11,590 ‫and therefore is going to be evicted, 222 00:09:11,590 --> 00:09:15,320 ‫it's called LRU for Least Recently Used, 223 00:09:15,320 --> 00:09:18,890 ‫or we set an item time-to-live or TTL, 224 00:09:18,890 --> 00:09:22,790 ‫which is saying, hey, you can live only for five minutes, 225 00:09:22,790 --> 00:09:25,900 ‫and in five minutes, I want the cache to evict you. 226 00:09:25,900 --> 00:09:29,400 ‫And the TTL are very helpful for really any kind of data 227 00:09:29,400 --> 00:09:33,330 ‫would it be Leaderboards, Comments, Activity streams etc. 228 00:09:33,330 --> 00:09:35,540 ‫And depending on your application, 229 00:09:35,540 --> 00:09:37,930 ‫your TTL can range from a few seconds, 230 00:09:37,930 --> 00:09:41,410 ‫so this is a very short TTL, to hours or days. 231 00:09:41,410 --> 00:09:45,560 ‫But even a TTL of a few seconds for some data that is very, 232 00:09:45,560 --> 00:09:48,680 ‫very requested can be extremely effective for a cache. 233 00:09:48,680 --> 00:09:52,760 ‫So TTL is a great strategy to keep a balance between 234 00:09:52,760 --> 00:09:55,730 ‫having data stay in the cache and also being evicted, 235 00:09:55,730 --> 00:09:59,040 ‫so that new data takes place and replaces it. 236 00:09:59,040 --> 00:10:02,030 ‫So if somehow you have too many evictions though, 237 00:10:02,030 --> 00:10:05,160 ‫because your case is always at full memory, 238 00:10:05,160 --> 00:10:08,010 ‫then you should consider updating your cache size 239 00:10:08,010 --> 00:10:09,360 ‫by scaling up or out. 240 00:10:09,360 --> 00:10:11,670 ‫So that means making your cache bigger. 241 00:10:11,670 --> 00:10:14,570 ‫Okay, final words of wisdom because caching is hard. 242 00:10:14,570 --> 00:10:17,290 ‫So the Lazy Loading and Cache-Aside strategy 243 00:10:17,290 --> 00:10:18,687 ‫is very easy to implement 244 00:10:18,687 --> 00:10:22,090 ‫and works with many situations as a foundation, 245 00:10:22,090 --> 00:10:23,970 ‫especially to improve the read performance. 246 00:10:23,970 --> 00:10:25,940 ‫So this is something that's very easy to do, 247 00:10:25,940 --> 00:10:29,750 ‫and that I recommend you to do in many applications. 248 00:10:29,750 --> 00:10:32,800 ‫The Write-through is a bit more hands on involved, 249 00:10:32,800 --> 00:10:34,940 ‫and it's more of an optimization 250 00:10:34,940 --> 00:10:38,230 ‫that comes in as an after effect on top of the Lazy Loading, 251 00:10:38,230 --> 00:10:40,000 ‫then a cache strategy on its own. 252 00:10:40,000 --> 00:10:42,960 ‫So make sure that Write-through is not your first priority. 253 00:10:42,960 --> 00:10:44,500 ‫Think out how it works 254 00:10:44,500 --> 00:10:47,600 ‫for your use case and implement it if necessary, 255 00:10:47,600 --> 00:10:50,553 ‫to improve your cache staleness. 256 00:10:51,770 --> 00:10:55,830 ‫And then finally, the TTL is usually not a bad idea, 257 00:10:55,830 --> 00:10:58,480 ‫except when you're using Write-through, Okay. 258 00:10:58,480 --> 00:11:00,330 ‫And you should set it to a sensible value 259 00:11:00,330 --> 00:11:01,950 ‫for your application. 260 00:11:01,950 --> 00:11:04,970 ‫Finally, also only cache the data make sense. 261 00:11:04,970 --> 00:11:08,800 ‫So it'd be user profiles, blogs, but maybe not pricing data. 262 00:11:08,800 --> 00:11:12,170 ‫And finally, or maybe not someone's bank account value. 263 00:11:12,170 --> 00:11:14,530 ‫And then finally, just one last thing on cache. 264 00:11:14,530 --> 00:11:15,977 ‫So there's this quote that says, 265 00:11:15,977 --> 00:11:18,347 ‫"There are two hard things in Computer science: 266 00:11:18,347 --> 00:11:20,410 ‫"cache invalidation, and naming things." 267 00:11:20,410 --> 00:11:22,210 ‫So that means that caching is really, really hard. 268 00:11:22,210 --> 00:11:23,420 ‫This was just an introduction 269 00:11:23,420 --> 00:11:26,550 ‫there's a whole field of Computer Science on caching. 270 00:11:26,550 --> 00:11:29,140 ‫But as you can see, for the exam, 271 00:11:29,140 --> 00:11:31,760 ‫you need to know about different caching strategies, 272 00:11:31,760 --> 00:11:33,700 ‫their Pseudocode and their implications. 273 00:11:33,700 --> 00:11:35,460 ‫So I hope that was helpful 274 00:11:35,460 --> 00:11:37,410 ‫and I will see you in the next lecture.