1 00:00:00,000 --> 00:00:01,230 ‫Now let's talk 2 00:00:01,230 --> 00:00:04,020 ‫about Dead Letter Queues in SQS. 3 00:00:04,020 --> 00:00:05,940 ‫The idea is that let's get into the scenario 4 00:00:05,940 --> 00:00:08,580 ‫where a consumer fails to process a message 5 00:00:08,580 --> 00:00:11,370 ‫within the visibility timeout period. 6 00:00:11,370 --> 00:00:12,810 ‫Then we know that the message 7 00:00:12,810 --> 00:00:15,060 ‫automatically goes back into the queue. 8 00:00:15,060 --> 00:00:16,320 ‫So the consumer reads the message, 9 00:00:16,320 --> 00:00:17,520 ‫maybe there's a failure, 10 00:00:17,520 --> 00:00:19,050 ‫maybe we don't have enough time. 11 00:00:19,050 --> 00:00:21,300 ‫The message goes back into the queue. 12 00:00:21,300 --> 00:00:23,611 ‫Now, if that thing happens very often, 13 00:00:23,611 --> 00:00:25,170 ‫that can be a problem. 14 00:00:25,170 --> 00:00:26,330 ‫For example, we read again the message, 15 00:00:26,330 --> 00:00:28,500 ‫maybe there's something wrong with the message. 16 00:00:28,500 --> 00:00:30,270 ‫Maybe our consumer doesn't understand the message 17 00:00:30,270 --> 00:00:32,010 ‫or can't process the message. 18 00:00:32,010 --> 00:00:34,050 ‫Then the message will go back into the queue 19 00:00:34,050 --> 00:00:34,950 ‫and it will happen again. 20 00:00:34,950 --> 00:00:36,870 ‫We'll read the message again from SQS 21 00:00:36,870 --> 00:00:39,540 ‫and it will go again back into the queue. 22 00:00:39,540 --> 00:00:40,890 ‫So we can set a threshold 23 00:00:40,890 --> 00:00:43,230 ‫to how many times this can happen. 24 00:00:43,230 --> 00:00:44,063 ‫And, 25 00:00:44,063 --> 00:00:45,480 ‫this failure loop can be a big problem, 26 00:00:45,480 --> 00:00:49,320 ‫but we can set a MaximumReceives threshold. 27 00:00:49,320 --> 00:00:51,750 ‫And if that threshold is exceeded 28 00:00:51,750 --> 00:00:53,055 ‫then we can tell SQS to say, 29 00:00:53,055 --> 00:00:55,440 ‫"Well, this message looks a bit odd. 30 00:00:55,440 --> 00:00:57,900 ‫It looks like it's being processed too many times 31 00:00:57,900 --> 00:00:59,460 ‫and not successfully." 32 00:00:59,460 --> 00:01:03,570 ‫Therefore, send this into a Dead Letter Queue. 33 00:01:03,570 --> 00:01:06,300 ‫And the Dead Letter Queue will contain that message 34 00:01:06,300 --> 00:01:07,650 ‫and for later processing. 35 00:01:07,650 --> 00:01:09,570 ‫And so the message will be removed 36 00:01:09,570 --> 00:01:12,120 ‫from the first queue and sent into the second one. 37 00:01:12,120 --> 00:01:14,130 ‫Why do we have Dead Letter Queues? 38 00:01:14,130 --> 00:01:17,460 ‫Well Dead Letter Queues are very useful for debugging 39 00:01:17,460 --> 00:01:20,310 ‫and in case a message goes in Dead Letter Queue 40 00:01:20,310 --> 00:01:22,440 ‫because it is an SQS queue, you have to process it, 41 00:01:22,440 --> 00:01:23,370 ‫but at least it gives you time 42 00:01:23,370 --> 00:01:25,710 ‫to understand what is happening. 43 00:01:25,710 --> 00:01:27,540 ‫Few things to notice that the Dead Letter Queue 44 00:01:27,540 --> 00:01:30,414 ‫of a FIFO queue must also be a FIFO queue, 45 00:01:30,414 --> 00:01:33,116 ‫and the Dead Letter Queue of the Standard queue 46 00:01:33,116 --> 00:01:36,390 ‫must also be a Standard queue. 47 00:01:36,390 --> 00:01:38,490 ‫Finally, because we have a Dead Letter Queue, you need to 48 00:01:38,490 --> 00:01:41,940 ‫make sure that the messages are processed before 49 00:01:41,940 --> 00:01:42,773 ‫They expire 50 00:01:42,773 --> 00:01:43,606 ‫from the queue. 51 00:01:43,606 --> 00:01:45,180 ‫So it's a good idea to set a long time 52 00:01:45,180 --> 00:01:46,862 ‫for example, 14 days of retention 53 00:01:46,862 --> 00:01:47,695 ‫from 54 00:01:47,695 --> 00:01:49,143 ‫within the Dead Letter Queue. 55 00:01:49,980 --> 00:01:51,840 ‫And the next feature for management 56 00:01:51,840 --> 00:01:54,510 ‫of your Dead Letter Queues are the re drive to 57 00:01:54,510 --> 00:01:55,950 ‫source feature. 58 00:01:55,950 --> 00:01:58,740 ‫So it's a feature to help you consume messages in 59 00:01:58,740 --> 00:02:02,100 ‫the Dead Letter Queue to understand what is wrong with them. 60 00:02:02,100 --> 00:02:03,480 ‫So you have your messages 61 00:02:03,480 --> 00:02:06,413 ‫now you know they haven't been processed in the source queue 62 00:02:06,413 --> 00:02:08,912 ‫and therefore they are in the Dead Letter Queue 63 00:02:08,912 --> 00:02:11,610 ‫and you're going to do a manual inspection 64 00:02:11,610 --> 00:02:13,980 ‫and debugging of these messages. 65 00:02:13,980 --> 00:02:16,500 ‫And then you're going to fix your consumer code 66 00:02:16,500 --> 00:02:19,260 ‫understand why the message wasn't processed 67 00:02:19,260 --> 00:02:21,900 ‫in case the message was correct. 68 00:02:21,900 --> 00:02:25,110 ‫And then what you can do is re drive that message 69 00:02:25,110 --> 00:02:29,453 ‫from the dead letter queue into the source SQS queue 70 00:02:29,453 --> 00:02:31,710 ‫And what's going to happen with this is 71 00:02:31,710 --> 00:02:35,160 ‫that the consumer can now reprocess that message 72 00:02:35,160 --> 00:02:37,410 ‫without even knowing that the message went 73 00:02:37,410 --> 00:02:38,760 ‫into the Dead Letter Queue 74 00:02:38,760 --> 00:02:41,460 ‫and then the message processing has happened 75 00:02:41,460 --> 00:02:42,870 ‫and as a cool feature. 76 00:02:42,870 --> 00:02:44,520 ‫So now let us go into the console 77 00:02:44,520 --> 00:02:47,370 ‫so I can show you the Dead Letter Queue feature.