1 00:00:00,210 --> 00:00:02,100 So here we are, you beautiful people. 2 00:00:02,100 --> 00:00:06,600 You're about to learn the crown jewel of the Linux operating system piping. 3 00:00:06,600 --> 00:00:11,310 So in the last few videos you've learned that you can use redirection to change the location of where 4 00:00:11,310 --> 00:00:15,110 you send command output and where you get command input. 5 00:00:15,120 --> 00:00:20,430 But what if you wanted to connect the standard output of one command so that it flowed into the standard 6 00:00:20,430 --> 00:00:21,990 input of another command? 7 00:00:22,140 --> 00:00:25,950 Well, that's where piping comes in and it's ridiculously powerful. 8 00:00:26,040 --> 00:00:29,820 Each Linux command is designed to do one task extremely well. 9 00:00:29,820 --> 00:00:35,130 So if you can continually pipe these highly specialized commands together and pass data between them, 10 00:00:35,130 --> 00:00:39,330 you can build advanced pipelines to do pretty much any task that you can think of. 11 00:00:39,330 --> 00:00:42,750 And once you know how to do it, it will really feel like a super power. 12 00:00:42,750 --> 00:00:46,920 So in this video, you're going to learn how to build your very own command pipelines. 13 00:00:46,920 --> 00:00:51,510 And then in the next video in part two, you're going to learn how to make your pipelines more flexible 14 00:00:51,510 --> 00:00:54,090 by using the T and the X args commands. 15 00:00:54,090 --> 00:00:58,770 This is going to be a great section of the course and you will not recognize yourself after you learn 16 00:00:58,770 --> 00:00:59,640 what I'm about to teach you. 17 00:00:59,640 --> 00:01:00,720 So I hope you're excited. 18 00:01:00,720 --> 00:01:02,100 Let's go ahead and get into it. 19 00:01:03,300 --> 00:01:03,660 Okay. 20 00:01:03,660 --> 00:01:08,910 So piping is all about taking the standard output of one command and connecting it to the standard input 21 00:01:08,910 --> 00:01:10,000 of another command. 22 00:01:10,020 --> 00:01:12,160 So let's take, for example, the date command. 23 00:01:12,180 --> 00:01:14,640 The date command, as we know, will show us the date. 24 00:01:14,670 --> 00:01:21,180 Now we can write the standard output of the date command to a text file called Date Dot text using redirection 25 00:01:21,180 --> 00:01:22,560 so as a mini challenge. 26 00:01:22,590 --> 00:01:24,780 See, see if you can remember how to do that. 27 00:01:24,810 --> 00:01:30,820 How would you take the date command and redirect it into a file called date txt. 28 00:01:30,930 --> 00:01:33,240 Give it a go and then come back to see how you did. 29 00:01:39,000 --> 00:01:39,840 Did you manage it? 30 00:01:39,870 --> 00:01:41,370 Well, this is one way you could do it. 31 00:01:41,370 --> 00:01:47,700 You could type the date command and then say, I want to redirect the standard output into date txt. 32 00:01:47,760 --> 00:01:51,390 Now you could have just had the symbol like this, or you could have put the number one to indicate 33 00:01:51,390 --> 00:01:56,450 that you wanted the the first data stream, which is standard output to be written to data. 34 00:01:57,060 --> 00:02:00,870 And if you did that massive congratulations, you're learning already. 35 00:02:00,870 --> 00:02:05,970 But now that we've done it, we are going to take a look at our file explorer here and we see a file 36 00:02:05,970 --> 00:02:11,700 called Data txt and we see that the data information has indeed been written into that file. 37 00:02:12,240 --> 00:02:17,280 But now what if we wanted to just cut out the day of the week, which in this case is Monday? 38 00:02:17,700 --> 00:02:19,860 How would we cut out just the day of the week? 39 00:02:19,890 --> 00:02:23,190 Well, one way to do it would be to use the cut command. 40 00:02:23,340 --> 00:02:27,930 Now, you haven't seen the command before, but it works very simply, and it's probably easier to show 41 00:02:27,930 --> 00:02:28,780 you by doing. 42 00:02:28,950 --> 00:02:35,250 First we want to read the date txt file into the standard input for the cut command. 43 00:02:35,250 --> 00:02:41,010 So to do that we would type cut and then read into the standard input date. 44 00:02:41,250 --> 00:02:42,810 TXT okay. 45 00:02:42,870 --> 00:02:47,160 And you don't have to have the zero, so you could just get rid of the zero and just just do that, 46 00:02:47,160 --> 00:02:51,870 have the lesson sign and then data and remember this space here, this is optional. 47 00:02:51,940 --> 00:02:54,120 Okay, I'll probably just keep it because it looks a bit nicer. 48 00:02:54,330 --> 00:02:59,910 Now what will do what the command does is it will cut up a file and give you specific columns. 49 00:02:59,910 --> 00:03:06,960 So if we look at data txt, you can imagine that the file is broken up into columns with a space between 50 00:03:06,960 --> 00:03:08,490 each column the day of the week. 51 00:03:08,490 --> 00:03:13,880 In this case, Monday is the first column or the first field, as it's known in the command. 52 00:03:13,890 --> 00:03:17,340 This would be the second field, this would be the third field and so on and so on. 53 00:03:17,340 --> 00:03:22,230 But we want the first field now to make the command work correctly for us. 54 00:03:22,230 --> 00:03:25,230 We need to give it two pieces of information using options. 55 00:03:25,350 --> 00:03:29,430 First, we need to give it the delimiter long form option. 56 00:03:29,430 --> 00:03:36,660 So delimiter and what what a delimiter is, is it tells you what divides the columns. 57 00:03:37,110 --> 00:03:43,920 So we need to tell it that spaces are what divide or delimit the columns because spaces are what divide 58 00:03:43,920 --> 00:03:45,990 up these specific columns in the file. 59 00:03:46,410 --> 00:03:54,000 So to do that, we just type delimiter as a long form, put a space and then in quotes put another space. 60 00:03:54,180 --> 00:03:58,410 So we're saying a space is what limits this file or this input. 61 00:03:59,160 --> 00:04:03,180 Then we need to say which field we want now that we've told it, how the file is cut up. 62 00:04:03,180 --> 00:04:08,580 So we give it again the long form option, which is fields and then a space and then one. 63 00:04:09,600 --> 00:04:15,660 And now what we've got now when we run that, what we're going to see is we're saying to the command, 64 00:04:15,930 --> 00:04:22,440 okay, read data txt it's broken up into columns with us, each with a space separating each column 65 00:04:22,440 --> 00:04:23,700 and we want the first field. 66 00:04:23,700 --> 00:04:27,390 Now when I press enter, you can see that we get Monday as an output. 67 00:04:27,420 --> 00:04:28,140 Isn't that awesome? 68 00:04:28,140 --> 00:04:28,290 Right. 69 00:04:28,290 --> 00:04:28,890 That's pretty cool. 70 00:04:28,890 --> 00:04:31,620 You've not seen the command before, but it's a very cool command. 71 00:04:32,460 --> 00:04:35,790 But this this works, but it's kind of clunky. 72 00:04:35,790 --> 00:04:36,100 Okay. 73 00:04:36,210 --> 00:04:41,310 First, we're writing the standard output of the date command to a file which takes up space on our 74 00:04:41,310 --> 00:04:41,940 computer. 75 00:04:41,940 --> 00:04:42,370 Right. 76 00:04:42,390 --> 00:04:45,510 Then we have to read that file into the command. 77 00:04:46,050 --> 00:04:51,300 And maybe we would also, after getting this data out, we then have to delete this original file because 78 00:04:51,300 --> 00:04:54,360 we don't need it anymore, which is another process that we'd have to undertake. 79 00:04:54,360 --> 00:04:59,460 So doing it this way by saving it the data into a file and then reading that file in two steps takes 80 00:04:59,460 --> 00:05:00,450 a lot more typing. 81 00:05:00,810 --> 00:05:04,920 It makes unnecessary files, which then you've got to remove, or they just take up space and it's just 82 00:05:04,920 --> 00:05:07,230 an inefficient and awkward way of doing things. 83 00:05:07,380 --> 00:05:14,190 So instead we can pipe the standard output of the data command directly into the standard input of the 84 00:05:14,400 --> 00:05:14,910 command. 85 00:05:14,910 --> 00:05:15,210 Okay. 86 00:05:15,300 --> 00:05:21,900 So if I just use the up arrow key to keep what we had before, now instead of saying read into the read 87 00:05:21,900 --> 00:05:24,010 into the standard input of the command data. 88 00:05:24,060 --> 00:05:30,330 XRT What I'm going to do is I'm going to type date and then use this pipe symbol. 89 00:05:30,330 --> 00:05:37,440 Now this pipe symbol you can find by holding shift and then backslash on your, on your most keyboards 90 00:05:37,440 --> 00:05:37,920 anyway. 91 00:05:37,920 --> 00:05:43,110 So yeah, that's the way it is on most keyboards, but if it's not there on your keyboard, if it's 92 00:05:43,110 --> 00:05:48,360 not possible by holding shift and backslash, just hop onto a search engine and look for how to do the 93 00:05:48,360 --> 00:05:49,980 pipe symbol on your keyboard layout. 94 00:05:50,010 --> 00:05:52,410 If you have trouble finding it, you should come up straight away. 95 00:05:52,500 --> 00:05:59,190 But now what's happening is we're saying pipe the data from the date command standard output into the 96 00:05:59,190 --> 00:06:01,170 cut commands standard input. 97 00:06:01,200 --> 00:06:06,030 Now because data FD was effectively the standard output from the date command, this is going to have 98 00:06:06,030 --> 00:06:06,660 the same effect. 99 00:06:06,660 --> 00:06:13,260 So when we press enter Delta, we get Monday out and this happened all in one step. 100 00:06:13,890 --> 00:06:14,850 So that's really cool. 101 00:06:14,850 --> 00:06:19,650 We managed to take the data straight from the date command and pipe it into the standard input of the 102 00:06:19,650 --> 00:06:21,600 cut command so that it could be operated on. 103 00:06:21,600 --> 00:06:28,110 We gave the command some options that allowed us to cut out directly the name of the day of the week. 104 00:06:28,110 --> 00:06:32,910 Now I'm just going to press the up arrow key, and now I'm going to show you something really cool. 105 00:06:32,910 --> 00:06:38,370 What we can do is we can actually output that data into a. 106 00:06:38,400 --> 00:06:38,650 File. 107 00:06:38,850 --> 00:06:39,300 Okay. 108 00:06:39,300 --> 00:06:43,320 So we could actually save that using redirection into a file called today. 109 00:06:43,560 --> 00:06:44,190 TXT. 110 00:06:44,220 --> 00:06:45,090 So check this out. 111 00:06:45,090 --> 00:06:48,470 We're using the date command to get the all this information here. 112 00:06:48,480 --> 00:06:55,170 We're cutting it out to get just Monday and then we're writing that data into a file called today txt. 113 00:06:55,800 --> 00:06:56,370 Pretty awesome. 114 00:06:56,370 --> 00:06:56,700 Right? 115 00:06:56,700 --> 00:06:57,460 So let's try it. 116 00:06:57,480 --> 00:06:59,780 Let's go ahead and press enter delta. 117 00:06:59,880 --> 00:07:01,370 And now that's gone out. 118 00:07:01,380 --> 00:07:03,160 It hasn't come up on the screen this time. 119 00:07:03,180 --> 00:07:08,310 And if we look in here, we've got a file called today txt and if we click it, we've now got Monday. 120 00:07:08,640 --> 00:07:10,160 Isn't that awesome? 121 00:07:10,170 --> 00:07:11,700 I think that's fantastic. 122 00:07:12,360 --> 00:07:18,480 Now, as a side note, it doesn't actually matter where I put the redirection, so I can have the redirection 123 00:07:18,480 --> 00:07:19,410 at the end here. 124 00:07:19,590 --> 00:07:25,860 I could have the redirection over here and it would still work exactly the same. 125 00:07:26,160 --> 00:07:28,830 Okay, that works absolutely fine. 126 00:07:30,090 --> 00:07:37,470 I could even actually put it in between the different options so I could put it here. 127 00:07:39,640 --> 00:07:41,440 And I will also work exactly the same. 128 00:07:41,710 --> 00:07:44,590 We're still getting today's text with Monday inside. 129 00:07:45,340 --> 00:07:47,230 So that's just something to bear in mind. 130 00:07:47,260 --> 00:07:47,430 Okay. 131 00:07:47,440 --> 00:07:49,450 Don't be scared about typing stuff wrong. 132 00:07:49,810 --> 00:07:53,410 All of these are valid ways to do it, but some are just a lot more readable than others. 133 00:07:53,920 --> 00:07:58,540 But let's say that instead of redirecting, we wanted to send the standard output of the cut command 134 00:07:58,540 --> 00:08:00,610 to yet another command. 135 00:08:00,640 --> 00:08:01,550 How would we do that? 136 00:08:01,570 --> 00:08:06,340 How would, instead of saving it into a file, how would we send the data from the Cook Command into 137 00:08:06,340 --> 00:08:07,480 yet another command? 138 00:08:07,630 --> 00:08:08,470 Well, it's simple. 139 00:08:08,470 --> 00:08:09,610 You just add another pipe. 140 00:08:09,880 --> 00:08:11,830 So you just say, okay, let me cut that out. 141 00:08:12,700 --> 00:08:12,880 Sorry. 142 00:08:12,880 --> 00:08:17,290 Let me just put another pipe and then let's say I'll put a command and I've got some options and then 143 00:08:17,290 --> 00:08:19,750 I've got some command line arguments and that's how that would work. 144 00:08:20,440 --> 00:08:26,800 This new command could take its own options, its own command line arguments, and it could also read 145 00:08:26,800 --> 00:08:28,730 standard input from the command. 146 00:08:29,020 --> 00:08:30,300 So you can see how this works. 147 00:08:30,310 --> 00:08:35,260 We're taking the date command piping that output into standard input of the command, giving the command 148 00:08:35,260 --> 00:08:36,159 its own options. 149 00:08:36,250 --> 00:08:41,289 Then we're piping that output into another command, giving that command its own options and its own 150 00:08:41,289 --> 00:08:44,740 command line arguments and so on and so on and so on. 151 00:08:44,950 --> 00:08:49,930 You can just keep on going, piping these things together, doing what you want, and giving them options 152 00:08:49,930 --> 00:08:50,790 as you wish. 153 00:08:50,800 --> 00:08:55,140 So this is what gives the the whole Linux terminal its power. 154 00:08:55,630 --> 00:09:00,180 Up until now, we've been doing things that are cool but not really necessarily useful. 155 00:09:00,190 --> 00:09:06,760 But when you can combine together really powerful commands and have them operating exactly way exactly 156 00:09:06,760 --> 00:09:11,050 the way that you want to and pipe the data between them and all over your computer, it really gets 157 00:09:11,050 --> 00:09:11,730 amazing. 158 00:09:11,740 --> 00:09:16,120 So you can see that this piping stuff can go on and on and you can see how to build up powerful pipelines 159 00:09:16,120 --> 00:09:16,780 this way. 160 00:09:17,440 --> 00:09:18,640 It's actually really straightforward. 161 00:09:18,640 --> 00:09:20,490 You just pipe together different commands. 162 00:09:20,500 --> 00:09:21,970 Isn't that awesome? 163 00:09:22,720 --> 00:09:24,090 So let me ask you a question. 164 00:09:24,100 --> 00:09:29,650 What if we wanted to actually keep the whole date in a file and then pass it into a pipe? 165 00:09:30,040 --> 00:09:36,370 Well, we can't simply use a redirection at the end here because we're already passing the the standard 166 00:09:36,370 --> 00:09:38,430 output of the date command into the command. 167 00:09:38,440 --> 00:09:38,950 Right. 168 00:09:39,310 --> 00:09:44,530 So we can't really save it in a file because we're already passing it on down the pipeline. 169 00:09:44,860 --> 00:09:52,210 And if we tried to do it, if we tried to just get rid of the NDA, if we tried to just pipe the output 170 00:09:52,210 --> 00:09:54,110 into TXT. 171 00:09:54,160 --> 00:09:58,960 If we do that, then let me just delete everything we've got in here so we can see the effect. 172 00:09:59,290 --> 00:10:03,020 If I try this, we'll see that all we get is just data. 173 00:10:03,030 --> 00:10:10,060 TXT The whole data is saved into data, but then there's nothing to pass on into the command. 174 00:10:10,720 --> 00:10:14,620 This is because redirections are processed before pipes when in the shell. 175 00:10:14,620 --> 00:10:17,950 So it will find the redirection and do that before it ever sees the pipe. 176 00:10:18,400 --> 00:10:20,500 And this basically breaks our pipeline. 177 00:10:21,070 --> 00:10:23,710 So the data can't really go two places at once. 178 00:10:24,930 --> 00:10:25,680 Or can it? 179 00:10:26,260 --> 00:10:29,680 Well, to make this work, we're going to need to use a command called the PT Command. 180 00:10:29,680 --> 00:10:33,820 And it's these kind of advanced stuff that we're going to be showing you in the next video.