1 00:00:00,250 --> 00:00:01,083 Hey, Cloud Gurus. 2 00:00:01,083 --> 00:00:02,917 Welcome back to our series on 3 00:00:02,917 --> 00:00:04,920 Meeting the Tools of the Trade. 4 00:00:04,920 --> 00:00:06,690 In this lesson, we're continuing on 5 00:00:06,690 --> 00:00:08,763 with Azure Synapse Pipelines. 6 00:00:11,220 --> 00:00:14,260 These are a component of Azure Synapse Analytics, 7 00:00:14,260 --> 00:00:16,330 and we're going to start off with a brief comparison 8 00:00:16,330 --> 00:00:19,290 between them and Azure Data Factory. 9 00:00:19,290 --> 00:00:21,260 We'll then jump over to a demo 10 00:00:21,260 --> 00:00:23,913 and wrap everything up with a review. 11 00:00:26,210 --> 00:00:30,210 So how do Azure Synapse Pipelines and Data Factory differ? 12 00:00:30,210 --> 00:00:32,140 Well, not a lot. 13 00:00:32,140 --> 00:00:34,280 It's kind of like looking in a mirror. 14 00:00:34,280 --> 00:00:37,370 Azure Synapse Pipelines are built on the same technology 15 00:00:37,370 --> 00:00:39,083 as Azure Data Factory. 16 00:00:40,870 --> 00:00:42,900 There are some differences. 17 00:00:42,900 --> 00:00:46,410 ADF has the SSIS activity. 18 00:00:46,410 --> 00:00:48,973 It also has the Power Query activity. 19 00:00:49,910 --> 00:00:54,070 Synapse has the ability to monitor Spark jobs for data flow, 20 00:00:54,070 --> 00:00:56,513 but ADF has Azure Monitor Integration. 21 00:00:57,560 --> 00:01:00,340 And this is just a small subset of the differences. 22 00:01:00,340 --> 00:01:03,020 For the full list, you can follow this link. 23 00:01:03,020 --> 00:01:06,640 I'll also include that link in the resources for the lesson, 24 00:01:06,640 --> 00:01:09,470 but which tool you use basically comes down to, 25 00:01:09,470 --> 00:01:11,580 what do you want to use it for? 26 00:01:11,580 --> 00:01:12,960 For Synapse Pipelines, 27 00:01:12,960 --> 00:01:15,950 we use it mostly for analytics projects. 28 00:01:15,950 --> 00:01:18,130 When building an analytics solution, 29 00:01:18,130 --> 00:01:20,740 Synapse Analytics is a one-stop shop 30 00:01:20,740 --> 00:01:24,210 with a fully-integrated, design experience. 31 00:01:24,210 --> 00:01:26,500 So you can look at Azure Synapse Analytics 32 00:01:26,500 --> 00:01:29,700 as an all-in-one data and analytics solution 33 00:01:29,700 --> 00:01:32,190 of which pipelines is just a part. 34 00:01:32,190 --> 00:01:35,290 For those mostly wanting to do ETL and migration work, 35 00:01:35,290 --> 00:01:38,163 you might lean toward Azure Data Factory instead. 36 00:01:40,230 --> 00:01:42,860 With that background, let's jump into the Azure portal 37 00:01:42,860 --> 00:01:44,360 and take a look at the action. 38 00:01:46,160 --> 00:01:47,560 Here we are in the Azure portal, 39 00:01:47,560 --> 00:01:50,350 within my Azure Synapse Analytics workspace. 40 00:01:50,350 --> 00:01:53,330 I've opened up Azure Synapse Studio, 41 00:01:53,330 --> 00:01:55,510 and already you can tell it has very much 42 00:01:55,510 --> 00:01:58,323 the same look and feel as Azure Data Factory. 43 00:01:59,180 --> 00:02:01,980 We do have some separate sections on the side here. 44 00:02:01,980 --> 00:02:06,500 Data, Develop, and Integrate is where we want to go 45 00:02:06,500 --> 00:02:07,933 to work with our Pipelines. 46 00:02:09,210 --> 00:02:13,000 Within here, again, things look very much the same. 47 00:02:13,000 --> 00:02:15,060 We can create our Pipeline 48 00:02:15,060 --> 00:02:18,103 and we have many of the same activities as we did before. 49 00:02:19,170 --> 00:02:20,240 There are some in here 50 00:02:20,240 --> 00:02:22,830 that you don't find in Azure Data Factory 51 00:02:22,830 --> 00:02:24,803 under our Synapse category, 52 00:02:25,790 --> 00:02:29,040 but we have our same Copy data, 53 00:02:29,040 --> 00:02:32,850 Data flow, HDInsight activities. 54 00:02:32,850 --> 00:02:36,010 And so again, very much the same experience 55 00:02:36,010 --> 00:02:37,960 as what you saw before. 56 00:02:37,960 --> 00:02:40,300 As the 2 products develop, Azure Synapse 57 00:02:40,300 --> 00:02:42,570 has some features that ADF does not, 58 00:02:42,570 --> 00:02:46,860 but ADF has some features that Synapse Pipelines do not. 59 00:02:46,860 --> 00:02:48,370 But the essence of how you move 60 00:02:48,370 --> 00:02:52,000 and transform data with them is very much the same. 61 00:02:52,000 --> 00:02:54,080 We're still using various activities 62 00:02:54,080 --> 00:02:56,993 running on Databricks or Spark clusters in the background. 63 00:03:00,140 --> 00:03:02,850 By way of review, Azure Synapse Pipelines 64 00:03:02,850 --> 00:03:06,163 are built on the same technology as Azure Data Factory. 65 00:03:07,300 --> 00:03:09,840 It does not share all ADF features. 66 00:03:09,840 --> 00:03:11,690 Which tool you use will be determined 67 00:03:11,690 --> 00:03:14,703 by your overall transformation and solution goals. 68 00:03:15,730 --> 00:03:17,310 Thanks for joining me for this quick look 69 00:03:17,310 --> 00:03:19,290 at Azure Synapse Pipelines. 70 00:03:19,290 --> 00:03:20,190 When you're ready, 71 00:03:20,190 --> 00:03:22,490 we'll jump into the next tool of the trade. 72 00:03:22,490 --> 00:03:23,543 I'll see you there.