%3CLINGO-SUB%20id%3D%22lingo-sub-2108778%22%20slang%3D%22en-US%22%3EHow%20use%20Azure%20Data%20Lake%20Store%20REST%20API%20to%20upload%20file%20via%20AAD%20access%20token%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-2108778%22%20slang%3D%22en-US%22%3E%3CP%3EAzure%20Data%20Lake%20Storage%20Gen2%20(ADLS%20Gen2)%20is%20a%20set%20of%20capabilities%20dedicated%20to%20big%20data%20analytics%2C%20built%20on%26nbsp%3B%3CSPAN%3E%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fstorage%2Fblobs%2Fstorage-blobs-introduction%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noreferrer%22%3EAzure%20Blob%20storage%3C%2FA%3E%3C%2FSPAN%3E%2C%20so%20it%20supports%20Azure%20blob%20Storage%20API%20while%20also%20has%20its%20own%20File%20System%20API.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%20class%3D%22lia-indent-padding-left-60px%22%3EBlob%20Storage%20API%3A%20%3CSPAN%3E%3CA%20href%3D%22https%3A%2F%2Fnam06.safelinks.protection.outlook.com%2F%3Furl%3Dhttps%253A%252F%252Fdocs.microsoft.com%252Fen-us%252Frest%252Fapi%252Fstorageservices%252Foperations-on-blobs%26amp%3Bdata%3D04%257C01%257Cshidin%2540microsoft.com%257C892fff28e69f4c5c2f8a08d8c34dafaf%257C72f988bf86f141af91ab2d7cd011db47%257C1%257C0%257C637474085715544255%257CUnknown%257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%253D%257C1000%26amp%3Bsdata%3DKixK4jCse0o0T4hhIjM6RKy56Iu3zU6yjryn4tW26mo%253D%26amp%3Breserved%3D0%22%20target%3D%22_blank%22%20rel%3D%22noopener%20nofollow%20noreferrer%22%3Ehttps%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Frest%2Fapi%2Fstorageservices%2Foperations-on-blobs%3C%2FA%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%20class%3D%22lia-indent-padding-left-60px%22%3EFile%20System%20API%3A%20%3CSPAN%3E%3CA%20href%3D%22https%3A%2F%2Fnam06.safelinks.protection.outlook.com%2F%3Furl%3Dhttps%253A%252F%252Fdocs.microsoft.com%252Fen-us%252Frest%252Fapi%252Fstorageservices%252Fdata-lake-storage-gen2%26amp%3Bdata%3D04%257C01%257Cshidin%2540microsoft.com%257C892fff28e69f4c5c2f8a08d8c34dafaf%257C72f988bf86f141af91ab2d7cd011db47%257C1%257C0%257C637474085715554207%257CUnknown%257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%253D%257C1000%26amp%3Bsdata%3DFDlocI%252Bi2aK8S3VpNYbQ1xLevdFuS8PUL3ARhD5nRw8%253D%26amp%3Breserved%3D0%22%20target%3D%22_blank%22%20rel%3D%22noopener%20nofollow%20noreferrer%22%3Ehttps%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Frest%2Fapi%2Fstorageservices%2Fdata-lake-storage-gen2%3C%2FA%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%20class%3D%22lia-indent-padding-left-60px%22%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EThese%20interfaces%20allow%20you%20to%20create%20and%20manage%20file%20systems%2C%20as%20well%20as%20to%20create%20and%20manage%20directories%20and%20files%20in%20file%20system.%20Azure%20Data%20Lake%20Storage%20Gen2%20APIs%20support%20Azure%20Active%20Directory%20(Azure%20AD)%2C%20Shared%20Key%2C%20and%20shared%20access%20signature%20(SAS)%20authorization.%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CSPAN%3EIn%20this%20blog%2C%20we%20will%20introduce%20how%20to%20use%20Azure%20AD%20service%20principal%20to%20upload%20file%20to%20ADLS%20gen2%20through%20file%20system%20API%20using%20Powershell%20script.%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CSTRONG%3EPart%201%3A%20%3C%2FSTRONG%3ERegister%20an%20application%20with%20the%20Microsoft%20identity%20platform%20and%20apply%20the%20valid%20role%20assignment%20for%20access.%20%3CSPAN%3E%3CA%20href%3D%22https%3A%2F%2Fnam06.safelinks.protection.outlook.com%2F%3Furl%3Dhttps%253A%252F%252Fdocs.microsoft.com%252Fen-us%252Fazure%252Factive-directory%252Fdevelop%252Fquickstart-register-app%26amp%3Bdata%3D04%257C01%257Cshidin%2540microsoft.com%257C892fff28e69f4c5c2f8a08d8c34dafaf%257C72f988bf86f141af91ab2d7cd011db47%257C1%257C0%257C637474085715554207%257CUnknown%257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%253D%257C1000%26amp%3Bsdata%3D07RUn7WXTYbl0P6kXVgZpGNhPBZ3h3rNd30mNW%252FpLhc%253D%26amp%3Breserved%3D0%22%20target%3D%22_blank%22%20rel%3D%22noopener%20nofollow%20noreferrer%22%3Ehttps%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Factive-directory%2Fdevelop%2Fquickstart-register-app%3C%2FA%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E1.%20Register%20a%20new%20application%20in%20Azure%20AD.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20image-alt%3D%22Frank_Pan_0-1611987361652.png%22%20style%3D%22width%3A%20999px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F250717i9DFEB63E453C1A1A%2Fimage-size%2Flarge%3Fv%3D1.0%26amp%3Bpx%3D999%22%20role%3D%22button%22%20title%3D%22Frank_Pan_0-1611987361652.png%22%20alt%3D%22Frank_Pan_0-1611987361652.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%20class%3D%22lia-indent-padding-left-30px%22%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E2.%26nbsp%3BSelect%20account%20type%20based%20on%20your%20business%20requirements.%3C%2FP%3E%0A%3CP%20class%3D%22lia-indent-padding-left-30px%22%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20image-alt%3D%22Frank_Pan_2-1611987494564.png%22%20style%3D%22width%3A%20999px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F250719iBEEEAA49640F6185%2Fimage-size%2Flarge%3Fv%3D1.0%26amp%3Bpx%3D999%22%20role%3D%22button%22%20title%3D%22Frank_Pan_2-1611987494564.png%22%20alt%3D%22Frank_Pan_2-1611987494564.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%20class%3D%22lia-indent-padding-left-30px%22%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E3.%26nbsp%3BAssign%20%3CSTRONG%3EStorage%20Blob%20Data%20Owner%3C%2FSTRONG%3E%20role%20to%20the%20service%20principal%2C%20which%20grants%20the%20service%20principal%20full%20access%20to%20blob%20data%20rights.%20You%20may%20assign%20other%20blob%20data%20role%20according%20to%20your%20business%20requirements.%20For%20the%20details%20of%20built-in%20roles%E2%80%99%20permissions%20please%20refer%20to%20the%20document%20%3CSPAN%3E%3CA%20href%3D%22https%3A%2F%2Fnam06.safelinks.protection.outlook.com%2F%3Furl%3Dhttps%253A%252F%252Fdocs.microsoft.com%252Fen-us%252Fazure%252Frole-based-access-control%252Fbuilt-in-roles%2523storage-blob-data-owner%26amp%3Bdata%3D04%257C01%257Cshidin%2540microsoft.com%257C892fff28e69f4c5c2f8a08d8c34dafaf%257C72f988bf86f141af91ab2d7cd011db47%257C1%257C0%257C637474085715564167%257CUnknown%257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%253D%257C1000%26amp%3Bsdata%3DnOClNsahL0XuWXqN85%252BoAuMB5wXdxfc4o8jK%252FtgZl0Q%253D%26amp%3Breserved%3D0%22%20target%3D%22_blank%22%20rel%3D%22noopener%20nofollow%20noreferrer%22%3Ehttps%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Frole-based-access-control%2Fbuilt-in-roles%23storage-blob-data-owner%3C%2FA%3E%3C%2FSPAN%3E.%3C%2FP%3E%0A%3CP%20class%3D%22lia-indent-padding-left-30px%22%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20image-alt%3D%22Frank_Pan_3-1611987578967.png%22%20style%3D%22width%3A%20999px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F250721i7D26242833FBFC5E%2Fimage-size%2Flarge%3Fv%3D1.0%26amp%3Bpx%3D999%22%20role%3D%22button%22%20title%3D%22Frank_Pan_3-1611987578967.png%22%20alt%3D%22Frank_Pan_3-1611987578967.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%20class%3D%22lia-indent-padding-left-30px%22%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CSTRONG%3EPart%202%3A%20%3C%2FSTRONG%3EGenerate%20an%20access%20token%20of%20the%20service%20principal%20for%20the%20Rest%20API%20calls.%20%3CSPAN%3E%3CA%20href%3D%22https%3A%2F%2Fnam06.safelinks.protection.outlook.com%2F%3Furl%3Dhttps%253A%252F%252Fdocs.microsoft.com%252Fen-us%252Frest%252Fapi%252Fazure%252F%2523client-credentials-grant-non-interactive-clients%26amp%3Bdata%3D04%257C01%257Cshidin%2540microsoft.com%257C892fff28e69f4c5c2f8a08d8c34dafaf%257C72f988bf86f141af91ab2d7cd011db47%257C1%257C0%257C637474085715564167%257CUnknown%257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%253D%257C1000%26amp%3Bsdata%3D702vmvmK%252BvItpf%252F86KezM3Mkt6Q2Lpyx4bZx%252FLhJhM8%253D%26amp%3Breserved%3D0%22%20target%3D%22_blank%22%20rel%3D%22noopener%20nofollow%20noreferrer%22%3Ehttps%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Frest%2Fapi%2Fazure%2F%23client-credentials-grant-non-interactive-clients%3C%2FA%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CSPAN%3E1.%26nbsp%3B%3C%2FSPAN%3EIn%20the%20Azure%20Portal%20application%20%3CSTRONG%3EOverview%3C%2FSTRONG%3E%2C%20we%20can%20obtain%20the%20%3CU%3EApplication%20ID%20(client%20id)%3C%2FU%3E%20and%20%3CU%3EDirectory%20ID(tenant%20id)%3C%2FU%3E.%3C%2FP%3E%0A%3CP%20class%3D%22lia-indent-padding-left-30px%22%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20image-alt%3D%22Frank_Pan_4-1611987728000.png%22%20style%3D%22width%3A%20999px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F250722i1D0355AD037FC2CE%2Fimage-size%2Flarge%3Fv%3D1.0%26amp%3Bpx%3D999%22%20role%3D%22button%22%20title%3D%22Frank_Pan_4-1611987728000.png%22%20alt%3D%22Frank_Pan_4-1611987728000.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%20class%3D%22lia-indent-padding-left-30px%22%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E2.%26nbsp%3BIn%20the%20%3CSTRONG%3ECertificate%20%26amp%3B%20Secret%3C%2FSTRONG%3E%2C%20create%20a%20secret%20with%20an%20expiration%20time.%3C%2FP%3E%0A%3CP%20class%3D%22lia-indent-padding-left-30px%22%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20image-alt%3D%22Frank_Pan_5-1611987780748.png%22%20style%3D%22width%3A%20999px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F250723iCF7BF7EEBCFD4CA2%2Fimage-size%2Flarge%3Fv%3D1.0%26amp%3Bpx%3D999%22%20role%3D%22button%22%20title%3D%22Frank_Pan_5-1611987780748.png%22%20alt%3D%22Frank_Pan_5-1611987780748.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%20class%3D%22lia-indent-padding-left-30px%22%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20image-alt%3D%22Frank_Pan_6-1611987813799.png%22%20style%3D%22width%3A%20999px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F250724i7A7E7C799861BB71%2Fimage-size%2Flarge%3Fv%3D1.0%26amp%3Bpx%3D999%22%20role%3D%22button%22%20title%3D%22Frank_Pan_6-1611987813799.png%22%20alt%3D%22Frank_Pan_6-1611987813799.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%20class%3D%22lia-indent-padding-left-30px%22%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E3.%26nbsp%3BTo%20generate%20an%20access%20token%20for%20the%20storage%2C%20we%20need%20to%20name%20the%20resource%20endpoint%20for%20storage%20resource%20provider%20as%20%3CSTRONG%3Estorage.azure.com%3C%2FSTRONG%3E.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EIn%20the%20document%20%3CSPAN%3E%3CA%20href%3D%22https%3A%2F%2Fnam06.safelinks.protection.outlook.com%2F%3Furl%3Dhttps%253A%252F%252Fdocs.microsoft.com%252Fen-us%252Fazure%252Factive-directory%252Fdevelop%252Fv2-oauth2-client-creds-grant-flow%2523get-a-token%26amp%3Bdata%3D04%257C01%257Cshidin%2540microsoft.com%257C892fff28e69f4c5c2f8a08d8c34dafaf%257C72f988bf86f141af91ab2d7cd011db47%257C1%257C0%257C637474085715574120%257CUnknown%257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%253D%257C1000%26amp%3Bsdata%3D1cBqHyXUJ0Rd3qZZkPgi2oYphesDI4mgzY%252B8kwiHyHo%253D%26amp%3Breserved%3D0%22%20target%3D%22_blank%22%20rel%3D%22noopener%20nofollow%20noreferrer%22%3Ehttps%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Factive-directory%2Fdevelop%2Fv2-oauth2-client-creds-grant-flow%23get-a-token%3C%2FA%3E%3C%2FSPAN%3E%2C%20we%20can%20see%20how%20a%20token%20endpoint%20work%20in%20a%20common%20scenario.%3C%2FP%3E%0A%3CP%20class%3D%22lia-indent-padding-left-30px%22%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EPowershell%20function%20Example%3A%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CPRE%20class%3D%22lia-code-sample%20language-powershell%22%3E%3CCODE%3Efunction%20Get-StorageAADAccessToken()%0A%7B%0A%20%20%20%20param(%24TENANT_ID%2C%20%24client_id%2C%20%24client_secret)%0A%0A%20%20%20%20%24URI%3D%22https%3A%2F%2Flogin.microsoftonline.com%2F%24TENANT_ID%2Foauth2%2Fv2.0%2Ftoken%22%20%23We%20are%20using%20the%20oauth%20version%202%0A%20%20%20%20%24CONTENT_TYPE%3D%22application%2Fx-www-form-urlencoded%22%0A%0A%20%20%20%20%24HEADERS%20%3D%20%40%7B%0A%20%20%20%20%20%20%20%20%22Content-Type%22%3D%24CONTENT_TYPE%0A%20%20%20%20%7D%0A%0A%0A%20%20%20%20%24grant_type%3D%22client_credentials%22%0A%20%20%20%20%24resource%3D%22https%3A%2F%2Fstorage.azure.com%2F.default%22%0A%0A%20%20%20%20%24BODY%3D%22grant_type%3D%24grant_type%26amp%3Bclient_id%3D%24client_id%26amp%3Bclient_secret%3D%24client_secret%26amp%3Bscope%3D%24resource%22%0A%20%20%20%20%24ACCESS_TOKEN%20%3D%20(Invoke-RestMethod%20-method%20POST%20-Uri%20%24URI%20-Headers%20%24HEADERS%20-Body%20%24BODY).access_token%0A%0A%20%20%20%20return%20%24ACCESS_TOKEN%0A%7D%20%20%0A%3C%2FCODE%3E%3C%2FPRE%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%20class%3D%22lia-indent-padding-left-30px%22%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CSTRONG%3EPart%203%3A%20%3C%2FSTRONG%3EUpload%20the%20file%20using%20File%20System%20interface.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3ETo%20upload%20a%20file%20using%20file%20system%20interface%20will%20use%20the%20three%20APIs%2C%20Create%20File%2C%20Append%20Data%20and%20Flush%20Data.%20All%20APIs%20will%20use%20the%20%3CSTRONG%3E*.dfs.core.windows.net%3C%2FSTRONG%3E%20endpoint%20instead%20of%20%3CSTRONG%3E*.blob.core.windows.net%3C%2FSTRONG%3E%20endpoint.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CUL%3E%0A%3CLI%3ECreate%3A%20%3CSPAN%3E%3CA%20href%3D%22https%3A%2F%2Fnam06.safelinks.protection.outlook.com%2F%3Furl%3Dhttps%253A%252F%252Fdocs.microsoft.com%252Fen-us%252Frest%252Fapi%252Fstorageservices%252Fdatalakestoragegen2%252Fpath%252Fcreate%26amp%3Bdata%3D04%257C01%257Cshidin%2540microsoft.com%257C892fff28e69f4c5c2f8a08d8c34dafaf%257C72f988bf86f141af91ab2d7cd011db47%257C1%257C0%257C637474085715584075%257CUnknown%257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%253D%257C1000%26amp%3Bsdata%3DvufjNYWCI0%252B%252BpY9%252BqzqSQ4tC0q%252FkBJPob3w%252FIxHhCq0%253D%26amp%3Breserved%3D0%22%20target%3D%22_blank%22%20rel%3D%22noopener%20nofollow%20noreferrer%22%3Ehttps%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Frest%2Fapi%2Fstorageservices%2Fdatalakestoragegen2%2Fpath%2Fcreate%3C%2FA%3E%3C%2FSPAN%3E%3C%2FLI%3E%0A%3CLI%3EUpdate%20(Append%20%26amp%3B%20Flush)%3A%20%3CSPAN%3E%3CA%20href%3D%22https%3A%2F%2Fnam06.safelinks.protection.outlook.com%2F%3Furl%3Dhttps%253A%252F%252Fdocs.microsoft.com%252Fen-us%252Frest%252Fapi%252Fstorageservices%252Fdatalakestoragegen2%252Fpath%252Fupdate%26amp%3Bdata%3D04%257C01%257Cshidin%2540microsoft.com%257C892fff28e69f4c5c2f8a08d8c34dafaf%257C72f988bf86f141af91ab2d7cd011db47%257C1%257C0%257C637474085715584075%257CUnknown%257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%253D%257C1000%26amp%3Bsdata%3D0wFK%252FdkJqZtcn%252BPZz1zu6NTHoY9aYcUjXRCNYwJQWm8%253D%26amp%3Breserved%3D0%22%20target%3D%22_blank%22%20rel%3D%22noopener%20nofollow%20noreferrer%22%3Ehttps%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Frest%2Fapi%2Fstorageservices%2Fdatalakestoragegen2%2Fpath%2Fupdate%3C%2FA%3E%3C%2FSPAN%3E%3C%2FLI%3E%0A%3C%2FUL%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EHere%20is%20a%20logic%20flow%20to%20upload%20a%20large%20file.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CUL%3E%0A%3CLI%3EThe%20first%20position%20is%200%3C%2FLI%3E%0A%3CLI%3EThe%20next%20position%20is%20the%20last%20position%20plus%20the%20last%20content%20length.%3C%2FLI%3E%0A%3CLI%3EWe%20can%20send%20multiple%20append%20data%20requests%20at%20the%20same%20time%2C%20but%20the%20position%20information%20needs%20to%20be%20calculated.%3C%2FLI%3E%0A%3C%2FUL%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20image-alt%3D%22Frank_Pan_7-1611988384485.png%22%20style%3D%22width%3A%20999px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F250725iAA63DC329532A368%2Fimage-size%2Flarge%3Fv%3D1.0%26amp%3Bpx%3D999%22%20role%3D%22button%22%20title%3D%22Frank_Pan_7-1611988384485.png%22%20alt%3D%22Frank_Pan_7-1611988384485.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%20class%3D%22lia-indent-padding-left-30px%22%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EThe%20Powershell%20methods%20example%3A%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E1.%20Create%20File%20is%20a%20Create%20API%20in%20the%20file%20system.%20By%20default%2C%20the%20destination%20is%20overwritten%20if%20the%20file%20already%20exists%20and%20has%20a%20broken%20lease.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CPRE%20class%3D%22lia-code-sample%20language-powershell%22%3E%3CCODE%3Efunction%20Create-AzureADLS2File()%0A%7B%0A%20%20%20%20param(%24STORAGE_ACCOUNT_NAME%2C%20%24ROOT%2C%20%24PREFIX)%20%23%23%20storage%20account%20is%20the%20name%20of%20the%20ADLS%20gen2%20account%2C%20root%20is%20the%20file%20system%20container%2C%20prefix%20is%20the%20path%20and%20file%20name%20of%20the%20storage%20account%0A%20%20%20%20%0A%20%20%20%20%24URI%3D%22https%3A%2F%2F%24STORAGE_ACCOUNT_NAME.dfs.core.windows.net%2F%22%2B%24ROOT%2B%22%2F%22%2B%24PREFIX%2B%22%3Fresource%3Dfile%22%0A%0A%20%20%20%20%24DATE%20%3D%20%5BSystem.DateTime%5D%3A%3AUtcNow.ToString(%22R%22)%0A%0A%20%20%20%20%24ACCESS_TOKEN%3DGet-StorageAADAccessToken%20-TENANT_ID%20%24TENANT_ID%20-client_id%20%24CLIENT_ID%20-client_secret%20%24CLIENT_SECRET%0A%0A%20%20%20%20%24HEADERS%20%3D%20%40%7B%0A%20%20%20%20%20%20%20%20%22x-ms-date%22%3D%24DATE%20%0A%20%20%20%20%20%20%20%20%22x-ms-version%22%3D%222019-12-12%22%20%0A%20%20%20%20%20%20%20%20%22authorization%22%3D%22Bearer%20%24ACCESS_TOKEN%22%0A%20%20%20%20%7D%0A%20%20%20%20Invoke-RestMethod%20-method%20PUT%20-Uri%20%24URI%20-Headers%20%24HEADERS%0A%7D%20%0A%3C%2FCODE%3E%3C%2FPRE%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EAfter%20create%20a%20file%20by%20the%20Powershell%20custom%20method%20below%2C%20you%20will%20get%20a%20zero%20size%20file.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CPRE%20class%3D%22lia-code-sample%20language-powershell%22%3E%3CCODE%3ECreate-AzureADLS2File%20-STORAGE_ACCOUNT_NAME%20frankpanadls2%20-ROOT%20test%20-PREFIX%20file1%20%3C%2FCODE%3E%3C%2FPRE%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20image-alt%3D%22Frank_Pan_8-1611988803457.jpeg%22%20style%3D%22width%3A%20999px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F250726i2AC2FAF1C6369CDB%2Fimage-size%2Flarge%3Fv%3D1.0%26amp%3Bpx%3D999%22%20role%3D%22button%22%20title%3D%22Frank_Pan_8-1611988803457.jpeg%22%20alt%3D%22Frank_Pan_8-1611988803457.jpeg%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E2.%20Append%20Data%20is%20a%20part%20of%20Update%20API%20in%20the%20file%20system.%20%22append%22%20is%20to%20upload%20data%20by%20appending%20to%20a%20file.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CPRE%20class%3D%22lia-code-sample%20language-powershell%22%3E%3CCODE%3Efunction%20Upload-AzureADLS2File()%0A%7B%0A%20%20%20%20param(%24STORAGE_ACCOUNT_NAME%2C%20%24ROOT%2C%20%24PREFIX%2C%20%24POS%2C%20%24BODY)%0A%20%20%20%20%0A%20%20%20%20%24URI%3D%22https%3A%2F%2F%24STORAGE_ACCOUNT_NAME.dfs.core.windows.net%2F%22%2B%24ROOT%2B%22%2F%22%2B%24PREFIX%2B%22%3Faction%3Dappend%26amp%3Bposition%3D%24POS%22%0A%20%20%20%20%24DATE%20%3D%20%5BSystem.DateTime%5D%3A%3AUtcNow.ToString(%22R%22)%0A%0A%24ACCESS_TOKEN%3D%20Get-StorageAADAccessToken%20-TENANT_ID%20%24TENANT_ID%20-client_id%20%24CLIENT_ID%20-client_secret%20%24CLIENT_SECRET%0A%0A%20%20%20%20%24HEADERS%20%3D%20%40%7B%0A%20%20%20%20%20%20%20%20%22x-ms-date%22%3D%24DATE%20%0A%20%20%20%20%20%20%20%20%22x-ms-version%22%3D%222019-12-12%22%20%0A%20%20%20%20%20%20%20%20%22authorization%22%3D%22Bearer%20%24ACCESS_TOKEN%22%0A%20%20%20%20%20%20%20%20%22content-length%22%3D0%0A%20%20%20%20%7D%0A%0A%20%20%20Invoke-RestMethod%20-method%20PATCH%20-Uri%20%24URI%20-Headers%20%24HEADERS%20-Body%20%24BODY%0A%7D%20%0A%3C%2FCODE%3E%3C%2FPRE%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EIf%20we%20have%20content%20below%2C%20we%20can%20get%20a%20list%20of%20position%20and%20content%20length.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%20class%3D%22lia-indent-padding-left-30px%22%3Edata%20row%201%3C%2FP%3E%0A%3CP%20class%3D%22lia-indent-padding-left-30px%22%3Edata%20row%2022%3C%2FP%3E%0A%3CP%20class%3D%22lia-indent-padding-left-30px%22%3Edata%20row%20333%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%20class%3D%22lia-indent-padding-left-30px%22%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20image-alt%3D%22Frank_Pan_0-1611989162194.png%22%20style%3D%22width%3A%20400px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F250727i86B7AB991CA234B3%2Fimage-size%2Fmedium%3Fv%3D1.0%26amp%3Bpx%3D400%22%20role%3D%22button%22%20title%3D%22Frank_Pan_0-1611989162194.png%22%20alt%3D%22Frank_Pan_0-1611989162194.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CPRE%20class%3D%22lia-code-sample%20language-powershell%22%3E%3CCODE%3EUpload-AzureADLS2File%20-STORAGE_ACCOUNT_NAME%20frankpanadls2%20-ROOT%20test%20-PREFIX%20file1%20-POS%200%20-BODY%20%22data%20row%201%60n%22%0AUpload-AzureADLS2File%20-STORAGE_ACCOUNT_NAME%20frankpanadls2%20-ROOT%20test%20-PREFIX%20file1%20-POS%2011%20-BODY%20%22data%20row%2022%60n%22%0AUpload-AzureADLS2File%20-STORAGE_ACCOUNT_NAME%20frankpanadls2%20-ROOT%20test%20-PREFIX%20file1%20-POS%2023%20-BODY%20%22data%20row%20333%60n%22%0A%3C%2FCODE%3E%3C%2FPRE%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EThere%20will%20be%20no%20data%20in%20the%20file%20until%20you%20flush%20all%20content%20in%20the%20file.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20image-alt%3D%22Frank_Pan_1-1611989257000.jpeg%22%20style%3D%22width%3A%20999px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F250728i3BFE983D2332F330%2Fimage-size%2Flarge%3Fv%3D1.0%26amp%3Bpx%3D999%22%20role%3D%22button%22%20title%3D%22Frank_Pan_1-1611989257000.jpeg%22%20alt%3D%22Frank_Pan_1-1611989257000.jpeg%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E3.%20Flush%20Data%20is%20a%20part%20of%20Update%20API%20in%20the%20file%20system.%20%22flush%22%20is%20to%20flush%20previously%20uploaded%20data%20to%20a%20file.%20This%20request%20is%20similar%20to%20PutBlockList%20in%20the%20blob%20storage%20api%2C%20but%20will%20need%20to%20specify%20position.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CPRE%20class%3D%22lia-code-sample%20language-powershell%22%3E%3CCODE%3Efunction%20Flush-AzureADLS2File()%0A%7B%0A%20%20%20%20param(%24STORAGE_ACCOUNT_NAME%2C%20%24ROOT%2C%20%24PREFIX%2C%20%24POS)%0A%20%20%20%20%0A%20%20%20%20%24URI%3D%22https%3A%2F%2F%24STORAGE_ACCOUNT_NAME.dfs.core.windows.net%2F%22%2B%24ROOT%2B%22%2F%22%2B%24PREFIX%2B%22%3Faction%3Dflush%26amp%3Bposition%3D%24POS%22%0A%20%20%20%20%24DATE%20%3D%20%5BSystem.DateTime%5D%3A%3AUtcNow.ToString(%22R%22)%0A%0A%20%20%20%20%24ACCESS_TOKEN%3D%20Get-StorageAADAccessToken%20-TENANT_ID%20%24TENANT_ID%20-client_id%20%24CLIENT_ID%20-client_secret%20%24CLIENT_SECRET%0A%0A%20%20%20%20%24HEADERS%20%3D%20%40%7B%0A%20%20%20%20%20%20%20%20%22x-ms-date%22%3D%24DATE%20%0A%20%20%20%20%20%20%20%20%22x-ms-version%22%3D%222019-12-12%22%20%0A%20%20%20%20%20%20%20%20%22authorization%22%3D%22Bearer%20%24ACCESS_TOKEN%22%0A%20%20%20%20%20%20%20%20%22content-length%22%3D%24POS%0A%20%20%20%20%7D%0A%0A%20%20%20%20Invoke-RestMethod%20-method%20PATCH%20-Uri%20%24URI%20-Headers%20%24HEADERS%0A%7D%20%0A%3C%2FCODE%3E%3C%2FPRE%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CPRE%20class%3D%22lia-code-sample%20language-powershell%22%3E%3CCODE%3EFlush-AzureADLS2File%20-AzureADLS2File%20-STORAGE_ACCOUNT_NAME%20frankpanadls2%20-ROOT%20test%20-PREFIX%20file1%20-POS%2036%20%3C%2FCODE%3E%3C%2FPRE%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EWe%20will%20see%20the%20flushed%20file%20like%20below%20with%20all%20content.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20image-alt%3D%22Frank_Pan_2-1611989481613.png%22%20style%3D%22width%3A%20999px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F250729iCA70E3E6A1D60164%2Fimage-size%2Flarge%3Fv%3D1.0%26amp%3Bpx%3D999%22%20role%3D%22button%22%20title%3D%22Frank_Pan_2-1611989481613.png%22%20alt%3D%22Frank_Pan_2-1611989481613.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-TEASER%20id%3D%22lingo-teaser-2108778%22%20slang%3D%22en-US%22%3E%3CP%3EAzure%20Data%20Lake%20Storage%20Account%20supports%20two%20types%20of%20interface%3A%20blob%20storage%20interface%20and%20file%20system%20interface.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%26nbsp%3B%26nbsp%3B%26nbsp%3B%26nbsp%3B%26nbsp%3B%26nbsp%3B%26nbsp%3B%20Blob%20storage%20interface%3A%20%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Frest%2Fapi%2Fstorageservices%2Foperations-on-blobs%22%20rel%3D%22noopener%20noreferrer%22%20target%3D%22_blank%22%3Ehttps%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Frest%2Fapi%2Fstorageservices%2Foperations-on-blobs%3C%2FA%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%26nbsp%3B%26nbsp%3B%26nbsp%3B%26nbsp%3B%26nbsp%3B%26nbsp%3B%26nbsp%3B%20File%20system%20interface%3A%20%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Frest%2Fapi%2Fstorageservices%2Fdata-lake-storage-gen2%22%20rel%3D%22noopener%20noreferrer%22%20target%3D%22_blank%22%3Ehttps%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Frest%2Fapi%2Fstorageservices%2Fdata-lake-storage-gen2%3C%2FA%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EThese%20interfaces%20allow%20you%20to%20create%20and%20manage%20file%20systems%2C%20as%20well%20as%20to%20create%20and%20manage%20directories%20and%20files%20in%20file%20system.%20Azure%20Data%20Lake%20Storage%20Gen2%20APIs%20support%20Azure%20Active%20Directory%20(Azure%20AD)%2C%20Shared%20Key%2C%20and%20shared%20access%20signature%20(SAS)%20authorization.%20Here%20is%20an%20example%20to%20use%20the%20Azure%20Active%20Directory%20access%20token%20to%20upload%20a%20file.%3C%2FP%3E%3C%2FLINGO-TEASER%3E%3CLINGO-LABS%20id%3D%22lingo-labs-2108778%22%20slang%3D%22en-US%22%3E%3CLINGO-LABEL%3EAzure%20Storage%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E
Microsoft

Azure Data Lake Storage Gen2 (ADLS Gen2) is a set of capabilities dedicated to big data analytics, built on Azure Blob storage, so it supports Azure blob Storage API while also has its own File System API.

 

Blob Storage API: https://docs.microsoft.com/en-us/rest/api/storageservices/operations-on-blobs

File System API: https://docs.microsoft.com/en-us/rest/api/storageservices/data-lake-storage-gen2

 

These interfaces allow you to create and manage file systems, as well as to create and manage directories and files in file system. Azure Data Lake Storage Gen2 APIs support Azure Active Directory (Azure AD), Shared Key, and shared access signature (SAS) authorization. 

 

In this blog, we will introduce how to use Azure AD service principal to upload file to ADLS gen2 through file system API using Powershell script.

 

Part 1: Register an application with the Microsoft identity platform and apply the valid role assignment for access. https://docs.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app

 

1. Register a new application in Azure AD.

 

Frank_Pan_0-1611987361652.png

 

2. Select account type based on your business requirements.

 

Frank_Pan_2-1611987494564.png

 

3. Assign Storage Blob Data Owner role to the service principal, which grants the service principal full access to blob data rights. You may assign other blob data role according to your business requirements. For the details of built-in roles’ permissions please refer to the document https://docs.microsoft.com/en-us/azure/role-based-access-control/built-in-roles#storage-blob-data-ow....

 

Frank_Pan_3-1611987578967.png

 

Part 2: Generate an access token of the service principal for the Rest API calls. https://docs.microsoft.com/en-us/rest/api/azure/#client-credentials-grant-non-interactive-clients

 

1. In the Azure Portal application Overview, we can obtain the Application ID (client id) and Directory ID(tenant id).

 

Frank_Pan_4-1611987728000.png

 

2. In the Certificate & Secret, create a secret with an expiration time.

 

Frank_Pan_5-1611987780748.png

 

Frank_Pan_6-1611987813799.png

 

3. To generate an access token for the storage, we need to name the resource endpoint for storage resource provider as storage.azure.com.

 

In the document https://docs.microsoft.com/en-us/azure/active-directory/develop/v2-oauth2-client-creds-grant-flow#ge..., we can see how a token endpoint work in a common scenario.

 

Powershell function Example:

 

 

function Get-StorageAADAccessToken()
{
    param($TENANT_ID, $client_id, $client_secret)

    $URI="https://login.microsoftonline.com/$TENANT_ID/oauth2/v2.0/token" #We are using the oauth version 2
    $CONTENT_TYPE="application/x-www-form-urlencoded"

    $HEADERS = @{
        "Content-Type"=$CONTENT_TYPE
    }


    $grant_type="client_credentials"
    $resource="https://storage.azure.com/.default"

    $BODY="grant_type=$grant_type&client_id=$client_id&client_secret=$client_secret&scope=$resource"
    $ACCESS_TOKEN = (Invoke-RestMethod -method POST -Uri $URI -Headers $HEADERS -Body $BODY).access_token

    return $ACCESS_TOKEN
}  

 

 

Part 3: Upload the file using File System interface.

 

To upload a file using file system interface will use the three APIs, Create File, Append Data and Flush Data. All APIs will use the *.dfs.core.windows.net endpoint instead of *.blob.core.windows.net endpoint.

 

 

Here is a logic flow to upload a large file.

 

  • The first position is 0
  • The next position is the last position plus the last content length.
  • We can send multiple append data requests at the same time, but the position information needs to be calculated.

 

Frank_Pan_7-1611988384485.png

 

The Powershell methods example:

 

1. Create File is a Create API in the file system. By default, the destination is overwritten if the file already exists and has a broken lease.

 

 

function Create-AzureADLS2File()
{
    param($STORAGE_ACCOUNT_NAME, $ROOT, $PREFIX) ## storage account is the name of the ADLS gen2 account, root is the file system container, prefix is the path and file name of the storage account
    
    $URI="https://$STORAGE_ACCOUNT_NAME.dfs.core.windows.net/"+$ROOT+"/"+$PREFIX+"?resource=file"

    $DATE = [System.DateTime]::UtcNow.ToString("R")

    $ACCESS_TOKEN=Get-StorageAADAccessToken -TENANT_ID $TENANT_ID -client_id $CLIENT_ID -client_secret $CLIENT_SECRET

    $HEADERS = @{
        "x-ms-date"=$DATE 
        "x-ms-version"="2019-12-12" 
        "authorization"="Bearer $ACCESS_TOKEN"
    }
    Invoke-RestMethod -method PUT -Uri $URI -Headers $HEADERS
} 

 

 

After create a file by the Powershell custom method below, you will get a zero size file.

 

 

Create-AzureADLS2File -STORAGE_ACCOUNT_NAME frankpanadls2 -ROOT test -PREFIX file1 

 

 

Frank_Pan_8-1611988803457.jpeg

 

2. Append Data is a part of Update API in the file system. "append" is to upload data by appending to a file.

 

 

function Upload-AzureADLS2File()
{
    param($STORAGE_ACCOUNT_NAME, $ROOT, $PREFIX, $POS, $BODY)
    
    $URI="https://$STORAGE_ACCOUNT_NAME.dfs.core.windows.net/"+$ROOT+"/"+$PREFIX+"?action=append&position=$POS"
    $DATE = [System.DateTime]::UtcNow.ToString("R")

$ACCESS_TOKEN= Get-StorageAADAccessToken -TENANT_ID $TENANT_ID -client_id $CLIENT_ID -client_secret $CLIENT_SECRET

    $HEADERS = @{
        "x-ms-date"=$DATE 
        "x-ms-version"="2019-12-12" 
        "authorization"="Bearer $ACCESS_TOKEN"
        "content-length"=0
    }

   Invoke-RestMethod -method PATCH -Uri $URI -Headers $HEADERS -Body $BODY
} 

 

 

If we have content below, we can get a list of position and content length.

 

data row 1

data row 22

data row 333

 

Frank_Pan_0-1611989162194.png

 

 

Upload-AzureADLS2File -STORAGE_ACCOUNT_NAME frankpanadls2 -ROOT test -PREFIX file1 -POS 0 -BODY "data row 1`n"
Upload-AzureADLS2File -STORAGE_ACCOUNT_NAME frankpanadls2 -ROOT test -PREFIX file1 -POS 11 -BODY "data row 22`n"
Upload-AzureADLS2File -STORAGE_ACCOUNT_NAME frankpanadls2 -ROOT test -PREFIX file1 -POS 23 -BODY "data row 333`n"

 

 

There will be no data in the file until you flush all content in the file.

 

Frank_Pan_1-1611989257000.jpeg

 

3. Flush Data is a part of Update API in the file system. "flush" is to flush previously uploaded data to a file. This request is similar to PutBlockList in the blob storage api, but will need to specify position.

 

 

function Flush-AzureADLS2File()
{
    param($STORAGE_ACCOUNT_NAME, $ROOT, $PREFIX, $POS)
    
    $URI="https://$STORAGE_ACCOUNT_NAME.dfs.core.windows.net/"+$ROOT+"/"+$PREFIX+"?action=flush&position=$POS"
    $DATE = [System.DateTime]::UtcNow.ToString("R")

    $ACCESS_TOKEN= Get-StorageAADAccessToken -TENANT_ID $TENANT_ID -client_id $CLIENT_ID -client_secret $CLIENT_SECRET

    $HEADERS = @{
        "x-ms-date"=$DATE 
        "x-ms-version"="2019-12-12" 
        "authorization"="Bearer $ACCESS_TOKEN"
        "content-length"=$POS
    }

    Invoke-RestMethod -method PATCH -Uri $URI -Headers $HEADERS
} 

 

 

 

Flush-AzureADLS2File -AzureADLS2File -STORAGE_ACCOUNT_NAME frankpanadls2 -ROOT test -PREFIX file1 -POS 36 

 

 

We will see the flushed file like below with all content.

 

Frank_Pan_2-1611989481613.png