%3CLINGO-SUB%20id%3D%22lingo-sub-1684836%22%20slang%3D%22en-US%22%3EHow%20to%20start%20Synapse%20Pipeline%20from%20Rest%20API%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1684836%22%20slang%3D%22en-US%22%3E%3CP%20style%3D%22margin%3A%200in%3B%20font-family%3A%20Calibri%3B%20font-size%3A%2011.0pt%3B%22%3EAnother%20day%20another%20case.%3C%2FP%3E%0A%3CP%20style%3D%22margin%3A%200in%3B%20font-family%3A%20Calibri%3B%20font-size%3A%2011.0pt%3B%22%3EIt%20took%20me%20a%20while%20to%20follow%20this%20step%20by%20step%20by%20using%20the%20new%20Synapse%20APIs.%20So%20this%20post%20has%20the%20intention%20to%20make%20it%20easier%20by%20example.%3C%2FP%3E%0A%3CP%20style%3D%22margin%3A%200in%3B%20font-family%3A%20Calibri%3B%20font-size%3A%2011.0pt%3B%22%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%20style%3D%22margin%3A%200in%3B%20font-family%3A%20Calibri%3B%20font-size%3A%2011.0pt%3B%22%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%20style%3D%22margin%3A%200in%3B%20font-family%3A%20Calibri%3B%20font-size%3A%2011.0pt%3B%22%3EYou%20will%20need%3A%3C%2FP%3E%0A%3CUL%3E%0A%3CLI%20style%3D%22font-family%3A%20Calibri%3B%20font-size%3A%2011pt%3B%22%3EA%20Synapse%20workspace%3C%2FLI%3E%0A%3CLI%20style%3D%22font-family%3A%20Calibri%3B%20font-size%3A%2011pt%3B%22%3EAn%20ADF%20pipeline%20that%20you%20want%20to%20start%20from%20Rest%20API.%3C%2FLI%3E%0A%3C%2FUL%3E%0A%3CP%20style%3D%22margin%3A%200in%3B%20margin-left%3A%20.375in%3B%20font-family%3A%20Calibri%3B%20font-size%3A%2011.0pt%3B%22%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%20style%3D%22margin%3A%200in%3B%20font-family%3A%20Calibri%3B%20font-size%3A%2011.0pt%3B%22%3EDoc%20references%3A%3C%2FP%3E%0A%3CP%20style%3D%22margin%3A%200in%3B%20font-family%3A%20Calibri%3B%20font-size%3A%2011.0pt%3B%22%3E%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fdata-factory%2Fquickstart-create-data-factory-rest-api%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noopener%20noreferrer%20noopener%20noreferrer%22%3Ehttps%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fdata-factory%2Fquickstart-create-data-factory-rest-api%3C%2FA%3E%3C%2FP%3E%0A%3CP%20style%3D%22margin%3A%200in%3B%20font-family%3A%20Calibri%3B%20font-size%3A%2011.0pt%3B%22%3E%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fazure-monitor%2Fplatform%2Frest-api-walkthrough%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noopener%20noreferrer%20noopener%20noreferrer%22%3Ehttps%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fazure-monitor%2Fplatform%2Frest-api-walkthrough%3C%2FA%3E%3C%2FP%3E%0A%3CP%20style%3D%22margin%3A%200in%3B%20font-family%3A%20Calibri%3B%20font-size%3A%2011.0pt%3B%22%3E%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Frest%2Fapi%2Fsynapse%2Fdata-plane%2Fpipeline%2Fcreatepipelinerun%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noopener%20noreferrer%20noopener%20noreferrer%22%3Ehttps%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Frest%2Fapi%2Fsynapse%2Fdata-plane%2Fpipeline%2Fcreatepipelinerun%3C%2FA%3E%3C%2FP%3E%0A%3CP%20style%3D%22margin%3A%200in%3B%20font-family%3A%20Calibri%3B%20font-size%3A%2011.0pt%3B%22%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%20style%3D%22margin%3A%200in%3B%20font-family%3A%20Calibri%3B%20font-size%3A%2011.0pt%3B%22%3EThat%20is%20my%20pipeline%20code.%20It%20is%20a%20simple%20one%20it%20just%20creates%20Spark%20Database%20using%20a%20notebook.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CPRE%20class%3D%22lia-code-sample%20language-applescript%22%3E%3CCODE%3E%25%25spark%0Aspark.sql(%22CREATE%20DATABASE%20IF%20NOT%20EXISTS%20DB_example%22)%3C%2FCODE%3E%3C%2FPRE%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%20style%3D%22margin%3A%200in%3B%20font-family%3A%20Calibri%3B%20font-size%3A%2011.0pt%3B%22%3EAs%20you%20can%20see%20in%20figure%201.%20That%20is%20my%20pipeline%3A%3C%2FP%3E%0A%3CP%20style%3D%22margin%3A%200in%3B%20font-family%3A%20Calibri%3B%20font-size%3A%2011.0pt%3B%22%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20image-alt%3D%22pipelin_noe.png%22%20style%3D%22width%3A%20667px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F219601i24E88BBBEA582425%2Fimage-size%2Flarge%3Fv%3D1.0%26amp%3Bpx%3D999%22%20title%3D%22pipelin_noe.png%22%20alt%3D%22pipelin_noe.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%20style%3D%22margin%3A%200in%3B%20font-family%3A%20Calibri%3B%20font-size%3A%2011.0pt%3B%22%3EFigure%201%20Pipeline%3C%2FP%3E%0A%3CP%20style%3D%22margin%3A%200in%3B%20font-family%3A%20Calibri%3B%20font-size%3A%2011.0pt%3B%22%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%20style%3D%22margin%3A%200in%3B%20font-family%3A%20Calibri%3B%20font-size%3A%2011.0pt%3B%22%3EThe%20name%20of%20my%20pipeline%20is%20User_not_test.%3C%2FP%3E%0A%3CP%20style%3D%22margin%3A%200in%3B%20font-family%3A%20Calibri%3B%20font-size%3A%2011.0pt%3B%22%3EI%20can%20run%20successfully%20this%20pipeline%20from%20Synapse%20Studio.%20But%20I%20want%20to%20run%20it%20from%20the%20Rest%20API.%3C%2FP%3E%0A%3CP%20style%3D%22margin%3A%200in%3B%20font-family%3A%20Calibri%3B%20font-size%3A%2011.0pt%3B%22%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%20style%3D%22margin%3A%200in%3B%20font-family%3A%20Calibri%3B%20font-size%3A%2011.0pt%3B%22%3E%3CSTRONG%3EStep%20by%20Step%3C%2FSTRONG%3E%3C%2FP%3E%0A%3CP%20style%3D%22margin%3A%200in%3B%20font-family%3A%20Calibri%3B%20font-size%3A%2011.0pt%3B%22%3EThe%20first%20step%20consists%20in%20using%20this%20documentation%20to%20register%20my%20pipeline%2Fworkspace%20as%20an%20application%3A%3C%2FP%3E%0A%3CP%20style%3D%22margin%3A%200in%3B%20font-family%3A%20Calibri%3B%20font-size%3A%2011.0pt%3B%22%3E(%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fazure-monitor%2Fplatform%2Frest-api-walkthrough%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noopener%20noreferrer%20noopener%20noreferrer%22%3Ehttps%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fazure-monitor%2Fplatform%2Frest-api-walkthrough%3C%2FA%3E)%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CPRE%20class%3D%22lia-code-sample%20language-applescript%22%3E%3CCODE%3E%24subscriptionId%20%3D%20%22%7Bazure-subscription-id%7D%22%0A%24resourceGroupName%20%3D%20%22%7Bresource-group-name%7D%22%0A%0A%23%20Authenticate%20to%20a%20specific%20Azure%20subscription.%0AConnect-AzAccount%20-SubscriptionId%20%24subscriptionId%0A%0A%23%20Password%20for%20the%20service%20principal%0A%24pwd%20%3D%20%22%7Bservice-principal-password%7D%22%0A%24secureStringPassword%20%3D%20ConvertTo-SecureString%20-String%20%24pwd%20-AsPlainText%20-Force%0A%0A%23%20Create%20a%20new%20Azure%20AD%20application%0A%24azureAdApplication%20%3D%20New-AzADApplication%20%60%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20-DisplayName%20%22My%20Azure%20Monitor%22%20%60%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20-HomePage%20%22https%3A%2F%2Flocalhost%2Fazure-monitor%22%20%60%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20-IdentifierUris%20%22https%3A%2F%2Flocalhost%2Fazure-monitor%22%20%60%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20-Password%20%24secureStringPassword%0A%0A%23%20Create%20a%20new%20service%20principal%20associated%20with%20the%20designated%20application%0ANew-AzADServicePrincipal%20-ApplicationId%20%24azureAdApplication.ApplicationId%0A%0A%23%20Assign%20Reader%20role%20to%20the%20newly%20created%20service%20principal%0ANew-AzRoleAssignment%20-RoleDefinitionName%20Reader%20%60%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20-ServicePrincipalName%20%24azureAdApplication.ApplicationId.Guid%3C%2FCODE%3E%3C%2FPRE%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%20style%3D%22margin%3A%200in%3B%20font-family%3A%20Calibri%3B%20font-size%3A%2011.0pt%3B%22%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%20style%3D%22margin%3A%200in%3B%20font-family%3A%20Calibri%3B%20font-size%3A%2011.0pt%3B%22%3EFill%20the%20gasp.%20For%20example%2C%20suppose%20my%20workspace%20was%20named%20as%3A%26nbsp%3Bsynapseworkspace_Demo.%20So%20this%20part%20of%20the%20script%26nbsp%3B%20you%20should%20add%20your%20workspace%20name%20as%20my%20example%3A%26nbsp%3B%26nbsp%3B%3CSTRONG%3Ehttps%3A%2F%2F%3CFONT%20color%3D%22%230000FF%22%3Esynapseworkspace_Demo%3C%2FFONT%3E.dev.azuresynapse.net%3C%2FSTRONG%3E%3C%2FP%3E%0A%3CP%20style%3D%22margin%3A%200in%3B%20font-family%3A%20Calibri%3B%20font-size%3A%2011.0pt%3B%22%3ETake%20note%20of%20the%3CSTRONG%3E%20password%3C%2FSTRONG%3E%20that%20you%20create%20and%20define%20here%3CSTRONG%3E%3A%26nbsp%3B%3C%2FSTRONG%3E%3CFONT%20color%3D%22%230000FF%22%3E%24pwd%20%3D%20%22%7Bservice-principal-password%7D%22%3C%2FFONT%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CPRE%20class%3D%22lia-code-sample%20language-applescript%22%3E%3CCODE%3E%23%20Create%20a%20new%20Azure%20AD%20application%0A%24azureAdApplication%20%3D%20New-AzADApplication%20%60%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20-DisplayName%20%22APP_synapseworkspace%22%20%60%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20-HomePage%20%22https%3A%2F%2Fsynapseworkspace_Demo.dev.azuresynapse.net%22%20%60%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20-IdentifierUris%20%22https%3A%2F%2Fsynapseworkspace_Demo.dev.azuresynapse.net%22%20%60%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20-Password%20%24secureStringPassword%0A%0A%3C%2FCODE%3E%3C%2FPRE%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%20style%3D%22margin%3A%200in%3B%20font-family%3A%20Calibri%3B%20font-size%3A%2011.0pt%3B%22%3EAfter%20that%2C%20you%20will%20execute%20some%20steps%20to%20actually%20invoke%20the%20API.%20The%20example%20is%20described%20here%3A%3C%2FP%3E%0A%3CP%20style%3D%22margin%3A%200in%3B%20font-family%3A%20Calibri%3B%20font-size%3A%2011.0pt%3B%22%3E%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fdata-factory%2Fquickstart-create-data-factory-rest-api%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noopener%20noreferrer%20noopener%20noreferrer%22%3Ehttps%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fdata-factory%2Fquickstart-create-data-factory-rest-api%3C%2FA%3E%3C%2FP%3E%0A%3CP%20style%3D%22margin%3A%200in%3B%20font-family%3A%20Calibri%3B%20font-size%3A%2011.0pt%3B%22%3EBut%20the%20Synapse%20APIs%20are%20here%3A%3C%2FP%3E%0A%3CP%20style%3D%22margin%3A%200in%3B%20font-family%3A%20Calibri%3B%20font-size%3A%2011.0pt%3B%22%3E%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Frest%2Fapi%2Fsynapse%2Fdata-plane%2Fpipeline%2Fcreatepipelinerun%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noopener%20noreferrer%20noopener%20noreferrer%22%3Ehttps%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Frest%2Fapi%2Fsynapse%2Fdata-plane%2Fpipeline%2Fcreatepipelinerun%3C%2FA%3E%3C%2FP%3E%0A%3CP%20style%3D%22margin%3A%200in%3B%20font-family%3A%20Calibri%3B%20font-size%3A%2011.0pt%3B%22%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%20style%3D%22margin%3A%200in%3B%20font-family%3A%20Calibri%3B%20font-size%3A%2011.0pt%3B%22%3EMy%20script%20will%20run%20with%20the%20following%3A%3C%2FP%3E%0A%3CP%20style%3D%22margin%3A%200in%3B%20font-family%3A%20Calibri%3B%20font-size%3A%2011.0pt%3B%22%3E%3CSTRONG%3Ehttps%3A%2F%2F%3CFONT%20color%3D%22%230000FF%22%3Esynapseworkspace_Demo%3C%2FFONT%3E.dev.azuresynapse.net%2Fpipelines%2F%3CFONT%20color%3D%22%230000FF%22%3EUser_not_test%3C%2FFONT%3E%2FcreateRun%3Fapi-version%3D2018-06-01%3C%2FSTRONG%3E%3C%2FP%3E%0A%3CP%20style%3D%22margin%3A%200in%3B%20font-family%3A%20Calibri%3B%20font-size%3A%2011.0pt%3B%22%3EDo%20not%20forget%20to%20add%20the%20password%20that%20you%20defined%3A%26nbsp%3B%3C%2FP%3E%0A%3CP%20style%3D%22margin%3A%200in%3B%20font-family%3A%20Calibri%3B%20font-size%3A%2011.0pt%3B%22%3E%3CSTRONG%3E%24%3C%2FSTRONG%3Epwd%20%3CSTRONG%3E%3D%20%22Password%20that%20you%20defined%20on%20the%20previous%20step%22%3C%2FSTRONG%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CPRE%20class%3D%22lia-code-sample%20language-applescript%22%3E%3CCODE%3E%24pwd%20%3D%20%22Password%20that%20you%20defined%20on%20the%20previous%20step%22%0A%24azureAdApplication%20%3D%20Get-AzADApplication%20-IdentifierUri%20%22https%3A%2F%2FYourWorkspaceName.dev.azuresynapse.net%22%0A%0A%24subscription%20%3D%20Get-AzSubscription%20-SubscriptionId%20%24subscriptionId%0A%0A%24clientId%20%3D%20%24azureAdApplication.ApplicationId.Guid%0A%24tenantId1%20%3D%20%24subscription.TenantId%0A%24authUrl%20%3D%20%22https%3A%2F%2Flogin.microsoftonline.com%2F%24%7BtenantId1%7D%2Foauth2%2Ftoken%22%0A%24cred%20%3D%20New-Object%20-TypeName%20Microsoft.IdentityModel.Clients.ActiveDirectory.ClientCredential%20-ArgumentList%20(%24clientId%2C%20%24pwd)%0A%0A%24AuthContext%20%3D%20%5BMicrosoft.IdentityModel.Clients.ActiveDirectory.AuthenticationContext%5D%24authUrl%20%0A%24result%20%3D%20%24AuthContext.AcquireTokenAsync(%22https%3A%2F%2Fdev.azuresynapse.net%22%2C%20%24cred).GetAwaiter().GetResult()%0A%0A%23%20Build%20an%20array%20of%20HTTP%20header%20values%0A%24authHeader%20%3D%20%40%7B%0A'Content-Type'%3D'application%2Fjson'%0A'Accept'%3D'application%2Fjson'%0A'Authorization'%3D%24result.CreateAuthorizationHeader()%0A%7D%0A%0A%0A%24request%20%3D%20%22https%3A%2F%2FYourWorkspaceName.dev.azuresynapse.net%2Fpipelines%2FYourPipelineName%2FcreateRun%3Fapi-version%3D2018-06-01%22%0A%24body%20%3D%20%20%40%22%0A%7B%0A%20%20%20%20%22name%22%3A%20%22YourWorkspaceName%22%2C%0A%20%20%20%20%22location%22%3A%20%22Region%22%2C%0A%20%20%20%20%22properties%22%3A%20%7B%7D%2C%0A%20%20%20%20%22identity%22%3A%20%7B%0A%20%20%20%20%20%20%20%20%22type%22%3A%20%22SystemAssigned%22%0A%20%20%20%20%7D%0A%7D%0A%22%40%0A%0A%0A%24response%20%3D%20Invoke-RestMethod%20-Method%20POST%20-Uri%20%24request%20-Header%20%24authHeader%20-Body%20%24body%0A%24response%20%7C%20ConvertTo-Json%0A%24runId%20%3D%20%24response.runId%3C%2FCODE%3E%3C%2FPRE%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%20style%3D%22margin%3A%200in%3B%20font-family%3A%20Calibri%3B%20font-size%3A%2011.0pt%3B%22%3EIf%20you%20face%20this%20error%3A%3C%2FP%3E%0A%3CP%20style%3D%22margin%3A%200in%3B%20font-family%3A%20Calibri%3B%20font-size%3A%2011.0pt%3B%22%3E%3CFONT%20color%3D%22%23FF0000%22%3EInvoke-RestMethod%3A%20%7B%22error%22%3A%7B%22code%22%3A%22Unauthorized%22%2C%22message%22%3A%22The%20principal%20'some%20number%20display'%20does%20not%20have%20the%20necessary%20permissions%20to%20perform%20this%20operation.%20%22%7D%7D%3C%2FFONT%3E%3C%2FP%3E%0A%3CP%20style%3D%22margin%3A%200in%3B%20font-family%3A%20Calibri%3B%20font-size%3A%2011.0pt%3B%22%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%20style%3D%22margin%3A%200in%3B%20font-family%3A%20Calibri%3B%20font-size%3A%2011.0pt%3B%22%3EGo%20back%20to%20you%20synapse%20studio%20-%26gt%3B%20open%20Monitoring%20-%26gt%3B%20access%20control%20and%20be%20sure%20of%202%20things%3A%3C%2FP%3E%0A%3CP%20style%3D%22margin%3A%200in%3B%20font-family%3A%20Calibri%3B%20font-size%3A%2011.0pt%3B%22%3E1)%20The%20user%26nbsp%3B%20that%20will%20start%20the%20rest%20API%20needs%20Workspace%20admin%20permission%3C%2FP%3E%0A%3CP%20style%3D%22margin%3A%200in%3B%20font-family%3A%20Calibri%3B%20font-size%3A%2011.0pt%3B%22%3E2)The%20APP%20that%20you%20register%20needs%20workspace%20admin%20permissions%20and%20to%20satisfy%20this%20requisite%3A%20Copy%20the%20number%20displayed%20on%20the%20error%20and%20add%20the%20permission%20like%20figure%202%3A%3C%2FP%3E%0A%3CP%20style%3D%22margin%3A%200in%3B%20font-family%3A%20Calibri%3B%20font-size%3A%2011.0pt%3B%22%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20image-alt%3D%22service%20principal.png%22%20style%3D%22width%3A%20999px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F219617i0734B2609F6950FF%2Fimage-size%2Flarge%3Fv%3D1.0%26amp%3Bpx%3D999%22%20title%3D%22service%20principal.png%22%20alt%3D%22service%20principal.png%22%20%2F%3E%3C%2FSPAN%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20image-alt%3D%22permission.png%22%20style%3D%22width%3A%20999px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F219618i0A4588CB7F04E00C%2Fimage-size%2Flarge%3Fv%3D1.0%26amp%3Bpx%3D999%22%20title%3D%22permission.png%22%20alt%3D%22permission.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%20style%3D%22margin%3A%200in%3B%20font-family%3A%20Calibri%3B%20font-size%3A%2011.0pt%3B%22%3EFigure%202%20Permission%3C%2FP%3E%0A%3CP%20style%3D%22margin%3A%200in%3B%20font-family%3A%20Calibri%3B%20font-size%3A%2011.0pt%3B%22%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%20style%3D%22margin%3A%200in%3B%20font-family%3A%20Calibri%3B%20font-size%3A%2011.0pt%3B%22%3E%3CSTRONG%3ENote%3C%2FSTRONG%3E%3A%20Use%20the%20number%20of%20the%20principal%20service%20id%2C%20not%20the%20APP%20name%20to%20get%20this%20permission%20done.%3C%2FP%3E%0A%3CP%20style%3D%22margin%3A%200in%3B%20font-family%3A%20Calibri%3B%20font-size%3A%2011.0pt%3B%22%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%20style%3D%22margin%3A%200in%3B%20font-family%3A%20Calibri%3B%20font-size%3A%2011.0pt%3B%22%3EYou%20could%20also%20monitor%20the%20pipeline%20execution%20adding%20this%20piece%20of%20code%3A%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CPRE%20class%3D%22lia-code-sample%20language-applescript%22%3E%3CCODE%3Ewhile%20(%24True)%20%7B%0A%20%20%20%20%24response%20%3D%20Invoke-RestMethod%20-Method%20GET%20-Uri%20%22https%3A%2F%2FYourWorkspaceName.dev.azuresynapse.net%2Fpipelineruns%2F%24%7BrunId%7D%3Fapi-version%3D2018-06-01%22%20-Header%20%24authHeader%0A%20%20%20%20Write-Host%20%20%22Pipeline%20run%20status%3A%20%22%20%24response.Status%20-foregroundcolor%20%22Yellow%22%0A%0A%20%20%20%20if%20(%24response.Status%20-eq%20%22InProgress%22)%20%7B%0A%20%20%20%20%20%20%20%20Start-Sleep%20-Seconds%2015%0A%20%20%20%20%7D%0A%20%20%20%20else%20%7B%0A%20%20%20%20%20%20%20%20%24response%20%7C%20ConvertTo-Json%0A%20%20%20%20%20%20%20%20break%0A%20%20%20%20%7D%0A%7D%0A%0A%3C%2FCODE%3E%3C%2FPRE%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%20style%3D%22margin%3A%200in%3B%20font-family%3A%20Calibri%3B%20font-size%3A%2011.0pt%3B%22%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%20style%3D%22margin%3A%200in%3B%20font-family%3A%20Calibri%3B%20font-size%3A%2011.0pt%3B%22%3EHere%20is%20the%20execution%20-%20Figure%204%3A%3C%2FP%3E%0A%3CP%20style%3D%22margin%3A%200in%3B%20font-family%3A%20Calibri%3B%20font-size%3A%2011.0pt%3B%22%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20image-alt%3D%22monitoring.png%22%20style%3D%22width%3A%20482px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F219622iC56C3EAC4F342DCA%2Fimage-size%2Flarge%3Fv%3D1.0%26amp%3Bpx%3D999%22%20title%3D%22monitoring.png%22%20alt%3D%22monitoring.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%20style%3D%22margin%3A%200in%3B%20font-family%3A%20Calibri%3B%20font-size%3A%2011.0pt%3B%22%3EFigure%204%20Execution%3C%2FP%3E%0A%3CP%20style%3D%22margin%3A%200in%3B%20font-family%3A%20Calibri%3B%20font-size%3A%2011.0pt%3B%22%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%20style%3D%22margin%3A%200in%3B%20font-family%3A%20Calibri%3B%20font-size%3A%2011.0pt%3B%22%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%20style%3D%22margin%3A%200in%3B%20font-family%3A%20Calibri%3B%20font-size%3A%2011.0pt%3B%22%3EThanks%2C%20Dan%20Rosales%20for%20help%20during%20this%20process.%3C%2FP%3E%0A%3CP%20style%3D%22margin%3A%200in%3B%20font-family%3A%20Calibri%3B%20font-size%3A%2011.0pt%3B%22%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%20style%3D%22margin%3A%200in%3B%20font-family%3A%20Calibri%3B%20font-size%3A%2011.0pt%3B%22%3EThat%20is%20it!%3C%2FP%3E%0A%3CP%20style%3D%22margin%3A%200in%3B%20font-family%3A%20Calibri%3B%20font-size%3A%2011.0pt%3B%22%3ELiliam%26nbsp%3B%3C%2FP%3E%0A%3CP%20style%3D%22margin%3A%200in%3B%20font-family%3A%20Calibri%3B%20font-size%3A%2011.0pt%3B%22%3EUK%20Engineer%3C%2FP%3E%0A%3CP%20style%3D%22margin%3A%200in%3B%20font-family%3A%20Calibri%3B%20font-size%3A%2011.0pt%3B%22%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%20style%3D%22margin%3A%200in%3B%20font-family%3A%20Calibri%3B%20font-size%3A%2011.0pt%3B%22%3E%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-TEASER%20id%3D%22lingo-teaser-1684836%22%20slang%3D%22en-US%22%3E%3CP%3EThis%20is%20a%20step%20by%20step%20how%20to%20start%20the%20pipeline%20from%20the%20rest%20api%20using%20power%20shell.%3C%2FP%3E%3C%2FLINGO-TEASER%3E%3CLINGO-LABS%20id%3D%22lingo-labs-1684836%22%20slang%3D%22en-US%22%3E%3CLINGO-LABEL%3ESynapse%20Pipelines%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3ESynapse%20Support%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E
Microsoft

Another day another case.

It took me a while to follow this step by step by using the new Synapse APIs. So this post has the intention to make it easier by example.

 

 

You will need:

  • A Synapse workspace
  • An ADF pipeline that you want to start from Rest API.

 

Doc references:

https://docs.microsoft.com/en-us/azure/data-factory/quickstart-create-data-factory-rest-api

https://docs.microsoft.com/en-us/azure/azure-monitor/platform/rest-api-walkthrough

https://docs.microsoft.com/en-us/rest/api/synapse/data-plane/pipeline/createpipelinerun

 

That is my pipeline code. It is a simple one it just creates Spark Database using a notebook.

 

 

 

 

 

 

%%spark
spark.sql("CREATE DATABASE IF NOT EXISTS DB_example")

 

 

 

 

 

 

As you can see in figure 1. That is my pipeline:

pipelin_noe.png

Figure 1 Pipeline

 

The name of my pipeline is User_not_test.

I can run successfully this pipeline from Synapse Studio. But I want to run it from the Rest API, actually, that is the post idea.

 

Step by Step

The first step consists in using this documentation to register my pipeline/workspace as an application:

(https://docs.microsoft.com/en-us/azure/azure-monitor/platform/rest-api-walkthrough)

 

 

 

 

 

$subscriptionId = "{azure-subscription-id}"
$resourceGroupName = "{resource-group-name}"

# Authenticate to a specific Azure subscription.
Connect-AzAccount -SubscriptionId $subscriptionId

# Password for the service principal
$pwd = "{service-principal-password}"
$secureStringPassword = ConvertTo-SecureString -String $pwd -AsPlainText -Force

# Create a new Azure AD application
$azureAdApplication = New-AzADApplication `
                        -DisplayName "My Azure Monitor" `
                        -HomePage "https://localhost/azure-monitor" `
                        -IdentifierUris "https://localhost/azure-monitor" `
                        -Password $secureStringPassword

# Create a new service principal associated with the designated application
New-AzADServicePrincipal -ApplicationId $azureAdApplication.ApplicationId

# Assign Reader role to the newly created service principal
New-AzRoleAssignment -RoleDefinitionName Reader `
                          -ServicePrincipalName $azureAdApplication.ApplicationId.Guid

 

 

 

 

 

 

Fill the gasp. For example, suppose my workspace was named as synapseworkspace_Demo. So this part of the script  you should add your workspace name as my example:  https://synapseworkspace_Demo.dev.azuresynapse.net

Take note of the password that you created and defined here$pwd = "{service-principal-password}"

 

 

 

 

 

 

# Create a new Azure AD application
$azureAdApplication = New-AzADApplication `
                        -DisplayName "APP_synapseworkspace" `
                        -HomePage "https://synapseworkspace_Demo.dev.azuresynapse.net" `
                        -IdentifierUris "https://synapseworkspace_Demo.dev.azuresynapse.net" `
                        -Password $secureStringPassword

 

 

 

 

 

 

After that, you will execute some steps to actually invoke the API. The example is described here:

https://docs.microsoft.com/en-us/azure/data-factory/quickstart-create-data-factory-rest-api

But the Synapse APIs are here:

https://docs.microsoft.com/en-us/rest/api/synapse/data-plane/pipeline/createpipelinerun

 

My script will run with the following:

https://synapseworkspace_Demo.dev.azuresynapse.net/pipelines/User_not_test/createRun?api-version=2018-06-01

Do not forget to add the password that you defined: 

$pwd = "Password that you defined on the previous step"

 

 

 

 

 

 

$pwd = "Password that you defined on the previous step"
$azureAdApplication = Get-AzADApplication -IdentifierUri "https://YourWorkspaceName.dev.azuresynapse.net"

$subscription = Get-AzSubscription -SubscriptionId $subscriptionId

$clientId = $azureAdApplication.ApplicationId.Guid
$tenantId1 = $subscription.TenantId
$authUrl = "https://login.microsoftonline.com/${tenantId1}/oauth2/token"
$cred = New-Object -TypeName Microsoft.IdentityModel.Clients.ActiveDirectory.ClientCredential -ArgumentList ($clientId, $pwd)

$AuthContext = [Microsoft.IdentityModel.Clients.ActiveDirectory.AuthenticationContext]$authUrl 
$result = $AuthContext.AcquireTokenAsync("https://dev.azuresynapse.net", $cred).GetAwaiter().GetResult()

# Build an array of HTTP header values
$authHeader = @{
'Content-Type'='application/json'
'Accept'='application/json'
'Authorization'=$result.CreateAuthorizationHeader()
}


$request = "https://YourWorkspaceName.dev.azuresynapse.net/pipelines/YourPipelineName/createRun?api-version=2018-06-01"
$body =  @"
{
    "name": "YourWorkspaceName",
    "location": "Region",
    "properties": {},
    "identity": {
        "type": "SystemAssigned"
    }
}
"@


$response = Invoke-RestMethod -Method POST -Uri $request -Header $authHeader -Body $body
$response | ConvertTo-Json
$runId = $response.runId

 

 

 

 

 

 

If you face this error:

Invoke-RestMethod: {"error":{"code":"Unauthorized","message":"The principal 'some number display' does not have the necessary permissions to perform this operation. "}}

 

Go back to you synapse studio -> open Monitoring -> access control and be sure of 2 things:

1) The user  that will start the rest API needs Workspace admin permission

2)The APP that you register needs workspace admin permissions and to satisfy this requisite: Copy the number displayed on the error and add the permission like figure 2:

service principal.pngpermission.png

Figure 2 Permission

 

Note: Use the number of the principal service id, not the APP name to get this permission done.

 

You could also monitor the pipeline execution adding this piece of code:

 

 

 

 

 

while ($True) {
    $response = Invoke-RestMethod -Method GET -Uri "https://YourWorkspaceName.dev.azuresynapse.net/pipelineruns/${runId}?api-version=2018-06-01" -Header $authHeader
    Write-Host  "Pipeline run status: " $response.Status -foregroundcolor "Yellow"

    if ($response.Status -eq "InProgress") {
        Start-Sleep -Seconds 15
    }
    else {
        $response | ConvertTo-Json
        break
    }
}

 

 

 

 

 

 

Here is the execution - Figure 4:

monitoring.png

Figure 4 Execution

 

 

Thanks, Dan Rosales for help during this process.

 

That is it!

Liliam 

UK Engineer