How to delete pipeline tags with special characters?
I want to delete specific tags attached to Azure pipeline builds, for example "hello: world". I've come to the conclusion that the ADO REST API endpoint for handling Tag deletions cannot parse special characters in the URL's slug i.e. colons and whitespaces. According to thehttps://learn.microsoft.com/en-us/rest/api/azure/devops/build/tags/delete-build-tag?view=azure-devops-rest-7.1, the tag should be specified in the URL slug, followed by query string parameters if applicable. I tried the following: 1. If I insert the tag directly into the URL it will look like this: https://dev.azure.com/organisation/project/_apis/build/builds/1234567/tags/hello: world?api-version=7.1 This returns: "Response status code does not indicate success: 400 (Bad Request)." 2. But if I encode my slug using `[System.Web.HttpUtility]::UrlEncode($tag)`, the URL looks like this: https://dev.azure.com/organisation/project/_apis/build/builds/1234567/tags/hello%3a+world?api-version=7.1 This returns "Response status code does not indicate success: 404 (Not Found)." So it seems the encoding might have worked, although it appears to be searching for a tag without decoding the URL first? Does anyone know if there is a way for deleting tags with special characters? I have over 1600+ tags that need to be deleted so manually doing this through the UI would not be a viable option. EDIT: I just realised the documentation has a small note saying: This API will not work for tags with special characters. To remove tags with special characters, use the PATCH method instead (in 6.0+) Tried the PATCH method instead of DELETE and still not working. And there's no examples provided in the docs.PowerShell script to delete all Containers from a Storage Account
After move the BootDiag settings out of the Custom Storage Account, the original Storage Account used for are still consuming space for nothing. This is part of the standard Clean Up stream need to be consider into the FinOps Plan. This script will help you to clean these Storage Accounts quickly and avoid cost paid for nothing. Connect-AzAccount #Your Subscription $MySubscriptionToClean = "MyGuid-MyGuid-MyGuid-MyGuid-MyGuid" $MyStorageAccountName = "MyStorageAccountForbootdiags" $MyStorageAccountKey = "MySAKeyWithAllCodeProvidedByYourStorageAccountSetting+MZ3cUvdQ==" $ContainerStartName = "bootdiag*" #Set subscription ID Set-AzContext -Subscription $MySubscriptionToClean Get-AzContext $ctx = New-AzStorageContext -StorageAccountName $MyStorageAccountName -StorageAccountKey $MyStorageAccountKey $myContainers = Get-AzStorageContainer -Name $ContainerStartName -Context $ctx -MaxCount 1000 foreach($mycontainer in $myContainers) { Remove-AzStorageContainer -Name $mycontainer.Name -Force -Context $ctx } I used this script to remove millions of BootDiag Containers from several Storage Accounts. You can also use it for any other cleanup use case if you need it. Fab
Powershell Script to remove all Blobs from Storage account
With large number of Blobs in Storage Account, the manual cleanup from the Portal is more complicated and time consuming, as it's per set of 10'000. This script is simple and and can be executed in background to clean all items from a defined Blob Container. You have to specify the Storage Account connection string and the blob container name. [string]$myConnectionString = "DefaultEndpointsProtocol=https;AccountName=YourStorageAccountName;AccountKey=YourKeyFromStorageAccountConnectionString;EndpointSuffix=core.windows.net" [string]$ContainerName = "YourBlobContainerName" [int]$blobCountAfter = 0 [int]$blobCountBefore = 0 $context = New-AzStorageContext -ConnectionString $myConnectionString $blobCountBefore = (Get-AzStorageBlob -Container $ContainerName -Context $context).Count Write-Host "Total number of blobs in the container Before deletion: $blobCount" -ForegroundColor Yellow Get-AzStorageBlob -Container $ContainerName -Context $context | ForEach-Object { $_ | Remove-AzureStorageBlob # or: Remove-AzureStorageBlob -ICloudBlob $_.ICloudBlob -Context $ctx } $blobCountAfter = (Get-AzStorageBlob -Container $ContainerName -Context $context).Count Write-Host "Total number of blobs in the container After deletion : $blobCount" -ForegroundColor Green It was used for large blob storage container with more than 5 millions of blob items. Sources: https://learn.microsoft.com/en-us/powershell/module/az.storage/new-azstoragecontext?view=azps-13.0.0#examples https://learn.microsoft.com/en-us/answers/questions/1637785/what-is-the-easiest-way-to-find-the-total-number-o https://stackoverflow.com/questions/57119087/powershell-remove-all-blobs-in-a-container FabAzure - PowerShell script to change the Table Retention in Azure Log Analytics Workspaces
With large scale implementation of Azure, the Log Analytics Workspace volume could increase and the default value for retention is quite long if you are not changing it. This PowerShell script will help you to reset the 2 retention values applied in Workspace Tables (Live and Total). I applied a selection criteria based in name as we are using a naming convention with status (prod, vs nonprod), you can anyway adapt this part with your context. #Install-Module -Name Az -Repository PSGallery -Force Import-module Az Connect-AzAccount $RetentionDays = 30 $TotalRetentionDays = 30 $AzureRetentionDays = 90 $AzureTotalRetentionDays = 90 $namecriteria = "nonprod" $All_Az_Subscriptions = Get-AzSubscription Foreach ($Az_Subscription in $All_Az_Subscriptions) { ################################################### #Set the context Write-Host "Working on subscription ""$($Az_Subscription.Name)""" Set-AzContext -SubscriptionObject $Az_Subscription | Out-Null $AllWorkspaces = Get-AzOperationalInsightsWorkspace foreach ($myWorkspace in $AllWorkspaces) { Write-Host " ---------------", $myWorkspace.Name ,"---------------- " -foregroundcolor "gray" if ($myWorkspace.Name -match $namecriteria) { Write-Host " >>> WORKSPACE TO APPLY RETENTION ADJUSTMENT:", $myWorkspace.Name -foregroundcolor "green" if ($myWorkspace.retentionInDays -gt $RetentionDays) { Write-Host " >>> APPLYING DEFAULT RETENTION PERIOD:", $RetentionDays -foregroundcolor "yellow" Set-AzOperationalInsightsWorkspace -ResourceGroupName $myWorkspace.ResourceGroupName -Name $myWorkspace.Name -RetentionInDays $RetentionDays } $GetAllTables = Get-AzOperationalInsightsTable -ResourceGroupName $myWorkspace.ResourceGroupName -WorkspaceName $myWorkspace.Name foreach ($MyTable in $GetAllTables) { if (($MyTable.Name -eq "AzureActivity") -or ($MyTable.Name -eq "Usage")) { if (($MyTable.RetentionInDays -gt $AzureRetentionDays) -or ($MyTable.TotalRetentionInDays -gt $AzureTotalRetentionDays)) { Write-Host " >>> APPLYING SPECIFIC RETENTION PERIOD:", $AzureRetentionDays, "- TABLE:", $MyTable.Name -foregroundcolor "yellow" Update-AzOperationalInsightsTable -ResourceGroupName $MyTable.ResourceGroupName -WorkspaceName $MyTable.WorkspaceName -TableName $MyTable.Name -RetentionInDays $AzureRetentionDays -TotalRetentionInDays $AzureTotalRetentionDays } else { Write-Host " >>> NO CHANGE FOR RETENTION PERIOD FOR TABLE:", $MyTable.Name -foregroundcolor "green" } } else { if (($MyTable.RetentionInDays -gt $RetentionDays) -or ($MyTable.TotalRetentionInDays -gt $RetentionDays)) { Write-Host " >>> APPLYING NEW RETENTION PERIOD:", $RetentionDays, "- TABLE:", $MyTable.Name -foregroundcolor "yellow" Update-AzOperationalInsightsTable -ResourceGroupName $MyTable.ResourceGroupName -WorkspaceName $MyTable.WorkspaceName -TableName $MyTable.Name -RetentionInDays $RetentionDays -TotalRetentionInDays $TotalRetentionDays } else { Write-Host " >>> NO CHANGE FOR RETENTION PERIOD FOR TABLE:", $MyTable.Name -foregroundcolor "green" } } } } else { Write-Host " >>> WORKSPACE NOT CONCERNED BY THIS CHANGE:", $myWorkspace.Name -foregroundcolor "green" } } } With this script, we reduced the Workspace cost for non prod drastically maintaining only the last 30 days live without any archive. The material used for this script is: https://learn.microsoft.com/en-us/azure/azure-monitor/logs/data-retention-archive?tabs=portal-3%2Cportal-1%2Cportal-2 https://learn.microsoft.com/en-us/powershell/module/az.operationalinsights/get-azoperationalinsightsworkspace?view=azps-11.6.0 https://learn.microsoft.com/en-us/powershell/module/az.operationalinsights/update-azoperationalinsightstable?view=azps-11.6.0 Fabrice RomelardAzure - PowerShell Script to delete a specific Tag for any resources in all your Subscriptions
A classical question after many months of usage and delegation to different admin is related to the TAG Cleanup. You can be faced to a large diversity of Tags created at one moment, but not useful and mainly not maintained. This small script will help you to execute this cleanup in all your subscriptions you are in charge. Import-module Az Connect-AzAccount [string]$TagName = "YourSpecificTagKey" $TagCount = 0 $All_Az_Subscriptions = Get-AzSubscription Foreach ($Az_Subscription in $All_Az_Subscriptions) { Write-Host " " Write-Host " --------------------------------------- " Write-Host "Working on subscription ""$($Az_Subscription.Name)""" -foregroundcolor "yellow" $TagCount = 0 Set-AzContext -SubscriptionObject $Az_Subscription | Out-Null $AllTaggedresources = Get-AzResource -TagName $TagName $TagCount = $AllTaggedresources.Count Write-Host " >> TAG "" $($TagName) "" found "" $($TagCount) "" times" -foregroundcolor "green" if($TagCount -gt 0) { $AllTaggedresources.ForEach{ if ( $_.tags.ContainsKey($TagName) ) { $_.tags.Remove($TagName) } $_ | Set-AzResource -Tags $_.tags -Force } } } This script was inspired by these pages: https://stackoverflow.com/questions/54162372/how-to-fix-this-error-in-azure-powershell-can-not-remove-tag-tag-value-becaus https://learn.microsoft.com/en-us/powershell/module/az.resources/set-azresource?view=azps-11.6.0 Fabrice Romelard
Azure function app- Powershell runtime stack
Hello community, Can anyone please help me with the query i have. I have a Azure Powershell function app set up in my environment. i have 2 timer triggered functions say F1 and F2 running at 5 mins interval. They both are connecting to 2 different SharePoint sites with the same tenant using Connect-PnPOnline with Client Id and Certificates. Issue is that both of them are conflicting the run of the other function when running. Errors i get- Save conflict. Your changes conflict with those made concurrently by another user. OR List 'X' is not present in the site F2 My understanding is the function even though are in the same App can run independently of each other as separate instances. Is there a way other than updating the apps to run at 2 different times to fix this issue?
Azure Pipeline - Powershell Task Output
Hello all, I am looking to leverage Azure Pipelines to replace an existing scheduled task that I have running on an on-premise server. Today, my script connects to a SQL database using the dbatools PowerShell module, runs a few queries, and dumps the files to a specific file format. Then uploads these files to an SFTP server. My repository contains the SQL files to run, the Get-Data.ps1 script that connects to the database, retrieves the data, and exports it to the format necessary. I also have a Send-Data.ps1 which takes the files created from Get-Data.ps1 and SFTPs them to an SFTP server. I've got 1 PowerShell script to retrieve and output the data to files, and another to SFTP the files to an SFTP server. My Send-Data.ps1 script has 3 parameters - localpath, remotepath and credential. I am curious how I could have the Get-Data.ps1 script write as output the data required for "localpath" parameter, and then pass this to the Send-Data.ps1 as a 2nd task in the pipeline? Additionally, is there a recommended way to pass credentials into PowerShell scripts in a pipeline? I saw where I can leverage the library to store a username and password to pass into a script. But this isn't a PSCredential object. Would I do this by using Azure Keyvault? Could I do it using the library and the inline script part of the pipeline? Or read it in from the key vault/library, create the PSCredential object, store that as a variable(?) and pass it into the PowerShell script task? Thanks in advance for any input. Steve
