Parsing JSON with PowerShell

Q: I try to parse my JSON with the handy dandy “ConvertFrom-JSON” cmdlet but it only works in PowerShell 7, not in my good old PowerShell 5.1.  How do I get it to work everywhere? 

 

A: PS 7 parses JSON a little differently than PS 5.1 by ignoring comments and accepting less than ideal JSON so you need to clean up your JSON for PS 5.1.  Use the function below to do it for you. 

 

Java Script Object Notation (JSON) is a popular format these days for sending and receiving data with Azure.   JSON is used for sending and receiving data using Azure REST API, deploying resources to Azure using ARM templates, configure governance in Azure using Azure Policy, and much more. PowerShell is a great tool for creating and modifying JSON. PowerShell 7.1 is my preferred solution since it has additional capabilities however sometimes we still need good old PS 5.1 so it is nice to be able to accomplish anything in either version.  

 

Disclaimer: The sample scripts are not supported under any Microsoft standard support program or service. The sample scripts are provided AS IS without warranty of any kind. Microsoft further disclaims all implied warranties including, without limitation, any implied warranties of merchantability or of fitness for a particular purpose. The entire risk arising out of the use or performance of the sample scripts and documentation remains with you. In no event shall Microsoft, its authors, or anyone else involved in the creation, production, or delivery of the scripts be liable for any damages whatsoever (including, without limitation, damages for loss of business profits, business interruption, loss of business information, or other pecuniary loss) arising out of the use of or inability to use the sample scripts or documentation, even if Microsoft has been advised of the possibility of such damages. 

 

First, here is a sample file I’ll use for the example or you can download it from here. 

 

 

 

{ 
    "Accounts": { 
        "Users": [ 
            { 
                "jdoe": { 
                    "givenName": "John", //Out on leave this month  
                    "surname": "Doe", 
                    "department": "Finance" 
                }, 
                "jdoe2": { 
                    "givenName": "Jane", //Department head  
                    "surname": "Doe", 
                    "department": "Marketing" 
                }, 
                "asmith": { 
                    "givenName": "Alice", /* Sits in the corner office */ 
                    "surname": "Smith", 
                    "department": "IT" 
                } 
            } 
        ] 
    } 
} 

 

 

 

 

Alice got promoted so we need to change her department to Senior Leadership.  PowerShell makes it easy to modify JSON by converting JSON to a PSCustomObject.  The object can then be modified easily like any other object.  The object can then be exported back out using ConvertTo-Json. 

 

 

 

$myJson = Get-Content .\test.json -Raw | ConvertFrom-Json 
$myJson.Accounts.Users.asmith.department = "Senior Leadership" 
$myJson | ConvertTo-Json -Depth 4 | Out-File .\test.json  

 

 

 

 

This gives a test.json file that accomplished exactly what we wanted: 

 

 

 

{ 
    "Accounts": { 
        "Users": [ 
            { 
                "jdoe": { 
                    "givenName": "John", 
                    "surname": "Doe", 
                    "department": "Finance" 
                }, 
                "jdoe2": { 
                    "givenName": "Jane", 
                    "surname": "Doe", 
                    "department": "Marketing" 
                }, 
                "asmith": { 
                    "givenName": "Alice", 
                    "surname": "Smith", 
                    "department": "Senior Leadership" 
                } 
            } 
        ] 
    } 
}

 

 

 

 

 Now if we’re on a computer without PowerShell 7.1 we try to run the same command in PowerShell 5.1 but it fails! 

 

 

 

PS> $a = gc .\test.json -Raw | ConvertFrom-Json 
ConvertFrom-Json : Invalid object passed in, ':' or '}' expected. (17161):{ 
<the entire contents of the json file listed here> 
} 
At line:1 char:28 
+ $a = gc .\test.json -Raw | ConvertFrom-Json 
+                            ~~~~~~~~~~~~~~~~ 
    + CategoryInfo          : NotSpecified: (:) [ConvertFrom-Json], ArgumentException 
    + FullyQualifiedErrorId : System.ArgumentException,Microsoft.PowerShell.Commands.ConvertFromJsonCommand  

 

 

 

 

Comments will prevent ConvertFrom-Json from working properly in PowerShell 5.1.  I like to use this simple function to fix my JSON for me. 

 

 

 

Function Remove-Comments{ 
    param( 
        [CmdletBinding()] 
        [Parameter(Mandatory=$true,ValueFromPipelineByPropertyName=$true,Position=0)] 
        [string] 
        [ValidateScript({test-path $_})] 
        $File 
    ) 
    $CComments = "(?s)/\\*.*?\\*/" 
    $content = Get-Content $File -Raw 
    [regex]::Replace($content,$CComments,"") | Out-File $File -Force 
} 

 

 

 

 

Simply run the following to fix the json file:  

 

 

 

Remove-Comments .\test.json 

 

 

 

 

Now test.json file looks like this: 

 

 

 

{ 
    "Accounts": { 
        "Users": [ 
            { 
                "jdoe": { 
                    "givenName": "John", 
                    "surname": "Doe", 
                    "department": "Finance" 
                }, 
                "jdoe2": { 
                    "givenName": "Jane", 
                    "surname": "Doe", 
                    "department": "Marketing" 
                }, 
                "asmith": { 
                    "givenName": "Alice", 
                    "surname": "Smith", 
                    "department": "IT" 
                } 
            } 
        ] 
    } 
} 

 

 

 

 

 Now that the comments are gone the JSON can be converted into a PSCustomObject using the following just like in PS7: 

 

 

 

$myJson = Get-Content .\test.json -Raw | ConvertFrom-Json  

 

 

 

 

PowerShell is a great tool to use for manipulating JSON which is used throughout Azure.   

Have fun scripting! 

 

Additional reading:  

• 7.1: ConvertFrom-Json (Microsoft.PowerShell.Utility) - PowerShell | Microsoft Docs 
• 5.1: ConvertFrom-Json (Microsoft.PowerShell.Utility) - PowerShell | Microsoft Docs 
• 7.1: ConvertTo-Json (Microsoft.PowerShell.Utility) - PowerShell | Microsoft Docs 
• 5.1: ConvertTo-Json (Microsoft.PowerShell.Utility) - PowerShell | Microsoft Docs 

• 7.1: Test-Json (Microsoft.PowerShell.Utility) - PowerShell | Microsoft Docs 
• Azure REST API reference documentation | Microsoft Docs 
• Overview of Azure Policy - Azure Policy | Microsoft Docs 
• ARM template documentation | Microsoft Docs