Export to PST via Powershell

So, I've been fighting to automate using powershell to export a PST from Office 365 off and on for a while.

AzCopy worked great for Sharepoint/OneDrive exports, but exchange data all came down as ".batch" files, which appear to need further post-processing to collect everything into a PST. Someone better than I might know how to piece them together, but I just moved on.

It's a bit of a workaround but here's what I did to automate the PST download.

When you manually download the export using the export tool, the click-once popup downloads their "Unified Export Tool" and populates it with what it needs to get it going. While it's open, you can right-click the process in task manager and click "Open file location". This will take you to the files their tool uses, in your appdata. For me, this location was "%LOCALAPPDATA%\Apps\2.0\QBKH9EZX.XMP\7R4KYC02.WJ7\micr..tool_51a5b647dacf4059_000f.0014_a4b60912e622c727\"

I'm sure Microsoft updates the tool every now and then, so this path may not work indefinitely.

Anyways, I just copied everything to a new folder (I don't recommend doing this, i provide an example to find the file automatically in the script below) and ran the "microsoft.office.client.discovery.unifiedexporttool.exe" which returned the parameters that it accepts. Much like AzCopy, it looks for the URL, Token, and a place to drop the files.

Here's what I slapped together to automatically download the export from Office 365:

The code below has been edited since this post was created to include improvements to the original code and provide a start to finish example to export and download a PST. If you want to use this as a user off-boarding script you'll need to add those tasks as well. Everyone has their own processes, but Tom Aguero's post below has some good examples too.

# Note that I'm not validating any input or providing any adequate error handling.
# This is just an example, you'll need to add these in yourself.

Param(
    [Parameter(Mandatory=$True, HelpMessage='Enter the email address that you want to export')]
    $Mailbox,
    [Parameter(Mandatory=$True, HelpMessage='Enter the URL for the user''s OneDrive here. If you don''t enter one, this will be skipped.')]
    $OneDriveURL,
    [Parameter(Mandatory=$True, HelpMessage='Enter the path where you want to save the PST file. !NO TRAILING BACKSLASH!')]
    $ExportLocation # = ""# you can un-comment the = "" to set a default for this parameter.
)

# Create a search name. You can change this to suit your preference
$SearchName = "$Mailbox PST"

# I'm using the Exchange Online Powershell Module v2. You can install it from an admin session with the following command: Install-Module ExchangeOnlineManagement
Write-Host "Connecting to Exchange Online. Enter your admin credentials in the pop-up (pop-under?) window."
Connect-IPPSSession

Write-Host "Creating compliance search..."
New-ComplianceSearch -Name $SearchName -ExchangeLocation $Mailbox -SharePointLocation $OneDriveURL -AllowNotFoundExchangeLocationsEnabled $true #Create a content search, including the the entire contents of the user's email and onedrive. If you didn't provide a OneDrive URL, or it wasn't valid, it will be ignored.
Write-Host "Starting compliance search..."
Start-ComplianceSearch -Identity $SearchName #Start the search created above
Write-Host "Waiting for compliance search to complete..."
for ($SearchStatus;$SearchStatus -notlike "Completed";){ #Wait then check if the search is complete, loop until complete
    Start-Sleep -s 2
    $SearchStatus = Get-ComplianceSearch $SearchName | Select-Object -ExpandProperty Status #Get the status of the search
    Write-Host -NoNewline "." # Show some sort of status change in the terminal
}
Write-Host "Compliance search is complete!"
Write-Host "Creating export from the search..."
New-ComplianceSearchAction -SearchName $SearchName -Export -Format FxStream -ExchangeArchiveFormat PerUserPst -Scope BothIndexedAndUnindexedItems -EnableDedupe $true -SharePointArchiveFormat IndividualMessage -IncludeSharePointDocumentVersions $true 
Start-Sleep -s 5 # Arbitrarily wait 5 seconds to give microsoft's side time to create the SearchAction before the next commands try to run against it. I /COULD/ do a for loop and check, but it's really not worth it.

# Check if the export tool is installed for the user, and download if not.
While (-Not ((Get-ChildItem -Path $($env:LOCALAPPDATA + "\Apps\2.0\") -Filter microsoft.office.client.discovery.unifiedexporttool.exe -Recurse).FullName | Where-Object{ $_ -notmatch "_none_" } | Select-Object -First 1)){
    Write-Host "Downloading Unified Export Tool ."
    Write-Host "This is installed per-user by the Click-Once installer."
    # Credit to Jos Verlinde for his code in Load-ExchangeMFA in the Powershell Gallery! All I've done is update the manifest url and remove all the comments
    # Ripped from https://www.powershellgallery.com/packages/Load-ExchangeMFA/1.2
    # In case anyone else has any ClickOnce applications they'd like to automate the install for:
    # If you're looking for where to find a manifest URL, once you have run the ClickOnce application at least once on your computer, the url for the application manifest can be found in the Windows Registry at "HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Uninstall" (yes, CTR apps are installed per-user).
    # Look through the keys with names that are 16 characters long hex strings. They'll have a string value (REG_SZ) named either "ShortcutAppId" or "UrlUpdateInfo" that contains the URL as the first part of the string.
    $Manifest = "https://complianceclientsdf.blob.core.windows.net/v16/Microsoft.Office.Client.Discovery.UnifiedExportTool.application"
    $ElevatePermissions = $true
    Try {
        Add-Type -AssemblyName System.Deployment
        Write-Host "Starting installation of ClickOnce Application $Manifest "
        $RemoteURI = [URI]::New( $Manifest , [UriKind]::Absolute)
        if (-not  $Manifest){
            throw "Invalid ConnectionUri parameter '$ConnectionUri'"
        }
        $HostingManager = New-Object System.Deployment.Application.InPlaceHostingManager -ArgumentList $RemoteURI , $False
        Register-ObjectEvent -InputObject $HostingManager -EventName GetManifestCompleted -Action { 
            new-event -SourceIdentifier "ManifestDownloadComplete"
        } | Out-Null
        Register-ObjectEvent -InputObject $HostingManager -EventName DownloadApplicationCompleted -Action { 
            new-event -SourceIdentifier "DownloadApplicationCompleted"
        } | Out-Null
        $HostingManager.GetManifestAsync()
        $event = Wait-Event -SourceIdentifier "ManifestDownloadComplete" -Timeout 15
        if ($event ) {
            $event | Remove-Event
            Write-Host "ClickOnce Manifest Download Completed"
            $HostingManager.AssertApplicationRequirements($ElevatePermissions)
            $HostingManager.DownloadApplicationAsync()
            $event = Wait-Event -SourceIdentifier "DownloadApplicationCompleted" -Timeout 60
            if ($event ) {
                $event | Remove-Event
                Write-Host "ClickOnce Application Download Completed"
            }
            else {
                Write-error "ClickOnce Application Download did not complete in time (60s)"
            }
        }
        else {
            Write-error "ClickOnce Manifest Download did not complete in time (15s)"
        }
    }
    finally {
        Get-EventSubscriber|? {$_.SourceObject.ToString() -eq 'System.Deployment.Application.InPlaceHostingManager'} | Unregister-Event
    }
}

# Find the Unified Export Tool's location and create a variable for it
$ExportExe = ((Get-ChildItem -Path $($env:LOCALAPPDATA + "\Apps\2.0\") -Filter microsoft.office.client.discovery.unifiedexporttool.exe -Recurse).FullName | Where-Object{ $_ -notmatch "_none_" } | Select-Object -First 1)

# Gather the URL and Token from the export in order to start the download
# We only need the ContainerURL and SAS Token at a minimum but we're also pulling others to help with tracking the status of the export.
$ExportName = $SearchName + "_Export"
$ExportDetails = Get-ComplianceSearchAction -Identity $ExportName -IncludeCredential -Details # Get details for the export action
# This method of splitting the Container URL and Token from $ExportDetails is thanks to schmeckendeugler from reddit: https://www.reddit.com/r/PowerShell/comments/ba4fpu/automated_download_of_o365_inbox_archive/
# I was using Convert-FromString before, which was slow and terrible. His way is MUCH better.
$ExportDetails = $ExportDetails.Results.split(";")
$ExportContainerUrl = $ExportDetails[0].trimStart("Container url: ")
$ExportSasToken = $ExportDetails[1].trimStart(" SAS token: ")
$ExportEstSize = ($ExportDetails[18].TrimStart(" Total estimated bytes: ") -as [double])
$ExportTransferred = ($ExportDetails[20].TrimStart(" Total transferred bytes: ") -as [double])
$ExportProgress = $ExportDetails[22].TrimStart(" Progress: ").TrimEnd("%")
$ExportStatus = $ExportDetails[25].TrimStart(" Export status: ")

# Download the exported files from Office 365
Write-Host "Initiating download"
Write-Host "Saving export to: " + $ExportLocation
$Arguments = "-name ""$SearchName""","-source ""$ExportContainerUrl""","-key ""$ExportSasToken""","-dest ""$ExportLocation""","-trace true"
Start-Process -FilePath "$ExportExe" -ArgumentList $Arguments

# The export is now downloading in the background. You can find it in task manager. Let's monitor the progress.
# If you want to use this as part of a user offboarding script, add your edits above here - Exports can take a lot of time...
# You can even comment this entire section and exit the script if you dont feel the need to monitor the download, it will keep downloading in the background even without the script running.
# This is only monitoring if the process exists, which means if you run multiple exports, this will stay running until they all complete.
# We could possibly utilize sysinternals handle.exe to identify the PID of the process writing to the $Exportlocation and monitor for that specifically, but I'm trying to limit external applications in this example script.
#
# Just an FYI, the export progress is how much data Microsoft has copied into PSTs from the compliance search, not how much the export tool has downloaded.
# We only know the actual size of the download after the $ExportProgress is 100% and $ExportStatus is Completed
# The actual final size of the download is then reflected in $ExportTransferred. Even then, our progress is still a bit inaccurate due to the extra log and temp files created locally, which will probably cause the progress to show over 100%
# We could make this a bit more accurate by just collecting the size of PSTs and files under the OneDrive folder, but I think this brings us close enough for most situations.
while(Get-Process microsoft.office.client.discovery.unifiedexporttool -ErrorAction SilentlyContinue){
    $Downloaded = Get-ChildItem $ExportLocation\$SearchName -Recurse | Measure-Object -Property Length -Sum | Select-Object -ExpandProperty Sum
    Write-Progress -Id 1 -Activity "Export in Progress" -Status "Complete..." -PercentComplete $ExportProgress
    if ("Completed" -notlike $ExportStatus){Write-Progress -Id 2 -Activity "Download in Progress" -Status "Estimated Complete..." -PercentComplete ($Downloaded/$ExportEstSize*100) -CurrentOperation "$Downloaded/$ExportEstSize bytes downloaded."}
    else {Write-Progress -Id 2 -Activity "Download in Progress" -Status "Complete..." -PercentComplete ($Downloaded/$ExportEstSize*100) -CurrentOperation "$Downloaded/$ExportTransferred bytes downloaded."}
    Start-Sleep 60
    $ExportDetails = Get-ComplianceSearchAction -Identity $ExportName -IncludeCredential -Details # Get details for the export action
    $ExportDetails = $ExportDetails.Results.split(";")
    $ExportEstSize = ($ExportDetails[18].TrimStart(" Total estimated bytes: ") -as [double])
    $ExportTransferred = ($ExportDetails[20].TrimStart(" Total transferred bytes: ") -as [double])
    $ExportProgress = $ExportDetails[22].TrimStart(" Progress: ").TrimEnd("%")
    $ExportStatus = $ExportDetails[25].TrimStart(" Export status: ")
    Write-Host -NoNewline " ."
}
Write-Host "Download Complete!"
pause

The nice part about this is the unified export tool includes it's own wait loop that checks the status of the export, so you don't have to create your own and worry about credential timeout, etc. Thanks to this, you can kick off the download and move on to other tasks in your off-boarding script.

In production, I've written this to execute the download on a separate terminal using Invoke-Command and email the team a transcript when the download's done.

Drawbacks:

The only way you know it's running is the process shows up in task manager.

The only way you know it's done is it's not in task manager. We can estimate how far along it is, but it isn't perfectly accurate. You definitely want to rely on the trace files to know if everything went OK.

If the export gets stopped for some reason (lost internet, restart due to windows updates, etc.) it does not resume the download when started again. I haven't been able to figure out how to get that to work.

Anyways, I wanted to give back to the thread that helped me get this far.