SOLVED

Export to PST via Powershell

Deleted
Not applicable

I am continually expanding on my offboarding process within Orchestrator. I decided to add the export of the email so that all that has to be done is go to the Content search and download it (unless someone knows how to do that as well...). My addition works, however, instead of creating a single PST file, I end up getting the actual folders with individual email in message file format. I have tried what is correct per everything I could find (honestly, not a lot of detail on the subject). The current single line in question is:

New-ComplianceSearchAction -SearchName $SearchName -Export -ArchiveFormat PerUserPST -EnableDedupe $true 

I have tried different values for -ArchiveFormat including leaving it completely off since a single pst per user is supposed to be the default.

 

 

I will include the entire part of the script responsible for the full function in case it is supposed to be declared somewhere else (but I haven't found anything).

# Create Compliance Search - Export Email

$SearchName = "Export - " + $term.Name
New-ComplianceSearch -ExchangeLocation $term365.UserPrincipalName -Name $SearchName

# Start Compliance Search and wait to complete

Start-ComplianceSearch $SearchName
do
    {
        Start-Sleep -s 5
        $complianceSearch = Get-ComplianceSearch $SearchName
    }
while ($complianceSearch.Status -ne 'Completed')

# Create Compliance Search in exportable format
New-ComplianceSearchAction -SearchName $SearchName -Export -ArchiveFormat PerUserPST -EnableDedupe $true 
$ExportName = $SearchName + "_Export"

#Wait for Export to complete
do
    {
        Start-Sleep -s 5
        $complete = Get-ComplianceSearchAction -Identity $ExportName
    }
while ($complete.Status -ne 'Completed')

Any help would be appreciated!

 

45 Replies

OK. Then it's time to log a support call with Microsoft. 

 

I too have some scripts that I use with searches. When I went to test one to investigate this problem, I also found that I had the wrong permissions and the export parameter wasn't available. After I amended permissions and restarted the PowerShell session, all was well...

I hate to bring back old threads, but it's always nice to find info in threads that show up high on search results.

 

As of today (05/10/2018) the export error still exists. However, swapping back to the "old experience" and using the export results link in the right column still works.

Content search exports are different in the new experience. For one thing, you can no longer use Chrome to export search results because Google has deprecated the feature Microsoft used to download and launch the export app.

 

But I have certainly been able to export information from a content search.

 

Export from a content searchExport from a content searchLeading to:

Export optionsExport options

Isn't that what you see in the new experience?

Yes, that's what I saw before clicking 'Export' (second screen shot). When I clicked it I got the error box that complains about 'failed to find parameter'. I was in IE when failing to export the results in the new UI. Trying a test export to get you a screen shot is failing now though. Apparently after one export went through others are working just fine in both browsers.

As for chrome not working, the ClickOnce extension still works for me. Is it failing for you?

ClickOnce has been deprecated by Google but is still supported in current builds... That's why it still works in the old interface, but it is blocked in the new interface.

So, I've been fighting to automate using powershell to export a PST from Office 365 off and on for a while.

 

AzCopy worked great for Sharepoint/OneDrive exports, but exchange data all came down as ".batch" files, which appear to need further post-processing to collect everything into a PST. Someone better than I might know how to piece them together, but I just moved on.

 

It's a bit of a workaround but here's what I did to automate the PST download.

When you manually download the export using the export tool, the click-once popup downloads their "Unified Export Tool" and populates it with what it needs to get it going. While it's open, you can right-click the process in task manager and click "Open file location". This will take you to the files their tool uses, in your appdata. For me, this location was "%LOCALAPPDATA%\Apps\2.0\QBKH9EZX.XMP\7R4KYC02.WJ7\micr..tool_51a5b647dacf4059_000f.0014_a4b60912e622c727\"

I'm sure Microsoft updates the tool every now and then, so this path may not work indefinitely.

 

Anyways, I just copied everything to a new folder (I don't recommend doing this, i provide an example to find the file automatically in the script below) and ran the "microsoft.office.client.discovery.unifiedexporttool.exe" which returned the parameters that it accepts. Much like AzCopy, it looks for the URL, Token, and a place to drop the files.

 

Here's what I slapped together to automatically download the export from Office 365:

The code below has been edited since this post was created to include improvements to the original code and provide a start to finish example to export and download a PST. If you want to use this as a user off-boarding script you'll need to add those tasks as well. Everyone has their own processes, but Tom Aguero's post below has some good examples too.

 

# Note that I'm not validating any input or providing any adequate error handling.
# This is just an example, you'll need to add these in yourself.

Param(
    [Parameter(Mandatory=$True, HelpMessage='Enter the email address that you want to export')]
    $Mailbox,
    [Parameter(Mandatory=$True, HelpMessage='Enter the URL for the user''s OneDrive here. If you don''t enter one, this will be skipped.')]
    $OneDriveURL,
    [Parameter(Mandatory=$True, HelpMessage='Enter the path where you want to save the PST file. !NO TRAILING BACKSLASH!')]
    $ExportLocation # = ""# you can un-comment the = "" to set a default for this parameter.
)

# Create a search name. You can change this to suit your preference
$SearchName = "$Mailbox PST"

# I'm using the Exchange Online Powershell Module v2. You can install it from an admin session with the following command: Install-Module ExchangeOnlineManagement
Write-Host "Connecting to Exchange Online. Enter your admin credentials in the pop-up (pop-under?) window."
Connect-IPPSSession

Write-Host "Creating compliance search..."
New-ComplianceSearch -Name $SearchName -ExchangeLocation $Mailbox -SharePointLocation $OneDriveURL -AllowNotFoundExchangeLocationsEnabled $true #Create a content search, including the the entire contents of the user's email and onedrive. If you didn't provide a OneDrive URL, or it wasn't valid, it will be ignored.
Write-Host "Starting compliance search..."
Start-ComplianceSearch -Identity $SearchName #Start the search created above
Write-Host "Waiting for compliance search to complete..."
for ($SearchStatus;$SearchStatus -notlike "Completed";){ #Wait then check if the search is complete, loop until complete
    Start-Sleep -s 2
    $SearchStatus = Get-ComplianceSearch $SearchName | Select-Object -ExpandProperty Status #Get the status of the search
    Write-Host -NoNewline "." # Show some sort of status change in the terminal
}
Write-Host "Compliance search is complete!"
Write-Host "Creating export from the search..."
New-ComplianceSearchAction -SearchName $SearchName -Export -Format FxStream -ExchangeArchiveFormat PerUserPst -Scope BothIndexedAndUnindexedItems -EnableDedupe $true -SharePointArchiveFormat IndividualMessage -IncludeSharePointDocumentVersions $true 
Start-Sleep -s 5 # Arbitrarily wait 5 seconds to give microsoft's side time to create the SearchAction before the next commands try to run against it. I /COULD/ do a for loop and check, but it's really not worth it.

# Check if the export tool is installed for the user, and download if not.
While (-Not ((Get-ChildItem -Path $($env:LOCALAPPDATA + "\Apps\2.0\") -Filter microsoft.office.client.discovery.unifiedexporttool.exe -Recurse).FullName | Where-Object{ $_ -notmatch "_none_" } | Select-Object -First 1)){
    Write-Host "Downloading Unified Export Tool ."
    Write-Host "This is installed per-user by the Click-Once installer."
    # Credit to Jos Verlinde for his code in Load-ExchangeMFA in the Powershell Gallery! All I've done is update the manifest url and remove all the comments
    # Ripped from https://www.powershellgallery.com/packages/Load-ExchangeMFA/1.2
    # In case anyone else has any ClickOnce applications they'd like to automate the install for:
    # If you're looking for where to find a manifest URL, once you have run the ClickOnce application at least once on your computer, the url for the application manifest can be found in the Windows Registry at "HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Uninstall" (yes, CTR apps are installed per-user).
    # Look through the keys with names that are 16 characters long hex strings. They'll have a string value (REG_SZ) named either "ShortcutAppId" or "UrlUpdateInfo" that contains the URL as the first part of the string.
    $Manifest = "https://complianceclientsdf.blob.core.windows.net/v16/Microsoft.Office.Client.Discovery.UnifiedExportTool.application"
    $ElevatePermissions = $true
    Try {
        Add-Type -AssemblyName System.Deployment
        Write-Host "Starting installation of ClickOnce Application $Manifest "
        $RemoteURI = [URI]::New( $Manifest , [UriKind]::Absolute)
        if (-not  $Manifest){
            throw "Invalid ConnectionUri parameter '$ConnectionUri'"
        }
        $HostingManager = New-Object System.Deployment.Application.InPlaceHostingManager -ArgumentList $RemoteURI , $False
        Register-ObjectEvent -InputObject $HostingManager -EventName GetManifestCompleted -Action { 
            new-event -SourceIdentifier "ManifestDownloadComplete"
        } | Out-Null
        Register-ObjectEvent -InputObject $HostingManager -EventName DownloadApplicationCompleted -Action { 
            new-event -SourceIdentifier "DownloadApplicationCompleted"
        } | Out-Null
        $HostingManager.GetManifestAsync()
        $event = Wait-Event -SourceIdentifier "ManifestDownloadComplete" -Timeout 15
        if ($event ) {
            $event | Remove-Event
            Write-Host "ClickOnce Manifest Download Completed"
            $HostingManager.AssertApplicationRequirements($ElevatePermissions)
            $HostingManager.DownloadApplicationAsync()
            $event = Wait-Event -SourceIdentifier "DownloadApplicationCompleted" -Timeout 60
            if ($event ) {
                $event | Remove-Event
                Write-Host "ClickOnce Application Download Completed"
            }
            else {
                Write-error "ClickOnce Application Download did not complete in time (60s)"
            }
        }
        else {
            Write-error "ClickOnce Manifest Download did not complete in time (15s)"
        }
    }
    finally {
        Get-EventSubscriber|? {$_.SourceObject.ToString() -eq 'System.Deployment.Application.InPlaceHostingManager'} | Unregister-Event
    }
}

# Find the Unified Export Tool's location and create a variable for it
$ExportExe = ((Get-ChildItem -Path $($env:LOCALAPPDATA + "\Apps\2.0\") -Filter microsoft.office.client.discovery.unifiedexporttool.exe -Recurse).FullName | Where-Object{ $_ -notmatch "_none_" } | Select-Object -First 1)

# Gather the URL and Token from the export in order to start the download
# We only need the ContainerURL and SAS Token at a minimum but we're also pulling others to help with tracking the status of the export.
$ExportName = $SearchName + "_Export"
$ExportDetails = Get-ComplianceSearchAction -Identity $ExportName -IncludeCredential -Details # Get details for the export action
# This method of splitting the Container URL and Token from $ExportDetails is thanks to schmeckendeugler from reddit: https://www.reddit.com/r/PowerShell/comments/ba4fpu/automated_download_of_o365_inbox_archive/
# I was using Convert-FromString before, which was slow and terrible. His way is MUCH better.
$ExportDetails = $ExportDetails.Results.split(";")
$ExportContainerUrl = $ExportDetails[0].trimStart("Container url: ")
$ExportSasToken = $ExportDetails[1].trimStart(" SAS token: ")
$ExportEstSize = ($ExportDetails[18].TrimStart(" Total estimated bytes: ") -as [double])
$ExportTransferred = ($ExportDetails[20].TrimStart(" Total transferred bytes: ") -as [double])
$ExportProgress = $ExportDetails[22].TrimStart(" Progress: ").TrimEnd("%")
$ExportStatus = $ExportDetails[25].TrimStart(" Export status: ")

# Download the exported files from Office 365
Write-Host "Initiating download"
Write-Host "Saving export to: " + $ExportLocation
$Arguments = "-name ""$SearchName""","-source ""$ExportContainerUrl""","-key ""$ExportSasToken""","-dest ""$ExportLocation""","-trace true"
Start-Process -FilePath "$ExportExe" -ArgumentList $Arguments

# The export is now downloading in the background. You can find it in task manager. Let's monitor the progress.
# If you want to use this as part of a user offboarding script, add your edits above here - Exports can take a lot of time...
# You can even comment this entire section and exit the script if you dont feel the need to monitor the download, it will keep downloading in the background even without the script running.
# This is only monitoring if the process exists, which means if you run multiple exports, this will stay running until they all complete.
# We could possibly utilize sysinternals handle.exe to identify the PID of the process writing to the $Exportlocation and monitor for that specifically, but I'm trying to limit external applications in this example script.
#
# Just an FYI, the export progress is how much data Microsoft has copied into PSTs from the compliance search, not how much the export tool has downloaded.
# We only know the actual size of the download after the $ExportProgress is 100% and $ExportStatus is Completed
# The actual final size of the download is then reflected in $ExportTransferred. Even then, our progress is still a bit inaccurate due to the extra log and temp files created locally, which will probably cause the progress to show over 100%
# We could make this a bit more accurate by just collecting the size of PSTs and files under the OneDrive folder, but I think this brings us close enough for most situations.
while(Get-Process microsoft.office.client.discovery.unifiedexporttool -ErrorAction SilentlyContinue){
    $Downloaded = Get-ChildItem $ExportLocation\$SearchName -Recurse | Measure-Object -Property Length -Sum | Select-Object -ExpandProperty Sum
    Write-Progress -Id 1 -Activity "Export in Progress" -Status "Complete..." -PercentComplete $ExportProgress
    if ("Completed" -notlike $ExportStatus){Write-Progress -Id 2 -Activity "Download in Progress" -Status "Estimated Complete..." -PercentComplete ($Downloaded/$ExportEstSize*100) -CurrentOperation "$Downloaded/$ExportEstSize bytes downloaded."}
    else {Write-Progress -Id 2 -Activity "Download in Progress" -Status "Complete..." -PercentComplete ($Downloaded/$ExportEstSize*100) -CurrentOperation "$Downloaded/$ExportTransferred bytes downloaded."}
    Start-Sleep 60
    $ExportDetails = Get-ComplianceSearchAction -Identity $ExportName -IncludeCredential -Details # Get details for the export action
    $ExportDetails = $ExportDetails.Results.split(";")
    $ExportEstSize = ($ExportDetails[18].TrimStart(" Total estimated bytes: ") -as [double])
    $ExportTransferred = ($ExportDetails[20].TrimStart(" Total transferred bytes: ") -as [double])
    $ExportProgress = $ExportDetails[22].TrimStart(" Progress: ").TrimEnd("%")
    $ExportStatus = $ExportDetails[25].TrimStart(" Export status: ")
    Write-Host -NoNewline " ."
}
Write-Host "Download Complete!"
pause

 

The nice part about this is the unified export tool includes it's own wait loop that checks the status of the export, so you don't have to create your own and worry about credential timeout, etc. Thanks to this, you can kick off the download and move on to other tasks in your off-boarding script.

In production, I've written this to execute the download on a separate terminal using Invoke-Command and email the team a transcript when the download's done.

 

Drawbacks:

The only way you know it's running is the process shows up in task manager.

The only way you know it's done is it's not in task manager. We can estimate how far along it is, but it isn't perfectly accurate. You definitely want to rely on the trace files to know if everything went OK.

If the export gets stopped for some reason (lost internet, restart due to windows updates, etc.) it does not resume the download when started again. I haven't been able to figure out how to get that to work.

 

Anyways, I wanted to give back to the thread that helped me get this far.

Is this still working for you? Have you made any changes/improvements since you posted this?

 

How'd you get the container URL?

 

Thanks!

Yes, this still worked quite well as of a week ago. I can only see this breaking if Microsoft changes the format of the "Results" property of the content search or splits it into separate properties of their own.

I haven't made any real improvements to it, but this reminds me that I should probably combine my code with Brad's so that there's a minimally functional example. I'll edit my previous post at some time for that.

 

The container URL comes from the "$exportdetails =" line.

It first gets the compliance search, selects the Results property that contains the container URL, then parses the URL and token from the property using the $exporttemplate

The results property is just one giant string of information, instead of separate properties of their own, so parsing it with Convert-FromString was the easiest way I could think of to get those properties.

 

Convert-FromString requires a template consisting of at least two lines of data, formatted so that it knows what you're looking to parse. The data in $exporttemplate is just two of our old searches, modified with the parsing tags. Sorry if that's not incredibly clear.

Awesome, I got it working and I figured out a way to monitor the process along with the percent complete. Sorry, I'm not sure how to do the fancy formatting for this post.

 

#Do while microsoft.office.client.discovery.unifiedexporttool.exe running
$started = $false
Do { $status = Get-Process microsoft.office.client.discovery.unifiedexporttool -ErrorAction SilentlyContinue
If (!($status)) {

Write-Host 'Waiting for process to start' ; Start-Sleep -Seconds 5 }

Else {

Write-Host 'Process has started' ; $started = $true

}

}Until ( $started )  

 

Do{

$ProcessesFound = Get-Process | ? {$_.Name -like "*unifiedexporttool*"}
If ($ProcessesFound) { $

Progress = get-ComplianceSearchAction -Identity $exportname -IncludeCredential -Details | select -ExpandProperty Results | ConvertFrom-String -TemplateContent $exporttemplate | %{$_.Progress}
Write-Host "Export still downloading, progress is $Progress, waiting 60 seconds"
Start-Sleep -s 60

}

}Until (!$ProcessesFound)

Oh nice! I forgot I parsed that and didn't even think about polling the server-side progress.

The only thing I can see being an issue is if it's a very large export, or you have a slow connection, your session with office 365 might timeout before it's done.

I'll try adding it to our implementation and see if I run into any problems.

 

Are you logging the entire session with a transcript? If not, I'd recommend writing the progress to a file so you can quickly check if it finished or not, in case your computer restarts for windows updates or something while you're not looking. "Write-Output $Progress | Out-File" is quick and easy, but you might want to include a timestamp as well.

@Tom Aguero Can you share the whole script to download pst physically from EXO mailboxes

@Edgar Guerrero Sure, I think I sanitized it

 

#region Connect to all the things
#Test O365 Connection
If($Null -eq $O365Cred){
$O365Cred = $Host.ui.PromptForCredential("","Enter your OFFICE 365 admin creds","ENTER YOUR ACCOUNT NAME HERE","")
}

Try{Get-O365Mailbox aguerot -ErrorAction Stop > $Null}
Catch{
$O365Session = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri https://outlook.office365.com/powershell-liveid/ -Credential $O365Cred -Authentication Basic -AllowRedirection
Import-PSSession $O365Session –Prefix o365
}

Try{Get-MsolUser -UserPrincipalName ENTER TEST USER HERE -ErrorAction Stop > $Null}
Catch{Connect-MsolService -Credential $O365Cred}

Try{Get-ComplianceSearch > $Null}
Catch{
#Get login credentials
$Session = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri https://ps.compliance.protection.outlook.com/powershell-liveid -Credential $O365Cred -Authentication Basic -AllowRedirection
Import-PSSession $Session -AllowClobber -DisableNameChecking
}
#endregion

#Get User Info
$User = Read-host "Enter user name"

$Logfile = "PATH TO LOG FILE\$User.txt"

Function LogWrite
{
Param ([string]$logstring)
Write-Host $LogString
Add-content $Logfile -value $logstring
}
$Date = Get-Date -Format "MM/dd/yyyy"
LogWrite $Date
LogWrite " "
LogWrite "Username: $User"

#Begin Compliance Search
$UPN = Get-ADUser $User | ForEach-Object{$_.UserPrincipalName}
$SearchName = $User + "_Termed"
New-ComplianceSearch -Name $SearchName -ExchangeLocation $UPN
Start-ComplianceSearch $SearchName
Logwrite " "
LogWrite "Compliance search $SearchName started"
Do{
$complianceSearch = Get-ComplianceSearch $SearchName | ForEach-Object{$_.Status}
Write-Host "Compliance Search in Progress"
Start-Sleep -s 30
}
While ($complianceSearch -ne 'Completed')
$Size = Get-ComplianceSearch $SearchName | ForEach-Object{$_.Size}
$Size = $Size / 1048576 | Out-String
$Size = $Size.SubString(0,6)

# Create Compliance Search in exportable format
New-ComplianceSearchAction -SearchName $SearchName -EnableDedupe $true -Export -Format FxStream -ArchiveFormat PerUserPST > $Null

#Wait for Export to complete
$ExportName = $SearchName + "_Export"
Start-Sleep -s 20
do{
$SearchAction = Get-ComplianceSearchAction -Identity $ExportName | Select-Object Status,JobProgress
$Status = $SearchAction.Status
$ExportProgress = $SearchAction.JobProgress
Write-Host "Export in progress, $ExportProgress complete"
If($Status -ne "Completed"){
Start-Sleep -s 60
}
}
while ($Status -ne 'Completed')

LogWrite "Compliance search completed"

$exportlocation = "C:\pst" #enter the path to your export here !NO TRAILING BACKSLASH!
$exportexe = "\UnifiedExportTool\microsoft.office.client.discovery.unifiedexporttool.exe" #path to your microsoft.office.client.discovery.unifiedexporttool.exe file. Usually found somewhere in ForEach-ObjectLOCALAPPDATAForEach-Object\Apps\2.0\

# Gather the URL and Token from the export in order to start the download
#We only need the ContainerURL and SAS Token but I parsed some other fields as well while working with AzCopy
#The Container URL and Token in the following template has been altered to protect the innocent:
$exporttemplate = @'
Container url: {ContainerURL*:https://xicnediscnam.blob.core.windows.net/da3fecb0-4ed4-447e-0315-08d5adad8a5a}; SAS token: {SASToken:?sv=2014-02-14&sr=c&si=eDiscoveryBlobPolicy9ForEach-Object7C0&sig=RACMSyH6Cf0k4EP2wZSoAa0QrhKaV38Oa9ciHv5Y8MkForEach-Object3D}; Scenario: General; Scope: BothIndexedAndUnindexedItems; Scope details: AllUnindexed; Max unindexed size: 0; File type exclusions for unindexed: <null>; Total sources: 2; Exchange item format: Msg; Exchange archive format: IndividualMessage; SharePoint archive format: SingleZip; Include SharePoint versions: True; Enable dedupe: EnableDedupe:True; Reference action: "<null>"; Region: ; Started sources: StartedSources:3; Succeeded sources: SucceededSources:1; Failed sources: 0; Total estimated bytes: 12,791,334,934; Total estimated items: 143,729; Total transferred bytes: {TotalTransferredBytes:7,706,378,435}; Total transferred items: {TotalTransferredItems:71,412}; Progress: {Progress:49.69 ForEach-Object}; Completed time: ; Duration: 00:50:43.9321895; Export status: {ExportStatus:DistributionCompleted}
Container url: {ContainerURL*:https://zgrbediscnam.blob.core.windows.net/5c21f7c7-42a2-4e24-9e69-08d5acf316f5}; SAS token: {SASToken:?sv=2014-02-14&sr=c&si=eDiscoveryBlobPolicy9ForEach-Object7C0&sig=F6ycaX5eWcRBCS1Z5nfoTKJWTrHkAciqbYRP5ForEach-Object2FhsUOoForEach-Object3D}; Scenario: General; Scope: BothIndexedAndUnindexedItems; Scope details: AllUnindexed; Max unindexed size: 0; File type exclusions for unindexed: <null>; Total sources: 1; Exchange item format: FxStream; Exchange archive format: PerUserPst; SharePoint archive format: IndividualMessage; Include SharePoint versions: True; Enable dedupe: True; Reference action: "<null>"; Region: ; Started sources: 2; Succeeded sources: 2; Failed sources: 0; Total estimated bytes: 69,952,559,461; Total estimated items: 107,707; Total transferred bytes: {TotalTransferredBytes:70,847,990,489}; Total transferred items: {TotalTransferredItems:100,808}; Progress: {Progress:93.59 ForEach-Object}; Completed time: 4/27/2018 11:45:46 PM; Duration: 04:31:21.1593737; Export status: {ExportStatus:Completed}
'@
$exportdetails = Get-ComplianceSearchAction -Identity $exportname -IncludeCredential -Details | Select-Object -ExpandProperty Results | ConvertFrom-String -TemplateContent $exporttemplate
$exportdetails
$exportcontainerurl = $exportdetails.ContainerURL
$exportsastoken = $exportdetails.SASToken

# Download the exported files from Office 365
Write-Host "Compliance Search Size:" $Size
LogWrite "Initiating download"
LogWrite "Saving export to: $exportlocation"
$arguments = "-name ""$searchname""","-source ""$exportcontainerurl""","-key ""$exportsastoken""","-dest ""$exportlocation""","-trace true"
Start-Process -FilePath "$exportexe" -ArgumentList $arguments

#Do while microsoft.office.client.discovery.unifiedexporttool.exe running
$started = $false
Do {
$status = Get-Process microsoft.office.client.discovery.unifiedexporttool -ErrorAction SilentlyContinue
If (!($status)) {
Write-Host 'Waiting for process to start' ; Start-Sleep -Seconds 5
}
Else {
Write-Host 'Process has started' ; $started = $true
}
}
Until ( $started )

Do{
$Finished = $False
Write-host "Sleeping 300 seconds"
Start-Sleep -s 300
$DownloadSize = Get-Childitem "C:\pst" -recurse -include *.pst | ForEach-Object{$_.Length}
$DownloadSize = $DownloadSize / 1048576 | Out-String
$DownloadSize = $DownloadSize.SubString(0,6)
Write-Host " "
Write-Host "Compliance Search Size:" $Size
Write-Host "Downloaded Size:" $DownloadSize
Write-Host " "
<## Commenting this out to test performance
$ProcessesFound = Get-Process | ? {$_.Name -like "*unifiedexporttool*"}
Write-Host "Process still running"
 
If ($ProcessesFound) {
$Progress = get-ComplianceSearchAction -Identity $exportname -IncludeCredential -Details | Select-Object -ExpandProperty Results | ConvertFrom-String -TemplateContent $exporttemplate | ForEach-Object{$_.Progress}
Write-Host "Export still downloading, progress is $Progress, waiting 300 seconds"
Start-Sleep -s 300
}
#>
If($DownloadSize -ge $Size){
$status = Get-Process microsoft.office.client.discovery.unifiedexporttool -ErrorAction SilentlyContinue
If (!($status)) {
Write-Host "Download has finished"
$Finished = $True
}
}
}Until ($Finished -eq $True)

$DownloadSize = Get-Childitem "C:\pst" -recurse -include *.pst | ForEach-Object{$_.Length}
$DownloadSize = $DownloadSize / 1048576 | Out-String
$DownloadSize = $DownloadSize.SubString(0,6)

Write-Host " "
Write-Host "Compliance Search Size:" $Size
Write-Host "Downloaded Size:" $DownloadSize
Write-Host " "

#Read-Host "Pausing script until you verify download. Seriously. If the download isn't done this gets really bad. You sure you're ready? Press enter to continue"

LogWrite "PST exported & downloaded"

#Remove compliance search & export
$ExportName = $SearchName + "_Export"
Remove-ComplianceSearchAction -Identity $ExportName -Confirm:$False
Remove-ComplianceSearch $SearchName -Confirm:$False
If($?){Write-Host "Compliance search & export removed"}

#Get & Remove O365 licenses
$UPN2 = Get-ADUser -Identity $User | ForEach-Object{$_.UserPrincipalName}
$MSOLUser = Get-MsolUser -UserPrincipalName $UPN2 -ErrorAction SilentlyContinue

#$IsLicensed = $MSOLUser.IsLicensed
$Licenses = $MSOLUser.Licenses[0].AccountSkuId

Set-MsolUserLicense -UserPrincipalName $UPN2 -RemoveLicenses $Licenses
If($?){Write-Host "$User $Licenses removed"}

#Move to Disabled Accts OU
Get-AdUser $User | Move-AdObject -TargetPath "" #Moves to an OU that doesnt sync to O365
LogWrite "$User account has been moved to Disabled Accts OU"

#Run DirSync on Azure AD Connect sync server
Invoke-Command -Computername "" -ScriptBlock {C:\DirSync-ForceDelta.ps1} #Runs an Azure AD Connect Directory sync on a remote server

#Move PST for upload
Get-Childitem "C:\pst" -recurse -include *.pst | Move-Item -Destination "C:\pst"

#Upload mailbox to Azure
Write-Host "Uploading to Azure"

Start-Process "Disable User Process\LaunchCmd-Custom.cmd" #Customized Azure Storage AZ copy upload script

#Confirm upload to finished
write-host " "
$confirmation = Read-Host "Did Azure upload complete successfully? You need to enter y here"
if ($confirmation -eq 'y'){
#Delete pst
Remove-Item "C:\pst\*" -Recurse -Confirm:$False
If($?){Write-host " "
Write-Host "PST Deleted"}
}
Also, here is the AZ copy script, you need to get your own key and destination info. I honestly forget how to get that.
@echo off
title Microsoft Azure Storage tools
set LaunchCmdFolder="C:\Program Files (x86)\Microsoft SDKs\Azure\AzCopy"
set PATH=%PATH%;%LaunchCmdFolder%AzCopy

Set azcopy="C:\Program Files (x86)\Microsoft SDKs\Azure\AzCopy\AzCopy.exe"
Set source="C:\pst"
Set dest=
set key=

%azcopy% /source:%source% /Dest:%dest% /DestKey:%key% /Pattern:*.pst /S /V:C:\AZCopy.log

Pause

Hi, @Tom Aguero Thanks for the help so far.

I am not able to start the export.

From the tracert file i got this message:
Unhandled Exception: System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.Office.Client.Discovery.UnifiedExport, Version=15.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35' or one of its dependencies. The system cannot find the file specified.

 

Do I need to do something with the Microsoft.Office.Client.Discovery.UnifiedExportTool.application file or should it work with Container url and SAS token?

 

I also fund this (he has stolen some part from you). Still no luck.
https://www.reddit.com/r/PowerShell/comments/ba4fpu/automated_download_of_o365_inbox_archive/

 

Any help is appreciated

Hi @Powershell_mannen69 

It looks like the export tool can't load one of it's dependency files. Did you copy the exe out of your profile folder without all the rest of the files? And, are you using Tom Aguero's code? If so, try replacing:

 

 

$exportexe = "\UnifiedExportTool\microsoft.office.client.discovery.unifiedexporttool.exe"

 

 

with:

 

 

$exportexe = ((Get-ChildItem -Path $($env:LOCALAPPDATA + "\Apps\2.0\") -Filter microsoft.office.client.discovery.unifiedexporttool.exe -Recurse).FullName | Where-Object{ $_ -notmatch "_none_" } | Select-Object -First 1)

 

 

and see if that helps?

 

This should find the exe within your profile, without having to copy the file or use any static paths, which should help if you have multiple people that may run the script.

 

Also, thanks for the Reddit link, I like their way of pulling the URL and token much better! I'll have to try it out on the next run of the script.

 

 

Additionally, If anyone wants to automate downloading the unified export tool:

# Check if the export tool is installed for the user, and download if not.
While (-Not ((Get-ChildItem -Path $($env:LOCALAPPDATA + "\Apps\2.0\") -Filter microsoft.office.client.discovery.unifiedexporttool.exe -Recurse).FullName | Where-Object{ $_ -notmatch "_none_" } | Select-Object -First 1)){
    Write-Host "Downloading Unified Export Tool ."
    Write-Host "This is installed per-user by the Click-Once installer."
    # Credit to Jos Verlinde for his code in Load-ExchangeMFA in the Powershell Gallery! All I've done is update the manifest url and remove all the comments
    # Ripped from https://www.powershellgallery.com/packages/Load-ExchangeMFA/1.2
    # In case anyone else has any ClickOnce applications they'd like to automate the install for:
    # If you're looking for where to find a manifest URL, once you have run the ClickOnce application at least once on your computer, the url for the application manifest can be found in the Windows Registry at "HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Uninstall" (yes, CTR apps are installed per-user).
    # Look through the keys with names that are 16 characters long hex strings. They'll have a string value (REG_SZ) named either "ShortcutAppId" or "UrlUpdateInfo" that contains the URL as the first part of the string.
    $Manifest = "https://complianceclientsdf.blob.core.windows.net/v16/Microsoft.Office.Client.Discovery.UnifiedExportTool.application"
    $ElevatePermissions = $true
    Try {
        Add-Type -AssemblyName System.Deployment
        Write-Host "Starting installation of ClickOnce Application $Manifest "
        $RemoteURI = [URI]::New( $Manifest , [UriKind]::Absolute)
        if (-not  $Manifest){
            throw "Invalid ConnectionUri parameter '$ConnectionUri'"
        }
        $HostingManager = New-Object System.Deployment.Application.InPlaceHostingManager -ArgumentList $RemoteURI , $False
        Register-ObjectEvent -InputObject $HostingManager -EventName GetManifestCompleted -Action { 
            new-event -SourceIdentifier "ManifestDownloadComplete"
        } | Out-Null
        Register-ObjectEvent -InputObject $HostingManager -EventName DownloadApplicationCompleted -Action { 
            new-event -SourceIdentifier "DownloadApplicationCompleted"
        } | Out-Null
        $HostingManager.GetManifestAsync()
        $event = Wait-Event -SourceIdentifier "ManifestDownloadComplete" -Timeout 15
        if ($event ) {
            $event | Remove-Event
            Write-Host "ClickOnce Manifest Download Completed"
            $HostingManager.AssertApplicationRequirements($ElevatePermissions)
            $HostingManager.DownloadApplicationAsync()
            $event = Wait-Event -SourceIdentifier "DownloadApplicationCompleted" -Timeout 60
            if ($event ) {
                $event | Remove-Event
                Write-Host "ClickOnce Application Download Completed"
            }
            else {
                Write-error "ClickOnce Application Download did not complete in time (60s)"
            }
        }
        else {
            Write-error "ClickOnce Manifest Download did not complete in time (15s)"
        }
    }
    finally {
        Get-EventSubscriber|? {$_.SourceObject.ToString() -eq 'System.Deployment.Application.InPlaceHostingManager'} | Unregister-Event
    }
}

$exportexe = ((Get-ChildItem -Path $($env:LOCALAPPDATA + "\Apps\2.0\") -Filter microsoft.office.client.discovery.unifiedexporttool.exe -Recurse).FullName | Where-Object{ $_ -notmatch "_none_" } | Select-Object -First 1)

 

Hi @JChup Thank you for sharing your code.

 

I get a problem when I started to export result.

From the trace.log file I got the error:

Fatal error happened. System.UnauthorizedAccessException: Access to the path '.\x86' is denied.

 

After that, the export process has been cancelled.

 

Do you know where is the folder '.\x86' and how can I set the permission of that folder?

 

Thanks

 

 

I finally found the reason. The script should be run by Powershell Administrator !

@JChup 

I am curious to know if this capability is available now since most of the conversation was about a year ago. I am also needing to automate the whole process via a script but it looks like the last piece of downloading the pst file is still a manual process from the GUI? 

 

Thanks! 

Hi @Riftsan,

 

Yes, everything still works, and I edited my original post a few months ago to include improvements.

The actual downloading of the PST does require Microsoft's ClickOnce "Universal Export Tool".

The script includes code to automatically download the tool if it's not already on your computer, and download the PST without having to interact with the GUI.

The only issues at this point are that we can't pull real stats from the local application, so we have to poll server-side stats to estimate how far along the download is, and you can't resume a download if it's been interrupted.

@Tom Aguero 

 

Edit: Never mind. I was calling C:\Users\USERNAME\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\Microsoft Corporation\Microsoft Office 365 eDiscovery Export Tool.appref-ms instead of C:\Users\USERNAME\AppData\Local\Apps\2.0\ABC123.456\ZZZZZ.789\micr..tool_123456789_abcdef_abcabcabc\microsoft.office.client.discovery.unifiedexporttool.exe.

 

When running the discovery tool with those parameters, it complains that it can't determine the URI format. I'm leaning towards it being 'key', as this isn't a valid URI (PowerShell fails to parse it).

@JChup 

 

If the application has already been installed, you'll get a misleading InvalidOperationException from AssertApplicationRequirements. This doc "suggests" to check GetManifestCompletedEventArgs before calling AssertApplicationRequirements.

 

 

        Register-ObjectEvent -InputObject $HostingManager -EventName GetManifestCompleted -Action {
            New-Event -SourceIdentifier "GetManifestCompleted" -EventArguments @($EventArgs.Error)
        } | Out-Null

 

 

 

 $HostingManagerEvent = Wait-Event -SourceIdentifier "GetManifestCompleted" -Timeout 15
        if($HostingManager){
            if ($HostingManagerEvent.SourceArgs[0]) {
                if($HostingManagerEvent.SourceArgs[0].Message.Contains("already installed")){
                    Write-Output "ClickOnce Application has already been downloaded"
                }else{
                    Write-Error -Message $HostingManagerEvent.SourceArgs[0].Message
                }
            }
        }

 

 

Additionally, VSCode warned me $event is an automatic variable. Unless it was intentionally named that way, I renamed mine to something else.