Forum Discussion
Export to PST via Powershell
- Aug 10, 2017
No way to do it without going to the SCC and initializing the download via the click-one app, sorry. Perhaps you can automate it via AzCopy or some other tool that takes container/token as input - you can get those via the Result property of Get-ComplianceSearchAction.
As for the cmdlet, make sure you also use the -Format parameter!
The Format parameter specifies the format of the search results when you use the Export switch. Valid values are:
FxStream Export to PST files. This is the only option that's available when you export search results from the Security & Compliance Center.
Mime Export to .eml messsage files. This the default value when you use cmdlets to export the search results.It's most likely what causes the issue in your case.
Awesome, I got it working and I figured out a way to monitor the process along with the percent complete. Sorry, I'm not sure how to do the fancy formatting for this post.
#Do while microsoft.office.client.discovery.unifiedexporttool.exe running
$started = $false
Do { $status = Get-Process microsoft.office.client.discovery.unifiedexporttool -ErrorAction SilentlyContinue
If (!($status)) {Write-Host 'Waiting for process to start' ; Start-Sleep -Seconds 5 }
Else {
Write-Host 'Process has started' ; $started = $true
}
}Until ( $started )
Do{
$ProcessesFound = Get-Process | ? {$_.Name -like "*unifiedexporttool*"}
If ($ProcessesFound) { $Progress = get-ComplianceSearchAction -Identity $exportname -IncludeCredential -Details | select -ExpandProperty Results | ConvertFrom-String -TemplateContent $exporttemplate | %{$_.Progress}
Write-Host "Export still downloading, progress is $Progress, waiting 60 seconds"
Start-Sleep -s 60}
}Until (!$ProcessesFound)
Tom Aguero Can you share the whole script to download pst physically from EXO mailboxes
- Tom AgueroApr 12, 2019Copper Contributor
Edgar Guerrero Sure, I think I sanitized it
#region Connect to all the things#Test O365 ConnectionIf($Null -eq $O365Cred){$O365Cred = $Host.ui.PromptForCredential("","Enter your OFFICE 365 admin creds","ENTER YOUR ACCOUNT NAME HERE","")}Try{Get-O365Mailbox aguerot -ErrorAction Stop > $Null}Catch{$O365Session = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri https://outlook.office365.com/powershell-liveid/ -Credential $O365Cred -Authentication Basic -AllowRedirectionImport-PSSession $O365Session –Prefix o365}Try{Get-MsolUser -UserPrincipalName ENTER TEST USER HERE -ErrorAction Stop > $Null}Catch{Connect-MsolService -Credential $O365Cred}Try{Get-ComplianceSearch > $Null}Catch{#Get login credentials$Session = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri https://ps.compliance.protection.outlook.com/powershell-liveid -Credential $O365Cred -Authentication Basic -AllowRedirectionImport-PSSession $Session -AllowClobber -DisableNameChecking}#endregion#Get User Info$User = Read-host "Enter user name"$Logfile = "PATH TO LOG FILE\$User.txt"Function LogWrite{Param ([string]$logstring)Write-Host $LogStringAdd-content $Logfile -value $logstring}$Date = Get-Date -Format "MM/dd/yyyy"LogWrite $DateLogWrite " "LogWrite "Username: $User"#Begin Compliance Search$UPN = Get-ADUser $User | ForEach-Object{$_.UserPrincipalName}$SearchName = $User + "_Termed"New-ComplianceSearch -Name $SearchName -ExchangeLocation $UPNStart-ComplianceSearch $SearchNameLogwrite " "LogWrite "Compliance search $SearchName started"Do{$complianceSearch = Get-ComplianceSearch $SearchName | ForEach-Object{$_.Status}Write-Host "Compliance Search in Progress"Start-Sleep -s 30}While ($complianceSearch -ne 'Completed')$Size = Get-ComplianceSearch $SearchName | ForEach-Object{$_.Size}$Size = $Size / 1048576 | Out-String$Size = $Size.SubString(0,6)# Create Compliance Search in exportable formatNew-ComplianceSearchAction -SearchName $SearchName -EnableDedupe $true -Export -Format FxStream -ArchiveFormat PerUserPST > $Null#Wait for Export to complete$ExportName = $SearchName + "_Export"Start-Sleep -s 20do{$SearchAction = Get-ComplianceSearchAction -Identity $ExportName | Select-Object Status,JobProgress$Status = $SearchAction.Status$ExportProgress = $SearchAction.JobProgressWrite-Host "Export in progress, $ExportProgress complete"If($Status -ne "Completed"){Start-Sleep -s 60}}while ($Status -ne 'Completed')LogWrite "Compliance search completed"$exportlocation = "C:\pst" #enter the path to your export here !NO TRAILING BACKSLASH!$exportexe = "\UnifiedExportTool\microsoft.office.client.discovery.unifiedexporttool.exe" #path to your microsoft.office.client.discovery.unifiedexporttool.exe file. Usually found somewhere in ForEach-ObjectLOCALAPPDATAForEach-Object\Apps\2.0\# Gather the URL and Token from the export in order to start the download#We only need the ContainerURL and SAS Token but I parsed some other fields as well while working with AzCopy#The Container URL and Token in the following template has been altered to protect the innocent:$exporttemplate = @'Container url: {ContainerURL*:https://xicnediscnam.blob.core.windows.net/da3fecb0-4ed4-447e-0315-08d5adad8a5a}; SAS token: {SASToken:?sv=2014-02-14&sr=c&si=eDiscoveryBlobPolicy9ForEach-Object7C0&sig=RACMSyH6Cf0k4EP2wZSoAa0QrhKaV38Oa9ciHv5Y8MkForEach-Object3D}; Scenario: General; Scope: BothIndexedAndUnindexedItems; Scope details: AllUnindexed; Max unindexed size: 0; File type exclusions for unindexed: <null>; Total sources: 2; Exchange item format: Msg; Exchange archive format: IndividualMessage; SharePoint archive format: SingleZip; Include SharePoint versions: True; Enable dedupe: EnableDedupe:True; Reference action: "<null>"; Region: ; Started sources: StartedSources:3; Succeeded sources: SucceededSources:1; Failed sources: 0; Total estimated bytes: 12,791,334,934; Total estimated items: 143,729; Total transferred bytes: {TotalTransferredBytes:7,706,378,435}; Total transferred items: {TotalTransferredItems:71,412}; Progress: {Progress:49.69 ForEach-Object}; Completed time: ; Duration: 00:50:43.9321895; Export status: {ExportStatus:DistributionCompleted}Container url: {ContainerURL*:https://zgrbediscnam.blob.core.windows.net/5c21f7c7-42a2-4e24-9e69-08d5acf316f5}; SAS token: {SASToken:?sv=2014-02-14&sr=c&si=eDiscoveryBlobPolicy9ForEach-Object7C0&sig=F6ycaX5eWcRBCS1Z5nfoTKJWTrHkAciqbYRP5ForEach-Object2FhsUOoForEach-Object3D}; Scenario: General; Scope: BothIndexedAndUnindexedItems; Scope details: AllUnindexed; Max unindexed size: 0; File type exclusions for unindexed: <null>; Total sources: 1; Exchange item format: FxStream; Exchange archive format: PerUserPst; SharePoint archive format: IndividualMessage; Include SharePoint versions: True; Enable dedupe: True; Reference action: "<null>"; Region: ; Started sources: 2; Succeeded sources: 2; Failed sources: 0; Total estimated bytes: 69,952,559,461; Total estimated items: 107,707; Total transferred bytes: {TotalTransferredBytes:70,847,990,489}; Total transferred items: {TotalTransferredItems:100,808}; Progress: {Progress:93.59 ForEach-Object}; Completed time: 4/27/2018 11:45:46 PM; Duration: 04:31:21.1593737; Export status: {ExportStatus:Completed}'@$exportdetails = Get-ComplianceSearchAction -Identity $exportname -IncludeCredential -Details | Select-Object -ExpandProperty Results | ConvertFrom-String -TemplateContent $exporttemplate$exportdetails$exportcontainerurl = $exportdetails.ContainerURL$exportsastoken = $exportdetails.SASToken# Download the exported files from Office 365Write-Host "Compliance Search Size:" $SizeLogWrite "Initiating download"LogWrite "Saving export to: $exportlocation"$arguments = "-name ""$searchname""","-source ""$exportcontainerurl""","-key ""$exportsastoken""","-dest ""$exportlocation""","-trace true"Start-Process -FilePath "$exportexe" -ArgumentList $arguments#Do while microsoft.office.client.discovery.unifiedexporttool.exe running$started = $falseDo {$status = Get-Process microsoft.office.client.discovery.unifiedexporttool -ErrorAction SilentlyContinueIf (!($status)) {Write-Host 'Waiting for process to start' ; Start-Sleep -Seconds 5}Else {Write-Host 'Process has started' ; $started = $true}}Until ( $started )Do{$Finished = $FalseWrite-host "Sleeping 300 seconds"Start-Sleep -s 300$DownloadSize = Get-Childitem "C:\pst" -recurse -include *.pst | ForEach-Object{$_.Length}$DownloadSize = $DownloadSize / 1048576 | Out-String$DownloadSize = $DownloadSize.SubString(0,6)Write-Host " "Write-Host "Compliance Search Size:" $SizeWrite-Host "Downloaded Size:" $DownloadSizeWrite-Host " "<## Commenting this out to test performance$ProcessesFound = Get-Process | ? {$_.Name -like "*unifiedexporttool*"}Write-Host "Process still running"If ($ProcessesFound) {$Progress = get-ComplianceSearchAction -Identity $exportname -IncludeCredential -Details | Select-Object -ExpandProperty Results | ConvertFrom-String -TemplateContent $exporttemplate | ForEach-Object{$_.Progress}Write-Host "Export still downloading, progress is $Progress, waiting 300 seconds"Start-Sleep -s 300}#>If($DownloadSize -ge $Size){$status = Get-Process microsoft.office.client.discovery.unifiedexporttool -ErrorAction SilentlyContinueIf (!($status)) {Write-Host "Download has finished"$Finished = $True}}}Until ($Finished -eq $True)$DownloadSize = Get-Childitem "C:\pst" -recurse -include *.pst | ForEach-Object{$_.Length}$DownloadSize = $DownloadSize / 1048576 | Out-String$DownloadSize = $DownloadSize.SubString(0,6)Write-Host " "Write-Host "Compliance Search Size:" $SizeWrite-Host "Downloaded Size:" $DownloadSizeWrite-Host " "#Read-Host "Pausing script until you verify download. Seriously. If the download isn't done this gets really bad. You sure you're ready? Press enter to continue"LogWrite "PST exported & downloaded"#Remove compliance search & export$ExportName = $SearchName + "_Export"Remove-ComplianceSearchAction -Identity $ExportName -Confirm:$FalseRemove-ComplianceSearch $SearchName -Confirm:$FalseIf($?){Write-Host "Compliance search & export removed"}#Get & Remove O365 licenses$UPN2 = Get-ADUser -Identity $User | ForEach-Object{$_.UserPrincipalName}$MSOLUser = Get-MsolUser -UserPrincipalName $UPN2 -ErrorAction SilentlyContinue#$IsLicensed = $MSOLUser.IsLicensed$Licenses = $MSOLUser.Licenses[0].AccountSkuIdSet-MsolUserLicense -UserPrincipalName $UPN2 -RemoveLicenses $LicensesIf($?){Write-Host "$User $Licenses removed"}#Move to Disabled Accts OUGet-AdUser $User | Move-AdObject -TargetPath "" #Moves to an OU that doesnt sync to O365LogWrite "$User account has been moved to Disabled Accts OU"#Run DirSync on Azure AD Connect sync serverInvoke-Command -Computername "" -ScriptBlock {C:\DirSync-ForceDelta.ps1} #Runs an Azure AD Connect Directory sync on a remote server#Move PST for uploadGet-Childitem "C:\pst" -recurse -include *.pst | Move-Item -Destination "C:\pst"#Upload mailbox to AzureWrite-Host "Uploading to Azure"Start-Process "Disable User Process\LaunchCmd-Custom.cmd" #Customized Azure Storage AZ copy upload script#Confirm upload to finishedwrite-host " "$confirmation = Read-Host "Did Azure upload complete successfully? You need to enter y here"if ($confirmation -eq 'y'){#Delete pstRemove-Item "C:\pst\*" -Recurse -Confirm:$FalseIf($?){Write-host " "Write-Host "PST Deleted"}}Also, here is the AZ copy script, you need to get your own key and destination info. I honestly forget how to get that.@echo offtitle Microsoft Azure Storage toolsset LaunchCmdFolder="C:\Program Files (x86)\Microsoft SDKs\Azure\AzCopy"set PATH=%PATH%;%LaunchCmdFolder%AzCopySet azcopy="C:\Program Files (x86)\Microsoft SDKs\Azure\AzCopy\AzCopy.exe"Set source="C:\pst"Set dest=set key=%azcopy% /source:%source% /Dest:%dest% /DestKey:%key% /Pattern:*.pst /S /V:C:\AZCopy.logPause- tylermontney_accSep 06, 2022Brass Contributor
Edit: Never mind. I was calling C:\Users\USERNAME\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\Microsoft Corporation\Microsoft Office 365 eDiscovery Export Tool.appref-ms instead of C:\Users\USERNAME\AppData\Local\Apps\2.0\ABC123.456\ZZZZZ.789\micr..tool_123456789_abcdef_abcabcabc\microsoft.office.client.discovery.unifiedexporttool.exe.
When running the discovery tool with those parameters, it complains that it can't determine the URI format. I'm leaning towards it being 'key', as this isn't a valid URI (PowerShell fails to parse it).
- Powershell_mannen69May 04, 2020Copper Contributor
Hi, Tom Aguero Thanks for the help so far.
I am not able to start the export.From the tracert file i got this message:
Unhandled Exception: System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.Office.Client.Discovery.UnifiedExport, Version=15.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35' or one of its dependencies. The system cannot find the file specified.Do I need to do something with the Microsoft.Office.Client.Discovery.UnifiedExportTool.application file or should it work with Container url and SAS token?
I also fund this (he has stolen some part from you). Still no luck.
https://www.reddit.com/r/PowerShell/comments/ba4fpu/automated_download_of_o365_inbox_archive/Any help is appreciated
- JChupMay 04, 2020Brass Contributor
It looks like the export tool can't load one of it's dependency files. Did you copy the exe out of your profile folder without all the rest of the files? And, are you using Tom Aguero's code? If so, try replacing:
$exportexe = "\UnifiedExportTool\microsoft.office.client.discovery.unifiedexporttool.exe"
with:
$exportexe = ((Get-ChildItem -Path $($env:LOCALAPPDATA + "\Apps\2.0\") -Filter microsoft.office.client.discovery.unifiedexporttool.exe -Recurse).FullName | Where-Object{ $_ -notmatch "_none_" } | Select-Object -First 1)
and see if that helps?
This should find the exe within your profile, without having to copy the file or use any static paths, which should help if you have multiple people that may run the script.
Also, thanks for the Reddit link, I like their way of pulling the URL and token much better! I'll have to try it out on the next run of the script.
Additionally, If anyone wants to automate downloading the unified export tool:
# Check if the export tool is installed for the user, and download if not. While (-Not ((Get-ChildItem -Path $($env:LOCALAPPDATA + "\Apps\2.0\") -Filter microsoft.office.client.discovery.unifiedexporttool.exe -Recurse).FullName | Where-Object{ $_ -notmatch "_none_" } | Select-Object -First 1)){ Write-Host "Downloading Unified Export Tool ." Write-Host "This is installed per-user by the Click-Once installer." # Credit to Jos Verlinde for his code in Load-ExchangeMFA in the Powershell Gallery! All I've done is update the manifest url and remove all the comments # Ripped from https://www.powershellgallery.com/packages/Load-ExchangeMFA/1.2 # In case anyone else has any ClickOnce applications they'd like to automate the install for: # If you're looking for where to find a manifest URL, once you have run the ClickOnce application at least once on your computer, the url for the application manifest can be found in the Windows Registry at "HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Uninstall" (yes, CTR apps are installed per-user). # Look through the keys with names that are 16 characters long hex strings. They'll have a string value (REG_SZ) named either "ShortcutAppId" or "UrlUpdateInfo" that contains the URL as the first part of the string. $Manifest = "https://complianceclientsdf.blob.core.windows.net/v16/Microsoft.Office.Client.Discovery.UnifiedExportTool.application" $ElevatePermissions = $true Try { Add-Type -AssemblyName System.Deployment Write-Host "Starting installation of ClickOnce Application $Manifest " $RemoteURI = [URI]::New( $Manifest , [UriKind]::Absolute) if (-not $Manifest){ throw "Invalid ConnectionUri parameter '$ConnectionUri'" } $HostingManager = New-Object System.Deployment.Application.InPlaceHostingManager -ArgumentList $RemoteURI , $False Register-ObjectEvent -InputObject $HostingManager -EventName GetManifestCompleted -Action { new-event -SourceIdentifier "ManifestDownloadComplete" } | Out-Null Register-ObjectEvent -InputObject $HostingManager -EventName DownloadApplicationCompleted -Action { new-event -SourceIdentifier "DownloadApplicationCompleted" } | Out-Null $HostingManager.GetManifestAsync() $event = Wait-Event -SourceIdentifier "ManifestDownloadComplete" -Timeout 15 if ($event ) { $event | Remove-Event Write-Host "ClickOnce Manifest Download Completed" $HostingManager.AssertApplicationRequirements($ElevatePermissions) $HostingManager.DownloadApplicationAsync() $event = Wait-Event -SourceIdentifier "DownloadApplicationCompleted" -Timeout 60 if ($event ) { $event | Remove-Event Write-Host "ClickOnce Application Download Completed" } else { Write-error "ClickOnce Application Download did not complete in time (60s)" } } else { Write-error "ClickOnce Manifest Download did not complete in time (15s)" } } finally { Get-EventSubscriber|? {$_.SourceObject.ToString() -eq 'System.Deployment.Application.InPlaceHostingManager'} | Unregister-Event } } $exportexe = ((Get-ChildItem -Path $($env:LOCALAPPDATA + "\Apps\2.0\") -Filter microsoft.office.client.discovery.unifiedexporttool.exe -Recurse).FullName | Where-Object{ $_ -notmatch "_none_" } | Select-Object -First 1)