Forum Discussion
Copy of a large number of folders from one sharepoint online site to another
Hi
For a Customer of ours, I try to use Powershell to automate the copy of a large number of folders from one sharepoint online site to another site (same tenant).
Here are some details:
- 2500 folders stored in one sharepoint online site
- 2500 office groups created. They all have one sharepoint site.
We have the lists in XLS files for:
Source URL and name
Destination URL
We want to copy whats inside fo one folder (source) to the officegroup sharepoint.
Any help would be appreciated.
Thank you.
You can use the below script for copying. Attached the sample csv to be used. This script does not preserve metadata.
function Copy-sharepoint { Param( [Parameter(Mandatory=$True)] [String]$sharepointUrl, [Parameter(Mandatory=$True)] [String]$UserName, [Parameter(Mandatory=$True)] [String]$Password, [Parameter(Mandatory=$True)] [String]$Path, [Parameter(Mandatory=$True)] [System.Management.Automation.PSCredential]$cred ) #create secure password $sPassword = $Password | ConvertTo-SecureString -AsPlainText -Force $loadInfo1 = [System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint.Client") $loadInfo2 = [System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint.Client.Runtime") $csvs = Import-Csv $Path Foreach($csv in $csvs) { $Sourceurl = $csv.SourceUrl $souceFoldername= $csv.Foldername $targeturl = $csv.TargetUrl $TargetDocumentLib = $csv.targetlib Connect-PnPOnline -Url $Sourceurl -Credentials $cred $Files= Find-PnPFile -Folder $souceFoldername -Match * Disconnect-PnPOnline $Loginsource =$false $webClient = New-Object System.Net.WebClient $webClient.Credentials = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($UserName, $sPassword) $webClient.Headers.Add("X-FORMS_BASED_AUTH_ACCEPTED", "f") $webclient.Proxy = $null Write-Host "Copying $($Files.count) files to $($targeturl)" foreach($file in $Files) { $pageUrl = $sharepointUrl+$file.ServerRelativeUrl $UniqueFileName = $file.Name $ByteArray=$webClient.DownloadData($pageUrl) $tfolderWithname= ($pageUrl -split $Sourceurl)[1] $tfolderwiths =($tfolderWithname -split $UniqueFileName)[0] $tfolderrmstr =$tfolderwiths.TrimStart("/") $tfolder =$tfolderrmstr.TrimEnd("/") $fstream = [System.IO.MemoryStream]($ByteArray) $tf=($tfolder -split "/") $test=$False If($tf.count -gt 1) { foreach($tf1 in $tf) { if($test -eq $true) { $tf2=$tf2+"/"+$tf1 } $test=$true } } Else { $tf2=$null } If($Loginsource -eq $false ) { Connect-PnPOnline -Url $targeturl -Credentials $cred $Loginsource =$true } $m=Get-PnPList -Identity $TargetDocumentLib $ctx=Get-PnPContext $mroot=$m.RootFolder $ctx.Load($mroot) $ctx.ExecuteQuery() $targetFolder=$mroot.Name+$tf2 $tf2=$null If($targetFolder -notlike "*Forms") { Add-PnPFile -FileName $UniqueFileName -Folder $targetFolder -Stream $fstream } } } } $cred=Get-Credential $Path = "C:\csv\copyfile.csv" $sharepointUrl = "https://tenantname.sharepoint.com" $UName=$cred.UserName.ToString() $Pass =$cred.GetNetworkCredential().Password Copy-sharepoint -UserName $UName -Password $Pass -cred $cred -sharepointUrl $sharepointUrl -Path $Path
12 Replies
- Guy JohnsonIron Contributor
This was the reason we started using Sharegate. The cost outweighed the aggrevation
- I also recommend migration tools for a scenario where you have several info. to move mainly because the throttling you will get from the SPO site. Sharegate and other tools will be able to deal with this…the other approach you could take a look is the Office 365 Import Service
- Christophe DAVIDCopper Contributor
I am trying Sharegate at the moment, but there seems to be a big limitation.
We have one Site with multiple folders. eg: customer1, customer2, customer3.
We have multiple Sites: Sitecustomer1, Sitecustomer2, Sitecustomer3.
When i edit the Excel file batch migrate in sharegate, i get a problem with the "DestinationPath"; i cant go up in the hierarchy with cd ../Sitecustomer1. I dont know if its even possible.
Chris
You can use the below script for copying. Attached the sample csv to be used. This script does not preserve metadata.
function Copy-sharepoint { Param( [Parameter(Mandatory=$True)] [String]$sharepointUrl, [Parameter(Mandatory=$True)] [String]$UserName, [Parameter(Mandatory=$True)] [String]$Password, [Parameter(Mandatory=$True)] [String]$Path, [Parameter(Mandatory=$True)] [System.Management.Automation.PSCredential]$cred ) #create secure password $sPassword = $Password | ConvertTo-SecureString -AsPlainText -Force $loadInfo1 = [System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint.Client") $loadInfo2 = [System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint.Client.Runtime") $csvs = Import-Csv $Path Foreach($csv in $csvs) { $Sourceurl = $csv.SourceUrl $souceFoldername= $csv.Foldername $targeturl = $csv.TargetUrl $TargetDocumentLib = $csv.targetlib Connect-PnPOnline -Url $Sourceurl -Credentials $cred $Files= Find-PnPFile -Folder $souceFoldername -Match * Disconnect-PnPOnline $Loginsource =$false $webClient = New-Object System.Net.WebClient $webClient.Credentials = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($UserName, $sPassword) $webClient.Headers.Add("X-FORMS_BASED_AUTH_ACCEPTED", "f") $webclient.Proxy = $null Write-Host "Copying $($Files.count) files to $($targeturl)" foreach($file in $Files) { $pageUrl = $sharepointUrl+$file.ServerRelativeUrl $UniqueFileName = $file.Name $ByteArray=$webClient.DownloadData($pageUrl) $tfolderWithname= ($pageUrl -split $Sourceurl)[1] $tfolderwiths =($tfolderWithname -split $UniqueFileName)[0] $tfolderrmstr =$tfolderwiths.TrimStart("/") $tfolder =$tfolderrmstr.TrimEnd("/") $fstream = [System.IO.MemoryStream]($ByteArray) $tf=($tfolder -split "/") $test=$False If($tf.count -gt 1) { foreach($tf1 in $tf) { if($test -eq $true) { $tf2=$tf2+"/"+$tf1 } $test=$true } } Else { $tf2=$null } If($Loginsource -eq $false ) { Connect-PnPOnline -Url $targeturl -Credentials $cred $Loginsource =$true } $m=Get-PnPList -Identity $TargetDocumentLib $ctx=Get-PnPContext $mroot=$m.RootFolder $ctx.Load($mroot) $ctx.ExecuteQuery() $targetFolder=$mroot.Name+$tf2 $tf2=$null If($targetFolder -notlike "*Forms") { Add-PnPFile -FileName $UniqueFileName -Folder $targetFolder -Stream $fstream } } } } $cred=Get-Credential $Path = "C:\csv\copyfile.csv" $sharepointUrl = "https://tenantname.sharepoint.com" $UName=$cred.UserName.ToString() $Pass =$cred.GetNetworkCredential().Password Copy-sharepoint -UserName $UName -Password $Pass -cred $cred -sharepointUrl $sharepointUrl -Path $Path- Christophe DAVIDCopper Contributor
Hi again.
I cant make it work. The script runs, i get no error, but the files are not copied.
I am testing as follow:
1 Site is named "testsource". it has a "123456" folder in it.
1 Site is named "testdestination".
It is a french tenant. Does it make a difference for the folder path ?
Any help would be greatly apreciated.
Chris
In your csv, the format is incorrect, the headers are missing. Attached the csv sample with your example.
- Christophe DAVIDCopper Contributor
Thank you very very much.
I will try now.
- Philip WorrellIron Contributor
HI if you want to do it with Powershell go look at the Powershell PNP for Sharepoint project
https://github.com/SharePoint/PnP-PowerShell
These give you a large variety of large cmdlets to use.
However note the copying between sites / libraries does not preserve the meta data of the files. Although it does appear to copy versions at least within the same site collection
If that is important to you to preserve the metadata and versions look at third party tools that can do this for you.
Metalogix, Sharegate and Avepoint are just three of a vast array out there. Also some do trial versions or free versions that will let you copy data up to a certain limit for free.