Forum Discussion

Christophe DAVID's avatar
Christophe DAVID
Copper Contributor
Feb 08, 2017
Solved

Copy of a large number of folders from one sharepoint online site to another

Hi

For a Customer of ours, I try to use Powershell to automate the copy of a large number of folders from one sharepoint online site to another site (same tenant).

Here are some details:

- 2500 folders stored in one sharepoint online site
- 2500 office groups created. They all have one sharepoint site.

We have the lists in XLS files for:

Source URL and name
Destination URL

We want to copy whats inside fo one folder (source) to the officegroup sharepoint.

Any help would be appreciated.

Thank you.

  • You can use the below script for copying. Attached the sample csv to be used. This script does not preserve metadata.

     

    function Copy-sharepoint
    {
    Param(
    [Parameter(Mandatory=$True)]
    [String]$sharepointUrl,
    
    [Parameter(Mandatory=$True)]
    [String]$UserName,
    
    [Parameter(Mandatory=$True)]
    [String]$Password,
    
    [Parameter(Mandatory=$True)]
    [String]$Path,
    
    [Parameter(Mandatory=$True)]
    [System.Management.Automation.PSCredential]$cred
    
    )
    #create secure password
    $sPassword = $Password | ConvertTo-SecureString -AsPlainText -Force
    
    $loadInfo1 = [System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint.Client")
    $loadInfo2 = [System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint.Client.Runtime")
    $csvs = Import-Csv $Path
    
    Foreach($csv in $csvs)
    {
    
    $Sourceurl = $csv.SourceUrl
    $souceFoldername= $csv.Foldername
    $targeturl = $csv.TargetUrl
    $TargetDocumentLib = $csv.targetlib
    
    Connect-PnPOnline -Url $Sourceurl -Credentials $cred
    $Files= Find-PnPFile -Folder $souceFoldername -Match *
    Disconnect-PnPOnline
    $Loginsource =$false
    $webClient = New-Object System.Net.WebClient 
    $webClient.Credentials = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($UserName, $sPassword)
    $webClient.Headers.Add("X-FORMS_BASED_AUTH_ACCEPTED", "f")
    $webclient.Proxy = $null
    Write-Host "Copying $($Files.count) files to $($targeturl)"
    
    foreach($file in  $Files)
    {
        $pageUrl = $sharepointUrl+$file.ServerRelativeUrl
        $UniqueFileName = $file.Name
        $ByteArray=$webClient.DownloadData($pageUrl)
        $tfolderWithname= ($pageUrl -split $Sourceurl)[1]
        $tfolderwiths =($tfolderWithname -split $UniqueFileName)[0]
        $tfolderrmstr =$tfolderwiths.TrimStart("/")
        $tfolder =$tfolderrmstr.TrimEnd("/")
        $fstream = [System.IO.MemoryStream]($ByteArray)
        
        $tf=($tfolder -split "/")
             $test=$False
             If($tf.count -gt 1)
             {
                 foreach($tf1 in $tf)
                 {
                     if($test -eq $true)
                     {
                        $tf2=$tf2+"/"+$tf1
                     }
                        $test=$true
                  }
              }
              Else
              {
              $tf2=$null
              }
        
         If($Loginsource -eq $false )
            {
                Connect-PnPOnline -Url $targeturl -Credentials $cred         
                $Loginsource =$true
               
            }
        
        $m=Get-PnPList -Identity $TargetDocumentLib
        $ctx=Get-PnPContext
        $mroot=$m.RootFolder
        $ctx.Load($mroot)
        $ctx.ExecuteQuery()
        $targetFolder=$mroot.Name+$tf2
        $tf2=$null
    
        If($targetFolder -notlike "*Forms")
        {
            
                Add-PnPFile -FileName $UniqueFileName -Folder $targetFolder -Stream $fstream
            
        }
    }
    
    }
    }
    
    $cred=Get-Credential 
    $Path = "C:\csv\copyfile.csv"
    $sharepointUrl = "https://tenantname.sharepoint.com"
    $UName=$cred.UserName.ToString()
    $Pass =$cred.GetNetworkCredential().Password
    
    Copy-sharepoint -UserName $UName -Password $Pass -cred $cred -sharepointUrl $sharepointUrl -Path $Path 

12 Replies

  • Guy Johnson's avatar
    Guy Johnson
    Iron Contributor

    This was the reason we started using Sharegate. The cost outweighed the aggrevation

    • I also recommend migration tools for a scenario where you have several info. to move mainly because the throttling you will get from the SPO site. Sharegate and other tools will be able to deal with this…the other approach you could take a look is the Office 365 Import Service
      • Christophe DAVID's avatar
        Christophe DAVID
        Copper Contributor

        I am trying Sharegate at the moment, but there seems to be a big limitation.

         

        We have one Site with multiple folders. eg: customer1, customer2, customer3.

        We have multiple Sites: Sitecustomer1, Sitecustomer2, Sitecustomer3.

         

        When i edit the Excel file batch migrate in sharegate, i get a problem with the "DestinationPath"; i cant go up in the hierarchy with cd ../Sitecustomer1. I dont know if its even possible.

         

        Chris

  • You can use the below script for copying. Attached the sample csv to be used. This script does not preserve metadata.

     

    function Copy-sharepoint
    {
    Param(
    [Parameter(Mandatory=$True)]
    [String]$sharepointUrl,
    
    [Parameter(Mandatory=$True)]
    [String]$UserName,
    
    [Parameter(Mandatory=$True)]
    [String]$Password,
    
    [Parameter(Mandatory=$True)]
    [String]$Path,
    
    [Parameter(Mandatory=$True)]
    [System.Management.Automation.PSCredential]$cred
    
    )
    #create secure password
    $sPassword = $Password | ConvertTo-SecureString -AsPlainText -Force
    
    $loadInfo1 = [System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint.Client")
    $loadInfo2 = [System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint.Client.Runtime")
    $csvs = Import-Csv $Path
    
    Foreach($csv in $csvs)
    {
    
    $Sourceurl = $csv.SourceUrl
    $souceFoldername= $csv.Foldername
    $targeturl = $csv.TargetUrl
    $TargetDocumentLib = $csv.targetlib
    
    Connect-PnPOnline -Url $Sourceurl -Credentials $cred
    $Files= Find-PnPFile -Folder $souceFoldername -Match *
    Disconnect-PnPOnline
    $Loginsource =$false
    $webClient = New-Object System.Net.WebClient 
    $webClient.Credentials = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($UserName, $sPassword)
    $webClient.Headers.Add("X-FORMS_BASED_AUTH_ACCEPTED", "f")
    $webclient.Proxy = $null
    Write-Host "Copying $($Files.count) files to $($targeturl)"
    
    foreach($file in  $Files)
    {
        $pageUrl = $sharepointUrl+$file.ServerRelativeUrl
        $UniqueFileName = $file.Name
        $ByteArray=$webClient.DownloadData($pageUrl)
        $tfolderWithname= ($pageUrl -split $Sourceurl)[1]
        $tfolderwiths =($tfolderWithname -split $UniqueFileName)[0]
        $tfolderrmstr =$tfolderwiths.TrimStart("/")
        $tfolder =$tfolderrmstr.TrimEnd("/")
        $fstream = [System.IO.MemoryStream]($ByteArray)
        
        $tf=($tfolder -split "/")
             $test=$False
             If($tf.count -gt 1)
             {
                 foreach($tf1 in $tf)
                 {
                     if($test -eq $true)
                     {
                        $tf2=$tf2+"/"+$tf1
                     }
                        $test=$true
                  }
              }
              Else
              {
              $tf2=$null
              }
        
         If($Loginsource -eq $false )
            {
                Connect-PnPOnline -Url $targeturl -Credentials $cred         
                $Loginsource =$true
               
            }
        
        $m=Get-PnPList -Identity $TargetDocumentLib
        $ctx=Get-PnPContext
        $mroot=$m.RootFolder
        $ctx.Load($mroot)
        $ctx.ExecuteQuery()
        $targetFolder=$mroot.Name+$tf2
        $tf2=$null
    
        If($targetFolder -notlike "*Forms")
        {
            
                Add-PnPFile -FileName $UniqueFileName -Folder $targetFolder -Stream $fstream
            
        }
    }
    
    }
    }
    
    $cred=Get-Credential 
    $Path = "C:\csv\copyfile.csv"
    $sharepointUrl = "https://tenantname.sharepoint.com"
    $UName=$cred.UserName.ToString()
    $Pass =$cred.GetNetworkCredential().Password
    
    Copy-sharepoint -UserName $UName -Password $Pass -cred $cred -sharepointUrl $sharepointUrl -Path $Path 
    • Christophe DAVID's avatar
      Christophe DAVID
      Copper Contributor

      Hi again.

       

      I cant make it work. The script runs, i get no error, but the files are not copied.

       

      I am testing as follow:

       

      1 Site is named "testsource". it has a "123456" folder in it.

      1 Site is named "testdestination".

       

      It is a french tenant. Does it make a difference for the folder path ?

       

      Any help would be greatly apreciated.

       

      Chris

       

       

       

  • HI if you want to do it with Powershell go look at the Powershell PNP for Sharepoint project

    https://github.com/SharePoint/PnP-PowerShell

    These give you a large variety of large cmdlets to use.

     

    However note the copying between sites / libraries does not preserve the meta data of the files. Although it does appear to copy versions at least within the same site collection

     

    If that is important to you to preserve the metadata and versions look at third party tools that can do this for you.

    Metalogix, Sharegate and Avepoint are just three of a vast array out there. Also some do trial versions or free versions that will let you copy data up to a certain limit for free.

     

Resources