.ps1 script: how to loop in a range of dates

[marco69]

Hi all,

I have to copy the content of the folders:

C:\mypath\YYYYDDMM\myfolder1

into one folder destination.

And: YYYYMMDD is from 20140101 to 20211231

I'm struggling to find a way to increment day by day my loop, extract day.month.year from the date and then compose the new string for my path......

can somebody help me?

thanks!!!

[#2]

If all myfolder1 folders are in the base of YYYYDDMM in the c:\mypath folders... Then you could just get a list of folders and loop through them if the myfolder1 folder is found, then copy all items to the one folder destination.

foreach ($folder in Get-ChildItem -Path c:\mypath -Directory) {
if (test-path $folder.fullname\myfolder1) {
copy-item -path $folder.fullname\myfolder1 -Recurse -destination c:\mypath\folder destination
}
}

something like that?

[#3]

Hi Harm_Veenstra!

I have this situation:

C:\mypath\20220421\myfolder1
C:\mypath\20220422\myfolder1
C:\mypath\20220425\myfolder1
C:\mypath\20220511\myfolder1
....
C:\mypath\20220718\myfolder1
C:\mypath\20220723\myfolder1
...
C:\mypath\20221212\myfolder1


starting from the very first folder (the oldest, in the example above is the folder at 21Apr2022) I have to copy the content of myfolder1 to a destination directory, for example:

c:\destpath\myfolder1

Then:

copy C:\mypath\20220421\myfolder1\* -> c:\destpath\myfolder1\*
copy C:\mypath\20220422\myfolder1\* -> c:\destpath\myfolder1\*
copy C:\mypath\20220425\myfolder1\* -> c:\destpath\myfolder1\*
...
copy C:\mypath\20221212\myfolder1\* -> c:\destpath\myfolder1\*

For info, just to better clarify, when copying I have to overwrite a file if it already exists and normally copy if it's new.

mf

[#4 · Accepted solution]

Just tested this script, this works:

foreach ($folder in Get-ChildItem -Path c:\mypath -Directory | Sort-Object Name) {
    try {
        Test-Path "$($folder.fullname)\myfolder1" -ErrorAction Stop | Out-Null
        Copy-Item "$($folder.fullname)\myfolder1" -Recurse -Destination C:\destpath\myfolder1 -Force:$true -Confirm:$false -ErrorAction Stop
        Write-Host ("Processing {0}" -f $folder.fullname) -ForegroundColor Green
    }
    catch {
        Write-Warning ("Folder myfolder1 not found in {0} or not enough permissions" -f $folder.FullName)
    }
}

Source directory:

Destination directory:

It will check permissions and copy if it has permissions and will overwrite if the file is already present, it will show a warning if the folder1 folder was not found in a directory.

You can test it for yourself with a test directory structure :) 

[#5]

You are my Hero!!!!
many many many thanks!!!!

[#6]

Let us know if it works out for you, please mark my answer as best response to mark it as solved if it did :)

[#7]

I forgot one important point: I have to overwrite only if the file is newer...
how can I modify the script to do that?
thank you and excuse me for this missing

[#8]

@marco69 Changed it to also check for that:

$sourcepath = 'C:\MyPath'
$destinationpath = 'C:\DestPath\myfolder1'
foreach ($folder in Get-ChildItem -Path $sourcepath -Directory | Sort-Object Name) {
    try {
        Test-Path "$($folder.fullname)\myfolder1" -ErrorAction Stop | Out-Null
        foreach ($file in Get-ChildItem -Path "$($folder.fullname)\myfolder1" -File) {
            if (Test-Path -Path "$($destinationpath)\$($file.name)") {
                $destinationfile = Get-ChildItem -Path "$($destinationpath)\$($file.Name)"
                if ($file.LastWriteTime -gt $destinationfile.LastWriteTime) {
                    Copy-Item "$($folder.fullname)\myfolder1\$($file.Name)" -Destination $destinationpath -Force:$true -Confirm:$false -ErrorAction Stop
                    Write-Host ("Processing {0} in {1} and overwriting old version" -f $file.name, $folder.fullname) -ForegroundColor Green
                }
                else {
                    Write-Host ("File {0} from {1} already present in {2}, no change needed" -f $file.Name, $folder.Name, $destinationpath) -ForegroundColor Green
                }
            }
            else {
                Copy-Item "$($folder.fullname)\myfolder1\$($file.name)" -Destination $destinationpath -ErrorAction Stop
                Write-Host ("Copying new file {0} from {1} to {2}" -f $file.name, $folder.fullname, $destinationpath) -ForegroundColor Green
            }   
        }
    }
    catch {
        Write-Warning ("Folder myfolder1 not found in {0} or not enough permissions" -f $folder.FullName)
    }
}

But... Are there multiple directories in c:\mypath\xxxxx\myfolder1? This script copies files from one directory to another, is that enough?

[#9]

Hi @Harm_Veenstra,

I have to copy myfolder1 to destination, so this is ok.

I have a weird behavior when I run the script, it gets me an error not founding the files under myfolder1, but some of those files exist...

When I? wun theprevious version of the script it works very fine(simply it didnt consider the age of the files but it worked very well).

This is the error:

Get-ChildItem : Cannot find path 'C:\MyPath\20210618\a\b\c\myfolder1' because it does not exist.
At C:\scripts\myscript.ps1:7 char:27
+ ... h ($file in Get-ChildItem -Path "$($folder.fullname)\$myfolder1" -Fil ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : ObjectNotFound: (C:\MyPath\20210618\a\b\c\myfolder1:String) [Get-ChildItem], ItemNotFoundException
+ FullyQualifiedErrorId : PathNotFound,Microsoft.PowerShell.Commands.GetChildItemCommand

this is the line where the error refers:

 foreach ($file in Get-ChildItem -Path "$($folder.fullname)\$myfolder1" -File) {

[#10]

@marco69 I fixed it :) It wasn't doing a Recurse in subdirectories if any and did not pass that along correctly, the script is now like this:

$sourcepath = 'C:\MyPath'
$destinationpath = 'C:\DestPath\myfolder1'
foreach ($folder in Get-ChildItem -Path $sourcepath -Directory -Recurse | Sort-Object Name) {
    try {
        Test-Path "$($folder.fullname)\myfolder1" -ErrorAction Stop | Out-Null
        foreach ($file in Get-ChildItem -Path "$($folder.fullname)\myfolder1" -Recurse -File) {
            if (Test-Path -Path "$($destinationpath)\$($file.name)") {
                $destinationfile = Get-ChildItem -Path "$($destinationpath)\$($file.Name)"
                if ($file.LastWriteTime -gt $destinationfile.LastWriteTime) {
                    Copy-Item $file.FullName -Destination $destinationpath -Force:$true -Confirm:$false -ErrorAction Stop
                    Write-Host ("Processing {0} in {1} and overwriting old version" -f $file.name, $folder.fullname) -ForegroundColor Green
                }
                else {
                    Write-Host ("File {0} from {1} already present in {2}, no change needed" -f $file.Name, $folder.Name, $destinationpath) -ForegroundColor Green
                }
            }
            else {
                Copy-Item $file.FullName -Destination $destinationpath -ErrorAction Stop
                Write-Host ("Copying new file {0} from {1} to {2}" -f $file.name, $folder.fullname, $destinationpath) -ForegroundColor Green
            }   
        }
    }
    catch {
        Write-Warning ("Folder myfolder1 not found in {0} or not enough permissions" -f $folder.FullName)
    }
}

[#11]

ok @Harm_Veenstra  thank you so much!!!

I'm running now: do you think it's ok that after the run of the script it stays without doing nothing? maybe it has to recurse on each subfolders before to go ahead?

I put a warning at the first statement as a log, but I don't get anything:

$sourcepath = 'C:\MyPath'
$myfolder1 = 'a\b\c\myfolder1'
$destinationpath = 'C:\DestPath\myfolder1'

foreach ($folder in Get-ChildItem -Path $sourcepath -Directory -Recurse | Sort-Object Name) {
    try {
        
        Write-Warning ("$($folder.fullname)\$myfolder1" -f $folder.FullName)

        Test-Path "$($folder.fullname)\$myfolder1" -ErrorAction Stop | Out-Null
...

[#12]

"$($folder.fullname)\$myfolder1" --> "$($folder.fullname)\$($myfolder1)" I guess?

[#13]

Hi @Harm_Veenstra,

sorry I'm not skilled on that.

I tried your change and also tried to put the true text path into the script, but it seems stuck. And no files copied into the dest path. the behavior is like it doesn't find the source files and path...

this is the script I run:

$sourcepath = 'C:\MyPath'
$myfolder1 = 'a\b\c\myfolder1'
$destinationpath = 'C:\DestPath\myfolder1'

foreach ($folder in Get-ChildItem -Path $sourcepath -Directory -Recurse | Sort-Object Name) {
    Write-Warning ("$($folder.fullname)\$($myfolder1)" -f $folder.FullName)
    try {
        Test-Path "$($folder.fullname)\$($myfolder1)" -ErrorAction Stop | Out-Null
        foreach ($file in Get-ChildItem -Path "$($folder.fullname)\$($myfolder1)" -Recurse -File) {
            if (Test-Path -Path "$($destinationpath)\$($file.name)") {
                $destinationfile = Get-ChildItem -Path "$($destinationpath)\$($file.Name)"
                if ($file.LastWriteTime -gt $destinationfile.LastWriteTime) {
                    Copy-Item $file.FullName -Destination $destinationpath -Force:$true -Confirm:$false -ErrorAction Stop
                    Write-Host ("Processing {0} in {1} and overwriting old version" -f $file.name, $folder.fullname) -ForegroundColor Green
                }
                else {
                    Write-Host ("File {0} from {1} already present in {2}, no change needed" -f $file.Name, $folder.Name, $destinationpath) -ForegroundColor Green
                }
            }
            else {
                Copy-Item $file.FullName -Destination $destinationpath -ErrorAction Stop
                Write-Host ("Copying new file {0} from {1} to {2}" -f $file.name, $folder.fullname, $destinationpath) -ForegroundColor Green
            }   
        }
    }
    catch {
        Write-Warning ("Folder $($myfolder1) not found in {0} or not enough permissions" -f $folder.FullName)
    }
}

[#14]

but I guess that I have so many folders when recurse and I think it spends a lot to recurse on them.

Since I try this (and it is stuck after the first 3 warnings, but yet not printed the folders, the fourth warning):

$sourcepath = 'C:\MyPath'
$myfolder1 = 'a\b\c\myfolder1'
$destinationpath = 'C:\DestPath\myfolder1'

Write-Warning ($sourcepath)
Write-Warning ($myfolder1)
Write-Warning ($destinationpath)

$folders = Get-ChildItem -Path $sourcepath -Directory -Recurse
Write-Warning ($folders)

foreach ($folder in $folders | Sort-Object Name) {
    Write-Warning ("$($folder.fullname)\$($myfolder1)" -f $folder.FullName)
    try {
        Test-Path "$($folder.fullname)\$($myfolder1)" -ErrorAction Stop | Out-Null
        foreach ($file in Get-ChildItem -Path "$($folder.fullname)\$($myfolder1)" -Recurse -File) {
            if (Test-Path -Path "$($destinationpath)\$($file.name)") {
                $destinationfile = Get-ChildItem -Path "$($destinationpath)\$($file.Name)"
                if ($file.LastWriteTime -gt $destinationfile.LastWriteTime) {
                    Copy-Item $file.FullName -Destination $destinationpath -Force:$true -Confirm:$false -ErrorAction Stop
                    Write-Host ("Processing {0} in {1} and overwriting old version" -f $file.name, $folder.fullname) -ForegroundColor Green
                }
                else {
                    Write-Host ("File {0} from {1} already present in {2}, no change needed" -f $file.Name, $folder.Name, $destinationpath) -ForegroundColor Green
                }
            }
            else {
                Copy-Item $file.FullName -Destination $destinationpath -ErrorAction Stop
                Write-Host ("Copying new file {0} from {1} to {2}" -f $file.name, $folder.fullname, $destinationpath) -ForegroundColor Green
            }   
        }
    }
    catch {
        Write-Warning ("Folder $myfolder1 not found in {0} or not enough permissions" -f $folder.FullName)
    }
}

[#15]

@marco69 Did another test, and changed the script a little bit again:

$sourcepath = 'C:\MyPath'
$destinationpath = 'C:\DestPath\myfolder1'
$myfolder1 = 'Test\Test2'
foreach ($folder in Get-ChildItem -Path $sourcepath -Directory) {
    try {
        if (Test-Path "$($folder.fullname)\$($myfolder1)" -ErrorAction Stop) {
            foreach ($file in Get-ChildItem -Path "$($folder.fullname)\$($myfolder1)" -Recurse -File) {
                if (Test-Path -Path "$($destinationpath)\$($file.name)") {
                    $destinationfile = Get-ChildItem -Path "$($destinationpath)\$($file.Name)"
                    if ($file.LastWriteTime -gt $destinationfile.LastWriteTime) {
                        Copy-Item $file.FullName -Destination $destinationpath -Force:$true -Confirm:$false -ErrorAction Stop
                        Write-Host ("Processing {0} from {1} and overwriting old version" -f $file.name, "$($folder.FullName)\$($myfolder1)") -ForegroundColor Green
                    }
                    else {
                        Write-Host ("File {0} from {1} already present in {2}, no change needed" -f $file.Name, "$($folder.FullName)\$($myfolder1)", $destinationpath) -ForegroundColor Green
                    }
                }
                else {
                    Copy-Item $file.FullName -Destination $destinationpath -ErrorAction Stop
                    Write-Host ("Copying new file {0} from {1} to {2}" -f $file.name, "$($folder.FullName)\$($myfolder1)", $destinationpath) -ForegroundColor Green
                }   
            }
        }
        else {
            Write-Warning ("Folder {0} was not found in {1}" -f $myfolder1, $folder.FullName)
        }
    }
    catch {
        Write-Warning ("Folder {0} not found in {1} or not enough permissions" -f $myfolder, $folder.FullName)
    }
}

In my case, it looks for a subdirectory pattern specified in the $myfolder1 variable, 'Test\Test2'.  It does that in the directory that is defined as $sourcepath. So, c:\mypath\20220421\Test\Test2 is processed with all files beneath that path including one in subdirectories below that. And c:\mypath\20220422\test\blabla is not.

[#16]

yes, just tried and it seems working, but this last version doesn't create the tree, it copies everything into the destpath...

I'm still waiting for the fourth log :)
I really think it's creating the folders list, if so it should work

I let it running and let you know

[#17]

It does copy everything in the root of the destpath without creating directories.. "into one folder destination." That's what you asked :p

[#18]

excuse me Harm, you are right

do you think it's possible to keep the source tree?

the very first version kept the tree (the only change to do there is to take the newest files and new ones)

[#19]

 @marco69 ok :) A light version using Robocopy mirrors the directories if the $myfolder1 pattern is found, copies changes files and removes files from the destination path if it's also deleted in the source path. Logfile is in c:\destpath\robocopy.log

$sourcepath = 'C:\MyPath'
$destinationpath = 'C:\DestPath'
$myfolder1 = 'Test\Test2'
foreach ($folder in Get-ChildItem -Path $sourcepath -Directory) {
        if (Test-Path "$($folder.fullname)\$($myfolder1)") {
            robocopy.exe "$($folder.fullname)\$($myfolder1)" "$($destinationpath)\$($folder.name)\$($myfolder1)" /mir /log+:"$($destinationpath)\robocopy.log" /X /TEE /FP /R:1 /W:1
    }
    else {
        Write-Warning ("Folder {0} not found in {1} or not enough permissions" -f $myfolder1, $folder.FullName)
    }
}

It shows the status on the screen, you can change/check the robocopy parameters if needed (Check them using robocopy /? in a command-prompt or on the website.

[#20]

EXCELLENT!!!!

in the afternoon I wll try soon

thank you again!!!

let you know

you have a wagon of beers (or whatever you want) for you approaching!!!!