Forum Discussion
KiranAlamuru
Jan 23, 2020Copper Contributor
Split large xml file to small using powershell
Hi All, I have requirement to split the large xml file (xmls will not be identical and with different nodes/tags/structure with in; and size will be more than 400MB+ ) to small xml files (around 2MB...
Erick A. Moreno R.
Feb 21, 2020Iron Contributor
KiranAlamuru I've used this snipped before for similar purposes. I've tested it with your requirements and should be good.
$FileFolder = "S:\--106-- Downloads\"
$FileName = "enwiki-20110115-abstract6"
$FileExt = "xml"
$LimitSize = 2MB
$FilePath = $FileFolder + $FileName + '.' + $FileExt
$ReadFile = [io.file]::OpenRead($FilePath)
$Size = new-object byte[] $LimitSize
$RemainingData = 0
$ID = 0
Do
{
$RemainingData = $ReadFile.Read($Size, 0 , $Size.Length)
If($RemainingData -gt 0)
{
$NewChunk = $FileFolder + $FileName + "-$ID-" + '.' + $FileExt
$ChunkFile = [io.file]::OpenWrite($NewChunk)
$ChunkFile.Write($Size, 0 , $RemainingData)
$ChunkFile.Close()
}
$ID ++
}While($RemainingData -gt 0)
$ReadFile.Close()
M-A_Charlotte
Jun 30, 2022Copper Contributor
Erick A. Moreno R. Thanks, it seems a good script to split a large file into multiple 2 MB files however it cannot keep the XML structure intact and in the end, you will get many broken XML files.
Best,