Forum Discussion
VBA or Power Query help: Iterate copy/paste from one sheet to another and save each iteration
- May 19, 2023
You can started with the VBA code to achieve your task.
Here's an example VBA code that you can use as a starting point:
Sub IterateAndCopyPaste() Dim wb As Workbook Dim wsInputs As Worksheet Dim wsAnalysis As Worksheet Dim wsSummaries As Worksheet Dim tblInputs As ListObject Dim tblAnalysis As ListObject Dim tblSummaries As ListObject Dim inputRow As ListRow Dim analysisRow As ListRow Dim lastRow As Long Dim i As Long ' Set references to the workbook and worksheets Set wb = ThisWorkbook Set wsInputs = wb.Worksheets("All Inputs") Set wsAnalysis = wb.Worksheets("Analysis") Set wsSummaries = wb.Worksheets("Summaries") ' Set references to the tables Set tblInputs = wsInputs.ListObjects("Inputs") Set tblAnalysis = wsAnalysis.ListObjects("Analysis_T") Set tblSummaries = wsSummaries.ListObjects("Summaries") ' Loop through each row in the Inputs table For i = 1 To tblInputs.DataBodyRange.Rows.Count ' Get the current input row Set inputRow = tblInputs.ListRows(i) ' Clear the cells in Analysis_T tblAnalysis.DataBodyRange.ClearContents ' Copy the input row to the first row of Analysis_T inputRow.Range.Copy tblAnalysis.DataBodyRange.Rows(1) ' Wait for calculations to complete (if necessary) ' Add code here to wait for the calculations to complete ' Copy all rows from Analysis_T to Summaries lastRow = tblSummaries.DataBodyRange.Rows.Count tblAnalysis.DataBodyRange.Copy Destination:=tblSummaries.DataBodyRange.Offset(lastRow) ' Delete rows from Summaries where Result 2 is "No" DeleteRowsWithNoResult2 wsSummaries, tblSummaries ' Clear the clipboard Application.CutCopyMode = False Next i End Sub Sub DeleteRowsWithNoResult2(ws As Worksheet, tbl As ListObject) Dim result2Column As Range Dim cell As Range ' Set the Result 2 column range Set result2Column = tbl.ListColumns("Result 2").DataBodyRange ' Loop through each cell in the Result 2 column For Each cell In result2Column.Cells ' Check if the value is "No" and delete the entire row If cell.Value = "No" Then cell.EntireRow.Delete End If Next cell End Sub
Please note that this is a basic example, and you may need to modify it based on your specific workbook structure and requirements. You'll also need to add code to wait for the calculations to complete, as indicated in the comments.
To run the code, press Alt+F11 to open the VBA editor in Excel, insert a new module, and paste the code into the module. You can then run the IterateAndCopyPaste macro to perform the desired iterations.
You can achieve the desired task using Power Query too in Excel.
Here's a step-by-step solution using Power Query:
- Open your Excel workbook and go to the "Data" tab.
- Click on "Get Data" and select "From Other Sources" -> "Blank Query". This will open the Power Query Editor.
- In the Power Query Editor, go to the "Home" tab and click on "Advanced Editor".
- Replace the existing code with the following Power Query code:
let Source = Excel.CurrentWorkbook(){[Name="Inputs"]}[Content], #"Filtered Rows" = Table.SelectRows(Source, each [#"Result 2"] = "Yes"), #"Removed Columns" = Table.RemoveColumns(#"Filtered Rows", {"Result 2"}), #"Renamed Columns" = Table.RenameColumns(#"Removed Columns",{{"Model", "Parts"}, {"Calculation 1", "Result 1"}, {"Calculation 2", "Result 2"}, {"Calculation 3", "Result 3"}}), #"Appended Queries" = Table.Combine({#"Renamed Columns", #"Renamed Columns", #"Renamed Columns"}) // Replace the number of repetitions here in #"Appended Queries" ā
5. In the code above, replace "Inputs" with the actual name of your table in the "All Inputs" sheet.
6. Replace the comment // Replace the number of repetitions here with the actual number of times you want to repeat the rows.
7. Click on "Done" to close the Power Query Editor.
This Power Query code will filter the rows based on "Result 2" column, remove the "Result 2" column, rename the columns accordingly, and append the resulting table multiple times based on the specified number of repetitions.
Once you've completed the above steps, Power Query will load the transformed data into a new table in Excel. The table will contain the desired iterations based on your requirements.
You can refresh the data in the table whenever needed by clicking on "Data" -> "Refresh All".
I hope the proposal solutions, with AI helps, using VBA or Power Query meets your requirements.
I have made some modifications to your template code to fit my needs. I am now at a point where I would like to paste as values instead of the traditional copy./paste. Do you have any suggestions for doing that? I suspect it will require modifying line 64. I have attached a copy of the xlsm file to this post.
Also, do you have any suggestions for improving the speed of this code?
Sub IterateAndCopyPaste_V2()
Dim wb As Workbook
Dim wsInputs As Worksheet
Dim wsAnalysis As Worksheet
Dim wsSummaries As Worksheet
Dim tblInputs As ListObject
Dim tblModel As ListObject
Dim tblAnalysis As ListObject
Dim tblSummaries As ListObject
Dim inputRow As ListRow
Dim analysisRow As ListRow
Dim lastRow As Long
Dim i As Long
' Temporary ovewrite value
Dim Temp As String
Temp = "Temp"
' Set references to the workbook and worksheets
Set wb = ThisWorkbook
Set wsInputs = wb.Worksheets("Inputs")
Set wsAnalysis = wb.Worksheets("Analysis")
Set wsSummaries = wb.Worksheets("Summaries")
' Set references to the tables
Set tblInputs = wsInputs.ListObjects("Inputs_T")
Set tblModel = wsAnalysis.ListObjects("Model_T")
Set tblAnalysis = wsAnalysis.ListObjects("Analysis_T")
Set tblSummaries = wsSummaries.ListObjects("Summaries_T")
' Helper variables
num_Inputs_rows = tblInputs.DataBodyRange.Rows.Count
On Error Resume Next ' This is a way to handle the error when SpecialCells(xcellTypeBlanks) is empty and returns an error
num_Inputs_empty_rows = tblInputs.DataBodyRange.SpecialCells(xlCellTypeBlanks).Rows.Count
On Error GoTo 0
data_num_Inputs_rows = num_Inputs_rows - num_Inputs_empty_rows
' Clear tblSummaries before beginning first/new iteration
tblSummaries.Range(2, 1).Value = Temp ' This is a temporary fix to allow table to clear in next line
tblSummaries.DataBodyRange.ClearContents
tblSummaries.DataBodyRange.Delete
tblSummaries.Range(2, 1).Value = Temp ' Temp is added back in to allow the For loop to identify LastRow for i = 1 (throws an error without this)
' Loop through each row in tblInputs
For i = 1 To data_num_Inputs_rows
' Get the current input row
Set inputRow = tblInputs.ListRows(i)
' Clear the cells in Model_T
tblModel.DataBodyRange.ClearContents ' This prevents a pop-up on every iteration asking to replace cells with existing data
' Copy the input row to the first row of Model_T
inputRow.Range.Copy tblModel.DataBodyRange.Rows(1)
' Wait for calculations to complete (if necessary)
' (In testing this is a buffer to observe impact of code modifications)
Application.Wait (Now + TimeValue("0:00:05"))
' Determine which row the Analysis data should be pasted in based on current iteration (i)
lastRow = tblSummaries.ListRows.Count
' Copy all rows from tblAnalysis to tblSummaries
tblAnalysis.DataBodyRange.Copy Destination:=tblSummaries.DataBodyRange.Offset(lastRow - 1)
If i = data_num_Inputs_rows Then
tblModel.DataBodyRange.ClearContents ' Clear tblModel
MsgBox "Analysis complete. Go to the Summaries sheet and make an offline copy."
End If
' Clear the clipboard
Application.CutCopyMode = False
Next i
End Sub
'Sub DeleteBlankRows_V2(ws As Worksheet, tbl As ListObject)
' Dim AllSummaries As Range
' Set AllSummaries = tbl.DataBodyRange
' AllSummaries.SpecialCells(xlCellTypeBlanks).EntireRow.Delete
'End Sub
This file is the right one.
Also, you can disregard the comment about speed for now. It is likely tied to Application.Wait
- NikolinoDEMay 23, 2023Gold Contributor
With out opening any file (for own security reasons at the moment) maybe you can replace the line inputRow.Range.Copy tblModel.DataBodyRange.Rows(1) with the following code:
inputRow.Range.Copy tblModel.DataBodyRange.Rows(1).PasteSpecial Paste:=xlPasteValues Application.CutCopyMode = False
This code copies the range from inputRow.Range and then pastes only the values into tblModel.DataBodyRange.Rows(1) using the PasteSpecial method. The Application.CutCopyMode = False line is added to clear the clipboard.
Regarding the speed optimization, here are a few suggestions:
- Disable screen updating and events: Add Application.ScreenUpdating = False and Application.EnableEvents = False before the loop and set them back to True after the loop completes. This prevents unnecessary screen refreshing and event triggering during the execution of the loop.
- Use arrays for data manipulation: Load the input range and destination range into arrays, perform the necessary operations in memory, and then write the updated arrays back to the worksheet. Working with arrays can significantly improve the performance compared to reading and writing cell-by-cell.
- Minimize interactions with the worksheet: Instead of repeatedly clearing the contents of tables or individual ranges, consider resizing and overwriting existing data directly.
- Use efficient methods for deleting blank rows: If you need to delete blank rows, consider using advanced filtering or the AutoFilter method to identify and delete the blank rows in a single operation, rather than row-by-row deletion.
Remember to thoroughly test any modifications to ensure they produce the desired results and maintain data integrity.
These suggestions are general performance improvement techniques that can be applied to various scenarios.
- Antonino2023May 25, 2023Brass Contributor
Hello,
Can you please explain point 2 about speed optimization more? For example, how would I re-write my existing code to use arrays in place of cell-by-cell operations?
- NikolinoDEMay 25, 2023Gold Contributor
Certainly! To optimize your code by using arrays, you would need to load the data from your input range and destination range into arrays, perform the necessary operations on the arrays, and then write the updated arrays back to the worksheet.
Here's an example of how you can modify your existing code to use arrays:
Dim inputRange As Range Dim inputData As Variant Dim destinationRange As Range Dim destinationData As Variant Dim i As Long Set inputRange = Sheets("Sheet1").Range("A1:D10") ' Update with your input range inputData = inputRange.Value Set destinationRange = Sheets("Sheet2").Range("A1:D10") ' Update with your destination range destinationData = destinationRange.Value ' Perform your operations on the arrays For i = LBound(inputData, 1) To UBound(inputData, 1) ' Example: Copy values from column 2 to column 3 destinationData(i, 3) = inputData(i, 2) Next i ' Write the updated arrays back to the worksheet destinationRange.Value = destinationData
In this example, inputData and destinationData are variant arrays that store the values from the input range and destination range, respectively. You can then perform your desired operations on the arrays, accessing and modifying the values directly.
Note that arrays in VBA are 1-based, so the index starts from 1 instead of 0. Therefore, the LBound and UBound functions are used to determine the lower and upper bounds of the arrays.
After performing the operations on the arrays, you can assign the updated array back to the destination range using the Value property.
Using arrays can significantly improve performance compared to cell-by-cell operations because you are minimizing interactions with the worksheet and working with data in memory. This is especially beneficial when dealing with large ranges or performing complex calculations.
- Antonino2023May 23, 2023Brass Contributor
Thank you for the suggestions! I will be sure to implement those performance techniques. I will definitely mark your responses as the Best Answers but am leaving this post open since I am adding more functionality to the code and will likely need some help when I get there.
- Antonino2023May 23, 2023Brass Contributor
I think I got it! I referenced this link: https://stackoverflow.com/questions/24294923/how-to-copy-only-values-in-excel-vba-from-a-range
I used the PasteSpecial xlPasteValues function as shown in the modified code below:
Also, please use this new file to test it if you would like. Let me know if it does not let you make a downloadable copy of it.
Sub IterateAndCopyPaste_V2() Dim wb As Workbook Dim wsInputs As Worksheet Dim wsAnalysis As Worksheet Dim wsSummaries As Worksheet Dim tblInputs As ListObject Dim tblModel As ListObject Dim tblAnalysis As ListObject Dim tblSummaries As ListObject Dim inputRow As ListRow Dim analysisRow As ListRow Dim lastRow As Long Dim i As Long Dim rSource As Range 'Source Range Dim rDest As Range 'Target Range ' Temporary ovewrite value Dim Temp As String Temp = "Temp" ' Set references to the workbook and worksheets Set wb = ThisWorkbook Set wsInputs = wb.Worksheets("Inputs") Set wsAnalysis = wb.Worksheets("Analysis") Set wsSummaries = wb.Worksheets("Summaries") ' Set references to the tables Set tblInputs = wsInputs.ListObjects("Inputs_T") Set tblModel = wsAnalysis.ListObjects("Model_T") Set tblAnalysis = wsAnalysis.ListObjects("Analysis_T") Set tblSummaries = wsSummaries.ListObjects("Summaries_T") ' Helper variables num_Inputs_rows = tblInputs.DataBodyRange.Rows.Count On Error Resume Next ' This is a way to handle the error when SpecialCells(xcellTypeBlanks) is empty and returns an error num_Inputs_empty_rows = tblInputs.DataBodyRange.SpecialCells(xlCellTypeBlanks).Rows.Count On Error GoTo 0 data_num_Inputs_rows = num_Inputs_rows - num_Inputs_empty_rows ' Clear tblSummaries before beginning first/new iteration tblSummaries.Range(2, 1).Value = Temp ' This is a temporary fix to allow table to clear in next line tblSummaries.DataBodyRange.ClearContents tblSummaries.DataBodyRange.Delete tblSummaries.Range(2, 1).Value = Temp ' Temp is added back in to allow the For loop to identify LastRow for i = 1 (throws an error without this) ' Loop through each row in tblInputs For i = 1 To data_num_Inputs_rows ' Get the current input row Set inputRow = tblInputs.ListRows(i) ' Clear the cells in Model_T tblModel.DataBodyRange.ClearContents ' This prevents a pop-up on every iteration asking to replace cells with existing data ' Copy the input row to the first row of Model_T inputRow.Range.Copy tblModel.DataBodyRange.Rows(1) ' Wait for calculations to complete (if necessary) ' (In testing this is a buffer to observe impact of code modifications) Application.Wait (Now + TimeValue("0:00:05")) ' Determine which row the Analysis data should be pasted in based on current iteration (i) lastRow = tblSummaries.ListRows.Count ' Copy all rows from tblAnalysis to tblSummaries ' tblAnalysis.DataBodyRange.Copy Destination:=tblSummaries.DataBodyRange.Offset(lastRow - 1) Set rSource = tblAnalysis.DataBodyRange Set rDest = tblSummaries.DataBodyRange.Offset(lastRow - 1) rSource.Copy rDest.PasteSpecial xlPasteValues If i = data_num_Inputs_rows Then tblModel.DataBodyRange.ClearContents ' Clear tblModel MsgBox "Analysis complete. Go to the Summaries sheet and make an offline copy." End If ' Clear the clipboard Application.CutCopyMode = False Next i End Sub 'Sub DeleteBlankRows_V2(ws As Worksheet, tbl As ListObject) ' Dim AllSummaries As Range ' Set AllSummaries = tbl.DataBodyRange ' AllSummaries.SpecialCells(xlCellTypeBlanks).EntireRow.Delete 'End Sub
I may have more questions as I develop the code a bit more.