Average a set of values based on date

So I am trying to average a set of values that we record each day, then drop them into a different sheet. I want to do this for every day of the year, but when I pull the formula down in the table, it only moves my set of numbers being average down 1 spot instead of 6 like I want. I have attached two pictures of the example data that I want to average as well as the picture of the final results that I want out of it. What I am confused is how to get the next set of averages to populate on the right date without having to do it manually for the entire year.

Data Set 1: 3/31/24

Data Set 2: 4/1/24

End Result average compilation: