Over the past few weeks, I’ve been building and updating a dashboard that shows the development of the Covid-19 infection in New Zealand. With the ever-changing data sources, I ran into quite a few challenges and had to come up with some workarounds. I want to share with you some of my approaches and decisions. This article gives just a short overview. Watch the video below if you want to go into detail.
My main data source is the New Zealand Ministry of Health web site. The format and presentation of the data on that site changed a lot and I needed to update my queries almost daily. I also kept an Excel spreadsheet as a data source, where I kept track of some numbers that were not available in that form on the web site.
Starting out with Excel, I used a map chart with colour shading to show the numbers for each New Zealand region. That was already challenging. New Zealand's map chart needs a lot of tweaking to be integrated in a dashboard. The source data required a lot of cleaning up and massaging in Excel's Power Query, but I finally got a usable map and arranged a few charts around it.
When the Ministry of Health changed the reporting units to grouping by District Health Boards, which don't overlap with the local government region boundaries, I switched to Power BI, so I could plot points on the map instead of regions. Using Power BI also meant that I could publish the dashboard on the internet and let people interact with it.
I could re-use the Power Query code from Excel in Power BI, so the online version was configured very quickly. What followed was a series of query adjustments as the Ministry of Health changed table positions, table structure, column names, etc., almost daily.
I used Power Query commands for data cleanup, a few lookup tables in the data model and a lot of checking every day to make sure my dashboard works without errors.
For a more detailed look under the hood of this dashboard, watch this video. It is a recording of a session from M365May.com, a free online conference that runs sessions throughout the month of May 2020.