Web Query not refreshing updated data from website

Copper Contributor

I have been attempting to pull flight schedules from a website using Excel's web query. 

The initial time I requested information, I got the information back exactly how I wanted it but it appears to be stuck on that view and when I refresh it, nothing changes even though the websites content has since changed.

 

I have attached some images highlighting the issue, the excel sheet shows outdated information by about an hour. 

 

I have been working on this across several computers and it seems the first time the web query is performed on a new computer it will update correctly but then get stuck on that info and wont refresh.

I've since learned about BF caching where the saves a snapshot of the website so that if the client returns the content is already ready, i think this may have something to do with the problem but all the tutorials ive seen online seem to show exactly what i did but their attempts at refreshing live data works.

 

I came across a post where they recommended Fiddler and I tried that, i recorded the http requests when I recorded myself trying to refresh the query and no direct link occurred with the URL i specified. there was a lot of tunnel to random things, just not the flight schedule website that I specified into the query.

 

I'd really appreciate any insight into how I may resolve this

 

Edit: URL is https://www.wellingtonairport.co.nz/flights/?direction=departure

In my last attempt, I added '&today=refresh' to the URL source, this appeared to work and it did initially refresh changes that were occuring. I then saved it, uploaded the workbook to onedrive. I have since reopened the workbook both through one drive and locally and it had reverted to being static.

 

Excel outdated web query data.pngLive Website.pngOutdated preview.pngURL from live website.pngadvanced editor.png

10 Replies

@Dezmond95 It's going to be difficult to help if you don't share the URL.

@Dezmond95 Thanks. Not much happening there right now, but I've connected to the site both Arrivals and Departures and will see what happens in the next hour or so.

@Riny_van_Eekelen thanks, I really appreciate you taking a look. 

@Dezmond95 Two arriving flights just had a status change and one disappeared from the table.

NZ457 initially had "Landed 21:18". A few minutes later it updated to "Arrived at Gate".

NZ459 was still in the air when I created the macro, thus no status. It landed 21:34 though the website took 12 minutes to show it. The query still doesn't pick up this one but it does reflect others.

Riny_van_Eekelen_1-1678351736516.png

The "First connection" info reflects the Arrival status from some approx. 30 minutes before the PQ status. So, there are updates, but it seems to be with quite some delay.

 

 

 

so i've just opened my old workbook and it's displaying information from over an hour ago.

 

but with the recent version showing arrivals, it is staying up to date, it seems to be when i reopen the document the connection seems to be lost. 

 

are you having any issues reloading the content after closing the file?

 

Dezmond95_0-1678353682421.png

 

 

@Riny_van_Eekelen 

@Dezmond95 Just re-opened the file. Refreshed and now it shows this for arrivals.

Riny_van_Eekelen_0-1678353778765.png

 

See attached. The file still has the status before a Refresh

so in the file it shows the scheduled time as 2355 and estimated time as 2355, the website now displays 2355 and 2550 respectively. Does your refreshed version include that update?
Has your query, refreshed to show the new days flights, mine is stuck from yesterdays point.

@Dezmond95 I attached my file earlier. When I open it now, it shows a long list of arrivals from 07:00 through 23:55

 

Departures don't want to update, though. Did you try my file?