Forum Discussion
Andytcc
Feb 20, 2024Copper Contributor
Get Data
I'm not sure if I'm wording this right. and I'm also new to the following. I am using the "get data from the web" in Excel. I am receiving the data no problem. but as the file gets larger it takes t...
SergeiBaklan
Feb 21, 2024MVP
Bit more information is here Chris Webb's BI Blog: Comparing The Performance Of Reading Data From Files With File.Contents And Web.Contents In Power Query And Power BI (crossjoin.co.uk)
Lorenzo
Feb 21, 2024Silver Contributor
I might have misunderstood you. Re-reading, Andytcc said I'm actually getting the data from a csv file on an SD card in a PLC. I'm doing it with an IP address and the name of the file
So, where you trying to say he actually uses Web.Contents (probably) and not File.Contents?
(that would explain the confusion re. I am using the "get data from the web")
- SergeiBaklanFeb 21, 2024MVP
Perhaps that's me was confused with "get data from the web". Yes, assumed web connector.
Anyway, the task is not clear enough, at least for me. If refresh is once per week that's one story, if every hour that's another. Do we do refresh manually (e.g. on file opening) or Power Automate triggered on csv file update could work. Could we modify csv file update not to keep entire history in it (or keep it separately). Etc.
- AndytccFeb 22, 2024Copper ContributorRefresh is done every time file is opened could be multiple times a day to weekly.
What I mean by hourly is: PLC will do an hourly write to the csv file of the 201 values.
The reason for hourly is: The logging is for temperature in a Grain Bin. So if I do an hourly log that gives me a good deviation in a 24hr period between min/max temp and therefore I can see which sensors are in the grain and with that the number of Bushels still in the Bin (not accurately to the bushel but a very good idea).- LorenzoFeb 22, 2024Silver Contributor
If you want to post your Query code here...
In Excel
- Data (tab) > Queries & Connections (the corresponding pane opens on the right
- Right-click on your query > CopySwitch to this site
- Reply on this Discussion > Open full text editor
- In the Toolbar: Insert/Edit code sample
- Language: 'Power Query'
- In the text box below: Paste > OK
- LorenzoFeb 21, 2024Silver Contributor
Perhaps that's me was confused with "get data from the web". Yes, assumed web connector
And you're probably right as after re-reading (I overlooked on 1st reading): ...with an IP address
Anyway, the task is not clear enough, at least for me
Same here. Let's see how it develops...
- AndytccFeb 22, 2024Copper ContributorI reread your reply and refer to Question 4 in you first reply.
Q4: Not sure I understand the logic here. Shouldn't this be "Reverse Rows" and "Keep first Rows"?
Not sure (to be tested and with Power Query expect 'surprises') but instinctively I would do - only - Keep Bottom Rows (201) instead.
I actually "Keep last Rows" then "Reverse Rows"
- mathetesFeb 21, 2024Silver Contributor
With Lorenzo and SergeiBaklan having joined in, you're now in good hands, far more competent hands than mine.
- LorenzoFeb 21, 2024Silver Contributor
mathetes You're too humble and I'm not that competent, don't set the bar too high 🙂
- SergeiBaklanFeb 21, 2024MVP
You're too humble and I'm not that competent, don't set the bar too high 🙂
Yes, join.