Need to download a zip file from a website and unzip it and call its content in Log Analytics worksp

[MiteshAgrawal]

 Hi All,

I want to download a zip file from an external website and unzip the file.

The unzipped file contains 4 different excel which I want to read and write data to Sentinel. How can I achieve this?

Once I get these excel files, I want these to be reference to txt files contained in BLOBs which will be referenced in rules in Sentinel.

Please help. I am very new to Azure Sentinel and Azure cloud.

Regards,

Mitesh Agrawal

[#2]

@MiteshAgrawal 

You can see examples in the [Azure-Sentinel] channel, posts like this https://techcommunity.microsoft.com/t5/azure-sentinel/mapping-ips-to-autonomous-systems-number-and-n... and the link contained within, explain the use of Externaldata for ingesting files from URLs or BLOB.

Thanks 

[#3]

Hi Clive,

The link is quite helpful. Actually I want to automate the task which I have mentioned.

The zip file should be downloaded everyday and it should be unzipped. Later I can call the csvs using externaldata and create rules. But how to achieve the first thing.

Regards,

Mitesh Agrawal

[#4]

@MiteshAgrawal 

I'd start looking at a Logic App - which has a schedule (recurrence option) to allow you to schedule and then move the file to blob https://social.msdn.microsoft.com/Forums/en-US/5643f6de-0c2b-4697-9b72-8307a6447db6/download-payload...   Note: I've not tried this.

Its the unzip operation I'm not sure about...

Thanks Clive 

[#5]

Hi @CliveWatson,

I am exploring the option for Logic App. Used "Recurrence", "HTTP" and Azure Blob storage connectors and created a blob with the zip file. 

Thanks a lot for the link. It helped a lot. 

Now, I am looking for an option to unzip this zip file and create 4 more blobs from those 4 excel sheets in the zip file.

If you get to know anything then please update here. Also, if you have any other idea on achieving this then please tell me.

Regards,

Mitesh Agrawal

[#6]

Hi @CliveWatson ,

I got the below error when I tried to unzip the file.

```

{

  "status": 413,

  "message": "The file contains 50.5 megabytes which exceeds the maximum 50 megabytes.\r\nclientRequestId: abcd",

  "error": {

    "message": "The file contains 50.5 megabytes which exceeds the maximum 50 megabytes."

  },

  "source": "azureblob-ci.azconn-ci.p.azurewebsites.net"

```

How can I increase this limit?

By the way I got an option under Logic App connector - "Extract Archive to Folder" in order to unzip the .zip file.

Regards,

Mitesh Agrawal

[#7]

@MiteshAgrawal 

I believe that is a hard limit.  Other options (from a quick search, so there maybe other ways) seem to be to use a Azure Function http://www.frankysnotes.com/2019/02/how-to-unzip-automatically-your-files.html