We have an Linux based queue triggered Python Azure Function App running on consumption plan. We have a process running on another server to ingest about 4 to 5 queue message per second after uploading a dozen blobs into Azure blob storage which can go up to a couple of thousand messages/ tens of thousands blobs put in the storage account in a span of 15 minutes.
The Azure function app then processes the queue messages reading a dozen blobs using the Blob URL mentioned in each queue message, processes and outputs the data to blob storage in parallel as it auto scales.
The whole program is written in python pandas, that reads a csv file using Azure blob URL, processes it and pushes csv back into a different container. A few instances of the function randomly fails saying that there's a temporary failure in name resolution of input blobs and URL cannot be opened.
I checked the BlobURL in question and was able to access and download the file directly using the browser. The failures are random and sometimes doesn't happen. Each blob itself is not accessed concurrently by different instances, however the container is accessed by tens of function instances concurrently.
One area of doubt is that since the blobs are uploaded asynchronously by another program, the function app might be trying to download it before the upload was complete but we do await the upload to be completed before putting in the queue message and even if it was a case I would expect to receive a blob does not exist error instead of a name resolution.
I tried to look up on the internet but couldn't find anything related to this specific case but a general search on the error itself suggests that the error might be related to server/host side which in this case is Azure function. Any thoughts/help would be much appreciated.
The error we get is below. Happy to post the full stack trace if needed
URLError: <urlopen error [Errno -3] Temporary failure in name resolution>
