Code Stucks at Microsoft.SharePoint.Client.File.OpenBinaryDirect Method

I am trying to download a file using ServeRelativeUrl  by OpenBinaryDirect Method and it works fine. But often,  irrespective of size, extension or any other file metadata it just get stucks.

Please see the code below, it gets stucks at either of the bolded lines (more often at ExecuteQuery()) and throws an operation timeout error, later I tried to give timeouts (shown in italic) but then it got stuck for infinite time until the main thread got killed.

clientContext.RequestTimeout = -1;
FileInformation fileInfo = Microsoft.SharePoint.Client.File.OpenBinaryDirect(clientContext, file.ServerRelativeUrl);
clientContext.ExecuteQuery();

using (var fileStream = new FileStream(location, FileMode.Create))
{

                    fileInfo.Stream.WriteTimeout = -1;

                    fileInfo.Stream.CopyTo(fileStream);
}
fileInfo.Stream.Dispose();

I have attached a snip of the explorer when my code got stuck at this particular file, it's size was 0kb after the main thread got killed.


Can anyone please help me out with this?