Determining sizing requirements for GPU enabled Azure VM
Greetings, We are trying to determine the correct VM sizing requirement for our AI workload, which is used for NLP processing. This workload does not require any training, but will only be used for inference. We have the following software configuration: a C# application that is heavily multithreaded using a lot of socket I/O. The application has concentrated bursts where 10-20 threads are fired concurrently to perform tasks (mostly socket I/O). This app communicates via dedicated sockets to: a Python application which performs various NLP tasks. This app is also multithreaded to handle multiple incoming requests from the .NET app. This app sends queries to a local LLM (model size will vary based on query type). We estimate we will need to support sub-second performance (at the very least) on a 7B parameter model. Ultimately, we may need to go to larger model sizes if accuracy is insufficient. The amount of text passed to the LLM will range from 300-3000 tokens. In short, we need: a) a CPU with sufficient cores to handle multiple concurrent threads on the .NET side. The app will have 5 or 6 background threads running continuously, and sudden bursts of activity which will require a minimum of 10-20 threads to run shorter-lived tasks. b) a GPU with sufficient VRAM to handle at the very least, a 7B parameter model. Ultimately, we may need to support larger models to perform the same task due to insufficient accuracy. We need the ideal configuration of GPU/VRAM and CPU/RAM to handle these tasks, and also, potentially, larger LLM sizes of up to 14B or 70B parameters. We are looking at the NC-series VMs, with a budget of about $1,000/month (see https://learn.microsoft.com/en-us/answers/questions/2150959/determining-sizing-requirements-for-gpu-enabled-az?comment=question). Any feedback on the optimal configuration in terms of CPU/GPU would be greatly appreciated. Thank you in advance.
Unable to load large delta table in azure ml studio
I am writing to report an issue that I am currently experiencing while trying to read a delta table from Azure ML. I have already created data assets to register the delta table, which is located at an ADLS location. However, when attempting to load the data, I have noticed that for large data sizes it is taking an exceedingly long time to load. I have confirmed that for small data sizes, the data is returned within few seconds, which leads me to believe that there may be an issue with the scalability of the data loading process. I would greatly appreciate it if you could investigate this issue and provide me with any recommendations or solutions to resolve this issue. I can provide additional details such as the size of the data, the steps I am taking to load the data, and any error messages if required. I'm following this document: https://learn.microsoft.com/en-us/python/api/mltable/mltable.mltable?view=azure-ml-py#mltable-mltable-from-delta-lake Using this command to read delta table using data asset URI from mltable import from_delta_lake mltable_ts = from_delta_lake(delta_table_uri=<DATA ASSET URI>, timestamp_as_of="2999-08-26T00:00:00Z", include_path_column=True)
Azure Machine Learning compute cluster - avoid using docker?
Hello, I would like to use an Azure Machine Learning Compute Cluster as a compute target but do not want it to containerize my project. Is there a way to deactivate this "feature" ? The main reasons behind this request is that : I already set up a docker-compose file that is used to specify 3 containers for Apache Airflow and want to avoid a Docker-in-Docker situation. Especially that I already tried to do so but failed so far. I prefer not to use a Compute Instance as it is tied to an Azure account which is not ideal for automation purposes. Thanks in advance.Azure VM automatisierung in 3rd Party Software
Hallo zusammen Wir haben eine Azure VM am laufen mit Windows Server 2019. Auf dieser läuft dauerhaft eine 3Party Software, welche aber möchte, dass man sich alle 12h neu mit dem Passwort einloggt. Würde sich dies irgendwie auf der VM Automatisieren lassen? Es müsste alle 12 Stunden folgende Befehle ausgeführt werden: -Abmelden Button Drücken -Anmelden Button Drücken -Passwort eingeben -OK Button drücken Wie würdet Ihr dies lösen? Lässt sich dies mit einem Script machen oder geht das nur via 3rd Party Software? Danke für eure Vorschläge! Lg Dario
Azure Batch AI example using tensorflow
Hi currently I am assessing the Azure frameworks. I found a nice example how to run CNTK on parallel CPUs/GPUs using the CLI interface https://docs.microsoft.com/en-gb/azure/batch-ai/quickstart-cli Now I wonder if it is similarly easy to run a convolutional neural network for the MNIST database using tensorflow? Does anyone have experience with this set up? Regards MarkusData Simulator for Machine Learning
Virtually any data science experiment that uses a new machine learning algorithm requires testing across different scenarios. Simulated data allows one to do this in a controlled and systematic way that is usually not possible with real data. A convenient way to implement and re-use data simulation in Azure Machine Learning (AML) Studio is through a custom R module. Custom R modules combine the convenience of having an R script packaged inside a drag and drop module, with the flexibility of custom code where the user has the freedom of adding and removing functionality parameters, seen as module inputs in the AML Studio GUI, as needed. A custom R module has identical behavior to native AML Studio modules. Its input and output can be connected to other modules or be set manually, and they can process data of arbitrary schema, if the underlying R code allows it, inside AML experiments. An added benefit is that they provide a convenient way of deploying code without revealing the source, which may be convenient for IP sensitive scenarios. By publishing it in Cortana Intelligence Gallery one can easily expose to the world any algorithm functionality without worrying about classical software deployment process. Read more on the https://azure.microsoft.com/en-us/blog/data-simulator-for-machine-learning-custom-r-module-cortana-intelligence-gallery/
