Fine Tune Mistral Models on Azure AI Foundry

RashaudSavage

Microsoft

Feb 05, 2025

Customizing and optimizing the performance of Mistral Large 2411, Nemo and Ministral 3B

We're excited to announce the general availability of fine-tuning for Mistral models on Azure is now live! Starting today, Mistral Large 2411, Mistral Nemo, and Ministral 3B fine-tuning are available to all our Azure AI Foundry customers, providing unmatched customization and performance. This also establishes Azure AI Foundry as the second platform, after Mistral's own, where fine-tuning of Mistral models is currently available. Azure AI Foundry lets you tailor large language models to your personal datasets by using a process known as fine-tuning. Fine-tuning provides significant value by enabling customization and optimization for specific tasks and applications. It leads to improved performance, cost efficiency, reduced latency, and tailored outputs.

Finetuning enabled Mistral Models

Mistral Large 2411

Mistral Large 24.11 is an advanced Large Language Model (LLM) with state-of-the-art reasoning, knowledge and coding capabilities. Designed to support multiple languages, including English, French, German, Spanish, Italian, Chinese, Japanese, Korean, Portuguese, Dutch, and Polish. Mistral large is highly proficient in coding, with training in over 80 programming languages such as Python, Java, C, C++, JavaScript, and Bash, as well as specialized languages like Swift and Fortran.

Mistral large emphasizes agent-centric capabilities, providing top-tier agent functionalities with native function calling and JSON output. It is equipped with advanced reasoning skills, featuring state-of-the-art mathematical and logical capabilities.

Mistral Nemo 2407

Mistral Nemo is an advanced Language Model (LLM) that excels in reasoning, world knowledge, and coding within its size category. Developed in collaboration with Nvidia, this powerful 12B model pushes the boundaries of language understanding and generation.

Mistral Nemo features multilingual proficiency with a new tokenizer, Tekken, designed for multilingual applications. It supports over 100 languages, including English, French, German, Spanish, Italian, Chinese, Japanese, Korean, Portuguese, Dutch, Polish, and many more. Tekken is more efficient than the Llama 3 tokenizer, compressing text for approximately 85% of all languages, with significant improvements in Malayalam, Hindi, Arabic, and prevalent European languages.

Mistral Nemo also boasts top-tier agentic capabilities, including native function calling and JSON outputting. Additionally, it demonstrates state-of-the-art mathematical and reasoning capabilities within its size category.

Ministral 3B

Ministral 3B is a cutting-edge Small Language Model (SLM) designed for edge computing and on-device applications. Its low-latency and compute-efficient inference make it ideal for standard GenAI applications that require real-time processing and handle high volumes.

With 3.6 billion parameters, Ministral 3B sets a new benchmark in knowledge, commonsense reasoning, function-calling, and efficiency within the sub-10B category. This model can be utilized or fine-tuned for various purposes, from orchestrating agentic workflows to creating specialized task workers.

Serverless Finetuning of Mistral Models

Fine-tuning is a powerful technique for customizing and optimizing the performance of large language models (LLMs) for specific use cases. By further training a pre-trained LLM on a labeled dataset related to a particular task, fine-tuning can improve the model's performance. This can be done with a large model for complex or dissimilar tasks, or with a smaller model to match the performance of a larger model, potentially leading to latency and cost benefits. The performance increase varies depending on the use cases.

To fine-tune a Mistral models model:

Sign in to Azure AI Foundry.
Choose the model you want to fine-tune from the Azure AI Foundry portal model catalog.
On the model's Details page, select fine-tune.
Select the project in which you want to fine-tune your models. To use the pay-as-you-go model fine-tune offering, your workspace must belong to the East US 2 region.
On the fine-tune wizard, select the link to Azure Marketplace Terms to learn more about the terms of use. You can also select the Marketplace offer details tab to learn about pricing for the selected model.
If this is your first time fine-tuning the model in the project, you have to subscribe your project for the particular offering (for example, Ministral-3B) from Azure Marketplace. This step requires that your account has the Azure subscription permissions and resource group permissions listed in the prerequisites. Each project has its own subscription to the particular Azure Marketplace offering, which allows you to control and monitor spending. Select Subscribe and fine-tune.

Note

Subscribing a project to a particular Azure Marketplace offering (in this case, Ministral-3B) requires that your account has Contributor or Owner access at the subscription level where the project is created. Alternatively, your user account can be assigned a custom role that has the Azure subscription permissions and resource group permissions listed in the prerequisites.

Once you sign up the project for the particular Azure Marketplace offering, subsequent fine-tuning of the same offering in the same project don't require subscribing again. Therefore, you don't need to have subscription-level permissions for subsequent fine-tune jobs. If this scenario applies to you, select Continue to fine-tune.
Enter a name for your fine-tuned model and the optional tags and description.
Select training data to fine-tune your model. See data preparation for more information.

Note

If you have your training/validation files in a credential less datastore, you will need to allow workspace managed identity access to their datastore in order to proceed with MaaS finetuning with a credential less storage. On the "Datastore" page, after clicking "Update authentication" > Select the following option:

Make sure all your training examples follow the expected format for inference. To fine-tune models effectively, ensure a balanced and diverse dataset. This involves maintaining data balance, including various scenarios, and periodically refining training data to align with real-world expectations, ultimately leading to more accurate and balanced model responses.

The batch size to use for training. When set to -1, batch_size is calculated as 0.2% of examples in training set and the max is 256.

The fine-tuning learning rate is the original learning rate used for pretraining multiplied by this multiplier. We recommend experimenting with values between 0.5 and 2. Empirically, we've found that larger learning rates often perform better with larger batch sizes. Must be between 0.0 and 5.0.

Number of training epochs. An epoch refers to one full cycle through the data set.

Task parameters are an optional step and an advanced option- Tuning hyperparameter is essential for optimizing large language models (LLMs) in real-world applications. It allows for improved performance and efficient resource usage. The default settings can be used or advanced users can customize parameters like epochs or learning rate.
Review your selections and proceed to train your model.

Once your model is fine-tuned, you can deploy the model and can use it in your own application, in the playground, or in prompt flow

Get started today!

Whether you're a newcomer to fine-tuning or an experienced developer, getting started with Azure AI Foundry is now more accessible than ever. Fine-tuning is available through both Azure AI Foundry and Azure ML Studio, offering a user-friendly interface for those who prefer a graphical user interface (GUI) and SDK’s and CLI for advanced users.

Learn more!