Announcing New Fine-Tuning Capabilities with o1-mini Model on Azure OpenAI Service

We are thrilled to introduce the latest advancements in AI with the o1-mini model on Azure OpenAI Service. This new model is a game-changer, bringing enhanced reasoning capabilities that are particularly useful for tackling complex problems in various fields such as science, coding, and math.

Video provider requires cookies to play the video. Accept to continue or watch it directly on the provider's site.

Accept

What is the o1-mini Model?

The o1-mini model is adept at complex reasoning tasks, representing a significant advancement in AI capabilities. It can be used by healthcare researchers to annotate cell sequencing data, by physicists to generate complicated mathematical formulas needed for quantum optics, and by developers in all fields to build and execute multi-step workflows.

Introducing Reinforcement Fine-Tuning

Reinforcement Fine-Tuning (RFT) is a new model customization technique that enables developers to customize our models using dozens to thousands of high-quality tasks. By grading the model’s responses with provided reference answers, this technique reinforces how the model reasons through similar problems and improves its accuracy on specific tasks in that domain.

How Does RFT Work?

Graders play a crucial role in RFT by evaluating the responses during training. The grader compares the response generated by the model with the correct answer and gives a score from 0 to 1. This helps the model to reason better and improve its accuracy over time. Unlike Supervised Fine-Tuning (SFT), where the model is given the prompt and the completion at one go, RFT involves defining the graders and helping the model arrive at the logic behind choosing the answer with every new example.

Real-World Applications of RFT with o1-mini -

Reinforcement Fine-Tuning (RFT) unlocks the potential of the o1-mini model to address highly specialized and complex tasks across industries, delivering superior accuracy and domain-specific insights.

• Healthcare: Researchers can fine-tune the o1-mini model using RFT to annotate cell sequencing data with exceptional precision. By leveraging high-quality, graded annotations, the model learns to deliver more reliable and insightful outputs, accelerating breakthroughs in genomics and personalized medicine.
• Physics: Physicists can customize the o1-mini model to generate intricate mathematical formulas for quantum optics. With RFT, the model refines its ability to handle nuanced calculations, empowering researchers to address the most challenging problems in their field.

These tailored fine-tuning processes highlight the flexibility and power of the o1-mini model when equipped with RFT, enabling domain experts to achieve transformative results.

 Region Availability

The o1-mini RFT is supported in North Central US and Sweden Central. Initially, it will be a code-first experience, with a user interface to follow later on.

Example: Base Model vs. Reinforcement Fine-Tuned Model

To demonstrate the effectiveness of RFT, we compared the results of the base o1-mini model with the reinforcement fine-tuned o1-mini model on a small dataset.

1. Accuracy of Responses: This measures if the model gives out accurate response for a given question/ prompt which is not part of training dataset.
   
   Prompt - 
   Which of the following will not decrease mother to child transmission of HIV?
   A. Avoid breastfeeding 
   B. Vaginal delivery
   C. Zidovudine given to mother antenatally and to neonate after birth
   D. Vitamin A supplementation given to mother

2. Conciseness of responses: To measure along with accuracy, finetuned model is better in giving crisp responses which results in Lower number of input and output tokens.
   Prompt:
   Which of the following is not true for myelinated nerve fibers?
   A. Impulse through myelinated fibers is slower than non-myelinated fibers
   B. Membrane currents are generated at nodes of Ranvier
   C. Saltatory conduction of impulses is seen
   D. Local anesthesia is effective only when the nerve is not covered by myelin sheath

Conclusion

As we embark on this journey together, we are not just providing a service; we are creating innovations that can define the future of domain-specific AI applications. We look forward to building use cases across various industry verticals with reinforcement fine-tuning.

For more information, please reach out to your Microsoft representatives or account managers.