Responsible Synthetic Data Creation for Fine-Tuning with RAFT Distillation
This blog will explore the process of crafting responsible synthetic data, evaluating it, and using it for fine-tuning models. We’ll also dive into Azure AI’s RAFT distillation recipe, a novel approach to generating synthetic datasets using Meta’s Llama 3.1 model and UC Berkeley’s Gorilla project.
Using Neural Network to Learn Profitable Trading in the FOREX Markets
I am using Neural Networks (NN) to teach them how to recognize profitable trading opportunities in the Foreign Exchange (FOREX) markets, using 10 currencies simultaneously. I am using 3rd-order Cubic Splines as input to give the NNs a sense of how the critical variables change over time. I am using free FOREX historical trading data to train the NNs how to trade profitably in the future. I don't just feed the trading levels of the FOREX currency pairs as input to the NNs. Instead, I use a variation of the computed DXY Index for all 10 currencies in order to isolate the value change of each of the individual currencies, using Cubic Splines to detail how those values change over various time periods. The end result is Neural Networks that recognize which currencies to Buy and which ones to Sell at the most profitable times. If anyone is interested in the details, please reach out and I will provide more details.
Training and Inference of LLMs with PyTorch Fully Sharded Data Parallel and Better Transformer
In this blog we show how to perform efficient and optimized distributed training and inference of large language models using PyTorch’s Fully Sharded Data Parallel and Better Transformer implementations, on the Spark platform. In this implementation, we combine Microsoft Fabric for data preparation and model inference, and Azure Databricks for model training, having all our data under Microsoft Fabric’s OneLake. The code for this blog is available at this GitHub repository, as a series of PySpark notebooks for Microsoft Fabric and Azure Databricks.
