RAG with Azure AI: why your retrieval strategy matters AMA

This is a fantastic question! Small Language Models (SLMs) are very new and there is lots of research going on to find out more about their use case. Here are a few articles from Microsoft that talk about them more in depth: Phi-2: The surprising power of small language models - Microsoft Research and Orca 2: Teaching Small Language Models How to Reason - Microsoft Research.  

My takeaways from the research are the following: 

1. Specialized Task Handling: SLMs like Phi-2 can be tailored for specific domains or tasks such as legal document analysis or technical support queries. In a RAG setup, these models can act as specialized agents that handle particular types of queries with high precision, complementing the broader knowledge base of LLMs. 
2. Efficient Re-Ranking: SLMs can be used to quickly re-rank the results retrieved by an initial search query based on more nuanced criteria or domain-specific knowledge. An SLM trained in specific criteria can refine and re-rank results before presenting them to the user, improving relevance and accuracy. Note, Azure AI Search has a re-ranker that is SOTA already 🙂 Semantic ranking - Azure AI Search | Microsoft Learn
3. Augmenting Search Queries: Before executing a search query, an SLM could preprocess the query to better capture the user’s intent or expand it with additional relevant terms. 
4. Post-Search Query Analysis: After initial results are retrieved and presented, SLMs can offer an additional layer of interaction, where they analyze user feedback or follow-up questions to refine the search results further.  

I think the integration of SLMs with LLMs in an RAG architecture offers a pretty compelling approach to creating more efficient, accurate, and context-aware search and information retrieval systems. However, this space is pretty new and we’re all continuously experimenting to see how SLMs work best in the context of RAG. Hopefully, more to come soon!