Powering the next generation of Generative AI applications
This year, the field of AI has experienced unparalleled growth, propelling groundbreaking advancements in generative AI technologies such as OpenAI's GPT-family of Large Language Models (LLMs) [and ChatGPT service]. By joining LLMs with contemporary retrieval systems, organizations can create generative AI applications that reflect their own content in natural language, regardless of the size of their data estates. Serving as a one of the foundational pillars for Generative AI applications, search and retrieval capabilities facilitate efficient navigation through immense data sets, enabling innovative and contextually relevant outputs for users spanning various industries.
A key capability that underpins how search can both leverage and enhance Generative AI technologies is Vector search. We are delighted to announce the public preview of Vector search in Azure Cognitive Search a fundamental capability for building applications powered by large language models.
Vector search is a method of searching for information within various data types, including image, audio, text, video, and more. It determines search results based on the similarity of numerical representations of data, called vector embeddings. Unlike keyword matching, Vector search compares the vector representation of the query and content to find relevant results for users. Azure OpenAI Service text-embedding-ada-002 LLM is an example of a powerful embeddings model that can convert text into vectors to capture its semantic meaning.
The integration of Vector search seamlessly extends to other capabilities of Azure Cognitive Search, including faceted navigation, filters, and more. Using Azure Cognitive Search’s Indexer, customers can use data from diverse Azure datastores, such as Blob storage, Azure SQL and Cosmos DB, to inform a single generative AI-powered application. The enterprise-grade security and scalability our customers expect is built in.
Vector search is integrated with Azure AI, allowing customers to,
- Build search-enabled, chat-based applications using the Azure OpenAI Service
- Convert images into vector representations using Azure AI Vision for accurate, relevant text-to-image and image-to-image search experiences. Learn more here
- Quickly and accurately retrieve relevant information from large datasets, to help automate processes and workflows
Unlocking the Power of the Retrieval Augmented Generation (RAG) Pattern
The emergence of the RAG pattern has been driven by the growing need to integrate LLMs with custom data into application workflows to create bespoke solutions and grounding for enterprise copilots.
Developers can retrieve relevant information using Vector search, analyze and understand the retrieved data, and generate intelligent responses or actions based on the LLM’s capabilities. This pattern enables organizations to create innovative applications while incorporating domain-specific knowledge and context from their own data sources.
Beyond Pure Vectors to Hybrid Retrieval
A hybrid search approach that harnesses both vector and traditional keyword scores delivers even better retrieval result quality than a single search method alone. With the introduction of hybrid search, Azure Cognitive Search now enables supercharged search experiences.
Enabling Powerful Search Applications
Azure OpenAI Service text-embedding-ada-002 is effective for developing vector embeddings. Numerous open source models are also available to perform this function. In some cases, custom models are necessary to support unique business solutions. With Vector search in Azure Cognitive Search, customers have the flexibility to leverage pretrained or custom embedding models depending on their specific use case.
For instance, Schroders, a financial services provider, relies on Azure Cognitive Search and Azure OpenAI Service to enable efficient multi-modal search within their AI solution, Genie.
Recognized as a Leader in Insight Engines by Gartner
Microsoft has been recognized as a Leader in the 2022 Gartner® Magic Quadrant™ for Insight Engines. With advanced capabilities and robust features, Azure Cognitive Search empowers organizations to unlock the value of their data, delivering impactful insights. To learn more, read the report:
Microsoft named a Leader in 2022 Gartner® Magic Quadrant™ for Insight Engines
Get Started with Vector search in Azure Cognitive Search
We can’t wait to see what you build with the advanced capabilities now enabled in Azure Cognitive Search! Unlock the true value of your data today and gain impactful insights. Unlock limitless possibilities of vectors in the cloud! Best yet, you can start your journey with Azure Cognitive Search for free!
Already an Azure customer? For more information on how to get started, please visit our documentation as well as additional details on Vector search in Azure Cognitive Search.
Want to learn more about Vector search, Generative AI, plugins & more? Join our subject matter experts for an online discussion July 25th at 9am PT.
Gartner, Magic Quadrant for Insight Engines, Stephen Emmott, Anthony Mullen, David Pidsley, Tim Nelms, 12 December 2022
Gartner is a registered trademark and service mark and Magic Quadrant is a registered trademark of Gartner, Inc. and/or its affiliates in the U.S. and internationally and are used herein with permission. All rights reserved
This graphic was published by Gartner, Inc. as part of a larger research document and should be evaluated in the context of the entire document. The Gartner document is available upon request from Microsoft.
Gartner does not endorse any vendor, product or service depicted in its research publications, and does not advise technology users to select only those vendors with the highest ratings or other designation. Gartner research publications consist of the opinions of Gartner’s research organization and should not be construed as statements of fact. Gartner disclaims all warranties, expressed or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose.
Microsoft
