Event banner
Event details
Introduction: This is Adam Koch and Todd Meinershagen from Paycor. We are in technical roles working to help deliver some prototype AI features into our application suite.
Some topics on our team's mind in preparation for the live session:
- Multi-tenancy 1: Are there any formal recommendations on having multi-tenant Cognitive Search-LLM via Azure AI Studio? (beyond having a full instance per tenant)
- Multi-tenancy 2: We are proofing the idea of having multiple indexes in a single cognitive search resource - one for each of our customers. We would then have a single LLM that would process the prompt along with the results of the particular index search based on the customer. Are there any limits to the number of indexes within one Search resource? Are there any risks or challenges we should be aware of in using this approach?
- In all of the samples, the pattern leverages a Blob Container with documents that are indexed with the index being automatically set up by the Open AI Studio. We are wondering how we would do that from a straight code/automation perspective. What are the commands/sdk that we use to create a new index for a Blob Container that pulls out the correct 5 pieces of metadata?
- Since Azure Cognitive Search can handle databases and json data - Does Search + Azure OpenAI also support pure data from Sql Server or json documents? Or are documents (Word, PDF, etc.) the only items supported in that scenario?
- What is the difference between the regular search and the higher priced semantic search with regards to the RAG pattern?
For multi-tenancy, Azure Cognitive Search has a few common patterns when modeling a multitenant scenario. One index per tenant: Each tenant has its own index within a search service that is shared with other tenants. One service per tenant: Each tenant has its own dedicated Azure Cognitive Search service, offering the highest level of data and workload separation.
Regarding multiple indexes in one resource, Azure Cognitive Search can import, analyze, and index data from multiple data sources into a single consolidated search index. You can use multiple indexers in Azure Cognitive Search to create a single search index from files in Blob storage, with additional file metadata in Table storage. You can also configure an indexer that imports content from Azure Blob Storage and makes it searchable in Azure Cognitive Search.
To create an index for a Blob Container that pulls out the correct 5 pieces of metadata, you can use the deploy-index.json file which defines the structure of the search index. It includes the typical information from blob storage (the content as well as file name, full path, file size, etc.).
Azure Cognitive Search supports pure data from SQL Server or JSON documents. It also supports documents (Word, PDF, etc.).
The difference between regular search and semantic search with regards to the RAG pattern is that semantic search uses natural language processing (NLP) to understand the meaning behind words and phrases. It can identify synonyms and related concepts to expand queries and improve relevance. Regular search uses keyword matching to find relevant documents.