Event banner
Event details
116 Comments
- How do you manage the cost of using Azure Search as vector store for RAG use cases? I had an experience trying out an azure search example that involves some 90k arvix and covid journals and it costs £500 just to load them. Would there is a way to estimate cost better and how should one approach it?
- fsunavala-msft
Microsoft
Cost ultimately comes down to your SKU and the number of Search Units (SUs) which many factors play a role in the decision making such as document count, availability and reliability, index schema design, count of vector fields, vector field dimensions, etc. The most accurate way to estimate cost is to perform a proof-of-concept index with the desired schema you want and load a sample of documents that will reflect your production workload. Then, you can extrapolate your sample document count and index size with your production document count and get your production index size. You can visit our documentation on service limits by SKU and decide which SKU and Search Unit Count (partitions/replicas) fit your needs best. https://learn.microsoft.com/azure/search/search-limits-quotas-capacity Once you have an estimated SKU and number of search units, you can visit our pricing calculator here, select your region, and see a cost estimate of your Cognitive Search service: https://azure.microsoft.com/pricing/details/search/
- Do we have any best practices or case studies for beginners to learn Azure OpenAI & Cognitive Search?
- fsunavala-msft
For Azure Cognitive Search, a good hands-on holistic overview of the whole product is here Knowledge Mining Accelerator, and specifically for Vector Search, Quickstart vector search - Azure Cognitive Search | Microsoft Learn.
For Azure OpenAI, a good start is Quickstart - Deploy a model and generate text using Azure OpenAI Service - Azure OpenAI | Microsoft Learn
- Thanks for joining us for this Azure Cognitive Search AMA! The event is now over, but we'll be posting a summary of the questions and answers here soon.
- liamca-msftThanks everyone for the amazing questions! It was great speaking with you today!
- Plug In environment roadmap - I've seen technical specs. how 3rd parties will build. Specific to parsing/reading/integrating docs. What is timing for integration/rollout? Reference to tech. articles showing partner examples?
Plugins support in Azure OpenAI is currently in a limited private preview. For now, I'd recommend checking out these resources on additional capabilities that you can leverage:
- Function calling is now available in Azure OpenAI Service - Microsoft Community Hub- Introducing Azure OpenAI Service On Your Data in Public Preview - Microsoft Community Hub
In particular, function calling allows you to do all the same things you could do with plugins. The main difference is that with functions, you're in control of the orchestration and need to call out to the plugins on the client side.
- How can I use this if OpenAI is not currently supported in my region?
I'd recommend creating an Azure OpenAI service in a region close to your search service. There will be some added network latency but it should be manageable. You can check out the regional availability of different models here: Azure OpenAI Service models - Azure OpenAI | Microsoft Learn
We're also working hard to add new Azure OpenAI regions!
- What is the approach recommended to store a chatbot conversation history for future context?
- liamca-msft
Hi Gerardo, I would highly recommend you check out this blog post from Semantic Kernel which discusses memory management, There is also this video on the topic.
OpenAI studio (Chat Playground) is able to connect to data sources only if they are in public network, which not the ideal scenario in corporate environments. Any plans to add private endpoint connectivity to studio?
- gia_mondragonSupporting private endpoints is part of that feature roadmap and ETA is the coming quarter (Q4 '23) as per the team supporting it.
- How will vector search change or improve upon the pattern in the blog post here https://techcommunity.microsoft.com/t5/ai-applied-ai-blog/revolutionize-your-enterprise-data-with-chatgpt-next-gen-apps-w/ba-p/3762087 and sample code here https://github.com/Azure-Samples/azure-search-openai-demo?
- liamca-msft
Jason, you are absolutely correct that vector search will have a very positive impact on this from the ability to get the most relevant results, and we have already updated the sample to leverage vectors (check out the "embedding" field).
In fact, we believe that it is best to leverage Hybrid Search along with the reranking layer of Semantic Search to get the best results. I talk a bit about Hybrid Search in my blog post.
Could you EricStarker provide insights into the scalability and performance aspects of the Azure OpenAI interface? How does it handle large-scale AI workloads and ensure efficient resource utilization?
The biggest thing to keep in mind when it comes to the scalability of Azure OpenAI service are the quota and limits. I'd start by taking a look at this page that talks about the limits. It's also important to learn how to manage the quota for your service.
In terms of performance, the best thing I can recommend is trying out your prompts in Azure OpenAI to understand the performance. One big factor for performance is the length of your prompts, particularly the number of output tokens.
- Sounds great. I will work on that and learn for myself how it works.
- Hello! I am not a product expert, just a community manager here, so I'll leave that for the Microsoft folks here to answer when they can.
- Oh, Sorry about that misunderstanding. I thought you was an expert.
