Bring your own Data to Azure OpenAI: Step-by-Step Guide

JamesPetra Azure AI Search (formally, cognitive search) can vectorize data from 5 different sources Blob storage, Cosmos DB for NoSQL, SQL Database, Table Storage, and Data Lake Storage Gen2. See the documentation here Data sources gallery - Azure AI Search | Microsoft Learn.

Currently, the bring your own data feature is only available for unstructured data like uploading a document to blob storage or document stores like MongoDB or Cosmos DB at the end of the day it's document-like data being queried.

1. It's not bad at all if you use it to its full potential (see the available features in the documentation Feature descriptions - Azure AI Search | Microsoft Learn)

2. You can definitely experiment with this more but it will surely return more relevant information but will this be enough for you?

3. That's totally up to your business needs, your data, and the results you are getting from only using the AI search (formally, cognitive search).

If you want to integrate Azure OpenAI with SQL DB.
You can have a more complex solution with an Azure OpenAI layer to handle the query generation and the returned response from the database. (Scenario, instead of making a call to the database the user query or natural language is sent to the Azure OpenAI model endpoint with a prompt to generate a SQL query and the names of the tables and columns to be used in the query then send this to your database and the response from the database is then being sent to Azure OpenAI to evaluate based on the previous prompt and make a new SQL if not satisfied and so on until the response is good then it will be sent to the user after being rephrased using the Azure OpenAI endpoint.)

I would like to point out to you that Azure OpenAI is not the only option available for RAG on Azure. see the documentation Retrieval Augmented Generation using Azure Machine Learning prompt flow (preview) - Azure Machine Learning | Microsoft Learn.