Semantic Kernel vs. Lang Chain
Readers have the opportunity to explore two different approaches - using either Semantic Kernel or Lang Chain.
For those interested, here's the link to the Lang Chain version of this tutorial: Teach ChatGPT to Answer Questions: Using Azure AI Search & Azure OpenAI (Lang Chain.ver)
- Semantic Kernel
NOTE:
NOTE:
In this tutorial we will use the Basic tier to explore semantic ranker with Azure AI Search. You can expect a cost of approximately $2.50 per 100 program runs with this tier.
If you plan to use the free tier, please note that the code demonstrated in this tutorial may differ from what you'll need.
(Azure AI Search is priced even when you're not using it. If you're just going through the tutorial for practice, I recommend deleting the Azure AI Search you created when you're done all tutorial.)
How to keep sensitive data private?
To ensure the privacy of sensitive data, Azure Cognitive Search provides a Personally Identifiable Information (PII) detection skill. This cognitive skill is specifically designed to identify and protect PII in your data. To learn more about Azure Cognitive Search's PII detection skill, read the following article.
Personally Identifiable Information (PII) Detection cognitive skill
- To enable this feature, select Extract Personally identifiable information.
You can change the fields to suit your data. I have attached a document with a description of each field in the index. (Depending on your settings for the index fields, the code you implement may differ from the tutorial.)
In series2, we will implement the feature to answer questions based on PDFs using Azure AI Search and Azure OpenAI, and implement this feature in code.
Intent of the Code Design
The primary goal of the code design in this tutorial is to construct the code in a way that is easy to understand, especially for first-time viewers. Each segment of the code is encapsulated as a separate function. This modularization ensures that the main function acts as a clear, coherent guide through the logic of the system.
Ex. Part of the main function. (Semantic Kernel.ver)
async def main():
…
search_results = await search_documents(QUESTION)
documents = await filter_documents(search_results)
…
kernel = await create_kernel(sk)
await create_embeddings(kernel)
await create_vector_store(kernel, embeddings)
await store_documents(kernel, documents)
related_page = await search_with_vector_store(memory, QUESTION)
await add_chat_service(kernel)
answer = await answer_with_sk(kernel, QUESTION, related_page)
…
Overview of the code
Part 1: Retrieving and Scoring Documents
We'll use Azure AI Search to retrieve documents related to our question, score them for relevance to our question, and extract documents with a certain score or higher.
Part 2: Document Embedding and Vector Database Storage
We'll embed the documents we extracted in part 1. We will then store these embedded documents in a vector database, organizing them into pages (chunks of 5,000 characters each).
Part 3: Extracting Relevant Content and Implementing a Function to Answer Questions
We will extract the most relevant page from the vector database based on the question.
Then we will implement a function to generate answers from the extracted content.
"outputFieldMappings": [
{
"sourceFieldName": "/document/content/pages/*/keyphrases/*",
"targetFieldName": "keyphrases"
},
{
"sourceFieldName": "/document/content/pages/*",
"targetFieldName": "pages"
}
]
mkdir azure-proj
cd azure-proj
mkdir gpt-proj
cd gpt-proj1
Python -m venv .venv
.venv\Scripts\activate.bat
pip install semantic-kernel==0.9.1b1
# Library imports
from collections import OrderedDict
import requests
# Semantic Kernel library imports
import semantic_kernel as sk
import semantic_kernel.connectors.ai.open_ai as sk_oai
from semantic_kernel.connectors.ai.open_ai import AzureChatCompletion, AzureTextEmbedding
from semantic_kernel.memory.semantic_text_memory import SemanticTextMemory
from semantic_kernel.core_plugins.text_memory_plugin import TextMemoryPlugin
from semantic_kernel.prompt_template.input_variable import InputVariable
from semantic_kernel.functions.kernel_arguments import KernelArguments
# Azure AI Search service settings
SEARCH_SERVICE_NAME = 'your-search-service-name' # 'teachchatgpt-search'
SEARCH_SERVICE_ENDPOINT = f'https://{SEARCH_SERVICE_NAME.lower()}.search.windows.net/'
SEARCH_SERVICE_KEY = 'your-search-service-key'
SEARCH_SERVICE_API_VERSION = 'your-API-version' # '2023-10-01-Preview'
# Azure AI Search service index settings
SEARCH_SERVICE_INDEX_NAME1 = 'your-search-service-index-name' # 'teachchatgpt-index'
# Azure AI Search service semantic configuration settings
SEARCH_SERVICE_SEMANTIC_CONFIG_NAME = 'your-semantic-configuration-name' # 'teachchatgpt-config'
# Azure OpenAI settings
AZURE_OPENAI_NAME = 'your-openai-name' # 'teachchatgpt-azureopenai'
AZURE_OPENAI_ENDPOINT = f'https://{AZURE_OPENAI_NAME.lower()}.openai.azure.com/'
AZURE_OPENAI_KEY = 'your-openai-key'
AZURE_OPENAI_API_VERSION = 'your-API-version' # '2024-02-15-preview'
# Configuration imports
from config import (
SEARCH_SERVICE_ENDPOINT,
SEARCH_SERVICE_KEY,
SEARCH_SERVICE_API_VERSION,
SEARCH_SERVICE_INDEX_NAME1,
SEARCH_SERVICE_SEMANTIC_CONFIG_NAME,
AZURE_OPENAI_ENDPOINT,
AZURE_OPENAI_KEY,
AZURE_OPENAI_API_VERSION,
)
# Azure AI Search service header settings
HEADERS = {
'Content-Type': 'application/json',
'api-key': SEARCH_SERVICE_KEY
}
async def search_documents(question):
"""Search documents using Azure AI Search."""
# Construct the Azure AI Search service access URL.
url = (SEARCH_SERVICE_ENDPOINT + 'indexes/' +
SEARCH_SERVICE_INDEX_NAME1 + '/docs')
# Create a parameter dictionary.
params = {
'api-version': SEARCH_SERVICE_API_VERSION,
'search': question,
'select': '*',
# '$top': 5, Extract the top 5 documents from your storage.
'$top': 5,
'queryLanguage': 'en-us',
'queryType': 'semantic',
'semanticConfiguration': SEARCH_SERVICE_SEMANTIC_CONFIG_NAME,
'$count': 'true',
'speller': 'lexicon',
'answers': 'extractive|count-3',
'captions': 'extractive|highlight-false'
}
# Make a GET request to the Azure AI Search service and store the response in a variable.
resp = requests.get(url, headers=HEADERS, params=params)
# Return the JSON response containing the search results.
search_results = resp.json()
return search_results
async def filter_documents(search_results):
"""Filter documents with a reranker score above a certain threshold."""
documents = OrderedDict()
for result in search_results['value']:
# The '@search.rerankerScore' range is 0 to 4.00, where a higher score indicates a stronger semantic match.
if result['@search.rerankerScore'] > 0.8:
documents[result['metadata_storage_path']] = {
'chunks': result['pages'][:10],
'captions': result['@search.captions'][:10],
'score': result['@search.rerankerScore'],
'file_name': result['metadata_storage_name']
}
return documents
async def main():
QUESTION = 'Tell me about effective prompting strategies'
# Search for documents with Azure AI Search.
search_results = await search_documents(QUESTION)
documents = await filter_documents(search_results)
print('Total Documents Found: {}, Top Documents: {}'.format(
search_results['@odata.count'], len(search_results['value'])))
# Execute the main function.
if __name__ == "__main__":
import asyncio
asyncio.run(main())
1. We will create functions related to Azure OpenAI and Semantic Kernel and run them from
the main function.
- Add the following functions above the main function.
async def search_with_vector_store(memory, question):
"""Search for documents related to your question from the vector store."""
related_page = await memory.search('TeachGPTtoPDF', question)
return related_page
async def add_chat_service(kernel):
"""Add a chat service."""
azure_chat_service = AzureChatCompletion(
service_id = 'chat_service',
deployment_name = 'gpt-35-turbo', # Azure OpenAI deployment name
endpoint = AZURE_OPENAI_ENDPOINT,
api_key = AZURE_OPENAI_KEY)
return kernel.add_service(azure_chat_service)
async def answer_with_sk(kernel, question, related_page):
"""Answer question with related_page using Semantic Kernel."""
prompt = """
Provide a detailed answer to the <question> using the information from the <related_page>.
<question>
{{$question}}
</question>
<related_page>
{{$related_page}}
</related_page>
Answer:
"""
execution_settings = sk_oai.OpenAIChatPromptExecutionSettings(
service_id = 'chat_service',
ai_model_id = 'gpt-35-turbo',
max_tokens = 500,
temperature = 0.0
)
prompt_template_config = sk.PromptTemplateConfig(
template = prompt,
name = 'chat_prompt_template',
template_format = 'semantic-kernel',
input_variables = [
InputVariable(name = 'question', description = 'The question that requires a detailed answer.', is_required=True),
InputVariable(name ="related_page", description = 'The text of the page that contains information relevant to the question.', is_required=True),
],
execution_settings = execution_settings)
chat_function = kernel.create_function_from_prompt(prompt_template_config = prompt_template_config,
plugin_name = 'chat_plugin',
function_name = 'chat_function')
arguments = KernelArguments(question = question, related_page = related_page[0].text)
answer = await kernel.invoke(chat_function, arguments)
return answer
2. Add the code below to your main function.
async def main():
QUESTION = 'Tell me about effective prompting strategies'
# Search for documents with Azure AI Search.
search_results = await search_documents(QUESTION)
documents = await filter_documents(search_results)
print('Total Documents Found: {}, Top Documents: {}'.format(
search_results['@odata.count'], len(search_results['value'])))
# Answer your question using Semantic Kernel.
kernel = await create_kernel(sk)
embeddings = await create_embeddings(kernel)
memory = await create_vector_store(kernel, embeddings)
await store_documents(memory, documents)
related_page = await search_with_vector_store(memory, QUESTION)
await add_chat_service(kernel)
answer = await answer_with_sk(kernel, QUESTION, related_page)
print('Question: ', QUESTION)
print('Answer: ', answer)
print('Reference: ', related_page[0].id)
3. Now let's run it and see if it answers your question.
- The result of executing the code.
```
Total Documents Found: 5, Top Documents: 3
Question: Tell me about effective prompting strategies
Answer: Effective prompting strategies are techniques used to encourage individuals to engage in desired behaviors or complete tasks. These strategies can be particularly useful for individuals with disabilities or those who struggle with executive functioning skills. Some effective prompting strategies include:
Overall, effective prompting strategies should be tailored to the individual's needs and abilities, and should be used consistently to help them develop independence and achieve success.
Reference: Prompting GPT-3 To Be Reliable.pdf_1
```
# Azure AI Search service settings
SEARCH_SERVICE_NAME = 'your-search-service-name' # 'teachchatgpt-search'
SEARCH_SERVICE_ENDPOINT = f'https://{SEARCH_SERVICE_NAME.lower()}.search.windows.net/'
SEARCH_SERVICE_KEY = 'your-search-service-key'
SEARCH_SERVICE_API_VERSION = 'your-API-version' # '2023-10-01-Preview'
# Azure AI Search service index settings
SEARCH_SERVICE_INDEX_NAME1 = 'your-search-service-index-name' # 'teachchatgpt-index'
# Azure AI Search service semantic configuration settings
SEARCH_SERVICE_SEMANTIC_CONFIG_NAME = 'your-semantic-configuration-name' # 'teachchatgpt-config'
# Azure OpenAI settings
AZURE_OPENAI_NAME = 'your-openai-name' # 'teachchatgpt-azureopenai'
AZURE_OPENAI_ENDPOINT = f'https://{AZURE_OPENAI_NAME.lower()}.openai.azure.com/'
AZURE_OPENAI_KEY = 'your-openai-key'
AZURE_OPENAI_API_VERSION = 'your-API-version' # '2024-02-15-preview'
# Library imports
from collections import OrderedDict
import requests
# Semantic Kernel library imports
import semantic_kernel as sk
import semantic_kernel.connectors.ai.open_ai as sk_oai
from semantic_kernel.connectors.ai.open_ai import AzureChatCompletion, AzureTextEmbedding
from semantic_kernel.memory.semantic_text_memory import SemanticTextMemory
from semantic_kernel.core_plugins.text_memory_plugin import TextMemoryPlugin
from semantic_kernel.prompt_template.input_variable import InputVariable
from semantic_kernel.functions.kernel_arguments import KernelArguments
# Configuration imports
from config import (
SEARCH_SERVICE_ENDPOINT,
SEARCH_SERVICE_KEY,
SEARCH_SERVICE_API_VERSION,
SEARCH_SERVICE_INDEX_NAME1,
SEARCH_SERVICE_SEMANTIC_CONFIG_NAME,
AZURE_OPENAI_ENDPOINT,
AZURE_OPENAI_KEY,
AZURE_OPENAI_API_VERSION,
)
# Azure AI Search service header settings
HEADERS = {
'Content-Type': 'application/json',
'api-key': SEARCH_SERVICE_KEY
}
async def search_documents(question):
"""Search documents using Azure AI Search."""
# Construct the Azure AI Search service access URL.
url = (SEARCH_SERVICE_ENDPOINT + 'indexes/' +
SEARCH_SERVICE_INDEX_NAME1 + '/docs')
# Create a parameter dictionary.
params = {
'api-version': SEARCH_SERVICE_API_VERSION,
'search': question,
'select': '*',
# '$top': 5, Extract the top 5 documents from your storage.
'$top': 5,
'queryLanguage': 'en-us',
'queryType': 'semantic',
'semanticConfiguration': SEARCH_SERVICE_SEMANTIC_CONFIG_NAME,
'$count': 'true',
'speller': 'lexicon',
'answers': 'extractive|count-3',
'captions': 'extractive|highlight-false'
}
# Make a GET request to the Azure AI Search service and store the response in a variable.
resp = requests.get(url, headers=HEADERS, params=params)
# Return the JSON response containing the search results.
search_results = resp.json()
return search_results
async def filter_documents(search_results):
"""Filter documents with a reranker score above a certain threshold."""
documents = OrderedDict()
for result in search_results['value']:
# The '@search.rerankerScore' range is 0 to 4.00, where a higher score indicates a stronger semantic match.
if result['@search.rerankerScore'] > 0.8:
documents[result['metadata_storage_path']] = {
'chunks': result['pages'][:10],
'captions': result['@search.captions'][:10],
'score': result['@search.rerankerScore'],
'file_name': result['metadata_storage_name']
}
return documents
async def create_kernel(sk):
"""Create a Semantic Kernel."""
return sk.Kernel()
async def create_embeddings(kernel):
"""Create an embedding model."""
embeddings = AzureTextEmbedding(
deployment_name = "text-embedding-ada-002",
endpoint = AZURE_OPENAI_ENDPOINT,
api_key = AZURE_OPENAI_KEY
)
kernel.add_service(embeddings)
return embeddings
async def create_vector_store(kernel, embeddings):
"""Create a vector store."""
memory = SemanticTextMemory(storage = sk.memory.VolatileMemoryStore(), embeddings_generator = embeddings)
kernel.import_plugin_from_object(TextMemoryPlugin(memory), "TextMemoryPlugin")
return memory
async def store_documents(memory, file_content):
"""Store documents in the vector store."""
for key, value in file_content.items():
page_number = 1
for page in value['chunks']:
page_id = f"{value['file_name']}_{page_number}"
await memory.save_information(
collection = 'TeachGPTtoPDF',
id = page_id,
text = page
)
page_number += 1
async def search_with_vector_store(memory, question):
"""Search for documents related to your question from the vector store."""
related_page = await memory.search('TeachGPTtoPDF', question)
return related_page
async def add_chat_service(kernel):
"""Add a chat service."""
azure_chat_service = AzureChatCompletion(
service_id = 'chat_service',
deployment_name = 'gpt-35-turbo', # Azure OpenAI deployment name
endpoint = AZURE_OPENAI_ENDPOINT,
api_key = AZURE_OPENAI_KEY)
return kernel.add_service(azure_chat_service)
async def answer_with_sk(kernel, question, related_page):
"""Answer question with related_page using Semantic Kernel."""
prompt = """
Provide a detailed answer to the <question> using the information from the <related_page>.
<question>
{{$question}}
</question>
<related_page>
{{$related_page}}
</related_page>
Answer:
"""
execution_settings = sk_oai.OpenAIChatPromptExecutionSettings(
service_id = 'chat_service',
ai_model_id = 'gpt-35-turbo',
max_tokens = 500,
temperature = 0.0
)
prompt_template_config = sk.PromptTemplateConfig(
template = prompt,
name = 'chat_prompt_template',
template_format = 'semantic-kernel',
input_variables = [
InputVariable(name = 'question', description = 'The question that requires a detailed answer.', is_required=True),
InputVariable(name ="related_page", description = 'The text of the page that contains information relevant to the question.', is_required=True),
],
execution_settings = execution_settings)
chat_function = kernel.create_function_from_prompt(prompt_template_config = prompt_template_config,
plugin_name = 'chat_plugin',
function_name = 'chat_function')
arguments = KernelArguments(question = question, related_page = related_page[0].text)
answer = await kernel.invoke(chat_function, arguments)
return answer
async def main():
QUESTION = 'Tell me about effective prompting strategies'
# Search for documents with Azure AI Search.
search_results = await search_documents(QUESTION)
documents = await filter_documents(search_results)
print('Total Documents Found: {}, Top Documents: {}'.format(
search_results['@odata.count'], len(search_results['value'])))
# Answer your question using Semantic Kernel.
kernel = await create_kernel(sk)
embeddings = await create_embeddings(kernel)
memory = await create_vector_store(kernel, embeddings)
await store_documents(memory, documents)
related_page = await search_with_vector_store(memory, QUESTION)
await add_chat_service(kernel)
answer = await answer_with_sk(kernel, QUESTION, related_page)
print('Question: ', QUESTION)
print('Answer: ', answer)
print('Reference: ', related_page[0].id)
# Execute the main function.
if __name__ == "__main__":
import asyncio
asyncio.run(main())
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.