In this article, I will introduce LangChain and explore its capabilities by building a simple question-answering app querying a pdf that is part of Azure Functions Documentation.
Harrison Chase's LangChain is a powerful Python library that simplifies the process of building NLP applications using large language models. Its primary goal is to create intelligent agents that can understand and execute human language instructions. With LangChain, you can connect to a variety of data and computation sources and build applications that perform NLP tasks on domain-specific data sources, private repositories, and more.
As of May 2023, the LangChain GitHub repository has garnered over 42,000 stars and has received contributions from more than 270 developers worldwide.
The langchain library is comprised of different modules:
This includes prompt management, prompt optimization, a generic interface for all LLMs, and common utilities for working with LLMs like Azure OpenAI. It supports a variety of LLMs, including OpenAI, LLama, and GPT4All.
Chains in LangChain involve sequences of calls that can be chained together to perform specific tasks. For instance, you may need to retrieve data from a particular URL, summarize the returned text, and answer questions using the generated summary. Chains can also be simple, such as reading user input, constructing a prompt, and generating a response.
Data Augmented Generation involves specific types of chains that first interact with an external data source to fetch data for use in the generation step. Examples include summarization of long pieces of text and question/answering over specific data sources. LangChain’s Document Loaders and Utils modules facilitate connecting to sources of data and computation. If you have a mix of text files, PDF documents, HTML web pages, etc, you can use the document loaders in Langchain.
Agents involve an LLM making decisions about which actions to take, taking that action, seeing an observation, and repeating that until done.
As we explained before, chains can help chain together a sequence of LLM calls. In some tasks, however, the sequence of calls is often not deterministic and the next step will depend on the user input and the response in the previous steps.
“Agents” can take actions based on inputs along the way instead of a hardcoded deterministic sequence.
Memory refers to persisting state using VectorStores. Vector databases are optimized for doing quick searches in high dimensional spaces. LangChain makes this effortless.
An embedding is a mapping of a discrete, categorical variable to a vector of continuous numbers. In the context of neural networks, embeddings are low-dimensional, learned continuous vector representations of discrete variables. Neural network embeddings are useful because they can reduce the dimensionality of categorical variables and meaningfully represent categories in the transformed space.
Neural network embeddings have 3 primary purposes:
Example of clustering of vector values for sentences
A vector database is a specialized type of database that stores data as high-dimensional vectors. These vectors are mathematical representations of the features or attributes of the data being stored. The number of dimensions in each vector can vary widely, ranging from tens to thousands, depending on the complexity and granularity of the data. In this article, we will explore the concept of vector databases and their applications in various fields.
Let’s build a tool that can read developers documentation – in this case Azure Functions Documentation as PDF.
Then answer arbitrary questions by referencing the documentation text.
We will follow these steps:
One time procedure:
The application:
Steps
OPENAI_DEPLOYMENT_ENDPOINT ="<your openai endpoint>"
OPENAI_API_KEY = "<your openai api key>"
OPENAI_DEPLOYMENT_NAME = "<your gpt35 deployment name>"
OPENAI_DEPLOYMENT_VERSION = "<gpt35 api version>"
OPENAI_MODEL_NAME="<gpt35 model name>"
OPENAI_ADA_EMBEDDING_DEPLOYMENT_NAME = "<your text embedding ada deployment name>"
OPENAI_ADA_EMBEDDING_MODEL_NAME = "<your text embedding ada model name>"
The flow of app_indexer.py is:
Note: As you probably know, LLMs cannot accept long instructions since there is a token limitation, so we will be splitting the document into chunks, see below.
Running this code takes time since we need to read and split the whole document and send the chunks to Ada model to get the embeddings.
from langchain.document_loaders import PyPDFLoader
from langchain.embeddings.openai import OpenAIEmbeddings
from langchain.vectorstores import FAISS
from dotenv import load_dotenv
import openai
import os
#load environment variables
load_dotenv()
OPENAI_API_KEY = os.getenv("OPENAI_API_KEY")
OPENAI_DEPLOYMENT_ENDPOINT = os.getenv("OPENAI_DEPLOYMENT_ENDPOINT")
OPENAI_DEPLOYMENT_NAME = os.getenv("OPENAI_DEPLOYMENT_NAME")
OPENAI_MODEL_NAME = os.getenv("OPENAI_MODEL_NAME")
OPENAI_DEPLOYMENT_VERSION = os.getenv("OPENAI_DEPLOYMENT_VERSION")
OPENAI_ADA_EMBEDDING_DEPLOYMENT_NAME = os.getenv("OPENAI_ADA_EMBEDDING_DEPLOYMENT_NAME")
OPENAI_ADA_EMBEDDING_MODEL_NAME = os.getenv("OPENAI_ADA_EMBEDDING_MODEL_NAME")
#init Azure OpenAI
openai.api_type = "azure"
openai.api_version = OPENAI_DEPLOYMENT_VERSION
openai.api_base = OPENAI_DEPLOYMENT_ENDPOINT
openai.api_key = OPENAI_API_KEY
if __name__ == "__main__":
embeddings=OpenAIEmbeddings(deployment=OPENAI_ADA_EMBEDDING_DEPLOYMENT_NAME,
model=OPENAI_ADA_EMBEDDING_MODEL_NAME,
openai_api_base=OPENAI_DEPLOYMENT_ENDPOINT,
openai_api_type="azure",
chunk_size=1)
dataPath = "./data/documentation/"
fileName = dataPath + "azure-azure-functions.pdf"
#use langchain PDF loader
loader = PyPDFLoader(fileName)
#split the document into chunks
pages = loader.load_and_split()
#Use Langchain to create the embeddings using text-embedding-ada-002
db = FAISS.from_documents(documents=pages, embedding=embeddings)
#save the embeddings into FAISS vector store
db.save_local("./dbs/documentation/faiss_index")
The flow of app_chatbot.py works something like:
Note: What is important to note here is that Langchain does most of the heavy lifting for us and this happens behind the scenes.
from dotenv import load_dotenv
import os
import openai
from langchain.chains import RetrievalQA
from langchain.vectorstores import FAISS
from langchain.chains.question_answering import load_qa_chain
from langchain.chat_models import AzureChatOpenAI
from langchain.embeddings.openai import OpenAIEmbeddings
from langchain.vectorstores import FAISS
from langchain.chains import ConversationalRetrievalChain
from langchain.prompts import PromptTemplate
#load environment variables
load_dotenv()
OPENAI_API_KEY = os.getenv("OPENAI_API_KEY")
OPENAI_DEPLOYMENT_ENDPOINT = os.getenv("OPENAI_DEPLOYMENT_ENDPOINT")
OPENAI_DEPLOYMENT_NAME = os.getenv("OPENAI_DEPLOYMENT_NAME")
OPENAI_MODEL_NAME = os.getenv("OPENAI_MODEL_NAME")
OPENAI_DEPLOYMENT_VERSION = os.getenv("OPENAI_DEPLOYMENT_VERSION")
OPENAI_ADA_EMBEDDING_DEPLOYMENT_NAME = os.getenv("OPENAI_ADA_EMBEDDING_DEPLOYMENT_NAME")
OPENAI_ADA_EMBEDDING_MODEL_NAME = os.getenv("OPENAI_ADA_EMBEDDING_MODEL_NAME")
def ask_question(qa, question):
result = qa({"query": question})
print("Question:", question)
print("Answer:", result["result"])
def ask_question_with_context(qa, question, chat_history):
query = "what is Azure OpenAI Service?"
result = qa({"question": question, "chat_history": chat_history})
print("answer:", result["answer"])
chat_history = [(query, result["answer"])]
return chat_history
if __name__ == "__main__":
# Configure OpenAI API
openai.api_type = "azure"
openai.api_base = os.getenv('OPENAI_API_BASE')
openai.api_key = os.getenv("OPENAI_API_KEY")
openai.api_version = os.getenv('OPENAI_API_VERSION')
llm = AzureChatOpenAI(deployment_name=OPENAI_DEPLOYMENT_NAME,
model_name=OPENAI_MODEL_NAME,
openai_api_base=OPENAI_DEPLOYMENT_ENDPOINT,
openai_api_version=OPENAI_DEPLOYMENT_VERSION,
openai_api_key=OPENAI_API_KEY,
openai_api_type="azure")
embeddings=OpenAIEmbeddings(deployment=OPENAI_ADA_EMBEDDING_DEPLOYMENT_NAME,
model=OPENAI_ADA_EMBEDDING_MODEL_NAME,
openai_api_base=OPENAI_DEPLOYMENT_ENDPOINT,
openai_api_type="azure",
chunk_size=1)
# Initialize gpt-35-turbo and our embedding model
#load the faiss vector store we saved into memory
vectorStore = FAISS.load_local("./dbs/documentation/faiss_index", embeddings)
#use the faiss vector store we saved to search the local document
retriever = vectorStore.as_retriever(search_type="similarity", search_kwargs={"k":2})
QUESTION_PROMPT = PromptTemplate.from_template("""Given the following conversation and a follow up question, rephrase the follow up question to be a standalone question.
Chat History:
{chat_history}
Follow Up Input: {question}
Standalone question:""")
qa = ConversationalRetrievalChain.from_llm(llm=llm,
retriever=retriever,
condense_question_prompt=QUESTION_PROMPT,
return_source_documents=True,
verbose=False)
chat_history = []
while True:
query = input('you: ')
if query == 'q':
break
chat_history = ask_question_with_context(qa, query, chat_history)
Now we can run app_chatbot.py and start asking questions:
you: what are azure functions?
answer: Azure Functions is a cloud service available on-demand that provides all the continually updated infrastructure and resources needed to run your applications. You focus on the code that matters most to you, in the most productive language for you, and Functions handles the rest. Functions provides serverless compute for Azure. You can use Functions to build web APIs, respond to database changes, process IoT streams, manage message queues, and more.
you: can I use events hub as a trigger for an azure function?
answer: Yes, you can use events hub as a trigger for an Azure Function.
`Azure Functions supports trigger and output bindings for Event Hubs. Use the function trigger to respond to an event sent to an event hub event stream. You must have read access to the underlying event hub to set up the trigger. When the function is triggered, the message passed to the function is typed as a string.`<|im_end|>
you: can I deploy azure functions in multi-region?
answer: Yes, you can deploy Azure Functions in multi-region. There are two patterns to consider: Active/Active which is used for HTTP trigger functions and Active/Passive which is used for event-driven, non-HTTP triggered functions. Azure Front Door needs to be used to coordinate requests between both regions when using the active/active pattern for HTTP trigger functions. When using the active/passive pattern, the second region is activated when failover is required and takes over processing. To learn more about multi-region deployments, see the guidance in Highly available multi-region web application.<|im_end|>
A full functioning example of this can be found in my github repo:
The indexer:
The chatbot:
Thanks for reading, hope you enjoyed it.
Denise
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.