LangChain

• Model I/O: this is where you can bring in your LLM and format its inputs and outputs
• Retrieval: In RAG applications, this component specifically helps you load your data, connect with vector databases and transform your documents to meet the needs of your application.
• Other Higher level Components
  • Tools: allows you to create Intergrations with external services and applications
  • Agents: these are responsible as a guide on what step to take next.
  • Chains: these are a sequence of calls linking various components to create LLM apps

1. Install the necessary libraries: pip install langchain openai
2. Login to Azure CLI using az login --use-device-code and authenticate your connection
3. Add you keys and endpoint from .env to your notebook, then set the environment variables for your API key and type for authentication.

import os
from azure.identity import DefaultAzureCredential

# Get the Azure Credential
credential = DefaultAzureCredential()

# Set the API type to `azure_ad`
os.environ["OPENAI_API_TYPE"] = "azure_ad"
# Set the API_KEY to the token from the Azure credential
os.environ["OPENAI_API_KEY"] = credential.get_token("https://cognitiveservices.azure.com/.default").token

4. Create your model class and configure it to interact with Azure OpenAI

# Import the necessary modules
from langchain_core.messages import HumanMessage
from langchain_openai import AzureChatOpenAI

model = AzureChatOpenAI(
    openai_api_version=AZURE_OPENAI_API_VERSION,  
    azure_deployment=AZURE_OPENAI_CHAT_DEPLOYMENT_NAME  
)

5. Use ChatPromptTemplate to curate your prompt

# Import the necessary modules
from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder

# Create a ChatPromptTemplate object with messages
prompt = ChatPromptTemplate.from_messages(
    [
        (
            "system",
            "You are a helpful assistant that translates tasks into Kiswahili. Follow these guidelines:\n"
            "The translation must be accurate and culturally appropriate.\n"
            "Use the {{$time_of_day}} to determine the appropriate greeting to use during translation.\n"
            "Be creative and accurate to communicate effectively.\n"
            "Incorporate the {{$style}} suggestion, if provided, to determine the tone for the translation.\n"
            "After translating, add an English translation of the task in the specified language.\n"
            "For example, if the question is 'what is the WiFi password', your response should be:\n"
            "'Habari ya mchana! Tafadhali nipe nenosiri la WiFi.' (Translation: Good afternoon! Please provide me with the WiFi password.)"
        ),
        ("human", "{question}"),
    ]
)

6. Chain your model and prompt together to get a response

# Chain the prompt and the model together
chain = prompt | model

# Invoke the chain with the input parameters
response = chain.invoke(
    {
        "question":"what is the WiFi password",
        "time_of_day":"afternoon", 
        "style":"professional"
    }
)

# Print the response
response

7. The output will be the requested translation.

 

PromptFlow

1. First, you install the promptflow extension on Visual Studio Code

2. Next, ensure you install the necessary dependencies and libraries your will need for the project.

3. In our case we will be build a chat flow with template. Click on somewhere and create a chat flow for the application

4. Once the flow is ready, we can open flow.dag and click on the visual editor to see how our application is structured.

5. We will need to connect to our LLM, you can do this by creating a new connection. Update your Azure OpenAI endpoint and your connection name. Click create connection then you will have your connection ready.

6. Update the connection and run the flow to test your application.

7. Update the chat.jinja2 file to customize the prompt template.

8. Edit the yaml file to add more functionality to your flow, in our case for the Tutor, we will add more inputs.

In Summary:

If you are getting started and building LLM applications at scale, you might need to use an AI Orchestrator to ease your development process. Choosing between LangChain, Semantic Kernel and PromptFlow, depends on your project scope, scale, flexibility level and your preferred programming language. You can get started with the tools using the following resources:

• GitHub Repository: https://github.com/BethanyJep/Swahili-Tutor
• Semantic Kernel: microsoft/semantic-kernel: Integrate cutting-edge LLM technology quickly and easily into your apps (github.com)
• Semantic Kernel documentation: Create AI agents with Semantic Kernel | Microsoft Learn
• Promptflow documentation: Prompt flow — Prompt flow documentation (microsoft.github.io)
• LangChain: Introduction | 🦜️:link: LangChain