Background
Much has been written about the powerful use cases enabled by Azure OpenAI. One such example is the transformation of the modern call center as their operators evaluate the potential of using artificial intelligence and natural language processing to optimize their processes and improve customer interactions. Call centers face challenges of high call volumes, long wait times, and agent turnover. Although many tools already exist in the modern call center, Azure OpenAI deployment leveraging Large Language Model (LLM) and Generative AI has the potential to improve operational efficiency and customer satisfaction by:
- Providing the call center agent with more precise information, more quickly.
- Assisting with more complex queries.
- Providing sentiment analysis of call center interaction to ensure customer retention.
In this article, we describe a deployment where Azure OpenAI serves as a platform to assist human agents. Existing systems such as conversational chatbots, interactive voice response (IVR), and customer relationship management (CRM) all remain and complement the new system.
Modern Call Center Architecture
The fictitious organization Contoso operates a call center to support its business. It decides to use Azure OpenAI to improve its customer service and reduce the cost and time of handling customer calls. It is useful to consider AI solutions in three layers of functionality that group together tasks in an end-to-end context.
Deployment Architecture
These three AI functional layers form the foundation of the Contoso modern call center architecture as illustrated below:
AI Platform: This comprises the baseline LLMs of Azure OpenAI (AOAI) and Azure AI Search.
AI Application: This is a Web app (Azure App Service) which makes API calls to the AI Platform. This application also interfaces to the Call Center CRM to gain insight of customer profile and call history. The application is presented to AI Usage and needs to have intelligent input handling.
AI Usage: This is the human call-center agent who uses the AI application. The agent is also holding a conversation with a caller/end-customer.
In addition to these functional layers, there are other critical components which interface to these three layers.
CRM: The CRM is a repository of customer profile and logs of all customer interaction. While Azure OpenAI itself is stateless, the call center operation may capture state of all customer interaction via the WebApp (AI Application) and store in the CRM based on corporate policy.
Knowledge Base: These are data sources which include industry specifications, corporate service level agreements, configurations, and any knowledge base useful to Contoso call center operations. The data sources may be in PDF, Word, Powerpoint, Excel and a variety of other formats.
PowerBI: Analytics may be run against the CRM, which is constantly updated.
There are two main use cases served by the architecture:
Use Case 1: In-Call Processing
During a person-to-person conversation (1) between the caller (end customer) and the call-center agent, the agent uses the Web app (2) to assist with the customer query such as researching the steps to troubleshoot an issue or recommending a service offering most suitable to the customer based on his/her profile and needs. The Web app interfaces to the CRM (3) to pull customer information to have context of the caller’s profile and any previous call history. The Web app is also the AI application which takes the agent’s query as input and passes that query to Azure OpenAI (4) for a response. Depending on the agent prompt, the AI application may instruct the model to call AI Search (5) to access data sources from the knowledge base (6). The agent is in the loop to validate the accuracy of the response.
This use case reduces the Average Handling Time (AHT), as well as improve First Call Resolution (FCR).
Use Case 2: Post Call
Conversations with call center agents are recorded and stored in a telephony server. After the call, the conversation is converted to text (using either an existing toolset or Azure Speech to Text AI service). The agent sends the file (1) using the Web app (2) to Azure OpenAI for summarization and for sentiment analysis (3). The agent provides a validation of the sentiment analysis and summarization, and this record is stored in the CRM (4). On a regular basis, the call center operations and business team can collect insights by using PowerBI analytics (5).
The agent is able to quickly move onto the next customer conversation, and this post call use case reduces the average After-Call Work (ACW) time because Azure OpenAI automatically summarizes the conversation.
Securing the Deployment Architecture
Call centers are bound by strict standards and regulations designed to protect customer data and privacy. To honor this commitment, they must establish a robust security framework that not only spans all elements of their systems—including AI platforms, applications, and external databases—but also ensures the security of the interfaces, whether they are accessed by human operators or through automated APIs.
Secure AI System Layer Analysis
By analyzing the components and interfaces of the architecture, we can identify risks and threats and can take measures for protection.
|
|
Risks |
Example |
|
Components |
|
|
|
1. Call Center Agent (AI Usage) |
Overprivileged access |
Agent should not have access to records/data not relevant to the current customer/callee |
|
|
Sensitive Data Exposure |
Intentional or unintentional leakage of customer data |
|
2. AI Application (WebApp) |
OWASP Top 10 (e.g. Broken Access Control) |
Bad actor may bypass access control by manipulating the website URL |
|
|
Prompt Injection Attack |
User prompt may include malicious instructions |
|
|
Data authorization |
|
|
3. AI Platform (Azure OpenAI) |
OWASP Top 10 for LLM (e.g. Sensitive Data Disclosure) |
Prompt injection may cause LLM to leak sensitive data |
|
|
Model Denial of Service |
Resource heavy operations may disrupt availability of system |
|
4. AI Search |
Data authorization |
Search on behalf of WebApp or LLM may return data for which they are not authorized |
|
5. Knowledge Base |
Sensitive Data Exposure |
Intentional or unintentional leakage of customer data |
|
6. CRM |
Sensitive Data Exposure |
Data theft of valuable confidential customer data |
|
Interfaces |
|
|
|
7. AI Usage to AI Application |
Adversary in the Middle Attacks |
Call Center Agent are often working from remote locations subject to open Internet based attacks |
|
8. AI Application to AI Platform |
Adversary in the Middle Attacks |
Although this interface is in a private Virtual Network, bad actors seek to find vulnerabilities including unencrypted links |
|
9. AI Platform to Grounding |
Adversary in the Middle Attacks |
Although this interface is in a private Virtual Network, bad actors seek to find vulnerabilities including unencrypted links |
|
10. AI Application to Customer Insight |
Adversary in the Middle Attacks |
Although this interface is in a private Virtual Network, bad actors seek to find vulnerabilities including unencrypted links |
Solutions to Protect the Architecture
Microsoft offers specific recommendations and solutions to protect against the above-described risks.
Elevating the security posture of Generative AI (GenAI) applications within Azure requires a holistic approach that addresses the full spectrum of data handling, compliance, and lifecycle management. A confluence of Azure Landing Zones, the Azure Well-Architected Framework, and robust security practices forms the backbone of a secure and scalable cloud solution.
Best Practices Integration:
- Secure Development and Deployment: Adhere to the principles outlined in the Azure Well-Architected Framework, incorporating secure coding, configuration management, and consistent updates. Use pseudonymization for data privacy enhancement.
- Data in Use Protection: Leverage Azure’s Confidential Computing capabilities and Integrated Identity Management solutions like Entra ID paired with Role-Based Access Control (RBAC) for identity and access management.
- Audit Trails and Logging: Maintain detailed logs and leverage Anomaly Detection to identify and address unusual patterns, using tools such as Microsoft Defender for Cloud and Microsoft Sentinel.
- Privacy Risks Management: Proactively align practices with Cybersecurity Frameworks, like NIST, and manage supply chain risks to ensure comprehensive data protection.
- International Data Standards Adherence: Follow data handling standards for cross-border transfers and conduct Privacy Impact Assessments to mitigate privacy issues.
- Implementation of RBAC: Employ this key aspect of the Microsoft security baseline to limit sensitive data access to authorized users only, supplemented by periodic audits.
Component-Specific Measures:
- AI Usage (Call Center Agent): Authenticate agents with Entra ID and Multifactor Authentication, and authorize access based on JIT and Just-Enough-Access principles.
- AI Application (WebApp): Validate inputs, manage authentication securely with Entra ID, and protect data at rest with encryption.
- Azure OpenAI (AI Platform): Adopt stringent data ingestion policies, utilize Azure Managed Identities, and implement runtime threat detection and compliance controls.
- AI Platform (Azure AI Search:( Ensure role-based access integration and data sensitivity assessment with Purview labeling.
- Knowledge Base (Grounding): Encrypt data at rest and enable Defender for Cloud and Defender for SQL for advanced threat protection.
- CRM: Encrypt confidential data, monitor threats, and use Purview for data labeling and loss prevention.
Inter-Component Safeguards:
- AI Usage to AI Application: Protect data in transit with encryption and guard against various web attacks using Application Gateway with Web Application Firewall and Azure DDoS Protection.
- AI Application to AI Platform: Secure AI application-API connections with Azure API Management (APIM), utilize Azure Private Link for secure communication, and segment network infrastructure for improved security.
Governance and Oversight:
- Logging and Monitoring: Monitor activity comprehensively with Azure Log Analytics.
- Sentinel Integration: Use Microsoft Sentinel for SIEM capabilities, establishing alerts and anomaly detection for unauthorized access and abnormal behavior.
- Defender for Cloud Deployment: Continuously assess Azure OpenAI and connected services' security posture and protect workloads with Microsoft Defender for Cloud.
- Purview for Data Governance: Discover, understand, and manage data with Microsoft Purview, ensuring policy and compliance for sensitive data protection.
Overall, these strategic measures and implementations across components and interfaces guarantee a proactive, inclusive security framework. The adoption of Azure Landing Zone architecture fundamentals offers a firm foundation for safeguarding deployments, while conforming to the Azure security baseline ensures adherence to established security protocols. This fortified structure enhances customer and partner trust while maintaining a rigorous security stance in call center operations.
Other Considerations
While this Call Center OpenAI deployment is limited to enterprise data, the architecture is extensible to include Bing search results. The concepts described here for privacy, integrity, and confidentiality may be extended to include Internet results. It is important to tailor the security considerations to the specific use case of your deployment, referencing the general guidelines provided by Azure to ensure that every aspect of your application’s infrastructure, from development to deployment, adheres to the highest standards of security and privacy.
Conclusion
As we explore the potential of Azure OpenAI to revolutionize call center operations, the implementation of rigorous security measures becomes paramount. The Zero Trust framework's layered security approach and continuous adaptation to emerging threats are crucial in securing Azure OpenAI deployments. By following the best practices outlined herein, advancements in AI will be coupled with robust defenses against cyber threats, ensuring a balance of innovation and security.
References:
- Azure OpenAI Service encryption of data at rest - Azure AI services | Microsoft Learn
- NIST AI RMF Playbook | NIST
- AI shared responsibility model - Microsoft Azure | Microsoft Learn
- Demystifying Azure OpenAI Networking for Secure Chatbot Deployment (microsoft.com)
- OWASP-Top-10-for-LLM-Applications-v1_1.pdf (llmtop10.com)
