How to identify Shadow AI traffic: the signals we've found actually useful
How to identify Shadow AI traffic: the signals we've found actually useful Shadow AI detection sounds simple until you actually try to build it. At first, the obvious approach is: Maintain a list of ChatGPT, Claude, Gemini, Perplexity, etc. -> detect traffic -> classify it as AI. But that breaks down surprisingly quickly. Take GitHub as an example. An employee visiting GitHub is not necessarily using Shadow AI. The same employee using GitHub Copilot potentially is. The same problem exists with Gmail/Gemini, Microsoft 365/Copilot, Slack AI, Notion AI, IDE extensions, browser extensions, and hundreds of SaaS products embedding AI into otherwise legitimate applications. So the real problem is not: "Is this domain an AI domain?" It is: "Does this particular interaction involve a user sending data to an AI system?" We have been working on this problem while building Shadow AI detection at Walled AI, and these are some of the signals I have found most useful. 1. Known AI domains and endpoints This is the easiest layer. Maintain intelligence for known: AI domains inference endpoints API paths model providers AI-specific subdomains SaaS AI endpoints For something like `api.openai.com`, confidence is obviously high. But domain-level detection becomes much weaker when AI functionality is embedded inside a larger application. So domain intelligence should be the first signal, not the final classifier. 2. Request characteristics The request itself can provide surprisingly useful signals. For example: POST requests relatively large text payloads JSON structures containing message- or prompt-like fields conversation history model identifiers generation parameters document or image uploads associated with an inference request None of these individually means "AI." A large POST request could simply be someone submitting a form. But several of these appearing together can increase confidence considerably. 3. Response characteristics The response can provide another independent signal. LLM applications often exhibit patterns such as: streamed responses Server-Sent Events token- or chunk-style incremental output long generated-text responses a request followed by sustained incremental response traffic Again, streaming does not automatically mean AI. Plenty of legitimate applications stream data. But: prompt-like request + AI-looking endpoint + streamed generated response is substantially stronger than any one of those signals alone. 4. Browser context matters This becomes particularly useful for embedded AI. The browser often knows more than the network does. It can potentially tell you: which page the employee is on what UI element initiated the interaction whether a prompt box exists whether text or files are being submitted whether the employee switched into an AI feature whether the user is on a personal or enterprise AI account This helps distinguish: GitHub -> normal development activity from: GitHub -> Copilot interaction without simply classifying all GitHub traffic as AI. This is one reason I do not think Shadow AI discovery can ultimately be solved through network intelligence alone. We have documented some of the multi-layer detection architecture we are building here: https://walled.ai/platform/shadow-ai-detection 5. Identity is another signal Finding an AI service is not necessarily enough. Consider: ChatGPT Enterprise -> approved versus Personal ChatGPT account -> unapproved The destination may effectively be the same while the governance implications are very different. Ideally, AI detection should therefore be correlated with: user device department account or tenant application approved or unapproved status At that point, you are moving beyond traffic classification into actual Shadow AI discovery. 6. Use confidence scoring instead of binary rules This is probably the biggest architectural lesson. Instead of: IF domain == X -> AI I would rather think about: Domain signal + endpoint signal + request signal + response signal + browser signal + identity signal -> AI confidence score High-confidence traffic can be classified deterministically. Low-confidence traffic can remain unclassified. The interesting part is the middle. 7. Use an LLM classifier only for ambiguous traffic One tempting solution is to send everything to an AI classifier. Technically, that is possible. Economically and operationally, it is probably not what you want. If you are inspecting endpoint traffic at enterprise scale, pushing every ambiguous HTTP interaction through an LLM adds: cost latency compute requirements another potential privacy consideration another dependency in the enforcement path We have therefore been experimenting with a different philosophy: Rules and patterns first -> confidence scoring -> LLM judge as fallback As the deterministic detection library improves, fewer requests should require expensive classification. In other words, the system should ideally become **cheaper and faster as its detection intelligence improves**. 8. Detection is only half the problem Once you have classified something as AI traffic, you can start answering more useful governance questions: Which AI applications are employees using? Which are approved versus unapproved? Which departments are adopting Shadow AI? Is sensitive information being submitted? Which models and services are receiving organizational data? What was allowed, redacted, or blocked? This is where we are feeding discovery and enforcement telemetry into the Walled AI Governance Dashboard, so security teams can get a consolidated view of AI usage rather than a large collection of individual network events: https://walled.ai/platform/governance-dashboard <a href="www.walled.ai">Walled AI</a> A practical architecture Known intelligence -> Network signals -> Request + response patterns -> Browser context -> Identity -> Confidence scoring -> LLM fallback rather than: List of AI domains -> Block The hardest part is not identifying ChatGPT. It is identifying AI functionality embedded inside otherwise legitimate applications without creating an unacceptable number of false positives. Curious how others working on Shadow AI are approaching this. What signals have you found reliable for distinguishing actual AI interactions from normal application traffic?
Where Can I post Videos Regarding AI in this community
Hi everyone, I create educational and informational videos around AI (such as Microsoft IQ content) and would like to share them with the community here, similar to how video content is shared in the Microsoft Fabric Community. Is there a designated hub, blog, or discussion category within the Microsoft Tech Community best suited for posting video content like this? For context, here is an example of the title/topic of content I produce: Implementing Microsoft IQ - The Microsoft Unified AI Framework Thanks in advance for any guidance!
How are AI agents changing the way enterprises automate business processes?
AI agents are becoming more capable of understanding requests, making decisions, and interacting with multiple systems through APIs and automation tools. I’m interested in how organizations are approaching this shift from traditional workflow automation toward more intelligent, agent-driven workflows. Some areas I’m exploring: How are teams deciding which processes are suitable for AI agents? What role do validation, monitoring, and human approval play in agent-based automation? How are organizations managing security and permissions when AI agents access business systems? Are AI agents replacing traditional workflows, or working alongside existing automation platforms? Would love to hear how the community is approaching AI agent adoption in real enterprise scenarios. What patterns or lessons have you learned while implementing AI-powered workflows?
AI questions
I'm building a survey and want to include a question about AI tools. When I look in the Glint library though I only see questions that are about Copilot specifically that are benchmarkable. Am I missing something? Questions i'm thinking of including are: I have access to the AI tools, resources and training I need to use AI effectively in my role. AI tools help me work more efficiently and focus on higher-value activities.Design Patterns for Building Reliable AI Agent Workflows in Enterprise Applications
Hi Microsoft community, AI agents are becoming increasingly common in enterprise applications, but designing reliable workflows around them introduces new challenges. A common architecture pattern is separating the AI reasoning layer from the execution layer, where agents can make decisions while business logic, permissions, and validations remain within controlled services. I would like to understand how teams are approaching: - Designing scalable AI agent architectures - Connecting agents with enterprise APIs and services - Managing agent state and workflow execution - Handling failures, retries, and human approval steps - Monitoring and evaluating agent performance in production What architecture patterns and Microsoft technologies are you finding effective when building production-ready AI agent solutions? Looking forward to learning from the community.
Windows AI Foundry (AI Examples) - Access Denied
Hi ! I am trying to play and run the Windows AI Foundry Apps - but keep getting an error message: Access denied Sample code: if (LanguageModel.GetReadyState() == AIFeatureReadyState.NotReady) { var languageModelDeploymentOperation = LanguageModel.EnsureReadyAsync(); await languageModelDeploymentOperation; } LanguageModel _session = await LanguageModel.CreateAsync();How GitHub Copilot Helps with Test-Driven Development (TDD)
Test-Driven Development (TDD) has been a cornerstone of modern software engineering for decades. By writing tests before implementing functionality, developers ensure better design, maintainability, and fewer bugs. But while TDD is powerful, it can sometimes feel slow or cumbersome, especially when setting up repetitive test structures or boilerplate code. This is where GitHub Copilot, the AI-powered coding assistant, becomes a valuable partner. It doesn’t replace the discipline of TDD, but it can accelerate the process and help developers stay in the flow. https://dellenny.com/how-github-copilot-helps-with-test-driven-development-tdd/Boosting Productivity with GitHub Copilot Real-World .NET Coding Examples
Software development in the .NET ecosystem is powerful, but projects can quickly get bogged down with boilerplate code, repetitive tasks, and lengthy setup. This is where GitHub Copilot comes in. Acting as your AI pair programmer inside Visual Studio or VS Code, Copilot suggests context-aware code snippets, tests, and even full method implementations. Let’s dive into real-world .NET coding examples that highlight how Copilot can make you more productive. https://dellenny.com/boosting-productivity-with-github-copilot-real-world-net-coding-examples/ML.NET Sentence Similarity scenario questions
How do you make the ML.NET Sentence Similarity scenario context-aware? I'm using the Model Builder to train, evaluate, and consume the model. I would like to create a context prior to comparing sentences. Is this possible? Is there a way to introduce this in the pipeline? Does a model in ML.NET rely entirely on the training dataset? Is there a pre-trained dataset behind the scenarios? Training a Sentence Similarity model only works when you use validation dataset. Using Cross validation or the Split option always returns this error. "Training time finished without completing a successful trial. Either no trial completed or the metric for all completed trials are NaN or Infinity" Is this a bug?
Let's Learn .NET - AI
Learn about AI, how to get started, and using ChatGPT to improve text completions. Come to learn something new and leave with something that we all built, together, live with experts! Community Links: https://aka.ms/letslearn/dotnet/ai Featuring: Luis Quintanilla (luquinta@microsoft.com) #AI #dotNET #ChatGPT
