How to identify Shadow AI traffic: the signals we've found actually useful

How to identify Shadow AI traffic: the signals we've found actually useful

Shadow AI detection sounds simple until you actually try to build it.

At first, the obvious approach is:

Maintain a list of ChatGPT, Claude, Gemini, Perplexity, etc. -> detect traffic -> classify it as AI.

But that breaks down surprisingly quickly.

 

Take GitHub as an example. An employee visiting GitHub is not necessarily using Shadow AI. The same employee using GitHub Copilot potentially is.

 

The same problem exists with Gmail/Gemini, Microsoft 365/Copilot, Slack AI, Notion AI, IDE extensions, browser extensions, and hundreds of SaaS products embedding AI into otherwise legitimate applications.

 

So the real problem is not:

 

"Is this domain an AI domain?"

 

It is:

 

"Does this particular interaction involve a user sending data to an AI system?"

 

We have been working on this problem while building Shadow AI detection at Walled AI, and these are some of the signals I have found most useful.

 

1. Known AI domains and endpoints

This is the easiest layer.

Maintain intelligence for known:

• AI domains
• inference endpoints
• API paths
• model providers
• AI-specific subdomains
• SaaS AI endpoints

For something like `api.openai.com`, confidence is obviously high.

But domain-level detection becomes much weaker when AI functionality is embedded inside a larger application.

So domain intelligence should be the first signal, not the final classifier.

 

2. Request characteristics

The request itself can provide surprisingly useful signals.

For example:

• POST requests
• relatively large text payloads
• JSON structures containing message- or prompt-like fields
• conversation history
• model identifiers
• generation parameters
• document or image uploads associated with an inference request

None of these individually means "AI."

A large POST request could simply be someone submitting a form.

But several of these appearing together can increase confidence considerably.

 

3. Response characteristics

The response can provide another independent signal.

LLM applications often exhibit patterns such as:

• streamed responses
• Server-Sent Events
• token- or chunk-style incremental output
• long generated-text responses
• a request followed by sustained incremental response traffic

Again, streaming does not automatically mean AI.

Plenty of legitimate applications stream data.

But:

prompt-like request + AI-looking endpoint + streamed generated response

is substantially stronger than any one of those signals alone.

 

4. Browser context matters

This becomes particularly useful for embedded AI.

The browser often knows more than the network does.

It can potentially tell you:

• which page the employee is on
• what UI element initiated the interaction
• whether a prompt box exists
• whether text or files are being submitted
• whether the employee switched into an AI feature
• whether the user is on a personal or enterprise AI account

This helps distinguish:

GitHub -> normal development activity

from:

GitHub -> Copilot interaction

without simply classifying all GitHub traffic as AI.

This is one reason I do not think Shadow AI discovery can ultimately be solved through network intelligence alone.

We have documented some of the multi-layer detection architecture we are building here:

 

https://walled.ai/platform/shadow-ai-detection

5. Identity is another signal

 

Finding an AI service is not necessarily enough.

Consider:

ChatGPT Enterprise -> approved

versus

Personal ChatGPT account -> unapproved

 

The destination may effectively be the same while the governance implications are very different.

Ideally, AI detection should therefore be correlated with:

• user
• device
• department
• account or tenant
• application
• approved or unapproved status

At that point, you are moving beyond traffic classification into actual Shadow AI discovery.

 

6. Use confidence scoring instead of binary rules

 

This is probably the biggest architectural lesson.

Instead of:

IF domain == X -> AI

I would rather think about:

Domain signal + endpoint signal + request signal + response signal + browser signal + identity signal -> AI confidence score

High-confidence traffic can be classified deterministically.

Low-confidence traffic can remain unclassified.

The interesting part is the middle.

7. Use an LLM classifier only for ambiguous traffic

 

One tempting solution is to send everything to an AI classifier.

Technically, that is possible.

Economically and operationally, it is probably not what you want.

If you are inspecting endpoint traffic at enterprise scale, pushing every ambiguous HTTP interaction through an LLM adds:

• cost
• latency
• compute requirements
• another potential privacy consideration
• another dependency in the enforcement path

We have therefore been experimenting with a different philosophy:

Rules and patterns first -> confidence scoring -> LLM judge as fallback

As the deterministic detection library improves, fewer requests should require expensive classification.

In other words, the system should ideally become **cheaper and faster as its detection intelligence improves**.

 

8. Detection is only half the problem

 

Once you have classified something as AI traffic, you can start answering more useful governance questions:

Which AI applications are employees using?

Which are approved versus unapproved?

Which departments are adopting Shadow AI?

Is sensitive information being submitted?

Which models and services are receiving organizational data?

What was allowed, redacted, or blocked?

 

This is where we are feeding discovery and enforcement telemetry into the Walled AI Governance Dashboard, so security teams can get a consolidated view of AI usage rather than a large collection of individual network events:

 

https://walled.ai/platform/governance-dashboard

<a href="www.walled.ai">Walled AI</a>

A practical architecture

Known intelligence -> Network signals -> Request + response patterns -> Browser context -> Identity -> Confidence scoring -> LLM fallback

rather than:

List of AI domains -> Block

The hardest part is not identifying ChatGPT.

It is identifying AI functionality embedded inside otherwise legitimate applications without creating an unacceptable number of false positives.

Curious how others working on Shadow AI are approaching this.

What signals have you found reliable for distinguishing actual AI interactions from normal application traffic?