Rethinking Azure OpenAI Token Economics: Processing Nasdaq Level 2 Market Data Before AI

Generative AI has fundamentally changed how organizations approach application development. Tasks that previously required specialized software or complex algorithms can now be prototyped with a prompt and an API call.

That simplicity is valuable, but it can also introduce an architectural anti-pattern: using a language model for repetitive, deterministic data processing that conventional software can perform more efficiently. A proof of concept may work beautifully and still become economically difficult to operate at scale.

I recently explored this challenge with a financial services customer that wanted to analyze Nasdaq Level 2 order-book data across a broad universe of equities. The objective was to identify notable buying and selling pressure, then generate readable market intelligence reports for financial analysts.

As an active trader, I could relate to the customer's need for fast, useful market insights. The interesting problem was not whether Azure OpenAI could create the report. It could. The question was how much raw market data the application needed to send to the model before it could write that report.

The business challenge: Finding signals in fast-changing order books

Nasdaq Level 2 market data provide visibility into displayed bid and ask prices and quantities at multiple levels of an order book. Examining this data can help analysts identify:

• Bid-versus-ask liquidity imbalances.
• Large displayed orders and sudden changes in quoted depth.
• Concentrations of liquidity near particular price levels.
• Changes in short-term buying and selling pressure.
• Patterns that merit further investigation alongside trades, volume, and other market context.

The customer's observations were captured in CSV files with thousands of rows. The following five rows are a synthetic MSFT example, not actual Nasdaq quotes:

timestamp_et,symbol,side,level,price,size
10:32:15.123456,MSFT,BID,1,429.99,7200
10:32:15.123456,MSFT,BID,2,429.98,5600
10:32:15.123456,MSFT,BID,3,429.97,4100
10:32:15.123456,MSFT,ASK,1,430.01,2600
10:32:15.123456,MSFT,ASK,2,430.02,1800

In a live environment, this evidence is updated rapidly. Even when timestamps have microsecond precision, the actual event rate depends on the feed and the security. The problem becomes much larger when the same workflow runs repeatedly across many tickers.

The initial architecture: The LLM as a data-processing engine

The customer exposed CSV data from Azure Blob Storage through an MCP server. An application or a Foundry agent in an agent-based implementation could invoke those tools and supply the returned records to Azure OpenAI for interpretation and report generation.

The initial workflow was straightforward:

1. Collect Level 2 market observations and write CSV files to Azure Blob Storage.
2. Expose raw file records through the MCP server.
3. Have the application or agent retrieve those records and provide them to Azure OpenAI.
4. Ask the model to find relevant pressure patterns, interpret observations, and write the analyst report.

This approach was functionally appealing: it minimized the amount of dedicated analytics code that had to be written. Yet it also placed repetitive data filtering, numerical comparisons, and narrative generation in one AI-intensive workflow.

As analysis cycles and covered securities multiplied, the application repeatedly brought substantial amounts of fast-changing data into model context. The cost risk came from a combination of large input token volumes, repeated tool interactions, model-generated intermediate analysis, and final report generation.

Why prompt caching was not enough

Azure OpenAI prompt caching can reduce the cost and latency of processing repeated input prefixes, such as stable system instructions or common tool definitions. It is a useful optimization, and those stable components may benefit even in a market-data application. However, new order-book observations frequently alter the dynamic portion of the request. Repeatedly sending fresh CSV records still creates substantial token demand. Prompt caching also does not directly reduce the number of output tokens generated by the model.

The issue was therefore architectural, not simply a missing prompt-cache configuration. Why ask a model to scan every structured record when a small Python service can determine which observations deserve an explanation?

The proposed architecture: Process first, reason second

I proposed separating deterministic detection from AI-powered interpretation. The original CSV remains available in Azure Blob Storage for traceability. A Python service running on Azure Container Apps reads and validates incoming records, computes predefined metrics, and identifies events relevant to the analysts' questions.

The service writes concise, timestamped results into Azure Managed Redis. Instead of returning entire CSV files, the MCP server then exposes relevant, fresh analytical summaries to the application or Foundry agent. Azure OpenAI uses this curated evidence to explain market conditions and produce the report.

1. Azure Blob Storage remains the durable data source

The raw CSV files should remain accessible for replay, investigation, and validation. Blob Storage acts as the source of historical observations; Azure Managed Redis is not a substitute for durable retention.

2. Azure Container Apps performs deterministic processing

A Python application can parse files, validate the schema, group records by symbol and time window, and evaluate conditions such as:

• Changes in displayed bid/ask depth.
• Order-book imbalance over matched depth levels.
• Significant additions or removals of displayed liquidity.
• Large quoted sizes relative to a rolling baseline.
• Repeated occurrences of a predefined event within a time interval.

These are structured calculations. They do not require a language model to evaluate every row. For larger or more continuous feeds, the ingestion and processing pattern should be designed for the actual event volume, but the division of responsibilities remains the same.

3. Azure Managed Redis stores curated, time-sensitive results

The Python service publishes a compact result for a symbol and observation window, together with timestamps, detection rules, and supporting statistics. Cache keys and time-to-live policies should reflect the freshness requirements of each metric.

For example, an illustrative record might look like this:

{
  "symbol": "MSFT",
  "observation_time_et": "10:32:15.123456",
  "price_reference": 429.99,
  "depth_levels_compared": 2,
  "displayed_bid_size": 12800,
  "displayed_ask_size": 4400,
  "order_book_imbalance_pct": 48.8,
  "observation": "Bid-heavy displayed depth at sampled levels",
  "source": "synthetic_example"
}

This example stays consistent with the MSFT CSV above. It is a description of displayed liquidity, not a prediction or a claim of institutional accumulation.

4. MCP and the application or Foundry agent orchestrate reporting

MCP standardizes access to the curated analytical data. The application or Foundry agent is the orchestration layer: it requests the appropriate MCP tool output, passes concise evidence to Azure OpenAI, and delivers the model-generated report to analysts.

The agent may request additional supporting evidence where needed, but the default response is already filtered to the relevant observations. The LLM is now focused on synthesis, explanations, caveats, and communication rather than large-scale CSV scanning.

Understanding the potential token savings

The architecture's key advantage is reducing the amount of data that must enter the model's context. The biggest direct reduction is typically in input tokens because the application sends derived signals rather than raw files. It may also reduce output tokens when unnecessary intermediate summaries and repeated model reasoning are removed.

To illustrate the potential savings, I modeled equivalent report workflows before and after the architectural change using the following assumptions:

1. Input, cached-input, and output tokens per completed report.
2. Model calls and MCP tool invocations per report.
3. End-to-end latency and estimated cost per report.
4. Report completeness, reproducibility, and analytical accuracy.
5. Cache hit rate and data age at the moment the report was generated.

In the original architecture, each stock produces approximately 2,000 Level 2 observations, resulting in 60,000 CSV records. Assuming an average of 30 input tokens per record, processing the raw data could require approximately 1.8 million input tokens across multiple Azure OpenAI model interactions.

With the proposed architecture, a Python application running on Azure Container Apps processes those same 60,000 records, identifies relevant market conditions, and stores the resulting findings in Azure Managed Redis. Assuming the processing logic identifies eight meaningful observations per stock, the MCP server would retrieve only 240 structured findings. At an estimated 40 tokens per finding, Azure OpenAI would receive approximately 9,600 input tokens of analytical data.

This represents a potential 99.47% reduction in data-related input tokens. The architectural advantage is straightforward: the LLM no longer needs to examine every individual order book observation. Instead, it focuses on interpreting a much smaller set of relevant findings while the application handles repetitive numerical processing.

The bigger issue: AI's ease of use can create expensive habits

The broader lesson extends beyond market data. AI is now so easy to integrate that teams can begin using an LLM as a universal query engine, calculator, file scanner, and report writer. This can be productive during experimentation. In production, however, a working AI application is not automatically a cost-efficient AI application.

I see five principles worth applying:

1. Use deterministic code for deterministic work. Search, filtering, arithmetic, and well-defined rule evaluation often belong in application logic.
2. Reduce data before calling the model. Keep only the evidence relevant to the user's question, while retaining source references for verification.
3. Design caching for the workload. Prompt caching and application-managed caching solve different problems; they can complement each other.
4. Make MCP tools precise. A tool that returns a well-defined analytical result is more efficient than one that routinely returns an entire file.
5. Measure business value per AI workflow. Cost, latency, freshness, accuracy, and analyst usefulness matter more than token counts alone.

Final thoughts: Let code identify the events. Let AI explain their significance.

In this financial services scenario, the customer did not need Azure OpenAI to scan the most CSV rows. The customer needed timely, understandable insight into changing buying and selling pressure.

That is why the proposed architecture keeps each technology in the role where it contributes most:

• Azure Blob Storage: retains the underlying market observations.
• Azure Container Apps + Python: processes structured records and detects predefined conditions.
• Azure Managed Redis: makes time-sensitive analytical findings available efficiently.
• MCP + application or Foundry agent: retrieves the right evidence and orchestrates the workflow.
• Azure OpenAI: transforms curated evidence into clear analytical narratives.

As generative AI adoption accelerates, one of the most valuable architectural decisions may be knowing when not to use the model - Marcio Morales

Additional resources

• Azure OpenAI prompt caching
• Azure Container Apps overview
• Azure Managed Redis
• Azure AI workload cost optimization