Blog Post

Educator Developer Blog
7 MIN READ

City simulator with AI agents for traffic congestion

Shoib's avatar
Shoib
Tin Contributor
May 11, 2026

Project Overview

Hi I'm Shoib Muhammad an MEng Mathematical Computation student at UCL, I set out to tackle one of the most persistent challenges in modern cities: traffic congestion. In London alone, drivers lose around 141 hours each year to congestion, with the wider UK economy losing approximately £5.1 billion in productivity. Traditional fixed-cycle traffic lights cannot adapt to changing demand, and even adaptive systems such as SCOOT and SCATS tend to focus on single intersections rather than understanding the full network.

The aim of this project was to build something practical and experimental: a high fidelity simulation environment where multi agent AI systems can be tested, broken, and improved without affecting real world traffic. It is a safe space to explore how intelligent systems could one day manage entire city networks.

The Stack

The project brings together a mix of gaming, backend, and AI technologies. Microsoft Foundry using GPT-4o handles both vision and reasoning, allowing the system to count vehicles from camera feeds and make decisions from a single cloud endpoint. The Microsoft Agent Framework enables multi agent coordination, structuring a pipeline from camera input through to intersection decisions and orchestration, with up to 20 agents running in parallel. Unreal Engine 5 provides the simulation itself, modelling a 20 intersection city with realistic vehicle movement and camera views. FastAPI acts as the backend, linking everything together with schema validation, a polling loop, and a deterministic testing suite to ensure reliability.

 

Technology

Core Function

Impact

Microsoft Foundry (GPT-4o)

Vision + Reasoning

Counted vehicles from camera frames and made per-intersection decisions through one cloud endpoint.

Microsoft Agent Framework

Multi-Agent Orchestration

Structured the Camera → Intersection → Orchestrator pipeline and fanned out 20 parallel agent runs per cycle.

Unreal Engine 5

3D Simulation Environment

Hosted the 20-intersection city, vehicle physics, and per-intersection cameras.

FastAPI

Backend & Schema Validation

Bridged UE5 and Azure with a polling loop, schema contracts, and a 32-test deterministic suite.

How a Decision Cycle Flows

The simulation models a small city with 20 intersections. Each decision cycle is powered by three types of AI agents working together. A Camera Agent acts as the perception layer, using GPT-4o vision to count vehicles on each road approaching an intersection. Each intersection is controlled by its own agent, which evaluates traffic pressure and decides whether to change the signal phase. An Orchestrator Agent then coordinates these decisions across the network, ensuring that neighbouring intersections do not switch at the same time and cause gridlock.

At the start of each cycle, Unreal Engine generates screenshots from each intersection along with traffic flow data. The backend processes and validates this data, then sends it to the Camera Agent. The output is distributed to all intersection agents in parallel, and their decisions are combined by the orchestrator. The final decision is written back into the simulation, updating the traffic lights. If anything fails, the system simply holds its current state and retries on the next cycle, making the system robust even with non deterministic AI outputs.

  • Camera Agent – the perception layer, using GPT-4o vision to count vehicles on every approach of every intersection.
  • Intersection Agents (×20, parallel) – each owns a single junction, evaluates queue pressure and recommends whether to switch the signal phase.
  • Orchestrator Agent – the coordinator, applying an adjacency-aware rule that prevents two neighbouring intersections from switching simultaneously and dumping traffic into each other's red phases.
Agent Topology

When a decision cycle begins, UE5 writes 20 PNG screenshots and a flow-direction JSON to a shared directory. The FastAPI backend polls the directory, validates the batch against a 20-unique-intersection coverage gate, and hands the images to the Camera Agent. Its structured output is fanned out to 20 parallel Intersection Agents through the Microsoft Agent Framework, then fanned back in to the Orchestrator. The Orchestrator's decision JSON is written back to the shared directory; UE5 polls it and updates signal phases. If any layer fails, the simulation simply holds its previous pattern and tries again on the next cycle.

The JSON payload shared between every layer has to follow strict formats. This matters because GPT-4o output is non-deterministic; early iterations would return the wrong schema, alter key names or silently omit intersections it judged unimportant. Treating prompt engineering and schema validation as one component is what makes an LLM-based control loop robust enough to leave running. 

Decision cycle flow diagram

One of the key challenges was ensuring consistency. GPT-4o outputs are not always predictable, so strict schema validation was essential. Early versions would return incorrect formats or omit data, which made the system unreliable. Treating prompt design and validation as one combined problem was critical to making the system stable enough to run continuously.

Results

Metric

Baseline

AI-Enhanced

Mean throughput (300s run)

204.4

127.1

95% CI

[195.6, 213.4]

[121.3, 132.9]

Throughput dispersion (IQR)

≈ 240

≈ 120

Across forty simulation runs, each lasting 300 seconds, the traditional fixed cycle system outperformed the AI driven system in terms of throughput. The AI system processed fewer vehicles, with performance affected by several factors including decision latency, small errors in vehicle detection, and the use of a general purpose language model for a task that ideally requires millisecond level control.

However, the numeric result was not the main success of the project. The primary goal was to explore and demonstrate what can be built using Azure AI Foundry and multi agent systems, and in that respect the project achieved its aim.

Forty 300-second runs across two demand profiles, ten replicates per condition. The fixed-cycle baseline beat the AI-enhanced mode on throughput by 37.8%. Three factors drove the gap: 3–10 second decision-cycle latency, vision noise of around one vehicle per approach and using a general-purpose LLM for what is effectively a millisecond-scale control task. 

The final result is not very important to the success of this project. The primary goal of this project was to demonstrate Azure AI Foundry's capabilities, which was readily achieved.

Main Deliverable

One of the most valuable outcomes of the project is the dataset produced. For every decision cycle and each intersection, the system logs a full state action outcome record. This includes screenshots, true queue lengths from the simulator, estimated queue lengths from the AI, throughput, signal states, and decision outputs. In total, the project generated around 48,000 labelled data points and associated images, producing roughly 4.8 GB of structured data.

This dataset has real value beyond the simulation. It can be used to train improved vision models that are more robust to noise, or to train reinforcement learning agents that learn traffic control policies from experience rather than predefined rules.

Every cycle, for every intersection, the simulation logs a state–action–outcome triple: a 512×512 screenshot, ground-truth queue lengths, GPT-4o-derived queue lengths, throughput, signal state, flow direction, and the orchestrator's recommended action. Forty runs across different modes and traffic densities produced roughly 48,000 triples and 48,000 paired screenshots, about 4.8 GB of structured, labelled data. Each screenshot has both a clean label (simulator ground truth) and a noisy label (vision estimate), making it directly usable for fine-tuning a vehicle-counting model robust to real-world camera noise. The same triples are in the format offline reinforcement learning algorithms expect.

Future Work

The next step would be to replace the intersection agents with a reinforcement learning model trained on this dataset, using traffic flow improvements and queue lengths as rewards. Another improvement would be to fine tune a dedicated vision model to replace the current GPT-4o based approach, using the existing data as training input. Expanding the simulation to include more realistic road types, such as roundabouts and T junctions, and using real traffic data to drive demand patterns would also make the system more representative of real cities.

The most direct continuation is replacing the Intersection Agents with a small reinforcement learning policy trained on the dataset above, using throughput delta and the queue length as the reward signals. Beyond that, the priority extensions are:

  • A fine-tuned vision model to replace the GPT-4o vision call - the screenshot–queue-length pairs already in the dataset are the training set.
  • Larger, more realistic networks with mixed intersection types (roundabouts, T-junctions) and time-varying demand curves calibrated against real traffic feeds.

Final Thoughts

This project is a great example of how modern AI tools can be combined to build complex, experimental systems. If you are a student developer interested in AI, simulation, or multi agent systems, this is a space full of opportunities to explore.  Microsoft Foundry,

If you're exploring the technologies behind this project, the Microsoft's AI stack, Microsoft Agent Framework, Agent Topologies and the AI Agents for Beginners are the resources I'd recommend.

I would also like to thank my external supervisor, Lee Stott, for his guidance and support throughout the project. 

Link to Project

Link to my LinkedIn 

Want to Learn More

Join us on Thursday 21 May at 12pm UK time for a live session in the Microsoft Foundry Discord Community where we will explore how a student built a full 20 intersection AI powered traffic simulation using Azure AI Foundry, GPT-4o and the Microsoft Agent Framework.

This is a great opportunity for student developers who want to go beyond theory and see how modern AI tools can be combined to build real, end to end systems.

In this session, we will break down how multi agent systems can be used to solve complex real world problems, how Unreal Engine was used to create a realistic simulation environment, and how Microsoft Foundry enabled rapid experimentation with vision and reasoning models. You will also get a behind the scenes look at the challenges of working with non deterministic AI outputs and how to make these systems more reliable through structured design.

If you are interested in AI, simulation, game development, or building intelligent systems that operate at scale, this session will give you practical ideas you can apply to your own projects. Whether you are just starting out or already experimenting with AI agents, you will leave with a clearer understanding of how to design, test, and iterate on complex systems.

Bring your questions, get involved in the discussion, and connect with others in the community who are building with Microsoft Foundry.

https://discord.gg/rMh9PkjF?event=1502936656506781827 



Updated May 10, 2026
Version 1.0

2 Comments

  • Really cool project! Curious whether you benchmarked the LLM-agent approach against a classical MARL baseline. That's where the interesting question hides: are the generative agents actually better at managing congestion, or just easier to reason about than a reward-shaped policy?

    • Shoib's avatar
      Shoib
      Tin Contributor

      Thanks for the comment! I have not benchmarked the LLM approach against MARL. This was mainly due to the time constraints for the project. The reason why I think that RL would be the best approach is from the research I did with existing solutions; the most successful ones used RL.