Traffic Detector Agent

A multi-agent system for predicting and monitoring highway traffic flow using XGBoost flow prediction and deviation analysis. Designed for the VCI, it monitors a network of traffic sensors across highways A20 and A28 and provides hourly assessments of traffic.

Overview

The dt4mob-agent-vci project implements a distributed traffic monitoring system consisting of:

• Coordinator Agent: Orchestrates the monitoring cycle and aggregates results across all sensors.

• Subagents: Individual specialized agents assigned to specific sensor/direction pairs that:

  • Fetch historical flow data and weather conditions.
  • Run pretrained XGBoost models to predict baseline traffic flow.
  • Compare predictions against actual observed flow using SMAPE (Symmetric Mean Absolute Percentage Error).
  • Classify deviations and provide contextual analysis ( football matches, holidays, rain).

The system integrates with:

• Ditto
• S3/MinIO: For model artifact storage and retrieval.
• MLflow: For experiment tracking and run logging.

Architecture

Core Components

config.yaml               → Goose agent configuration
mcp_server.py             → MCP (Model Context Protocol) tool server
coordinator_recipe.yaml   → Goose coordinator recipe
subagent_recipe.yaml      → Goose subagent recipe
APItoken.py               → Authentication service
custom_skynetollama.json  → LLM model configuration
Dockerfile                → Docker configuration
docker-compose.yaml       → Docker Compose configuration

Sensor Network

The system monitors traffic across two highways through counters along the network. Each detector has two directions, creating a graph-based corridor topology for tracking traffic flow patterns.

Features

Baseline Traffic Prediction

• Model: XGBoost regressors per sensor/direction trained on historical flow data.
• Features:
  • Temporal encoding (hour, day, month, weekday via sine/cosine).
  • Lag flows (1h, 24h, 168h).
  • Weather (temperature, precipitation).
  • Holiday/weekend indicators.
• Data: Hourly flows and weather, football games data from 2022-01-01 to 2024-12-31.
• Scalers: Fitted on all continuous features for normalization.

Anomaly Detection

• Metric: SMAPE (Symmetric Mean Absolute Percentage Error).

  • Compares baseline prediction vs. actual observed flow.

• Threshold: Configurable via ANOMALY_THRESHOLD (default: 0.20 or 20%).

Contextual Analysis

Each subagent evaluates deviations and provides one of six verdicts:

Verdict	Condition	Recommendation
within_baseline	SMAPE within threshold	No action
expected_conditions_football	Elevated SMAPE + football match detected	Do NOT flag as deviation
expected_conditions_rain	Elevated SMAPE + precipitation > 10mm on >5 sensors	Do NOT flag as deviation
expected_conditions_holiday	Elevated SMAPE + holiday/school vacation (is_holiday=1)	Do NOT flag as deviation
localized_event	SMAPE above threshold, no external explanation	Flag as deviation
pending	Actual flow reading not yet available in Ditto	Defer decision

Simulation Mode

• Set SIMULATION_START environment variable to a past timestamp.
• The simulated clock advances, allowing multi-hour test runs.
• Writes to a historic database.

Installation & Setup

Prerequisites

• Python 3.10+
• Docker (for containerized deployment)
• S3-compatible storage (AWS S3 or MinIO)
• Ditto instance (historic and live)
• MLflow tracking server

Environment Variables

Create a .env file with the following:

# Ditto URL
VCI_HISTORIC_DITTO_URL=https://dt4mob.av.it.pt/historic/

# Authentication
USER_DITTO=<username>
PASS_DITTO=<password>
TOKEN_URL=<auth-server-url>
CLIENT_ID=<oauth-client-id>

# MOnitoring parameter
ANOMALY_THRESHOLD=0.20

# MLflow
MLFLOW_TRACKING_URI=http://mlflow:5000
MLFLOW_EXPERIMENT_NAME=goose-traffic-detector

# S3 Model Storage
AWS_ACCESS_KEY_ID=<access-key>
AWS_SECRET_ACCESS_KEY=<secret>
AWS_DEFAULT_REGION=us-east-1
AWS_ENDPOINT_URL=https://dt4mob.av.it.pt/s3
AWS_BUCKET_NAME=dt4mob-public

# Simulation
RUN_MODE=test
SIMULATION_START=2024-07-21T15:00:00
TEST_START=2024-07-21T15:00:00
TEST_END=2024-07-22T15:00:00

Installation

# Clone the repository
git clone https://github.com/ATNoG/dt4mob-agent-vci.git
cd dt4mob-agent-vci

# Use Docker
docker compose up --build 

MCP Tool Server

The MCP server exposes traffic detection tools via the FastMCP framework.

Coordinator Agent Flow

1. Initialize: Coordinator calls get_current_timestamp() to get the evaluation hour.

2. Dispatch: Spawn subagents for each sensor/direction pair.

3. Subagent Execution (per sensor):

   • Call get_flows() → fetch lag flows, football context, weather.
   • Call baseline_predict() → run XGBoost model.
   • Call get_actual_flow() → fetch real observed flow.
   • Call check_deviation() → compute SMAPE and flag deviations.
   • Return structured verdict with contextual analysis.

4. Aggregation: Coordinator collects all subagent payloads.

5. Logging:

   • Call log_run_summary() → log metrics to MLflow.
   • Call log_final_ditto() → write assessment back to Ditto.

Model Artifacts

XGBoost models are stored in S3 with the following structure:

s3://models/agent_models/
├── 716_-1.0/
│   ├── model.ubj           (XGBoost binary)
│   ├── feat_scaler.pkl     (StandardScaler for features)
│   └── tgt_scaler.pkl      (StandardScaler for target)
└── ...

Models are downloaded and cached in:

/tmp/model_cache/

Configuration File

config.yaml

Goose framework configuration for the coordinator agent:

• GOOSE_MODE: auto
• Extensions: traffic-detector MCP server
• Environment variables passed to the MCP server

Monitoring & Logging

All tool calls are logged to MLflow with:

• Parameters: Input arguments to each tool.
• Metrics: SMAPE, baseline predictions, actual flows.
• Tags: Sensor ID, road, km marker, tool name.
• Artifacts: Full JSON output payloads.

Run summaries aggregate metrics across all sensors and corridors, providing visibility into:

• Data coverage (sensors completed / attempted).
• Anomaly counts and classifications.
• Mean and max SMAPE across the network.
• Corridor-level verdicts (e.g., macro drift vs. localized events).

Ditto Authentication

Verify that TOKEN_URL, CLIENT_ID, and credentials are correct.

The AuthenticationService handles OAuth token refresh automatically.

References

• Goose Documentation
• FastMCP Framework
• MLflow