🚆 Railway Wagon Inspection

Cameras watch a moving train. The system reads each wagon's number and scores its condition — even at night, in tunnels, and through camera shake.

---

The problem, in one paragraph

A person cannot reliably read wagon numbers on a moving train. A camera can — but only when the picture is good. And the picture is worst exactly when inspection matters most: at night, inside tunnels, in rain, and when the train is moving fast enough to blur every frame.

So the real problem is not "detect wagons". It is "what do you do when the image is bad?"

---

How it works

Data flows from the bottom of the stack to the top. Watch the glowing packets climb through it:

▶ Open the interactive 3D version — drag to rotate it

_{Or clone the repo and open docs/index.html in any browser — it needs no internet.}

Five layers, in plain terms:

	Layer	What it does
1️⃣	Cameras	Ordinary RTSP cameras beside the track. Several run at once.
2️⃣	Edge device	A small computer at the trackside. Finds wagons, checks picture quality, repairs the picture if needed, reads the number.
3️⃣	MQTT broker	The postbox between trackside and server. Carries text only.
4️⃣	Backend	Checks each message, saves it, and pushes it onward instantly.
5️⃣	Dashboard	Shows results the moment they arrive. No refreshing.

Video never leaves the device. Only small JSON messages are sent. A trackside 4G link cannot stream video, but it can easily carry a few hundred bytes per frame. This one decision is what makes the system deployable.

---

The three ideas that make it work

1. Repair the picture only when the picture is actually bad

Most systems have a "night mode" that switches on at a fixed time. That is wrong inside a tunnel at noon, and wrong under floodlights at midnight.

This system measures every frame instead — how blurry it is, and how dark it is — and decides from that:

Repairing an image costs a full pass through a model. So if the number can already be read, nothing runs at all.

flowchart LR
    A[Frame arrives] --> B[Measure blur<br/>and brightness]
    B --> C{Can we already<br/>read the number?}
    C -->|Yes| D[Do nothing<br/>saves GPU time]
    C -->|No| E{What is wrong<br/>with it?}
    E -->|Blurry| F[Sharpen it]
    E -->|Too dark| G[Brighten it]
    D --> H[Read the number]
    F --> H
    G --> H

    style D fill:#065f46,stroke:#34d399,color:#fff
    style F fill:#5b21b6,stroke:#a78bfa,color:#fff
    style G fill:#5b21b6,stroke:#a78bfa,color:#fff
    style H fill:#0369a1,stroke:#38bdf8,color:#fff

Loading

2. The system proves its own worth

Every frame is read twice — once before repair, once after. Both answers and both confidence scores are saved.

That means the benefit of the repair step is a number the system reports, not a claim the team makes:

curl -H "Authorization: Bearer $TOKEN" \
     http://localhost:8000/api/metadata/analytics/

{ "total_frames": 1284, "enhanced_frames": 402, "avg_accuracy_improvement": 0.0 }

(That figure starts at zero. Run the pipeline over real footage and it fills in with your own measured result.)

3. Many frames agreeing beats one confident guess

A wagon appears in dozens of frames as it passes. Instead of trusting whichever frame looked sharpest, every reading is collected and the answer most frames agree on wins, weighted by confidence.

frame 41 ─ "W12345"  87%  ┐
frame 42 ─ "W12345"  91%  ├─ 4 frames agree  ──►  W12345  ✅
frame 43 ─ "W12345"  84%  │
frame 44 ─ "W12345"  89%  ┘
frame 45 ─ "W12B45"  99%  ──  1 confident mistake  ──►  rejected ❌

This defeats the classic OCR failure: being 99% certain about one wrong character.

The same idea handles defects. One crack is visible in 30 consecutive frames — counting it 30 times would be wrong. Defects are grouped by (part, damage type, position along the wagon), so one crack counts once.

Health score starts at 100, and each unique defect subtracts:

Damage	Penalty
Crack	−40	🔴 critical
Missing bolt	−30	🔴 critical
Bent / deformed	−15	🟠 major
Foreign object	−10	🟠 major
Rust	−5	🟡 minor

Below 60 → CRITICAL. Below 85 → INSPECTION REQUIRED.

---

What's in this repo

railway-inspection/
├── backend/         Django + Channels + Celery — API, ingest, live push
├── edge-pipeline/   Portable pipeline — runs on any laptop
├── edge-device/     Jetson node — gRPC services, RTSP, TensorRT
├── ocr-service/     Document and video OCR
├── dashboard/       Live operator screen
├── assets/          Sample video      models/  YOLO weights
├── datasets/        COCO8 sample set  submission/  hackathon files
└── docker-compose.yml · Makefile

Two edge versions exist on purpose: edge-device/ is the real Jetson node, edge-pipeline/ is the portable one that runs anywhere for demos and tests.

---

Run it

With Docker (easiest)

make up

	Address
📊 Dashboard	http://localhost:8080
🔌 Inspection API	http://localhost:8000/api/
📄 OCR service	http://localhost:8001/api/

Use your own video: drop it in assets/ and run VIDEO_FILE=my_train.mp4 make up.

Run it locally

Needs Python 3.11–3.13.

make setup     # one virtualenv per part
make migrate   # create the database
make test      # all 86 tests
make demo      # run the pipeline over the sample video

Start Redis and the MQTT broker:

brew install redis mosquitto
brew services start redis
mosquitto -c mosquitto.conf -d

Then, in separate terminals:

cd backend && .venv/bin/daphne -p 8000 config.asgi:application
cd backend && .venv/bin/celery -A config worker -l info
cd backend && .venv/bin/python manage.py run_mqtt_consumer

No Celery worker running? Use run_mqtt_consumer --eager — it saves messages directly.

---

API

Everything needs a login token first:

curl -X POST http://localhost:8000/api/auth/token/ \
     -H 'Content-Type: application/json' \
     -d '{"username":"USER","password":"PASS"}'

Send it back as Authorization: Bearer <token>.

Endpoint	What you get
GET /api/metadata/live/?hours=1	Recent frames
GET /api/metadata/analytics/	Totals, plus the measured repair benefit
GET /api/metadata/top_blur/	The worst frames the system rescued
GET /api/wagons/	Inspected wagons
GET /api/wagons/flagged/?threshold=85	Wagons needing attention
ws://localhost:8000/ws/live/	Live frame updates
ws://localhost:8000/ws/dashboard/	Live wagon results

Test it without a camera:

cd backend && .venv/bin/python manage.py publish_test_payload --kind frame --count 5

---

Built for the real world

🌊 Never falls behind	If processing slows down, the oldest frame is dropped, not the newest. Live inspection needs now, not a backlog.
🔁 Safe to repeat	After a network outage, devices resend everything. Repeats update the same record instead of creating duplicates.
⏱️ Two different times	When the frame was captured and when it arrived are stored separately. A device offline for ten minutes still reports the truth.
🛟 Degrades, never dies	Missing GPU weights, missing OCR library, broker down — each is logged and skipped. The pipeline keeps running on a simpler path.
🔌 Reconnects itself	Cameras and the broker both retry with backoff. Runs under systemd and restarts on failure.

---

Tests

make test

86 tests, no external services required:

Part	Tests	Covers
backend/	36	Ingest, repeats, bad data, analytics, REST API, login, WebSockets
edge-pipeline/	23	Health scoring, defect grouping, OCR voting, asset lookup
edge-device/	27	Repair decisions, enhancement, camera loop, crop, publishing

---

Where it stands

Working today

• ✅ Full path verified end to end: edge → MQTT → database → dashboard
• ✅ Wagon detection, OCR, agreement voting across frames
• ✅ Quality-based repair decisions
• ✅ Health scoring with defect grouping

Not done yet — stated plainly

• ⚠️ No trained repair models ship here. The deblurring and low-light models have no weights in this repo, so repair currently uses classical image processing (CLAHE + sharpening). Drop TorchScript weights into models/ and the learned versions load automatically.
• ⚠️ Detection is not wagon-specific. It uses stock YOLO11n, where large vehicles stand in for wagons. A real deployment needs a model trained on wagons and their parts.
• ⚠️ Accuracy is not yet measured on real footage. The system computes this itself — run it on real video and read avg_accuracy_improvement.

---

_{Built for the Adani Hackathon · Interactive 3D architecture}