Analytics dashboards need a metric and chart vocabulary grounded in established LMS analytics

[zaira-bibi]

Summary

Milestone 1 ships a single login-activity chart (#487) — but the epic has no agreed reference for what those dashboards should show or how. Without a shared vocabulary of metrics and chart types, each dashboard slice risks inventing its presentation ad hoc, and the chart primitives built for #487 may not anticipate what Analytics dashboard actually needs.

Why it matters

• The dashboard scoping ("engagement, activity" views) is currently one line in the roadmap; it needs concrete candidate metrics before it can be decomposed.
• The Analytics are not visible anywhere in the product UI #487 charting approach (hand-rolled SVG primitives, no charting dependency) was chosen partly on the bet that near-term dashboards need only simple time-series charts — that bet should be validated against what mature LMS analytics actually display.

Prior art surveyed (Canvas LMS, Totara)

Canvas Admin Analytics (admin-facing; three dashboards, 20+ charts/tables; data served from ~8-hour-lagged rollups, not live):

• Overview: courses by status; student & instructor engagement over time (weekly page views / participations as line charts); enrollment trends; average grades.
• Course: grade + engagement statistics; interactions by feature; watch-list tables ("courses with low average grades", "courses with low student activity").
• Student: grade averages; submissions by status (on-time/late/missing); per-student engagement; detail tables.
• Filters: date range, term, sub-account. Every chart is paired with a data-table view and CSV export.

Canvas New Analytics (instructor-facing, per course):

• Course Grade: average grade as an interactive line chart, comparable against a section, assignment, or individual student.
• Weekly Online Activity: average page views + participations per week — the closest analogue to our login-activity series (activity counts bucketed by time).
• Students: per-student page views, participations, last-activity time.

Totara ships no fixed analytics dashboards — reports built in Report Builder (from sources like Site Logs or course completion) are placed on per-audience dashboards as table or graph blocks. Chart types (ChartJS): column, horizontal bar, line, area, pie, donut, progress donut, scatter. Site-usage stats are exactly the time-bucketed-count pattern (Site Logs + time aggregation + user-ID count).

What the survey indicates

• Time-bucketed activity counts rendered as bar/line charts with a date-range filter are the entry-point visualization in both products — line/bar covers the large majority of what both display.
• Chart-plus-companion-table (or summary tiles) is the standard presentation.
• Serving dashboards from lagged/materialized rollups is normal practice (Canvas is openly ~8 hours behind), relevant to the roadmap's real-time-freshness cross-cutting concern.
• A proven metric vocabulary for complete analytics dashboard to draw from: activity/engagement over time, active-user trends, per-course activity ranking (low-activity watch-lists), submissions by status, grade averages/distributions.

What's needed

A decision, when the complete dashboard is scoped, on which of these metrics and chart types the operator/instructor dashboards adopt — recorded so the chart primitives and read API grow deliberately rather than per-slice.

This description was written with the assistance of an LLM (Claude).

[zaira-bibi]

Decision: self-serve dashboards

We won't fix a metric list. Operators and instructors compose their own charts from the event vocabulary instead of picking from pre-built ones.

The survey supports this. Time-bucketed counts as line/bar with a date filter are the entry point in both products and cover most of what they show. Totara already works this way: no fixed dashboards, reports composed from sources.

What changes

Per-metric plumbing means every new question costs a migration, SQL, an endpoint and a component per question, which is
the ceiling this issue flagged.

Two endpoints replace it:

• Schema projection — lists event types, versions and payload fields from the existing event registry (ANALYTICS_EVENTS), so the UI's vocabulary stays current as producers are added.
• Constrained query — event type, bounded time range, bucket, metric, optional group-by, optional filters; returns a tidy series. Not SQL: every field is validated against the registry.

Surveyed metrics against that model

Metric	Status
Activity over time	Covered — event count by bucket
Active-user trends	Covered — distinct actors by bucket (actor_id is already a raw_events column)
Interactions by feature	Covered within one event type; comparing across types isn't
Course activity ranking, watch-lists	Deferred — needs top-N ranking and a course dimension no producer emits
Grade averages and distributions	Blocked — the platform holds no grade data
Submissions by status	Blocked — assessment.submitted has a schema but no producer

Three shapes are out of scope for the first cut: top-N ranking, histograms, and cross-event-type comparison.

#487 chart primitives

The hand-rolled SVG bet holds. Line and bar cover the entry-point charts. One gap: a group-by returns N series, so the primitives need multi-series support with a legend and a categorical palette.

Adopted: line, bar, area, summary tiles, and the companion data table both products pair with charts.

Pie, donut and scatter deferred.

Freshness

Canvas runs roughly eight hours behind, so near-real-time isn't table stakes. The first cut queries raw events under mandatory time bounds, with continuous aggregates added later behind the same query contract.

Producers come first. Self-serve over one event type shows nothing, so we will add chat authoring events for now and the self-serve slice follows.

This comment was written with the assistance of an LLM (Claude).