Skip to content

AI call analysis: feedback on interviews, syncs and presentations (all surfaces) - #115

Merged
ralyodio merged 1 commit into
masterfrom
feat/call-analysis
Sep 24, 2026
Merged

ralyodio merged 1 commit into
masterfrom
feat/call-analysis

Conversation

@ralyodio

Copy link
Copy Markdown
Contributor

AI call analysis for PairUX: feedback on interviews, team syncs and presentations. The host decides before starting a screen share or voice call, on every surface, and turning it on turns "Record the call" on too.

Surfaces

Surface Pre-start choice Capture Participant notice
Web / PWA Card on /host (kind + record toggle) Mixed call audio (host + all participants) + screen stills; auto-starts the video recorder Banner on /session/[id] and /view/[id]
Desktop Panel on the home screen, applied to Start a Meeting / Create Link / Voice / screen picker Same, in CapturePreview; auto local recording Banner in the viewer
Mobile Switches on the Host screen Host mic via expo-av, 5-minute segments (needs a new app build) Banner in the viewer

Pipeline

Chunks go to the private call-analysis bucket. pg_cron calls /api/analyses/run every minute, which:

  1. joins the audio (ffmpeg, added to the web image);
  2. transcribes it with OpenAI gpt-4o-transcribe-diarize (falls back to whisper-1);
  3. measures talk share, wpm, fillers, questions and interruptions;
  4. has claude-opus-5 write a structured report (refusal fallback default enabled);
  5. keeps or deletes the recording as the host chose, and emails the host.

Reports live at /analyses. The feature is gated to Pro/Team.

Ops needed at merge

  • Apply 20260924140000_call_analysis.sql (bucket, table, claim_call_analysis).
  • Add pg_cron job call-analysis posting to https://pairux.com/api/analyses/run with the reminders secret.
  • Keys OPENAI_API_KEY and ANTHROPIC_API_KEY are already set on Railway.

Tests

  • web: 952 (new metrics/storage tests)
  • desktop: 724
  • mobile: 314, plus verify:bundle

Typecheck and lint are clean on all three.

Not exercised end to end: a real call through ffmpeg, OpenAI and Claude. That can't run on this box (no Electron, and I didn't want to spend API credit on a synthetic call). The first real analyzed call is the live test.

🤖 Generated with Claude Code

Hosts switch on AI call analysis before a screen share or voice call on
pairux.com, the desktop app or the mobile app; "Record the call" turns on
with it. After the call they get a private feedback report.

- Session settings carry `analysis` ({ enabled, kind, keepRecording }),
  set at creation only; Pro/Team gate in POST /api/sessions and the
  scheduled-meeting start route.
- Capture: web and desktop upload the call's mixed audio (host + everyone)
  in 15s chunks plus a screen still every 20s; mobile records the host mic
  with expo-av in 5-minute segments. Chunks are grouped by recorder run so
  reloads and segments join cleanly.
- Job: pg_cron -> /api/analyses/run claims one analysis, joins the audio
  (ffmpeg, now in the web image), transcribes with OpenAI
  gpt-4o-transcribe-diarize (whisper-1 fallback), measures talk share, pace,
  fillers, questions and interruptions, and has claude-opus-5 write a
  structured report (interview / team sync / presentation / general).
  Recording kept or deleted per the host's choice; host emailed.
- Consent: every participant sees a recording notice on web, desktop and
  mobile for the whole call.
- Reports at /analyses and /analyses/[id]; dashboard link; desktop links out.
- Visible: home section, /features entry and /features/call-analysis,
  pricing (Pro/Team), AI note taker use case now available, sitemap, llms.txt.
- Root @types/react pinned to 19 so adding expo-av cannot re-hoist React 18
  types into the web app.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@socket-security

Copy link
Copy Markdown

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Added@​anthropic-ai/​sdk@​0.128.06610092100100
Addedexpo-av@​15.0.21001008993100

View full report

@github-actions

Copy link
Copy Markdown

ThreatCrush Security Scan

47 finding(s)

HIGH/CRITICAL: 4 | MEDIUM: 27 | LOW: 16

Severity Rule Location
HIGH sh-eval-expansion .githooks/pre-commit:33
HIGH js-electron-node-integration apps/desktop/src/main/window.ts:49
HIGH sh-unquoted-expansion-destructive apps/installer/scripts/install.sh:715
HIGH sh-unquoted-expansion-destructive apps/installer/scripts/install.sh:917
MEDIUM sh-remote-script-execution apps/installer/scripts/install.sh:691
MEDIUM sh-remote-script-execution apps/installer/scripts/install.sh:820
MEDIUM sh-remote-script-execution apps/installer/scripts/install.sh:822
MEDIUM sh-remote-script-execution apps/installer/scripts/install.sh:1078
MEDIUM sh-remote-script-execution apps/installer/scripts/install.sh:1080
MEDIUM sh-remote-script-execution apps/livekit/setup-livekit-server.sh:93
MEDIUM sh-remote-script-execution apps/turn/deploy-droplet.sh:62
MEDIUM js-unescaped-html-sink apps/web/src/app/blog/[slug]/page.tsx:48
MEDIUM js-unescaped-html-sink apps/web/src/app/blog/[slug]/page.tsx:73
MEDIUM js-unescaped-html-sink apps/web/src/app/c/[handle]/page.tsx:192
MEDIUM js-open-redirect apps/web/src/app/cli/authorize/consent.tsx:66
MEDIUM js-unescaped-html-sink apps/web/src/app/l/[joinCode]/page.tsx:129
MEDIUM js-unescaped-html-sink apps/web/src/app/l/[joinCode]/page.tsx:213
MEDIUM js-unescaped-html-sink apps/web/src/app/layout.tsx:142
MEDIUM js-unescaped-html-sink apps/web/src/app/live/page.tsx:145
MEDIUM js-unescaped-html-sink apps/web/src/app/page.tsx:123
MEDIUM js-unescaped-html-sink apps/web/src/app/pricing/page.tsx:286
MEDIUM js-open-redirect apps/web/src/app/pricing/UpgradeButton.tsx:50
MEDIUM js-unescaped-html-sink apps/web/src/app/u/[username]/page.tsx:282
MEDIUM js-open-redirect apps/web/src/hooks/useDesktopHandoff.ts:24
MEDIUM redos-nested-quantifier apps/web/src/lib/deliverable.ts:11
MEDIUM js-unescaped-html-sink apps/web/src/lib/player/player.ts:124
MEDIUM js-unescaped-html-sink apps/web/src/lib/player/player.ts:393
MEDIUM js-unescaped-html-sink apps/web/src/lib/player/player.ts:396
MEDIUM js-unescaped-html-sink apps/web/src/lib/player/player.ts:405
MEDIUM js-unescaped-html-sink apps/web/src/lib/player/player.ts:608
MEDIUM sql-template-interpolation packages/ai-core/src/prompts.ts:36
LOW secret-generic-credential apps/desktop/src/main/agent-cli/commands.test.ts:138
LOW secret-generic-credential apps/desktop/src/main/agent-cli/commands.test.ts:139
LOW secret-generic-credential apps/desktop/src/main/agent-cli/commands.test.ts:340
LOW secret-generic-credential apps/desktop/src/main/agent-cli/commands.test.ts:341
LOW secret-generic-credential apps/desktop/src/main/agent-cli/commands.test.ts:371
LOW secret-generic-credential apps/desktop/src/main/agent-cli/commands.test.ts:372
LOW secret-generic-credential apps/desktop/src/main/agent-cli/commands.test.ts:383
LOW secret-generic-credential apps/desktop/src/main/agent-cli/commands.test.ts:397
LOW secret-generic-credential apps/desktop/src/main/agent-cli/commands.test.ts:398
LOW secret-generic-credential apps/desktop/src/main/agent-cli/commands.test.ts:405
LOW secret-generic-credential apps/desktop/src/main/agent-cli/commands.test.ts:415
LOW secret-generic-credential apps/desktop/src/main/agent-cli/commands.test.ts:416
LOW secret-generic-credential apps/livekit/fly.toml:12
LOW secret-generic-credential apps/turn/fly.toml:11
LOW secret-generic-credential docs/API.md:747
LOW secret-generic-credential docs/API.md:753

Snippets are redacted; ThreatCrush never prints matched credential material.

@ralyodio
ralyodio merged commit fea8943 into master Sep 24, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant