Skip to content

Commit 3bc7cdc

Browse files
authored
Merge pull request #1262 from OpenAdaptAI/privacy/scrub-outbound-by-default
feat(privacy): scrub everything that leaves the machine by default
2 parents 5c6a76e + 057d12d commit 3bc7cdc

22 files changed

Lines changed: 2578 additions & 67 deletions

‎packages/capture/CLAUDE.md‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ Key responsibilities:
1010
- Process raw events into structured actions (clicks, drags, typing)
1111
- Retain optional action-time structural evidence where a provider exists
1212
- Keep raw captures local unless a caller invokes an explicit transfer path
13-
- Expose an optional privacy integration without claiming automatic scrubbing
13+
- Scrub a copy before `capture share` sends it; never claim the local recording is scrubbed
1414

1515
**Always use PRs, never push directly to main**
1616

@@ -65,7 +65,8 @@ openadapt_capture/
6565
video.py # External FFmpeg process boundary and frame extraction
6666
audio.py # Audio recording + transcription
6767
visualize/ # Demo GIF and HTML viewer generation
68-
share.py # Magic Wormhole sharing
68+
share.py # Magic Wormhole sharing (sends a scrubbed copy by default)
69+
share_scrub.py # Builds that scrubbed copy
6970
cli.py # CLI commands (capture record, capture info, capture share)
7071
experimental/ # Unfinished prototypes kept out of the package (Chrome extension, WebSocket bridge)
7172
```

‎packages/capture/README.md‎

Lines changed: 34 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -179,9 +179,39 @@ is installed, and it deletes the audio after transcription unless you set
179179

180180
A recording can contain anything that was on screen or typed, including
181181
passwords and personal data. It stays on your computer. Capture uploads
182-
nothing unless you run `capture share` or set `send_profile=True`. The
183-
`privacy` extra installs `openadapt-privacy` but doesn't remove anything from
184-
a recording by itself.
182+
nothing unless you run `capture share` or set `send_profile=True`, and neither
183+
sends the raw recording by default.
184+
185+
`capture share send` scrubs a copy of the recording and sends only the copy.
186+
Scrubbing needs the `privacy` extra:
187+
188+
```bash
189+
pip install "openadapt-capture[privacy,share]"
190+
python -m spacy download en_core_web_sm
191+
capture share send ./my_recording
192+
```
193+
194+
The copy keeps the recording database with text and accessibility values
195+
scrubbed and screenshots redacted. It leaves out typed keys (a name typed one
196+
key at a time can't be recognized), audio, video, and any file type that can't
197+
be scrubbed, and lists what it left out in `SCRUB_MANIFEST.json`. Scrubbing is
198+
automatic and can miss data, so review the copy before you share it further.
199+
Without the `privacy` extra, `capture share send` refuses and sends nothing.
200+
201+
To send the raw recording, for example synthetic demo data, opt out
202+
explicitly:
203+
204+
```bash
205+
capture share send ./my_recording --unscrubbed --reason "synthetic demo data"
206+
```
207+
208+
The opt-out is appended to `share-unscrubbed.jsonl` in Capture's data folder
209+
(`~/Library/Application Support/ai.openadapt.capture` on macOS, or
210+
`OPENADAPT_CAPTURE_DATA_DIR`), and the archive carries an
211+
`UNSCRUBBED_SHARE.json` file so the receiver knows.
212+
213+
`send_profile=True` sends only the numbers and flags from `profiling.json`;
214+
folder names and window titles are withheld.
185215

186216
Shadow-mode pattern mining, which notices repeated tasks and offers a draft,
187217
is off by default, needs the person's acknowledgement to turn on, and keeps
@@ -225,7 +255,7 @@ are qualified are covered in [docs/COMPLIANCE.md](https://github.com/OpenAdaptAI
225255
| --- | --- |
226256
| `transcribe-fast` | Local transcription with faster-whisper |
227257
| `transcribe` | Local transcription with openai-whisper |
228-
| `privacy` | The `openadapt-privacy` library, for your own integration |
258+
| `privacy` | `openadapt-privacy`, which `capture share send` needs to scrub a recording |
229259
| `share` | `capture share`, using Magic Wormhole |
230260
| `linux` | Accessibility observation on Linux (AT-SPI) |
231261
| `all` | All of the above |

‎packages/capture/docs/COMPLIANCE.md‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -34,8 +34,11 @@ See [AUTHENTICATION_HANDOFF.md](AUTHENTICATION_HANDOFF.md).
3434

3535
Capture doesn't upload a session. The sharing command (`capture share`) and the
3636
profiling transfer (`send_profile`) are the only two uploads, and both run
37-
only when the operator asks for them. Installing the `privacy` extra doesn't
38-
scrub anything by itself.
37+
only when the operator asks for them. `capture share send` scrubs a copy of
38+
the recording and sends only the copy; it refuses when the `privacy` extra is
39+
missing. Sending the raw recording needs `--unscrubbed --reason ...`, which is
40+
recorded in a local ledger and inside the archive. The profiling transfer
41+
sends numbers and flags only. The local recording itself is not scrubbed.
3942

4043
Structural observations carry control names, window titles, and accessibility
4144
ancestry, so they stay inside the same local boundary. Accessibility text is

‎packages/capture/openadapt_capture/cli.py‎

Lines changed: 15 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -572,23 +572,36 @@ def uninstall_ffmpeg() -> None:
572572
print(f"Nothing to remove at {root}")
573573

574574

575-
def share(action: str, path_or_code: str, output_dir: str = ".") -> None:
575+
def share(
576+
action: str,
577+
path_or_code: str,
578+
output_dir: str = ".",
579+
unscrubbed: bool = False,
580+
reason: str | None = None,
581+
) -> None:
576582
"""Share recordings via Magic Wormhole.
577583
584+
``send`` scrubs a copy of the recording and sends only the copy. It needs
585+
the ``privacy`` extra and refuses to send without it.
586+
578587
Args:
579588
action: Either "send" or "receive".
580589
path_or_code: Recording path (for send) or wormhole code (for receive).
581590
output_dir: Output directory for receive (default: current dir).
591+
unscrubbed: Send the raw recording without scrubbing (needs --reason;
592+
the choice is recorded).
593+
reason: Why the raw recording must be sent.
582594
583595
Examples:
584596
capture share send ./my_recording
597+
capture share send ./my_recording --unscrubbed --reason "synthetic demo"
585598
capture share receive 7-guitarist-revenge
586599
capture share receive 7-guitarist-revenge ./recordings
587600
"""
588601
from openadapt_capture.share import receive, send
589602

590603
if action == "send":
591-
send(path_or_code)
604+
send(path_or_code, unscrubbed=unscrubbed, reason=reason)
592605
elif action == "receive":
593606
receive(path_or_code, output_dir)
594607
else:

‎packages/capture/openadapt_capture/recorder.py‎

Lines changed: 33 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -218,16 +218,48 @@ def _send_profiling_via_wormhole(profile_path: str, timeout: int = 60) -> None:
218218
print(f"Profiling saved to: {profile_path}")
219219
return
220220

221+
# The profile holds timings plus a few strings (the capture folder and the
222+
# window title filter) that can name a patient. Only numbers and flags
223+
# leave the computer; every string is withheld.
224+
import json as _json
225+
import shutil as _shutil
226+
import tempfile as _tempfile
227+
228+
try:
229+
with open(profile_path, encoding="utf-8") as _f:
230+
minimal = _numbers_only(_json.load(_f))
231+
except (OSError, ValueError) as exc:
232+
print(f"Profiling not sent: could not read {profile_path} ({exc})")
233+
return
234+
send_dir = _tempfile.mkdtemp(prefix="capture-profile-")
235+
send_path = os.path.join(send_dir, "profiling.json")
236+
with open(send_path, "w", encoding="utf-8") as _f:
237+
_json.dump(minimal, _f, indent=2, sort_keys=True)
238+
221239
print(f"Sending profiling via wormhole (waiting up to {timeout}s for receiver)...")
240+
print("Strings (folder names, window titles) are withheld; numbers only.")
222241
print("Give the wormhole code below to the receiver.\n")
223242
try:
224-
_sp.run([wormhole_bin, "send", profile_path], check=True, timeout=timeout)
243+
_sp.run([wormhole_bin, "send", send_path], check=True, timeout=timeout)
225244
except _sp.TimeoutExpired:
226245
logger.warning(f"Wormhole send timed out after {timeout}s. File at: {profile_path}")
227246
except _sp.CalledProcessError:
228247
print(f"Wormhole send failed. File at: {profile_path}")
229248
except KeyboardInterrupt:
230249
print(f"\nCancelled. File at: {profile_path}")
250+
finally:
251+
_shutil.rmtree(send_dir, ignore_errors=True)
252+
253+
254+
def _numbers_only(value):
255+
"""Return ``value`` with every string replaced by ``"<withheld>"``."""
256+
if isinstance(value, dict):
257+
return {str(k): _numbers_only(v) for k, v in value.items()}
258+
if isinstance(value, list):
259+
return [_numbers_only(v) for v in value]
260+
if isinstance(value, str):
261+
return "<withheld>"
262+
return value
231263

232264

233265
Event = namedtuple(

‎packages/capture/openadapt_capture/share.py‎

Lines changed: 116 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,12 @@
11
"""Share recordings between computers using Magic Wormhole.
22
3+
``send`` scrubs a copy of the recording first and sends only the copy (see
4+
:mod:`openadapt_capture.share_scrub`). Sending the raw recording needs
5+
``--unscrubbed --reason ...``, and that choice is recorded.
6+
37
Usage:
48
capture share send ./my_recording
9+
capture share send ./my_recording --unscrubbed --reason "synthetic demo"
510
capture share receive 7-guitarist-revenge
611
"""
712

@@ -79,15 +84,82 @@ def _ensure_wormhole() -> str | None:
7984
return _install_wormhole()
8085

8186

82-
def send(recording_dir: str) -> str | None:
87+
OPT_OUT_LEDGER = "share-unscrubbed.jsonl"
88+
OPT_OUT_MARKER = "UNSCRUBBED_SHARE.json"
89+
90+
91+
def opt_out_ledger_path() -> Path:
92+
"""Local file where every unscrubbed share is recorded."""
93+
from openadapt_capture.ffmpeg_runtime import data_root
94+
95+
return data_root() / OPT_OUT_LEDGER
96+
97+
98+
def _record_opt_out(recording_path: Path, reason: str) -> dict:
99+
"""Append an unscrubbed-share record to the local ledger and return it."""
100+
import getpass
101+
import json
102+
import time
103+
104+
files = [f for f in recording_path.rglob("*") if f.is_file()]
105+
try:
106+
user = getpass.getuser()
107+
except Exception: # noqa: BLE001 - no login name in some sandboxes
108+
user = None
109+
record = {
110+
"schema": "openadapt.capture.share-unscrubbed/v1",
111+
"recorded_at": time.time(),
112+
"user": user,
113+
"recording": str(recording_path.resolve()),
114+
"file_count": len(files),
115+
"size_bytes": sum(f.stat().st_size for f in files),
116+
"reason": reason,
117+
}
118+
ledger = opt_out_ledger_path()
119+
ledger.parent.mkdir(parents=True, exist_ok=True)
120+
with ledger.open("a", encoding="utf-8") as handle:
121+
handle.write(json.dumps(record, sort_keys=True) + "\n")
122+
return record
123+
124+
125+
def _zip_folder(folder: Path, zip_path: Path, extra: dict | None = None) -> None:
126+
import json
127+
128+
with ZipFile(zip_path, "w", ZIP_DEFLATED, compresslevel=6) as zf:
129+
for file in folder.rglob("*"):
130+
if file.is_file() and not file.is_symlink():
131+
zf.write(file, file.relative_to(folder.parent))
132+
if extra is not None:
133+
zf.writestr(
134+
f"{folder.name}/{OPT_OUT_MARKER}",
135+
json.dumps(extra, indent=2, sort_keys=True),
136+
)
137+
138+
139+
def send(
140+
recording_dir: str,
141+
unscrubbed: bool = False,
142+
reason: str | None = None,
143+
) -> str | None:
83144
"""Send a recording via Magic Wormhole.
84145
146+
By default the recording is scrubbed first: a scrubbed copy is built in a
147+
temporary folder and only that copy is sent. Scrubbing needs the
148+
``privacy`` extra; without it nothing is sent.
149+
85150
Args:
86151
recording_dir: Path to the recording directory.
152+
unscrubbed: Send the raw recording without scrubbing. Requires
153+
``reason``. The choice is written to the local ledger and to an
154+
``UNSCRUBBED_SHARE.json`` file inside the archive.
155+
reason: Why the raw recording must be sent (required with
156+
``unscrubbed``).
87157
88158
Returns:
89-
The wormhole code if successful, None otherwise.
159+
``"sent"`` if the transfer finished, None otherwise.
90160
"""
161+
from openadapt_capture.share_scrub import ShareScrubError, scrub_recording
162+
91163
recording_path = Path(recording_dir)
92164

93165
if not recording_path.exists():
@@ -98,22 +170,38 @@ def send(recording_dir: str) -> str | None:
98170
print(f"✗ Not a directory: {recording_path}")
99171
return None
100172

173+
if unscrubbed and not (reason and str(reason).strip()):
174+
print("✗ --unscrubbed needs --reason explaining why the raw recording")
175+
print(" must leave this computer. The reason is recorded.")
176+
return None
177+
178+
# Check the transfer tool first so a scrub is not wasted.
101179
wormhole_path = _ensure_wormhole()
102180
if not wormhole_path:
103181
return None
104182

105-
# Create a temporary zip file
106-
zip_name = f"{recording_path.name}.zip"
107-
108183
with tempfile.TemporaryDirectory() as tmpdir:
109-
zip_path = Path(tmpdir) / zip_name
184+
tmp = Path(tmpdir)
185+
opt_out: dict | None = None
186+
if unscrubbed:
187+
folder = recording_path
188+
opt_out = _record_opt_out(recording_path, str(reason).strip())
189+
print("! Sending the RAW recording without scrubbing.")
190+
print(f" Recorded in {opt_out_ledger_path()}")
191+
else:
192+
print(f"Scrubbing a copy of {recording_path.name}...")
193+
try:
194+
folder = scrub_recording(recording_path, tmp / "scrubbed")
195+
except ShareScrubError as exc:
196+
print(f"✗ {exc}")
197+
print(" Nothing was sent. To send the raw recording anyway, run")
198+
print(" capture share send PATH --unscrubbed --reason '...'")
199+
return None
200+
_print_scrub_summary(folder)
110201

111-
print(f"Compressing {recording_path.name}...")
112-
with ZipFile(zip_path, "w", ZIP_DEFLATED, compresslevel=6) as zf:
113-
for file in recording_path.rglob("*"):
114-
if file.is_file():
115-
arcname = file.relative_to(recording_path.parent)
116-
zf.write(file, arcname)
202+
zip_path = tmp / f"{folder.name}.zip"
203+
print(f"Compressing {folder.name}...")
204+
_zip_folder(folder, zip_path, extra=opt_out)
117205

118206
size_mb = zip_path.stat().st_size / (1024 * 1024)
119207
print(f"Compressed to {size_mb:.1f} MB")
@@ -140,6 +228,22 @@ def send(recording_dir: str) -> str | None:
140228
return None
141229

142230

231+
def _print_scrub_summary(folder: Path) -> None:
232+
import json
233+
234+
from openadapt_capture.share_scrub import MANIFEST_NAME
235+
236+
manifest = json.loads((folder / MANIFEST_NAME).read_text(encoding="utf-8"))
237+
counts = manifest["counts"]
238+
print(
239+
f"Scrubbed {counts['text_values']} text values and "
240+
f"{counts['images']} images."
241+
)
242+
for item in manifest["withheld"]:
243+
print(f" withheld {item['path']} ({item['reason']})")
244+
print("Scrubbing is automatic and can miss data. Review before sharing further.")
245+
246+
143247
def receive(code: str, output_dir: str = ".") -> Path | None:
144248
"""Receive a recording via Magic Wormhole.
145249

0 commit comments

Comments
 (0)