Enhance sysdiagnoseProcess.py for a process tree using ps.txt on Sysdiagnose

[mathisdesaulty]

What this changes

Adds a new artifact module, sysdiagnoseProcess, that parses ps.txt and ps_thread.txt files from iOS sysdiagnose extractions and renders the process list as a clean, theme-aware parent/child hierarchy tree in the HTML report. It includes user-type badges (root, mobile, etc.) for better readability.

For a new or changed artifact

• Ran the tool against a real extraction and confirmed the row counts, not just that it imports.
• Ran python admin/scripts/check_artifact_output.py <report folder> on that report and fixed or documented every finding.
• Checked it against a second app data directory where the platform provides one (--compare <multi-container report>). (N/A: system-wide sysdiagnose logs, not app-specific containers)
• notes, description and sample_data say only what the data shows, and the numbers were re-derived from the finished run.

Anything reviewers should know

• Tested and verified against multiple distinct sysdiagnose extractions across different iOS versions (e.g., builds 20A362 and 19H349 (https://github.com/EC-DIGIT-CSIRC/sysdiagnose-testdata)).
• output_types is restricted to "html" only — since there is no standard timestamp column in ps/ps_thread output, this artifact deliberately avoids feeding timeline/TSV/KML outputs.
• Fully compatible with iLEAPP's light/dark theme toggle via standard Bootstrap/MDB classes (badge-*, text-muted, <pre class='mb-0'>).

---

Maintainer note (2026-09-20). The description above tracks an earlier draft. What merges renders the hierarchy as a PNG rather than HTML badges, and output_types is ["html", "tsv", "lava"] rather than html only. The image is drawn on a fixed dark background, so it does not follow the report's light or dark setting.

One commit was added on merge:

• %CPU, %MEM and TIME are read but no longer reported. Each held a single value on all 1,668 rows of the four captures tested: 0.0, 0.0 and 0:00.00 on iOS 16 20A362, iOS 17.3 21D50, iOS 26 23G71 and iOS 26.5.2 23F84.
• The ps_thread.txt note now gives the measured reason. Its columns sit in a different order from ps.txt, and its header is byte-identical across the three sysdiagnose captures tested, each carrying no data rows.
• The tree artifact pointed at "Sysdiagnose Process List", which names no artifact. The sibling is "Sysdiagnose Process".
• sample_data now cites corpora in the project registry, with counts from real runs: 407, 546 and 382 rows on iOS 17.3, 26 and 26.5.2, and 0 on an image that carries no sysdiagnose.

Thanks for the module, and for the test data.

[github-actions]

Test data is now included for every changed artifact module. Thank you!

[abrignoni]

Thank you for the contribution, @mathisdesaulty! A process tree from a sysdiagnose is a useful addition. I checked the tree against ps.txt on your test data and on three other sysdiagnoses (one iOS 17.3, two iOS 26), and every process sits under the right parent with the right user and command.

A few things need to change before this can be merged:

1. LAVA support is mandatory. Right now output_types is "html" only, so nothing reaches LAVA or the TSV export. The tree can't simply be switched on for LAVA either: it is HTML markup inside a single cell, and LAVA shows cell values as plain text, so the markup would appear as text. Please change the artifact to one row per process with plain columns, for example PID, Parent PID, User and Command, plus any other ps.txt columns you want to keep as stored (UID, %CPU, %MEM, STAT, STARTED, TIME). If you keep STARTED, report it as text. Values such as 1:25PM carry no date, and none of them carry a time zone, so they are not full timestamps. "output_types": ["html", "tsv", "lava"] fits this artifact.

2. Drop ps_thread.txt. It only adds an empty row. In this PR's iOS 15 test data its columns are in a different order (the fourth column is %CPU, not PID), so every line is skipped. On the iOS 17 and iOS 26 sysdiagnoses I tested, it holds only the header line. The description should then name only ps.txt.

3. Name the file each row came from. source_file is overwritten on every pass of the loop, so the report's "located at" line shows ps_thread.txt, the file that produced nothing, instead of ps.txt. And when the input holds more than one sysdiagnose, each tree lands in its own row with nothing saying which capture it came from. Please return every file you read, joined with newlines, and add a source column built with context.get_relative_path(). sysdiagnose.py does both.

4. Read packed sysdiagnoses too. A full file system extraction can hold sysdiagnoses packed as sysdiagnose_*.tar.gz, and this artifact doesn't open those. On an extraction I tested that holds two of them, it reported "No file found". get_sysdiagnose_files() in scripts/ilapfuncs.py reads a named file both loose and from inside those archives, and sysdiagnose.py uses it with '*/sysdiagnose_*.tar.gz' in its paths.

5. Two CI checks fail. I ran them locally on this commit:

   • check_html_safety.py flags line 69, where the badge class goes into the markup unescaped. Plain rows remove this.
   • lint_changed.py reports 10 new pylint warnings. The one-argument signature, def sysdiagnoseProcess(context): with context.get_files_found(), removes the four unused-argument warnings. Moving the helper functions out of the loop removes the five cell-var-from-loop warnings. The last one is the except Exception; please catch the specific errors you expect.

6. Smaller items:

   • notes is empty and there is no sample_data. python3 admin/scripts/validate_sample_data.py --emit <your sysdiagnose> --modules sysdiagnoseProcess prints paste-ready sample_data values. It reads its counts from the LAVA output, so run it after the LAVA change.
   • Please run check_artifact_output.py again after the change. It reads the TSV export, so it could not see this artifact while it was HTML only.
   • Re-record the test baseline with python admin/test/scripts/test_module.py sysdiagnoseProcess -a all -c all and keep only the newest snapshot. There are three now: the oldest (333 processes) doesn't match the committed test zip (244 processes), and the other two have the same rows.
   • The log message in the except block is still in French.

A suggestion, not a requirement: I think the tree view is worth keeping, and it could be its own artifact that draws the tree as an image. springboard.py already does this for the home screen: "iOS Home Screen Layout" holds the data as a table, and "iOS Home Screen Layout - Visual" renders a PNG with Pillow and checks it in with check_in_embedded_media() in a media column, so the picture reaches both the HTML report and LAVA. A "Sysdiagnose Process" table plus a "Sysdiagnose Process - Tree" image would follow the same pattern. If you do that, neutral colors per user would work better than red and yellow, which read as warnings.

Thanks again for the work on this!

[mathisdesaulty]

Thank you so much for the feedback. I tried to follow your advice; I have one page with a complete table and a second page containing images of all the processes to make it more visual.
I hope you like it—I also tried to include all of your feedback!
I've changed the dataset because of one column where the value was only "0.0".
I'm very interested in this project and hope to be able to contribute to it!