Make browser automation work on real ATS forms (upload, custom selects, stable refs) - #113
Merged
Merged
Conversation
…s, stable refs) Dogfooding `tron automate` on Greenhouse and Ashby job applications hit seven problems; each one broke form filling: - No way to set a file input. Add browser_upload (MCP), `tron upload` (CLI) and Page.upload, via DOM.setFileInputFiles. Snapshots now list file inputs as role "file" even when an Attach button hides them. - browser_select only handled native <select>. It now also drives custom comboboxes (react-select, ARIA listboxes) with trusted CDP mouse and text input, then clicks the matching [role=option]. [role=option]/[role=combobox] are collectable in snapshots. - browser_press dispatched untrusted KeyboardEvents that widgets ignore. It now uses Input.dispatchKeyEvent. Adds `tron press`. - Every snapshot renumbered all refs, and every mutating tool takes one, so opening a dropdown shifted later refs and the next fill hit the wrong field. Refs are now stable per element; new elements get new numbers and a vanished element's ref is never reused. - scrollIntoView under scroll-behavior:smooth measured mid-animation, so trusted clicks landed off-screen. Scroll instantly. - extract text used body.textContent: script source filled the 20k cap before visible words, hiding "Thanks for applying" on Ashby. Use rendered text. - Snapshots carry `required` (attribute or aria-required), since ATS forms mark it outside the label; fill on a file input now says to upload instead of throwing InvalidStateError. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
|
|
||
| describe('text extraction', () => { | ||
| it('returns visible words, not <script>/<style> source that would eat the 20k cap', () => { | ||
| document.body.innerHTML = `<script>${'var bundle = 1;'.repeat(3000)}</script><style>.x{color:red}</style><main><h1>Thanks for applying</h1></main>`; |
Contributor
ThreatCrush Security Scan52 finding(s) HIGH/CRITICAL: 3 | MEDIUM: 28 | LOW: 21
…and 2 more. Full results in the Security tab. Snippets are redacted; ThreatCrush never prints matched credential material. |
ralyodio
marked this pull request as ready for review
September 24, 2026 07:05
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Found by driving
tron automate(MCP) through real job applications on Greenhouse and Ashby. With this branch the same MCP session filled and submitted Greenhouse forms end to end: resume upload, react-select dropdowns, location autocomplete, checkbox groups, and the emailed security-code resubmit. Before it, none of those forms could be completed.Fixes
browser_upload(MCP),tron upload <ref> <file...>,Page.upload, viaDOM.setFileInputFiles; snapshots list file inputs as rolefileeven when visually hiddenbrowser_selectonly handled native<select>Input.insertText, then click the matching[role=option]; options and comboboxes are now collectablebrowser_presssent untrustedKeyboardEventsInput.dispatchKeyEvent; addstron pressscrollIntoViewunderscroll-behavior: smoothmeasured mid-animation, so clicks landed off-screenbehavior: 'instant'extract textusedtextContent, so script source used up the 20k cap and "Thanks for applying" was never seen on AshbyinnerText, script/style-stripped fallback)fillon a file input threwInvalidStateErrorrequiredin snapshots ([required]in text); a clear "use upload" errorVerification
pnpm -r testexit 0 (browser-core 98, sdk 51 + 1 skipped, new tests for each fix);pnpm -r typecheckand lint clean;sh -non install.sh and the extractedtronCLI heredoc.tron automateover stdio MCP, including the security-code resubmit.Not in this PR
libatk*/libatspi/libXcomposite/libXdamage/libXRes, the bundled engine cannot start. The installer only warns about AppArmor, not the missing libs. I worked around it with a user-space lib dir +TRONBROWSER_BROWSERwrapper; the installer could detect this.tron uploadneeds theinstall.shdeploy (CLI dispatch) plus a tagged release (the automate runtime) to reach users.🤖 Generated with Claude Code