Repository navigation
fix(e2e): harden nightly legs against transport-level failures - #5129
Merged
Merged
Conversation
mziolkowskii
force-pushed
the
fix/e2e-nightly-flakes
branch
from
September 22, 2026 06:53
3b2fcb9 to
d96011f
Compare
Nightly 2026-09-21 and 09-22 (iOS 26, RN 0.87) failed smoke-mute-volume at `evt-muted`: Maestro reported the btn-mute tap done 2.4 s after onEnded, the simulator log shows touch-down and touch-up delivered to the app's window, and onPress never ran, so `muted = true` was never called. Same symptom as the tap lost 264 ms after onEnded on 09-15, so the settle in wait-for-end.yaml is not a complete answer. The other iOS legs were 12/12 green those nights. Root cause not established: #5130. The test app now counts presses per control and renders one marker per press, `pressed-<testID>-<n>`, in the fixed marker row above the scrolling log, so a flow can prove a specific tap arrived on an element that is never off-screen. e2e/shared/press.yaml taps a control and waits for that marker, re-sending the tap (retry, 2 max) only when it never appeared. The marker is set synchronously in onPress before the player is touched, so the retry covers tap transport and nothing else; every player marker is still asserted once, and every control sets an absolute value, so a second delivery is harmless. The five flows that press on an idle player (after wait-for-end.yaml) use it; taps issued during playback stay bare, because iOS delivers those only once the player is idle, later than the subflow waits.
… no file Nightly 2026-09-19 (RN 0.82, API 35) failed before Maestro started: 3 s after install `uiautomator dump` returned 0 without writing /sdcard/settle.xml (the guest was still busy; adb had reported "device offline" moments earlier), and the unguarded `adb shell cat` under `set -e` killed the script. The retry loop assumed only the dump command itself could fail. Each attempt is now one guest shell (rm, dump, cat) and the file, not the exit status, decides whether it produced a dump; a missing one is retried. Only a dialog still on screen in the last dump fails the leg: no dump at all, or a dismissed dialog followed by dump-less attempts, ends in a warning and exit 0, because there is nothing to fail on before a single flow has run and Maestro's own driver start-up is the next check of the guest. android-settle.test.mjs runs the script against a fake `adb` and `sleep` on PATH (bun test), including the dump-less and dismissed-then-dump-less cases.
mziolkowskii
force-pushed
the
fix/e2e-nightly-flakes
branch
from
September 22, 2026 07:17
d96011f to
f24723d
Compare
…rror Nightly 2026-09-20 (RN 0.82, API 35): Gradle could not download gson-2.9.1.jar for the build classpath, Maven Central answered 403, and the build failed after 8 s. Gradle retries 5xx and timeouts itself, not 403. `assembleDebug` gets one more attempt after 30 s, only when its log shows a 403, 429 or 5xx from a repository; a compile error or a missing artifact (404) still fails once.
mziolkowskii
force-pushed
the
fix/e2e-nightly-flakes
branch
from
September 22, 2026 07:20
f24723d to
d4249ca
Compare
fnwk
approved these changes
Sep 22, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The first four nightly E2E runs from
master(2026-09-19 → 09-22) each had one or two red legs, for four unrelated reasons, none of them in the library. This PR removes three of those failure modes at the layer they live in, without retrying anything the flows assert, and documents the fourth.android-settle.sh:uiautomator dumpreturned 0 without writing the file; theadb shell catunderset -ekilled the script before Maestro startedgson-2.9.1.jarwhile resolving the build classpathsmoke-mute-volume: thebtn-mutetap never reached JS, soevt-mutednever appearedDeviceServerDiedException ... UNAVAILABLEon the first command of the first flow, 3 s after the driver came up; the other nine flows passedMotivation
Nightly is meant to become a required check after 10–15 consecutive clean runs. Each of these failures would reset that streak while telling us nothing about the library.
For the iOS one, the simulator
device.logfrom both failing nights shows UIKit delivering touch-down and touch-up to the app's window, followed by noonPress. In both failures the touch-down was followed by ~120 ms without touch updates, then a UIKit system-gesture state change, then the touch-up; in every passing run of that flow the state change came after the touch-up. The tap went 2.4 s afteronEnded, so the settle added inwait-for-end.yamlafter the 09-15 incident is not a complete answer. Evidence and open questions are in #5130.The Android driver death gets no mitigation here. A warm-up
maestro testbefore the suite (tried in an earlier revision of this PR) does not help: everymaestro testprocess installs, connects and uninstalls its own driver, so the suite's first connection is still a first connection. A transport-level retry of flows that die in the driver is a possible follow-up.Changes
eventLog.press()counts presses per control andEventLogPanelrenders onepressed-<testID>-<n>marker per press in the fixed marker row, above the scrolling log, so a flow can wait for a specific tap on an element that is never off-screen. Unit tests added.e2e/shared/press.yaml(new): taps a control and waits for that marker, re-sending the tap (max 2) only if it never appeared. The marker is set synchronously inonPressbefore the player is touched, so the retry covers tap transport only; every player marker is still asserted exactly once, and the controls set absolute values, so a second delivery is harmless. Used by the five flows that press on an idle player (smoke-mute-volume,smoke-seek,smoke-hls-seek,smoke-loop,smoke-replay-after-end); taps issued during playback stay baretapOn, since iOS delivers those only once the player is idle.scripts/e2e/android-settle.sh: one guest shell per attempt; the dump file, not the exit status, decides whether an attempt produced a dump; only a dialog still on screen in the last dump fails the leg. Newandroid-settle.test.mjs(fakeadb/sleepon PATH), 6 cases including "dialog dismissed, then no dumps"..github/workflows/_e2e-android.yml:assembleDebuggets one more attempt after 30 s only when its log shows a 403/429/5xx from a repository; a compile error or a 404 still fails once.e2e/CONTEXT.md(lost-tap gotcha linking E2E: iOS 26 tap on an idle player reaches the app window but never fires onPress #5130, the second transport-only retry, why a driver warm-up does not work),e2e/README.md.Platforms affected
Type of change
Test plan
test-app/src/e2e/eventLog.test.ts(press counting, snapshot identity, reset),scripts/e2e/android-settle.test.mjs(6 cases).bun run test197 pass,bun lintandbun typecheckclean.retryattempt in the flow'scommands.jsonin the Maestro debug artifact, so the rate of lost taps stays observable.Checklist
bun run test,bun lintandbun typecheckpass locally.skills/react-native-video/) to match. (Not applicable: no library change.)