RESOLVED FIXED276081
run-webkit-tests failing doesn't fail EWS
https://bugs.webkit.org/show_bug.cgi?id=276081
Summary run-webkit-tests failing doesn't fail EWS
Sam Sneddon [:gsnedders]
Reported 2024-07-01 11:09:05 PDT
c.f. https://github.com/WebKit/WebKit/pull/30272#issuecomment-2198130465 https://ews-build.webkit.org/#/builders/42/builds/30643 shows layout-tests passing, despite the logs containing: webkitpy.layout_tests.servers.http_server_base.ServerError: WPT Server process exited prematurely with status code 1 We should instead be failing the EWS test job as run-webkit-tests should be exiting with non-zero.
Attachments
Radar WebKit Bug Importer
Comment 1 2024-07-01 11:09:18 PDT
Sam Sneddon [:gsnedders]
Comment 2 2024-07-01 11:34:35 PDT
Looking at the code run-webkit-tests should definitely be exiting with non-zero. However, the problem here might be the fact that we have logged: /bin/sh -c 'python3 Tools/Scripts/run-webkit-tests --no-build --no-show-results --no-new-test-results --clobber-old-results --release --ios-simulator --results-directory layout-test-results --debug-rwt-logging --exit-after-n-failures 60 --skip-failing-tests --child-processes=5 imported/w3c/web-platform-tests 2>&1 | Tools/Scripts/filter-test-logs layout' in dir /Volumes/Data/worker/iOS-17-Simulator-WPT-WK2-Tests-EWS/build (timeout 19800.0 secs) watching logfiles {'json': 'layout-test-results/full_results.json'} argv: [b'/bin/sh', b'-c', b'python3 Tools/Scripts/run-webkit-tests --no-build --no-show-results --no-new-test-results --clobber-old-results --release --ios-simulator --results-directory layout-test-results --debug-rwt-logging --exit-after-n-failures 60 --skip-failing-tests --child-processes=5 imported/w3c/web-platform-tests 2>&1 | Tools/Scripts/filter-test-logs layout'] using PTY: False program finished with exit code 0 elapsedTime=25.727919 Because we're running a sub-shell, we're actually just getting the exit code of `sh`, and running without `-e`. This makes this seem like potentially a regression from 279663@main, bug 270651, "Upload layout-test logs to S3".
Sam Sneddon [:gsnedders]
Comment 3 2024-07-02 18:58:13 PDT
Sam Sneddon [:gsnedders]
Comment 4 2024-08-05 16:28:14 PDT
I'd held off from landing this because I was a bit concerned that Bug 277654 would mean this wouldn't have actually fixed the original bug, but it would, so I guess we land that and treat that as a separate issue.
EWS
Comment 5 2024-08-05 18:56:00 PDT
Committed 281869@main (44f912123399): <https://commits.webkit.org/281869@main> Reviewed commits have been landed. Closing PR #30424 and removing active labels.
Elliott Williams
Comment 6 2024-08-29 14:50:24 PDT
*** Bug 277654 has been marked as a duplicate of this bug. ***
Ryan Haddad
Comment 7 2024-11-22 15:54:43 PST
This change was reverted, reopening to reflect that.
Ryan Haddad
Comment 8 2024-12-16 08:40:36 PST
*** Bug 284659 has been marked as a duplicate of this bug. ***
Sam Sneddon [:gsnedders]
Comment 9 2025-04-07 17:04:44 PDT
(In reply to Angelos Oikonomopoulos from comment #4 on bug 277749) > We got bitten by this again, with `run-jsc-stress-tests` throwing an > exception and `filter-test-logs` accidentally eating it up, presumably > because it relied on `-o pipefail`. I've asked internally at Igalia and all > our buildbots should have `bash` in the PATH. For the post-commit bots could > we change `remote_command` in `build-webkit-org/steps.py` to use `bash`, > `-c` instead of `/bin/sh`, `-c`? Would that work on other platforms? This is probably an acceptable option for now; we probably should longer term try to move back to `sh`, given shells are slowly adding it after its addition in POSIX 2024.
Angelos Oikonomopoulos
Comment 10 2025-04-08 00:49:35 PDT
Yah, `-o pipefail` is going to be in the version of dash included in the next debian release (to be released in a few months), but I'm not sure when we'll be able to move to it. I don't think we want to wait that long to introduce `-o pipefail` anyway, so perhaps we can re-land the above commit with the command set to `bash`?
Sam Sneddon [:gsnedders]
Comment 11 2025-04-14 17:04:37 PDT
EWS
Comment 12 2025-05-01 11:54:24 PDT
Committed 294384@main (afddd6334cd1): <https://commits.webkit.org/294384@main> Reviewed commits have been landed. Closing PR #44077 and removing active labels.
Note You need to log in before you can comment on or make changes to this bug.