Bug 174443

Summary: Add BenchmarkRunner patches with WebDriver support for each testing plan
Product: WebKit Reporter: Matthew Stewart <matthew_r_stewart>
Component: New BugsAssignee: Nobody <webkit-unassigned>
Status: RESOLVED FIXED    
Severity: Normal CC: buildbot, commit-queue, dewei_zhu, glenn, slewis
Priority: P2    
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   
Bug Depends on: 174390    
Bug Blocks: 174056    
Attachments:
Description Flags
Patch
buildbot: commit-queue-
Archive of layout-test-results from ews122 for ios-simulator-wk2
none
Add BenchmarkRunner patches with WebDriver support for each testing plan
slewis: review+, commit-queue: commit-queue-
patch none

Matthew Stewart
Reported 2017-07-12 16:46:28 PDT
Add BenchmarkRunner patches with WebDriver support for each testing plan
Attachments
Patch (2.30 MB, patch)
2017-07-12 16:51 PDT, Matthew Stewart 		buildbot: commit-queue-
DetailsFormatted DiffDiff
Archive of layout-test-results from ews122 for ios-simulator-wk2 (16.15 MB, application/zip)
2017-07-12 18:36 PDT, Build Bot 		no flags
Details
Add BenchmarkRunner patches with WebDriver support for each testing plan (788.10 KB, patch)
2017-07-18 19:35 PDT, Matthew Stewart 		slewis: review+
commit-queue: commit-queue-
DetailsFormatted DiffDiff
patch (788.12 KB, patch)
2017-07-24 18:01 PDT, Matthew Stewart 		no flags
DetailsFormatted DiffDiff
Show Obsolete (2) View All Add attachment proposed patch, testcase, etc.
Matthew Stewart
Comment 1 2017-07-12 16:51:04 PDT
Created attachment 315308 [details] Patch
Build Bot
Comment 2 2017-07-12 16:53:40 PDT
Attachment 315308 [details] did not pass style-queue: ERROR: Tools/Scripts/webkitpy/benchmark_runner/benchmark_builder.py:39: [BenchmarkBuilder.__enter__] Instance of 'BenchmarkBuilder' has no '_driver' member [pylint/E1101] [5] Total errors found: 1 in 35 files If any of these errors are false positives, please file a bug against check-webkit-style.
Stephanie Lewis
Comment 3 2017-07-12 17:18:24 PDT
Comment on attachment 315308 [details] Patch This looks rubber stampable to me but I'd like Dewei to weigh in.
Build Bot
Comment 4 2017-07-12 18:36:03 PDT
Comment on attachment 315308 [details] Patch Attachment 315308 [details] did not pass ios-sim-ews (ios-simulator-wk2): Output: http://webkit-queues.webkit.org/results/4110662 New failing tests: storage/websql/execute-sql-rowsAffected.html
Build Bot
Comment 5 2017-07-12 18:36:05 PDT
Created attachment 315325 [details] Archive of layout-test-results from ews122 for ios-simulator-wk2 The attached test failures were seen while running run-webkit-tests on the ios-sim-ews. Bot: ews122 Port: ios-simulator-wk2 Platform: Mac OS X 10.12.5
dewei_zhu
Comment 6 2017-07-17 17:38:21 PDT
Comment on attachment 315308 [details] Patch View in context: https://bugs.webkit.org/attachment.cgi?id=315308&action=review > Tools/Scripts/webkitpy/benchmark_runner/benchmark_builder.py:44 > + if self._driver == 'websocket': > + if 'websocket_benchmark_patch' in self._plan: > + self._apply_patch(self._plan['websocket_benchmark_patch']) > + else: > + if 'webdriver_benchmark_patch' in self._plan: > + self._apply_patch(self._plan['webdriver_benchmark_patch']) It would be clearer: patch_file_key = "{driver_name}_benchmark_patch".format(driver_name=self._driver) if patch_file_key in self._plan: self._apply_patch(self._plan[patch_file_key])
Matthew Stewart
Comment 7 2017-07-18 19:35:37 PDT
Created attachment 315878 [details] Add BenchmarkRunner patches with WebDriver support for each testing plan
dewei_zhu
Comment 8 2017-07-18 20:57:37 PDT
Comment on attachment 315878 [details] Add BenchmarkRunner patches with WebDriver support for each testing plan View in context: https://bugs.webkit.org/attachment.cgi?id=315878&action=review > Tools/ChangeLog:6 > + Reviewed by Dewei Zhu. r=me
WebKit Commit Bot
Comment 9 2017-07-24 17:16:06 PDT
Comment on attachment 315878 [details] Add BenchmarkRunner patches with WebDriver support for each testing plan Rejecting attachment 315878 [details] from commit-queue. Failed to run "['/Volumes/Data/EWS/WebKit/Tools/Scripts/webkit-patch', '--status-host=webkit-queues.webkit.org', '--bot-id=webkit-cq-01', 'validate-changelog', '--check-oops', '--non-interactive', 315878, '--port=mac']" exit_code: 1 cwd: /Volumes/Data/EWS/WebKit Dewei Zhu found in /Volumes/Data/EWS/WebKit/Tools/ChangeLog does not appear to be a valid reviewer according to contributors.json. /Volumes/Data/EWS/WebKit/Tools/ChangeLog neither lists a valid reviewer nor contains the string "Unreviewed" or "Rubber stamp" (case insensitive). Full output: http://webkit-queues.webkit.org/results/4181291
Matthew Stewart
Comment 10 2017-07-24 18:01:54 PDT
Created attachment 316335 [details] patch
WebKit Commit Bot
Comment 11 2017-07-24 18:57:07 PDT
The commit-queue encountered the following flaky tests while processing attachment 316335 [details]: http/tests/security/cross-origin-xsl-BLOCKED.html bug 51054 (authors: abarth@webkit.org, jochen@chromium.org, and rniwa@webkit.org) The commit-queue is continuing to process your patch.
WebKit Commit Bot
Comment 12 2017-07-24 18:58:32 PDT
Comment on attachment 316335 [details] patch Clearing flags on attachment: 316335 Committed r219854: <http://trac.webkit.org/changeset/219854>
WebKit Commit Bot
Comment 13 2017-07-24 18:58:34 PDT
All reviewed patches have been landed. Closing bug.
Note You need to log in before you can comment on or make changes to this bug.