Bug 174201

Created attachment 314720 [details] Patch

Created attachment 314721 [details] Patch

Created attachment 314725 [details] Patch

Should we skip style checks on `PerformanceTests/Speedometer/resources/todomvc/`?

> Speedometer 2.0 should use production builds of frameworks and libraries whenever possible. We might want to file separate bugs for each app-specific change we're making here and treat 174201 as a meta-bug that is blocked on them landing. A similar approach was suggested by rniwa@ when we landed Speedometer 2 initially. I'll defer to him for the call here however. > Should we skip style checks on `PerformanceTests/Speedometer/resources/todomvc/`? I believe rniwa@'s preference was the contents of Speedometer/resources matching the current style guide but will defer to him again for whether this is something that is possible. I applied a tab->space transform when working on the apps previously.

This patch is way too big to review. Please split per framework. I also don't agree with the premise that we should be always using production libraries. For each library and framework, we need to carefully evaluate whether it makes sense to use the production version or not. Remember, what we're trying to do is to measure the performance of browser, not production websites.

Comment on attachment 314725 [details] Patch r- because the patch is way too big to review, and we ned to understand the implication of changes to each libraries & framework separately.

(In reply to Ryosuke Niwa from comment #6) > Remember, what we're trying to do is to measure the performance of browser, > not production websites. Addy asked me for feedback on this issue as I gave input the initial v2 changes and as the Spidermonkey team started looking at Speedometer v2. I understand it is hard to clearly separate Speedometer from the frameworks it tests from the real world. Speedometer was announced [1] as "a benchmark that measures the responsiveness of real-world web applications". While best practices could differ between frameworks (I could not find any examples), the default premise should be that real-world web apps use production builds. [1]: https://webkit.org/blog/3395/speedometer-benchmark-for-web-app-responsiveness/

(In reply to Harald Kirschner from comment #8) > (In reply to Ryosuke Niwa from comment #6) > > Remember, what we're trying to do is to measure the performance of browser, > > not production websites. > > Addy asked me for feedback on this issue as I gave input the initial v2 > changes and as the Spidermonkey team started looking at Speedometer v2. > > I understand it is hard to clearly separate Speedometer from the frameworks > it tests from the real world. Speedometer was announced [1] as "a benchmark > that measures the responsiveness of real-world web applications". While best > practices could differ between frameworks (I could not find any examples), > the default premise should be that real-world web apps use production builds. I wouldn't make such an assumption. When I first wrote Speedometer, many real world Ember apps were using debug builds. We really need to do a careful assessment of each framework & library separately.