Down and to the Proper: Firefox Bought Quicker for Actual Customers in 2023 – Mozilla Hacks
One of many largest challenges for any software program is to find out how adjustments influence person expertise in the actual world. Whether or not it’s the processing velocity of video modifying software program or the smoothness of a looking expertise, there’s solely a lot you may inform from testing in a managed lab setting. Whereas native experiments can present loads of metrics, enhancements to these metrics could not translate to a greater person expertise.
This may be particularly difficult with complicated consumer software program operating third-party code like Firefox, and it’s a giant purpose why we’ve undertaken the Speedometer 3 effort alongside different internet browsers. Our objective is to construct efficiency exams that simulate real-world person experiences in order that browsers have higher instruments to drive enhancements for actual customers on actual webpages. Whereas it’s straightforward to see that benchmarks have improved in Firefox all year long on account of this work, what we actually care about is how a lot these wins are being felt by our customers.
With a view to measure the person expertise, Firefox collects a variety of anonymized timing metrics associated to web page load, responsiveness, startup and different features of browser efficiency. Amassing knowledge whereas holding ourselves to the very best requirements of privateness will be difficult. For instance, as a result of we depend on aggregated metrics, we lack the flexibility to pinpoint knowledge from any specific web site. However maybe much more difficult is analyzing the info as soon as collected and drawing actionable conclusions. Sooner or later we’ll discuss extra about these challenges and the way we’re addressing them, however on this publish we’d prefer to share how a few of the metrics which can be elementary to how our customers expertise the browser have improved all year long
Let’s begin with web page load. First Contentful Paint (FCP) is a greater metric for felt efficiency than the `onload` occasion. We’re monitoring the time it takes between receiving the primary byte from the community to FCP. This tells us how a lot quicker we’re giving suggestions to the person that the web page is efficiently loading, so it’s a vital metric for understanding the person expertise. Whereas a lot of that is as much as internet pages themselves, if the browser improves efficiency throughout the board, we count on this quantity to go down.
Picture 1 – Median time from Response Begin to First Contentful Paint in milliseconds
We will see that this time improved from roughly 250ms at the beginning of the yr to 215ms in October. Because of this a person receives suggestions on web page masses virtually 15% quicker than they did at the beginning of the yr. And it’s vital to notice that that is all the results of optimization work that didn’t even explicitly goal pageload.
With a view to perceive the place this enchancment is coming from, let’s have a look at one other piece of timing knowledge: the period of time that was spent executing JavaScript code throughout a pageload. Right here we’re going to have a look at the ninety fifth percentile, representing probably the most JS heavy pages and highlighting a giant alternative for us to take away friction for customers..
Picture 2 – ninety fifth Percentile of JS execution time throughout pageload in milliseconds
This exhibits the ninety fifth percentile enhancing from ~1560ms in the beginning of the yr, to ~1260ms in October. This represents a substantial enchancment of 300ms, or virtually 20%, and is probably going chargeable for a good portion of the lowered FCP instances. This is smart, since Speedometer 3 work has led to important optimizations to the SpiderMonkey JavaScript engine (a narrative for an additional publish).
We’d additionally prefer to know the way responsive pages are after they’re loaded. For instance, how clean is the response when typing on the keyboard as I write this blogpost! The first metric we accumulate right here is the “keypress current latency”; the time between a key being pressed on the keyboard and its consequence being introduced onto the display. Rendering some textual content to the display could sound easy, however there’s quite a bit happening to make that occur – particularly when internet pages run principal thread JavaScript to answer the keypress occasion. Most typing is snappy and primarily restricted by {hardware} (e.g. the refresh fee of the monitor), but it surely’s extraordinarily disruptive when it’s not. This implies it’s vital to mitigate the worst instances, so we’ll once more have a look at the ninety fifth percentile.
Picture 3 – ninety fifth Percentile of the keypress current latency
As soon as once more we see a measurable enchancment. The ninety fifth percentile hovered round 65ms for many of the yr and dropped to below 59ms after the Firefox 116 and 117 releases in August. A ten% enchancment to the slowest keypresses means customers are experiencing extra instantaneous suggestions and fewer disruptions whereas typing.
We’ve been motivated by the enhancements we’re seeing in our telemetry knowledge, and we’re satisfied that our efforts this yr are having a optimistic impact on Firefox customers. Now we have many extra optimizations within the pipeline and can share extra particulars about these and our general progress in future posts.