Hello everyone, hey, I'm Eric, I'm an engineer and the android by performance team and I'd
like to talk to you a little bit about things we did for chromium and android to make
it nice and fast and to give you a little sneak peek, this is what the graph looks like
for speedometer performance on android and chromium over the last two years roughly.
It's an graphic quite proud of, it's not just me that accomplished this of course, it's
a large group of people that helped us get here, but yeah, we basically doubled web performance
measured as speedometer score on android through a lot of work.
To maybe briefly take a step back and ask how many of you do know what speedometer
is.
That's a good number, it's really a web interaction benchmark, it tries to measure how fast
as a browser respond when users interact with a website.
It does that using synthetic workloads, here what you can see a lot is a to-do application
where we add items to to-do list and then remove them, we measure how long does that take.
Why does this matter?
Speedometer improvements, they mean interactions on the web get faster, but they also
mean that page loads get faster.
And this is an example of loading a Google Doc in a Chrome on Android back two years ago
and now you can kind of see that two years ago that was almost 50% slower to do and it's
all due to these kinds of improvements.
So with all that bait out of the way, let me say thank you to a lot of people that helped
us get here and tell you what I want to talk about here today.
First I'd like to break down this timeline of what do each of these improvements, where
do each of these improvements come from, how do we get to 2x?
And then I'd like to dive deep into a rather geeky area and nerdy geeky area of the built-up
optimization that we added.
And then talk a little bit about the tooling that I enables us to understand this kind
of browser performance on the lower level close to the hardware and how we can identify
where there are optimizations.
So it's very much a view from a browser developer close to the barebones hardware here,
which might not be quite the same to a web developer, but some of the tooling, some of the
insights, some of the processes might actually also be quite relevant for a web developer.
So yeah, speedometer speedometer, lastly I also want to talk a little bit about what we are
doing to get workloads in the lab closer to real page loads, real interactions and
page loads on real websites.
So yeah, this timeline, a lot of the improvements here came out of these three areas.
First, improvements to Chrome's build to make it faster to execute on modern Android hardware.
Second, they were a bunch of improvements to the rendering and the JavaScript engines in Chrome
that contributed kind of the other half to mostly the other half of these improvements.
And lastly, we had to work quite closely with other OEMs and Android partners to make
sure that browsing is actually scheduled correctly on the hardware.
So digging into the build first, the main thing that enabled us to make a change here was
the fact that we split Chrome's build into two halves for Android.
Previously, we were shipping the same APK, the same binary to all Android devices.
And you might know Android devices, they range from $100 phone to $2,000 flip phone and
these devices perform very differently.
For the low end, it's really important that we ship an APK a build that's really small
and size and has low memory overhead because these devices ship with poor flash, very
little amount of flash and a lot less memory than the high end phones.
So for poor, these poorer, more economy-style phones, we can't really ship a very well-optimized
Chrome binary because a lot of these optimizations that we can land into Chrome, they increase
our binary size, they increase the memory footprint.
So splitting Chrome's build into these two halves and shipping one APK to lower end phones
and another APK to a more premium end phones made a lot of these improvements possible.
What are we doing differently for this high end build, I guess?
The first thing we're doing differently is that we don't optimize for size in the compiler,
we optimize for speed instead.
We also don't have to target 32 bit, are many more, we can say, well, this build, we are
only going to ship it to 64 bit devices.
That's actually also something that increases the memory footprint slightly.
64 bit means every pointer becomes double the size.
We can make some tweaks to that, we can use pointer compression, for example, to reduce
pointers in V8 and the garbage collection heaps and the JavaScript heaps and the bling heaps
potentially, but overall there's still going to be a memory impact.
Even on low end phones where we have 64 bit capability, we wouldn't ship a 64 bit
built, we would ship only a 32 bit built because of the memory impact.
And finally, what we can also now start to do is profile guided optimization, PGO, this
is basically a mechanism that runs workloads in the lab on Chrome and figures out what
of the code is hot, what of the code is, which code is less hot, which code is cold and
applies different optimizations to a hot and cold code, and we'll get into more details
of what that actually means later.
So initially, we just enabled all of these things.
Second, we then dug deep into how can we improve on that.
We switched the generation of these provided optimizations to use different profiling data.
Previously, we were kind of reusing profiles that were already present for Mac 64 bit arm devices.
Now we are switching to 64 bit profiles collected on actual Android phones.
And then it improves performance.
We also made sure that the PGO profiles that were used in the binary built later are profiles
that were very recently computed.
If you let a lot of time pass between generating the profile and doing your build, these profiles
become stale and might not actually apply the correct optimizations to the correct code.
Another thing that we then discovered later is that we can increase inlining in the compiler
so that the compiler kind of prefers to pull in more code into inline functions.
In this rigorous binary size, but it's actually beneficial on a modern hardware.
And lastly, we also added improvements to the order file.
That's a Chrome specific compiler, more like a linker feature that tries to arrange functions
across the binary in a sensible order.
That's now based on speedometer and that helps a lot too.
So that's all built.
Beyond built, I mentioned there a lot of improvements in the Chrome-UM engine itself,
like the blink rendering engine, for example, there were a bunch of small improvements
that were landed across the engine and a little bit there, a little bit there, here it
adds up.
With even 11, 13% plus improvements just by adding up lots of tiny little things over a year,
two years that adds up.
But there were also a couple of bigger changes that were landed.
The first one called out here is an improved parser that makes it faster to parse HTML when
it is inserted dynamically via the NIHDML attribute.
Again, something that we didn't ship on Android before, because binary size.
Adding this extra parser means we regress low-end devices.
The eight also added a new baseline compiler here that's basically a tier that sits in between
the really quick to generate code, ignition interpreter and the eight,
and the next level up compiler here that crunches out really well optimized code using a
Jet compiler.
Spark plug is a baseline compiler that is really quick to spit out somewhat better code.
And adding that into V8 improves speedometer but also improves spatial significantly.
The last thing to call out here is garbage collection.
I think there's more opportunities in that space.
But in the last few releases last year, we landed a few improvements to make sure that
garbage collection happens in a better moment in time.
Rather than triggering in moments when it will affect negatively speedometer scores or
interactions on pages.
Last area is scheduling an operating system.
So it turns out that if you don't tweak your kernel to prioritize the right threats,
your performance suffers. Who would have guessed?
This is an active area for us. I think we landed a couple of initial winds here with some of the
OEMs and Android, but Android is really fragmented in this area. A lot of OEMs use very
different scheduling heuristics policies in their platforms and making sure that web browsing
is effectively prioritized there is a very tricky topic.
All right, so to dig a little bit deeper into this built thing, I wanted to show you a little bit
of data. This is a data that we collect during speedometer on an Android device.
This is in this case a pixel 8 device. This was before we landed all these PGO improvements.
And what you can see here is that in speedometer, the execution in the CPU is often
stalled in the front end of the CPU. The front end of the CPU is kind of the piece in the CPU
that tries to fetch instructions from the memory to then pass them on into the execution units
in the back end to execute. So what this means really is that the front end is having trouble
identifying where to fetch the instructions from. Stalling here means we have to wait for
the data to come in so that we can actually take these instructions and put them into the back end.
And typically what that again means is that you are waiting for memory, you're waiting for
these instructions to come out of your cache hierarchy or out of your DRAM.
And what we discovered is that in Chrome, before all of these PMU, sorry, before all of these
provided optimizations, we were seeing a lot of stalls were due to branches in some form or
another cache misses that happened because we were mis-predicting branches.
And speedometer is a workload that is very different to many other workloads that CPU engineers
typically utilize to benchmark their CPUs. That's a very quick statistic here is on branches.
We don't matter, it's about 20% of its instructions being branches. That's every 5th instruction
is a conditional branch that wants to go somewhere. In workloads like peak bench,
something that compiler engineers or even CPU engineers would be more familiar with,
that number is about half. Given these many branches, it's really important that
these branches are predicted correctly in the CPU. When you mis-predict a branch on an ARM CPU,
you end up paying not only for the mis-predict itself, right? Like you mis-predict the branch,
that means you have to roll back all your execution to the beginning of this branch
throughout all the instructions that you executed in a predicted fashion and then restart instructions
instruction execution there. But when you did this mis-predict what you also did is you mis-predict
that what memory to fetch into your cache hierarchy to load your instructions from.
So you end up polluting your cache hierarchy with the wrong instructions and you end up not having
the correct instructions in the cache hierarchy, which means that again you increase the time
needed to fetch data from memory in the front end. The way that you solve this is by making
sure that in the code of your application, you align your code in such a way, your branches,
on the assembly level, and such a way that the fall-through branches, like the fall-through
path through a branch, is the one that is most often the taken one. That's what provided optimization
attempts to do. It tracks which path in each branch is taken, while executing a workload.
It then sees that maybe 80% of the time you have to go that way at 20% of the time you have to
get that way. So it then takes that information and during compilation, it will make sure that
the 80% branch is actually the path that falls through the branch. So in 80% of cases this branch
does not have to be taken, you just fall through. This helps CPUs enormously. What also helps is to have a
CPU that has higher branch predictors, larger branch predictors. When you, and another reason
why making sure that the fall-through branch is the one that is the most prominent one, the
one that is always taken, is because fall-through branches, when the CPU predicts that a branch
will be not taken, it doesn't have to take up any space in the branch predictors memory,
in the branch predictors' caches. Branch predictor caches only store the taken branches,
the branches that you have to follow. So jumping forward in both software and in hardware
one generation to a CPU that is a little bit better in Pixel 9 now, and to Chrome that uses PGO
optimizations. You can see that this bottleneck in the CPU moves, they no longer as much front and
bound, but instead we are now back and found. That means the instruction bottleneck that fetching
the instructions, doing correct branch prediction is a lot more optimized now. But what you can
also still see is that the instructions per cycle, so the efficiency of executing this workload
in the CPU is still lower when the front and stalls are higher. It's still very important for us
to continue optimizing in this area for the front end. So I mentioned the order file earlier,
and the quick call out here is that the the order file improves on this in a separate way.
The order file tries to make sure that we are across functions, across different
functions, we are pulling them together in memory, and then more continuous, and more and
more continuous space when these functions are often executed in temporal proximity. So often
function B is executed after function A, you would want to make sure that function B is close to
function A in the binary. This helps reduce pressure on the TLB. Again, improving a different
bottleneck, but also still in the front end. And of course, we also now need to look into the
back end stall a little bit more, what's causing all these back end stalls. I think our early
insights here are also that these back end stalls are bound on cache hierarchy lookups in these CPUs.
Most of the time when we are stalling in the back end, it's actually because we have to go beyond
the L3 cache in the CPU, so beyond the cache in the CPU into the caches or the DRAM itself.
This means we need to understand why the cache hierarchy doesn't work for the back end
accesses either. Guess likely something there that again scatters memory accesses in a way
that the CPU finds hard to predict. So I wanted to take a step at explaining a little bit
how we might go about doing that, understanding why data accesses are scattered as well.
And give you a little bit of an insight into what tooling we used to get these insights. I just
show it to you. Most of these insights come from profiling and I know many of you are probably
familiar with DevTools and Chrome and it's profiling options. As Chrome engineers we use another tool
that gives us a little bit more insights into the browser's inner workings and that's
performance tracing based on Pefetto. Pefetto gives us an attribution of all the execution
in Chrome to browser tasks and also allows us to combine that information with system profiling
data like scheduling information or other system counters. You might be familiar with the
performance.mark and the performance.measure APIs in the web as well. These allow us to
annotate these workloads with user journey information. For speedometer what we did is we added
instrumentation to annotate different sub-tests in speedometer to be able to then break down
are these all behaving the same way are the specific tests in speedometer that have different
bottlenecks than others and we can then also bring in additional data into the traces on
top of that. For example we can bring in CPUPU new counters so performance counters that the CPU
tells us how many the old cycles are there at this moment in time, how many instructions
that I execute, how many cycles that that take me and we can also bring in calls to examples
from Chrome and from JavaScript. Together all of this should allow us to effectively find
functions in JavaScript or functions in the browser that have a very poor instruction throughput.
For example because they often miss encasches. So here this is really tool tooling that allows
us to go down from the big picture of I'm executing speedometer it takes me 30 seconds to
this particular function this particular sub-test is executing instructions really really
slowly and you should better look into why. So if you want to go even further you can also
let me skip this slide actually. If you want to go even further you can go down a level further
and try to understand within these functions which instructions in these functions cause bottlenecks.
So we can add in data from low-level sources on armships that are called ETM and SPE.
ETM is a tracing mechanism on arm CPUs that allows you to get the whole instruction stream
basically and identify ranges in the instruction stream where instructions were taking very
very long time or where they were hiccups and SPE is a statistical
statistical tool to do something very similar. It samples loads or branches and it can
try to identify branches that often miss in the cache or branches that were often mispredicted.
Or loads that were often missing in caches. So for example in this screenshot there's a load here
that took thousands of cycles to fulfill and that's because you had to go down to DRAM to
fulfill it. Now with all the symbolization data we can actually go and understand which load is
that which instruction is that where in the JavaScript or in the page resources is this happen happening.
And yeah. Speedometer is a good workload for us to do all of this with. But speedometer is
quite a synthetic workload and it really only helps us look at a small piece of whatever
browser has to do. And for that I've got a nice diagram that shows you which areas of the browser
is our exercise by speedometer. Speedometer is this benchmark here and I've contrasted this with
page load and with scrolling here for now. In page load we see that there are a bunch of other
browser components that are exercised. In page load also affects pieces in the browser. For example
we have to utilize the network. We have to prepare requests sent them out and get responses back.
We have to parse a lot more of that content. We also have to do a lot more rendering during
page load. There's a lot more restoration of new resources than there's in speedometer.
And some pieces in speedometer while they are exercised they don't actually affect the score
that was ultimately computed in speedometer. So they are less relevant there either.
So overall for us that means that when we are talking about this kind of low-level data
the low-level optimizations that we can attack together with partners and OEMs
there's a lot a lot of the browser that we need to have a better coverage for in the lab.
Chromium in the past has mainly focused on using field data to optimize for these use cases.
That's why you see all these webbital metrics like INP and FCP and LCP have so much prominence
in the browser world. But when you want to look at instruction level bottlenecks you cannot use
you can't get this data from the field. So we need a good workload to approximate page load
and scrolling in the lab. And for that we set out to create one because really we didn't have
one that was up to date in Chromium. We call it load line and Gemini helped us generate a
logo for it. To talk a little bit about how we approach this problem we wanted to make sure
that this was a maintainable workload. A benchmark like this in the past when Chromium has attempted
this it often became irrelevant because it was very hard to update the pages that were part of
the benchmark or the metrics used. So we made the choice to focus on a small workload that we could
maintain and that we are happy to update in the future. So we limited ourselves to only five sites
and chose those based on product needs but also based on performance characteristics of those sites.
We wanted a little bit of coverage of both fast websites and slow websites and websites that
exercise the JavaScript engine very heavily and websites that exercise the layout engine a lot more
heavily and so on. We did this by analyzing a bunch of more popular websites and then selecting
once that had different characteristics in a way to maximize coverage across a bunch of
dimensions. We also noticed that many of these metrics that we use in the field like FCT or LCT,
they don't work that well when you try to apply them only to five websites in the lab.
So instead we utilized the fact that we've chosen only five sites by building site specific
metrics that utilize some knowledge of well in this case it actually matters when this element is
shown or when this element becomes intractable and another aspect of utilizing this custom
instrumentation custom metrics is that we can build metrics that actually behave well
in terms of statistical properties in the lab. If you have metrics that are by model for example
that's really problematic for a lab benchmark introduced a lot of noise. We also have to make
sure that when we are measuring today we measure tomorrow that we get somewhat consistent results.
So while we were using real websites we had to make sure that they kind of stay fixed in one
point in time and for that we are using a tool called web page replay or WPR that takes a recording
of a website and then later replace that. It's not perfect right but what we really are looking for
here is a workload that we can utilize that is reasonably relevant.
To give you an example this is a page load in one of our pages where we are looking at LCP
and right before LCP begins or right before LCP is finished we have a very long script execution
and the browser is in the deterministic sometimes that script execution happens before that paint
sometimes it happens after the paint. This creates it by modality in the metric.
So you see a little bit of a bump in an earlier around an earlier page load time and a little
bit of a bump in a later page load time. That's something we can't have so instead we had to choose
it better moment in time for for the metric for this page. Another example LCP doesn't always
track a moment that's actually really relevant. This is a page load of a CNN page. It turns out
that LCP happens when this image is shown but from an end user perspective that's not really
relevant that's only the image there's no text. I can't really look at this page yet.
I also don't I can't really scroll it yet. I can't interact with any items on the on the page yet.
So instead we built a metric for this particular page that waits for
these main pieces of the content to be loaded and for that content to start to be interactable.
That's kind of how we approach these things and then you end up with a set of pages.
For us this is the initial set, the initial set of the first version of this benchmark that we're
utilizing. We have one configuration for phones. That's mainly striving to create good coverage
over a void variety of websites. But websites that are a website types that are somewhat popular
on the wider grind scale of things. So you have fast pages like Wikipedia. You have pages that
are really slow like a news article. You have pages that are more on the average site like a product
page on Amazon. And for each of those pages we develop metrics that either wait for a piece of
the content to be ready or maybe in some cases actually we can use LCP because it does the right
thing for this page. We also looked at tablets and our tablets are up and coming.
On tablets the use cases that are relevant for browser performance are a little bit different.
There's a lot more focus on productivity and challenging things. Also from a competitive standpoint.
And so we chose this slightly different set of websites here skewed towards larger content
more challenging content for the browser.
This at the moment is an internal benchmark for Chromey engineers really. It's built also primarily
for Android not so much for desktop and it only covers the fundamental browser performance. It doesn't
cover absolutely every browser feature. It doesn't cover the networking part very well in the browser
given that we are replaying network responses as opposed to using a live server that behaves
in quite different ways. So there's a lot that you have to take into account when
when utilizing this but it is available and it's quite easy to run. So give it a try if you're
interested. With that let me open it up for questions.
Hi, I was just curious if you could explain why the CNN page didn't show the text initially.
Is that they lazy to text itself or if they have some font problem or why the metric doesn't
work for CNN? I don't understand what the metric doesn't work but I'll come the image appears
before the text. I don't know. It's something that I see at an engineer might want to look at.
Okay, I want to look at one of the quick questions. Do you have any recommendations for
framework developers to increase the chances of branch predictions being correct?
Like there's sort of mechanism since C++ where you can kind of annotate the guide to CPU but
it's not going to look like that in JavaScript. There's sort of so kind of presidents or certain
patterns. It's a very good question and the way that I would answer it is that that's really a
problem for the JavaScript engine to deal with probably. I mean, yes, you can try to reduce branches
even in JavaScript code, right? You might want to avoid conditions on the hot path, but if you can.
But primarily, the JavaScript engine should take care of all of this for you because
the JavaScript engine, it doesn't have profile guided optimization as we use that for a native code,
but it does have all the runtime jit information. So it effectively builds up the same
the same data, which branches are often taken, which branches are not often taken,
which functions are hot, which functions are cold. It has all of this data so that it's able to
optimize very hot code and higher compiler tiers. So it should already be doing these optimizations
to a degree to make sure that the hot path in a branch is the one that is the fall through.
For example, we have to remove branches from the hot path, but that doesn't really include
functions that doesn't really include the time spent executing a function before it is optimized
by the jit. I think I know why the image erase first because they're optimized for ACP.
Yes, it's a good idea to take away all the content for LCD, right? So yeah, it's another aspect
of these metrics is that developers start to gain them.
I do have a question. I've seen you mentioned both speedometer 2 and 3.
Is that because you waited for the 3 to be released before doing the work on that or how did you
work with speedometer while it was developed? And I worked on that that's why it's so interesting.
Yes, I have some data that is from speedometer 3. I have some data that is from speedometer 2.
That's mainly because over this time period, at the beginning of this graph, we only had speedometer
2 available to us. At the end towards the end of this graph, we had speedometer 3 available.
So we switched eventually to tracking the newer benchmark. Speedometer 3 is not significantly
different from speedometer 2 from the low level perspective of what works well in the CPU versus
what doesn't. So a lot of the work loads in speedometer 3 are the same ones as they are in speedometer 2,
just slightly updated in the framework from the framework perspective.
And the newer workloads are probably a little bit more stressful for a device overall.
It's slightly larger workloads. Maybe a little bit more GPU work. So there's a little bit more
there for us to look at and look at now. But yeah, they overall they don't look too different.
For load line you talk about these custom metrics, can you say a little bit more about
what they are and if or how that might scale for more than the five sites.
Yeah, it doesn't scale to more sites. That is very clear to us. At the moment, they have really
weight for specific elements on the page. For the pages that we need the custom metric,
it's like for CNN. We make sure that the headline element is there too. On some other pages,
we might interact with an element via JavaScript and then measure the time taken up to that point.
For example, we wait until the menu icon appears and we'll click on the menu icon and we'll
wait until the menu appears. That's a proxy for us to be able to say actually they are the
contentist all there and you can interact with it. It's not like you've shown the content,
but the big JavaScript that has to make the button interactive hasn't run yet. So you can't
actually do anything on the page. To ask that's not something we can at least not with a lot of
work generalized to any website. How many brands do you do of tests on a single website in order
to get stable results? About a hundred. But there's work in progress to try and reduce that.
The goal that we set out for initially was to be able to detect like a 1% difference in score
and time taken to render our website in one hour. Running this benchmark a hundred times on each
of these pages takes about an hour at the moment. We are roughly there for this goal,
I think there's opportunity for us to optimize this. Most of the reason why it takes too long
to run it is even though the page load may be takes about 500 milliseconds for a medium side.
We have to tear down the browser and bring it all back up in between each iteration to make
sure that things like caching and process creation, etc., is all taken into account correctly.
But there's work underway to try and identify how much of that can be emulate rather than having
to re-initialize everything from scratch.
Do you have test related to the late CSS loading because sometimes the CSS comes in
and after that the whole website has to be re-styled.
I think that's an equal mouse website that can get quite some issues.
It's a good point. I don't think we've covered for that particular case. At least not from what I've seen.
It's possible that one of these sites does have that characteristics, but I'm not sure.
All right. Thank you very much.
Thank you.