You're right, so can people in the back hear me okay?
Yeah, louder, okay.
I'll try to go loud and we'll see how far my voice goes.
So my name is Stephanie Hansen.
I work at Oracle in the G7 team.
I've been doing G7 for the past 12 years.
Mostly focusing on G1.
And lately I've been doing more and more CGC work.
And that's what I will be talking about today.
How we're making CGC as easy to use as possible.
And this is kind of a sibling talk to one
that my colleague Eric Ostlin will be giving a jaw one.
Since we have a bit shorter time here,
I will be focusing on some of the things that we've seen with Linux
when you're around CGC and some configuration problems there.
And yeah, point up if I need to raise my voice.
First of all, short introduction to CGC.
We want CGC to be a really good low latency alternative for Java.
And that means providing pause times below one millisecond.
We also want CGC to be very scalable.
So it should be able to handle terabyte size tips
without what the pause is growing in the larger.
And we also want CGC to be very seduced.
And the user should be required to minimal amount of configuration.
So if we look at the current status of the CGC,
and when it comes to low latency,
we feel like we tick all of the boxes.
And the pauses are most therefore synchronization reasons.
So to make sure that the CGC and the application friends agree
on what they're currently in or remarking objects
or are re-releicating objects or what's going on.
So all the heavyages that work is done concurrently
with the Java application running.
So that's really great.
From a scalability point of view,
we also feel very okay.
And most of the boxes are ticked.
We can support up to 16 terabytes of tips
and the pauses are still short.
And what's really fun is that we recently started
to get some reports that people are using large hits,
with CGC.
They found some issues,
and we're trying to even out the rough edges
around the corners here.
So that's really great to see.
When it comes to the out-to-tuning part,
we do a lot of configuration automatically.
But there's still a few boxes left to tick.
But still the number of threads are sized automatically.
The generation sizes are sized automatically.
We figure out when to promote all these to the old generation automatically.
We basically tell the user,
just set the heat size.
The problem with that is setting an optimal heat size.
It might not be as easy as it sounds,
because you need to take into account
what other things are running on the same machine or in the container.
You need to understand how much memory the JVM itself needs.
The code cache needs memory and other things
in the JVM also needs memory.
So if we could size this automatically,
it would be great.
And this is a product that our guests are in this leading
to add automatic heat-sizing for CGC.
And he will be covering that in much greater detail
in his talk in JVM.
So if you have the opportunity to go there,
do it and watch his talk.
Because it's a really interesting topic.
Yeah, I think this is it for the introduction to CGC.
Now I want to talk a bit about some of the design decisions
that we've done in CGC and how they differ a bit from the traditional uses.
The first question is why different.
And the reason for this is to achieve all those goals
that we stated,
CGC needs to be a concurrent collector.
Because otherwise we won't be able to write this really low latency.
And when you have a concurrent collector,
you get slightly different trade-offs
compared to the traditional uses
where you do most of the JVM's inside the JVM posts.
When you do a lot of work in the posts,
you need to make sure that the post is very effective and short.
Because otherwise it will affect the latency of the application.
When you have a concurrent collector in the process
or way below a millisecond,
like 100 microseconds having a post is not that big of a deal.
But instead you need to focus on other things.
And two of the main focus areas for CGC
is to keep the concurrency overhead low.
And what will basically mean by that
is make sure that the application throughput
isn't affected too much by also doing concurrent G
is a work alongside the Java application.
We also want the heat memory headroom
or overhead to be as low as possible.
So since the application is running
alongside the GCC,
the application will continue allocating memory.
So we need to make sure that the GCC free up memory
before the application allocates it all.
So this is a big part of the concurrent collector too.
To keep the headroom low.
So one of the key design decisions
is to achieve its colored pointers.
And we basically make use of most of the 64 bits
and a 64-bit pointer with colored pointers.
There are 46 bits for addressing.
And we have 12 metadata bits.
And more here are the metadata bits.
They allow us to solve some really complex problems
in a really efficient way.
So for example, it enables us to have very low GCC overhead.
But just a few instructions,
we're able to determine whether it's safe to use a pointer
or not if it needs to be fixed before they use it.
And having that feature, being able to very efficiently look at a pointer
and know, can we use this pointer right away or not.
We're able to do very eager memory recognition.
And the way we do this is,
since we can look at a pointer and know if it needs to be fixed or not,
we know that we will never look in a region
where an object is not located anymore.
So if we have a located all live objects in a memory region,
we can hand that back to the application as soon as the
location is done.
We don't need to remap all the pointers point the
into this region.
Because we know that no state pointers will ever be
accessed into this region.
So we can reuse the memory very quickly.
And another thing that this makes us make us possible
to do since we have 46 bits for addressing,
we can have a discontinuous heap.
So basically, we reserve additional virtual memory
address space for the heap in CGC.
A traditional GGC basically reserve the same amount of
virtual address space as we have the heap size.
But in CGC, we reserve up to 16 times as much
virtual address space compared to the heap size.
And the main reason for this is to avoid fragmentation.
So if we do this, we'll almost always be able to find room for
large allocation.
And you might use G1 in the past or you might be using
G1 and have some problems with you on this object.
And the handling of you on this object in
G1 has improved a lot over the last few years.
Still they introduce some issues.
And by doing this, having the additional
virtual address space and always being able to find room,
we can get away from most of those problems.
So that's pretty neat.
And other thing that differs between CGC and
many other businesses is backed by shared memory.
And the reason for this is that in non-generational
CGC, we needed to do this because we mapped the heap
at multiple addresses.
So the way that the caller pointers worked in non-generational
CGC was that we had multiple views over the heap
and the metadata bits told us which view was the current one.
With generational CGC, we instead have a feature called
LLS Roots.
So the efficient code is able to save away the metadata bits
and we only have a single view of the heap and a single address
for the heap.
So that's nice because it gives us additional
and we can support even larger hits with generations.
So that's pretty nice.
And still, we make use of shared memory even though
we don't need it for generational CGC.
Because we can do some things that we can do in
on this memory.
For example, it allows us to do lazy and mapping.
So for example, if you need to fit the logic
into the virtual address space, you can
map memory to a new location where you have room for this logic.
But you don't have to un-map the location of the old memory.
You can do that lazily in the thread that is not
performance critical.
So there are still some pros of using shared memory.
But we do experiments with moving to on this memory
because being the old one out and not using
the same techniques as others can come with problems.
And that's what I'm going to talk about in this next
section that these new techniques that we're using.
On Linux, they give us some configuration problems
or issues.
And we want to try to avoid those because we want
to minimize their amount of configuration.
But I'm going to tell you a bit about two of the things
that we're seeing on Linux.
The first one being that with CDC,
sometimes get very strange need of old memory.
I say strange because it's still plenty of memory left on the machine.
But still like M map or M protect,
or any of the other low-level calls,
making changes to the virtual memory mapping in Linux
return you know map.
So there is no memory left.
And the reason for this is that we hit a kernel limit
called VM max map count.
And this limit tells you how many mappings of process
can have at most.
And when you have a very large heap,
CDC, since we have this discontinuously,
might need a lot of mappings.
Because depending on the allocation pattern on the application,
we might do more mappings than a traditional disease.
So on startup, CDC looks at this value,
compared to the heap size that is set,
and sends out a warning if the value is too low.
And then also gives us a suggestion on how
to configure this value properly for your given heap size.
In many cases, you won't run into hitting this limit
because your application doesn't have allocation patterns
that triggers this.
But sometimes this warning actually tells you what you need to do
to really be able to use CDC in an efficient way.
The problem is that depending on how you're deploying your system,
seeing this warning might be problematic and you might just
see it out of memory and don't really understand why.
So what can we do about this?
Well, to get to the bottom of that,
we need to first understand this limit a bit more.
And the math count limit seems to be mostly there for historical reasons.
Back in the day, the elephant had restrictions,
and this led to core files only being able to handle
64K case of mapping.
So the default value was set to 64K.
Since then, the elephant has been extended,
and the core files have been improved to be able to handle this extension.
So this limitation is no longer a reason for this limit being low.
It still is, and it's not only so you see the running
to problems with this.
There are numerous cases online,
if you search for outer memory problems,
where people have spent a lot of time
to try to understand why there are certain application or game run
into problems with outer memory,
while there is still a lot of memory left on the system.
And there is games in memory databases
that actually state clearly in their documentation.
You need to reconfigure this value,
otherwise our in memory database won't work properly.
So our plan going forward is to try to influence
a change to this default value.
And we'll see how that goes.
We're not the first ones that have thought this question
and seen the problems with this.
So there are multiple emails on the Linux mailing list
suggesting either to remove this limit or to increase it.
Still, most of those attempts have been shut down
or all of them.
But we'll try to work probably with internal Oracle Linux teams
to see if we can argue for this limit being raised.
Because we want y'all to be able to run properly.
Without configuration on all distributions.
There are actually a few distributions that already
up this limit by default.
I think you've been to for example,
ships with a default value one million.
But it would be really great if the default from Linux
would be larger.
Yeah, kind of, but you can set it at runtime as well.
So it's easy to configure.
And it's really no drawback in configuring this to a larger value.
You will use a bit more kernel memory
because they are kernel data structures.
And this is kind of one of the reasons
or arguments made for not changing this that you could
maybe force out of memory using kernel memory
if you raised this value too high.
Yeah, we will have to argue around those things
if we want to change this.
Another thing that requires configuration on Linux
and it's something that's very important.
If you want great performance with y'all, it's huge pages.
In the path, we've encouraged people to use explicit use pages
or a huge TLB use pages because they are very deterministic
and once you have reserved them, they will be available for use.
The problem is that you have to reserve them up front
and to be able to know how many to reserve you need to decide
upon a heap size.
And as I mentioned, we want to make the heap size automatically
than this is not the very good effect.
So what kind of alternative do we have?
Well, we have the transparent huge pages.
They were introduced a long time ago, more than 10 years ago at least.
And at the time of introduction, they came with some performance drawbacks.
We saw latency issues when using transparent huge pages
due to the kernel having to defragment memory
to be able to satisfy transparent huge pages requests.
Later, or recently, we don't see as many problems with transparent huge pages
so we're getting more and more comfortable in encouraging uses
to use transparent huge pages.
So this is great.
Still transparent huge pages that require some configuration
and especially for shared memory, it often requires configuration
to be able to be used.
So let's look a bit closer at what kind of configuration
you have to do for transparent huge pages.
And the THP or transparent huge page mode,
basically how three different values,
it can be configured as always,
meaning that as soon as you have a mapping
that has the correct alignment and the correct size,
the mapping will be backed by transparent huge pages.
The Linux system will try to back the mapping
with transparent huge pages.
Or it can be configured as advice.
In that case, US and application developer or JVM developer
need to specify that,
this mapping with a correct alignment and the correct size
I want this to be backed by transparent huge pages
by using M and vice.
Or it can be configured as never,
and kind of explains itself,
you will never use transparent huge pages to back any mappings.
And all of this memory and shared memory are configured separately
and this is a bit problematic to suggested,
mainly because the defaults are different.
So on many distributions, the default for a shared,
for anonymous memory is always,
while the default for shared memory is never.
So you should really check your configuration
and you do this by looking at those two paths,
shared memory, of course, being the second one.
The problem here is that it gives unfair
out of the box comparisons.
So for example, if you compare G1 to CGC
on a default configuration existing,
G1's here will be backed by transparent huge pages,
but the CGC will not be backed by transparent huge pages.
And from a performance point of view, this is very unfair.
And we've seen in the mirror's cases,
both from the outside internal groups,
where we get the feedback while CGC looks due age from the late
point of view.
But when it comes to the throughput performance,
would you see like a 10 to 15% performance
or throughput regulation?
And most of the time, this is due to the fact that
they compare again something that may use
of transparent huge pages,
and CGC don't in the default configuration.
So once they're able to configure shared memory properly,
the performance regulation goes away.
The problem with this is that it forces you to do configuration,
and we want to avoid this.
And for some people doing this kind of configuration
is not possible because we run in a container or a system,
we don't have access to update the OS configuration.
So what can we do then?
Because yeah, we could move to anonymous memory.
Still, that requires that the configuration is correct
for anonymous memory.
But recently, we actually found this new flag
to a device called Maddy Collapse, which looks very promising.
The drawback is that's only available in the 6.1,
so it's a fairly new thing.
But it might be something for the future.
With this flag, you get much better control
on which mappings will be backed by transparent huge pages,
because when you say Collapse, my mapping with them,
and device, that mapping will be backed by transparent huge pages.
This can of course come with a performance penalty unless
if there are no transparent huge pages available,
then the underlying system needs to do
some different implementation work.
So you need to use this with care.
But the good thing is that it doesn't depend on configuration.
Even though the configuration for shared memories
never using this flag, we're able to get transparent huge pages
for that memory.
So this might be a way forward to avoid configuration.
Again, and making it even easier to use,
and of course, other things that use German memory.
And this was my last main slide.
A few key takeaways from this presentation.
Don't remember the problems.
We're going to fix those.
Instead, remember, or know that,
you see, basically, has no process.
It can handle terabyte size tips,
and it's very, very easy to use.
Thanks.
Thank you.
I spoke too fast, so we have time for questions.
Any questions?
Everything, please, crystal clear?
Looks like it.
Oh, question.
Can I ask you a question?
The collapse flag, it's a flag term advice.
So you can say, like, for this mapping,
I want to make sure that this memory is backed by transparent huge pages.
Yeah.
Okay.
Yeah.
So we should be able to use this.
Oh, yeah.
So the question is about m and vice collapse.
If it's synchronous and from what I understand it is,
and that's why it can come with some performance drawbacks.
So you need to use this with care,
make sure kind of have a priming thread
that makes sure that you don't do this in the application threads,
because that could be expensive.
I mean, you probably wouldn't.
You probably wouldn't do it upfront on the whole thing.
The plan would be, for example,
if you have automatic heat sizing,
you would start with a very small heat.
And then you would grow the heat,
and you would do that in a thread that's not performed as critical.
And you would do this having advice in that thread,
making sure that we'll always have transparent huge pages.
Remember we left, but you don't have to take the heat
in your application thread.
So that's kind of the idea.
Yeah.
Go to all along.
No more questions, I guess.
Thank you.