Well, in the back seat, you probably can't hear me.
Anyway, I'm Roman.
I'm going to talk about Project Lilipot.
What is about a little bit of the history and what we're trying to do?
It's mostly about reducing the memory footprint of Java applications.
And specifically, it's about what we're trying to do is by reducing the size of object headers in the Java VM from currently 12 bytes down to 4 bytes.
As you probably know, in typical Java applications, you have like tens or hundreds of thousands of objects and objects tends to be relatively small in many workloads.
So this adds up to typically about 20 to 30 percent reduction in heap usage.
Your mileage may vary. If you do the math, currently the minimum size of an object in the hotspot VM is 16 bytes.
If we reduce that to like 4 bytes, this doesn't really happen because we have object alignment of 8 bytes.
So the minimum object size would be 8 bytes with that. So the maximum savings that you can get out of this is like 50 percent.
The minimum savings would be close to zero if we haven't heap full of really huge arrays or something like this, then you would have no savings.
Most workloads tend to be in the middle of this around 20 to 30 percent heap reductions.
And we can see that in, I run a, it's a well known benchmark that simulates a web shop, hey, simulates Amazon.
And I run a steady workload on it and measure the heap size after periodic full cheese.
And with the current JVM, we get around like almost 1,200 megabytes of heap usage after GC.
With little put, we get much less is around 900. It's about 22 percent savings and heap usage.
And saving heap memory is the primary effect of this. There's also secondary effects because this lowers the pressure on GC.
This improves cash locality and so on. So this often also results in improved performance.
In this same workload, we have this benchmark produces latency measure and a throughput measure.
So the left two bars are the throughput measure.
With baseline current JVM and little put. And this means that we fully get about 8 percent improved performance in this particular workload.
The right-hand bars, the two bars on the right-hand side, this is the latency measure, the critical jobs.
This improved even more by about 11 to 12 percent.
So how did that start? So this is a bit of a rundown of the history of linear put.
Now that all back in 2020, I believe at first them, I was just talking about a new big new change that I implemented in the Shenandoah GC.
Back then, the Shenandoah GC used to require an extra word per object to store the forwarding pointer.
And at this first them, I talked about a big change where I removed this requirement. I removed this whole word per object requirement there.
And I had a chat with Andrew Haley in a taxi I believe. And he said, that's great, but could we do this for all it?
Could we do this for the whole one-time? Because we used like 16 bytes of memory for each object header, but other JVM's like GCJ, they only use alt bytes.
So why don't we do this in the notebook JVM? And well, how hard can it be?
Right? It's good that nobody tells you how hard this could be. You would never start projects like this if you know how hard this would be in the end.
So this is how it started in at first them five years ago. It took another year to actually start it because I had other things to do and more work to do on Shenandoah, so the first commit happened in March 2021.
We did lots of prototyping, Thomas Stufe started to work on class pointer improvements. He will give a talk later about those improvements and how we did this for Lillipot.
So I don't want to talk about this here in 2022. We realized after a bit of prototyping that we that the way that synchronization works in hotspot JVM doesn't really fit what we want to do in Lillipot.
That is because synchronization, so this is what happens when you do a synchronized statement and try to need to synchronize up.
This was very racy on the object header because it's changing lock, it's changing the lock bits in the object header concurrently, basically with the running application.
And this didn't really work well with Lillipot. So we decided to change this the way the locking works in the hotspot JVM.
And this is again the case of how hard can it be.
As a kind of funny side story, there used to be a file in this code where it says as the top do not ever talk unless you really know what you're doing.
And even then don't touch it. And we basically ripped it out and replaced it with something new.
So this took a whole year to the prototype was quite quick at one month of work to get something working.
And then it took a whole year to get the PR reviewed and reviewed again and change things and file another PR.
And it was probably like 600 review commands and a lot of work on the side of Oracle and others to test it and review it.
And we finally finished it in May 23 and it went into JDK 21.
Even then it wasn't really finished because we also realized that the first implementation didn't have to put recursive locking and we realized that some workloads.
This is quite important for performance. So from Oracle implemented recursive locking in this new stack.
This went in JDK 23 and this new lightweight locking became the default locking mode in JDK 23.
So this was good progress.
Even then it was not really finished because there was this was only the lightweight locking side.
And there's also the kind of heavyweight locking that uses monitors.
And this still uses the object header to, yeah, for some purposes and that kind of work, but we wanted that to go away.
So,
Excellent and other thought that Oracle implemented the new object monitor table and this went in August 2024.
So, 2023, what happened? We finished lightweight locking. We first proposed the version of the jet 450 for project lily port.
This is what we call lily port 1.
In 2024, the recursive lightweight locking and object monitor table landed and we finally integrated the first jet for this project into JDK 24, which is about to ship in March.
In 2025, this is where we are now. We are prototyping what we call lily port 2, which is the actual four byte errors.
And, yeah, later this year we get the EA of JDK 24, where we ship the lily port 1 basically. So, what's in it?
To the current state of object layout in hotspot looks like more or less like this, though we have a header at the top, which is, so each of those rectangles here is one word, which means eight bytes, right?
So, the first word of each object is what we call the object header or Mark word for some.
Then we have a class pointer, which identifies, which kind of class the object has, it's useful for things like virtual cores etc.
And then starts to field or the array element. For arrays, we also store the array length in this 32 bit block there. And then we have the array elements.
We want to, so for lily port 1, this is what is going into JDK 24. We improve this to only take eight bytes per object. So, we merge the class pointer into the object header.
So, we only have the single header here. For arrays, we still have the array length, but it shifts up a little bit and makes some free space, which we can use for the array elements.
And the long-term plan is to reduce that to long-term means like, a couple of more releases.
To reduce the header to just four bytes, make some more room there for actual fields or to store the array length on the other side.
Let's look at the detail. So, what is really in this object header. So, this one, the, the other sketch, schema here is showing what is in those bits in the object header. So, we have two log bits here, which is used for locking is the name implies there's an unused bit here.
There's GCH. This is where we track the edge of an object. This is four bytes. Then we have another unused bit here. We have 31 bits that is used for the identity hash code. And then we have some more unused bits here.
And in the second word, I showed you in the slide earlier, we have a compressed class pointer which takes 32 bits.
The one of the main insights of project Lily put is that most objects never get identity hash code.
And most objects never get locked. So, this is really a tiny percentage of objects that are usually locked at all.
So, we try not to push, push the burden for these features on every object. And instead, let only those objects that really need it pay for it.
There's another little twist when the object is locked or when the object is GCH forward, which I will get to later.
This whole header is replaced by a pointer into native memory, which basically points to the log object or the stack location where the, the, the monitor is.
And this is the reason why we, we couldn't deal with lightweight locking so, so well.
So, for Lily put one, we wanted to change this and you, you have seen all those unused bits that we make use of those unused bits.
We, we packed this a little bit better. We still have the log bits. We have a bit that, in the same forwarding, which is such a detail.
Still, we have the object H here. We reserve four bits for the project Valhalla because those guys told us, well, we probably going to meet those four bits.
So, we have those bits here. We still have the hash code bits here. And we have a compressed class pointer in the upper 22 bits.
For Lily put to the plan is or the current implementation is that we packed this even better. We only want to have two log bits still the self forwarding with those bits are all the same.
We only have two bits to support the identity hash code. And then we have an even smaller compressed class pointer up there and only nineteen bits.
So, how are we doing this? Thomas is going to talk about the class pointer bits. I want to talk a little bit about the identity hash code.
So, how do we get from 31 bits for the identity hash code down to just two bits for the identity hash code?
Just as a quick recap what this is doing. So, the identity hash code the contract is if two objects are the same object, then the identity hash code for one and the other reference must also be the same.
And this is about object identity not about object equals. So, this is an important distinction. But identity hash code is used as a default implementation for object equals. So, this is used more often than you probably think.
The inverse is not necessarily true. So, if the identity hash code is different doesn't mean that the objects must.
Could be sorry, the inverse is not true. Anyway, the identity hash code must remain stable throughout the object lifetime which is an important property must not change.
So, I want to shoot for great entropy. And yeah, as I said, the default implementation for object identity hash code.
So, the current situation is how it's implemented as we have 31 bits in every object that is reserved for storing the object, the identity hash code.
The system identity hash code generates basically a random number and stores it there. And whenever this identity hash code is called again, we read that number that we just stored there and then return it.
How can we avoid using those 31 bits? So, the first knife idea is to always calculate a hash code that is based on the object address.
The problem with that is that GCs can move the object around the address is changing. This is this violets one of the important properties that the identity hash code must stay the same.
The other idea allocate a slot for the identity hash code on demand only when we need it.
The problem is that usually the heap is filled consequently which means there is no space to do this.
So, how can we solve this? We basically combine the two ideas.
I get to this in the second. I first wanted to describe what those bits are doing. So, we only have two bits now and the meaning of those bits is the first bits says the object has been hashed or not hashed.
In the second bits says the object is expanded or not expanded. And from there we basically have three states that we are using.
We are going from the first state. The object is not hashed and not expanded. This is the default when the object is freshly allocated.
The first time we call system identity hash code this state transitions to the second one the hash not expanded state which means we calculate the identity hash code based on the address.
As long as the object does not move we can keep doing this.
So, if we subsequent calls to identity hash code will recalculate the identity hash code based on the object address.
So, what happens as soon as the gc moves the object around and it observes an object that is in this particular state.
Recalculate the identity hash code once again based on the original object address. Then it copies the object.
Allocates and x was lot for what we are doing this allocates and x was lot for storing the identity hash code.
Sticks the identity hash code there and then every subsequent call for identity hash code will return the hash code that we just stored there.
And we move the state to this hash and exponent states. So, we know what we are doing.
And as soon as we are on this state every subsequent call will simply read this identity hash code slot.
This where do we store this? We can, so the, if the field layout of the hotspot class loader can figure out a good place to do it.
Sometimes there are gaps in the field layout and we can use those gaps.
Sometimes there is an alignment gap at the end of an object. We can use the alignment gap.
And if there is no gaps available, then we attach it at the end of the object and allocate an x for slot basically.
So, often we don't even need to extend the object for real. We can reuse some free space at this there.
So, this is the result of this is the only required to bits per object.
And we allocate the space only when we really need it.
As I said, majority of objects never need any identity hash code, so we save a lot of bits here.
Question is, do I have enough time? I will quickly go over the other one.
Need to make this quick. She see forwarding. Current situation is when it she see needs to copy an object.
Then we need to store the new location somewhere.
And this happens by usually by storing in the header itself.
We flip those bits here to say one one, which means there's forwarding pointer and all the rest of the header would be interpreted as a pointer into the new location of the object.
There are two cases.
One is copying compaction, the other one is sliding compaction.
Copying compaction is relatively easy because we copied the object to a fresh region that is basically empty.
And we retain the old copy as long as we need it for updating the references.
Which means we don't, the problem is that when we store the forwarding pointer here, it all writes the class pointer information that we have here.
And this class pointer information is crucial for operation of GC.
So it's still quite easy because we just copied the object and we can find the class pointer here.
And this is relatively easy to solve and need to be careful, but it's not that too big deal.
So this is how it works. I skipped this because I'm out of time.
As I said, it's relatively easy to implement because we have a copy of the object and there's no problem with reading the class pointer from the new copy.
Sliding compaction is a bit more tricky because what's sliding compaction does it doesn't copy the object to a new empty space.
It basically, if you imagine the heap, it's a big stack of objects.
It throws out all the unused dead object here and it slides all the life objects to the bottom of the heap.
And the way it does is it first finds all the life objects in the marking phase.
In the second phase, it calculates the forwarding addresses for each of the objects.
And it stores those addresses in the object headers, at which point we would lose the class pointer information.
And then updates or the references and then copies the objects to the new location, basically sliding them down.
And we must not lose the class pointer information. This is the really difficult part here.
But we also don't have a copy from which we could read this class pointer information.
And we only have nine bits here for actually storing the forwarding pointer information.
So what we can do, the idea is that those nine bits in the object header that we have is enough to address a 512 words of a range in the heap, which is kind of 248 bytes or 4 kilobytes even.
So the idea is let's divide the heap into blocks of 412 words each.
And then use a side table with one entry for each block.
And each entry basically stores the forwarding address for objects in that block.
And then we can use those nine bits in the header to find the forwarding address for the object.
So the basically means look at the block in which this object resides, find the forwarding base of that block.
At the bits that we have shifted by three and then you get the forwarding address.
This is a kind of a forwarding table, but it has the advantage that it has a fixed size.
It's basically a fixed size 1, 512 of the heap size.
It's not really that big. We have basically no performance loss using this approach.
We have a potential to reduce the size even more by using the bits efficiently or by extending the block size by one bit or so.
And eliminates the eight terror by the limit of a little put one.
So this is the advantage. I need to make it short because I'm running out of time.
And putting it all together we have tiny class pointers which is going to talk about the compact identity hash code improved forwarding table.
And we still have four bits basically currently unused, but we'll be used by Wahala.
Which is great. And the best thing about all of this is that you can basically forget it because you don't need to remember all this.
You only need to enable this feature by using this flag and you will get memory savings for free.
In the future we intend to make this hopefully make this the default in one and only had a layout in hotspot JVM.
And that's it. Do we have time for questions?
Question.
So the question was if all objects using identity hash code do we use more memory or what was.
The performance.
Yeah. Okay.
I did some experiments. I could not see any performance degradation.
I did experiments similar to what you are describing.
We haven't done the full experiment yet.
So I expected as long as the object doesn't move, we are recalculating the object identity hash code.
And we believe that this would be expected to be much even a bit faster than reading the identity hash code for memory.
Because memory access can depending on if it's in the L1 cache or if it's in the RAM, right?
You can have a lot of cycles just to get this stuff into the L1 cache.
The identity hash code calculation is really designed to be very fast.
So this should not take many cycles. Yes?
The question was if we have gaps in the layout and we have smaller gaps,
smaller than 30 to bits, can we split the identity hash code?
No, we currently don't do this.
Yeah. I guess it would be possible.
So I guess adding the complexity of making this decision is probably more expensive than I think.
Yeah.
So I've got a single question here.
So if we have this, we're right there.
Yeah.
There could be a field, like an infield or something.
No.
Typically four bytes.
Yeah.
Unless you have four byte references, so this typically fits in there.
There might be a gap there, right there.
Exactly.
Yeah.
Correct.
Okay.
Any more questions?
No.
Thank you.