Okay, hi everyone, I'm Joaquilian, so there will be a percent of native member checking
tool extending empty beyond hotspot.
So I guess the first question everyone has, native member tracking, what is that?
Well native member tracking is a subsystem that we have in the hotspot virtual machine today.
We shortly need to end them to each other using NMT over and over again.
What does it do?
It groups native applications into categories.
So if you're thinking about you perform a mallet call and you supply some category along
with your mallet call.
For example, you can say this category is part of the C2 compiler.
Okay, so we have the native applications grouped in the categories, what do we do with them?
Well we present statistics regarding these applications to the user.
You do this by, for example, saying, hey, Jake, come on, they don't native member tracking
and you get the report.
What do you want to do with the report?
You want to check stuff like my memory usage is increasing a lot, but I'm pretty sure that my
job application is a great, fully functioning.
So I think it's the VM, that's got some sort of memory leak, for example.
Then you can start looking at the native member applications to see, hey, it's my hypothesis.
Correct.
So what does it look like?
Well, I look something like this.
So now I don't have a list of points, and I guess you can see my mouth pointer.
But you can say stuff like the total reserve memory.
You can see the categories here, you have, like, a Java heat category, you can see how
much it's mapped in, you have the class category, you can see the number of the mallocks,
it's done.
It's done, it's done, a thousand mallocks, and over that it's mallocked, 424 kivites.
You can also see, you know, in that M map stuff.
And then we have sometimes we have special stuff, like how many classes have you loaded
in?
665 classes.
OK, so we told about extending it, right?
That's the title of the talk.
So what's going on?
Well, native memory track in the day, we've got hotspot, native memory tracking, which is the
Java virtual machine, right?
But we don't have it for the core libraries, where the core libraries.
We have a bunch of Java libraries, which we then write C code 4, which A and I.
And then you can, for example, make ellipsic findings.
There's, you know, maybe you use like the inflated deflator stuff in Java.
That's actually calling out to C.
It's stuff that we written.
We don't track those memory allocations, even though they're native, right?
We got the new form function and the memory API, which let's do this C Java interrupt
stuff without J and I, right?
And we don't do any tracking on this either.
Finally, we have the third port libraries.
So you see thinking about when you are writing your user application and you write
J and I findings to something, right?
Then you, then that third port library that you're interfacing with, might be doing native
allocations.
We don't track those either.
So we're going to imagine a new tomorrow where you get this native memory tracking hotspot.
You get it for core libraries and you get it for FFM.
But you don't get it for the third port libraries.
And if you think back to what I was saying earlier, that this whole category thing, that's
something that you were supplying, right?
So you kind of, this is kind of like a cooperative process.
So asking like third port libraries, C libraries, which have no control over to do that,
might not be so simple.
It depends on their design essentially.
So when I try to implement something, I typically ask myself a very simple question.
What's the minimal change that require for this change?
And I kind of want to go on a small journey with you where we kind of try to figure that out.
So in order to implement the feature, you kind of typically have to know some of the
internals of what you're implementing, right?
So I've been talking a lot about categories in NMT in hotspot, we call these mem tags.
So when you're doing a malloc, you supply a mem tag.
This mem tags can also be seen as indices into a statically sized array.
So you're going to allocate some array, each entry in this array have the statistics for
a specific category.
Mem tags, they're put into each malloc at the start via a header data structure.
So you can see that we have this header here, that's 16 by its long.
Nothing bites most specifically because malloc requires your allocations to be.
I think it guarantees that allocations are 16 by the lines where you want to keep that guarantee.
So let's say that you want to add your own mem tag.
How do you do that?
Well, you would first go into the appropriate C++ source file and you're going to go into
the in-nom definition and you're going to add your category and you're going to recompile
your JVM.
Now, if you're into interfacing via FM, I'm pretty sure that you're not going to tell
your users to like, oh, just recompile the JVM with this new category, right?
So we need to find something else to do this with.
But all functionality in NMT is summary mode and there is a data mode, we're not going
to go into it.
But all the pants on this, handing out this mem tags, perceiving this mem tags, right?
So can we find a way to just give Java and see access to mem tags?
That's the core of the idea, so here's a short plan for attacking the problem.
We're going to ditch mem tags as you know members, because I've said we can't recompile
this stuff.
Instead, we're going to make them dynamically createable.
Then we can expose them via an interface in JVM.h, which is where we expose all of
the JNI methods for the native libraries, especially at the core library bindings that
we have.
Then what you can do on top of that is just add a Java interface for FM, so you can have
some interface which does this Java JNI calls for you.
Okay, so we have a plan, let's enact it.
We are going to need a few things for this dynamic, createable mem tags.
We're going to need lock free access to the memory tag you're counting, this is because
we don't want to do any synchronization when doing this statistics, right?
Of course, we said we want to be able to dynamically add this mem tags, so that's going
to be another part of the solution here.
And use this on extra, we'll come in here, we should really only use as much memory as
needed for the use case.
And this can be some serious, of course, we don't want to waste memory, right?
It makes sense when you look at what I've actually done for this.
So what we're going to do is we're going to make mem tags for bytes long instead of one.
So today they are one byte long, that means you get two hundred fifty six mem tags, probably
two few, right, if you're exposing it to so many of the more users.
But two to the power third to two, mem tags, that's definitely enough.
The other thing we're going to do is we're going to replace our statically sized array
with something which can grow.
So the thing is, it has to be able to grow in place because it's going to be very difficult
to have an array base that moves around while you're trying to do this lock free changes.
So what are we going to do?
We're going to do the thing where we take a big chunk, our virtual address base.
And we don't implement, use the linear bump point allocate on top of that.
And that means that we can have very gracious behavior where we resize by just paging in
another small page, which is just like 4k, 64k.
And it allows for quite graceful failure if you run out of memory.
So you try to paging a new page and it says, oh, oh, you can't do that.
There's not enough memory.
Then I can just return like a mem tag standing in for like the other category.
And the user is going to be known to the user about this.
We're just going to have like a mem tag factory, which takes strings returns mem tags.
So the name is going to be your identifier in this case.
How are we going to implement this mem tag factory?
Well, I thought, you know, let's go for something simple here and just have a dual table.
So we're just going to map the name to the mem tag name being a string and the mem tag to the name.
Simple closed hash tables with some three saving tricks applied.
So by closed hash table, I'm basically talking about the type of hash table.
You would be implementing in the university, right?
You have your buckets and you go into linked lists and the linked lists go into the
pointers to the keys and values, right?
And I thought, OK, actually seeing this, they can probably be done on their lock,
because I'm not really expecting that we're going to be creating these mem tags
of finding out what their names are that often.
We're probably going to create a mem tag, still be the way somewhere,
and just apply that mem tag over and over.
For the data structure layout, I was thinking that we are going to have our category statistics.
Because that's really the key thing here, right?
Those are 64 by it's long.
That's exactly one catch find, that's very nice, because it avoids full sharing.
We are going to use four byte indices instead of pointers.
So you can see here that I'm talking about these entry reps,
I'm talking about these string reps, right?
And that's because we can use recisable arrays for stuff when we're under the lock,
because it's fine if they move around.
We don't want to use pointers down, of course.
And the nice thing is that we can use four bytes.
So that saves us four bytes per reference.
And if you're trying to do some kind of contiguous mapping here,
you kind of want to shave off these small bytes,
use the fit stuff in, so you get better cache behavior.
I call them with this, a negative is that you get a four
giveaway to limit per array, of course,
because you're not going to be able to index it out if you have one byte.
Access, I find this unlikely to be an issue.
There are ways around this where you can say, well, you know,
actually it's not, you know, index plus of base plus offset.
It's actually, you know, base plus the offset times some alignment and stuff like that.
If you look at this as a hold, this is really when I thought I would have a mouse pointer.
So I'm going to have to try the pointer for you.
Here's our, here's like a function, make tag if absent.
As I said, it's a closed hash table with just hash module,
but the bucket size to get into our quite tight array of buckets, which are entry reps, right?
So all the four bytes. And they point into this other array where we have our linked lists.
So the nice thing is that, you know, you can just put your linked list nodes into an array.
You can have your array as an allocator. And that makes things quite nice, quite tight.
If you actually do have to iterate through the array, if you know, if your linked list has several nodes,
you might, you might have the next node in cache and so on. So it gives a better performance that way.
Finally, you can, you can see, you can't see that actually,
but there's our string array over there holding all of the names.
Pretty simple stuff. Finally got the stats of here for directly accessing this.
The other thing that you can't see, which is where we store our statistics.
And that means that you basically can skip this entire table mechanism and just access it directly,
which we need, right? So that's really all of the kind of, if you implement this in code,
it's not a lot of code. You get the dynamic, you create a bold amount of tags.
You get everything you need in order to kind of stop pushing this out to see, pushing this out to Java.
So let's see what that looks like when we export it to see.
So for the JVM.hapi, I was kind of imagining that we're going to have an array in a style API.
So that's when you kind of think about allocations as belonging to an array now.
In other words, they belong to a mem tag.
I think this kind of abstraction, which it's a fairly thin abstraction,
but I think it's going to be quite familiar to devs who are used to the FFM API,
where everything is structured in this array in our space, OK?
So what does it look like? It's just this very simple thing.
You got your JVM, make a arena, gives you an arena back.
You pass in this arena to all of your arena lock, arena free, and so on.
Super simple stuff.
So the next part, if you got an API, you better use it, right?
So let's say what that looks like.
So the core thing here, if you're thinking about it, is basically,
I need to get this arena and I just need to replace all my allocation calls.
And you're going to have this kind of interesting problems regarding the lifestimes, right?
You need to make sure that the arena creation is done before you start allocating,
because otherwise you're not going to have a valid arena.
And there are basically two cases.
So what I did is I went into our lipstick bindings, and I ported it over to this new API,
and I found that there are basically two ways of doing it.
Here we can see something which actually implements a Java native method,
and it's a static initializer.
So if you have a static initializer, which you know is going to run before everything else,
right? You can just make the arena there, and then you can change all your local locations
to use your arena location API.
Super simple.
What if it's not that simple then?
Well, then you're going to have your other case, which is this.
That's also quite simple, right? We're just going to say,
when we're accessing the arena, we're going to do so through an accessor, and we're just going to check, you know,
is it not initialized, then we're going to make the arena and set the initialized one and return it.
So we only take the heavy lock taking code in the worst case.
One thing that you might be wondering about, which I haven't mentioned, is
this like global variables or something like that, and the idea I have here is essentially,
no, this is this is something you're placing your C file, right?
So it's like a file, global variable, I guess you could say.
Yeah, that's a really basic trick that we've got there.
Okay, so I kind of want to talk to you a bit about F and FM also, right?
This kind of what we really want to have, I think.
So the idea I have there was we're just going to expose an empty via G and I.
We're going to have a set of native methods.
They're going to correspond to this C API that we looked at, right?
We're going to replace because if, when you read the FM code, you kind of, when you dig into the
nitty gritty, this all comes down to a few some misconcepts, of course.
So if you can just replace the usage of this son misconcepts,
of course, and instead use an mt, that would be great.
And then the final thing we really have to do is we used a quick hour
arena with constructors, new constructors, which take strings as names, and then we can know.
Okay, this guys really want to use an mt, so we let them do that, right?
So this is where I started hitting kind of a wall out.
There was a lot of the indirection in the Java code that makes this quite a bit painful to implement.
So but really it all boils down to this, and say, okay, memory zero thing, you know,
if I had kind of, if I knew about scoped values before this, this would have been a lot simpler,
and you know, known that they were real cheap, because then I could use the puttoes on the stack
and stuff, were really simple, really simple solution.
What I finally kind of want to talk about is like, okay, so we have this, we've just gone through
like a bunch of code and stuff, right? But what does this mean? Why should we care about this?
And first I want to talk a bit about just hotspot, hotspot DM, right?
I think it's really cool that we have made tags like cheap, we made them dynamic,
it means that you can do some things like you can have namespacing, right?
So you could say, okay, let's say I want to create a bunch of global race in the C2 compiler.
I'm really worried, like how much memory do these global race take up?
You could imagine that you pass in like a super group mem tag, and you can get local
allocation profiling for that. That's really exciting. Another thing I talked about was
namespacing. Now the sole identifier for an identity mem tag is its name so you can start doing things
like adding peer exercises, say, well, GC dot core table, for example. And this allows us to do
some mixing and matching here. If you are a hotspot DM guy, you're probably quite aware that
sometimes you pass in the mem tags as template arguments, that's of course not something that's
going to fly if it's not statically defined, right? So today there's no problem tomorrow,
there's no answer to that. So for I think this is thing is just to keep the number of
definition there. And finally, okay, the final thing here is like, we're talking so much about,
this is so exciting. We can do so much cool stuff. That's going to require some sort of
consistency rooms and the things like that. So everyone who's like, you know, developing hotspot
or kind of like have to have the around there some campfire and talk this out and figure out
like what should we be doing. Finally, I just like to talk a little bit about what does this
mean for the JDK. And I think the best thing is that we're just going to have so much better
analysis of memory issues here, right? We're going to have to be, we're going to be able to split
out like okay, which area of Open JDK is really doing these allocations, what's going on here.
If we have easier ways of doing foreign function memory stuff in FFM, if you can get more
data from there, I think people are going to be less scared about doing C interrupt, meaning like
maybe it's good to have better memory analysis here. And I'm really excited about the idea that
M&MT is going to have like more possible input, right? So if you could switch from our current
reporting format, which is very like meant for humans to really think and you do something like
XML, JSON, CSV, whatever, and kind of just give that tooling back to the users for them to develop.
I think we could see some really cool analysis coming out of this. So that is really all I have today.
Thank you very much for listening. If there are any questions.
I still don't understand how this works, it's the quality of the race, where we need to
include certain talking libraries like C, but not fast, right? We have to change the old dollar
system to explain it. Yeah, well, you mean, are you saying in the code that? So you have, you have
to see, you have the hard-bust labor code, right? And you have the C code that Joel and I'm talking
to that we write. Yeah, so those parts are really hard to switch. So what the nice thing is that
if you have something like, you know, leave a lipstick, right? They actually expose like, hey, do you
want to use a custom allocator just hook in here, then you get it for free? If you want to do something
else, right? Then you're not going to see it and switch out hard-bust, right? But then you also,
what I, okay, so this is actually entirely separate. So this talk about what I hope that we could potentially
have is to do some sort of LD preload them. So we, and then maybe we can do some stack walking to
see, because if you have the debugging information, you can kind of automatically infer what memory
tags something should have. So that is what I think is a very cool thing to do on top of this.
So you have to say that louder. It works with third-party libraries. Exactly, yes. Exactly,
so that's, you know, that's a really, you know, that would be fun if you could extend it to third-party
libraries and that way, exactly, yeah. Any other questions? Yeah? The memory extension is a powerful
sharing. The memory extension for our sharing. Oh, that's the point of provenance thing, right?
Yeah.
I have no clue. That's a really good question. So that may be, I don't know, where-
Yeah. Yeah, that's a good question. I don't know. That's a good one, yeah. Anything else?
Yeah? Yeah. Yeah.
Yeah? I don't think there's any extensions of the standards. It's just, yeah, it's JVM underscore, right?
So it's no problem, I think. Yeah. Okay.
you