All right, hello everybody, I'm glad to be back here again, finally.
I don't practice my talks, so this might fit and it might not, so we're just going to
get going a little early on this.
I'm going to talk today a little bit about how we use invoke dynamic and J Ruby, the things
that work, the things that don't, places where we see that there's possible improvement,
and hopefully you'll learn a little bit about how invoke dynamic works for us.
Basic contact information for me, probably the most interesting thing there, up until Jet
July, I was working for Red Hat, they were funding the development of the project.
Since then, we have been building our own open source support company to help keep J Ruby
going to help fund other open source projects, just finding a way to connect up commercial
concerns with open source projects like J Ruby so that we can keep the developers funded
and give the companies the support they need.
So I got stickers in business cards, if you're into that kind of stuff, J Ruby stickers, and some
information about the company too.
So my role today is now kind of two things, I am still one of the J Ruby leads, so I do a lot
of the core development, a lot of the research, experimentation with all of these different
JDK projects, I do most of the community outreach, make sure that pull requests are going
through, make sure we're finding the right people in the community to help develop the
features we need.
Also, a co-founder of the new business, HETIUS Enterprises, we are focusing right now on
providing commercial support to J Ruby users, and it turns out there's a lot of large applications
out there running on J Ruby that are very excited to have us on their team, essentially.
And hopefully trying to bring that to other open source projects.
So if you or someone else has an open source project that you know a lot of companies are
relying on, we can probably find a way to help get some funding organization, funding arrangements
happening.
It's generally days for us, but that's what we're looking to do.
So J Ruby, pretty straightforward, hopefully everybody's heard of it by now.
There's Ruby on the JVM of course, we focus very, very tightly on making it as much like
the standard Ruby experience as possible.
All the same command line, all the same command lines work, all the same libraries work.
If it's pure Ruby, everything should just run great like Ruby and Rails, all the different
database frameworks that are available for Ruby.
And if we have native libraries, we even have FFI support so that we can do the same native
calls and native integration that C Ruby does.
It's also just bringing the best of the JVM to Ruby.
We have been pushing the edge of Ruby performance.
Ruby concurrency and scaling for 15, 20 years now of JVB being able to run all these applications.
And that means leveraging all of these different Open JDK projects.
Every single thing that you would see in this room or at the JVM language summit is absolutely
useful and important for JVB and for us to make a better Ruby experience on the JVM.
This question always comes up, so I'm going to just answer it right now.
One about this Truffle Ruby thing, I thought that was the way that JVM Ruby was going
to go.
Truffle Ruby has very different goals from us.
For those of you not familiar, Truffle Ruby is a Ruby implementation using the Truffle
framework on top of Rall VM.
JVB is a JVM implementation of Ruby.
We want to run on every JVM, not just Rall VM.
We want to run on embedded environments and Android and everything.
So we focus on doing as good a job on implementing Ruby as we can with what the JVM provides,
with what the JDK has for us.
Truffle Ruby being limited to Rall VM, being kind of focused on just two major platforms,
Mac OS and Linux, that's too limited for what we want to do.
And there are trade-offs with the Truffle framework.
You can get some amazing long-term stable performance out of it.
There's a lot of startup overhead, one up overhead, and a much larger memory footprint.
So again, very different goals.
JVB focused on being a JVM targeted Ruby implementation.
I'm not going to talk much about method handles, but in this, it's kind of the other half
of in-vokedynamic, the job aside of the API to build up these method graphs and the adaptations.
My talk from PhasDM 2018 is still very relevant.
It gives an intro to method handles and intro to an API I wrote, called invoke binder, that
makes it easier to work with method handles, and actually shows that you can implement an entire
little language in method handles, and it will compile and optimize really well.
So check that out if you want to learn about that side.
So invoke dynamic in JVB.
Envokedynamic really makes JVB possible.
It makes things optimize the way we want.
It allows us to do all of the different call forms and adaptations that we need to do.
It allows us to shrink down the amount of byte code that we generate so that the optimizations
of the JVM work better.
It really is an essential part of the JVM for getting a language like Ruby running.
Unfortunately, we also still need to support a non-involvedynamic mode in JVB.
Indie takes a little bit longer to warm up because of all of the method handle logic
that goes into it.
There's more profiling that's required to optimize and inline things.
So we still support running without invoke dynamic.
Hopefully, as we go forward, that will be less and less the case.
And if any folks in the room are working on improving method handle and lambda form performance
and memory footprint, that's what we're really looking for here.
So the first area we're going to talk about method calls, it's kind of the obvious thing
that we use invoke dynamic for in JVB.
And it was the earliest case for us to use invoke dynamic.
JVB was pretty much the first dynamic language on the JVM to make use of invoke dynamic.
We actually were integrating it into JVB before it was finalized and released in Java 7
and helped drive a lot of the invoke dynamic development at that point.
This not a straightforward as it seems, it's not just like calling into reflection and getting a method point.
We have lots of different targets, all from Ruby to Ruby to Java.
You can call Ruby in the native through our FFI.
Hey, hey, this got the room sound work.
And okay, yeah, yeah.
So lots of different types of calls, different paths from Ruby to Java, Ruby to native.
Validation and binding of all these Ruby classes and method tables can be mutated at runtime.
It's part of the dynamic characteristics of the language.
So we can't just bind it once.
We have to be able to fall back.
We have to be able to detect changes in the Ruby class structures.
And then we need to be able to bind into all the different overlords of Java.
We need to take calls from a dynamic language, turn them into a static call to a Java thing
and try to make that all wire up correctly.
And then all of the adaptations along the way.
Ruby can support optional arguments, variable length argument lists, keyword arguments.
Ideally, we don't have to box all of these things and throw them into a hash table for keywords or an array for
our orgs.
Ideally, we can get those to go straight through on the stack without any doing extra allocation
and putting more load on the jit to optimize for us.
So Indy really does allow all of these adaptations to happen in J Ruby and to inline an optimized
well.
Here's the eye chart.
I'm not going to go through all of this.
A few key things here.
I mentioned that we have to invalidate if the type structures change if method tables change.
That is an active invalidation.
So whenever we cache a method at a call site, we grab a switch point which is basically an abstraction
around JVM safe points to say if this class changes, if a new class is introduced or a module
is introduced, if a method table change happens, go and invalidate all those call sites
and the next time we'll go back through, we'll get the new methods and re-catch it.
The more complex the binding is, the more adaptations we need to do, the longer the chain
of method handles from our call site to the target method.
Usually those will all get compressed down and inline and optimized away but there are thresholds
that we can crash.
If we create too much complexity in a given call site, we may end up not inlining those
pieces.
Now we're actually performing worse with invoke dynamic because we've actually broke in the
inlining process.
And there's obviously a lot more opportunities here.
I mentioned that some special types of calls in Ruby like doing super class calls or refined
calls, which are methods that are patched within a certain scope.
Those don't do any optimization currently, we're still adding those pieces, hopefully
in the upcoming versions of J Ruby will finish those pieces as well.
So a simple example in Ruby, we've got a full method, the full method calls bar and bar calls,
the dump stack method so we can see where we actually are in our execution.
And I'm running this a couple times in a loop and we don't have the extra overhead there.
J Ruby supports an interpreter, we're actually a mixed mode runtime on top of the JVM.
So most code will run in our interpreter for a while, eventually we will jid it to JVM
bytecode.
And the interpreter, this is what it looks like.
You see these specially named methods, interpret block, interpret method.
This is how we take our interpreter frames and the JVM stack frames splice them together
and produce a normal stack trace.
It's kind of a complicated way of doing things, but it allows us to avoid the overhead
of piling everything to JVM bytecode when a large portion of it will be called never or
maybe only once.
If we actually turn on the JVM jit compiler that produces bytecode, now we see that our Ruby
methods here have actually turned into JVM stack frames.
So at the bottom, the block that we used in the times loop, and then above there the
full method and the bar method.
And this is actual JVM stack frames here.
I use a special little syntax to encode Ruby information about the method.
So I know it's a deaf, it's a method.
The heart marks it as being one of our Ruby frames that we want to pull out.
The number here is basically which not index of that name in this particular script so that
if you have two classes in the same file with the same method, we don't overlap on those.
Now if we keep going with this, the version that doesn't use invoke dynamic and you can
see we've got just a caching call site, basically an inline cache that we use.
If we're not using envy, we just go through all of our own plumbing and then there's multiple
layers of adaptation that goes through our utility classes and finally we make the call.
If we wear that all up with envy, this job of stack trace reduces down to that.
All of the stuff in between the call and the receiver is done as method handles which
turns into lambda forms in the JVM.
Those get compressed down, it generates a bit of code, inlines the whole thing.
Now we can see we're actually doing these calls directly and we can get the optimizations
we would expect from inlining food and bar together, for example.
But this actually hides the complexity.
If you actually force the JVM to do a stack dump, you're going to see something that
looks more like this.
These are the layers of lambda forms that secretly exist between the call and the receiver
to do all of those adaptations and they have these horrible names because it's little bits
of generated code that are created to let the JVM do its normal sort of optimizations.
The same optimizations it does for regular JVM byte code, but as part of this invoked
dynamic method handle adaptation, then it can just do the same, use the same jet process,
the same profiling, optimize and inlining it altogether.
But this is a challenge for us when we deal with things like profiling tools which I'll
talk about later.
Going back here, changing the structure of these methods a little bit, here we're calling
on the food side, we're calling with one argument, it's going into a version of bar that
has variable length argument list, still inlines all correctly, all those adaptations
of turning the one argument into an array of arguments works just fine.
We started running the problems if we get more complex than this.
So here is the same stack trace, but showing where we have the times call, we'll be
fixed num.times, the loop we did, that calls back into the Ruby code.
Now we have these extra things because there's no way to do invoked dynamic calls from Java
directly.
So we have to call through our lock interface, that has to do some juggling of arguments
and moving things around, and then finally it gets back into our compile Ruby code.
This is an area we're looking to improve, try to find a better way that we can do invoked
dynamic from Java so that we can get the Ruby code in line back to the Java code just
like vice versa.
Another example, adapting Ruby's many different ways of doing argument lists to a target
job.
Here we're calling the same dump stack method, but because we don't know how many arguments
there might be in this incoming array, we're splatting this argument array, we have to go
through some extra additional adaptations, we can't do the inlining all the way to the
target method, and this is more of just working in J Ruby to try and bind these two sides
together, try and avoid having to go back into our utility code and make sure it's all
invoked dynamic.
And then other call forms that we're still working on exploring, I mentioned doing dynamic
calls from Java, I've opened the suggestions about ways we can rewrite the Java implementations
of these methods to do dynamic calls, I don't have a good pattern for this right now.
We also have the same problem that Java does with lambdas.
If you call a single method with many different lambdas, well, that lambdas dispatch becomes
mega-morphic, you can't inline through all of that, and the JVM currently does not profile
across that mega-morphic call.
We're looking at doing some of our own manual specialization where if we know that it's
a simple method that receives a simple block, well, let's admit a new copy of it, let's
actually specialize and split that method into another version, then the JVM can see through
it.
We really don't want to have to do this on our own.
We would like to be able to hint to the JVM that you should specialize this path based
on the lambda or the block we passed in rather than each piece along the way, find that
common path and inlining.
We've always thought about and wanted to try to do numeric unboxing, the lure of partial
escape analysis has kind of teased us for so many years that we wouldn't have to worry
about all these boxes.
Now possibly with Valhalla, we'll have value types, and we can double up our arguments and
be able to pass a native version and the box version.
But still, we don't do any unboxing.
We're kind of hoping that the JVM will catch up with what we need to be able to represent
objects or represent primitives as objects, and optimizes.
So the next area that we started using in both dynamic and JVM is for handling Ruby instance
variables or fields basically.
So here's a simple class.
We assign two instance variables name and number to the values passed in to the initialize
constructor here.
Adder accessor is a Ruby feature to just add accessors, getters, and setters for those
two fields.
And then an example of how you can dynamically add new fields to a class.
This is all at runtime and we need to be able to efficiently represent objects in memory
even though the set of fields they contain might change while we run.
So instances variables are basically dynamically allocated object fields in a typical Ruby
implementation.
The way that we get around having this be a separate box of values, a separate array we carry
along, is that we statically look through the method table, look for all the different variable
accesses we see, and then make a best guess about what the shape of this object is going
to be.
That allows us to put most Ruby instance variables directly into Java fields, even though
technically there's no declaration syntax for an instance variable.
And then if we if we turns out that we're wrong we still have our spill array that gets
carried along with an object.
Most of the time objects will be pretty well behaved and won't do a lot of dynamic instance
variables.
And then in both dynamic can actually have us wire up our access of this named field.
This instance variable straight into the Java field and the objects we cut out all of that
access look up all of the validation of it and actually go straight to the memory location
to get the Ruby field.
We also use invoke dynamic heavily for managing Ruby constants and globals, which oddly enough
both of these are mutable, constant values.
So here we have the debug constant being set to true, a debug global variable being set to true.
When you declare modules and classes in Ruby that's actually just assigning new constants
to a module object or a class object.
And then at the bottom there is a fully qualified access of that BAS class in the middle.
So for constants and globals, constants are scoped, lexically scoped, and then also scoped
within the class hierarchy.
They can be already, but it's typically not done, it's considered bad form in a typical
Ruby application and usually you'll get warnings about it.
Formals on the other hand can be modified all the time, there's no warnings for that,
but usually they end up falling into either is constantly being mutated or it's never
mutated, like a debug variable, probably not going to be turned on and off at runtime.
The ending call sites actually work really well for this.
We look up our constant value.
We use a global invalidator based on the location of the constant or the name of the global
variable.
We can get that value to fold in as a constant using the existing invoke dynamic features.
Similarly, we do the same thing with global variables, but we usually have a fallback in case
it's a variable that's actually being used to mutate quite a bit.
If it's being changed a lot, we fall back on a slow path so that we're not constantly
throwing out code and invalidating an entire call graph just because it's a mutable value.
There are places to improve here.
The through bar buzz is actually currently in J.W.B. free separate constant lookups, but it's
always going to produce the same result.
And if we're not doing a lot of changing of those constants in a typical application, it
really should be one lookup.
We can shove that all behind invoke dynamic with do-star byte code size.
So this is another area that we're working to improve in the future.
We actually use invoke dynamic for creating Ruby literal values.
Ruby has a much richer set of literals than what we can store in a constant pool in Java.
We have our numerics, of course, but we have a literal big integer, a big num format that
we need to be able to have as a literal value without constructing it every time.
Ruby strings are not Java strings.
We represent a string as a byte array and an encoding.
So we need a way to re-constitute that into a Ruby string object and ideally cache it
in place like it's a constant value like it was a Java string literal, similarly with regular
expression literals.
There are cases where we're going a little bit beyond this.
If we have arrays or hashes where it's all literal values, we should be able to just
tell invoke dynamic, create an array that looks like this, ideally share some of that
store, not have to recreate the entire structure of the array every time because we know
it has a mutable values in it.
Other things that we're still working on, things like other hack more complex hash
formats, composite types like complex and rational, but most of this we can easily put into
invoke dynamic call sites and have one instruction to create even a large structure like
a hash.
Do what those byte codes look like if you look in J. Ruby.
So at the top we have our fixed num which is a long, we just call into our fixed num
site that does a bootstrap, goes out and creates a Ruby fixed num object and then it's
cached at that point in the code forever.
The next one is a frozen string, Ruby has both mutable and immutable strings.
So here we have our hello string, we know that it's going to be UTF8 encoding.
The 16 here is basically an encoding that says this particular string is only seven
bit characters, so we can optimize accesses to it, and then Ruby also supports debugging
if someone accidentally goes and tries to mutate a frozen string, we can print out an error
that says where that string was allocated and let them know that they're not supposed
to be modifying it.
Similarly, the regular expression here, the execution is full, the encoding is UTF8 and
the 512 is basically just regular expression flags that are embedded in it.
I mentioned the array full of all literals, so this is an array that has two literal numeric
values, the fixed num one and fixed num two.
That gets shoved into a structure passed out to invoke dynamic and we can quickly create
that array without too much trouble.
Big nums of course we have here too, we embed that as a string, turn it into a big integer
that's inside of our big num object and only have to do that allocation once.
And then of course there's a range from one to ten as well.
One of the newer areas that we're playing with invoke dynamic is using it to encode string
interpolation.
Similar to the way that the JVM now does string concatenation using invoke dynamic, reducing
the byte code, making it a little bit more optimal.
We do the same thing, so we embed some additional information, the constant, the static
parts of that string interpolation plus a sort of map that says here we pull a static
string, now we need a dynamic string, now we need a static string.
Based on the size and structure of that, we do it a few different ways.
We can call it a number of different overlodes that will stitch it together.
We can just loop over all those values or we just fall back to a slow case, so we don't
create such a giant tree of method handles and overload the JVM that way.
This example of the string here, the value A was passed.
We embed our two static strings into the invoke dynamic call site.
Then we have the second to the last value here is a bit map that shows where the static
values and the where the dynamic values should go.
And then it's just a matter of telling invoke dynamic to stitch all those pieces together,
and they're stringed back out.
But then we only have this one instruction in our byte code rather than what would have
been dozens of instructions to create these strings, put in the dynamic values, and then turn
it into one string at the end.
We also use it internally just for some of our own runtime plumbing.
So when we create a block, we don't want to have to do that over again.
We use invoke dynamic to cache it in place.
We have heat-based local variables, blocks can close around a scope, and still mutate
values outside of them, unlike Java lambdas.
So we need to maintain a separate heap structure for that.
We use invoke dynamic to try and speed the process of digging into that heap structure
and finding those local variables.
We also have to support Ruby's ability to interrupt threads at any time.
So there's a poll that we would do to check and see, have I been interrupted?
Am I supposed to raise an error?
Am I supposed to, as this thread is supposed to kill itself now?
We don't want to constantly be paying a memory location for that.
So we do it with safe points.
If the thread is interrupted, we flip the safe points, the code de-optimizes, we go and do
our interrupt, and then we go back in.
That's obviously very heavy because we throw out a lot of code.
But in general, these are shut down cases, critical cases in a Ruby application, where
you want to cleanly tear down a thread, the only way to do it is to cause it to raise an
error or kill itself.
We also have other little constant dynamic things.
There's places where we still use our old inline caching, call site.
We create that with invoke dynamic and cache it in place just to save some trouble.
Now the last area here I want to talk about, this is kind of a cool thing we're doing
with method handles.
There are certain cases where we can turn a Ruby method into a chain of method handles just
to make it inline a little bit better.
For example, object construction in Ruby involves going to the class, telling it to allocate
an object, and then calling initialize on that object.
That actually is a method, class.new that we call, that creates that, that allocates the
object and calls the constructor, we don't want to have that mega-morphic new method
that we're calling through for every allocation.
So we can turn that into a chain of method handles inline all the way through the constructor
and not have to worry about it, not inlining.
Similarly, Ruby and recent versions added the ability to override not equals as the version
of not with the equals method.
We don't want to have to dispatch through the not equals to get to equals, so we do a
little extra magic with method handles to inline that all.
There are other places we're looking at doing this.
If we know we're calling one of the core loop methods, we can turn that into IR, and then
inline all the things back and have our loop inline with the method that we call.
And again, my PhosDM 2018 shop shows an example of this.
You can basically compile anything into method handles and it will all execute and inline
properly.
We're using it in small cases to sort of intensify some of these Ruby features.
So there's an example of the new method in the class with allocating the initialize.
We turn that into a call from the food method to a method handle chain that does allocate
and initialize as one operation and now it inlines and has the characteristics we need.
All right, I'm basically done here.
The big challenges that we have are with Lambda forms stack traces end up looking pretty
horrendous when you do a thread dump.
We need better ways to deal with that stack trace and more importantly, we need ways to
identify that those are not interesting for profiling purposes.
You might have a call from food to bar that goes through a couple different Lambda form paths
should be considered the same piece in a profile.
The more we use Lambda forms, the more complex this gets and we're putting a lot of load
on the JVM2.
We're also seeing a lot of job-alang and vote objects that are taking up additional
memory.
The more complex our call sites, the more of these objects are in memory, the more allocated
and that impact start up and run time.
I'll skip these two samples.
Again, in the last thing, no Indy from Java makes it very difficult for us to get the
same characteristics calling back into Ruby from Java.
We're hoping that the JVM is able to help us with this more in the future.
So future things that we're going to be doing.
We are going to be filling in a lot of these different invoke dynamic gaps in JVM, doing
better adaptations, getting wired up with Panama for native calls, really looking forward
to working with all of the different Open JDK projects to try and leverage as much as possible
in JVM and give you feedback about what we use and how it can be improved for languages
in the future.
That's all I've got.
Thank you.
We've got room for one question if you want.
Yes, right or right?
Dynamic constants at all.
Dynamic constants.
Do we use dynamic constants?
They can be useful for us for some of our runtime stuff that does not depend on the current
JVM.
But in most cases, we need to know what instance of JVM we're running, what we're fixing
in some class.
There's a lot of runtime data that we need for those dynamic constants that's hard
to get into constant dynamic right now.
So it's good for plumbing, but not really for the core of Ruby.
Yeah.
All right?
Thank you.
We're moving on to work.