Get there, I'm going to go.
Good.
Now it works.
Good.
Now we'll move.
Again?
Now.
Oh, that is too loud.
Is it better now?
Thank you very much.
So.
We were coming from that idea that we have a bunch of LVMI options.
we have a bunch of LVMI files that we feed into org and org would figure out which of them
we need to run the code for the symbol that we were asking for.
And it would take these materialization units and compile them to native objects with the
LVM target for the LVM backend for the target that we need.
And then it links them in memory and loads them for execution so that eventually we get back
the address of the symbol that we did ask for.
And how does that work in practice?
If we have a hello world example like this with the main function and the print statement,
we can compile LVM IR for that with clang in this case and feed it into LLI, the LVM
interpreter.
This is the program that uses org kit under the hood and if we run that like this it looks
right it seems to work as long as we look at small examples.
So if we take something bigger like these are two for example it's a simple compression
program that is often used for benchmarks.
This consists of like 13 C files and one make file and the make file describes the build process
for that around 200 lines of code.
If we want to like usually we would just run make and get an executable and run that executable.
But if we want to get the IR code for the C files and run this through LLI we need to do something
here right.
We could say well let's tell all the C compile commands to embed LVM instead of binary.
And that works pretty well for the first seven targets.
But then we want to link a static archive that's a bit of a problem because we did not
emit the object files and there's nothing to link and even if we tried LVM IR has no counterpart
for linking archives or anything like that so there's not so much we can do except well we already
have almost all the bit code files we need so we could just compile the driver code manually
and now we have all the codes for the main program and we can run this in LLI.
It gets a bit more complicated but still works if we executed like this.
Great that could be a good approach right or is it really so nice?
Well there's a few issues here let's start with one that is maybe not so obvious.
One of the problems is that we load all code that start up and why is that?
Well before we enter main we have to get all the symbols somehow because we need to figure out where
is main and where's the code for it and also we have to allocate a memory for global variables
because main will think oh it must be accessible right and we also have to initialize all these
global variables because we come from a world where everything used to be static and compile the
head of time. There is a way to get the first two from the NLTO summaries that we can get from
the compiler and I showed that in a talk long ago on the LLI and deaf meeting but the third one
is really a problem and it won't go away because initialization can call arbitrary code and
are executable so we can just generate that ahead of time so the status quo really is we load all
code and start up. We'll be coming back to this in a second let's go first to the previous slide again
and see what was maybe another problem here well of course the MLVM was the big problem right
we can't just do that for real world projects we have static archives we have dynamic libraries
we want to pass linker flags all of this would not work together with emitting only LLLM code so
all in all the observation is not necessary from this only but also from here we need this build
process and it's quite complicated because our platforms are a bit messy and the build process is
what holds everything together when we look at other projects and how they build their code for
example Julia or come in this implementation class that you selected they are built on the dynamic
execution model and they bring everything they have everything included that needs to be done to build
the code they have first class trip support but cc++ rusts with they don't have that and most of all
we are missing build system integration so what could we do well the idea is that we could build
a LLM plugin to somehow connect these two worlds because plugins work very well with LLM based
compilers and they're easy to be used with existing build systems the project is on GitHub and
I code is here on the slide and the idea is this we hook in the compiler pipeline very early
early as in before we do a lot of work with the code and we cut out all the code and put it somewhere
like on disk maybe and then we inject the grid loader instead of it and otherwise we just compile
everything as regular into a kind of shallow binary after that if we do that for our bc2 example
it looks a bit like that we pass the plugin in the cflex here and this will obviously
close no problem as long as we use the compiler like clang because it will not break our build
and this works not only will make but we see make cargo if you want to have a few samples there's
CI GitHub actions with one example for each project and that should be yes everything here
so back to the idea that was the first part what about the next steps we cut out code and
inject the grid loader how would they look well let's say we have a function like this bc2
compressed block looks like that it's a lot bigger but they didn't fit on the slide
and now we would replace that function body with a grid loader so that would maybe look like
something like this we have three blocks an entry a materialize and a call block in the entry we say
oh did we already materialize the code for this function if so we go to call and just invoke it
otherwise we go to the materialize block and call into the runtime and instead of
requesting a function by name we request a number here and build an ID system around that
that's basically like thin LTO summaries have ID's global unique IDs that identify functions
instead of names because that's a lot easier and we can just store all this information
that we need to identify which file it is in and what code we need to load into the study binary
this code doesn't live in thin error we also have to inject a few more information
because for example the bit code file that we store the code in and we append a global static
initializer that will register this file with the runtime so that it knows where it is
for regarding the last point what do I mean with a shallow binary how would that look
if we look at the call graph of our bc2 example it looks about like this we have a main function
over here and all the other things are functions that are reachable that's a regular executable
I added boxes are functions and arrows are calls and now imagine we replace all function bodies
with a dead loader well then we also remove all the calls right and we look like that
almost because we also run that code elimination of course and then it would run then it would look like this
um what remains are the exported entry point of our translation units like that's like the
public functions in in our c files and this is great for the iD system because it's a lot fewer
entries than before and it's also great for a bi compatibility because our translation units from
the outside stay as they are they don't differ from regular compile compilation units and we can mix
and match these with regular ones so if something doesn't work yet we can just say okay let's not
audit it this object file we just keep the original one for in in this case
exactly and all the remaining functions are basically really just symbols with a little
stop attached that would load the actual code and then that is something we see in the binary size
the original binary is 190 kilobytes on the left side and the audit kit compiled one is 43
only and it's important to note that this is something like that the size reduction is really
only in the text section or maybe exception data or something like that but not in a data section
because this is another idea and the concept that we just leave all the data all the global variables
in there because that means that we can solve our status quo we don't have to build these things
on startup anymore we can implement real laziness because all the things that we need
upfront at startup are already in the exit and the static executable
on the project there's a benchmark mode or some scripts that run benchmarks it's about
outputs things like that it shows binary sizes it shows like average compile times
it doesn't get much faster yet because LVM compile times are not such a big part when you compile
your C++ and run times are a bit slower of course because we have to materialize this code
on the fly but it's not that big of a difference if we look at the run times there is a proof of
concept release this is really like super early stage there don't expect anything to work
but there's two types of run times one static out of process run time where
you can link a static archive that would implement the run time functions materialize on
register and that would talk to a demon auto to D process to do all the grid compilation because
this is pretty heavy like the run time is pretty big it brings a whole LVM compiler
and the dynamic in process run time is just a shared library but that makes issues sometimes
like for example rust developers don't want to link clips of C++ in their code and things like that
there's also issues when we compile LVM binaries with that because it confuses symbols
and last but not least there is a fast-blocking auditory with that I would say
that's kind of a novel approach for running orchid at scale because you can in principle build
projects of any size with that and not just small examples at least I don't know any other
project that does that so far yes we reduce binary size and compile time ideally
one idea is that incremental builds might be able to avoid relinks because if you only change
the implementation of a function then the actual object file will not change and the
linker doesn't need to do anything actually we just load a different bit code file from disk
and we can mix a match objects so we don't have to do that for the whole program we could say
there's something that doesn't work okay then let's don't do it not do it here
and the real core part is free to leave global variables in the study executable
so that we can support real laziness here's the project and if you have questions then we have five minutes
yes
you come to me say that you reduce the binary size because you have all the because that
nice on your file system and you just keep into the application so I don't know if this is really
something for production actually but the point here is more like that if you reduce the binary
size then you also reduce the link size and if you link huge binary is like chromium or something
then link times really huge and if you link only every tiny bit of it then like like the
edit compile tests cycle gets much smaller because if you only link a few megabytes instead of a
few gigabytes might be benefit
if you're responding it to the runtime then many starts it starts to get compiled by the runtime
exactly the idea is that the static executable has none of the actual code it's just like a frame
that is executable and knows where all the code is that it needs and then when it starts it will
first of all initialize global variables and we'll call all these constructors and for that it
will start loading code and then it will enter main and go the regular code path
just continue this seems like across platform compilation as well so for example if I compile
and basically I have an output the edit VMIR and it is we are able to use it for any application
because it's equivalent only to compile it I know LVMIR is not platform-agnostic it's very specific for
platform yeah yeah the question right the question was if this would be a cross-platform solution
but it is not because the IR is very specific to platform yes if I had correctly you're now running
the like global constructor functions delayed or an amazing instead of it started
yes the question is the global constructor constructors need to be run yes that's true that
doesn't work at compile time and yes this is just like every other code that runs
does it still run before main or executing yes yes global constructors need to be run before main
executes because main expects everything to be initialized but in this concept global constructors
are not different than any other function calls it's just calls into the runtime and it will
find the code and materialize it and run it is you've initially mentioned Julia
other languages that have written plus jits support is that the pair of comparison because
like the language has first class of jits support typically does tracing or some sort of like
runtime optimizations can also like invalidate code so I think like it's it's a different
idea than like what you're following here is the correct assumption yes the question is if this is
if it's fair or or reasonable to compare dynamic languages with study relief wild languages and
yes it's not it was more meant as a comparison in what other built systems are there for for
languages that use objects so far yes yes yes the question was what is the overhead for the dynamic
loading and it depends it depends if you have endless small functions for example or if you have
reasonable size functions for example and there's no threshold here implemented it could all be
done it's just not it's just an experiment so far but for like this is a real code bc2 is a real code
and here the runtime difference is not very big but there's also this is also a trick actually
it's it's bigger because this is using the tpde back and for all the others that is like very fast
but it's any other time just it's good for baseline jitter
can you say that again how does lb and auto jit affect runtime performance of larger projects
probably not so much probably like this as I said that the difference is if the code is in many
big for many big functions or only these small functions everywhere thank you there's everything
you can ask me more questions in a whole way thank you really