So, hello everyone, as mentioned, I'm Luis Cielbaum from Synopsis. I'm a junior
compilet engineer working on GCCC, and I'm here to present a talk on understanding
the AI compatibility between compilers targeting Luis Cielbaum. So, for our agenda today,
we will talk about the why we started this project about the two implementation details
and how it works and then finally the conclusion. Let's start with the problem statement.
So, our goal is to be able to link
resupply object files from different compilers and be able to make sure that the
combination would work and our focusing is identify potential discrepancies in the ABIs
used by different compilers or syncopiler with their different options. We want to make sure that
we can identify deviations from the standard ABI, which is defined by resupply documentation.
So, first of all, what's an application binary interface? So, according to the Wikipedia,
ABI specifies an interface between the binary proposals for example object files or a program
that depends on library. It also defines all-level details, for example properties of fundamental
types, calling convention and main usage of registers, also defines high-level details.
And it is managed by compilers operating systems in develops of libraries.
But our focus is under the compatibility between compilers for a specific target.
So, within ABI, it defines resupply usage. So, what is it? So, it specifies a purpose for each register.
Here we have a list from the risked file ABI documentation that specifies each register
in their purpose. You can see that on the right, we have something called caller and call E.
So, basically, a caller means that this register needs to be saved before a function call.
And the call E means that after the function call in the call E function, the register that will
be needed to, needs to be stored in the stack. It also defines here, we have an example of the
function arguments, which is defined by an ABI, which means that these are the registers from X to X
17, expected to pass the arguments between functions.
Data layout, this is a really important thing about the ABI that defines it.
Specifies the sizes and alignments of fundamental C and C++ types.
It specifies our structs, unions and classes are laid out in memory as well as for the bits fields
or in the struct. And here, this is, again, a table from the risked file document that tells us
for each, it's the 32 and 64 bits architecture, the size and alignment for each data type.
Now, there's a really important thing called Indianness. There's two main ones,
lipland and in big Indian, there's also mixed, but real quick, big Indian means that a value is
stored in memory starting by the list significant bits. While the big Indian, it starts for the
most significant bit. This is important because if we have two object files, one leafland and another
in one big Indian, when we are combining each other, then we won't work because they'll try to
fetch the different things for memory. Calling convention. So, between function calls, we need to
understand which registers or memory locations are used to pass the arguments, how we turn
value we specify, even either on a register or on the stack, who's responsible for saving these
registers and the struct frame structure. So, here I have a really quick example. On the left, we have
a simple SQL that does a function call with nine arguments and on the right, we have the corresponding
assembly code. So, we can see the Y700 on CTS and line two to fifth on the same way. We have the
pull log of the function, which will allocate a space on a stack and save the return address register
and SZO register to the stack because we will be used during the function. And yeah, so for the yellow part,
which is between the line six to 16, is the actual preparation to the function call. You can see that
each value, it's been stored for the registers except for the argument nine, which is the value nine,
it's actually been stored on the stack and the stack. So, the rest of them will be stored in registers
from the A7 to A0, it starts from the end to first. So, line 17 is preparing for the return,
which we set the return as one to three. So, we will save the value to I5 and later on specify
to A0 register, which is what we expected from the ABA. And finally, there's the epilogue where
it's restoring the registers that is saved earlier and then exiting to the caller function.
Why barriers? Why barriers are really important for the ABA because if we have a C++ program,
they can interact with each other as long as they are ABA compatible. It means for, for example,
the header file, if we are calling a function, we are in C and calling a function in C++,
the arguments need to be the same, data types, returns, etc. Also for macros,
there are, there can be used together as long as they are compatible. It ensures functions
from library that can be calculated correctly, either linked separately or dynamically,
specifies all functions, our name and location and memory, and also the version compatibility
libraries. Exceptional handling for C++ is also defined by the ABA and systems. It allows
programs to run on different OS or other versions as long as they follow the same ABA.
Now, there are a number of projects we see below the goal, but slightly different.
First one is ABA dump, which is bought from the ABA compliance shaker. This one has a goal
of extracting ABA properties from libraries, and how does it do it? He analyzes the dwarf
debugging information from the libraries. Here have a really quick example, how it works,
it will extract all the data using a JSON format. ABA compliance shaker ensures backwards
binary source level compatibility of C and C++ libraries, how does it do it?
He uses ABA dump from the previous slide to analyze the dwarf information.
Lead that big ABA, the goal is to detect interface and capabilities in L shared libraries.
How does it do with it analyzes the L shared libraries in their associated debugging
information, again, with the dwarf, to build internal representations of expert functions and variables.
It identifies changes in function variables and their data types, for example, the return type changes.
ABA is cafe, so this one, the goal is to identify again with two compilers or languages they
agree on the same ABA. How do they do it? They do it by both brute forcing. They generate
random test cases, focusing on detecting ABA incompatibilities, and if they agree, and great,
if they don't agree, then even better because we just learned something. It's essentially an ABA
fuzzer, although it's more like an ABA fuzzer that detects ABA incompatibilities.
Now let's jump into the about the framework and let's start with the end, basically the summary
report, which is the output from our framework. This example is from GCC,
exactly from GCC 32 bits, which fly. So we selected a number of test cases to display here,
so the first one will be the most important one, is extracting the data types sizes.
As you can see here, for the signs shot, an unsigned shot, we have a volume of 1 byte of size,
and for an ink loan pointer and a float, we have the size of 4.
This is everything corrected to what we expected. Second one is a stock direction, which is
really important. We need to understand if the stock is going upwards or downwards in memory.
And according to the ABA, it's downwards, which is exactly what the framework extracted.
Next one is alignment of the stack. Again, the stock is aligned to 16 bytes, which is exactly
what we expected. Now, this test case is a little more tricky. So here we want to understand,
which registers or the stack is used when passing arguments from one function to one other.
So here we can see in the summary that the short short ink loan inflow, they all use the same
registers from 1 to 8 arguments. And then when we reach 9 argument, it starts putting the
values to the stack. Now for a long long interval, it's a little different. Now remember that we are
in a 32-bit architecture, and we saw that a long long interval, they are 8 bytes in size.
That means that we cannot fit a whole value into one register. We need to, the compiler will
split the value into and low and high. And this is exactly what our framework is telling us.
For the long long interval, we can only store four values in the registers, because each one will
be split in the low and high. And then when it reaches the stack, it continues to do the same thing.
It's low and high. Next one is more tricky. We will talk about it more later, but really rough.
This one for the structs, it doesn't go for the number of members, but the size of a struct.
So we want to understand what's the boundary of the size of a struct until it starts going
to pass by reference on the stack. And here we can see that a size of a struct less or equals to 8 bytes.
It's best on registers. When it's greater than 8, it is best on the on by reference on the stack.
Short short ink and long float, they use all the same registers, is your A1. And long double
use the same registers, representing these ways for a low and high for each one.
Now we also test to see if an empty struct went passing through one of the function
is ignored by the second pilot. And this is exactly what the different work is selling is.
Now, and in the test, this is what I discovered before we need to understand if our system is
little end and begin or mix. Now, for what I test of this is for a little end and this is exactly
what the test determines. So we wrote a volume and then it read from memory and from that
he can understand which one it is. Colour saved and calles saved. This is what I also discussed earlier.
We need to understand if the compiler is going according to the ABI, which registers are being
called or saved and which registers are being called e-save. The framework was able to identify
which ones are what and produce a report on it. Return registers is this is when we are going
from the calle function to the color function. So as a return, we need to understand where the
volume will be located when doing the proper function call. So this is the fact that we put the
framework telling us. So for a short short, in long and float, the register is a zero for long,
long and double the volume is split. Now, design choices. We decided not to use dwarf. So we wanted
to eliminate all the layers between the compiler and what we can extract from it. So and also from
different compilers might include information in the different way. Another thing is that we don't
want to restrict this framework to compilers that only use that's produced dwarf. So we want to
make sure that we can use this framework in any compiler even if he has dwarf or it doesn't work,
we use dwarf. We assume the availability of a compiler. When I say compiler, I'm mentioning the
whole full chain as a compiler and a template linker and a simulator to run the tests.
The framework is developed in Python and it basically uses generic C test cases and runs them on
the simulator and then extract to extract the behavior. It produces a summary report that we already
discussed. It's easy to extend. So the base framework, it generates C tests, it runs them in extracts
output. It's easy to add new analyzers and it's possible to share information between those analyzers.
We have support for feedback loop and what does that mean? It means that for some tests,
they are dynamically generated based on the results from the previous run. It doesn't all decide the
same test case. For example, finding the boundary where it structs our best between function
calls. We will talk a little bit more later for us to explain the example.
Now let's go into implementation details. So how do we extract all this information? We use
something called architecture don't be formation. We will extract the information from the current
execution. We will be able to extract the contents of all the registers and then the stack
and then also some header information. For example, size of a pointer, the register banks, etc.
This one is used for mostly for the colony convention. We need to understand the location of the
known special values when you're making the function call and it helps the analyzer reduce
how the compiler passes the arguments. Also for the caller and call these saved register
test case. Now let's go deep into one of the examples. The struct's arguments best in this case.
So as I discussed earlier, we need to understand how structs are passed between function calls.
We need to determine how non empty and empty structs are passed. The term in which
registers are used and identifying the which conditions the struct is best by reference.
So according to the ABI, aggregates larger than two times x length meaning the size of a register
are passed by reference in the replacement in the argument list with the address. This is great.
This is what we expect to happen but we need to understand if the compiler is behaving exactly like this.
So we need to identify the maximum size at which the struct is best by reference.
And so what we do, we will start by generating multiple tests, incrementing the size of the struct
by one byte. We will use source as a for started until the boundary is reached.
So for things first, we create a struct automatically generated a struct with one one one
and we add a value to it and then we do a function call and then we will have the dump of it.
So here we have a register bank zero and below we'll be all the register values.
You know we're in it on the left and green we have the specified the corresponding register to it
to help us analyze. So we can see that the value that we assigned to the struct is actually in the
register is zero. So this is good. This is starting to look good although this is not we have already
we haven't already which the limit. So we add another chart and so we do exactly the same thing
we look into the dump and we try to identify where those values are and we can
you know we can see that the value or the values are actually in the same registers one
to another which is great. It's still being passed in registers but we still haven't found
our answer yet. So we continue to do this until we reach a point where in this case with non
arguments we'll see something different in the same register. So if you're looking to the A0
there's something different there and if you look into the stack it's actually the stack value there
and if you look into the stack values itself you will see that all the arguments values are there.
So this is great. What does it mean? It means that we have reached the bottom of the boundary at
eight bytes it goes to two range in size. Okay great information so far so good but we need to do this
for all data types. It can be a little time consuming so this is where the feedback loop kicks in.
So we already have the information that the size of the struct greater than eight bytes are expected
to be passed by reference from the previous slide. So what we do is for the 32 bit architecture
an inch is for bytes. So a struct we two the int are already equal to eight bytes so it's expected to
be passed by registers. If we add a char it will be expected that it will be passed by reference.
So this is exactly what we do. Instead of going from one inch to the integer we start
all already we two int and we test it and if it doesn't reach the stack great we will add a char
and it is expected to reach the stack. Now there might be moments that that doesn't happen so
we have a fall through that if the expected boundary is not reached we increase the size by one
and we repeat again we have three inch and then we have a char in so one. So for this test case we
have the summary report which we already discussed. So we have the limit of eight is passed by
reference by registers when it's greater than eight there's a typo there sorry when it's greater
than eight is passed on the stack. Now what happens when we enable out of the where 14 points we have
some different information here you can see that the boundary we have some information about the
boundary being a limit of 16 and why does this happen is this above what is going on exactly right
here. So we have the size if less or equals to 16. So we're in registers if it's greater than 16
is passed by reference on the stack but it's only for double and why is that?
So according to the the risk file ABI a short content in 2014 point reels is passed in 2014
point registers if neither real is more than ABI flank bits why at least two registers
four devices are available what this is meant it means that independent of the size of the
struct if we have two floats two doubles float double or double float which the first one will be
eight bytes rest of them will be 16 doesn't matter the size it matters the number of elements.
So for this one we had to create a special taste case for risk file that will determine
this behavior so they all will have the same behavior if we increase the number of elements
in the struct resulting the struct being passed by reference this is exactly what the new
summary special summary tell us so a member last sort equal to two it's passed in registers
if when there's more than two members is passed by by reference on the stack and then below we have
the least of the the contract system members so a float float double double double double and a double
flow. Now for an empty struct how do we do this how do we make sure that an empty
struct is being ignored by the sequimpiler so we make use of a sentence of all in which is a
marker that indicates the end of a data structure so here we have an example the k k test case
we have a signal value which is coffee everyone loves it and then we have a empty struct so for each
argument possible argument that we we already know how many arguments we can we can test so this is also
information from previous tests so we will test for the out possible eight arguments so the first one
we will test for the first of the store empty structure then the Sentinel second one Sentinel empty
struct and so on so if you look into the that place and empty struct between citizen of
values for each possible argument position our registers should contain exactly so each
register argument register should contain the keyword coffee that will tell us that indeed it's
ignored by the signal bothers so if you're looking to the fourth argument position so we have
three three Sentinel empty struct and a Sentinel if you're looking to the dump for the
registers a zero to a three we have four keywords meaning that's indeed we are sure that in this case
the empty struct is ignore by sequimpiler so we test this for all of the positions
so this is a quick summary between the software 14 points and the hardware 14 point
do you already discuss it the other 40 points it's more density has more information but it's more
complete each everything that we want to know now there's some there's some situations where the
ABI extract you cannot be sure what it's extracting so for example here we have a simple
GCC extracted information everything it's great what we already discussed but when we put
playing we have some differences so if you look into line five and nine playing there is a warning
there so multiple value occurrences detected in t zero and stack being t zero the one of the temporary
registers why not this happen so the framework detected the same value the same special volume
in two different places and cannot be sure which one is actually being used for the other
argument the passing argument to a function so we will tell us now if we look deeply into this
which creates a simple set that sends nine values to a dump function doesn't matter the name
and we're looking to the assembly we can see that actually what is going on is the compiler
is saving the value to t zero and later on we'll be used to starting stack okay now this also happens
in gcc but gcc uses a different argument with different registers sorry it uses the A5 register which
is one of the arguments one so basically there's the same thing but it will start in the stack
earlier and then we'll rewrite the register so extendability portability of the framework
oops so we don't want to this to be exclusive for gcc or playing or whatever and the same
for the simulator so we wanted to make sure that we could easily add new compilers similar to
the way so we went ahead with the idea of having batch rappers each batch rapper will tell
the framework how to execute the proper compiler we can use the same compiler we
different ABI options and everything that will be placed in the rapper same thing for the
simulator in this case we are using QMU to execute the test and on the right we have a tree
of the folders and how it looks currently so we have inside scripts in rapper we have the compiler
and then each each rapper which will be the for example the seal this is the playing the
risk 532 and gcc risk 532 each one will have for the assembler the compiler and the linker
rapper same thing for the simulator which can easily add new compilers to it no portability the same thing
we don't want to be focusing only in a risk file we want later on to add new architecture x86 etc
so to do that we had to make sure that we for example this is one of the reasons we didn't use
more but to add that is we have a class in Python that specifies the register banks just specifies
the names of it so that when generating the report we will give you the correct names for the
registers all pretty and we have one assembly file specific which is also part of the architecture
dumpy formation this assembly file will be responsible for saving the registers to the memory
and later on bring them to as a dump it also have functions to reset the register to know
nonvolve so that we don't have trash on it now future extensions this these are the least of
potential extensions we are working on so currently we are working on supporting 64 bit
long double and vector types so let's jump to the conclusion so here I presented the framework
ABI extractor is able to extract ABI properties for a compiler and dumps the information in a
human readable format it simplifies comparison between extracted ABI from different compilers
and it helps with the validation that the ABI implementation for compilers is correct
now where where can you find the ABI extractor so the ABI extractor is open source it's available
in our synopsis repository but it's currently dummy repository we just got to approval from the
legal team yesterday so in the following days we'll be working on the publishing the source code
so for now if you're interested what you can do we subscribe to the open issue in the repository
and once the framework source code is available you will get an notification
and also we are hiring so if you're interested in working in compilers or any other hardware
and the position feel free to look into our careers page we can go through the link or that you
are code you can use LLVM or GCC as a search keywords to find relevant positions because
synopsis has to provide to compilers GCC and LLVM based so if you can also reach out to me if
you're interested thank you
we have
So, thank you for the presentation.
So, if I have a respite code, where I have my own custom
let's say some registers, right?
How would, let's say I'm trying to also get it up
speed with GCC and C-land, right?
How do I make it compatible?
Should I run it with your case?
Extended in your case?
How do you see this happening?
Whether it's a different TCG group from this by extension, let's say,
that comes up that does that, or somebody else who is doing
some custom thing.
How do you see this being compatible with those?
So, you ask me if you develop a code and you want to make sure
that it works, or what's that expected API for it, right?
Yes.
So, what you can do is there are two options.
You either have a witness file that you create or someone creates
for a generated compiler and then you create the report from your execution
and then you will do a diff between those.
Exactly what we did for the GCC and C-land.
And if there is no differences, it means that everything is fine.
If there is differences, then either something is wrong, right?
Or yeah.
So, this is a way to make sure that the developers make sure
that the API is correct, everything is well implemented.
There is no deviations.
For example, internally, we can say that our LLM-based compiler
for GCC has differences.
So, if we try to combine both objects rather than work,
if we use long double data types, because by default,
we set all of the compiler as eight instead of 16 bytes.
This is one way also to determine it.
Thank you.
Anymore?
All right, we hear me?
Yeah.
Sorry, I missed a few first minutes of your presentation,
so I don't know it could be repetitive.
But how do you go down the rabbit mode?
Like, from your slides, I notice you will check for
a short floating point integer.
Do you go for complex?
I mean, because I can see that this can never end.
Right now, we are not going for the complex, et cetera,
that might be in the plans, but we don't promise anything.
There's so much way to be connected to the framework.
There's so much ideas we want to look into,
for example, checking the L, compiler compatibility, et cetera.
But yeah, currently now, we're just looking
to the fundamental data types, which, in this case,
would be this one's.
That's what I mean.
Hi, further talk.
Just while I get it, would you be possible
to make your work available as a plugin for you?
Does it make sense?
Applying for?
Sure, you will.
Ah, QMU.
I've never thought of it, so I don't think so, honestly.
I will put this as an external project.
And not it, because we don't want this to be a for one thing.
We want this to make this available for all possible
c-related components that said they're not for the specific one.
Hello.
Thanks for the talk.
I was curious if there are any plans in trying to incorporate
all these extracted information into a nest boom.
There's a relevant depth on for that.
And there would be great interest in my view.
Sorry to incorporate the extractor to.
In a nest boom, software will look materials.
Expo.
Yeah.
OK, yeah, I've never thought about it.
I don't have a proper answer for it.
Hi.
Thank you for your presentation.
I was just curious about one thing.
So in this one example, you have each of the phasing
from the simulator you're using.
In this case, QMU.
The thing is that on the other hand, you
do manage to be compiler as well as a simulator
into combat by not pressing on T-bike
version instead of formats instead.
You are essentially looking for an execution trace.
But these tracks, I guess, involve that.
You are also using a way to instrument the light, the code,
which is not dependent on one-specific simulator.
And so I was curious as to where the instrumentation
the light is usually between the test case
usually combined, like the two chains,
or what kind of process it is here.
And it is instrumentized with code without pressing
on a specific simulator.
So what we do is you just execute the code
and get the output from the test case.
And from there, we will analyze either.
There are some tests that already give us everything
that we want others.
Then we need to look into the API.
For example, this is this architecture
don't be formation.
This is the output from the test case itself.
We don't look into assembly code or anything.
We're looking to the output of the simulator.
Because if we would look into the assembly code
for each architecture that would never end,
because if we were to add a new architecture,
then we need to add a new one, a license for it.
And et cetera, and it will be not easy to do.
It doesn't mean that in the future,
we might need to do it depending on certain tests
that we might not be able to extract information
from the execution.
But so far, we don't look into the assembly or anything.
So please, we don't waste time editing the code.
And then the code can be executed.
Yes.
Thank you for the talk.
I'm just wondering, is it possible
to infer whether a parameter is being passed
as an integer or a pointer?
I'm thinking of a, sort of, cherry specification.
Points are not the same as integers anymore.
Do you have a way of handling that?
Currently, no, we haven't done it.
Maybe in the future, we'll open the end.
All right, no more questions, so, thank you.
Thank you.