So, yeah, my name is Mikhail, and I'm going to talk about IA, and so about what's new
in Rust for a BFF.
We were talking about IA in previous editions of PhosDem, so that's not going to be an
introduction or a talk, quite the opposite, we'll deep dive into what's new, but to people
who don't know, IA is a BFF library written purely in Rust, and it allows you to write
both the user space part and BFF part in Rust.
Of course, mixing with other languages is not possible, like the BFF component in certain
other languages and still using IA in the user space or the other way around, but you
can also do both in Rust.
So the first question, which usually here from people, when we say that we do BFF in Rust
is why, because the main selling point of Rust is not so valid here, because BFF is
more used here, because we're a verifier, and well, my answer to that is usually that our
choice of Rust, and our preference towards Rust is more about the developer experience
rather than the memory safety.
So we like cargo, and many people who are Rust developers, they today, like cargo, they
like music packages, and they like all the tooling, which is built around cargo, and in IA,
it's similar.
For example, in the start with IA, we can start with cargo generates cover of the project.
You can also use dependencies from Crase IA, so actually even in the BFF part, you can
use crates if they are small enough, and plus, but not least, like Rust, there is option
and result in pattern matching, and you can leverage this in the kernel space in BFF as well.
So a cargo generates, like a sub-command of cargo, which you need to install additionally,
which generates projects from templates, and we have a template as well.
So the best way to start with IA, if you want to do it, is to do cargo generates IA, or S, IA template.
And when you type this comment, it will prompt you, it will ask you some questions,
it will ask you like what type of program you want to do, whether it's XDP, whether TCE, whether LSN,
it will ask you some follow-up questions specific to the program, and then,
yeah, it will generate something, which should be like kind of a hell of work for
the given program type, so for XDP it will show you a program which locks packets for
LSN, it will do some basic lesson hook for, I know, accepting the certain action.
And about crates, so I mentioned that crates, which use a dependencies in BFF part need to be small enough,
in RASN there is a concept called NoSDD, which means no study of library,
which means that you can use only like the smaller version of study of library, PortCore,
which has no operating system dependency, it doesn't have allocations, it doesn't have
a synchronization primitive, but otherwise it's still like a basic library,
which raspberries out of the box, and one of the main crates, external crates, which you can use
in the kernel space when you write programs in IR, it's network types, it provides you the
definitions of two and three networking headers, when you write BFF programs with 3 BFF,
the way you go with generating network types is usually like getting them from VM Linux,
so you use BFF tool to generate a Cheader 5 from BTF, and that includes the network types,
and networks, but on the other hand, network types don't really change, they are standardized,
so we decided that, yeah, we will ship it as a crate, and the big pro of this approach is that
in kernel space you can use the network types crate with like minimum set of features,
but then in user space you can do some more sophisticated packet parsing and cellularization,
and yeah, we were presenting about IR, previous times as well, and the main points which
changed since of time is first of all we have integration test framework, we are testing
staff or arm, and we are also testing unit test and many other architectures,
we ported all the infrastructure around IR from using perf buffers to ring buffers, and that includes
the logging part of IR, so like the standard way of printing clocks in IR is using IR log,
and IR log is using ring buffers to basically push the information from kernel space to user space,
and then user space component is basically formatting those arguments and printing the outputs
on your SDD out by default. We added more map types, for example, SK storage,
task storage was coming very soon, we added for the sector and iterator programs,
we added support of entirety of raw types in BTF, and I will dive into that and I will dive
exactly into the status of BTF support in IR ecosystem, and we added also TCX links.
Well, to give you some context, BTF is the type format like the debug sort of type format,
which is way smart and worth, and it's used in the BTF world to do relocations of kernel types
across versions, so for example, if your BTF program with kernel 6.2 and you'll love it in kernel 6.12,
the BTF relocations are basically way of figuring out what's the offset of the field
of the kernel structure accessing, and BTF is generated in two ways, so for BTF programs is generated
by the compiler, so if you build your BTF program, then LLVM is basically responsible for producing
BTF. When you build the kernel, you know, kernel is contains dwarf debug information,
so what happens on the kernel site is that there is a tool called Payhole, which then
transpires dwarf into BTF. For people who are not familiar with VM architecture,
I mean, it's a very basic outline of it. LVM is you could say library for writing your own compilers,
which provides you all the backends and LLVM provides all the infrastructure to build
binaries for different architectures, and how you can use LVM is to create your own programming
language, or like, support some programming language, while not worrying about
compiling the target code, and both clank and trust see, like the default implementation of trust
see are using LLVM underneath, and then how LVM works, well, frontends are supposed to generate
immediate representation of LLVM, which is like a very minified language, which then
decide LLVM is being optimized first, and then compile to the machine code, and when you have
simple rust code like this, LLVM, when it's compile to LLVM IR, LLVM has this representation
of the back info, which is like very similar to Dwarf, it already has Dwarf tags, it's not technically
Dwarf yet, but it's like, once that before having a Dwarf, it's a lot of information, basically
for every structure you are using, and for like all the details of your code, it has information
like including also the, which part of file, in which part of file that code is, but BTF is
minified, BTF doesn't need everything from that, so like from this debug info, translating it
do Dwarf would be like basically translate it into one, and then BTF we kind of modify it just to
have enough information to make relocations in the offsets, and BTF relocations lift in an
LF header and VM produces them using the in-frenzix, and all the language front and in LLVM, which
support BTF are supposed to use those in-frenzix to define the access to the fields for
the structs in which we want to support BTF relocations, and in LLVM there is a concept of
good element PTR instruction, which is basically accessing the fields of structs or enums or
arrays, and what happens, what should happen in the compiler front ends, which want to leverage
BTF relocations, is to detect, get the element PTR calls, which are concerning the structs related
to the kernel, and then kind of rub them in those in-frenzix, and in clank is achieved by
defining this preserved access index attribute to the struct, for which we want relocations,
so for example, the task struct is very common, it's struct in the kernel which represents the
tasks, the processes, for to make this example very short, I'm just outlining the PID of this process,
and when you write the code, where you I know access or return the PID field,
and you have this preserved access index attribute, you have this BTF, and let's assume that in the
kernel, where you are compiling this program, PID is the 96 fields, and it has offset of those
like 21,000 beats. What you end up with in BTF assembly is having a pointer of this task,
and then adding offset of certain amount of bytes, like this amount of bytes is like this
corresponds to this amount of beats, and yeah that's how it ends up in the assembly, but at the
same time because we added this preserved access index, in the f section of BTF relocations,
BTF X, you had this information that for tasks struct, and the field PID, in the assembly line 5,
like in as offset, like refers to the number of line in the assembly code of your program,
like here, and this is the information that the loader of the BTF program needs to do a relocation
there and make sure to check out what's the offset of PID fields in the kernel to which the
program is being closed is, and BTF loaders, like IR or BTF are responsible for doing those
relocations, so relocations are not done in the kernel, they are done by the user space loader,
which is about to load the program, and both IR and BTF are able to do it, so basically when
you load the program and you have the relocation, it will also like alongside load the BTF from
the kernel and we'll fix up your byte code, so for example in this case, again your program has
different offsets, and the kernel has different offsets, for example in the kernel like there's one
more field added before in the task struct, and like the offsets is like a bit forward,
the BTF loader is supposed to do this patching, and IR does this kind of thing,
but the missing piece in Rust is emitting the relocations, so IR is capable of doing
relocations when they are there, so if you load BTF program, written in C compiler with clank,
and load it with IR, IIS able to do the relocations, the missing piece which we are working on
is adding the relocations support in the Rust compiler, and our idea is basically like
to do something very similar as you have in clank to add this preserve access index
attribute to the types and then the magic wood happen of about generating the relocations,
and to sum it up, so like the BTF support in IR, you know, one year, two years ago it would be
mostly red, but nowadays like the first thing is that when you have Rust code and all the
Rust types can be converted into a correct BTF information by the VM, IIS able to do
relocations, and very soon already on Master, we have like fully compatible with BTF map definitions,
the last missing piece is the compiler support in Rust for emitting the relocations,
and actually like last year we did quite a bit of work to about the first point,
and we had to do some work at the VM to support all the Rust types, and we had to basically
do two things, we needed to add this support for variant parts, and if you don't know what is
that I will explain in the next slide, and we also had to strip name of names of pointer types
in the debug info, so basically like the small difference between C and Rust is that C by default,
if you have a pointer type, like for example start const, taskstract, it's not gonna generate
name for it in debug info, Rust would do it, and then kernel and verifier and all the BTF
infrastructure kind of realize on pointer types like not having a name, so we had to do a little fix
up there, but variant parts was the most crucial work, we had to do in a little VM in order to
support generation of really entirety of Rust types in BTF, and variant part is kind of like union
in C, but it has a discriminant at the very beginning, and for example like in C, the union is
segment of memory which can carry different variants, and like usually like it would take the
amount of memory of the biggest variant, and for example like you can have to this union has to
variants, it has this struct with to integer fields or has just one integer, so like by default
this union like regardless of which variant it carries would have like enough space for this
like upper struct, and in Rust like a very similar thing to to see unions are in us, and like
they look the same in the code, but the crucial differences that in C like the unions they don't have
the discriminator, and the most of C developers kind of work it around because like normally
if you just have union by itself, if you do a mistake, and you do an assumption that it's a
variant B and it's variant A, like you are kind of in trouble, it's undivided behavior, of course
like they also see developers who would like embed this union into some struct and have a
field indicating which variant it is or have some like integer outside, or like write the code
in a way that like they are really sure that in the certain part of code, the certain variant
like it's always going to be there, but in Rust they have this discriminant just for like
extra safety, and well at the beginning like when we started the work on BTF, and we weren't
aware of you know the BFF bucket in LLVM not supporting variant parts, if you had this kind of
in Rust and you would try to compile it to BFF target with the back info, the M was just crashing
like it just like didn't it was panicking, it's not what to do in variant part, after adding
support to it, there was a lot of discussion how to represent this thing like one idea was to like
represent it as a struct with the discriminator of field and then second field which would be the
which would be represented as union, what we end up doing is that we just represent the whole
thing as union, and the first field is a discriminant, and then we represents the variants later on,
and yeah to sum it up, our future plans and ideas like in general, first of all,
yes, doing the BTF locations Rust compilates really the last piece missing, to have a full
query support, the full everything written in Rust in IA projects, second thing which we want to do,
so for now current the IA has its own custom linker using the MS library,
which is called BFF linker, and it operates on LLVM bitcolt, we would like to eventually
like stop relying only on BFF linker and like align more with upstream and allow it to link
with the new tiers, or also LLD if possible, but we would like to start with the new tiers because
being used is already has BTF support, a BTF target support, and then also the idea which
like is very widely discussed, or did there is like a first draft PR, a scabbing support for
structups and scabics, I adopted by different projects and different companies,
how offline just the new ones, so one of the heavy users of IA, or whom I work actually,
it's Anza, Anza is a blockchain company which works on Solana, if you know it, and
basically Anza develops agave the main Solana by data implementation, we care about performance
a lot, and you know in distributed system it's very important that all the nodes communicate with
each other very fast, so we use AFXZP for it, and we already started developing agave XDP library
period and rest for using AFXDP, it's not mature enough to like move it to IA, but that's the
long-term plan, a new recent user of IA is the main in the middle proxy, or meeting proxy,
MIT and proxy, they started using IA, like not ever, ever yet, but they have like certain modes,
like including wiregun and local redirect, where they offload some parts to BTF, and they
write everything in IA for that, and yeah, I'm done here, and thank you for listening,
and yeah, it's time for questions.
Thank you.
So my question would be in the description, it is said that you are now a tier 2 rest package,
and you are going to go to tier 1, so what's missing to be there?
Actually, it's a bit worsening, this is perhaps where we are in tier 3, and we are trying to go into tier 2,
so yeah, we are in touch with the Rust team about that, there are two requirements from them,
which we need to meet to like move up to this year, actually I mentioned that we want to
support new tiers, like other upstream linkers, that's one of the requirements, like the Rust team
is not really happy with us having like completely custom linker made just for one architecture,
and the second thing is, so BTF's calling convention allows only having five arguments per
function, and if you define a function with more arguments than that, it's not supported,
Rust team is also not happy with that, so we will need to address that and find some solution for it.
Those two things.
So I wanted to just say, I work in the Rust container team, so I want to talk about this afterwards,
maybe, let me make some progress there, and you have a question as well, yes.
Yeah, you mentioned in the inam that you are omitting the discriminant and the debug info,
how does this work with niche optimization?
This is when Rust inam gets optimized the way, like the discriminant gets optimized the way,
because the first variant, for example, is none, and the second is some of none, no pointer,
which means that you don't need a discriminant to actually determine the variant,
so there is no discriminant memory.
Okay, so honestly for that, and does this optimization also affect the debug info,
is the debug info fixed up as well, then it should be fine, because like what we do at the end,
is that at the very end, after all passes are done, we basically transpile the VN debug info into BTF,
so if the optimising away the discriminator already happens through passes, then we are safe.
And so you can translate arbitrary debug info at this point.
Yeah, exactly, okay, so if the debug info doesn't have this discriminator, it will not end up in BTF.
Any other questions?
Yes.
Is CP1 a why a utilisation during buffers?
Yeah, they are faster than per buffers.
And thirdly, I mean, it's a synchronisation hotspot kind of.
Yeah, I mean, so per buffers are an older type of a mapping VPF, but I mean, don't get me around,
like per buffers support, in general, it's not going away.
If you want to write your own program, which uses per buffers, it's still possible.
We just switched over like the core components of IA to use ring buffer at the different buffers.
And it's mostly for logging, so to give you some context,
well, in the main way of logging and having messages from kernel space to user space,
in the LBPF is using BPF printk, so basically like prints messages to the traceFS.
In our case, we decided like first to push the logs through perf buffers,
and we migrated that to ring buffers.
Yeah, but per buffers support is not going away.
Any more questions?
Thank you.