Yeah, happy to be here. Thank you all for joining.
So, I'm going to talk about EBITPF talks.
But first, a little bit about me. My name is Diana Amlinck.
I work at Isabelent, which is now part of Cisco.
I'm a Selium contributor.
And I am a reviewer for the Selium EBITPF loader library.
I've been doing EBITPF stuff since about 2021.
And that is to do with why I started the box project.
So, EBITPF, as we all know, is great.
We can do super cool stuff at the kernel.
It's safer than the alternatives.
And it's super fast.
But, EBITPF is also really complex, quickly evolving.
And honestly, not super wall documented.
As maybe perhaps any of you who are still learning at MIT,
might be aware of.
So, I personally, so I took this graphic, which is normally a marketing graphic.
Not something for a developer.
But what it's nice to explain is, it's called the adoption curve.
And what I believe that, like, from nothing other than my gut,
that we are somewhere at the Red Arrow.
So, we had the innovators, the people who did the early network of EBITPF,
who were users.
And we're now somewhere quite far along, probably,
in the early adopter stage.
What's coming is the main early majority in the late majority.
So, basically, most of the people that will eventually use EBITPF,
still have got to adopt it.
So, when you start learning EBITPF, or at least,
you start to Google about a bunch of terms for EBITPF.
And you're still more on blogs and articles,
which often explain, like, a very specific thing,
or, like, a very specific use case.
But they don't explain everything about the exact EBITPF feature they use.
They each quickly, because a lot of articles,
you'll still find state things that are just no longer true.
There are great videos, like, Linux plumbers and full stem on EBITPF.
But they are, like, everything in there as much searchable.
So, you have to either know exactly what the video is about.
Or, you'll just not find what you're after.
Colonel sources are demost reliable.
It's the source, basically.
So, it's the most reliable way to learn about EBITPF.
But, especially for EBITPF beginners now,
and a lot of people that are using EBITPF now,
are not necessarily Colonel developers.
So, reading Colonel sources is just not very beginner-friendly.
And then they come to docs.
So, there are, I want to shout out a few resources that are,
like, the most like docs that are already out there.
And they all have, like, a little bit,
that is really great.
The Colonel books are best source that we have,
but they just don't cover everything,
and they are often really focused on Colonel developers and
users with the main pages for helpers, which is now no longer,
not really being maintained, it's a searchable.
So, I had, like, when I started doing this,
I had so many gripes with, like, why, why come?
I just find what I want to.
Like, why come?
I just search something and find what I need.
So, that's where a lot of the, where I'm on motivation
for doing this project comes from.
When I, after, after I became a little bit more knowledgeable,
I started to help other people on Slack,
and the same questions get popping up, like,
so I found myself repeating myself often,
and having to maintain this, like,
you list of resources, like, depending on what you're asking,
I had to send people to completely different sites,
or a combination of sites, like you can find this here,
but if you want to know this, you need to go to this other place.
They, they didn't link to each other.
So, and there were just certain questions that we never answered,
like, that are quite practical.
Like, can I use this on Colonel X, why,
this, this, this program type support this helper?
How do I do loops in EBPF, which is, is not possible?
So, what my goal was, is to create the one-stop shop,
like, I drew the limit at everything that is kernel,
and everything that is maintained,
backer all developers, such as LibBPF, and so on,
and I wanted to make documentation that was for users
for the end user, and not necessarily for the people writing it,
because they probably know a lot of this already.
I want to, like, internal linking, because there's a bunch of stuff
that is often, like, a program can go,
how a function, which might reference something else.
So, I wanted to be able to follow it,
and I wanted to searchable, like, both on the site itself,
but also in search engines, because, because that's where I started,
like, just type in something.
I wanted to answer a few of these practical,
or as many of these practical questions that I, that I could.
So, what I ended up with is the EBPF docs project.
This is the homepage, and it's likely to do the big,
the big chapter for anyone who hasn't seen it.
What I, so, there's, you can internally search on this,
which I find one of the nicest features, and I use a lot.
And this is, like, the example of one of the pages.
On the left, is your, like, site navigation,
your navigation will page itself.
And then what I particularly want to do,
or potentially, for example, is here.
So, you'll see these a lot, is the little tags with version number.
And these are everywhere I can place them on the site,
and they, so they state the kernel version,
in which something was added, and if you follow the link,
it will go to the actual commit-on-get-up,
where that features added.
So, if you want to check, check something,
or, um, to actually dive into the sources,
because this isn't enough for you, then you can do so quite easily.
One of the other things it does is, every, all of my,
all of the program pages have this generated list of all of the helpers
that are, that you can use from it.
So, this answers, like, the practical question of,
can I use X and Y?
And if, um, you can only do it from a certain version,
we also have, like, the tags behind it.
So, the helpers exist, that the program exists,
that only from this version, onward, could you use them together?
Same for, for, for, k-fungs.
So, it just is, uh, every program that has this list,
and the reverse also, too.
We have map types, um, where you'll immediately notice
that I missed one, um, I just haven't had, uh,
I did and dive into the internal,
so I haven't explained that one.
Contributions are very welcome.
Uh,
Is this where you think stop this one?
Uh, I'll get to, I'll get to the behind the scenes a bit.
Yeah.
So, um, maps, we have maps, it links to a few other things,
internally, for example, the value size, which, um,
uh, gets explains, uh, what kind of things it does,
where you can use, or what Cisco, you can do for in case,
that's important to you.
Uh, I don't imagine it is for a lot of people.
Um, also, like, flags like these,
these typically don't get explained very well.
When are they added, what do they do, um, stuff like this?
Um, then we come to the helpers, so this is the index of the
helper, uh, of the helper, so this is like the very long page.
Uh, and notably, I like, bit my best to categorize these.
So if you're looking for like related helpers or whatever,
they're like, sort of listed together.
Um, and if they're like for specific maps, maps,
then it's, uh, also tells you that here immediately.
Um, so, uh, these are, for example, the memory, the memory helpers.
That's, uh, I hope a page looks like this.
Uh, again, tech, uh, we have the definition, uh, which comes from, um,
is so indirectly from the kernel, but the libbps, the libbps, uh,
that it's, uh, best to like collect all the separate definitions into like one file.
So I, uh, have their copyright homework, because that's where I pull my other data from.
Uh, and they've explained it, it shows you what the, what the definition is.
So if you need to, like manually define it, you can just grab the,
grab the definition and use this in your program.
Uh, and it explains it, uh,
we have, again, here to this. So this is the reverse, um, for,
which, in which program start, can I use this helper, which is, uh, which is also helpful.
Uh, and we always attempt to add a little example.
So this is often missing, I have to admit.
Um, and of course, credits do to everyone who helped, uh, maintain all of the pages.
Um, cavefunks, so cavefunks, this is actually the only place I know of that cavefunks are actually documented.
So the helper functions, we have the mampage holder, but there are no mampages for cavefunks.
Uh, so I endeavor to add these.
Um, an example of this is, again, along the very, the, the same line.
So if you're, we are used to one kind of page, then you'll probably find that this is also, uh,
that this is, uh, familiar to you. Uh, it explains a bunch of things.
Um, it, uh, so these cavefunks are used with different mechanism.
And they have, like, flags to, uh, to tell you, uh, roughly about what, what, what the purpose of, uh,
uh, over cavefunks, uh, and it tells you, for example, here that this may only be used in a sleepable program,
et cetera, and it links to a bunch of page that explains what a sleepable program is.
Um, again, with an example, program types that it can be used with, because that is the same, uh,
that's the same kind of deal.
Um, and these, uh, these are typically from also from the kernel sources, um, uh, which I, uh,
uh, which I didn't have here for, so you don't have to go, go fetch them yourselves.
Um, so these are concepts.
So besides all of, like, documenting explicitly all of the individual things to our,
I also endeavor to do a more, uh, more article style thing.
So for everything that's, that needs explaining, like, what is spinning?
How do I do, concurrent your loops or resource limits?
Stuff like this, where you actually need a little bit of explanation to introduce a concept.
Those are the concept pages.
Um, so we have, uh, stuff like, uh, how to do loops, uh, in different versions.
Well, when you can do loops, how do they work?
What are the caveats? Um, the explaining VP of tokens? Um, uh, what they are, what they do,
I need them, uh, dynamic pointers and a bunch of functions that you might use with this.
So there, there are quite a few of these, and they are more in, like, a sort of football style.
Um, I've also, uh, this is quite recently actually started to, uh, make pages for the,
for the libbps side of this. So, um, because typically, you see, like,
everywhere you go on the internet, you see these libbps, uh, internal macros.
Um, the, the, the, the, the, the second, uh, all of the little macros that are used to define specific things.
But no one ever explains why did this come from, what they've been mean. Um, uh, so, uh,
these are all here, and they also can be linked in the future and articles where you explain, like,
how to do something in EBPF. It can just link to these, these individual things, that is the idea.
Uh, same goes for, so again here, we said, okay, we have the bpf core, vetoed h, and you can find all of these macros,
and uh, explains what all of these individually do.
So, to, behind the curtain, so this was, like, everyone who, who can go to the page can all,
uh, my memory seen this. So I also want to explain what, what's behind the project.
So, um, it's a, um, it's a make-dokes project, so we, it's all mag-down, um, with a, with a theme and a bunch of plugins.
Uh, I use Docker to, uh, I use Docker to, um, make sure that everyone can contribute and run and do stuff,
get up actions to actually deploy this, they'll once and get up on get up pages.
Um, and then there are the tools, so I wrote a bunch of, uh, these are included in the repo.
A bunch of these tools to the help me, um, maintain, maintain this.
So the first one is the, the, the version feature finder and the feature, uh, tech generator.
So the way this works is I create a ammo file. Um, it's your, the, the two users get,
grab and get bisexual, um, Linux get up repository. Um, to, and then it will figure out, like,
when we're certain symbols added. Uh, it will create a big data file of this and from this data file,
I can insert it into the markdown automatically. So again, uh, we have here that, uh,
I want to point out again the, the label here. Uh, so this would be a file.
So I have a bunch of, so I said, okay, in the PF lives in these, in these,
a lot of files, that's optimization thing. Um, please go through all of these tags.
These are the tags that that BFF exist in and then go find all of these patterns. Um,
so by default, it's just a normal one, but it also supports red taxes.
And then it's, uh, it spits out, uh, they so just says, okay, this, this symbol was added in this version,
this commit, and then, um, it's, I don't have to, most of the time,
I don't actually manually have to go into the sources, and it gets inserted.
So only have to create a tag, tell it what symbol I'm looking for, and it will insert, uh,
nice little link in there. Um, I have the help, uh, reference generators, so these two lists of, uh,
uh, the from helper to program program to helper. Um, this is a little bit more labels.
So, uh, I have manually write down a sort of data file, which is a very, very small representation,
and it actually models the way that the kernel results, which helper files, um,
can can be used in which programs, which in the kernel is like a bunch of functions with
switches that call each other, uh, and I emulated this. So I have a group.
We have the base, the base group with all of these, uh, these, these helpers in there.
And then, uh, I have, for example, the C group, base group, which has these,
plus all of the ones that are in the base, and then it can be, uh, and then it is used by a program.
So these programs use the, uh, SK filter group or a few individual ones, plus the tracing group.
And these all get merged and translated into these big lists, so you can actually resolve,
which, uh, uh, actually resolve, uh, how, how does work. And this was, this is all, uh,
automated, so I don't have to do it by hand. Um, same, uh, uh, uh, we have the deaf, the deaf
scraper, so this helps a lot with the helper functions specifically. So it takes the, uh,
all of the definitions from libbpf about helpers. Um, it parses the commands that have, like,
explanations on them, which also the managers generated for covert to mark down and we can just
insert these. So you'll, uh, you'll see that, uh, that we have a bit here where, uh,
where we have the helper definition, it is actually like the mark down version.
It says we return this and this is, uh, uh, and we have the lag line. So that is all,
like automatic updated, the libbpf gets updated, we update, uh, we updated on the box as well.
And then for the cave function, uh, it's, uh, a little bit of the same idea.
So I manually curate the list of all cave functions that are, that are in there.
So I just basically watch and watch the mailing list or watch the two versions.
Um, I compile a VM Linux. So I compile a Linux kernel, take the VM Linux out of it
with all cave functions that are defined in there. And then I can use the kbpfk
from the contact to like validate that I got all of them, but I also use the VM Linux,
the btf and there to generate the signatures of all of the cave functions. So that are always, uh,
correct. And then the, the flags I manually do. So I say okay, we have a set of these IDs.
These are the functions. These are the flags. So that's sort of the notification things.
And these are the programs, um, on which you can, you can use them. Um, and then on the page,
I still have to create, uh, often manually take descriptions of what these do.
But the, uh, like the, the boring part, they basically get it done for me.
So that's all of these toolings really, all of these tools really help.
Uh, and then the reference with all the programs that it can be used from and the reverse,
are generated, of course. Um, so this makes it really, uh,
this really takes away a lot of maintenance burden from this. Um, so, um,
um, future things that I still want to, of course, we need to keep up a changes,
because otherwise this is all, um, useless work. Um, I still want to do the little bit of
user space so that we can also start to do more things on the, on the user space side of things,
explaining, uh, for example, in the tutorial styles. Um, there are so much more concepts
that that can use concept pages, but these are the most labor intensive to create.
So, um, future work. Um, there are some gaps. Uh, I have to admit like I, uh,
I don't understand everything myself. So I can't always document them. So as I'm learning,
I'm filling in the box myself. Um, and one of the big things I want to do is to start explaining
verify logs, because this tends to be like the biggest obstacle to new users. Like,
what is how do I redevelop, how does this work? Um, and what I like, if I get
error x, how do I, what do I need to do to fix it? Um, but to like get, I need some information
like what are common errors, um, get some good examples of the, uh, of the looks. Um,
so I want to shout out, uh, this link. It's by, uh, Johannes, uh, who's also speaker
at the conference later, uh, later this day, uh, where we are trying to collect, uh,
like voluntarily, collect as many, uh, verify logs and the solutions. Like, this is the
load I got, this is what I did wrong. This is how I fixed it. Uh, so that we can,
uh, I can use it in, uh, to document a lot of these things, um,
and, and Johannes wants to, uh, try to train an AI on this to make a tool to help
help programmers out, though I don't know. Let's see how that goes. Um,
uh, are there any questions?
We have time for questions. I, uh,
uh, what we have. Um, uh, what we have. Um, uh,
uh, this is a separate procedure that you've been doing, but these types of
you being doing most of that, you're sent information to me, uh,
like our communities, uh, uh, uh, uh, yes. Well, uh, so, uh,
yeah, so the question was, do we, uh, that, that I do this work myself, or do we have
other contributors? So the bulk of the work, uh, like adding adding most of the pages is, uh,
a currently still done by me. The, there was a section on lip XDP, uh, which has been
contributed, as all of the pages they are contributed, and we have around 20
contributors who mainly do, uh, like, uh, uh, uh, fixed my work,
spell checks, miss, I like things I go along, uh, stuff like that. So, uh,
most of these small corrections, uh, and of course, every, like,
please help out if you, if you see anything wrong, just submit it to
our, um, everything else, if, uh, as long as it's good and correct.
I don't have a question, I have a couple of them. Thank you.
I do using the site for many months now, it's great.
You're welcome. That's what I wanted.