All right, let's do this welcome full room, let's cool, I'm Benjamin, and yeah, we're here
to talk about the Zaffer, the good, the bad and the ugly, I guess, he words about myself,
I'm a few disclaimers, I guess.
So, yeah, my name is Benjamin, I'm a developer advocate for the Zaffer project, so I should
be telling you about why it's awesome, except that I also know that there's some limitation,
so let's talk about them, and yeah, a few disclaimers or like a caveat, I'm not necessarily
like a developer, an embedded developer by training, I guess I'm a, some would say I'm
a full stack developer, and I certainly can get my way around Zaffer, I've learned a lot
if I see the site join, and I can feel some of the paint points, so yeah, let's talk about
them.
Zaffer, actually question, because that the answer to that question might actually change
the way I will make this particular presentation, how many of you in the room are actually
using Zaffer?
That's a lot.
Okay, so the idea of this talk is that I know already what are some of the paint
points that you guys are experiencing, or at least I think I know, and I wanted to try and
go to some of them really quick, and hopefully we'll have time for more of your own like
genuine questions and feedback at the end, so I might not have to remind you what Zaffer
is, but yeah, open source, turning 10 years old next year, lots of like really really active
community, we just passed 100,000 commits, sometimes during 2024, thousands of contributors,
every week, like literally tens of new contributors every week, and it's an autos, or is
it?
Like, I mean, a piston on paper, it's called Zaffer Autos, so surely it should be, it is a real
time of bringing system.
In fact, over time, it's room, right?
Like, I guess you could say there's Zaffer, the Autos, and there's Daffer project, which
is a little bit more like Zaffer, when you take Zaffer, it's for more of the things for giving
you some hardware abstraction, maybe too much, some would say, but it's that, certainly
like making you completely independent of the underlying hardware, whether it's, we're
talking like architecture, whether we're talking socks, whether we're talking sensors, like
all that is of suckity doubt.
It's effectively a framework, it's much more than just a real time kernel, it's a framework
that provides you with, we have a state machine framework, we have integrations with LVGL
for, for you to build newies, we have obviously all the connectivity stacks, so that's,
I mean, that's not maybe what people would call an autos, when they think autos, they think,
oh yeah, free autos, scheduler, like really low level stuff, only a few, a few sea files
that I'm going to use to support my goal of a cost programming, Zaffer is more than
there, today you learned, if you didn't know that, people building all sorts of connected
devices, so Gardina, they will do a smart irrigation, and 15.4-based protocols, stuff,
in the background, it's just, just a random image of a winter mind, but investors, like
they run and they use Zaffer and the current protocol of all things to control, like to
do live a real-time monitoring of their winterbikes, and also that's that's the effort
based.
So what's the point of the roasting party, I mean, I think, I know, I've joined the Zaffer project
exactly two years ago, like two of the day, I forced them, 2023 was my, like, I was two
days into the job, I love Zaffer, I hope many of you do as well, but sometimes it's not
perfect, it's grown, it's grown over time, there's some, some pain points, so let's
run and talk to them, try and see maybe what's the reason behind it, if there is one,
and I try to let you see the lights maybe at the end of the tunnel for those of you who
might still experience some of those pain points that are listed here, I mean, it's,
yeah, overheard from reddit, discord, devil preserve this that we ran, you've maybe you think
that some of those are true, right, like Zaffer is too big, like it's an autos, how come I need to
download gigabytes of stuff to blink an LED fair enough, it's slow, it's bloated, it's yet another
hardware abstraction, why did you guys, like, divide three four embedded, like, that doesn't
make sense, that's yet another, yet more overhead that you're putting on us, west, why did you bring
your own, and like, build slash Swiss Army Knife sort of tool, just give me the dumb source,
source files, so yeah, there's, and start thinking hopefully we have a few minutes at the end,
if you have more to add to the list, but yeah Zaffer is too big, maybe it is, the thing is,
one thing that we try and, and do is make sure that the time-to-hello world, let's say,
is as quick as possible, and that you get a fully functional workspace as easily as possible,
and effectively, it's pretty much all that you need to do, use west, and we'll talk a bit more
about west in a minute, but basically, provision of workspace pointed to the Zaffer, the main version,
and the upstream version of Zaffer, and then fetch all the dependencies that are given point
in time, are needed to have this Zaffer, this particular version of Zaffer functional,
it means that it actually is going to pull a lot, we're going to see that, bringing in
SDKs as well, so what we call SDKs is effectively tool chains for all the architectures
that you may consider targeting, but like there's a lot, like you might only care about
expressive, and yet installing those SDKs and those tool chains is effectively going to bring
you XADCX and ARM and whatnot, but then the good news is that once you're here, you can
start building a hell of a world, or more, like we have hundreds of code samples, which will
just work on whatever board you're talking, like we have almost 700 boards in Zaffer that
are supported, so you can compile against those boards, flash, debug, it will just work,
but of course that means the drawback is that, and I actually didn't realize that much,
but it's basically you end up with 15 gigs or close to 15 gigs of on your laptop or on your computer,
because that's a lot, like you get the Zaffer sources, which already is actually quite a lot,
when one gig, and then modules, what are modules? Many things can be like embed TLS or nano-PD,
like we mentioned before, and the big bulk of the modules would be the house from the vendors,
which all the rest of the Zaffer like porting layers are being built on top of, and some
house are like over one gigabyte, and same story for the SDKs, we have an easy way for you to
install those, all those tool chains, but some of them, like you will like you never need them,
it's great to have them, I guess, when you work on Zaffer as a Zaffer maintainer, Zaffer,
like someone who touches all the areas of the kernel, and like, and drivers, etc, like you can
really quickly try and see, oh yeah, like, can I really compile my, like, is my drive on?
And we're back. Yeah, like, you might be happy to have all those SDKs, but maybe you don't
need them, so just a reminder, I guess, it's effectively not like you're not forced to do that,
and as a matter of fact, when you build, like, when you start and create a new workspace,
if you follow the Zaffer getting started, yes, you will set up a workspace that effectively
tracks Zaffer upstream, but oftentimes, you will, like, create your own application for which
Zaffer is one of the, one of the dependencies, and when you effectively create the manifest
that describes what it is that you need, you can suddenly allow lists, only those things that
you really know that you're going to ever need, and you don't bring in the how from esteemed
micro, if for the foreseeable future, you only care about NXP, and then you can maybe bring it later,
but that's, yeah, like, that's something for which you can find lots of examples and
best practices in the example application that we maintain in the Git repo, and kind of the same
story recently, we introduced a command that helps provisioning Zaffer SDK into your local environment,
by default, it will install those, I'm actually like, 8 plus gigabytes of tool chains,
but you can easily use the command to be like, I'm on Mac and I care about ARM just bring me
that couple hundred megabytes of tool chain. So, yeah, I guess, I hope that helps with the, like,
yeah, this is too big, but there's a reason, like having those 15 gigabytes that do
download once, you might actually not read it in the end, because you have, you know, that you have
an environment that's already set up to do anything you might ever want to do with Zaffer, so yeah,
Zaffer is bloated, as in it is, yeah, lots of abstracts, however abstractions, a lot of layers,
it's, I want, like, just, sort of almost a bare metal, like really low-level autos
due to real time, because because autos, right? Zaffer is not that, perhaps, but the thing is that,
I mean, Zaffer is, allows you to build, like, really allows you to build really complex applications,
for which you might care about some of those things, maybe not all of them, right? Like, if you're building
some kind of connected Bluetooth battery power, kind of device, surely you're going to care about
having, like, a really good rock solid connectivity stack, you're going to care about power efficiency,
maybe security to some extent, but speed, like, like, as in performance and throughput,
being able to have, like, really intense multitasked environment, maybe not, so you might have,
you might be in that situation, where you actually don't mind having a slightly above-aged
developer experience in that hardware stack protection would be nice to have, like, stack
calories would be nice to have, really nice fancy logs would be nice, because you're,
you don't mind, you kind of forward it, you think it's a good thing in your, for your use case,
and maybe you have something that's completely different, and the thing is that Zaffer allows you
to tweak all those knobs the way you want to, and by default, it takes some reasonable decisions
with regards to what could be a good compromise, something that works for pretty much all use cases,
all sorts of architectures, will run correctly on all the boards that might not be what you need,
but yeah, by default, hardware stack protection is enabled, that might have a small impact on performance,
that also means that should you have a stack overflow and should your hardware platform support it,
you will have, like, a hot fold when something funny was just going, was just about to happen,
because you actually had stack overflow. By default, we optimized for size, not first speed,
so, something to be aware of, but it's just one cake on figure way to be, to be changed,
which if you don't do a lot of defensive programming in the kernel for performance reasons,
but still, there's a few places whereby default, like allocating memory from a memory slot,
like by default, it would be thinking a few extra checks, you might want to disable that,
if you really, really care about the performance, maybe you don't, because you rather have the,
sort of, like, the safety net that it provides you. So, I think, I guess, all that you say,
and kind of related, when it comes to selecting an autos, like, there's many things to look at,
and performance as in, like, I want something that's, like, real time as in fast and high throughput,
first, like, make sure that this is actually video requirement, it might not be or, or even if it is,
it's just, like, one axis among the many that I was showing on, on the radar chart earlier,
and in any case, like, out of the box's effort might not be fine tuned for performance,
and especially, like, benchmark wise, there's always, you need to fine tune for benchmark,
specifically, which, by the way, we've been working on, so there's, there's a lot of benchmarking
tool and, and code in the in the Zephyr repo, but there's one that historically comes from,
from Threatex, that's now been pointed to, to, to Zephyr, which has been a good opportunity for us to,
actually identify a few areas where we were, like, maybe not being optimized, so you can actually
rent those numbers by yourself, and we're not pretty much parity with, with, like, equipped Streatex,
but yeah, like, again, think about this radar chart from earlier, and I think all the accesses there,
matter. Yeah, device 3 device 3 for embedded, that's thousand indexes, that's, again, adding
too much complexity, too much bloat, why, how, why do we use device 4 in Zephyr, while describing the
hardware, that's for sure, providing the hardware initial configuration, whatever, it is that you
configure in your, like, when you configure the, the node corresponding to an LCD display to a nice
spreadsheet bus, whatever, like, all those parameters in terms of speed, resolution, call-off format,
whatever, this is what's going to end up being a past to your drivers, so that there are initialized,
and they have the, like, the right configuration to function, and, yeah, and, and, and effectively,
you end up just, like, just, like, a Linux describing, describing your hardware, and then,
based on what, uh, compatibles are being indicated, and, and, and, and cared for for each node,
then the build system will figure out, like, what is the, uh, what is the, is there a driver available
for this particular compatible, and then, uh, past the data, uh, to it, and, and, and, and, and configure it,
which means that basically there's a lot of stuff happening, um, a build time, where we take all
the metadata, if you will, uh, from, uh, from the device tree, and turn that into,
lots of macros, really, that, that end up, uh, being what's needed to feed, uh, to feed data,
into, uh, and, and, and, and configuration into the drivers, and also to expose the, um,
uh, what, the developers will need when, uh, at, at runtime when they need to basically access,
and get a handle to a particular GPIO, so that they can toggle it, or they need to,
to get a reference to, to a display that, that's works, but that's really, like, a build time thing.
We, we, we generate all that, and, uh, when it works, it's pretty awesome, like, you end up as a developer
to be like, a, I want a display, like, hopefully, like, my, um, my, um, my device supports display.
If yes, um, it will have indicated, uh, in the device tree that there's a chosen node that
points to a particular, um, instance of a display, and then, that means that it can display stuff.
This code, and I actually wanted to bring the demo, but, um, the point here is that, like, the,
the client code, the C code corresponding to that, couldn't care less about the fact that the display
is effectively an LED strip. It's just an LVGL application that only, that only knows that it
runs, uh, it's talking to a bit, a display that's 16 by 16 pixels, and it's just, it just uses it,
and, and if you want to switch, uh, from this to, um, a simulated device running on your on your Linux
machine, it would work just the same, this, the really, the very same code will display the very
same numbers, except that it will be running in an SDL window on your, um, you know, machine.
So that's, um, that's good. When it works, the theory is quite nice, like, you end up, uh,
using all the labels, all the aliases configured in your device tree to get handles and references
to the, the various nodes, in practice, sometimes it doesn't work, and this is something that's
painful confusing for people. The first time this happens to you, and maybe this has happened to
you in the past, like, just like, really go to the bottom, uh, to the bottom of it to understand what's
going on, um, uh, under the hood, or, uh, and, or refer to the documentation to get some,
some tips and tricks, but basically, the error message itself will be really painful to the
server, but, uh, in practice, like, if, if the error happens really at compile time, it's, like,
that you've been, like, the, the node that you're trying to, to basically, uh, uh, get a reference to,
it basically doesn't exist. So whatever, macro, that should have been generated at build time,
just, actually, just doesn't exist. So, um, yeah, it's that, and, or you have a node that requires
a node that itself is disabled. Or maybe that, um, uh, the issue, I'd happen slightly later, uh,
during the linking time, uh, and in which case, probably your device free is all right, but your, um,
your effectively, um, um, um, you might have some drivers that are not, um, um, um, uh, like, uh,
the, the example earlier, I have an LED display, uh, for which, when I configure it, the LED
matrix, it requires an LED strip. And so I might have done everything right there, uh, but if
I didn't enable the LED strip driver, then the LED matrix, driver, uh, I will, uh, we'll have some
issues and, uh, linking what just don't work. Um, West, something that's actually, like, a really
really nice tool in the effort that makes Zephyr, uh, like, when you use West, you have this sort of
single pane of glass to, um, to interact with your devices, like, when you want to, to flash something,
you don't have to remember, uh, whether it's like your flashing for any SP32, what is the Python
command that I can, I should call, uh, or STM or whatever, what is the, uh, um, the command to, uh,
to debug, to, to build, like, this, this takes care of all of that, takes care of, um, managing
all your dependencies. You have, like, this manifest that I referenced, um, reference earlier, which allows
it to, to, um, basically when you have a project that the, the build system and the command line
are very much aware of all the things that make up your, um, uh, your system, and it knows what
board your targeting and sort of, like, what, um, you can have, like, really nice content as Zephyr,
so it's, it's, it's quite nice, um, it's quite extensible as well, uh, some of those, uh, sort of, like,
things that I, I am animating here, um, maybe you're not familiar with all of them, so check out the
documentation that you can use, like, special, uh, build targets in West to, again, in a very
consistent way no matter whether you're targeting arm, intel, STM, espricef, whatever, uh, getting, um,
uh, ram usage report for your application, West build dash key, ram report, you get that, uh,
generating a, uh, software build of material, running code checker, uh, all that is available through
West, so some people don't kind of like, uh, more like, yeah, bring me the, uh, the couple of source files
that make up your autos, make up your kernel, and I'll, uh, and I'll have some fun with it.
I'd rather myself have something that, that gives me something that's, uh, a more, more integrated.
If you don't want to use West, that's fine. Uh, you might have to provide more, uh, information manually
that having West doing the job for you by fetching all the metadata already available in the, in the
manifest and elsewhere, but, uh, it's fine, that works, it's supported, uh, but, uh, um, uh,
last but not least, something we hear often in kind of a segue into the, the, the few minutes, we may
have four for discussion. Uh, my board, my sensor is not supported. Um, yeah, I mean, sure,
if your board, your sensor is not supported, if it's, uh, a board or a sensor from one of those vendors
really engaged, uh, uh, uh, with, with the project, not them, maybe, uh, but if it's just like a
random board random sensor, it's an open source project. It might be a good opportunity for you
to make your first contribution. One thing though, uh, though, um, when we get this kind of feedback,
sometimes it's also along the lines of, yeah, this board or this sensor is supported, but, uh,
it looks still, like it's, it hasn't, uh, and how do I know how, um, how well it works, like, and
does it really support all the, all the features of set sensor? And yeah, sometimes it's,
that the support is not necessarily complete. Uh, we don't have, like, good tools for,
for helping you, um, uh, be, besides just, like, look at the get history, uh, to know whether, yeah,
the file hasn't been touched in five years, maybe it's not up to date and, uh, up to the latest
standards in the effort, um, but if you have tips or, like, suggestions on how we could improve that,
to give better visibility on, sort of, like, the maturity of the various, uh, sock,
portings, board, portings, drivers, um, they don't be great, uh, to do here. And, uh, we actually have
one or two minutes for some additional criticisms, maybe. Thanks.
And now one minute. I would say I love the advisory from a user perspective, but if I want to try
a white driver, it's terrible, because then you go into this mark rose, you think about parsing
that. So I'm curious if, like, we as a community could figure out how to make that's error,
reporting of that, uh, mark rose from the advisory better, so people can actually write more
drivers as well. Yeah, I agree. I, and yesterday in my hotel room, I was trying to, I'm sure there's
it, there's, there are ways. Um, um, I think I wrote a driver turning minutes and the device
he grabbed took me one of ours, right? So, yeah, that goes on right, but that doesn't sound
familiar either or so. Yeah. One more, maybe real quick.
Uh, I really like Zephyr a lot, so thank you for your work on it. Um, my question is,
so you're talking about hardware abstraction earlier. Uh, it makes Zephyr makes hardware abstraction
really nice, especially between different microcontrollers and such. Uh, have you noticed vendors
being upset about this type of thing? Because, um, in the chip shortage, it became easier to, um,
basically change out of part. Uh, I, I haven't experienced that. I, I think it's a feature
to some extent. I think like the vendors, they, they embrace it or they, they have to,
I, I thought of the, the deal and that brings them compete on maybe other other stuff and
stuff that there, there might be stuff that will become a commodity in terms of like IP, IP blocks.
They need to compete on what's like really unique to them. Um, yeah. So thanks.
Thank you.