So this one is going to be done by somebody who needs no introduction, because we
weird for me to introduce myself. My lightning talk is tiny grad on microcontrollers.
So we had one talk about, you know, people trying to run AI on MPs. I figured out, you
know, I need AI on an MPU, because I happen to be part of the community that is about
to tape out. It's a pretty interesting microcontroller level MPU. It's one of the first
fully open source tape outs at 16 nanometers. If you're interested, go at the foundry
GitHub, you can find all about it. It's a pretty capable MPU at about one top, 16 megabytes
of AMRAM, so yeah, it's not super tiny, so you can actually do a lot of interesting things
with it, like depth models, y'all models, you know, stuff like that.
It actually came from a much bigger architecture. That again, we're now dealing with
an AI foundry called ET. It used to be done by this company called Esperanto Technology,
but now it's sort of like we're dealing with it. And back then, it was scaled out to the
1000 cores. And, you know, if you want to play with those types of CPUs, I actually have some
in my pocket and in my lab. But again, this microcontroller that we're taping out is just
much more scaled down version of this 1000 core CPU. So, if I want to run something, you
know, anywhere, what am I usual suspects for a small kind of constraint device? Well, again,
I mean, there is M-Learn, which again, last year at Fauston was a great talk, so if you're
curious, you know, go check it out. It's kind of very zephyr OS friendly, you know, you
have a lot of fun using it, but it's really constrained in types of models that it can
run not available, you know, to sort of the kind of models that I was interested in.
Then there's obviously a light attitude for microcontrollers. Again, the supports for zephyr OS
is kind of like, I mean, nobody knows. Basel is not my favorite thing. And then, of course,
there's exicotarch. Exicotarch, we actually had a talk about it. You know, here is this weird
kind of combination of small things and big things. So, I'm not quite sure, you know, what
to think about it yet, is pretty young, so maybe it will develop into something that I would
actually find the joy to use. But with Fauston, right? So, like, why should we constrain ourselves
to the things that I kind of like pre-cammed and given to us by big vendors, you know, like
PyTorch community? So, what I considered them was, you know, micro-TVM, but apparently
that thing died. So, like, I didn't know. And then I'm like, okay, fine, I know G-G-ML, I know
tiny grad. And there was actually this other project that the clips from the foundation
called age that kind of tries to bridge the gap between big accelerators and small ones.
And the way I look at them is, like, all of them are basically looking at a compute graph
and trying to lower it into, like, big devices or small devices. So, daisy tuner, again, there
was a talk about that. There's one of the cool ones that I really would like to play with,
and even, like, push it to the micro side of the devices. A-I-H-H graph is something that they
did lower it to a lot of MPUs and actually lower it to the A6 even. So, that actually
definitely has a backend, but, again, I'm unfamiliar with it. I-I-E, and I'm L-I-R, I just
like, I don't talk to me about that. So, tiny grad. So, tiny grad is this framework, not
a lot of people, for some reason, know about it. But it's basically kind of this idea that
if we have a really optimized internal representation of a compute graph, it's very compact.
It contains only small amount of operators. We can do a lot of good things with it.
And, give you kind of like a toolbox of things that can be applied to it, right? So, that's
what I chose. My additional constraint was to basically make sure that it can be done
by cloud code, because I'm lazy these days, and it just won't cloud to do everything for
me. And, by the way, tiny grad is brought to you by the same person who hacked PlayStation
way back when, and, you know, that blog post back in 2010 was such a breath of fresh air
for me, it's the same guy. As my friends, I'm Luca, likes to say tiny grad is small enough
to actually fit into cloud codes, context window. So, that is literally my kind of, you know,
it's not quiet, but like, that's my cloud.md, right? You know, that's what I wanted cloud
to do. And, amazingly, that actually went pretty well. So, like, there's a really good series
of blog posts, and it's not just for cloud to read them. You're welcome to read them as
well. They were reading for humans. They've kind of introduced you to the tiny grad.
What I decided to do is to look at how they implemented web GPU backhand, because, you know,
tiny grad has it. And, I kind of went from there and just, you know, it's actually
amazingly well worked, you know, with cloud code. So, takeaways. Takeaways from, like,
experimenting for basically a week with cloud code. I actually ended up generating, you know,
some semblance of an L file. Kind of tiny grad is all predicated on, you know, taking, basically
a graph expressed somehow and producing a bunch of things that you can then push into either
a device itself. Like, you can literally produce an L file, and all of that is done through
the renders. So, with the renderer, you basically have, again, either a graph that's given to
you, or you can construct a graph, like, I'm doing here with the u-ops. You can kind of like,
you chain them and build that graph in memory. And then you can ask it to render, and it would
render into something like, you know, a cuda actual kernel. So, that's what I did. One of the things
that didn't work out, I didn't actually look quite managed to generate, like, c and c++ code,
that would be compact. So, I need to kind of like, look into tiny grads, pattern matching,
and optimizations like that. But other than that, I'm actually on the way of using tiny grad,
to produce code for microcontroller imputes. Again, my models are yellow and depth perception.
So, I recommend you all try that, because, again, it just happens to be one of the toolboxes.
There is not a product per se, but it's so easy to combine and work with, that you can
run a lot of experiments, especially if you use cloud in really short amount of time. So, that's it.
Thank you so much.