Okay, welcome everyone at the next talk. We have Sebastian who had really nice talk about
I'm really looking forward to see the right ones and run everywhere approach for not only AI but also for
all kind of GPU operations that we can do on tensors and using Rust and Slang and maybe Rust in the future
fully, let's see Sebastian, it's all in your hands now.
Yeah, thank you very much for the introduction. So yeah, I'm Sebastian Kose, so we'll be talking
about Slang and Rust. So a little bit of background. I'm not entirely like an AI guy, so at first I was mostly
specialized into Lina algebra and physics engines. So if you are familiar with the Rust ecosystem,
you might know some of my libraries, the main ones being in algebra for like Lina algebra,
matrix operations etc. And rapier, which is a physics engine for rigid bodies, so you can use it for
games, for robotics, etc. And more recently I've got some interest into Lina min France, so I started making this toy
AI inference library, which is named Slang. And the SL in Slang means Slang. So my goal here is that
as a programmer, I really don't want to spend too much time rewriting again and again the same piece of code for
the same GPU operations. So right now if we look at a lot of the existing ecosystem like the
GPU, candle and many inference engines, what happens is that depending on the platform you want to
target, you have to write your GPU code as many times as you have platforms. So if you want to target
metal, you want to you need to write metal shaders, if you want to target QDA, you need to
write QDA shaders, etc. So it's always the same thing, but with different languages. So my goal here is,
can you just run write things once and have it run everywhere? And this becomes especially important to me,
because I'm not just on AI, I want to do this for physics, I want to do this, I don't know for some
synthesis for all kinds of domains. And right now GPU programmers, there are not a lot of
GPU programmers because it's very, very difficult. So if a programmer has to write things 10 times
to support every platforms, that's just impossible. So just even for from a community perspective,
it's extremely important to provide these tools, to provide these languages, but you can use
to write things once and then everywhere. So I've been experimenting with slang, so slang is kind of all
you at the same time. So it's all because it has been run for a few years, it was developed by
Nvidia, but at the end of 2024, what happens is that the project was transferred to Kronos.
So now it's a Kronos project, it's open source, open governance, so it's way more appealing
from a community perspective to start using that. So what's slang is two things, it's at the same
time a programming language for the GPU, and at the same time it's a compiler for that language. So the idea is you
write your shader once in slang and then the compiler will translate your shader into your platform specific
code. So, for example, if you want to support the web, it can convert your slang shader into
WJSL, if you want to support meta, it creates MSL, it can create pts5 for QDA, it can even
support the CPU, and it has some things here, targets I have never really tried, like by
each other, but you can see it has a very, very wide coverage in terms of targets. And actually before
doing some slang, I actually made the same experiment that using WJSL, which is very present
in the Rust ecosystem, and WJSL has this library called Naga, and Naga is kind of very
much like slang, except that it takes WJSL as input or also spv as input, and it will generate
the same kind of outputs, but it cannot do compute only apis because it's a lot more focused on graphics.
So why would you want to use slang? So of course, open source, open governance, it sounds
very promising for its future, you write your shader once, it runs everywhere, and also it
supports both compute and rendering. So if you have a pipeline like some AI inference and then you
need to use a result of that on some rendering shaders or some physics or whatever, you could do this
because you remain within the same API for compute and graphics, so you wouldn't need any synchronization
between CPU GPU, you can just keep everything on the GPU. And like web GPU, you are not limited
by the web GPU standard, so you do have access to more low-level operations, which can be
important if you are interested in supporting higher performance tools for native, so you can
have some sort of conditional comparisons if I am targeting the web, and I don't use these kinds of
very operations, but if I am targeting native, then you can fall back to a more efficient
implementation. It has automatic differentiation built in, I haven't used that actually,
but I suppose it can be very convenient for training, and the most important part is
it has a very modern syntax. So even if you just use slang for writing shaders and creating
sprv kernels from it, that's always your win just because it's so much nicer to use
vanWGSL or GLSL or any kind of these languages. And finally, it supports reflection, so you can
ask the slang compiler information about your shaders, so you can let the slang compiler
allocate binding sets and binding groups for you, and it will automatically tell you, okay,
for this kernel argument, you need to bind it that way, so you don't have to worry about this
kind of registered location, which sometimes can be very annoying to do.
There is this QR code here, so this is a talk from Kronos, which I found extremely interesting,
so if you are interested in slang, you might want to start with that talk because it gives
a lot of information about what it's about, how it works, the limitations, potential use cases,
so I found it very, very interesting. So this is slang, so this part here,
this is only about writing and converting shaders. Now, this does not tell you how you can use this
from rest, if you actually want to run it for your inference or for anything else. So you need
four parts, first you need the slang compiler, so this translate your shaders from slang to your
target, you need responding, so you can actually call the slang compiler, so the two blue things here
so slang C++ and shaders slang, these already existed when I started this work, so we already
had the rest bindings, but these are low level bindings, so I kind of had to make a very small
wrapper on top of that to make it easier to use for the simple use case of translating the shader,
and then the hard part, and I believe it's the hardest part, is to write an i-ch i, so i is
run the hardware interface, so the idea is that on the one hand slang allows you to write your shader
ones and convert it for every platform, and on the other hand you need to write your CPU code,
so all the CPU side orchestration, creating buffers, launching a kernel, etc, all that you also want
to write them once and have them run on every platform, so it means that they need when you write
create buffer, it needs to translate to the meta API if you are on the quest to translate
to the Q-der allocation, if you are targeting Q-der, etc, so you kind of
still need this kind of multi-backhand interface, but only for the GPU API not for the shaders themselves,
so I've started walking on sling a trail, which is essentially that API that you just call
and depending on your platform it will automatically use the proper low-level hardware API,
it's very incomplete right now, so right now it's only support compute and only web GPU and some
Q-der, what's interesting is that if you are using C++, be aware that they already have
so Microsoft already has an IHI implementation for sling, which means that you will probably
already be in a very good place for calling sling shaders directly from C++, but in the rest
but it doesn't exist, and finally the more interesting part of the actual tensile, so
as tensile is the tensile library which provides tensile operations like metric multiplication,
addition of shovectors, these kinds of very general operations you might want to use for
inference, but also for physics simulation or anything else, and that's kind of the reason why
this is a new library instead of just creating a sling backhand to something like the MSB or
candle, because I really need these operations to be reusable from different domains and not just
actual intelligence, slay is the inference library and an exclusive slash phase are just
physics simulation, so it's not related to IHI at all at the moment, so to give you an idea of
what sling looks like, this is a very basic kernel, and the most important part, and I think it's
kind of silly that this today, this is still important, you can see that A and B, the true tensile
arguments, they are specified as arguments to the add assigned kernel function, so these are not
global variables that you can see in a lot of shading languages today, and just that makes the code
extremely more easy to maintain and to read, and the rest is kind of very standard,
some things this does not show, is that sling has a modular system, so you can from what's
kernel, and you can see here this is, and everything else, this is generic,
graphic up to the backhand, so you can write this once, and then it will specialize automatically
depending if you select web GPU backhand, a queue that backhand, etc. So the rest is kind of similar,
you can specify your kernel arguments, so you have a bunch of GPU buffers, and you don't have to know
like from what backhand is the first original from, and then the lunch function just takes
these arguments and trigger the add assigned operation. So not anywhere here, you've seen any kind of
bending groups and bending indices, because this is automatically allocated by sling, and yeah,
you don't need to worry about the backhand, because this is automatically handled by your generic
arguments. So here I've put a bunch of links, which are the main libraries I mentioned here,
slay is very incomplete right now, so I'm still at the experimentation phase, and you get a bunch
of operations, so five implemented Yama, and whispers, whispers, model, and the idea is to continue
to provide these low-level operations, and more models in the future. There is a similar experiment,
I've done before starting slanks, so if you are interested in a purely web GPU, the WGSL implementation,
then you can have a look at the WGML. So what's next? Right now, like I said, I'm
still in the exploration phase, and my goal is to promote this idea of writing shader once, and
have it run everywhere, so that we get more and more contributors from the way the range of
backgrounds, and not just AI, and sling is very nice, it's extremely close to the ideal solution,
except that today does not have a package manager, but I think there's some potential
work in the future that makes that, but more importantly, it does not allow any kind of CPU GPU
co-sharing, just because it's just a completely different language from Rust. So,
I do have one last experiment, I've started recently, which is to use something called RustGPU
and RustGPU. So these are libraries which provide a computer that can for the RustGPU,
which helps translate compile Rust code directly into spv or directly into pts.
So with that, we could automatically use the cargo package manager, and also for sharing shader code,
and also we could have some ways of sharing code between GPU and CPU, which is especially important
if you want to share the definition of structure, for example, to avoid some kind of mismatch,
which can be very difficult to debug. So that would be my last experiment before I actually
dive into selecting my preferred solution for this Rust platform, single source GPU physics
implementation, and I also have a very strong focus with here on generating data sets for
embedded AI based on GPU physics simulation. So yeah, thank you all for being here, and if you
have any question of hesitate.
So, the question is if I support generics, so, well, if free aspects to that, so we can talk about
generics on the language itself, so this one I don't need to support them, because the
slang compiler does that for me, generics on the Rust side, right now it's only about the backhand
selection. So the only generic part you can specify is whether you want to use the web GPU backhand
or the QW backhand, but it does not let you change something like what group sizes, so you can
visualize these yet.