Thank you for coming to my liking, so I am unmuted, I think there's no red light.
It's not red.
All right.
So I work on Foothark, a data-parallel function program language.
It looks like this.
I'm not really here to talk about the language itself.
The idea is that you write this kind of very conventional style in the compiler, choose
on your program, transform it, optimize it, and the language generates code for GPU.
So I'll have a parallel platform, so I will talk about GPUs today, and then hopefully
it was pretty fast.
That's what least they did to go.
So it's by itself that there will be a GPU language, it's highly good, but we have a compiler
at generative GPU code.
And it has three production quality GPU backgrounds that are supposed to work on every
program.
One of them targets the API, OpenCL, which is an open standard for, well, GPU programming
a little while, and that actually, and it's implemented to various degrees, by
Nvidia, MT, Intel, and so on, so it's pretty wide as a port, it's an open standard.
So in principle, we should not need anything else, but we do.
We also target CUDA, which is Nvidia's proprietary API for running code on GPUs, which
is pretty dominant, and it's worked with well, and VDGQs, but only in VDGQs.
And then we also target CIP, which is an MP taking hip and filing of the serial numbers,
or running a pro step in retaining CUDA to hip, and then that works pretty well on MPTPUs
as well.
So ideally, when we generate equivalent code from compiler, targeting each of these three
APIs, we would expect identical performance.
So we had actually the case, buvigate identical performance across these backgrounds, as
far as we don't.
So the idea is that we will take equivalent benchmark programs, written in this two-time
language, but we just imagine it's a live VQ program, it's not really the point,
it's a language, it's not really important, but then we take these 48 benchmark programs,
and pile them with different baguettes, and run them on different GPUs, and see how
how fast they run, whether they only change this, how which API we target.
And that would do a lot of data, because we have 48 programs, two, three baguettes, and two
different GPUs.
But this, on readable graph shows, how fast OMCL is compared to CUDA, and hit respectively
on an MPTPU, and then in VDGQ, and a number greater than one means that OMCL is
faster than CUDA, or hit respectively.
And you can see that in many cases, the difference is less than 20%, but there are some
significant outliers.
And I would like to talk about briefly the reasons for these differences.
The compiler largely generated equivalent code, but there will be some exceptions, and
the actual drivers will also have an impact.
So one of the most tedious reasons, or less interesting ones, is that by default, OMCL
allows a floating point of single position floating point numbers to have incorrectly rounded
motivation and square root operations.
But on other hand, they will be faster, because they are wrong, and it's usually easier
to get things run faster, you don't care about the results.
So this explains the fact that for example, on the GA 100, GPU, the input benchmark is 1.43
faster for the OMCL and CUDA, and that's almost entirely due to the fact that if you
allow a CUDA to round incorrectly, it'll go really fast.
It still behaves nicely.
It's actually in stop validating with this, but it's kind of an unfair comparison.
So if you tell OMCL to please give me the right numbers, then many of these differences
will go away.
I don't actually know why that's a default on OMCL, it seems a bit weird to me.
Another more interesting difference is that some advanced program requires an algorithm
called a prefix sum, or a scan.
The best algorithm for running for computer prefix sums on a GPU is the so-called decouple
look-back scan, developed by some Nvidia researchers, and it really goes against how
you're supposed to program GPUs.
It depends on some very sophisticated communication between running threads at the same
time, so they write to some shared memory, and they read each other's results, so you
need some progress guarantee, you need about cash mechanics and so on, that are not provided
by the OMCL spank, and are sort of only provided by CUDA, because these Nvidia researchers
came out of the algorithm, it worked in practice, so Nvidia says if you do exactly this,
then it's okay.
They form us a little bit better later, but the point is, on OMCL, you can rely on you
run this fairly far fancy scan algorithms, we have to pull back for a slower one, and that
explains performance difference for these benchmark programs that we like heavily on
scans, which unfortunately, but that's, in principle, you should be also able to do this
in OMCL, but I don't know if anyone who's actually managed to get a turn work
to get a reliable one.
Another problem is that on MGTPUs, these OMCL implementation limits thread blocks to 256 threads,
so that's the limit of how it is group your threads, that's the hardware limit, because
it doesn't exist with hip on the same hardware, it's just a kind of weird driver software
limit, and there's some fine print that applies to that, so that explains the performance
difference for some benchmarks that are very sensitive to the block size, again, kind
of a strange software limit in MGT stack, and another similar weird thing is that you
cannot rely on BigQuery, they level two cache size, no problem, it's a less as kind of thing,
you can recover, you cache size, but it's kind of unclear whether it's mostly level one,
they will two or something else, and it doesn't give the same number on the video, and
give you the little two cache size, and then do the level one, and this affects algorithms
that query the cache size to make intelligent choices, because then on the way we wrote
it, we assume that it's a little too cache size, and if you get the level one on MG, then
you get some automatic tuning, and it doesn't really work so well.
And very another instance of Ubisoft may get difficult to query hardware information, is to
cannot query how many threads you need to fully set your GPU or how many will fit on it,
and we do generate code that tries to query the GPU to figure out how much can we fit,
and then make choices based on that, now the interesting question is that we can't find
information on an open shell, make a horrific guess instead, which generally is smaller than
the precise number, but it's actually not given that the right number produces faster code,
if tuning how many threads you've given to your GPU program is a pretty complicated process
or it needs a very hard to predict, and in some cases the heuristic actually runs faster
than the correct number, so this is kind of, it explains difference, but it's not really
something that's a big problem, or at least the heuristic is not necessarily worse than the right information.
Another problem is that when you're talking to a GPU you're going through an API, which has
some overhead on the host, and in some cases we can see that a program is slower, for
a little reason that we can attribute to the code actually running on a GPU, it's only due to
some host side overhead where even just touching a GPU is slower for a little reason that we can't
really determine, generally we see that OMCL is slower, so if you just want to run the minimal
unit program on a GPU it does nothing, then we have absolute OMCL is little bit slower than both
hip and then with your. This usual does not matter because usually when you touch the GPU you
tell us to do something costly, but there are some programs or benchmarks with the degenerate inputs
where that really do almost no work, and then it's basically just measuring the the host side
overhead line. One of the small data sets for the trace benchmark or anybody has some very
large ones and some very small ones, which we obviously have a big variance here. It's difficult to
find the exact reason for this long. Then there we also have bounce checking, this language also
support full bounce checking on a GPU, it's done with your code transformation that generates
some slightly odd code, and it's something that we suspect that the code generators for the in the
OMCL code and hip implementations are sometimes a little bit bad at handling, so for no other
reason that the bounce checkings, which by itself is not an expensive, we see that that code
is just turned into much worse code with OMCL, then in code that we're running on the Nvidia
stack or on the AMD stack, it's the opposite if the OMCL code action becomes faster than hip,
so that's just because there's a black box compiler after us, it does something to the code
which generates, I mean always quite sure what, in this case there's difference or we can
really attribute to anything else, and that's all things we can't figure out, there are some
programs where there's a difference performance between OMCL and hip, and we simply cannot see why,
we see that a GPU curve runs faster or slower, and even inspecting the generate machine code,
I can't figure out exactly what the reason is, I suspect that it's due with reduced allocation
or something like that, it's a very similar issue, very low-level thing, because it tends to
have a for kernels that are compute down, not memory-bounders, where such things might have
a major impact, but I really haven't figured out why. We can solve some of these issues,
because that's really too category-see, one is telling OMCL to do a property off,
they are, they're still managed to provide all this hardware information that OMCL does
and allows to carry otherwise, if we do that, then this becomes of the summary table, and now
the vast majority of the benchmarks, there's less than 10% difference between OMCL and
CUDA or OMCL and hip, and most of the significant cases where OMCL slow is because of this
issue, where we cannot run the best known scan implementation, because it requires some memory-bounders
guarantees that we don't have OMCL. So, my conclusion is that it's not really difficult to tie
it all of these APIs with a single code generator library, but there are some performance differences,
performance-bounders is tricky, and most of it is down to missing hardware information,
and I would say that if you're only careful in video and MG, just probably tightening
OMCL, it's not worth it, because CUDA and hip are pretty similar, and overall less trouble.
All right, that's my talk.
Perfect timing. One question. Who has a question?
Yeah, so there are different versions of the CL and we tend to have the ability to work.
What originally did you use across the board?
Uh, this will allow me to explain the case at the beginning of the board anything besides
true one, true one, true one, the one, right, and then MG things.
Uh, you know what we just used before, we said we used some of the number data.
We're going to need that library to pick.
I think it's, I think it's, I think it's going to be fun soon, I think it's been,
You