Yes, thank you, Kenneth. You can hear me, okay? Back there? All right, perfect. So I'm
JP, some of you may know me from Twitter or Macedon or whatever, and I'm going to talk
about the program and models that we have in Rockham. Now, I somehow miss how many minutes
I have, so I've waited many slides probably, so let's keep it up. First off, AMD has
too many compilers that some complaint that are here are the times that there's two compilers.
There's, for instinct, there's Rockham, and for Epic, there's AOCC. Don't confuse them.
Anything else is for like development and research or whatever. So Rockham, if you want to
do GPU offloading AOCC, if you're targeting the Epic processors, okay? Two compilers, not too many.
Okay, Rockham, what is Rockham? Rockham is like our software stack, right? So we go from
deployment towards and low-level stuff at the bottom all the way up to actual applications and
benchmarks. And there is libraries in between. We have support for different operating systems.
We're bringing in more support for operating systems. But today, I will be mostly looking at this
part of the stack. So this is the programming models, which is hit, open MP, and to some extent,
open CL, and then sitting on top of that is the library stack, right? So this is like a rock
glass and rock FFT or something. Okay, so thinking about the programming models,
there is hip. Hip is our grid language. So this is basically, when you're used to doing CUDA,
you would, for AMD GPU, you would do hip. So it's a grid language, and I'm going to explain more
what I mean with that. And hip is meant for giving you maximum control of how you program your
kernels, right? But it's, in a sense, vendor specific, albeit, you can run hip applications on
and video hardware. So keep that in mind. Then there's open MP. Now open MP traditionally would use
a for joint model from thinking, even though it has transition more to a task-based idea,
but it's standardized. So there's a standard, and we implement that standard. So it's not vendor
specific. That's also important and keep that in mind. And then there's open CL, which I tend to
think also of like a grid language, because you actually have more control than with open MP,
but it's also standardized. It's actually standardized from the kernels group. And so you can rely
on having a standardized thing. So when one vendor implements the open CL specification, you can
potentially pour to that vendor. Now the other thing is when you think about, when you think
about programming models, you also, or at least I also think about the languages that you can
use these programming models from, right? Because that's important. The programming model doesn't
help you anything if you're programming in the wrong language, right? So this is why I also have
the programming languages here. So we support in Rock and we have C++, C, and Fortran. So basically
you can mix and match here. And then I also tend to think of what libraries do I get, because
library support is also important since you don't want to redo all the work necessarily yourself,
right? And so this is, I have the Rock and Lips, kind of a umbrella term here, and then we're
going to look at Stutt PAR, and also hit for them. I'm going to touch on briefly later in the stock.
All right, so this matrix, we're going to see further along, and I will highlight which parts
of the stack I'm going to show you parts of. But first off, let's look at the models and the
languages. They actually all go through the same LVM compiler back end. So that gives you, if we
improve some part of the compiler that typically reflects to all programming models and all languages,
right? So that's also an important part. And we are, if you download Rock and Right now,
that's not entirely true, because our Fortran compiler is based on an older version of LVM,
because it's still based on classic flying, but we're moving towards this. And you will get a link
where you can actually get access to this stack later on in the talk. Okay, so as I said, I'm going to
show this matrix more. So I will have examples for hip open and PC++ Fortran,
Stutt PAR rock and lips, and I actually think hip forward too. I may have missed to make that
thing visible. Thank you. So first look at, first let's look at hip and C++. So getting back
to the grid fundamental or grid programming languages, what do I mean with that? So when you think
about the grid programming model, then it's how you have to program a kernel and map basically
your problem to the GPU, right? Or how do you execute your program on the data on the GPU?
And this is what it looks like. So you have a grid, you have a grid of blocks and in those blocks,
you have warps. And I know that's in video term, but we have similar things like way
fronts and whatever, but this is people are more used to these terminology, right?
Now one thing I'm going to make is whenever I speak of a lane, I'm actually making what would
be called a thread because I prefer to term lane in this regard, right? That's just a distinction
whenever I say lane, think about like a kuda thread potentially. So when you do hip grid programming,
what do you do? You write a kernel. So here's an example. You put a global specifier here,
and then we're going to have a running example of a XB because that's basically a nice and
easy enough to put on the slides, right? You're nodding, okay? That's right. So in hip you would do a
XB, you get some floats, okay? That's great. Then you compute your specific lane ID in the whole grid
using this. You potentially want to check if you're outside of your data set. And if you're not,
you're going to do the computation, right? And so here you see that all of these things is just
identifying your specific lane, your specific work item, in this grid thing to do the computation.
But so people who are used to program with kuda, this should not be a surprise to anyone, right?
That's just how you do it. And it's also just what you do in hip, right? So that's GPU part,
and then how do you get that onto the GPU? Is he actually say, okay? I need some pointers,
I need some memory, I need to copy some data, and then I'm going to execute the kernel on the GPU.
And we also have the triple chef runs syntax here for actually launching the kernel. You can put
that onto stream so you can have multiple streams and flight at the same time. So that's very much,
what do you would expect from kuda? Speaking of kuda, let's say you have a kuda application, right?
And you want to use that for an AMD GPU, what you're going to do? So we have something that works
sometimes, that's called hipify. I know it is not perfect, I know that, so you know,
but it's there and it gets you some way towards actually being able to run stuff on a
AMD GPU. So that would then give you a hip application to a certain extent. The neat thing about
hipify, what and hip what I think is that you can actually do incremental porting because you
can set hip to comply for in video. So you can replace parts of the application with just
called so the hip run time and actually like hip things. And then at compile time say oh by the way,
I want to run this on an video and you would still be able to execute the same thing on in video.
And you can mix and match this while you're doing actually the porting. And I think that's neat.
Now there's two versions of hipify. Of course, again there's too many versions, right?
There's a text based translation that's more for like if you have a single file you want to do this
like easily in a sense. You would use the text based version and then there's a compiler based version
does is that's more elaborate, but also you have to make sure that your application already compiles
with a whole thing, right? Because you have to give all the inkless paths, all the definitions,
everything you need to actually compile that thing just for doing the hip translation.
So that's something to keep in mind. Okay. Now open mp. I have to say I am actually more on the
open mp team so maybe I'm going to show you more open mp than other people will show you but anyway.
So open mpc plus plus and four to the next. First open mp fundamentals. Open mp I said for
join model. What does that mean? That means that theoretically you would do okay you start
sequentially and then you fan out into a parallel region and then you basically synchronize and
get back to this. Yes, hip basically does the same, right? You start sequentially then you fan out and
whatever. The difference is that in hip you are the person who does how do I map all the data to all the
lanes. In open mp it's the compiler. So it's not you. It's in a sense easier. You don't get the full
control. That's right. But you also do not have to do all the work with oh. How do I you know
which lane is going to write to which data items during kernel, right? So that's I think a positive.
albeit not suitable for every problem you have. Okay. So open mp and c plus plus. Sex be again.
So again we have the same signature. Oh boy. Thank you. Ten minutes. We have the loop that does a
computation and then we basically do impractor on ptarget a super parallel for. We map some data
to the device because we do not need to transfer the result back for the x part and then we map
two from for y. So we bring the data of y to the device and back. Right? So that's xp running on the
GPU with open mp and c plus plus. And that makes it actually possible to have a main function that
has the data does the computation and brings everything to the GPU. So that's working. That's the
whole example basically. Of course I admitted some things for gravity here. So you know I'm not going
initialization here or printing out the values but basically this is a fully functional GPU
offloading sex be implementation. So that's great. I think that's great. The same actually also
applies for fortune. So I know the fortune person by the way. Okay. So I copy paste it this basically
and let's see. So we have some reals, we have some integers, we have a do loop. And then we say omptarget
a distributed parallel do. Again we do the mapping and we get a functioning GPU offloading
for-term program that does the sex be on the GPU. And I think that's neat.
All right. C plus plus stood par and rockham dips. By the way, anybody here who prefers to write
pure C plus plus and does really dislike open mp for the pragmas? Yes. Okay. Okay. I assumed there
were people here in the audience that would have that. So sex be. Let's do a so transform here.
That's kind of nice and nice C plus plus potentially. How do you bring that to the GPU? Well you could
simply do another execution policy here which is stood execution par on sequence, give a compile
of like an apple offload that transform to the GPU using some hip magic. Okay. So if you prefer
to stay within C plus plus that might be a way for you to go. Rockham libraries. Maybe you don't want
to do actually writing the kernels yourself, right? You can use the libraries. And so here
what you would do is you would do rockblas for example. You would create a handle. Then you still need
to do some memory allocations here and memory transfers. So you would need to interact with hip
in that sense, doing hip malloc and hip mem copy. But then basically you make rockblas be able to
access the a pointer on the host. We're using that just for the alpha value which is a scalar.
So we don't bother transferring that to the GPU ourselves. Then we call into the rockblas API
with us for doing this xp. And we're going to copy the result back to the host. And basically
the only actual or all the GPU work happens here. And we have to destroy the handle again. So
that's xp when you're using rockblas. Okay. Now I don't have to preview here. Let's see.
I think now I'm talking about more 4-trend. And that's the next generation 4-trend compiler.
Yeah. Great. So that's the journey of our team. So the next generation 4-trend compiler,
the in 2017 upstream LVM started a project to come up with the next generation 4-trend compiler.
So they started implementing the base language features. Then at some point they realized,
okay, we are far enough when it comes to base language features. Let's do some open-MP host side
support. And then what AMD does is for two years, two and a half years now, I think.
We are actively contributing open-MP GPU support upstream for open-MP target offloading
in upstream LVM's 4-trend compiler. And we also do that downstream.
So there was a recent blog post about the journey of one sum or people from our
apps team with the next generation 4-trend compiler. And you can find the actual article.
There was a blog post. You can find that at that link. They go through what they had to do,
what worked, and what didn't. And I think it's worth a read. Everybody's done taking photos?
Okay. And one more. Okay. All right. So these code examples are taken from the blog post.
Okay. So again, I'm not a 4-trend person. The example is a Jacobi solvers. So we have,
okay. I think I can make that. So we have Jacobi. We have a module. So we have some type here.
And then we continue on the next slide. There are not too many surprises here. So we
simply allocate the components. And then here we actually map the components to the GPU
doing the open-MP target metadata. And then we have some more code that's removed for brevity.
And then in the actual run Jacobi, we have a do loop. Excuse me. And we call it some routines.
And I'm going to show you two routines from that, from that example. So update and norm.
And I'm only going to show this because the target annotation here, that brings us,
brings this code to the GPU. And we have collapse here. So we can collapse the two loops.
And that's an important feature when you're going to the GPU because you want to increase the
iteration space so you can map these things better to the teams that you're running on the GPUs.
And the other part is here is the norm because we have the collapse here too, but we also have
the reduction. And we want to reduce on the GPU too. And so we support that, of course, we actually
have a pretty fast implementation for the reduction on the GPUs, both in flying and in
in Fortran and in in C++. So that's something to to keep in mind as well.
And here is that's where I actually wanted to come to. So we make now preview versions of
rock and available for download through the infinity hub that has the next generation
Fortran compiler. So this is kind of a preview release. So you can download it, you can install it,
you can use it. And we are happy if you do that to evaluate it and open tickets against any of
the components in our GitHub. So you can download rock and hear in the preview build and do your
experimentation. So in addition or sitting on top of the program models and the libraries,
we also have other frameworks that we internally also test to make sure that we hit the performance
every one. And you can make use of that. So that's for example the name one is Kokos.
Sometimes it's a little, at least if you're using the OpenMP target, back end from Kokos,
compile time to kind of bit. Sometimes a little too long. But we enable to use these frameworks
and test internally that, you know, you get the performance you want. So wrapping up,
AMD Rockham is high performance open source and portable. High performance. So it powers
some of the top 500 list leaders. We have solutions for HBCN and I and AI. We have compilers,
libraries and frameworks as part of the stack. It's open source and we are committed to the open
ecosystem. So we contribute a lot of work upstream for LVM at least. That's why I work. So
that's where I'm most familiar with. We're active in community engagement and we're driving
development both in implementation and in standardization. So we are also a member for example
of the OpenMP ARB and we contribute there for the standardization efforts. And it's portable. So
in the sense that we think portable models should be what everybody uses. So you can easily switch
between vendors. And I think that's a good solution also for the evolving landscape of accelerators.
It's not necessarily just tied to GPUs. And of course, Rockham or our products are also supported
through these third-party libraries as I mentioned. So if you're uncocus, you can more or less
easily move to our products or GPUs. And with that, I think I have to show you this slide.
And I thank you very much and I'm happy to take questions.
Any questions? Can you tell me if I can tell you whether there's NPU support in Rockham?
Yeah. I don't know. Sorry. I would love to have the answer myself, but I don't know.
So the question is, LVM's backend has been more opinionated towards CPUs and what challenges
we face. I'm not necessarily a GPU person, but I believe the sum of the optimization passes
to not necessarily assume some of the address-based difficulties that we face. And we've stumbled
over address-based problems every now and then that we need to fix them because there were assumptions
in the actual optimizations or in the co-gen that we had to fix for targets that use more than one
address-based. Yeah. Thank you.
Yes. So whether the Zylings FPGA will be supported in Rockham at some point, I don't know. Sorry.
Okay. Okay. Also the problems.
Yeah. Yeah. So the common was that there's one programming model missing, which is
a sickle, and I'm not sure if there's- so this is the officially supported programming models.
Right? And I think there's a distinction to be made because we actually have more stuff in Rockham
that you can do, but the question is whether it's officially supported. Because we inherited
a bunch of upstream stuff, but we don't necessarily test all of it, but yeah. So on AMD GPU,
you can go the route of sickle through things like adaptive CPP for example, which was formally
known as hip sickle, and so you would get that too. But I don't think it's officially
in part of the Rockham stack. Good comment though.
What's that? What about, so I mentioned that hit the support and video, what about Intel?
I don't know. Yeah, I'm sorry. It's a very short answer, but I don't know.
Currently, it doesn't. For sure. Right? Okay.
Is around, when you're talking about credibility, the most things we have is what's going to be
intermediate by R, where does it lead across parts? My understanding is that AMD has now,
you've got a portability to code low, but when you're actually targeting, when you have multiple
targets in your cluster, then I handle a dozen of this review in my university data set,
re-outwork, and I can't do any easiest on the way to the right amount.
I've already planned to fix that, and something else fear me in the long-standing sound.
Yeah. Do you agree with the different parts that we have?
Yeah. So the question was whether there are any plans for an intermediate representation
that we would be able to do jit compilation more or less to other targets, because right now,
you would basically always have to compile to specific ice, and that kind of limits portability.
So we currently land, or we recently landed patches upstream, that would allow us to compile
for generic targets, so you would say gfx10 generic, and that would give you access to the actual
all of the gfx10 series GPUs. You may still want to compile to the gfx10,
30, for example, that's the most specific one, the ones you care about most.
So that it's easy, or you get the best performance on the target you care most, but you could still
target all the other GPUs, too. For the SPRV question, I'm not complete to sure what exactly the plans are,
but of course we are looking into ways to exactly solve the problem, because it is a problem,
even internally. Thank you.
I did not quite catch that question. Maybe we can take it off line.