Sorry, great, great.
Scoping out the 10th floor, warm hall.
Take it away, Peter.
OK.
APPLAUSE
OK.
So I'm holding here the 10th floor and 300
asked my purchase from 10th floor and so online
store.
And if you like what you hear in this talk and in the next
talk, you can go to this store and via own warm hall card.
A bunch of other form parties in the store as well.
But this is not a sales talk.
It's a first time talk.
I will here to talk about open source software.
So 0.1, 10th floor, make a open source software stack for that
hardware.
And all of that that hardware is up on a GitHub.
We can find it.
It's all great.
Why did I want to talk about software?
I want to talk about hardware.
Can I believe that good software,
it's really understand the hardware on which they are running?
So I'm going to tell you all about the hardware here.
And then you can think of what the software should be or better
understand why the transfer software is the way it is.
Now, I don't want to take my card apart.
It's like, hey, it's good.
Money for it.
But if you did, take off the case and the heat sink in the pan.
You would see this photo here.
But I'm not talking to talk about PCBs because I don't do PCBs.
So we're going to switch to this much simpler block.
So on the bottom of it, a piece I express thinking on the left,
got two QCP pages, and then two proprietary internet ports on the top.
And then in the middle of the card, what is two,
one-for-seps each fine-fire bunch of DRAM.
Now, if I show things around just a little bit,
I can show you how everything is wired together.
There we go.
So I've drawn three types of connection here.
Each green line is representing DRAM.
The red line is representing PCI express.
And each blue line is representing byte write 100 gigabit.
Ethernet.
Now, in theory, each one-on-one chip can drive 16 ports of that 100 gigabit.
Ethernet, but the form part of this particular board,
only gives you 8 ports with that on the left chip,
and 4 ports with that on the right chip.
Which is good, but it could be better.
But in any event, the scale up factor here,
the scale up so here is Ethernet.
At some point, we're going to get larger and larger,
and we're going to get too large for a single chip.
Even if your single chip is like the whole size I hold,
wait, that's super us.
You will still need multiple chips at some point.
So the hope is you could buy as many chips as you need,
and then link them together with point pointer
if that in whatever topology makes sense for you.
Is that what we wouldn't say here?
I think so.
Yep, okay, so.
However, if you scale out too much,
then giving each chip its own PCI Express connection,
it's going to get a little bit pricey.
So the hope is that we can get away with giving
only some chips a few sites,
access, connection, and rely on Ethernet to reach the other chips.
And this board is designed to give you a little taste about.
So as you can see, the chip on the left,
has a PCI Express connection,
but the chip on the right does not.
So it is only vegetable beer that is that link,
printed on to the circuit boarder.
So from the software point of view, this means that you can
aim that memory from the left chip,
you can't aim that memory from the right chip.
Implications for software there,
I'm just going to leave that since there,
and let me flip the arm.
So if we zoom in to one of these chips,
we see a 10 by 12 grid of tiles.
I've got various types of interface tile around the edge.
We've got Ethernet tiles, left and right,
DRM tiles, top and bottom,
also in the bottom, one piece I express how and one art tile.
Now, on the form factor of this particular board,
two columns of computer tile, which is all this stuff,
in the middle, are disabled for yield reasons.
So I'm just going to mark two columns here.
I can randomly chosen two columns,
but it will vary from chip to chip,
just based on the yield,
where the defects are on these tiles.
If you buy other form factors,
you can get more computer tiles,
if you buy an N15S board,
which I think someone has.
Back here, you get some two tiles,
and if you buy a Galaxy system,
you get all 80 computer tiles on every chip.
I, that they say of the really good yielding chips
for that particular product,
which will cost you lots of money,
because people have to make some money.
Okay.
So what's next?
Yep.
So I've numbered each DRM chip.
Each DRM tile, based on the DRM model,
it takes,
connects to,
I've numbered each,
it's not tile initially,
because point point,
it's that the things are unique.
I've not numbered any,
if the other tiles,
because all the computer tiles,
are identical to each other.
Now, before we delve into the various types of tile here,
I'd like to talk about the memory hierarchy.
Or,
more to the point,
the lack of a memory hierarchy.
So on most GPUs,
if you get one,
today,
you'll have a single other space that spans the entire
chip,
kind of covering everything from the computer to the memory,
and you'll have various layers of caches in between the compute and the memory.
But we have,
not that here,
no hierarchy of memory,
no caches.
And instead,
each tile has own other space.
And if you're running code on one of these tiles,
then you can only access the memory of the local tile.
So your load and saw,
as far as you can,
only access the local tile.
But of course,
we need some way to move things around further than just a single tile.
Otherwise life would be incredibly boring.
So that's where the network on chip comes in.
Now, before I can add that network to this diagram,
I need to make some space.
So I'm going to get rid of the external things here.
Unshuffles around from the physical grade to the logical grade.
So all the same tiles,
but we don't just know,
viewing them is slightly changed way.
And now I can add the network on chip or knock to the diagram here.
So this is all of these are purple arrows.
And this is things every tile to every other tile in a to the tourists.
So if you go off the white edge,
you end up on the left edge,
if you go off the top edge,
you end up on the bottom edge here.
And each,
not transactions,
any tile can initiate a nox transaction.
And each transaction is effectively an asynchronous memory copy.
Now, I've put the conceptual API to this on the site here.
You choose two tiles by giving the x and i coordinates in the tile grid.
And you copy some bytes from the other space of one tile to the other space of the other.
And in most cases,
the tile that is causing this transaction to happen will be either the source or the DST.
But you can do more exotic things if you are so wish.
Now, I should have also mentioned that each knock transaction has some overhead to it.
So if I could have peak performance,
the transfer size,
the number of bytes,
what's been the range like 1 to 8 kilobytes?
But if I used to the N video world,
it's going to sound like quite a lot.
So quick change over to the CUDA world.
In that world,
you're thinking like 120 bytes,
it's a kind of like transfer size,
because in that world,
you have 32 threads in a warp.
And each thread is used for white load,
and the harder it will coalesce all of that in the units of 128 bytes.
But sitting back to DST,
we have none of that.
We have no coalescing of memory.
You have one thread,
and you wish you a single large 1D memory copy.
Which is a little bit limiting,
but hopefully not too so.
I guess also limiting is that we can only move
white words and upwards on this particular network.
But for that problem,
we have a very similar solution,
which is,
a second network on chef that is identical in every way.
It could all of the arrows go in the other direction.
So this guy can go,
you know,
left and down,
and again it's a 2D towards.
You've got the power left,
you've got the power right.
If you've got the power,
you've got the power on the far top.
And both of these networks,
in addition to doing plane copies,
can do multicars,
and all three of these things,
plane copies,
multicars,
and a tonics,
are naturally a synchronous.
So you far them off,
and then if you care about whether they.
Complete,
and then you can,
then you can pull to ask whether they have,
yet completed.
And this is,
the second major difference,
to the usual GP model,
in that if you need to hide any,
latency,
I of memory,
then you are using asynchronous API as to do that.
So automatic,
context searching,
between multiple boards.
So again,
another major thing to bear in mind that.
So,
you're going to want to hear about,
these are various types of tile.
And I'll see that the,
the compute ones are,
the magic really happens,
but we're going to keep you waiting just a little bit,
and look at,
every other type of tile first.
So.
So,
looking at the art tile first,
this guy is boring,
but it has to be that to,
power,
some more thought,
in research management,
you know,
things that have to happen somewhere,
but you don't really have to care about what they,
really are.
So code for this particular tile is currently closed source,
but it's not really a problem,
because you can basically figure out
that this tile is even there.
Other than if you need to be set the board,
or change the plot speed,
so we can move the swiftly on to a more interesting type of tile.
So,
Dram tiles.
Each Dram tile is a bridge between the,
never come,
but you don't hear any purple and in till,
and they're Dram module.
Each Dram module having two gigs of GDR6,
and these tiles kind of initiate
not translations of their own,
but you can read or write from them from other tiles,
using a not read or write.
And if you do do that,
then the other space of these tiles is extremely simple.
The first one gig is the first one gig channel.
The second one gig is the second one gig channel,
and the rest of the other space is a map.
So, these lines are, you know,
where you're restoring all of your large...
T-t-t-t.
If we move on again,
we reach the PCI-e-x,
the PCI-e-tile.
So, this guy is a bridge between the network on tip,
again in purple and in till,
and the host address space.
So, if the host system issues are moved or write,
against the board,
this tile will translate that to a PCI-expertsuit,
moved or write into a knock,
moved or write against the address space of some of a tile.
And we've got a four hundred,
nine, six, no white,
aperture for that purpose,
broken up into a number of regions,
and each region can be mapped to a sun per tile.
Other states,
depending on who you want to map them to.
Going the other way,
if some other tile issues are moved or write,
against this tile using the knock,
this tile will translate it to a PCI-expertsuit,
right, against pin physical post memory.
And if we're using one gig,
huge pages,
you can pin four,
gigabyte aperture going in that direction.
So, moving on again,
we finally reach something particularly interesting.
So, this is an internet tile.
And at the top now,
we have a, for example, risk five-quarter,
which is what makes this guy interesting.
So, this is the RB32,
I am instruction set.
So, we've got some of that.
Inter, GPR's death, 32,
but it's wide and 32-bit,
and you can do all the usual risk-five things.
So, you've got to ensure,
hey, I'll do that.
We can do control flow with branching and calling.
And we put a little load-stoy unit.
And then a number of things on the memory box,
which can be accessed by that load-stoy unit.
So, you've got to block up S-fun there,
in ping that holds all the code and
data for this risk-five-quarter.
And then on the left of S-fun,
to not interface units or N-I-U's,
and these are how the risk-five talks to the not.
So, these guys can either initiate
not transactions of their own,
or they can service transactions
which have been initiated by other tiles.
So, if you are running code on this risk-five,
and you want to initiate a not transaction,
then you take the argument for the A-sync.
Then copy, you put an interesting memory map to registers on this,
load-stoy unit, and then you write a go signal again
with turn memory maps.
But just enough.
And if you care whether a thing has a computer
yet again, you can read from a memory maps.
But just search to find out what other things
have have finished.
So, if you are used to doing any kind of low-level
kind of like driver-dev,
then this pattern should be a community.
And if not, you can wrap up the nicer.
If I hide all of the finicky little details.
Now, on the other side of S-RAM,
we have two other units,
we have the E-Net transfer unit,
and the E-Net receive unit.
And this per unit is again an A-sync-less
memcopy thing.
So, you can do a A-sync-less
memcopy all bit only from the E-Net hall
that is the app source here to some other E-Net hall.
So, you know, data can flow from the S-RAM in
the style through the E-Net transfer unit
over a point pointer.
Is that linked to some other E-Net
tile through that tiles are actually,
you know, and into that other tiles, RAM.
And the S-RAM,
thus, by the Dust Nation Raster,
to write to hence why we have not had to put
the specific E-Net on the memory bus here.
So, I should also say that if the E-Net link
flaps, then the risk-5 is responsible for
noticing this and retraining the E-Net link.
And again, there's a tiny little bit of a closed source
somewhere for doing that retraining should you need to.
Which is unfortunate, but it is what it is.
Time, heck, good.
So moving on again, we reach what we are actually
kind of properly interesting.
So this is a computer.
Somewhat similar to what we just saw.
All that was five risk-five cores up there,
rather than just one.
Again, it's RB32, I am instruction set on each.
And one bare metal thread running on each.
So again, it is one thread per risk-five core.
And if you want to hide any latency, then you
are using asynchronous APIs to do so.
There is no automatic context switching
between multiple threads.
Yeah, OK.
So again, on the left of SVM2 and I use just as previously,
and then a huge block of RAM here.
We've got more RAM here than on the previous tile,
so that it and tiles had 256K of RAM on them.
These guys have a meg and a half.
But it serves the same role that holds all the code
and the data for these five risk-five cores.
And then the majority of the tile on the right
is taken up by this 106 copress, which is where
the magic really happens.
Now, I've drawn it on the general memory bus here,
but in practice, only the three risk-files
on the right can meaningfully talk to it.
Which leads to a very natural division of labor,
whether the three risk-files on the right
are the copress or leaving the two risk-files
on the left to drive the two NIUs.
And then they can also communicate to SVM2 shattings.
So obviously, we need to zoom in.
So this guy, because this is about the magic,
really happens.
And there we go.
So I've drawn to a front-end type bits
in green, back-end type bits in brown,
memory is in pink, and instruction arrows in lilac,
if the lilac is about on the slider.
So at the top here, we've got three instruction
fibers, fly-forging, another word for Q.
If that's not a turner's familiar to you.
And there's no real control flow in the front-end here.
If there isn't a control flow, which is there,
there will be then the risk-file calls to now zoomed out.
They will resolve all of the control flow.
That you have, and then each risk-file
will push a stream of tensile instructions
into one of these fly-files.
And we've got three fly-files here.
Hence, why I only three risk-files
can meaningfully talk to this thing.
And then, so there are two ways to push instructions
into these fly-files.
The first one you'll probably guess
on the theme of what I've been saying
is that you can do memory-mout rights to some memory-mout
just to write into these fly-files.
You only pushing a lot of tensile instructions
into this guy to do the work.
So there's a second, chief-water to push things in,
which requires a quick diversion into risk-file.
So if you're familiar with risk-file,
you should know that three courses of the off-code space
is preserved for the RVC extension.
But none of these calls infinitely RVC extension,
which leaves that to be courses of the off-code space
available for other use.
So think that's repurposed.
And any of these risk-file calls are given
an instruction in the RVC off-code space.
They will not treat it as a RVC in such
and they will instead treat it as a opaque
sedative value to push onto the fly-file attached
to that call.
Whichever way we're pushing,
in these fly-files, if we then look down,
we get to the next unit, which are these macro-expanders,
which are configurable units that allow
one instruction to expand out to several.
And if you configure and use these guys properly,
then you can keep the back end of the tensics unit fully
if it's saturated, only need an issue
of one instruction every kind of ten cycles also from the risk-file,
because you can expand out one to ten on average,
which frees up the risk-file to do other things,
such as resolving all of the control flow.
Stopping down again, the next unit here is the sixth phase.
Asian unit, and it contains, well,
it gets three streams of instructions flowing
in from the top, and it then marks us now
to the various back end units underneath.
And it contains the mutixes and some of the force
to control the wealth of all three of things
between these three streams,
which we're going to want one to use
because the various back end units here
are shared between the three incoming streams,
both there, execution units, and the registers in all of them,
which is a little punchy.
Didn't rock quite right, just coming up.
OK, fine.
I will therefore give you very briefly
on all of the units here.
Unpacked on the left to move stuff from S-Farm
into the unit, we've got mammals on the middle unit there,
that's where the mammals really happen.
FPA-T4, or BF16, or half-rate.
Well, BF16-ish at half-rate is a little weird.
F16-ish, it should curate,
and then you can push out the data back out
from your packing it back out to S-Farm, after you finished,
working with it, and then all of the contact on the side.
You can either drive from Memum-Out rights
to the rest files or from the skeleton
on the side that was that helped you drive
all of this contact, and thought, and out of time,
so that will have to do that.
And you can then zoom out and we are back
to, well, we solved it with this nice, perfect, thank you.
Give him a round of applause.
And just like the previous speakers, I'm going to kick Peter
out over there, so you can go find him if you have some more questions
about his presentation.
Martin, there you are, ready?