We are going to explain what this title means, hopefully by the end of the talk.
So this is the agenda.
Is it microphone too high?
No.
Just fine.
So, we are going to explain, we are going to introduce the ETI minion, what it is, by
scaring the architecture, the customer extension, and then some very brief conclusion.
But me, well, this hasn't changed since last year, this slide except this line and this line.
Oh.
Right.
So, that the only thing is that now I am not an echo, it has zero to three times to do my
personal projects.
So what is an echo and why am I here?
So it's our elistage startup, we are really trying to take the world open seriously when
it comes to AI and what we have amongst other things we started, I found that we will
find it at that link.
We famously, I think, for many people, acquired the experimental technologies IP and then we
have been sourcing it.
We don't stop there and in every discussion that we have internal to the company, opens
us harder and what it means and now we are going to do it is like things that takes 90%
of a fund discussion.
So what is it for the ETM union things?
ET is actually a rest here but essentially it's the prefix that we use for everywhere the
stunt of the architecture technology when it's been open source because all the code was
as planned to the technology and it's okay, let's call it ET.
Of this slide, that that values links here just to show how open we are on every time, this
is for more emulator, for example, everything, but the things that is still important
for this stunt case test is that they here we are going to find all the manual and even
the schematics of the board are you going to see soon.
So what am I here, why am I here, good question.
What I wanted to do is like now that we have to open source this and we found we got the
IP and we saw the amount of work that was done for Esperanto and we open source and get
that think this was the perfect room to actually discuss this architecture and what they
did and things that I like but I'm not going to express opinion in this talk, by the way.
This is not going to be a comparison between existing operables in the director of this
fire extension, we kind of had a talk about that and it's not a declaration that this extension
and whatever we have in the regional it is a one is absolutely perfect, far from it and you
would see why.
So in order to start doing this we need to start talking about the architecture because
you cannot really express anizer if you want to scrap your architecture for which you
was created.
So here's the board, the board of course behind this very optimistic hit sink you will
find the extra one ship, you find that PDR4 which is the most valuable thing of the whole
board and a P-MIC and a FDDI.
So behind this trip actually what is the it is a one is it is actually going to find
a thousand eighty eight minions, so small is five core, four out of all the core and they
all learn about the under eight under my guides again do not plus that hit sink for that.
Right, so this is how logically I see the board when I have to think about it.
You're going to find the it is a one which is the main ship, you're going to find the PCIe
that connects the activity is a one, so it's the LPDR4 that's a microcontroller that controls
the voltage regulator which is I ask us his slave of it is a one and then you got the
values you had to actually control the board.
And now we're going to start looking inside it is a one, at least one person in this
room will be offended by this slide on this slide because this is a logical view and doesn't
show what kind of beautiful mesh architecture the ship is actually made of.
The essentially you got six by six, shires everything is a shire and you got four and four
memory shire on the side and we need to start getting used to this kind of no
magnitude that the Esperanto domain for sure they got shire, they got minion and they got
so the shire essentially is a module that in which the chip is divided into.
So let's start from the things we're now going to look at the IOS shire.
The IOS shire is contains all the values devices but that's a contains full
actions which is the opposite of minion which are big out of all the core will connect
to know them and there's a very small minion that is just minion called the service processor which
is the actual mic of a controller that controls the board. PSA shire memory shires can imagine there's
a lot of IP there but it's not ours and the minion and so what is going to look at this is the
minion shire. The minion shire is the actual compute shire and this is where all this compute
calls that I saw are. Let's and so this is what I'm going to in so what is a shire in a minion term?
It's going to be a hierarchy of definitions so bear with me. So a shire is a compute shire
minion shire is actually composed by full neighborhood again no manager and full megabytes
or L choose L3 cache. Now what does L2's L2's L3 cache pins is a complicated thing and
I'm going to drive there to successive approximation. So H shire has this full megabytes
of dual T cache which means that actually in Esperanto you can actually see the manual
how it's done is have to be quite elegant in my opinion. The cache module can actually
be configured across the whole chip into this full megabytes of S3 can actually be divided into
12 to cache part of an L3 system wide cache that then becomes actually the global cache before
the RAM or L2's cache pad and this can be actually configured runtime but I wouldn't suggest
doing that. So keep in mind the cache is going to be very important for a thing.
Then we go to the neighborhood and we open up the neighborhood and what we found in the neighborhood
is eight minions which is the actual CPU and I shared and one cache, iCash, instruction cache.
That's the simplest slide so we go down and we actually see what a minion is like and I repeat
we have a thousand eighty eight of them in the chip and well it's a simple another core
two hearts per it per core it's a Lv64 IMFC. So it's a 64 bit as you can see it's
cleverest in the italics and we'll see why and then as you can see there's a big thing called
a VPU with eight lanes and that's going to be also the talk. There's also a full keyword L1 cache,
L1 data cache so please note the instruction cache was shared in the neighborhood. The D cache is
specific for the CPU and you start to see the pattern of the design here the D cache is
actually configurable in its minion and I can split it so I can there's the simplest mode in which
the both hearts share the same D cache which is only for a kilobite or I can split it and then I
can not only split it but also create fifteen half a kilobite per a heart of cache and
a kilobite of scratch pad. So what is scratch pad is just a buffer and of course you have to
scratch pad is a bit farther away and share custom minion the L1 scratch pad is a very fast memory
right by the chip is and this is when things start to get complicated if you think this was
fun to program because there's absolutely no currency about all the caches that I spoke to.
So you need to think in terms of cache lines when you program this chip and you have to decide
that which sits where and there's actually quite a quite a few important things so actually you
control the CPU but the caches would be the things we'll speak in the most.
Right so as I said the minion which is now we know what is the ET minion I say means
has custom extensions but let's start from the basics so right so the manual is originally optimistic
it says that of course as you can see as fancy and although fancy as I essentially not quite
useful but you know like this is an IMFC standard but he has machine models supervisor I say
nice I can run operating system in it except for some slightly major router so pitch table are
unusable and but there is a PMP of some sort so you're going to be fine one of the biggest
minor deviation of the of the design is that the performance monitor units actually
can't everything in the everything in the neighbor then you can actually start seeing how this
thing is supposed to be programmed because when you have an i cache and the performance
consider a share essentially the neighbor becomes to compute unit so this is where we start from
this is the major bug and the basicizer and from this we built so and we're going to look at
three basic extension the first one is the SIMD extension that might get some people excited in
the room and here there is the atomic extension and what it means to have atomic in a system with cache
a non-coerent and the third one is the tensile extension I don't have the time no one I think has the
time to explain the full tensile extension at all but I'm going to discrete basic mechanism for it
right let's talk about the SIMD instructions so this is the thing about the SIMD like
this chip has not been the same recently I think it was definitely less second and 16 more done
for that and so this was before a VV and to me that I come from x86 feels like more of a
classic also in the extension for his file it doesn't feel like it when you have to a program it
the biggest thing that I did is that they definitely didn't do for example what the p-accession
does that just groups or just so together they didn't do whatever VV does that kids a different
bank or register what they did was extend the floating providers to 256 bits and people can
start imagine working possibly go wrong with that but in general it's actually quite elegant
if you fix a compiler when spilling the registers which we did not so we have this 250 bits
they're actually this is why if you look at the previous slide you saw seven lanes because each of this
is considered like eight eight that you do bit facadelament.
The other things that I did is of course a mask zero is that we implicit that kind of a standard thing
but they actually added eight bit eight bit eight mask registers so the reason is that is of
course that's 64 bit 64 bit registered is added and that is the mask.
So, how can we start if you actually look at the manual you will see a lot of instructions
so I decided to kind of logically group them into this and going from the simplest to the most
complicated. So the first one that you're going to look at is a masking structure to the LSSV not much to say
you got this eight register and then you're just going to see the various operations so you know you
save it from the register you get a bunch of podcasts from that and then you can pop the count
the one count to zero I end up not so. So, this is our essential you can actually start masking
operation because you will find mask operation like you actually read some about mask
much easier like this. Load and store again think to take in mind that this is an architectural
where caches are non-credent so all of this load is still go to LL1 so to the L1 decash and so
keep in mind this because it will be important for the future. This is essentially a mask load store
of the full floating point register, very useful for stealing. These are actually just this that
feel like very simple to me because it feels very classical you can see there's the load store that is
mask the broadcast that some weird things going on for example you can see there are two different
prefixes one is PS your API one's time to pack packet signal which means floating point
your days packet integral which means we're into 32 or into 32. For example, the things that you
will find when you look at these are the isolated are some very strange surprises like the immediate
of the broadcast is actually when it's a floating point it's the top 20 bit and the last four
bit repeat the three times so because we do that has zero. As you can see the scatter gather
everything that you expect condition move permutation and all this thing so this feels kind of normal
this feels almost normal at the beginning until you go to the last page and this is the
converter to convert the elements so you know they supported p16 to fp32, into 32 to fp32 and
the vice versa but this is when you start seeing that the mean was meant for graphics or
each or even he was designed they actually want to get graphics GPU not an axlator so
we support and I don't really want to see the format of this for these things but we support fp
and fp like when I know the values integrate normalization numbers including you know too
it's part of the legacy of the of the same thing and then yes this is the values
the actual integral for the incorporation that they're actually things that we actually don't
use imagine these are all master of course interestingly the the floating point has the
mole at and not the integral one things and so this is essentially where most of the
specification is I mean I could list all the instructions but I think it's very useful because
there's an interaction to it right so this is where you know all the values things start happening
because for example not all the instructions are under finding that you'll find the manual
are actually implemented in the hardware and if you look at the very log which soon will be able
to because we're open so soon that you will see that after if they're out so they were implemented
and then they moved for saving space and so for example the touch and then actually instruction
like sign cosine I think even exponential wasn't implemented and there are things that
actually create problems with after we've programmed these things with a compiler which is not
that way we don't have an integral divide and we're actually breaking the actual C programming
because you cannot translate the normal register size which is you know in the 64
to have to teach you because there's no converter in hardware so you need to be very careful
if you're going to see these things to actually use float long. The architectural actually
generated a new exception which is M code emulation instead of relying on the illegal
instructions because this way the exception can actually pass you some later that helps you
understand which instructions that to it is how I think you have a more complicated the code
but it's not used by the firmware so that's something we need to add. This is something
that I wanted to see this is how the actual CPU looks like when you program and I don't know
to you but this feels very inspired to me the same time essentially what we did is essentially just
added PS as a as a prefix a suffix and this essentially is part of something I take when
a code I was apparently convolution using the same name section so as you can see like
actually the from a point of view of the user the idea of actually using floating point
like so makes a feel much more natural and that's something I like despite all the very
details of that project. This is where cash becomes important so this is why I introduce it first
and now we're going to see the consequence of it. As I can see the way I like to think about the
system despite what soon we realize this is a lie is that you essentially have a
meaner level and a meaner and each meaner CLL1 and an L1's cash fund. There's an L2 or an L2
's cash fund there is shared with the Shire and then there's an L3 global man so the plan that we have
in a system that is actually there is actually non-career and is that it gets very hard to flash
everything not the L1 and not controlling and then since all the cash not communicate with each other
you can actually easily create a lot of problems on the cash line. So the way this is solved essentially
and the atomic is stuck to you in a way you actually create essentially create implement so
in actual order this is actually implementing an L2 and L3 in the actual cash module so essentially
the operation actually don't have the cash level module and so all these operation that you do
to find as a topic you specify whether you're going to execute an L2 or an L3 and this is usually
specified with global and local and when you use an atomic operation you actually completely bypass
the one so essentially the way you program this machine is by decided on some data of Shire
local some data global you're just going to access the atomic instruction to exercise this
it requires a lot of how can I say discipline but I had to fix out of the maximum format for that
so it can be done note is not a real there does no real LSE of course you can actually have the
usual reason we we're not GC we don't have the A extensions because the atomic's needs to do this
and so you can have like the various amoxyl global and local for wood so you actually when you see
that you essentially realize that it's quite the same of the atomic instruction extension with
the global and local so everything is stopped there are any questions about this stats are
I'm going to continue right so what else is there um to something that I am personally
I don't know I was shocked because I remember working on circus back in my previous life
and so for example they actually have a comparison what we sell the amocas and of course it's local
in global so that's nice although not exactly the architecture would like to have a spin look on
yes and what is actually becomes interesting is since essentially
the what you're going to see is since atomic is actually used to really not work
so you can have atomic comparison because it's the cache layer that actually prevents them that
those are the all you part but is actually used to control a rich level of the cache in which cache
you actually write in the data and so actually you're going to find scatter and data, scatter and
local and global we're going to find all the various operations that usually are for
for floating point in the version of farmer up global and local so actually all the
operational usually you have for floating point they actually become atomic because of to this
in this architecture atomic are not meant to prevent someone to do something because everything is
an uncreated it's actually meant for you to be able to control at which level of the cache
of item data and that's very important but now we actually I hope I was running out of time before
starting this right so this is if you think that this cache was complicated you should say this one
um right now we definitely have 15 minutes I'm definitely going to finish this to uh okay so
that's our processor the view you had at the minions so far was very simple and nice because you
are simply atomic and things the tensile operations actually work on matrices and
this side actually looks nice because essentially you have a matrix which is 64 byte by 16 rows
and of course since it's the funnest bite you can actually have different size of matrix as
so you can actually do that um I'm I mean that's about one line here but it's found by essentially
like the fact that you actually can have in date 15 minutes to 2 you can actually have this kind of
m and mkn and so this is the kind of operation that you have and you can actually define
because you have various FMA operations for matrices so the problem is where do you put all this
all the status because if you make some calculation it is 64 by 16 that's 500 12 bits I believe
I had it at home and so this this becomes kind of important for us where we
where we do this and this is where it gets complicated because essentially
when you have this kind of copper this kind of operation of matrices in this kind of machine where
the cashes of you know you have a distributed chip in a knock the way you handle data becomes very,
very important and this is where we actually start saying that what I told you before about
the cashes was an actual lie because the way cashes work in it's ok one is that actually
they're all marked in a global space not the L2 but essentially like I cannot access
from another minion the L1's cashes but I can access someone else's L2's cashes but so this is
all this around get marked in different ritual at the space and not so I can access them.
So the two things that the way I should see when I'm talking about my minion how the memory
how the memory work is I'm going to see that as I'll if I enable the sketch but which
you need for transformation I'm going to see that is a L1's cashes but I'm going to see that
there's an L2's cashes but and to me I'm not going to only buy the hard idea I can know which one is
my L2's cashes but others what I'm going to notice is that the one that is closer to me because
it's a much higher is going to be faster to access and that's what I'm going to use. So essentially
you need to understand that even though the model to SAP can be can be passed. So how does the
transformation work in this case? Well it's a game between two matrices I B and C and what is
interesting about this is that the FMA which is the operation there are many of them by the way
there's a bunch of simplification it takes the first matrix from the L1 SAP so I need to be there
there is something called the tensor B tan B register you structure a sleeping register
these are not the simplification but essentially you issue a load B and then immediately you issue
a tensor operation or whatever uses you up to this and so this is not an actual this you don't
really load ever the full value of the matrix into this tensor just something as streams and then
the tensor operation actually saves the result matrix in the full floating point in the full
floating point registers of the machine and then you need to issue a tensor store. So why did I
put this because usually and it's going to be I guess talk that it's going to be much
because size I'm pretty sure it's going to be like a tensor load to a Cp that I can actually do
parallelly and then actually I can do a tensor load that can load from everywhere but usually the
weight works is that I have something to load from an external memory of from wherever it wants to
to Cp I do a Cp and I load from from here and also stick the tensor load A if that is the case
and then I do the tensor store back so essentially you should see this two parallel just two things
are parallel but the system doesn't force you to do that and usually what happens is that you actually
load A and then you change B because you stream that so this is the system
it's kind of complicated as I said at the beginning in a very first slide this is not going to be
a solution to avoid reading the manual I simplified massively because the two days have a
tensor FMA can answer read from L1 SP the for the B matrix so completely avoiding that
and so as you can see we started from this very beautiful idea what this doesn't cause
views and then now we are here and it's extremely silent zoom right there's going to be a
tall country now found the later but actually I believe it's going to create I don't
have a much better explanation of this it's going to actually show how to program these things
it's in the air I found the and I planned as the room is this afternoon so I'm actually past the
tens of things I'm quite happy so what to make of this personally I felt very stupid when I
program at VV in my past life I felt that I don't know like it really hit me in a wrong way
mostly with rage looking at this in the code that I program from time to time and you know despite
all the cashy issues and all the stuff I felt it extended refreshing there's a lot more
extension it is a one for example I took a lot about the cashes and now to handle the
non-Korean stuff but of course there's a lot of talk about this I have the things that I ignore
very much is that actually the ET allow you to have to send tens of magic to another there's actually
there's a lot of way too there's a lot of way to actually control that there's a lot of things for
synchronization of that from data passing so I end up cooperating low to cooperating so it's quite
complicated for that I didn't go into that I just wanted to see how this cash system and of this
very interesting scratch parts like configurable cashes at L10Q can become very interesting
is it ready to be like the future of the next thing now
this case is tough if you see like you for example you're going to find
pack eight pack for example you're going to find pack eight and various various instructions
that were necessary for them but clearly like this file I've defined it on so you know they're
not exactly very well defined this is the thing that for example is 20 meters and the
push to be neutral some to ashamed of doing something like this because as we know we all have
in this file some issues with the instruction length instruction and the way the space is actually
allocated and the designer of these is a quantity that I didn't care so they just said ah I'm
going to use the 48 and 64 bit space which in LVM is actually fine because you just defined
the table generator done in binodils you actually need to have a niff death when they calculate
the instruction length and I think that makes the batch completely unabsimable and these are good news
we actually have a lot of support I did you have steam you can actually go now to compare
explorer and play with it and I guess that's the end of my talk I beat this is I found it so
it's not a company technically behind this like you can just join discord I don't like this
god but we have discord so we I love discord now but we definitely go there and it will be quite interesting
to do what you do soon we'll we'll also have the TL for this so yes please let us know and
does everything here in case you want to see manual implementation the actual details such
scheme of the board and everything else and how the film will handle everything
any question I thought you were excited that the beginning says that
if we should you mentioned that some instructions are implemented as a mobile
driver can correct it was a rational behind this decision low overhead or specific
reports and that's that's a very good point and that's exactly what I asked because I didn't
decide it I wasn't there we are quite happy but I could talk to the very smart people that
ah the question is sorry question is what was the reason for adding another exception which is
type 30 by the way to have these these instructions not implemented please I'm called emulate
what was the reason behind it because the two this like I remember the realization was done
with top and emulate when I was young and illegal instructions or you need
ah the reason I've been I asked this question first thing and this one I've been told it's like
well we did the code in the chip why should we waste that information which is possibly
fine but I don't know how much the TL was actually complicated for that so essentially in the
in the arrow in the T1 or whatever that is you get information about this structure you should emulate
that's why
you're not able to buy you can be ah can can he buy the it is ok one we do have the chips
we do have the boards we do have actual open access to developers if you want to play with it
usually we do have some scheme not my decision is not I'm not in the loop of deciding who
guess the board it would doesn't but essentially yes we definitely have boards we definitely have
chips we definitely going to do something with them and even today if you join this code
you say can I have a SSH access to the machine you're going to have a machine with these things
we can do whatever you want you're going to film the the compiler everything everything is open
please
and for presentation my question is that since you plan to open source the very low code and also
obstructing the compiler stuff the complexity of this design both at the compiler and when you
can level for this complex architecture it's made in a way and then then having a code
upstream it's just two complex for this design so do you think that having for your company or
the community to maintain that when you try to simplify or will you want to continue with
this kind of project so the question is this is complicated this architecture is complicated
and we can open source of the thing and what happens to the mountain ability of code and it's a
very good question to be honest we notice it's not that complicated like we notice just implemented
the feelings and you're done and the problem is actually gcc engine gcc for the spilling for the
staff and you write it's complicated but I believe that if they manage to support 886 they can
support this architecture right essentially like the the thing is this like yes but this is why you
abstimate right you have I mean I can imagine that there are definitely now some bug in the
motor last 68000 code in gcc because no one uses it but if you abstimate this is where we should go
because if somebody uses it and it's upstream we can fix it so are we going to keep going
towards that architecture well first of all if a chip exists it should be supported we cannot
say ah sorry we have no software right that's not our open source work especially if you have access
to this level of details we're going to change architecture yes it's not that we're going to
take out the same thing all over again and by the way if you join our discord and if I found
we are actually going to a process of tape out right now and define the chip and since we have
really serious about open source you can actually find the spec of the chip we're going to
tape out the moment the other engineer actually changes it so you can actually find a school
album and you will find actually the specs and this is and we're definitely evolving with fixing
your router we are this is like the first test chip that we're going to do but then of course
yeah this you have the people that are at the back on our know how long our next architecture
know passionate to use our new teleworld the discussion about architecture have been so
one is laughing because I didn't as well I promise so yeah times up thank you very much for everything
you