Okay, I guess we're good now. So again, good morning, everyone. And I'm going to be talking
about, obviously I just said, research optimizations in the FMP project. So I'm going to be going
to be a little bit of a story of how we did that stuff. And then talk a little bit about,
this is all done, so you can pass me a guess of this one. This is done. And then also a little
bit of mark, kind of a feedback or like, post-mortem, it's not finished, but like, experience
from this activity. So this is only my personal opinion and all the societies in my employer,
thank you. And if I speak too fast, although that don't articulate enough, do interrupt me.
So always please take the questions to the end of this talk. So who am I? I've been attending
for them for more than 20 years now. And this is my second presentation. I do open source
mostly on my free time, also I do, I am a software engineer, but I am the risk argument in a
force of MPEG, but I mostly don't use the opportunity of, I'm not normally, but I'm the biggest
developer for the view, see me in the employer. So what I do, I'm a little bit of history
because this, as I've been playing a high-class stuff, so until 2022, there wasn't fairly
anything. I think there was one patch on open, open, BSD to just allow it to compile and
on race five, I was just playing C record because it insisted on knowing the architecture,
but that was basically it. And that's when we talked on the 2022, just the end of COVID,
that's when I started, I got a board from a star five tech and I started, well,
I thought things as I said, because automation first, and also writing some simple
spike and, I said, writing some simple optimizations using spike because at the time
key image and rework, and there was the hardware I got, obviously don't support vectors yet.
So I think we're getting full, so please. Anyway, so the point is that the time,
normally, if I think that doesn't get optimizations, this early in the process, we only
target real hardware, and at that time there was, there was a risk of hardware, obviously, but
there was no vector hardware. Anything was quite tedious to use spike and TK.
In the same year, later on, I actually started doing some, as if I'm not
optimizations, not just VFC, because obviously I've been playing this much more relevant
in terms of in our loop, so that you can optimize these vector units, and
you can finally find a good vector support so we could actually use more
proper emulation than spike really. Then we got the first hardware with the
B extension, and I implemented the KKSM tester, but we will see later on what
KKSM actually is. For a year, I started posting patches, but didn't carry on.
Then came the second, I mean, the vector, the drafts, the hardware, I think,
some of you have probably heard about this problem with the incompatible drafts,
the action of the vector specification, that was implemented in some
hardware, but while it was not really compatible, we were able to, back
forth the code, or I was able to back forth the code with some hacks, and
I've just get some very smart, some real hardware, because still it was
similar to what would be the final spike. And then you know to me, finally,
the first hardware that was small K-230 camera board, and we started getting
patches from China. For a few years, when we started getting money,
that's when I started funding, it's a six-for optimization,
and fortunately, that's now more than finished, and there's no money
anymore, anyway. Then we got the X60 board, or the banana pie, which is a
single, and most people would be using nowadays, if they want to do vector
organizations, or I would generally speak, with very popular RF5 board, I guess.
And that's when we started having the problem with the SVMX, which
we get from later also. And then last year, we got the X-280 from
size for which was the first 512 bit vector unit. It's a bit expensive,
obviously I don't have more than, I don't have any benchmark. So all the
benchmark comments I'm going to make in the rest of the talk are
for the C908 and X60, and they're not pretty sizing size 5,
but also when this, that I just don't have experience with.
So the development of, honestly, has an opportunity, and
I must say it has slowed down last year, and it's still slow.
On one computer from China, I was very assigned to it.
There was a project, another contributor, not from China quit.
There's a job, so that's left-retime, actually.
And as you can guess, nobody really likes to do it, because
especially for free, and especially not when it's long as
there are things, and we'll see why it's assembled or not, something
or something else. So how do you do, as you can get, as you can get,
that's an important thing, I recommend see that this point is
cross-compining it, because the boards are improving, but they're still
slower, and still seeing if you can't discover, and they're
looking at it on an X86 desktop. And then, I understand,
you should, instead of QMU, with the B70 support, you can
transfer and be pretty much running the arm, which is a space
program on Linux, so that's very easy. So in
mind, you can just download, configure, and compile, and run the test
to it. You can also do that natively, but it's going to be
going to take maybe one hour. So the workflow, while you're
finding one function doesn't be an optimizer yet, obviously,
then you reflect your refactor, the C code, so that it looks like
a loop, that is easy to optimize in assembler, then you write
the assembler, then you compile, then you test, then it doesn't work.
Then you debug it, and you compile and you restart the
process, and when you're finally done, you can go and
boast about your benchmarks on the mailing list. What you do
not do is you see in 26, and we explain why, using an
assembler, because you can't express a clover,
correctly, in a rest 5. So, absolutely, you can not use
an assembler for vector optimizations. You can use an
assembler for normally, if 5, but not for vectors.
Optimize function, we have test cases, that's a bad idea.
Specialize the vector length, this is controversial, but
get to it. So, do use the other set,
adversaries, making instructions, it helps a lot. Obviously,
because we don't have the clear performance profile for all the CPUs that
might exist in the future and for more different vendors, just try to
avoid the data dependencies between consecutive instructions, so
that the CPU pipeline is installed, especially on
in all the execution CPUs, and do check benchmarks until
hardware is shut up, possible. So, why we use a
assembler, I'm not in 26, partly it's historical,
on X86, when they've actually started, they didn't
have mixed experience with in 26. There's general
problems, defector, all platforms, including this five,
and you can't really prevent the compiler from
spilling vectors, or in general, because
registers at the wrong time and breaking of the
organization, I'm a Hingit slower, rather. So, you don't have
finally enough control, of course, it's easier, because
you don't have to worry about naming your files. You can name your
variables and let the compiler do the registered
allocation, but the result code is usually, is not
stable and sometimes it's just slower. Also, you can
do one-time detection, because you need to enable
the target extension for vector, in this case,
statically and your binary, is this means your binary.
It's a compiler with nus vectors, five racing, and you won't
be able to run your binary on the non-vector capable hardware with
one-time detection. So, we also don't like that, because
this was one-two has the same binary for both vector and
non-vector CPUs. And in the list, five vectors,
in case specifically, you have one-or-is, and which is
the group multiplier, where you can adjust the register pressure,
the size of the register depending on the register pressure of
your function, which means you need to have really
fine-drying control and the standing of the register pressure
function you're currently optimizing. And you cannot do that is
the compiler is the register allocation for you. So, do not
use intrain 6. I mean, you'll be seen as an effect, and we'll say
no, if you try it, but the nus approaches are
recommend against using intrain 6. Check it is same,
it's a bit more complex than that. So, it's specifically designed to
test assemblable optimizations. So, it has, it will
enable it to also feature a CPU has different as defined by the
project. It then gives random inputs, compares the assemblable
against the reference secret to actually that's how it's
testing. It's not just normal, you just check the results. It
also has all sorts of micro benchmark, and maybe I can
test a very conformant, it's not something you will normally do
really test, because you see compiling a text curve, but if you
find it as similar, you have to worry about it yourself.
So, this comes from x6 store, but it has been copied into
weshing a pack, and now there's also separate, you have
available a project that you can take into use into
other projects with similar issues, any computational project
fairly. So, I do recommend looking at it, if you want to
use it in some of the project. Now, getting involved with
the same pack specifically, I must warn you that there is a
high barrier of entry at this point, because when the easy
functions have been optimised by me or by other people. So, what's
left is either stuff that doesn't have test cases, and it's
there if you're going to have to write the test case, and you're going
to know, and I have to understand what you need to test, or
it's going to be a complex function. So, if you're familiar with
these five vectors, you come and help us. If you're not
recommend starting on user projects first, there are
also non-technical aspects, as I mentioned, code reviews are
a bit slow, because you can read all the new and my
free time. And, as a simple business authority is
for being a bit of a difficult community to use certain
reasons that I will not get into, and that this community
can do nothing about. So, there's probably plenty of
your open source projects to help us if you want to play with
them. So, I've of course, you're welcome to join the
fashion package, you know what you're doing, but they're
warning. So, right. And, I mean, never in multimedia, there's
quite a few projects, I think, like, all exist, these projects
set is completely an optimised moment. So, the pain
point. So, I will try to separate between what I think
implementation issues of the two CPU implementation that we have
been working with, and also, then what I see in this kind of
missing in the specifications has compared to my
competition that is neon SVE and also X86AVX. So, first
problem, and that's an implementation problem, is the
what I call the VHS, this is the N-Maxcombo VLC.
It's basically the question is, if the number of input, so
if the size of your useful input is smaller than sits in
the vector, what is the execution time? What is the
execution time for the CPU? So, specifically, if you
need less than half of the vector for the calculation, because
you have a vector that can sit 16 elements, and you only need
to calculate on seven or eight elements, then you are
wasting the second half of the vector, but since you have
that you have, you can't really just change the vector size.
And the spirit of the specification is that you should
more or less scale to the number of the actual useful elements you
have in the vector, so that's the VLS of vector lines.
But, both of the two implementation data have tested,
instead, scale the execution time to the L-Max, which is a
number of elements that your vector can sit. And this is a
problem, because it means that now, if you have an input
that fits in 128 bits, or an input that we have a lot of
6,000 inputs, and you optimize it for a 110-grade bit,
CPU, if you work right, and then you take the same function,
and you're already from a 256-bit vector CPU. And if you
run at the same speed, even so you have the vector size, and the
both typically the vector computation are units. And that's
not really what you want. So, there's two ways to fix this.
One is fix the implementation to actually scale according to
the L, which is the spirit of the specification. The
other approach, which I really hate, is to just say that
just half the vector size using the property player, which
is then this, well, the complexity, but if you're not
familiar with a vector spec, it's going to be a bit obscure,
but essentially, you can choose. So, you normally have
32 vectors in the specification, and they have a size equal to
the declared state bit width width width of the vector unit
you have. So, on 128 to 56, 5, 12, or on one,
12, 24 at this point. But you can also use a
multi-player tool, and in that case you will get double
size vectors, but of course, you will only get 16 vectors in
the 32, because that you're not inventing bits out of
senior, or you can use multi-player by four, or you can
use multi-player by eight. And this is useful when you want to
enroll, because for instance, if you do the same set or
the same simple function, you only need four or even less
and four vectors in your loop, you only need four variables
basically, then you can use larger vectors in your CPU with
run faster, because you'll have a bit bigger bandwidth.
Or you can also use fractional, a multi-player is
a vector is too big, and you want a small vector. And normally,
you should only do this. If you are mixing with, like you
have 32 bit values, and at some point you can
have them down to 16 bit, and then you do something,
and you can have them back to 32 bit, and realize that.
Then you can use fractional sizes. In this case, you
will shrink the vector size, but you will not shrink the
number of vectors, because the instruction set has no space to
describe more than 32 vectors. But so, you can scale,
you have this seven different multi-player sets, you can use
to scale the vector size. And so, if you use yellow
max, then you need to change, depending on the CPU
bit, which is, you need to select a different multi-player
for your optimization, and draw on that optimal speed.
And this is no good, because now, it means your optimization
that you wrote for 100 or 28 bit CPU, you have to
rewrite it for 5, 2 or less, the 6 bit CPU, the X.
60, and then you would have to rewrite it for the 5, 1 or
1 or 1 third bit CPU, which is already one model, and now
you also have, as a yesterday, or the day before, you
have to rewrite it for 1,024 bit CPU, because space
meter is announced for 1,024 bit CPU. Now, that doesn't
really work, especially for trying the process project.
And this is not the idea of having a scalable vector
extension. So, I'm told that 5,5, this is correctly
and they're clearly scale according to the year. I haven't
been able to consume this, because I don't have access
to the hardware, but if you don't want to take my
work for it, the specification not as I've said,
that's the same thing on the official
respective vector, meaning this. So, the
spirit of respect is to use the L, and also what we need as
open source developers is that you use the L, but
unfortunately so far, are these two implementations
are using the L-Maxine, and they're going to have
problems, and it's not just multimedia, because
think about, again, some simple example, like
let's say you have a même set or a même
copy in your own ellipse, an optimizer in your ellipse,
and that could be any.
Sorry, yes, it's so mic.
So, and now let's say, so a typical même
copy on the same set function doesn't need many variables,
right? You know, you basically only need one variable.
This is the vector of data you're copying, or
initializing. So, you can use multiple
vectors, you need less than four vectors, and this
means now on the 120-forbit CPU, you're going to
get your going to process one kilobite of data per
loop iteration. Your même set is probably going to
be copying initializing a lot less than one kilobite of
data in most coal sites, and now if you wait for the
roll one kilobite of data to be processed, every
time you want to just do that function, you're going
to have your performance, and you're going to end up in
a situation where increasing the bit, the capacity of
your CPU is going to make your code slower.
Or at least not faster, and so you're just wasting
gates on your CPU. So, point being, it's not just a
problem with multi-media code like FFM, which has
a lot of fixed size function, a lot of function working
on fixed size inputs. It also affects generally
code, which has variable but typically small inputs, which
will also be affected by this. So, in my opinion,
I don't think it's a windows 3 in two fixed. Also, as I
said, this kind of in fact, I would also know the
process project to have specialization for what I
want to expect. There were only three commissioning
available links, and I would seem that the
source of the available links, so it's even worse.
So, anyway, so that's the biggest problem in my opinion.
The second problem is poor performance of what's
known in the speakers segmented loads and store. So, in
the rest five, you can do, you have two aspects when you
load or store vectors to it. For memory, you have the
stride, which is the distance between the elements, normally
it's because you need stride and all the elements are
together. In memory, you can also have a variable stride,
when you want to load, for instance, the column of the
metric of 2D matrices. Of course, this is kind of
slow because you have to send the CPU has to eat
many, many different frequencies on the memory
bus. But there's also segments, which is almost
interliving. There is, so, for instance, if you have a
RGB pick up, you have the component like RGB for
it speaks together and you want to load. You probably
want to load each value. So, each R value in
long vector and it's green value in long, another
vector and blue value in the server vector. And for that
we use segmented load, which is automatically
displays the data into a continuous
elements, the interliving to multiple vectors. And on
arm, for instance, this is as fast as segmented loads and so
are as fast as non-signamented loads and so on. And so
but unfortunately, on both implementations that seem so far,
the signature and performance overhead. And that's, again,
an issue in terms of why it's an implementation issue,
obviously, but it's an issue in terms of competitive
activity with at least arm. And it affects us in
much amount of clutter, because we need this kind of
instructions a lot. Mostly for power of two segments.
I mean, three, five non-traditional seven. I absolutely
won't care. Six really use really. We want two, four,
eight, please those needs to be fast. So, yeah, on arm,
it's, it's, it's fast, it's a basic as fast as
if there's no segmentation. Yeah, so that was
for the implementation issues and the spec issues. So,
stuff that we really need to actually write and we can't
transposition. So, in, you know, with the segmentation
and so you can do memory to register transposition,
vector to register to memory transposition, but you
can do, register to register and, and in, in
much easier, we'll find out, especially to the
mentioned transforms, which, I mean, I go through the
mass, but, as you can do some calculation, like,
horizontal and then do it vertically on a matrix,
matrix and so forth, you need to transpose the data
with two steps. And currently, we have to
speed on the stack, because there's no way to
transpose the data inside the CPU registers. And that's
kind of slow. There is a ZVP, a ZVP extension being
developed. So, the first is, let's see,
how long it takes for it to be actually implementing
real hardware. Big sign now in clips. So, there's sign
now in clips and then sign now in clips, which is,
so a clip is when you take a integral and you
shrink it to a smaller bit with, and instead of
dropping the high bits, you clip to the machine
momentum minimum values. In multimedia, we
for some reason, because of our video code,
you always need to take sign 16 bit values and
clips and to un-sign 8 bit values, and that's not
possible in the spec, because it doesn't have
this mixed sign that has a spec. And because of
this, we need three, two, six instructions,
combination for back again. This is something that
interl and arm has, as a single instructions, so
it's a bit. It's not fatal, because we can
still do it in a reasonable way, but it is a
disadvantage. And then there's integral
distance of absolute difference.
What is that one is 25,000? So, in inverted
CG, you can do it in two instructions. You
take the difference between a and b, and then you
take the absolute value, with a signed
instructions. But in integral, it doesn't work
exactly, you have to calculate a minus b and a d minus a,
and then you have to take the max of both, and
you have to hope that there's no overflow in the
terms, otherwise you actually, your
calculation doesn't work, and you have to now
do a widening instruction, and you can
take up to six instructions, where you only
will need it to inflot. And again, there is an
extension being worked on a CG, it's now
in stabilization state, so it's looking quite
good, CV, ABD, but obviously it's still probably
a few years before we see that in real hardware.
There would have been nice to have a way to
change your memory without changing the
memory, or just interesting to play.
That goes back to the VL Max argument, and
I'm about to run out of time, so that's less
important anyway. So, yes.
These are references, obviously, the
restrived vector extension on the zero, if you
need to learn the vector specs, that's, you're
going to have to read it. It's obviously not
meant as a, you know, one-one, but I mean, I
learned from that one, so it's, if you know
at this, the basic of assembler, you can learn
the vector programming from the spec, a
same thing, it opens automatically, there's also
the link to the nightly, okay, nightly test suit,
and then you also have to check the same link.
Yeah, I think I spent 25 minutes, which is what I
said I would do, but, all right, I realized
we started early, so, so we all have one mic, so
there's an investment you have to with you.
Okay, sure. Sorry, man, first here.
There's a link of instructions for specific
instructions, sequences you have to run instead.
What is instruction, fusion, help, and
our vendor's intro.
So the question is, when we have, um,
combination of instructions for
instruction that doesn't exist, could,
instruction version, I guess, in the CPU pipeline,
help, well, obviously in theory, yes,
the problem is more like, from our standpoint, we have to
support kind of all vendors, and it's hard to rely
on, on one, on the fact that there will be
fusion anywhere, because unless, unless there's
a new version of the spec that, like, says,
you really should, uh, because it can't, the
spec is a specific performance, but it kind of
gives in, right? So unless it really says, like,
you're really, really should use this combination,
with something that we're going to be able to rely on.
But if, if, if that would happen, then maybe yes.
So what's the minimum version of the risk pipeline,
I say, this get run on?
So, uh, FSMPega, at least, as, yes, sorry,
so the question is, um, what, um, what's, um,
minimum, um, a spec, uh, I guess a version that,
that, that, that, that to, that's required.
So FSMPega has, uh, extensive, uh, one-time
CPU feature detection, at least on
using hardware probe, um, so, uh,
we only need, technically, you can run FSMPega
sync with only IMA, uh, of course you will definitely
want to have F&D because it's always performance
for floating point benchmark is going to be ready
bad. Uh, technically, you can run FSMPega,
which is just RV820, and then if we run
time detect, if your CPU has anything, uh, on top of
this, that it can use, especially the B,
extension and the V extension. So in this case,
V, versus version 100. Of course, if you want all of
these stuff, um, if you want the CPU that has all
of the stuff that we have in FSMPega at the moment,
I think you would not only need RV820, but you probably
also would need ZVBB, which I'm not sure if it's
included in RV820, so I think somebody
was more familiar with the profile definition.
But we already have a few optimizations. Okay, so
basically, we need RV820, sorry, but then if you
would get the new extension that I mentioned earlier,
then you would need some even future stuff.
Yes.
If there is a chip with RV820,
0.7, it is at least part of it used already.
So the question is, do we have any kind of
support for RV82071, version of the spec?
Okay, and now we're there's called XT-ed vector extension.
No.
There's, to my knowledge, there's only two major
major designs that have been released with this.
One is really, really small C906, which is not interesting
for FSMPega, really, because it's really a macro controller,
mostly kind of thing.
And the other is the C910, and also, so
there's no new hardware coming with this,
and we have to mind that I think both of these hardware
are affected by alt and cache fire instruction,
with a regular ability, which requires you to disable
the vector unit to not trigger.
So effectively, if you want to be secure, you can't
have the vector unit, so it's kind of pointless.
I did write some macros at partial,
the automated process, but it's compilation time.
And we don't want to drop.
So there's no, in my opinion, there's really no
same way to have a runtime support for both at the same.
So it is possible to, with some, not so big changes,
but some changes to take the existing code,
and make it work on 071, but you have to do it manually,
and you have to recompile, which is not very convenient.
I mean, it won't work for this course, basically.
Yes.
So the point is that there's already ways to do transfer,
yes, there's plenty of ways to do a transfer position.
I mean, you can do a bunch of slides and moves.
I'm not sure I heard everything, you were saying,
but so the point was, yes, that you can't already transfer,
so yes, you can do it with the slides,
move, you can probably do it with this compression instruction,
that you cannot, so we use speed and then read back,
because that's the simplest, and it's slower.
It's not that slow.
So I think, I think, night and egg,
it was supposed to work here two years ago,
it was sick, three bunch of different things.
We saw five guys and nothing really planned out,
but I wasn't in those discussions for a contrary say.
I think the point is more like a performance.
Yes, I mean, because obviously there are plenty of ways you can do it,
in the worst case you can do a gas arise.
It's actually inefficient, but you can do it,
but it's only taken more time than going for the memory.
So I think it also depends on the device,
because it's 60 finances as a rather slow memory, but
yeah, there might be slightly better ways,
but anyway, there is an extension being worked on
that's likely to do this properly, and in a way that would be.
So yes, maybe, I mean, by all means,
if there's way to improve, but we currently have
without requiring an extension, and we're interesting to hear about it,
but I expect that it's going to be,
maybe, marginally, faster.
The question is, mainly,
because obviously, you've worked on the X86 side of it,
I think, as well, we're going to specify it now.
Are there any major goals that you've seen,
miss a declaring ambitions for the title of the title of the title?
What is the date like fours on the weekend and the title of the title?
So the question is, having worked on X86,
and then there's five, what would be the,
Mr. Shelley's a gaps in, in, in, in, in, in,
five, four, forgetting similar performance on,
on similar, always on title workloads.
So, to be clear, I have an introduction, X86.
I don't know, X86 has some reason.
I'm starting to beat, I'm sorry, I know,
I've just, job, and a bit of,
it's a 64 bit, but.
So, I mean, the big, the big gaps are,
as I mentioned, there's also,
another list, more completely,
so I think, again, from the nation,
A.G. from David, and also,
it's, I think, is on the rise,
system library, or something like that,
a committee.
So, yes, there's a few of those,
there's also, I mentioned,
now, in terms of implementation,
we are missing a lot, like,
we have nothing for VVC,
which, because nobody cares about VVC,
I guess, but,
X86 is not finished,
XVC is, there's some,
initial implementation coming from,
from the,
the Chinese Academy of Science,
and from Alibaba,
but, I don't think,
you have time to review them.
Yeah,
things you only want,
that's doing kind of well,
at the moment, is a view on,
and that's because it's done by David,
and on by HMPEG.
MPEG, too,
is not implemented,
of course.
Now, as you can,
pretty much, with MPEG, too,
on scale out,
with that,
that's perfectly,
but performance of,
it won't be as fast as,
on the, on the,
on the,
on the, the coverage,
the code coverage is,
is way, way worse,
and X86.
We do have a few things that
aren't,
because I implemented them,
and nobody cared to implement them,
usually,
especially on the audio side,
all, like, things like,
slicing,
so, like, more like on the,
the,
the, the, the, the, the, the,
but there's a lot of work that's not been done,
including on some major codex,
and also major,
major video processing filters,
because the,
because it has,
a filter,
and so,
right.
well,
this is a somewhat special problem,
in case it doesn't anyhow work,
but it contains,
it contains,