Well, welcome to my talk. I'll go over what happened in the past year on in terms of mainline
kernel and he would support in Raptor. Also, may so. So first of all, who am I? I'm Nicola
of Raptor O'Reilly. I'm from Switzerland, which explains why the first name is
France, the French way. The second name is Italian and the accent is German. Yeah, I got
started working on rock-tip stuff in 2021. First thing I upstream was the year
object by the STDM Audio Driver. And I joined Colabora basically a year ago as a kernel
engineer and I've made several contributions to rock-tip in subsystems in Linux in that
year. That's good luck among this probably, undercounting, but it's the basic of quick
little pup. However, this talk is not just about like what Colabora did, but the community
has a whole. So there were a lot of outside contributions as well. First of all,
I rock-tip. The boards are very affordable and widely available. So before the great
time for the kit, you could get the flagship SLC boards for like 70 to 180 euros depending
on the amount of RAM you wanted. That's 4GB to 32GB. Many of them have their origins in the TV
boxes and stuff, which is why they have very good multimedia capabilities for like their
price range. They actually have a lot of their own hardware decoder encoder designs, which is
how they were like the first company that I'm aware of that had H264 10-bit hardware decoding,
which is a very obscure profile or at least was until HGAR had to see and HVC licensing became a mess.
Also, the TORMs are extremely, easily available. Sometimes, officially from rock-tip
themselves, other times you can find them floating around on the internet. A lot of times,
like the SBC vendors have just a copy of them up there. I don't know how sanctioned
that is, but even if you get them from rock-tip officially, you will like see a big water
mark or rock-tip confidential on the TORMs. So evidently that is not to be taken too seriously.
And yeah, the vendor code is also like, you'll do notice that their engineers are like
frequent contributors to upstream not just fundamentally, but also how the drivers are written.
In that it does look like Linux kernel code. They're still like that, the vendor drivers. They're
relatable. I'm 50 hours into a six-year-old work week and I need to shift this and I don't have
time to refact the entire subsystem kind of bad. It's not the whole let's write tens of thousands
of lines of whatever kind of bad. So it's good to go over the naming of the sheet so quick because
that makes you kind of a rare of where each chip in stack is, but it was here in the name.
This is just one naming scheme to have other chips. We don't have time to go over the other chips.
First, you have the ORK3 prefix and then you have one digits. Did I have labeled X here and then you have two
digits falling that. The X is kind of the generation. You will notice if you pay attention, there is no fourth generation.
Now, I don't get a guess. I went on Wikipedia. I looked up Chinese numerology and saw that
for is an unlucky number. That kind of plays into the next point which is that generation
here is mostly a marketing term. They will happily re-work the entire part of designs in chips
within the same generation. It's not that just because it's in the same generation, you're going
to have this driver work on this other thing in this generation. It's not how it works.
What follows is kind of the performance tasks of the chip in that generation, the higher the
better and you will see a lot of repeats, digits, eights there. It's also because eight is considered a
lucky number, I assume. Sometimes you have like suffixes, that is one letter, but those you know
can vary widely and usually, just assume it's a different chip.
So some examples, the current flag shape is the RK3588. It's a four performance for
efficiency core design. There's a Mali Valholt CSF GPU which is the first kind of
GPU, Mali GPU design that was explicitly designed around kind of the multiple
GPUs of work and stuff like that. They also didn't scheme about on the memory control or
you get quad channels of up to LPDDR5 and it performs the get is about 2022 entry level Intel Chromebook
which is quite good for an SPC. Then one SOC that might be a bit confusing, RK3588S, it's a cut
down version. I don't think it's a bit thing, I think it's legitimately just different silicon,
but stages have removed a whole bunch of IO stuff, however the CPU cores and GPU is still the same.
When we will talk about a lot today, it's the RK3586. That's kind of the cheaper entry points
to the kind of the stuff that can be found on the flagship and then we have the RK3589 which
was the previous generation flagship and has great mainline support because it was in a Chromebook
and at a time Google kind of said if you make a Chromebook with a chip in it, you have to upstream
support for the chip so that's how a lot of the upstream for the chip happened.
Then we don't have time to talk about those. So when you mainline stuff, one of the fun things is
that generic UEFI ARM system already kind of images to standard work like fedor IoT,
but the good benchmark is also, can you just treat like a PC? Can you use a generic R64 UEFI installer
on a USB thumb drive to lock it into a board, install that distribution to local storage and
boot from that local storage. So that's what I've done here. It's a RK3586 board and I've got an
NME drive clocking below and I've got open-source tumbleweeds on the USB thumb drive, HM1 and Torkey board
and so forth. So I'm trying to get open-source tumbleweeds installer and first kind of hurdle.
It initializes a video output clearly but it displays the installer only as text on serial.
Now that could have been my fault because I skipped the grubbed prompt by entering it on a
hitting enter on serial and that's because usually you don't see that because we don't have video
output in UEFI yet. But once you do that then maybe it will work in a way. It stumbles through
the finish line. It shows a fatal error well installing that is not actually fatal. It works as fine.
It boots from NME after installation but into the terrible unaccelerated X11 session which does not
feel good to use. After we switch away Lanto you can see we have fully accelerated OpenGL and
Vulcan so you can even enjoy some VK Quake and not just Synodic. So the conclusion of this experiment.
This reason stars are terrible but like I tried another one I tried Fedora which also didn't go well
but that's something for them to look into. If there's like anything missing from our Uboot
staff configs to make like certain EV wars writable or something, please let us know but I don't think
that is the problem at this point. I'm not super sure. But it's important that like distributions
might want to just test a mainland Uboot setup like that just to make sure they're in
solution making it wrong assumptions because at the end of the day if people go out and try
this and then it doesn't work and are going to be like oh support for this SLC is bad even though
it's actually good and it's just installed tripping over itself and that doesn't feel very nice when
you put a lot of work in. But yeah you can move from it just funny just works you don't need anything
fancy device specific images become redundant. So let's go into what kind of change.
During this year and one big topic was molecule abuse and you say well Molly's last
design why we bring it up will be there is it's no coincidence that when we have the
compliance test suit passing it will pass on our pay 3588 because that is the platform used for
developing the new pan store kernel driver for and that's what pan for us to pan became a user space
driver's target and that whole stack gained Vulcan 1.4 conformance this year it officially passes
the test we officially have permission to use the trademark insert it also passes open feel yes 3.1
conformance that is new this year. Then we start the out the year getting an HDMI output on our
K3588 fairly quickly later during the year we also gained video output for our K3576 and it's up to
HDMI 1.4b right now but HDMI 2.0 2.1 is still a working progress part of it has already been
posted to list part of it is still in development and yeah this includes HDMI audio and CEC.
Display port this cool because some of whom rock ship actually wrote a new main land driver
from for this it's not just the dance stream copy base it's been cleaned up it was actually a new
gear and bridge driver for it. It's a merge track name or arcade with 588 I think just today or yesterday
there's a middle day new region for arcade 3576 support so it's working progress on this but
cusbc alt mode has some complications because cusbc is a whole complicated mess and of course
come forget me pdsi that's usually used for like embedded displays and stuff like that
this also supported for both of those ssc's now we didn't put so the non s or k3588 has actually
hmi inputs and we've upstreamed the video for Linux based driver for that it doesn't do audio yet I don't think
and it doesn't do 10 bit yet I don't think but you can totally just like capture hmi on the board
and you can like set your own custom edits and make it pretend to be some monitor or whatever
then we have if you attended the talk I think yesterday we have new me pcs ad drivers for various
structured ports that work was done by Michael Rich it's partially merged work is still ongoing
it's a really complex area of work and if you like heard the thoughts before this also involves
hardware ISPs and we don't have a driver for those yet things are being worked on in that direction
but until then you can use the software ISP to avoid having a green picture
and you was another big edition this year
to me we chose a 10 bit as on his own in his own time he wrote rocket which is a new mainline driver
for the npu funnel in the or k3588 this is an inference and few several generations removed from the open source
and media laid design the user space is implemented as like a teflon delegate in maza so you just
pick up three maza it has that software in it the kernel side is using the new DRM Axel subsystem
the other stuff SLC is released around that time I have very similar and few designs
but they're not precisely the same and I did have some difficulty trying to get it make it work
for our k3586 but at the time I didn't have a register map for the npu there
I assume it won't take as much work as the initial driver to support those but it does still need
work so it's not like a just problem play kind of thing
multimedia is another great area that's a lot of stuff during this year
the hardware we did the coding drivers on this in mainline use the beautiful links to request
stateless decoding interface this is a cross vendor u api this is nothing to do with the
rock chip mpp library so the ffnp licensing drama is not relevant
we started a year out of already having a via one decoding on our k3588 that's a gave you one decoder on
there is a license design so other licensees of that hardware design will also profit from that
if they decide to move up stream then that left us an over route a decoder driver for the
h264 and h265 horror decoders which are in-house rock chip designs this was merged just recently
I think I wrote your first AK decoder driver in mainline I think Qualcomm might have beaten us by one release
but we can still claim we're the first stateless 8k decoder in mainline
and also I saw some people were like playing with using those drivers on our k3588 and 66
so I shouldn't be too much more work to get those working on those two chips
and yeah the ffnpx side of things is handled by Jonas Pibu Kormann who has been
as I mean that for review that has been quite a long story getting the support for that API there
and upstream this stream of food already is supported
then a lot of stuff happened with our k3588 support we basically started the year out not being able to
really boot that and now a lot of stuff works but first like why even bother
yeah it is all the license design but if you look more closely they actually operated a lot of
stuff on the like SSE peripherals side so they have a new audio controller driver they have
UFS now new PWM hardware etc etc they also have a new license interconnect which has DPCA coherence
extensions so you can just slam an AMD GPU in there no kernel patches needed it just works
you only get PCA to one lane but that's actually how I've been doing some AMD GPU work at work
because I already had my ok service and 6 ports set up for net booting a kernel on each boot so
why not use that and yeah it's a good mix of like being kind of closed or k3588 where we can
pick a pack of some work but it has enough new stuff to where we can get prepared for new
or chips and kind of smoothly transition into supporting those as well so basically as I
see support lenses this year which is like clock 3 so it's all that good stuff and a lot of
peripheral drivers needed basically no adjustment because they burn change at all and it's a lot of
license stuff then the UFS driver was actually upstream by rocket themselves which was cool
to see for those model where that's a new flash source standard it's fairly common in phones now
I think it's kind of replaces EMMC and it's basically you get NVME speeds but they very
look kind of small footprint then we have a new fire driver for USB display port because of course
display port now often goes over USB connectors audio driver has been upstream by me I did that
and yeah I needed to blockchain the clock controller a bit for that arm trusted firmware that's good
because with upstreaming of drone trusted firmware code it's the only blob that remains is
the DDRM training blob that runs once a boot and the rest of the stack is fully open
Sebastian Raquel who is also here in the audience today has been going around
squashing all of suspend resume issues I think okay 3586 should now suspend resume fine
okay 3588 I think not yet but I'm also way on reselling him here because whenever there is a
complex issue he's basically the person handles it again you see rock chip actually contributes
they are upstreaming can support which is interesting for cars industrial automation
3D printing now I think and yeah also roads two hardware RNG drivers because they decided to
change up the hardware the newer driver the RK 3576 one doesn't yet make use of all the features
of the hardware but neither does the downstream driver so I don't feel too bad about it it works
and yeah one of the big things still missing is Peter William output and capture I'm still working
this time allows it's posted on the list I need to iterate it on it's some more but yeah it's
a lot of fun and the Peter William work also made me re-factor a lot of rock chip drivers
in the like they had a hybrid update macro that was copy-pasted all across like the Linux
kernel and the unified all of those and accidentally north-sniped Linux Torvals into coming up with
cursed replacements for the Gen Mask macro for like the next two weeks which was extremely good to see
going to give you a few flat out again you're not a squibble carman just doing amazing work
getting your boots to boot on these ports mainline you boot that is even on like this smaller ones
well like the okay revivo 6 which is an interesting as a seat to look out for if you are into
very cheap stuff I got entrepreneur who is the mainline maintainer for rock chips off systems
is just doing amazing work it does a great job at just always being on time I guess
we rarely have problems with having a missed full request and stuff like that then we have a lot of
rock chip employees who contribute now you can see all those names I want to try to pronounce them
all because I will horribly boot for them but I want to give a special shout out to dumb
who actually during like my PWM upstream he replied to me and explained some part of the hardware
to me because he was literally the guy who wrote the downstream driver and I think probably
wasn't both in the hardware design and yeah I also want to give you a shout out to thanks
12 for working on VR putting you boot for okay 3668 because as I mentioned that will make a lot
of the user experience mover when you use generic images looking at the future
yeah I think okay 3586 is going to land this year I think that's going to become bootable it
depends on where they get the matrix IO upstream which is like the new way of having you will be able
to mobs every pin function to every other pin function every other pin and yeah I think you
boot needs to market itself better because a lot of people don't seem to realize you can just use
main land you boot and use it as a platform firmware and boot QEFI images just the old thing
you need to use like the vendors device specific images the video field for Linux requests
say it is important in UAPI is a big on-resolved question but I think there's a movement on that front
we should move up ship to the generic interconnect framework because right now I'm not sure
how well the QOS is handled I vaguely remember it's not handled very well and on for example
okay 3328 which is don't have video output control or QOS so if the memory bandwidth is under strain
you just get video glitches because it doesn't have to bandwidth and yeah I'm hoping you'll see more
camera things in 2021 and there is a lot to do we have a status matrix for the two SSEs we
actively work on there are so more capabilities faster than six drivers that I assume will work
on these other SSEs most of these aren't very interesting SSEs but at least to me but they use the same
kind of order designs and of course please do write device 3s for your device against main
i'm bannings questions yes
about secure boot yes so some of the reverse engineer has a cool reboot works on RK 3588
officially I think rock chip makes this on an NDA for that kind of OTP fuse info
it's one of a few things today do require an NDA for RK 3566 we don't have anything yet
I assume it's not going to meet too much different so no secure boot yet there is an unofficial way
someone has made it work and that's so far yeah well I can say about it
yes
did you try out the UDK 2 firmware for the RK 3588 the EDK 2 TNO Core firmware for that
um I have seen it I don't like it because it's EDK 2 which means it's not a code
base I like very much and it's a fork of EDK 2 which means like all of this work is going to
kind of shamblon in a device specific for that in 10 years I don't know if it's going to be
still maintained the good part about you would is that since it has the same lessons as Linux
we can borrow a lot of driver codes from Linux itself and because it's all all device support
is in the same repository and everyone licenses the same to you like USB controller hardware and
stuff like that you have a lot of like codes that doesn't need to be double-cated
we're good
is the 3.06 which is the option that said there they have it may not be the same yet so
yeah so 3.506 that hasn't been fully mainland yet there is rock ship is working on it
some other from other people from the community are working on it as well
I think you can already do mainland UBoot with a onlist patch set by Kribu
however kernel will take a bit more there's currently some difficulty figuring out how
we do the deep findings for the new rock ship matrix I.o but definitely expected to happen this year
and I would say they'll look out for that
okay thank you
you