1, hello and welcome to this session on keeping up with the SP. You already know Ahmed, he presented
the writing talk earlier. My name is Sumit Simbar and I am the senior technical lead for Android
at Linnaro. RTM at Linnaro works on all things Android. So, before we begin just a trademarks
disclaimer and so, what's the agenda? Well, it's simple. I will talk about some of the
many new features in ASP in the recent past and then Ahmed will talk about the enablement
of these along with some gotchas and pain points that were there. A quick recap on Linnaro
and development boards and ASP. So, I hope some of you may know about Linnaro and the work
that we do in the open source ecosystem. We have been a very long time advocate of ASP and
developing boards in it. We support ASP on a variety of member debboards. We do extensive testing
including CTS, BTS, BTS combinations and these we report and fix ASP and upstream
regulations. So, that's why these debboards are relevant and important for most of us and
the work that we do. For years we have been talking about the importance of debboards in ASP at
different forums like plumbers, ELC, Linnaro, Connectet, etc. And at the ESS in mid-23 I
may talk about the pain points and rewards of maintaining debboards in ASP. So, we will look
at some of the new features that have landed in ASP since then. Let's start with a boot
process. So, Android boot process like many other things is complicated and requires vendors
to implement almost annually changing boot requirements. The developers recognize that and as
a result, they have obstructed the Android systems of the boot process in an Android boot for
UFI application which they call generic bootloader or GBL. More details on the link there.
But the idea is that any bootloader that supports running an EFI app and provides some
basic requirements should be able to have an Android enabled bootloader part. There are various EFI
protocols that have been implemented and proposed. So, if you are interested, go and have a
look at those links there. Also related to the boot process previously on Android systems,
the specification of boot devices was via a platform specific string. This proved to be very
non-stable because things like success paths can change and suddenly your platform specific
string doesn't work. So, across kind of upgrades. To help with that, support was added recently
to system code to allow searching for boot devices via UIDs and this allows making booting
from external device allows much easier. For me, this has been an exciting change in ASP
recent times. The support of 16k page sizes in Android starting from Android 15.
This results in a five to 10% performance boost and about 9% cost on the space site.
How it is achieved is with a page-agnostic OS view, which includes doing runtime
page size calculation, OS minorities are aligned to 16k page sizes etc. And then with that,
once an application is updated to be page-agnostic, then it can run without change on a 4k or a 16k
page size supporting device. Now, the most optimal way to work with file systems is if the
file system block size was also the same as page size. So, file systems like Erofers and FTFers
have been made 16k block size compatible. The next relevant feature is something that is more
relevant for folks that keep their device drivers while they are working on upstreaming them
out of tree. So, this is called the driver development kit and it helps vendors to develop
GK modules against the GK builds using the right tools like hermetic beetles and
correct resources like headers and make files. This has them keeping the vendor modules
easy to manage with GKI while they are working on upstreaming them. When we work with GKI,
there are upstream kernel modules that are already upstream and but need to be built as modules.
So, those are signed and made immutable during fees on the KMI and then they can't be over it
in by vendors. So, basically if you say take Android 6.12 kernels, then the upstream kernel
modules are not mutable by the vendors after they have been upstream. These are called protected
modules and sometimes what might happen is vendors may have pushed out a basic support for a
kernel module and later on added new features. But if they are within the KMI free time,
then as protected modules, they cannot be over it. To allow these new features to be available
in existing products, for example, during an update, GKI provides the option of unprotected GKI
modules. So, they can backport these newer features that are upstream and still use in an updated
in an update for a device that comes later on. These are called unprotected GKI modules,
but the catch is that treated as vendor or out of tree modules and have the same restrictions as
KMI symbols for KMI symbols. Talking about these, how do we make sure that this atomic update
happens? So, there is protected, there is a thing called DLKM or dynamically loadable kernel
module partitions in the Android system, which allows updating GKI and GKI modules independently
of the rest of the partitions. And the protected modules then go in something called system DLKM
while the vendor and unprotected modules are put in vendor DLKM partitions. A related thing
around modules in module loading is parallel code in module loading, which has the name suggests
allows multiple independent modules kernel modules to be loaded in parallel. And a driver can
advertise its participation availability for parallel loading by setting a probe type of its
platform driver to prefer asynchronous mod details on the link here.
You may be aware of the Android shared memory or Ashman, which was an Android specific
file descriptor based shared memory. Now, upstream has something called MFT, which provides pretty much
the same functionality and to allow to use that with Android with ASP and Ashman compatibility
layer for MFT was added, which allows your older apps to still be able to use an upstream
MFT FD. As new GKI kernel branches are tagged, there may be some issues that are found
during these merges to Android common kernel branches. And so, they have started the developers
at Google have started keeping these eratos public. And this is an example of the latest
under 16, 16, 16, 12 kernel code based and issues they found while merging.
This is also something new and it's been talked about at various conferences. So, virtualization
and Android. So, AVF or PKVM, there is an overview link there. I can just talk briefly about it.
So, Android virtualization framework, it provides a secure and private execution environment for
executing code in Android. PKVM is the KVM hypervisor, but further hardened to disallow host
from accessing guest memory by default. And Google's implementation of AVF uses PKVM as the hypervisor
cross VM as the VMM building block. And then, various services are implemented on top of this.
So, this allows you to run, for example, you can run Debian in a VM on an Android device today.
Recently, a new hypervisor driver is being upstream to support SMMV3 in PKVM. And there are also
various how do's in case you are interested in experimenting to how to use custom virtual machines,
updateable virtual machines and device assignment in AVF. You could go to the link and have
look at that. There is a full presentation next by Serban on Transstable. So, I will just
give a one-line introduction. There is a new development process for Android, which is called Transstable.
And with that, I hand over to Ameth, who is going to take you for the rest of the presentation.
Hello. So, in this part of the session, we will look into the
technicalities and the realities of ASP development from a device maintainer and developer point of view.
Read the device, part of ASP or the device being hosted on the extended repositories.
We will cover both the cases. I will share the common pitfalls that we encounter while working on
ASP, the struggles, the failures, the success stories. I will talk about the set of ASP
breakages, set of stream breakages and our experiments with some of the new features that
Sumith has just talked about. So, this is what the ASP reference board page looks like as of today.
These reference boards are the hardware platforms that are used as reference for developing and testing
Android open source project. These reference projects provide a starting point for the device
manufacturers to build their own device. So, one of the most important aspect of supporting a
big device, supporting a device in a big project like ASP, is to keep up with always
churning changes. In this slide, I will share the common pitfalls that we encounter while
working on ASP. I will start with the most common one which is related to the ASP reposing.
At times we run into random failure, runtime failures which are totally or which could be totally
unrelated to the project that we have been working on. So, this is most likely related to the
ASP reposing timing. So, it is advisable to re-sync the sources after some time or wait for
the internal pre-submits to catch and fix the regulations internally. It is the problem persists,
then you know that it may be one or more device configuration that you may need to change based
on the new changes which you have just downloaded. And then there are some core frameworks
specific changes that need corresponding device configuration changes. The examples which I
can think of is the transition from HIDL to AIDL. Another example which we run into
from time back is that the lib hardware UI will come manager. There are few changes which assume
that the underlying Vulcan backend supports two or more cues for the full Vulcan framework to
work. But it is not true for all the Vulcan drivers at least some drivers which we have played with.
So, we could not switch to the full Vulcan support and we fall back to using OpenGL backend for that.
Keeping up with the average changing build configuration is something which we need to look out for.
If you maintain external projects then make sure that the soon properties or the soon
just to cut for the soon changes. I know because I ran into a build failure and it took me a while
to figure out that it is not the codebase which I am using it is something which has changed
certainly in the core framework that I took care of. The design migration seemed to be so it was a
big hype like two years ago three years ago that everything will move to visual build systems because
they wanted to move to more hermetic styles of hermetic style of building. But as whatever I have
seen so far the move to bezel is being limited to just GKI and GBL. The next up is tracking upcoming
years features as a new features keep getting added in a sp the device maintenance had to take
care of that. They do not necessarily have to implement everything but it is good to have the
core features or the important bits implemented if not all. I will start with GBL so we talked about
the GBL we were actually looking into GBL since the inception time we provided feedback we were
testing the half-baked things and everything because we have been picked by legacy bootloaders a lot.
If you are working on an older device you may be aware maybe you are lucky but at least in our
cases our bootloaders did not get updates as much as they would love to right and though it was
keep changing bootloader requirements year after year and starting from bootloader had a version
two three four then add the init boot it seemed like for every solution the problem the solution
was to add one more partition. So again so as Smith mentioned GBL is the generic bootloader
bootflow GFI app which decouples Android specific components from the vendor implementation.
It is very promising it is going to help a lot of devices and small audience to move away from
vendor move away from locked but locked implementations to the latest Android bootloader features.
Once you found out that the GBL is in a relatively stable state we enable that for a farer
forward devices in a USB right so the link is there if you can just click on that you will figure
out the how to enable the configurations which you need and I will not go into much details in that.
Second up error 16 k page size bills we also error 16 k page size bill support for all of our
devices. For those who are not aware of this 16 k page sizes do not support 32 bit apps.
At least the source code which is in errors I know that vendors will find a way to put some kind of
layers in between so that you can run through a bit apps but if you are product is based on a USB
if you are playing with a USB then 16 k is going to support only 64 bit apps.
Okay so we also ran into a bootloader crash while trying to enable 16 k page sizes it turned out that
our bootloader is only 4 k kept for was only 4 k capable and luckily if you were not the only ones
and as you were discussing earlier I have quick graphs in the build source code we could figure
out that okay there is a flag if it just set it you can just boot with if you can set it to
2009 6 it will take care of the image loading part and your bootloader do need to worry about
the user space build whether it is 4 k 16 k or something else.
Right so recently there are few again few more system core changes which caught our attention.
These changes simplify booting and write from external storage devices.
If you are booting from USB disk I mean if you have tried it earlier it was a pain I mean you
cannot do it without changing the system core framework maybe you can but I at least I could not do that
so now the solution is now okay so the existing solution is that there is a canal command line
you must be aware of android boot device where you specify the ccfs entry it will look for that part
it will again backtrack very backtrace and it will find the block device which is used to boot and
right from it works good but what if you have an internal UFS device internal UFS storage and internal
EMS storage but you are trying to boot from external devices what happens is that the platform
devices gets an emulated first so android first agent will find the device it will say okay this
is the boot device but it is because the system as entry is different because just set it for USB not
for the platform device so it will not boot it will just get stuck and it will time out now this new
boot hack what it does is you can specify a partition UID as an argument to this android boot
that bootpad UID it need not to it need not be any boot partition it could be just any partition
from that boot device the logic is that it will trace back the boot device out of that and it
will make sure that it can boot from that external storage another thing here is that if you have
encountered this or not you cannot boot with multiple block devices which have this android
partition layout for example I have an android device the internal partition is at android partition
layout I have plugged in a USB disk or let's say an NCSD card it again has a USB partition layout
GBL will trip on that I forgot to mention that just GBL will trip on that even android
will go crazy that okay what is happening I have no idea why there are multiple duplicate
partition of super or user data so now this approach it fixes that so that's I think this is one of
the main headlines of using this main the important point of using this partition new ideas that
we do this then booting from external storage it's going to be a little spin sorry I exaggerated a lot
on this because I have run into a lot of issues just because of this thing alone and I'm
really glad that the premium guys fixed it in a USB. The next one is the Ashmem recently saw
I renewed effort from the USB guys and they're moving away I hint that they're now finally
moving away from Ashmem Ashmem is the shared memory driver it has out of three driver since five
dot 18 I think but it's still maintained in android common can well because there are some
legacy applications which depend on Ashmem I have tools so this compact patches which I'm talking
about they make sure that you can switch to memory of the interface which is already upstream and
those compact hotels will take care of running those legacy applications which depend on Ashmem
I have tools right so checking relevant of stream projects maintaining a USB underboards is
like maintaining a device on a regular next this row so we have to make sure that the air stream
modifications that should be do a specific modifications or new additions which we do it should
align or I hope that it aligns with the upstream code base for this session I have listed down
the usual projects like the Linux kernel you bootboard loader the Mesa afraideno projects but
the list can go on and on depending on the kind of features that are being supported on your device
I'll start with the frequent help list down few breakages or few pain points which we encounter
the occasional upstream as I build breakages it used to be a lot thankfully we don't catch
new ones or there are no new regression maybe
I'm talking about the building upstream Mesa project and then right so the GFX stream so this is again
a point graphics stream is the I don't know if it's a graphics pipeline it's a graphics
subsystem these are the changes which help in your virtualized displays right so the graphics
stream changes whatever they have done in AOSP these are cross platform changes right
which are very bad you cannot have a dependency of one project on a different project especially
when the second project is an external project like Mesa right so we ran into a lot of
bell issues because of that so we end up having hooks in between and I hope that no one oils
have to deal with this sorry then as then our Linux breakages there are few Linux breakages which
are specific to Android common kernel we track core Android common patches like Ashman
or LFS for a debut amount and DM default key for metadata encryption you make sure that
these patches don't break on almost every much window as this FS nodes as everyone I mean I'm
hoping so it's a FS entries as this FS nodes are not stable interfaces right so kernel
developers have made it clear from day one that okay you can use this FS but don't rely on it
because it's not a stable API Android however uses SSFS a lot right as Linux access control is
based on your SSFS part so if you just change anything if you move from one kernel version to another
there's a chance that SSFS may not sorry as the Linux may not like it another thing is that if
you cannot reproduce ASP bug on a regular Linux distro then it is very hard to sell right so
there are some times they can be very fixed on that okay I cannot I don't see this issue on
my system or on my Linux distro so is it even a valid bug? Next up is the firmware bug
we sometimes run into issues because of firmware if it's our OEM sign firmware then it's worse
because then you know that you're completely logged on that recently there's the spinner quite
search in the LTS packages due to auto select cherry picks but in the end we get to see the
benefits of all the hustle and bug fixes and things when the Android release boots on your device
with no additional changes well mostly because they've been making sure that our devices stay
in sync with ASP code base throughout the development cycle here next up I'm going to talk about
maintaining so I've talked about the devices which you maintain in ASP the futures we enable it
get them with this how about the devices which devices which are not part of ASP right so I'm going to
talk about maintaining our running ASP on the development devices which are on external repositories
on your personal repository on your local stations and by the way if you are not if you are in
ASP developer which I'm assuming that most of us here are and if you have not already joined
this new ASP system's developer community yet then please do we believe that this open collaborative
initiative will be instrumental in information sharing and code-alloping genics of their solutions
and it would be beneficial for the entire ASP ecosystem this is a snapshot from the website and it's
this handful of devices and the projects that run ASP reliably now coming back to the main topic of
running ASP on external devices all the pain points that we discussed earlier this still holds for
these as well but the process I think is a bit relaxed you don't need to stick to the GKI if you
are okay with having a standard the funfic build based best build you can just stay with that
again some perks of running ASP on an external device is there is no obligation to run or support
ASP main development branch you can run a more stable GSI release branch and go on with the project
you can set customs and rules in your local manifest and that will save a lot of download time
as well as data usage you don't need to worry about breaking other ASP targets or breaking internal
targets it could be anything you need to worry about namespace conflicts because if you have
because you don't care about other projects like in ASP
you can skip this part side use these so we use these devices which we don't host in ASP
and I Android play ground or for our proof of concepts speaking of using the words for
ASP play ground and proof of concepts I'm plugging in the the words for Android projects
which you maintain we maintain a few devices few built targets outside of ASP we use it for our
little experiments like running with the stream canal no Android changes see if you can reproduce
the burlare if you can do all sorts of things you can play where they are all the things which
we have which I talked about earlier right the we came as thing the software entering bits
we can just experiment on all these things you can try the unified boot images and all those
experiments we use this workspace as a staging area for devices or features before pushing them
upstream that brings us to the end of the session the summary or the key takeaway from this
session today is that keeping up with the ASP is hard let's work together to engage on comments
of their solutions and join the ASP developer community and help us improve the ASP ecosystem