Okay. Hello everyone. Today I'm going to talk to you about Bluezy, Bluezy, I don't think
Bluezy project for implementing Bluezy in production.
First of all, where am I? I'm George. I'm a principal software engineer at Klabera.
I have been working on multimedia things for the past 14 years. I worked with just your
mirror, I'll show you a pipe wire, I'm the maintainer of wire plumber. I've contributed
to automotive grade Linux, audio staff there, and I'm near to Bluezy.
Now, in the past seven, eight months, I had a new adventure that stepped out of my audio
zone, let's say. I teamed up with my awesome colleague Frederick Dines, and together we deployed
Bluezy as the Blue to stack of real world vehicle in for payment system, which is currently
under going testing for qualification, and as far as I know it's the first vehicle that is
going to actually use Bluezy in production.
The project involved actually switching away from a proprietary stack. Previously, that
system had a proprietary Bluetooth stack that the customer that we worked with wanted to
remove and replace with Bluezy, and the goal was to just implement the same features,
no, nothing different. Along this process, we also had the chance to improve the code base,
improve a lot of open source components, and I personally learned a lot. I learned about Blue Tooth,
the audio part. I learned also about audio use cases on the automotive, like how the audio
DSPs work and what kind of requirements they have there. Now, in this talk, my intention is to
pass on what I learned for other people that might want to do the same thing later on,
like a change from a proprietary Bluetooth stack over to Bluezy. What should you expect
if you want to do that? First of all, let me say that it works. It works pretty well.
There is no difference from the proprietary stack. We support media player, phone calls, text
messaging, the contact list, access of course, and pairing controllers, like gaming and
roles, for example. The architecture that we used looks like this. So inside the Linux
kernels, there are some parts of Bluezy. There are drivers that talk to the Bluetooth
adapter, and implement some protocols, and then on top of that, we have Bluetooth D,
the main Bluetooth D, and then we have OBEXD that implement some of the profiles for
objective change. For example, the contact list exchange works through OBEXD. On the other
side, we have pipe wire and wire plumber, which was used to implement the audio profiles.
Namely, the A2DP profile, which is the advanced audio distribution profile, which is used for
transporting audio from the media player or the video player, and the HSP profile, which is the
hands-free profile, which is used for phone calls. On top of that, there is some proprietary
middleware and UI components that the customer already had, and they just wanted to
switch that to talk to Bluezy instead of the proprietary stack that they were previously
using. As part of this process, we added some things. First of all, we added
telephony support in pipe wire in the hands-free profile implementation. Like the hands-free profile
implementation was already there in pipe wire, which was being used for enabling headsets.
So if you have a Bluetooth headset and you want to enable bi-directional audio with a microphone,
if you are using the HSP profile to do that. That was working, so the audio part was working,
but what we were missing was some control to be able to send commands to the phone to let's
say make a call or answer a call or put a call on hold. Now this with this functionality
pipe wire is now able to replace a phone, which was the only project that the only component
that could do that before, and we decided to do that in pipe wire instead it was somewhat simpler
to add that in pipe wire. We also added the basic imaging profile support in Bluezy.
The basic imaging profile is a profile that controls for images to Bluetooth. We added a subset of that
that is able to download the cover art from the media player, like for the media player
that is on the car. If you're using something like Apple Music or Spotify on your phone,
it's a requirement that you have the same information together with the album art
on the infotainment system. That was Merrick's in Bluezy, while the pipe wire telephony support
is not yet merged. It lives in a merger quest. It needs a little bit more attention from me.
We also fixed some other issues, mostly related to the audio video remote control profile in Bluezy.
Now what were the challenges? As you can understand, we're working with people that came from a world where
they only knew about how the proprietary stacks work. For us, for me, at least, I didn't know
how the proprietary stacks work. I lived in my Bluezy open source world and they lived in their
proprietary world and we had to understand each other. As we moved along the project, we found
this difference with what is different. On proprietary stacks, it is very common that the
Bluezy controller is exposed as a serial device and then you have a user-space demon
that which is the proprietary stack. This is the software, the proprietary software.
That basically grabs the serial port and then talks with the controller using the
HCI protocol. That's the host controller interface protocol. There's nothing in the
kernel space except for the serial driver or something like that. It is probably a good
part for the people that implement this demon because they can reuse the same thing
in other operating systems. It doesn't matter if you're implementing this one Linux or
something else. It's just a serial port as far as the operating system is concerned.
Now with Bluezy, this is different. The serial port is attached directly inside the kernel.
The kernel then exposes. The kernel actually implements internally the main Bluetooth
protocols. It talks HCI with the controller. It exposes L2 cap which is a higher level
protocol and score which is another higher level protocol. You can basically request
rockets from the kernel. This rockets have the Bluetooth address family. Similar to
how we have the internet address family and we have the CPP IP to DP sockets.
So what makes sense?
Yeah, that makes sense but it is different from what people are expecting.
Now some other expectations that were different and things that we were looking at.
One is that in Bluezy, if you're using Bluezy, the profiles are not really implemented
in one place. You don't have one demon. You have multiple demons and multiple components in those
multiple versions because they are different projects. You cannot really say that this version
of Bluezy supports Bluetooth 5.2, for example, 5.4 or 6. There is no such thing.
You have to collect the PCs based on the version of the Linux kernel that you have.
Based on the version of Bluezy that you have, the version of pipe wire. Bring it all together.
Bring in whatever UI that you have or other management software that you may have to make this all work.
Then you have to assess what version you are supporting. You have to run actually the tests.
You have to do the testing and say, okay, this supports this feature.
Take the list from the Bluetooth qualification.
Look at each test and say, okay, this should sound. So this is what we support.
But yeah, that wasn't something that people were expecting. So they were asking, like, okay, which version of Bluetooth does this version of Bluezy support?
And we're saying there's no such thing.
Another thing is there's no straightforward way to filter the API packets, which sometimes can be a security requirement to implement firewalls,
like to filter what is going on between the controller and the host.
Unfortunately, there is no, even though the architecture looks similar to the internet address family, where we have net filter in the kernel,
and filter the packets, there's no such thing for Bluetooth.
There are some solutions, but they're not as straightforward as it is with internet packets.
There's no support for containers as well.
So right now, it is a requirement that you run Bluezy on the host operating system, on the host container.
You cannot really isolate using network namespaces and run Bluetooth D in a different network namespace than the main one.
These are fixable of course, but needs some work.
Other differences that we saw is Bluezy hides complexity in general. It tries to be abstracted, tries to provide you with a nicer interface.
But the customer codebase had been already developed using the proprietary stack, so they had access to low 11 information before.
And that was clashing sometimes, like the logic that they were implementing didn't match exactly to the logic that Bluezy exposed.
So we had to work around that.
They were also using a lot of logging to monitor things, to monitor problems.
And we had to add some logs also in Bluezy and pipe wire all around the place, just to make sure that they collect all the information.
Also, they were expecting to collect the logs from one place, from one demon, and that was no longer the case.
Depending on which profile you want it to debug, you have to look at different demons and different logs and correlate information, and that is more challenging.
Sometimes there's not enough error granularity in Bluezy as well.
So one use case that I can remember is you have a phone that disconnects and Bluezy says it disconnected, but why?
There can't be a number of reasons why this connected. It could be because the user just disconnected manually.
It could also be because the phone went out of range.
And these are different use cases, but there is no such information given in Bluezy, while there was in the other stack.
And also, Bluezy maintains state. So Blue Tuesday, for example, maintains the list of pair devices internally, and it has that in a directory structure.
That wasn't the case with the proprietors stack, and there was another layer that was already doing that in the customer's middleware.
And when we discovered that we realized there was some friction there, because the higher level middleware was already trying to do stuff that we were already doing, and sometimes mismatching with Bluezy.
Another pain point is vendor specific HCI commands. So HCI, as I said earlier, is the host controller interface, is the protocol that you talk to between the host FFC and the controller.
There are standardized HCI commands that talk to controllers, but there are some vendor specific as well, like vendors add stuff to make sure that they implement custom functionality.
So many profiles, we actually found many profiles that need the vendor specific HCI commands, and that wasn't easy to handle.
One particular that I personally handled was the HFP profile, the HHP profile, which typically on embedded systems, the audio that goes from the phone to the Bluetooth chip.
It doesn't pass to the main FFC, but instead there is a hardware path between the controller and the audio DSP, where the audio travels directly through the hardware path, and the main FFC never gets the audio.
And that's a little bit obscure, we don't exactly know how that works, every vendor has a different way of implementing that.
And in the case of the particular chip that this customer is using, in order to make this hardware path work, we had to send vendor specific commands to configure the audio format, the sample rate, like the codec that was being used, and things like that.
And we had to do that every time that the audio path was opened and then closed. So every time that you had a phone call, you had to send this commands to open the path and then as soon as the phone calls stops, send the commands again to close the path and continue.
Which is pure unnecessary pain, this could have been part of the driver, like this is something that belongs in the kernel, really, and it's so static.
Like the format, the sample rate, etc, are static information that depend on how the scholarships have been wired together, it doesn't depend on parameters that come from the phone or anything.
It's a really information that could be encoded in the device tree or somewhere else to make this work, but it's not there.
And yeah, that's a pain point, we had to patch you the space to do that, you the space sending HCI commands to enable that path.
And with that comes the next problem which is security concerns of doing that. So if you allow you the space to send HCI commands, you need to give it capabilities and the first capability you need to do is to give the row the network row capability at allows sending row packets.
With some commands, some functionality we also need this cabinet admin, so we need it to be root.
On this particular system, nothing runs as roots, they are all isolated, they have, they are giving capabilities one by one, and the customer was not pleased, obviously, when we decided to add cabinet row to pipe wire in order to be able to send these commands to enable the audio path.
I'm happy they were not pleased, that's actually a good thing they care about security, that's good.
And well, eventually we found the solution to that, which was very specific to their system, and it's not something generic.
We need to, for a generic solution, we need to really fix that somewhere else, I think the kernel is the better place.
Also, there were questions about the music demon itself requiring those capabilities.
Like they ask why does bluesy require cabinet admin to run?
But the answer they are is that this is part of the bluesy design, so bluesy requires that these permissions, so that it can execute privileged operations, which is a security feature.
Basically, the kernel only allows the bluesy demon to run those operations, and everybody else has to talk to bluesy with debas.
Instead of having all the user space components, talk directly to the kernel and try to do privileged operations that would have been a disaster.
And some non-challenges, something that we thought was a challenge, but it wasn't, eventually, the card parsing.
So the card is the file that describes contacts, when you download contact list from the phone, the contacts cameras as the card files.
And initially we thought, okay, we need to find the parser for this, we looked at parsers, we found parsers in GNOME and KDE, other places on GitHub,
they were too hard to package to many dependencies.
And then we looked at automotive red Linux, which had something else, it had actually a parser itself, like it was very simple parser there.
And we adopted the solution.
We implemented the VCAR parsing again, which is so simple, it's much easier to write a parser than to package any data.
Now, next steps, this project concluded, of course, for the community, the things that have been contributed are there, all the open source bits have been upstream or have been pushed to merge requests.
Something that I would like to do after that is to expose the telephony bits of pipe wire to the desktop, so have an application that you can make calls.
I think GNOME calls exist, which is possibly an interesting target, maybe there are other desktop apps that I haven't looked at yet.
But yeah, I was thinking that if functionality that you have on the Mac, for example, where if you receive a phone call, it automatically shows you on the laptop that you have a call and you can answer from the laptop directly, that's something useful.
And hoping to get that also into what if great Linux, like get all the changes into automotive green Linux again and have a demonstrator again in 2025.
Thank you very much.
Do we have any questions?
So you touched mostly upon all the challenges for implementing this.
We're also some notable advantages over choosing this, this open source stack over the proprietary solution.
No, the goal was to achieve future parity, the advantage is open source, right, so the customer wanted to use open source for, you know, for the future to to be to have access to the source code and be part of the community.
That was the reason, no technical advantage.
So I know most of the proprietary Bluetooth stack solutions, they, the vendors basically say, well, we've tested with this, you know, several hundred phones.
Does that, is that something your OEM customer worries about or is there discussion about trying to enable more testing against lazy upstream.
So in our case, no, we didn't care about this hundreds or thousands of phones.
We mostly cared about five or six specific ones that are, you know, the 99% of phones that are being used by their customers because.
They have the telemetry data to understand which phones are being used the most, so that's what we've tested with.
How did you solve the who owns the, the data about pair devices, did you.
I'm impressed about, did you change, Blue Z.
We modified the customers logic there, so we kept Blue Z as it is, so Blue Z maintains the pair devices and the, the layer above adopt.
Yep, I think that's the end of the time.
Right, thank you very much.