So, it's just time and I will start now, so I'm too to and I will present to you the
USB-D 2.0 device class in Zephyr.
So, first of all, Holtz started on why I'm talking to you today.
So, it's a friend of his amylization and he had a project where he has a custom controller.
The goal is to have a performance where he moves on top of that so we have some old distance sensors
like IRB's.
The problem is it's all analog and we have lots of cables, so it's very noisy and it's
difficult to have some precise movement and also there isn't any feedback for him to see
anything because it's an input only thing.
So we built a very first prototype, so we see the box.
We now have touchpad RGB, some encoder with an RGB light as well and some new fancy
time of flight sensor and soon enough we had a custom prototype, so you can see I'm not a PCB
designer probably but yeah, this is working and so we started to do things, make some
sounds, we also broke some parts by the way, that's happens.
But then finally it is the final case with the nice casing and the distance sensors are
on the top.
So why is Zephyr here, lots of talk about the device pretty today but definitely I like
it, but also the key config system, the west tooling etc.
It's free and up and so on, so I can read the code and simply update it if I want, there's
a large collection of drivers, we can just try different components and try and swap
them out.
Also Zephyr comes with some subsystem and OS services like you have the USB subsystem
built in, we also have the logging, the shell etc etc.
And if you ever worked with Linux, then it's definitely a different code base but you will
feel at home in Zephyr.
But there wasn't any MIDI support in Zephyr, so let's build it.
Let's first of all start with the very old MIDI 1 protocol from the 80s, so it's a very
old thing and the other hood, it's simply a serial link on which we can convey at two 16
channels of instruments I would say.
You may recognize the Iconic D5 connector, which was at the time pretty much all the
rage, but now it's basically only the MIDI Iconi in every software you see.
And basically it's a protocol that's allows to connect some controllers like a keyboard
and knobs and fliders to some synthesizers, so things that we'll produce sounds or maybe
do other things.
So what's coming on this serial link, so there are some messages, but the most important
thing to remember is that it's a stream of bytes.
The first one will always have the first bits set to one, which means it's a start of
the packets and all the other ones will be on seven bits with the highest one being zero.
So we have a different type of messages, but the most common one are not on or not
off, which start by eight or nine and then the channel number, which can go from zero to
16, then the note number and then the velocity, the velocity being like a whole heart,
you press on the key for instance, you can also send some control changes, which is what
we use in our controller, so knobs and fliders, simply a value, and also this has which
control number, so which knob you're actually rotating and again the value.
Also you can tell a synthesizer to change its program, so to move from a sound, which
is a piano to the sound from a duck, I don't know, to the pitch band, which is like
if you have a guitar, you push the strings, so you pitching up or down the sound, etc.
There are also the system exclusive, which allow you to send arbitrary payload, this one
is a bit special, because you have a start by it, which is f0, then f7 to completely
so just a few examples, to play a note to sound the message 90, 46, which means play a sharp
of 4 on channel 1, by the way, as programmer, we start counting a zero, but musicians
start at 1, it will be the same for all the slides, we can also say, hey, synthesizer, please
load that program or I can even send a nascist string as a 6-6 message, so just to be clear,
if we look at the midi sequence or something, the notes on event is sent here, to tell
I start playing the notes, and then we have the notes of event, when you stop playing the notes,
so it's like when you press the keyboard and release the keyboard, and also the velocity, which
indicates, how hard you press the keys. So we are known in the 90s and the 90s, if
you want, and so there is this new thing called USB, everyone has it, and it's fun, and so we
are going to connect midi to the USB, so the specification is curious, I would say, but there's
the USB midi function, which has some inputs and outputs, so we have the embedded connectors,
which are actually the one that goes to the host, and then we have the external connectors that
go to the outside world. And so an embedded midi is actually something coming from the host, which
ultimately will be a note put. Yeah, so for example, if you look at what happens in this
midi interface, USB midi interface, this means that if we have the input from the keyboard,
it corresponds to an external midi that goes to an embedded midi out from midi in and points,
then to the host. In notes, and you get other way around, if you have an external midi output
to do some synthesizer, this is how it looks like. So from the specification, we can represent a lot
of different topologies, so there's a typical midi in out, but also you have the opportunity
to describe something that's simply conveys midi from the external world to the external world,
without going to your PC, you can tell that's midi, USB midi one, or also this complex example
from the specification. But in practice, I've always seen, always this one used, I've never seen
any other topologies being used in the wide. So in 2020, came the USB midi two specification,
which I've compromised midi two specification. So it's very simplified, as you can see,
we don't have all those external embedded and going from in and out and a lot of
tringling around. So it's just, no, we have some group terminals, and so group terminals linked
the USB to the midi function I would say. And in this context for instance, we have no a pair of
output, and those two pair belong to the same block. So they form like 32 channels that are together
two times 16. So you can have like wider outputs for some complex interchange or stuff.
And the same way, as input for instance, so this described that we have one thing, which is one
only thing on the instrument or something that goes through all these ports. And this is something
you can't tell in USB midi one. And so what do we send through all these pipes? It's the
universal midi packet. So no, it's aligned on four bytes, and it's one to four of these words
along together. So we have those function blocks, which are the the group terminal, we
allocated in the previous slide. And this packet can carry either midi one or midi two messages.
In midi two, we have an extended numerical range. We can have some notes attribute, so we can
specify some detunes or some tuning that are represented in midi one. For instance, in non-Western
music, we have notes that can be represented in midi one. And also things like calls, tempo,
time signatures, we can even do karaoke over midi two. But the main point here is that the
midi two specification acknowledge that modern device communicate on high-speed link and that
are bidirectional like USB. Just one point, midi two is not USB midi two, which itself
is not USB two. So you can, for instance, send midi one message with USB midi two on USB one.
Just a week here. So just to be very clear, midi two is the presentation layer. USB midi two is
the transport layer. And then USB two point is all the left-house view. So some example of those
universal midi packets, which are not aligned on four bytes, they all have a field which is called
the message type, which identifies, yeah, what's inside this thing, in which specified the length
of the message. So we have the midi one message, which starts like before, message type two,
then the group, which is on which output we're going to send is. And then it's actually
exactly a midi one message like we saw a few slides before. Then we have midi two, the same message,
not on the top. But you can see that we now have an extended value for the velocity, which is
known in 16 bits. And we have some attributes which are related to all those two things. I told
about not going into the details, but the main idea is that you can put more data, I would say,
in the notes. And then we have some utility messages, like with the message type f,
UMP stream, which allows the host to discover some midi device and possibility to configure
them the dynamically to allocate some synthesizer or all things. Finally, one last word about
USB, so I mean the embedded rumor, I assume that you know a bit about that, but just to know
on the USB bus, if you are device, you must expose tons of descriptors, like it's just say to the
host, hey, if you find me, I'm capable of this, this and this. And also, I have this and point,
so this pipes to communicate from the device to the host or the URL. So now, overall, we have seen
all that. We must expose all this ton of descriptors to the host, so it can find or device.
We must respond to class specific requests, so the host will ask us, hey, what's your topology,
and I will answer with the map of my inputs and outputs, which is the case in midi two.
USB midi two, excuse me, and then we can exchange some universal in the duplicates on this
endpoint. So in Zephyr, we have USB device next, which is the new API for implementing this
b device. So what we have as an implementer, the hardware window will provide us or some
is people. The USB device controller, which is the part responsible for actually getting the
wires at the correct voltage and sending some data. And on top of that, you are going to write
some classes. Usually, when you write a USB device class, you will wrap it into a Zephyr device,
which allows the end user application to simply use device dtget and get a handle to your thing.
And all this is managed by the USB D context, which manage all the wiring, making sure that stuff
enabled at the right time, that the descriptors are open together. I have to say it's much more
easier than with the old USB stack, where you have to do all of this manually. So the operational
models, so we are going to exchange up to 16 UMP groups between our Zephyr application and the host.
So to do this, we can receive them by registering the callback. And we are, it's a bit off,
but we can just use USB D to send all of that. And all of this happens through the USB
dv device that we get from the device 3. So if you look at the descriptors, unfortunately,
the specification requires us to have first a fully compliant MIDI streaming interface from
the USB MIDI 1. So in that version, we had to put all the jack-in-out embedded external
inside the USB descriptors domain USB descriptors, I would say. But in this implementation,
I just choose to put an empty class. So there's a USB MIDI 1 interface, but it's non-functional.
It's functional, but it hasn't any input or outputs. And then we have the MIDI streaming
for USB MIDI 2, which staying contains only the basic definition. We just send the topology
through some class specific requests. But this implies that your host PC has to select the
else setting number one to enable MIDI 2. And this is true for any USB MIDI interface.
So let's make some noise. First of all, we just enable USB device tech next in our
config and USB D MIDI 2 class. That's all the project configuration. And then in the device 3,
we are going to insert some noise, which is a Z4 MIDI 2 device. And then we can declare
our input and output blocks. Like this, we just put some Chinese insights. So for instance,
here the block one here has two bi-directional input bi-directional connectors. And so we have
it started the group 0 for us programmers, which means one from musicians. And spans two
groups, which means 32 channels, 2 times 16. And then we also have the block 3 here. I skipped the
block 2 for a confusion, but you get the AD, which starts at two transitions. And this one's
suddenly capable to send MIDI 1 data. And it's an input only from the perspective of the host.
Then we can register some event handlers on the USB device, the USB MIDI device. So we have an
event on ver if our dear host select the right interface. And USB is correctly enabled. And so for
instance, we can simply light up the led when the host has selected the interface USB MIDI 2.
And we simply just say to our MIDI device, yeah, I want you to have these operations, like it's
usually the case in that section of the code. And we are going to see this one just a bit after,
but that's not just a good fast. So if you want to implement a simple MIDI keyboard, so an input
coming from the external in and going to a host, it's actually fairly easy. So we have our USB
MIDI device, which we get from the device free. We are going to use the input subsystem from Ziffer.
So actually it's very easy, you just enable it. And when you press a button, you can simply have
a call back that's called. Then we simply build the UMP packet. So if we do a key press, it's an
atone. If we do a key release, it's not off. And then we simply build the thing. So I've just added some
small models. So you can even understand what happens. Actually, not every detail is very important
here, but you get the ID. And the other way around. So if we want to get some events coming from the host,
now, we can choose to only respond to MIDI 2, this time, MIDI 2 packets. We get the channel.
So as I recall, there are two groups. So we have up to 32 channels. And then we just look that
which type of MIDI 2 event it is, and atone or not off. In which case we can just start
play some tone or stop playing it with notes that has been given in the packets, as well as
the velocity, which I recall is how hard you press a key. Again, I have some models, so you can
follow it later on. So, it's time for a demo. I have this board on my desk here. So I have a
bitteria, so you can even understand what happens. So there's the SD link on the left, which just
is, which is just used for power today. We have the user button, which will serve me as the input
keyboard. We have the LED, which is green, if we have USB MIDI 2 enabled, then we have the famous
USB MIDI going to my computer and maybe some audio outputs. Now, you have to tell me if that's
readable. I will actually try to put that. So, I've scripted it, I've scripted it, so it's a bit
easier for me to present. So let's first start by finding all the device. On Linux, we can just
look at the other sequential appliance. And we see that you have like all the blocks I presented
in the other slide. I can read some events in the basic row, UMP formats, directly from the FSD,
UMP blah blah blah. And so I can also record them here on my MIDI Sequencer. So if I play,
I'm just like press the button, you can see the notes on, the notes off, that's impressive, no?
Also, we can see it's in the old other formats. So we have like the two groups, I will stop this.
Oh, yeah, I had a timer, so I didn't demo that you can also read them in the old formats.
Oh, yeah, I should pop my speaker now. So yeah, probably if I just had that, I can play from
my sequencer and it doesn't work. And maybe I haven't selected the right outputs. I haven't selected
any input for two. So yeah, it's very, very impressive, no?
So, okay, I can play some note like this, but I can also, yeah, play some notes from all
sides. I used that picture, but I can also replay you.
And so that's what's my presentation. The main points I would like to tell
it's just been merged yesterday, evening. So we know you have USB-C2 coming in the
for a 4.1. I hope that we can do some fancy thing. You can find me on the Zephyr Discord
as I do and all those platforms as well. And if you have any question, I'd be glad to have them.
Question.
Thank you. That was quite interesting. Do you know how far the penetration of MIDI 2.0
USB-Med2.0 in the other device segments is?
So I don't know really well. I can't tell even the vendor who put them. It's very
new, so it's from the 2020. So until the spec is completely released, the vendor wants to implement it.
And then there's the development time and the time to market. So we are only seeing the first
you MIDI device coming up. I've never seen it once, so all the thing I developed here is like
I've just read the specification, tried to figure out if that matches what the specification does.
So I'm not a product person. Any more questions?
If I want to adapt this to my existing keyboard and hardware, what would I need to change
because it's older than 2022?
So if you have a keyboard which has a MIDI outputs, unless you're prepared to
unsold or you're the microcontroller inside, we sold or won from you, being able to
reprogram it using the server because I have a stack of whatever you want. This is what it would take.
I mean it's software, so somehow you need to update the software of the thing running on your
keyboard. I mean it's Sapphire also able to talk the old MIDI protocol.
Yeah, so there's the old MIDI one like the physical protocol which is a serial link and this
is not what my presentation is about. It's a very simple UI, so you just need to implement a very
simple parser. And then if you want to plug it to the USB, you just wrap it into UMPs
and use the Ziffer USB MIDI API to send them to the host if that's something you're up to.
But if you have an existing device and expect it to speak MIDI too, then you will need to
reprogram it somehow. Yeah, sure. Now I was fine with the first use case. Yeah. Thank you.
You can ask the question I will repeat.
Is it possible to use the pair to translate from MIDI to the zero?
So the question was, is it possible to use Ziffer to translate from MIDI 1 to MIDI 2?
So translating from MIDI 1 to MIDI 2 is actually quite easy. You use the 7 bits for instance from
the velocity. You shift them by 9 to the left and this gives you the velocity on 16 bits.
So sure, you can do that. The other way around is a bit, it's possible, but you lose either
the resolution or also to not attribute the two links and stuff. So going from MIDI 1 to MIDI 2 is
like trivial, but the other way around is not possible. You lose some information.
That's all from me. Thank you very much. Have a nice talk to everyone.
Thank you.