Thank you so much.
Yeah.
Last talk of the day.
We'll talk about machine learning for very, very small systems,
especially compared to what's all the rage today,
with giant language models, generative AI,
and so on.
This will be completely on different side of the spectrum,
and talking about how we can do machine learning that is useful,
for different kind of tasks at the machine,
to be somewhat size on small microsystems.
So when it is you and them from sound sensing,
we monitor ventilation systems,
such as the ones shown here,
and our customers are operating buildings such as this,
and for us we use machine learning to automatically analyze
to detect anomalies.
And today we'll talk about why machine learning on micro-controllers
is relevant, like how can we do useful things,
which on such small systems?
This niche is called tiny ML,
or edge AI is also getting popular as a term.
We'll talk about the EM Learn project,
the open source project that I maintain,
some projects that has been made,
real world projects with EM Learn,
and we'll cover a quick how through too.
So machine learning on micro-controllers and embedded devices,
what is it used for?
It's primarily used for sensor data analysis.
So we have some sort of sensor node.
We have the sensor itself,
can be camera, experimenter, microphone, radar,
and many lightar, et cetera.
And we want to process that information,
or all that data, and extract the useful information
from that large amount of data.
And only, like for example, directly act on this,
in robotics, for example,
or we might transfer it over to a larger system,
or potentially to a cloud.
But just the relevant information,
not all the useless data.
So this allows some key benefits.
We can make standalone systems that operate
with low latency, guaranteed latency.
Highly power efficient systems that can,
where the sensors can run for multiple years on the single battery.
And we can also make privacy compatible systems,
because we don't transfer all the potentially sensitive data.
And we can also make things low costs,
which is relevant because it enables massive scale
in terms of the number of units we will deploy.
And this is already used in many areas,
for example, in consumer technology,
keyboard spotting, so detecting hay Google, or hay Siri.
And so on, that's using microtroller,
then wakes up the system.
Or if you have a sleep quality tracker,
or a fitness tracker, that will also use this kind of technology.
And in industrial case, for example,
you can use this to track the health of animals.
We use it to track the health of machines.
Or you can just use it for fun projects at home.
For example, you can make a smart doorbell,
which opens the door for your cats,
but only if it's your specific cat,
and not any cat, or you can make a magic wand,
which you can use to make a gesture
to control some of the devices in your home.
And these microtrollers,
they are essentially a full computer,
but very, very small one.
You have, for example, maybe one kilobytes of RAM,
or maybe the largest system has 1000 kilobytes of RAM,
and similar in terms of the program space.
All the machine learning, all the code,
needs to fit in there.
And we're mostly doing inference on the edge.
Not so much learning on the edge.
The price point might be as low as 10 cents,
or as much as 10 dollars for a microtroller.
And this is a huge area.
These like any electronics will have one or more.
Like your car will have 100 microcontrollers these days.
So there are over 20 billion devices shipped each year.
So this is tiny chips, but massive scale.
And this is increasingly available for hobbyists.
So you can program these kind of things with the R.
We know ID, for example, or with micro Python.
But efficiency is key.
So the EMLARM project I started in 2018,
as part of my master in data science,
before I have a background lesson.
In general, I wanted to combine those skills.
And there's two aspects to these kind of systems.
You have the training process.
That's quite standard.
It happens on your computer.
Use familiar tools, such as psychic learn or caras.
And you have a pipeline that fits out of the deploy model.
And then we have the EMLARM project.
It's a Python library that you install.
And you have a one line export to a efficient device.
And then on the device side, you can then deploy this either with EMLARM C library,
which is written in portable C99.
It will run any system that can run C, which is basically any embedded system.
This is the link of Franca of that area.
And it's even run used inside the Linux kernel modules, for example.
Or you can use it with EMLARM micro Python,
which is quite interesting because most people
let the machine learning they know Python primarily.
So this enables to do all the application logic for even this kind of
tiny ML type systems in Python.
And then the EMLARM library is implemented in C and exposing Python APIs.
EMLARM supports the most common tasks,
such as classification, regression, and analytics that are relevant for
these kind of sensor data and better use cases.
Support a range of simple and effective embedded friendly model.
So this is a subset.
So there's no large language model.
There's nothing like that.
But you have, for example, decision trees, random forest,
classification, k-nearest neighbors.
You can do simple and device learning or adaptation.
And also some small neural networks, multi-layer perceptrons,
convolutional neural networks.
And here are some projects you think in learn, of course,
being open source.
Most of the projects out there we will never hear about,
but it's at least referenced in over 40 publications
by scientists across the world.
And here's this selection of examples to keep it kind of real
for what is it used for.
So from Virginia Tech, they created a system to track the health of cattle.
So using an experimenter that is mounted on the cow.
And in that system, you automatically
reanalyze with a decision tree model.
The extra amount of data in determining if the cow is flying,
walking, standing, grazing, or resonating.
These are like the stages of food processing for the cow.
And then you transmit this data over lower one.
Just the not that raw data, but the activities,
like the classes, what is this cow up to.
And then this is used then to track, to check for abnormal behavior.
So like if the cows are not eating for a long time,
that's concerning.
That's something you might want to check out.
And doing this all on the sensor,
enables it to run in their one milliwat,
which is 50 times lower power consumption and sending the raw data.
Which is really important in this kind of case,
because basically it allows you to have 50 times the battery life.
Here's a project from Samsung Research,
where they estimate the breeding rate of the person wearing a
this kind of earbola or device.
And they use a combination of audio data as well as the
extraometer data to determine how quickly you're breeding,
which is very important to track for those that have respiratory health problems.
And this is a practical solution that can be used in worn everyday,
not just in a clinical context.
And again, this is important that the power consumption is not too high,
so that you can still use this for like a couple of days without having to
worry about battery life.
Here's a more like fun project from of mine,
using an extraometer in a timer that can be attached to a toothbrush,
and you automatically track how long you're actively brushing your teeth.
Because you are supposed to brush your teeth for at least two minutes,
every time, and most of us don't do it.
So this device can help you.
So this is something that a Christmas, just a fun,
and one thing I'd like to highlight here is that I collected and
labeled the data for this in just a couple of hours.
So these kind of projects are possible also with a low amount of data,
because the problem is not so complex,
like a realistic type of shading that you do in the University,
that can be done by just selecting like five sessions of training data
with two minutes each time.
So that makes it really fun to get into.
Like you don't need as massive data sets,
you can go directly and solve things that you're interested in
around your house or home or business or whatever.
So there's full example code on that available online.
Here's like more of a research project of mine,
like trying to really find out like how cheap can we make this system,
like it's still a useful system.
And I post the question, can we make a useful animal system
for under $1 in total component costs?
And the preliminary answer is yes,
you can do either motion analysis,
so use an extrometer and you need a battery,
and the BLE transmission all that for under $1.
So there's like the bomb is around 70 cents for that.
And the model needs to fit that in the microintroller we have for that,
which has 8 kilowatts of RAM and 64 kilowatts of RAM.
And that's actually quite a lot from my perspective.
So we can should also be able to do audio analysis with that
with small neural networks.
So that is also possible.
So you can't do both.
You can't have a,
you don't have enough money for both sensors,
but you can do either.
So Emlearn has this C library.
So that's the easiest way to deploy on the vast majority of devices,
because he's the most common.
And you train the process using the standard Python ML libraries.
So for example, keras or psychic learn.
And yeah, we support selection of different models there.
Then you use the EM Learn library with this convert function to
convert into an efficient model.
We do a couple of simple optimizations there.
And then you can use the same function to generate some C code in this case.
So this is just some example code or like,
I'll sample output for a neural network.
And we'll look quite similar for the other kinds of models.
You can then include the generated header file.
And you construct your input data that you would typically read from your sensor.
And it might also include some feature engineering or pre-processing there.
And then you pass the data to the predicts function that we provide.
And that's all you need in order to run machine learning models on these kind of devices.
The alternative is to the EM Learn C library.
It's used EM Learn Micro Python library.
Because as I mentioned many that to machine learning,
they speak primarily Python.
And it's very interesting to be able to use those skills directly on these kind of devices.
So Micro Python implements a subset of Python 3.
It's a really nice practical way of getting into these kind of microcontroller devices.
And they have a really nice feature, which is that you can load C modules at runtime.
So similar as on a real computer, you can just do MIP install and a MPI native module.
And then you can drop in efficient C modules that you can use from Python.
And that's quite unique in this embedded space.
Very useful.
So we provide them a range of modules.
They're standalone.
So you can install just what you need.
For example, infinite impulse response filters for doing digital filtering.
Very common to do random forest or tree based ensemble models.
They're very power efficient and small.
They're very useful.
And also fast for a transport.
So we saw that it's useful to also provide some DSP functionality because it was a bit lacking in the Micro Python space.
And you can do convolutional neural networks.
So there's examples doing image classification, et cetera, available in the repository.
And doing this is at least 5 to 20 times faster than, for example, generating Python code and executing that with Micro Python.
So you get the convenience of using Python, but you get the speed benefits and the size benefits of the C modules.
And the process of using that is very similar.
You train your model.
Use the convert function.
In this case, we often save the model as a file that because Micro Python has a file system.
So we can copy the model definition over as a loadable file.
And then, yeah, when installs the library, we're just one line as well.
You can create the model.
You have to specify the capacity.
And then we load these the model definition from the file system.
And then again, you read your sensor data.
And you pass the data to the model predict function.
And you get the classification or regression or anomaly detection output.
So that's all that we had.
So in summary, machine learning's used in embedded systems to automatically analyze sensor data.
That's the predominant use case.
And then we can make practical applications in this space.
We just, a few kilobytes of RAM.
And within just a few million watts of power.
Or just a few dollars in terms of build materials or hardware costs.
And EM Learn is an open source project to help deploy machine learning models to microcontrollers.
And you can use it either with a C library or micro Python library.
So there's some resources.
And you'll find in the slides, of course, a line with more information, including some other talks,
and go a little bit more into depth in particular aspects of this niche and the library.
So thank you so much.
Thank you. Thank you very much.
This, this two graphs was, this two brush.
I have to buy it like this.
No, you know, you, it opensource.
So you're just download the three different files.
You have three different theory yourself, use it there.
If not, you send it to someone print and you download the code.
Okay, that's cool.
Any questions?
Yes, please. I'm shouted.
How come is this small device has some kind of a code processor?
Like a vector.
And can you explain on this network?
Yeah.
So the question, I'll repeat it.
So how are there devices that have some co processors that you can use like vector processing and stuff like that?
Is it common or not?
Yep.
It's getting more common because this is using doing machine learning is more and more common and relevant.
But hardware takes time, right?
So it's not like 99 will have it.
But there are some recent instructions.
So arm has like standardized,
Cindy instructions for the cortex and four.
That is what useful.
Can you give it like a 4x benefit on like neural network type problems?
And then there's helium that is coming.
They have standardized it like four years ago.
But devices are not really out there.
But then there are on the ESP32.
There's the S3 which has a bunch of smaller vector instructions.
So it's possible.
It's getting possible to do like a get like a 4 to 10x type speedups.
And then there are now like SC just announced a new microcontroller just before Christmas.
That has like a dedicated NPU they call it neural processing unit.
So definitely this is becoming more and more common.
And to on the high end like if you do like powerful image classification or object detection,
for example, you want to use that.
But then if you're doing with extra amount of data for example,
it's usually overkill.
And sometimes it can be less power efficient because you have this copressor,
you need to shift data in and out.
So it's definitely getting there.
But you can do quite a lot without them right now.
So let's see.
We will see how much we implement like those instructions.
They want to get common.
Yes.
But right now we prefer just doing plain.
See runs everywhere.
Simple and usually enough.
That's a question.
One question from mine.
Do you see any time when those microcontrollers will be around in LLM inference?
LLM.
Oh yes.
This is there's a surprising amount of buzz about LLMs.
And I don't quite understand the use cases yet.
But it might be relevant for Baltics.
But there's a lot of hype around it.
We will see in a few years if there's like useful things or not.
I don't really know.
But you have definitely a lot of hype.
I'd like to talk to you.
You need to do more or more like if personal trainer.
I don't know.
Thank you.
Thank you again.
Thank you.
And that was the last talk.
And let me pass the mic to time.