So, let's start it, so we'll have Augustine talking to us about accelerating
robotic development for simulation, and I am personally very interested about that, so
that's going to be awesome.
No pressure, never.
Hello everyone, well, thank you very much, just edit a bit, I'm working at
correct human consulting films, focus on robotics, and other topics based in Argentina,
but we also have a pretty much distributed team, and this is super helpful for the topic of this
talk.
Some of the projects that we have are nano and educational robots that we are going to use
as a target robot for the top, Beluga, and Monte Carlo localization package, check it out,
super cool.
This is my colleague, Hiku and make it, I'm helping him with the talk, he's the simulation
lead, I take him in, this is me, if you don't recognize me, here I am, you can talk to me
through social media or after this talk, and this is our journey today.
So, who has ever used the simulation engine for robotics, is your hands, awesome,
who has ever used a robot without the simulation engine?
Nice, okay, so let's talk a little bit about robots and simulators, this is a super
simplified scheme of what a robot is, right, so we have a hardware platform, software platform,
if we think of the hardware platform, we have sensor actuators and a compute engine,
we think of a robot suffers structure, we have an operating system driver,
application and system libraries, and then we have Taiga, and the robot is interacting with an
environment, the physical world, surrounding it, people, things, this is a super cool example,
here I am, I'm part of an environment, but this robot may also be interacting with a
combo, sorry, of more compute, services, other systems running a like a robot, so we need to account
for that one building robot. Let's talk about the simulator, what's that? Well, a simulator is an
application that runs on a scenario, right, and tries to follow a given model, and a scenario
is a representation that they both do at a time, and it has a lot of systems on it, and the key thing
is that it's not static, right, it evolves, what is the model, a model is an error representation
of the rules that governs a process, so if we think of robotic simulators, it's an application
which allows to model the scenarios with robotic systems, it seems to be pretty straightforward,
so far, right, but when we think of an architecture, this is a super simplified one, and it
becomes to add more and more complexity, why? Because we need logging, every simulator has logging,
we need a physics engine to evolve the models, and we also need rendering, so, closing the
limit, so we need a rendering engine to get the visuals to process the sensors that need to capture
environment, that's super important. All these three things alone, they don't do anything, so we need
to put everything that glutes, glutes, all the systems together, and that's the causing,
that's a main responsibility. But then we need to interface with outside world of the simulator,
those are interface APIs, there may be many like the communication system that interacts with
the external world, or direct clients that get into the simulator itself, and they want to talk
with the robotic software stuck directly, or any third-party external services, even to control
the simulator. Maybe we want to step the simulator just once, at the time, or let it flow.
With this, we get the basics of the simulator, and then we may get a target for an application,
or get a target for a testing environment. So, let's recap on the robot diagram and think of
what are we going to replace here with the simulator? Probably we want to replace a hardware
and environment. And this is how we achieve a robotic simulator, right? We get the simulator
in the middle and interacts with the onboard software platform. So, this huge blue block here thinks,
but there's a real hardware under the hood, but actually it's similar, it's similar, it's
running on a computer platform. It can directly use or access the external of board services,
and your board service is also can interact directly to the simulator. So, far so good,
excellent. So, this is an A-R for robotic arm that we build in the company.
It's open hardware, open source, thermal, open source controllers, and right now we are using
move it, with a motion, which is a motion planner through service, in the middle, and we have a simulation
in Gazevo with it, right? What can we do with the simulator? It's not because of Gazevo.
But we can do a lot of things. We can model the actual physical thing that we have, we can test it,
we can validate it, learn and tune parameters, let's say that we want to go faster to achieve the
role. We need to modify the controllers and we can learn that in simulation. We can generate data,
so as to later on get closer to the real thing. We can train,
valid data and train an A-R, or M-O-D controller, and we can even solve, right, because we may not
have the final configuration of your robotic system that is going to be deployed to your customers,
but maybe we can use this in order to let your customers know how it would look like,
how it would proceed on the real scenario. So, this comes to a bit problem,
like, how do I find the right tool for the right job? There are many, many possibilities in the market.
So, what we want to present today here, it's like a set of questions that would guide
hopefully objectively towards that conclusion. So, let's start with a rename.
You have to identify use cases. So, it's not the same if you are going to use it on a recurrent
manner for testing purposes or for validation purposes. It might look like the same,
actually it's not. Imagine that you have a team of 50 developers with five products on the market,
and you want to have that on a CI, and you have, I don't know, 50 PRs per way,
HPR gets five commits until it gets merged. So, you have like 250 commits, and you want to run
a simulation, test suite for that. You need to think of a compute power for that. That's a lot of
cost that you need to put into your infrastructure team, so as to get it. Fast, efficient,
and achieve the goal. But if you want to just validate things before release, you've made a
wrong of QA. Probably it's not going to be that exacerbated the number of times that you're
going to run. Is it going to be a development tool that do your workforce need actually
an actual simulator because they don't have access to automation software example? Well, that's
super important to know. The kind of requirements that the development tool has is completely different
from the one that you have in CI, right? Hardware in the loop. Do you know what hardware in the
loop is? Is it fun? Okay, pretty much. Everyone, so for those that don't know what hardware in the
loop is. Imagine that you have this Raspberry Pi, and you connect it to a laptop, you run the simulation
here, but the full software stack is in the Raspberry Pi, so you can run simulations against it,
and you don't have to have this all-interview full of people to actually check that it can avoid
a crash with them. Ground, aerial, vehicle, there are many robotic systems are super
indigenous. Identifying your use case is super important. If you have robotic arms, the type of
simulation that you need is completely different to the ones that you have on mobile robotics. For
example, there are other physical features that you need to account for. When talking all the scenarios
and thinking of the scenes, you have indoor and outdoor or mixed environments, which will affect
differently the sensors and actuators there, you may have one of multiple robots, and that's a
completely different thing. Imagine that you want to make tests on a bit of 50 robots. You don't
care how much one will perhaps is turning on each prototype. Maybe you want to see if the system
as a whole is behaving as you expected, right? So it's a completely different thing. Do you have
automatisms? Do you know the different videos between an automatism and an autonomous system?
Raise your hands. Okay, go into that. An automatism will decay. The deterministic elite
an autonomous system may be exposed to new situations and behave differently, or it's not such
deterministic system. So in case of an automatism, you may lower a bit, there are on the simulation
engine required for that because the number of constraints to the system is much bigger. You know
how to control the variables, so you may not need as many requirements on the simulation engine.
Do a scenarios last long. You have simulations that are running for hours,
pace, or is it just a few seconds that you need to actually validate that your system is behaving as
expected. The unique determinism that goes back to the previous question,
sing creation tools. This is a super interesting topic. So imagine the robot, the aroma that we have
there, or the vacuum cleaner, you know, any specific robot, is it required to actually vacuum clean
every single living room on planet Earth? So you need to actually account for different geometries.
If you have a carpet, if you have a wooden floor, that requires some kind of random
generation of homes to actually validate that your system can behave properly on all those different
scenarios. And did you doubt that it unlikely? Maybe you have a super nice coach that you want to
actually detect, or you have, I don't know, a set of spare cables on the floor, and you don't want
your vacuum cleaner to go over it. You need to detect that, and that's certainly not a tool,
or sorry, a task for a roboticist, maybe it's something that you need to get away to a graphic designer.
And the list keeps growing to be honest, this is a never-ending list. So if we get into the physical
requirements, it's kinematic simulation enough for your use case. Are you going to be able to speed up
your simulation by doing that, or do you actually care about dynamics of the system? Sometimes you
need to deal with complex dynamic systems, and there's more pressure on the physics engine.
Since there are many models for different types, maybe some are simplified, maybe you need to
get more resolution and imagine that you have a rotary lighter, and you care about how the point
cloud that you generate moves with your robots, so actually it's not static. That's something
really hard to get, and you need to get a super custom model for it. It's CPU power enough,
maybe you have to delegate that to the GPU, and that's not simple. More on that later on, probably.
Rendering requirements, do you need photo-rallism? Maybe you don't care about photo-rallism?
Why are we using a super expensive simulator for that? Then, mind-tenance model. So when getting
up to this point, you think, okay, I have two or three options out there, but I'm going to rely
on the community to get upgrades or fixes on my system. It's my system right now. I don't know.
I'm going to 18, and my simulator needs to run and I want to 24, who's going to do the
party with it. That's a lot of burden on the development teams to take care of, and you have to
think ahead of that. Otherwise, it's going to be a problem. So what did we do? We got a lot of
these analysis many times, right? Because on every new custom run, every new project, we have
two answer all these questions, and doing that for all the option available out there, what's
a really complicated task. So we kind of model, and you know, which has this structure, as I
mentioned before, and it's composed with a 2D LiDAR, actually the camera, IAMU, actuators,
which are models, motors, wheeling coders, frames, battery, etc. This is pretty much what every
more robot out there has, probably you may have multiple cameras on it, or maybe you have a 3D
LiDAR, maybe you have multiple LiAMU, maybe you lack of wheeling coders or it happens.
This is a Diff drive robot. You may have an acarmonic configuration, you may have a tricycle,
a non-nidrive, but still it's one configuration that helps you to benchmark, but actually to evaluate
different simulators. Super simple, everyone understands this, and then we model all those things
for all of these simulators. So we got a COW without so-vide, Mujogo, and Flatland. There are many
others, probably not suitable for this kind of conference, there may not be open source, so we have
leading those outside of discussion, but it's been super useful to actually understand what
can we do with each one. I'm not going to get into the details of everyone, but all these things are
in public repositories, I have only videos, I want to emphasize some strengths of each of those,
and how we use them. You can scan now, you will be directed to a link free, a good manual
way, if you're organization, you will find all the Andino underscored something there, and you will
find other projects. It's important that you also start them. I'm looking at you, so. All right,
so, if you download, let's start with Casivo.
There we are. How it's very just in case. So we have over there, Casivo for this simulation,
it's in our HQ office, and it's running to land on it, so it will be mapping the entire office.
This simulation is running four times, if I remember correctly, and real time, it's a super small
robot like this, and it's mapping a big area. Casivo is a general purpose simulation,
simulator, sorry. Right now, it's using Dart as it's physics engine and ORE2, which comes from
the gaming, perfect thing here. The gaming industry, as a rendering engine, the cool thing
out is that Casivo has a super modular structure, so you need to change for example the physical engine,
there's another one that is much faster right now that you can use. It's not fully implementing
in terms of all the features that you might get, but you might speed up simulations that don't
require such a detail on the physics engine. It might be the full choice, and not a bad one,
per se, when you are starting a simulation project, but sometimes in terms of rendering, it falls short
to be honest. We may need other things. Cas, a lot of examples, a lot of sensor support, so if
much in that you want to have a segmentation model, a ground truth, a segmentation model, you can just
use the one that is out there, you don't have to implement something at all to it, and that happens
with many other things. It works with a aerial robotic model, as well as with robotic arms and
wippers, so it's a great choice. Let's see if we can use another one. So the other one that I wanted
to present is wewots, let's see if it loads, otherwise I'm going to switch here, now it's not working.
So, wewots. Now I just mine now. There we are. So, wewots is solved for a general purpose simulation
engine. It has a monolithic structure with respect to a sieve, it's not full-blown of components,
so it might be better if you are looking for something that is lighter. It has a lot of
planes as well, and support for different types of robots. It's been used also in the AV industry,
and it's a great tool. In this case, we are only celebrating the robot. So on the right side,
we can see all the information that we are extracting from the simulation engine, and we also have
on the left side, the camera feed. It's relying on ODE for the physics engine, and open GL for the
brandening. It's great. It has a lot of bindings, so you don't have to be proficient in c++,
you can use other languages to use it, come into the AV and so on. It's simple, a great choice as
well. So now, I'm going to switch to this one and speak briefly because we have an R section
with Jan. How about this is ODE, short video, great graphics. We can get everything will learn more
about it later on in the next stop. This is in Java. I don't know if you know it, but this is great
for gathering, like very fine rain, physical data, like you can get a lot of different models,
integrators, physics engines, and so on. We are using it within our R and D team, so as to tune
the process of getting a great model, great physical model of a robot, and then try to aid
the development with that, get some structure on how to do it. It's out there as well,
different use case, not great graphics, certainly. And this is flat land. I really like this video.
So in this one, we get a lot of robots moving around. This is using a 2D physical engine.
We are running about 20 robots in here. Of course, we are not getting the camera,
rendering for them, but we can achieve RTFs real time factors up to 20 even more with hundreds
of agents. So this is great to look at other use cases. So with that, this is the A4.
I think I finished. Thank you very much. I'm not sure if you can.
Question.
Hello. Thank you very nice for speech.
I wonder, how do you handle misalignments between the simulation and the real world?
In general. What are your techniques? Misalignments between?
Between simulation and real world. Great question. So if it depends.
So the question was, how do we handle misalignments between the simulation and the real robot?
It depends on what you want to actually do with it. Sometimes you don't care about
the misalignments and it's more expensive to actually try to develop something that matches real life.
So we first analyze what we want to do. That's the first thing. Sometimes you don't care.
When you care, you go back to the list of questions and we see a limitation on the simulation
or the physics extension. Maybe the problem is that the simulator is suffering you and not what
the physics extension is actually offering. So you need to go over that gap. Maybe you are not doing
a proper model, offering like let's say you have got zero, right? In a zero you have a
description model, which is SDS, a specific format and you cannot account for all that.
So in those cases what you end up doing is changing the simulator to cover that case.
We have done regressions. So what do you want to do?