I started here, because I have a lot to cover, also unfortunately couldn't get video working
on my laptop, and my robot is up on tail scale, so I'll do the demo after for people who
are interested, I'm really sorry about that, we tried a bunch of times earlier, so let's
get started, so this is my robot, these are some of my other robots, and I also actually
get paid to work on other people's robots, which is just like the best job ever, like
far, I work on big industrial arms, agricultural robots, spot, spacecraft, hotel robots,
it's just, it's great, but let's go back to this robot, this is a hacker bot, it's pretty
spy, it's a small startup in Colorado, called Hacker Bot Industries, the founder is a colleague
of mine in Bernstein, you might remember, no him, from being a founder of the Sphere Robot
all, I don't even remember that, or the Misty Robot, he likes doing social robots, and so he came
about two years ago, we met up at a conference in Boulder, and he's like, I think you
know the social robot, and become like a turtle bot kind of thing, and, you know, the
expressive, and, you know, told me about it, I was like, sure, you know, he's an industrial
crowd funder, and I really, sure, you can't get turtle lots anymore, they're not making
the create three, you know, there's just not a, like a cheap hackable base out there,
everything other robots have gotten to be like two to four thousand, like four or five
thousand dollars, I love the Leo tech guys, and they raised for their latest version, they
raised the price from like $255,000, I was like, okay, great, a cheap robot base, yeah,
sure, signed me up, and then he mentioned that it was going to be like basically a Ruben
off, he kind of kind of just company in China that I was making a Ruben off off that had
a built-in wide R, and it is all integrated into the two-year system, and you just push
button and it wakes up, the first thing it does is rolls around your house and it maps out
the space, you before it goes on clean, and then you can, and that says the map internally,
and I was like, okay, I have to have one of these, you're going to go off and draw this
AI stuff, it's going to have an arm, it's going to be expressive, and I just want it from
the mapping, I mean, if I could, like, just get it from that and figure out how to get
the map out of there, I'd sign me up, and so he helped me out a lot with like, getting
the commands, figuring out the commands and stuff for it, and this is basically what it does,
this is my goal was a one-touch mapping utility robot so that I could map a space and not
have to, like, follow around with a joystick and one hand and tell me up and a laptop and
the other, and I also spend time here. He didn't get tired and you're following it out
of Wi-Fi, and it's just the idea of having this thing, you push button and it goes around
and it does it for you, and then it's usable for other robots, like your roster robots,
I have a lot of really, really wanted to do that. So the thing here is to robot generates
its own onboard map, it auto colors, figures out, it has some sort of algorithm inside for
figuring out where what it think a room is, and then it stores that as a bit map and it's
own custom format, and then you have to have your phone and you get this app called TUIA,
which I guess is like if you're not doing Siri and you're not doing some other home environment
things, you use TUIA, because they control, like they're also trying to, and they have like air
conditioners and rice cookers and window shades, just like everything is hooked into that.
And then you can also, it turns out that there is a plug-in for homocystent, there is like
an integration with TUIA, but I looked at that and it turns out that they're using homocystent
is using the TUIA SDK, which is great except that they want like a lot of money to buy a
license for the TUIA SDK, and then homocystent is also its own lifestyle, which I appreciate,
but I just kind of wanted to get the map off the robot. So that invites some hacking, like
all good open source or open-ish hacking robots, there's a Siri report. Ian has added a
TUIA, not a TUIA, but a TUIA, armed board from Adafruit, on to it, to make the Siri lane facing
a little bit easier, so you don't have to play with the actual wires, but I've done that before,
and basically that you talk to it is you send it hex commands and it spits out data about
like all the bumpers activated, what is the battery status. And this also turns out that there was
a map, a command that they did not document, that was also like displayed map, and this is how
the map gets from the robot, from the flash on the robot over to TUIA. It's all done over wireless,
so we figured that out, and then let's go back one, that led to realizing you can get this map.
This is my apartment, San Francisco, it's a Victorian, so if you know anything, it's basically
it's a single long hallway with rooms off the side of it, and so it looks a little weird,
but that's really like the top part there is entrance to it on the hallway, and then then there's
like a main room, and then there's kitchen, and so that's the map. So we got the map as a bite stream,
off the serial port, and start looking for headers, start looking for repeating patterns.
I actually did some more on the XV11, the old Nito, a wide R for controlling the wide R, and
had it dump, just from, again, serial and interpreting hex codes, got it to dump out a wide R data.
So I was an entirely unused to this, it's kind of a pain, but so we found the header,
and then realized that it is an occupancy grid, because all the thing is doing it with the map is that
it's cleaning, and so it just needs to know where the obstacles, where the boundaries and things are.
So it's actually pretty simple, but converted to gray scale, and we still have, so here's
like the interesting part, or getting into the interesting part.
We're able to convert some gray scale, but it's still encoding some colors. The white part,
the white region is actually part of the image, native coming off of the, coming off the robot,
which is going to be our problem. So it encodes the color for user convenience, but actually when
it goes around and navigates, it doesn't have any concept of events, it's like you're just supposed
to on the phone point, but it doesn't actually really matter operationally.
And this is a little bit different from Ross, where it's just a single layer, and everything in
the single layer, it represents everything. Ross abstracts things like rooms and boundaries,
and user preferences to other layers. You have a global cost map, here at the local cost map,
then you have an inflation layer, then you have, you know, you can set waypoints, and you can
customize it. So first thing I realized is that because Ross encodes white as open to
explore and open, the boundary, just where the map is, is the robot would consider that open.
Ross or all, consider that open. So the entire image is just this open space, and
this is where the conversion starts. I have this media idea, hey, open CV. Open CV detects
edges, it has some, you know, has like camion fine contours, and a, you know, fine poly and a bunch
of other things. And well, as I say, you know, now you have two problems or end problems.
This is not going to be an easy solution, because I realized really pretty quickly, because
candy edge detection is really, it's great, it tries to find an edge for everything. It tries
to find a line or an edge for everything, even the tiniest little feature it thinks is an edge.
And what do you get with light R? You get a bunch of, if you've ever worked with Ross and with
light R, you get a bunch of little spikes, a bunch of little edges just natively. And it turns out
that's actually not a Ross thing, that's a light R thing, because the rays point through and they
all get, oh, hey, maybe that is some area that is unexplored. Maybe that is some area, you know,
you don't want to just throw away things, you take in everything, and hold on to it for later,
and maybe you'll explore that area, or maybe it'll just be this. So that's really just hard to
filter out with candy, you know, if doing a lot of carpet fitting, a lot of tuning, and it still
just didn't work properly, and you're creating boundaries, where there weren't any before,
and I tried a couple other things, like, you know, finding a kernel, to do
finding several kernel operations, and using some of those functions in OpenCV. That didn't
really work either, because it wants to apply to the whole image or to the whole subsection. Now
I'm going to define subsection, so it's just epicycles upon epicycles. So turns out,
if OpenCV says no, go ask NumPy. And if you're familiar with NumPy, and OpenCV are just like this,
they're just like the bestest buddies. And it turns out that to NumPy, an image is just an array shape
of, you know, a 2 by 3 by N matrix, and so anything that you could do with like machine learning,
you know, knowing that work model, training data, where you still have to be out your way
and you're selecting out your parameters and things like that. You can do the same thing with an
image. It just doesn't care. It just shows up as it takes a bit map. It's just an array of data.
So I went poking around in the API, and I found this, NumPy.org, which selects out anything that is a
non-zero value. And that turned out to be one of the answers because you select out, if we go back.
Here, to the gray, now you can just, I can select out where all the boundaries are, and
get that as a separate array and start marching through that. So apply a 6-way kernel
where you're looking at, like, for an edge, you're looking at the left to the right,
up, down, and down the diagonals, and just come into march around the entire inside of the image
and find the boundaries. Get to this. And you get the same map, but now it has black boundary
around it. And now that we've done that, you could out the white, you can invert the colors
because, and just kind of do with it, whatever you need. And so at this point, we've
downloaded the byte stream, converted it to binary, which gives you a bit map. We've flushed
all of the user-defined color regions into a single region, and I've got a border. And so now this
is ready to make available as a cross-cost map. And I found that also, like, this worked well enough,
that actually cleaned up the apartment a bit. And I got a lot more free space, and so now you can
actually through the rooms better, and you can see some artifacts. And so, you know, that's another thing
is, you know, inspired to clean up my room because they actually, this is it. So that is,
buying the demo, I can take questions because this is pretty much about it. This is
up on GitHub for people to look at and play around with. I am actually actually uses to map out
my co-working space in Oakland, California. I work at, I have a kind of appointment there,
as a robot assistant residence at a hardware incubator call, a circuit launch. So it's all up on
up on my GitHub, and then there's the hacker bot industries linked to her if you, they're,
he's still selling hacker bots. And I'm currently also looking for work, always, you know,
contracting life, always looking for interesting projects, interesting companies to help with
their robots with. And come find me, I have cards if you're interested in that. And then also
wanted to pass on some thanks to the Phospham for having me here. I circuit launch family in Oakland
for giving me the space to work on robots and access to tools and 3D printers and the kind of things
that let me build my robots. And then the home robotics club, the Silicon Valley, who I've been a
member of for just like years and years, they got me started on convince me that I could build robots.
I was only a software guy at one point, and then also the character CVC, CV2, R, O, and S,
and also people like you for, I mean, listening to me. And I think with that, I have a few minutes for questions.
Thank you. Definitely come see me afterward if you're interested in looking through the
code or if you want to see it, it actually work as a demo. Hi, yes. It's listening, does it work? Yes.
You know, I've got it. I've got it on there. Yeah, but sure, it's, you know, it's a typical
San Francisco. I don't know if you're familiar with the Victorian style, where it's a really,
really super long hallway. Yeah, it's, it's that. And it's a little bit of a mess, but, you know,
that's where I don't remember a lot. Anyone else? So you also used the Velodine. So did you also
prepare the accuracy between them? I'm sorry, what? You also showed that you used the Velodine
in your apartment? Did you like a variety of accuracy? Yeah, it's, let me go back to that slide here.
So you can see that the Velodine, I mean, it's just like, it's an amazing piece of tech. It's like,
it takes about like maybe 15 or 20 minutes for the hackerbot with the consumer-wide R,
which is like a, you know, like a slick and harpsied R, kind of a lane. It's for it to roll around
and to map that. The Velodine just did this in like five minutes, because the distance, I mean,
this is why it shows up on autonomous vehicles and AMRs and things like that. It just shoots down,
like, I had it like starting here, and it just shot down at the end of the hallway. And so,
didn't, like, five minutes. And you can see the, I mean, I can compare like accuracy in terms of,
like, you know, some meters or, you know, but I noticed that it just, I got much cleaner, just the
plain map. I got much cleaner lines. There's much, it's just much higher resolution, like,
per degree turn. And it just saw everything. I mean, like those raised down there at the bottom,
that's into the back of the laundry room. It saw that, like, it just comes across the, you know,
the second barrier into the big room. And it just saw everything. So it was almost like,
no point in, you really do much more with it, because it's like, okay, I have not been five minutes.
I actually ended up moving, that was for a client project and actually moving it over to my co-working
space, because it turned out my apartment was just like too small. And it wasn't useful for
the kind of data that we were doing or trying to, kind of, data we were trying to get.
Hi, Hugh. Oh, that's fun. Thank you.
My question is, so you, you said that the hacker votes has a shock wrench later, and it's
moved to be able to, to get everything in range, let's say. Right. Do you know how the robots
look at themselves? Um, I think it has, like, it has, like, an onboard IMU, it has the wheeling
co-orders. Um, I, it wants to, like, one of the things that it does after it's done the initial
mapping is that it can, it seems to consult the map that it just made, because they ask you to, like,
consumer device, so they say, hey, put it right here and push the button and say,
want you to start on the base. So it has a little, it has a little doc. Um, so it uses the map that
it just made, and it re-homes itself towards the base, and then it does kind of like the classic
room of docking where there's some sort of an IR, you know, and then it docs itself. And then
it either charges, or if you push the button again, then it goes out and actually, it realizes
it has a map, and it goes out and cleans. Um, so I, so just like my experience happened, like,
tapped into the, um, the serial port while I was doing all of this. Um, but my, my guess is that,
um, it is localized, it knows kind of where it started from, and then it just localizes back to
that, and then it finds the base, stores that somewhere in memory. Um, and then it, it runs with that,
because once it finds the, once the map is done, it finds the base pretty easily, um, just again.
So I think there's like, it makes the map, and then it's kind of like relocalizing sort of,
on the spot. Um, it's all, um, kind of like when, you know, it's a little hard.
Yes, thanks, but yeah, uh, for me, it's, uh, quite impressive that it manages to stitch correctly
all the later information, uh, because for example, on carpet, if you have carpet on the floor,
yeah, it's, it's, um, I, I, I don't want to say other than, um, that's kind of China.
I mean, that's, that's the level of hardware and, and devices that they, they produce, um,
don't ask me about the actual soft, you know, like the user software, but the firmware, um,
that's about average, uh, for devices that I've seen, uh, from them for, like, for that,
for some of the arms, um, the mechanisms and the control, you know, the onboard control firmware,
is actually really quite impressive. Um, and it's like, this is why it's harder to be. It's a consumer product.
It was for 150 dollars, you asked. Um, it's just, why would you be even predicate with that
in the marketplace, um, and just does it? Thanks.