Hello everyone.
I hope you can hear me.
I have this low voice today.
It's me.
I'm Anna.
Then there is two hours.
And there is another person who couldn't make it to Brussels today.
And we have started the web developed marcha.
We call it the open source capture that improves open street map.
And to start speaking about marcha.
I want to start from when it actually was born.
So marcha was born here, more or less.
Not exactly here, but like a Nairobi at the state of the map last September,
where I was presenting my academic research about a tool,
which is developed by HOT, the humanitarian pursuit map team.
This is the website, the school is called Fair.
And this is a tool that allows you to basically
the call it AI assistant mapping tool.
And at the moment it allows you to basically identify buildings from satellites.
And the way it works is that there is a pre-trained model,
which can find tune on a specific city using data,
which makes available for a website, which is called Openeram app.
But I'm showing you in general how it works for this is a computer vision task
of images segmentation.
So the labels come from a street map.
They are in a Victoria format.
And they need to be transformed to a binary mask,
where you have other buildings on buildings basically.
So this is a raster, which is compatible with the ground through the data,
which is from the website that was mentioned before.
Then you go through the machine learning model,
while the training validation takes place.
You build up a checkpoint place,
where with which you can run prediction on another area of a city.
Okay?
So this can find tune up a pre-trained model on whichever city you want,
as long as the RGB imagery is available.
And the data from up a street map I show how it works.
So this is an image from a panorama from a place in Russia.
Yeah, this is the website that was mentioned before,
which actually now is done if you try to make it work,
I mean, to do it with them work, but anyways,
that's where the images come from.
This is the area of interest, the training area.
You fetch the data from up a street map.
And if the data is not available there,
you can actually map it yourself,
which I think is great, because you don't label it just for you on your computer.
And it goes straight to a street map, what you do for the labels.
Anyway, so after you run your training,
I've done this for 25 cities for my research.
Okay?
So that's the performance of fair,
because when you run the training,
you get your model for your city.
This is a, sorry.
This is, this is in Kakuma,
in, this is actually refugee company in Kenya.
And when you run the prediction,
you can get fairly good results somewhere,
as well as it works worse.
Okay?
So yeah, for example, these buildings are not great.
So in the audience, like, like you,
there was someone that thought,
wait, this could be a good idea for something.
Okay? I tell you.
So the way it works is that this prediction
doesn't go straight to a street map,
but then it's again feedback.
So what you can do,
this is the botanier you don't see,
which you leave a feedback.
Okay?
So the guy who was watching in the crowd was saying,
like, what if we try to get this feedback
on the AI prediction from a wider audience,
so we can help port an open street map.
So that's the idea of the map,
plus,
while doing that,
what if you create a tool that helps block the bots,
so a capture,
but without improving proprietary maps and software,
or exposing user information to third parties,
so that's the lock part.
And then if you add the matcha t,
so you get the map-cha logo.
Okay? So that's the idea.
The person behind the idea and the name is Giam.
I have connected the two of them
and plucked everything,
so that then Stuart arrived and developed actually the tool.
So that's the few of us.
What are we trying to do?
We ask ourselves, can we build something
that can reliable reliably,
that the difference between our human and the bot,
because that's what the capture does.
Because you are allowed and not allowed into our website,
with the capture.
And can we actually get that a data for us?
Because in those data,
we don't want to put them straight to a specific map.
And so also,
we got good validation data to improve the models themselves.
So yeah, we have Stuart.
It's going to speak about what is developed.
Let me see it.
Yeah.
Great, thanks.
Yeah, so basically what we wanted to do was test this idea.
So where we are with the project is not like a live capture,
that you can install on your website,
we're kind of prototyping and everything on the idea
to see if we can get something that works,
and that we can then, and in the first instance,
use an upper sheet map,
and then add a wider context as well.
So this first iteration of the platform,
you can try it out here.
If everybody wants to give it a go,
it'll give us more data, which is always good.
But basically you land on the website,
it gives you a little bit of a brief introduction.
And then we have actually two different interfaces.
So when we were trying to figure out how to
make this fast and fun and accessible,
we wanted to try a couple of different modes of interaction.
So the first, the second one is the grid one,
and that's kind of more of a traditional kind of capture,
like you would see on Google,
where you see a bunch of images with the buildings
outlined, the outputs of the AI model,
and you're asked basically which ones of these look correct,
which ones look like the AI has properly outlined the building.
And what that allows us to do is,
as to quickly identify places where the AI is gone wrong,
or places where it's completely messed what's there in the first place.
And the second interface is the swipe interface,
which is a little bit more like a dating app,
I won't mention which one,
but it's a quicker interaction where you see a larger version of the image,
and you just kind of swipe left or right to tell us
whether or not you think that the AI has got the answer correct.
And the idea is that we show a mix of images on here,
we have show images that we knew the answer for already,
to verify if somebody is human or not.
The data that's being generated from the AI model is already wrong,
so we think that it's a good candidate for a reading out bot,
so that we need to test that idea.
And then we also show images that we don't know the answer to,
and so we gradually collect multiple people's opinions
or whether that image was correct or not,
even though we don't know the answer,
and then that helps us with the models and retraining.
So they kind of work like this, the swipe interface has buttons at the bottom,
where you can click on them to very quickly,
just say whether one's right or not,
if you don't want to swipe,
or you can click on the main image,
or use your finger in the main image and swipe left and right,
to kind of just tell us whether it's correct or not, which is fun.
The second interface is the grid one,
which I get said is a little bit more traditional capture,
where you just like all the images you think are right,
and then you get another set of images to have a look through.
So the straight off between these two,
the swipe interface gives you a larger image,
so you can see more detail,
if the grid one will get through more images over time,
so it's a little bit faster.
And we wanted to test out how well these worked.
So when you land in the website,
as I'm sure some of you are doing in the room right now,
you'll get randomly assigned either the swipe interface or the grid interface,
so we can do a little bit of an eB latency,
which one is better, which one works in different contexts.
The entire thing is built on open source software,
so the interface itself is just web components,
which if you haven't played with them,
or are really good alternative to things like React,
or spell, or Angular,
they're just built into the browser,
and give you components to build interesting things really quickly,
and they're a browser standard,
so they kind of work everywhere without too much hassle.
And then the backend,
we just set up a little super base database,
which is open source alternative to Firebase,
to kind of just gradually grab the data and store it.
So once we've come to build the fill version of this,
it'll be using a more robust backend,
that'll have callbacks for logging people in,
and things like that,
but for now we're just collecting basic data,
which is a very simple database in the backend.
And so we'll be running this for about three weeks.
We launched it on half January.
Yeah.
We launched it,
initially, and what you say,
we're among friends,
and then I posted a must-do on,
and we got a big skyrocketed with a number of people who tried it.
So we got,
kind of, yeah, seven of people who used it,
and of these,
we got to one of the people who run the survey after.
We could run it filled up,
and we started from 3098 images,
which is what we have in the database,
and then we got more than 20,000 clicks.
So we got all these data.
How do the images look like,
and try to make it clear?
So these data,
where we have already labeled data,
so we can know if persons are correct or not.
So in case,
so this is the case where the building should have been predicted,
and it has been predicted, because we have labels there.
Yeah, we don't have labels, but it has been predicted.
Yeah, it should have been predicted, but it has not.
Basically, so we have labels here,
and yeah, it's a true negative,
so it's a true force, positive and negative.
So yeah, there's not a big index,
and there should be buildings, which is okay.
So when we asked the question,
is there a shape correctly,
which is the question that we put in the map chart,
and it was very hard to find a good question.
If you have an input for that, please come up with something,
because we're never happy with any of the versions that we try.
So when we asked this question,
we expect people to reply like that.
Yes, no, no, yes, for all these cases, okay.
So going through it, so eventually,
so this is the amount of images we had,
mainly true positive,
for positive force negatives.
And this is how the people responded,
like this is what we were expecting them to reply.
And you see that with the force negatives,
we have really little people who agree with the image,
somehow, then we go through that, if we have time.
And yeah, and mainly,
they agree with the true negatives and for positive.
But the point for us in the,
like it on the SNF people are humans or not,
about how much they agree with the image,
with the labels themselves,
but how much people agree among themselves.
So because they are humans,
and they might be more crack than the labels, right.
So anyway, I'm showing you where most of the people agree,
here, this is like 100% of people agree that this
is where correct or false, okay.
And this is where most of the people didn't agree,
like maybe just 1% of the people agree that this was a building somehow.
Yeah, I just showing you what it is, and even just
them, that's what happens, came out from the test.
Then Stuart, that's what he has to say here.
Yeah, so what Anna was just showing was kind of like the overall results.
So for each one of those categories that were interested in identifying
on average how many people got it right.
But we also can look at the distribution of how well
somebody did an image.
So basically what we're showing here is if we take each image,
that image will be shown to roughly about 100 people.
Or so, I'm given the numbers that we have.
And we can then look at the fraction of people who said the right thing,
given what we know about the label.
And so what you see along the bottom here is the fraction of users,
and then the fraction of the images on the histogram as well.
So essentially, the further to the right here we are on this graph,
the more images, the more people agree to the images,
and the further to the left, the less the agreed on images.
So we can see here in the sniping interface,
there's kind of a peak around here where about 60%
70% of users agreed on the label for the image,
and they got that correct.
And so what that means is if we take images from these buckets on the histogram,
those are places where we know the image was relatively easy to classify.
And so we can use that for human verification,
or we can use that to get information on the site.
It's interesting to see how these compare between the sniping interface
and the grid interface.
Well, and this is, and probably the one we're working.
Oh, here we go.
There we go.
There.
And the grid interface, the false positives.
People got them almost entirely correct.
Like they got them to the point where we,
everything shoved out against the right hand side,
and it's like basically almost like perfect.
But the true positives have got this much longer tail here.
And so really what we're trying to understand is like the differences between the interfaces,
and what they lead to in terms of trade-offs between the swiping interface and the grid interface.
And this might be due to the fact that the swiping interface is much larger screen size,
so you can see more in the image.
But in the grid interface, you see more images alongside each other,
so you've got some comparison points.
So you can sort of maybe tell the difference between them.
So we're still analyzing this data, we're trying to figure it out.
But the important thing is that we have a bunch of images,
and these context that we can use then for further follow-up,
and also do the kind of like the verification step.
We also had a survey.
If you try this out, you'll see it that kicks in after about 35 images of being classified,
which is about three or four pages of the grid,
or about 35 swipes in the swiping interface.
And we ask people kind of a number of different questions about who they are,
so whether or not they've done belly identification before,
on an open sheet map or somewhere else,
and it turns out that we have a fairly large,
strongly agreed there and agreed.
So we've got quite a specialized audience,
so we're reaching a biased audience through the Master Dawn,
through the places that we put out there in friends,
which makes sense, but it's also not necessarily that bad of a biased audience,
simply because we want to use this first all-no-push sheet map
to like walk into different things there.
So this is actually quite close to the audience.
It's going to be the first user audience for this, which is great.
People say, we ask people if they could easily identify the types of buildings,
and we got some strongly agree, a good chunk of agree,
and then there's kind of a mix of the end here.
There's a hard task.
We know that people find this hard to do it,
and we're going to iterate on the question we ask,
and the kind of things we're asking in there as well.
People ask if they would like more or less in the imagery.
This is on surprising that on the swiping interface,
very few people said they wanted more of them then,
whereas on the grid interface, most people said they wanted more of them then.
So we're going to look to see how we can make the grid interface a little bit clearer,
and then a benefit more of instructions and find this cool.
We were very reassured that people found this cool.
So we're happy to say that people enjoy doing this,
and it's a good idea, which is very nice.
So I'll pass over to Tyler for some of the other feedback.
Yeah, yeah.
So yeah, I was saying we posted this on MasterDone,
on the 15th of January,
and we got several comments there.
Maybe easy to access to this and to the survey.
Yeah, so it was mainly about accessibility.
We just read out the line.
Probably it was, we could do something else.
So somebody was suggesting to put a kind of,
not to fill up, not only the outline,
but to fill up the building as well, the rectangle, whatever.
So people were saying that in terms of accessibility,
we should take into account people with visual impairment,
and that was an idea well as well about translating to other languages,
or give the possibility at least.
So for someone, so I write a left was not clear.
If the direction was very come from where it goes to,
and then, yeah, and the tiles with no outline got a big,
like logical question, like if you ask me,
if there is a, the right outline,
but there's no red right, but what am I doing here,
but for us, because of the way we built it up,
with the true negative for us,
it was a lot of data more.
So once we explained it in the comments,
I think, understood, because they did it correctly,
and then we can add instructions,
which is something we thought about too.
The survey, many people didn't get to it,
and yeah, the zoom level.
People were asking, in the sort of,
we just made it worth in general,
we'd like to change the zoom level,
and people asked, yeah, in which sense,
but yeah, for us, it was obvious like the zoom more in,
but yeah, that's it.
Again, yeah, future work.
There is a lot to do.
I mean, this is just an initial trial.
We would like to improve the images,
first of all, because of the data we had was limited,
and we only had the, always the same tiles,
we could just switch the buildings that were selected in there.
But the idea for us would be to have the,
the image centered on the building,
and also improve the outlines, as we were seen before.
We should introduce the unknowns,
because yeah, that's what we were seen before.
We already have label data, so we can understand
if people agree or not with what we already have,
and eventually we can get new information on the images,
which are not labeled yet,
so basically we're going to show our mix of all these two categories.
Then we'll add instructions, as we said,
and we'll have to have a good thing on how to integrate
the validative buildings into the wider opposite mountata set,
we're thinking of the other two inputs into,
input the results into the map rule add,
of, you know, well, we have no battery,
and also need back to self training.
Like I, ideally, if we have,
there results from people, we can,
where we have my confidence,
we can send it back to for training on the same model,
so that we get better checkpoints,
and we have better prediction,
good, ideally, forever, right?
Yeah, then we should also add a skipper,
a lot of new buttons as well,
which is also in so many captures,
and the translation,
and there also just accessibility
that there was mentioning before.
You want to see something else?
Okay, so that's it, I think.
Thank you.
I mean, again, I guess just to say,
if MD wants to collaborate with us on this,
on the data site, on the eye site,
or on the, this up to itself,
on the coding site,
please feel free to reach out.
We'd love to have more cards of users,
and more people getting involved.
Yes.
Yeah.
Good question.
Yes, sir.
I think your height is data set,
to make sure you find buildings.
So, is the question,
like our word?
Ah, yeah, okay.
The question is, yeah, so the question is,
are we using any height information
when we're identifying the buildings
through lighter and things of that?
Currently, we're not.
This is, we think about this as satellite data,
but it's often not,
it's often drawn slaying over an area
with cameras and looking down.
And for areas where,
the humanitarian,
which street map is kind of like,
needs often you're just getting fairly straight
forward RGB images of the ground.
So, in that context,
we're not using the hay information or lighter,
which is quite expensive and hard to gather.
Yeah, yeah.
Yeah, just so out on these,
how to want to improve the model
to identify not only buildings,
but other features.
And there is a project in the future
about, which should be starting
about using street-level imagery,
which should be integrated as well.
So, yeah, there's already a deal.
Using it in general,
image segmentation tool for other stuff too.
But not lighter, I think,
would be harder to put it together.
Hmm.
Any other questions?
I think we've probably got a lot
about the actual time event?
Yeah, okay.
No, it's also fine.
You're all just busy playing the thing, right?
Yeah.
Yeah, please.
Okay.
So, you can just come to this
to find out about somebody that captures
not just different types of buildings,
and type them, like, more kind of awesome
to do.
Maybe this might actually also just be more
like sensible to type, so like,
something that's different, that's not what I'm going to do.
Yeah.
I mean, we'd like to think, so, I think it is.
So, the question is,
have we compared the other captures
and kind of the usability of it,
compared to some of the other ones out there?
Like, the crazy text with,
on the top of which is barely readable or things like that.
And we haven't done that yet.
We'd love to do some user research for that,
which would be really cool.
Do you want to do it?
I've been no extra.
But I think we probably want to,
I think one of the interesting things that
captures is the more popular they get,
the more people are going to try to break them.
And so I think almost like there's a nice idea here
that we keep it will key and for OSM and a few other websites.
And build it up gradually that way,
rather than trying to make this the replacement
for all Google catches out there in the world.
So I don't think I don't work trying to supplant the,
you know, click all the more banks
or click all the crosswalks catches of the world.
We're just trying to create something
that's more appropriate for this context.
And for the OSM community,
means that they're not having to rely on proprietary services
like Google's capture system.
I mean, some of the new captures are not even clicking on things.
They're just click the button.
And apparently the way the work is just the way that you're
mouse moves as it goes towards the button,
tells whether or not you're a person or not.
So bots will just go in a straight line
or kind of do some convoluted thing,
but the random motions of your hand
is you're moving towards the thing,
everything off to tell whether you're not your human.
So it's not even clear that these telecaptors
are necessary anymore, but it's still kind of an interesting thing.
And we can gather good data from it as well,
which is the whole point.
We wouldn't be doing this if it wasn't for the fact
that it could be used to improve models,
which is the ultimate goal.
Cool.
It's going to be extensive work,
so that we can use it all over the world.
I thought, like, not just, just really got nice,
but, yeah, it's a great question.
Sorry, yeah.
Yeah, so the question was, is this going to be
extensible to be used with other types of data?
And all the types of images are kind of tasks?
Yeah, I think so.
I think what's we've got to up and running.
It's fairly easy just to substitute
and the images that are being shown.
And then on the back end, give the label that you expect
for the ones that you know about already to test this out.
So it should be very easy to add another things there
and use the interfaces.
It's all fairly, the software itself
is not particularly complicated.
The thing that's going to be complicatedly
around is us doing the integration with login systems.
But that's for a future, to worry about.
But yeah, no, for sure, the kind of interface
and the data should be extensible.
And we've talked a little bit about other ways.
Other types of data we may want to put in there ourselves.
Yeah.
Is there any much I'd like to do just one type of data
that I want to put the website of, does it?
Do you have lines on each of the next user
that can be, in terms of, what's it, what does it, what's it?
Yeah, so right now, I think, so the question is,
are we planning on doing different data types
than the current version?
Is that a question?
Is it going to be just a one type of application?
Right.
Is it going to be possible for weeks?
Yeah, okay.
So the question is, so if we put this on, say, for example,
Humanitarianism or Pistrate Matt website
to get a walk in there, would we have the same task each time
or would it be different tasks on that website?
I think that's up for debate.
I don't think we've got a good answer to that yet.
But we want to start with this model, because we know
it has really good value and really prove the idea of using that.
And then we might have some more data types in there for sure.
If I understand the question is like, you would
that makes in the same cop chat, images for buildings,
and images for solar detection, that's what you said.
Or in general, yeah, I mean, if you have one task,
specifically, especially because that one question, right?
That's what I guess, from where?
But I guess, like if you came to Monday and came to
and choose, do you make it?
Yeah, yeah, yeah, it couldn't be.
Yes.
Cool.
I think that's about our time.
Well, sir, one more question.
I think that's how to see one of the findings,
you know, we can include this country.
And one guy program, simple web interface,
or SM buildings.
Yeah.
State artillery and the social rescue record
at the building during the area, and he put it
to the original engineering one,
below the social rescue material, the artillery,
though, and users were selecting.
Yeah.
Which is the static, so you can be used to detect
to verify the detection of the strike building
so you know, sir.
Yeah, so the question I guess was, yeah, yeah, totally.
So the question was from a previous example
of hot using new images and all the images
to see which buildings were destroyed and which ones aren't.
Yeah, I think we could build an interface that looks like that.
I will say we're not trying to replace all of what
humanity in a push-to-beb steering,
like this isn't an extra add-on to get a few more views
a few more clicks to do something so different.
Like, hot has got so many amazing tools
that are focused on getting people who are really invested
in this and kind of have a lot of time spending it
integrating with it.
The idea here is like, can we just get a little bit more attention
for people who might not be interested in doing
right to the airport or shoot me up,
but have a few extra bits of time.
But yeah, I think in the future we could extend this
to be like detection of like whether buildings there
or not at the end.
I think the challenge is always going to be
we need it to be both easy to do, quick, fun,
and then also we let bots.
And so that's always the kind of tension
between these two things, right?
So we need to think about that really carefully.
OK, that's great.
Thank you so much, everybody.
I'm for a pleasant pleasant.
Bye now.
APPLAUSE