WEBVTT

00:00.000 --> 00:07.000
Hello everyone.

00:07.000 --> 00:08.000
I hope you can hear me.

00:08.000 --> 00:10.000
I have this low voice today.

00:10.000 --> 00:11.000
It's me.

00:11.000 --> 00:12.000
I'm Anna.

00:12.000 --> 00:13.000
Then there is two hours.

00:13.000 --> 00:16.000
And there is another person who couldn't make it to Brussels today.

00:16.000 --> 00:21.000
And we have started the web developed marcha.

00:21.000 --> 00:25.000
We call it the open source capture that improves open street map.

00:25.000 --> 00:27.000
And to start speaking about marcha.

00:27.000 --> 00:31.000
I want to start from when it actually was born.

00:31.000 --> 00:34.000
So marcha was born here, more or less.

00:34.000 --> 00:39.000
Not exactly here, but like a Nairobi at the state of the map last September,

00:39.000 --> 00:44.000
where I was presenting my academic research about a tool,

00:44.000 --> 00:48.000
which is developed by HOT, the humanitarian pursuit map team.

00:48.000 --> 00:51.000
This is the website, the school is called Fair.

00:51.000 --> 00:55.000
And this is a tool that allows you to basically

00:55.000 --> 00:58.000
the call it AI assistant mapping tool.

00:58.000 --> 01:06.000
And at the moment it allows you to basically identify buildings from satellites.

01:06.000 --> 01:11.000
And the way it works is that there is a pre-trained model,

01:11.000 --> 01:15.000
which can find tune on a specific city using data,

01:15.000 --> 01:19.000
which makes available for a website, which is called Openeram app.

01:19.000 --> 01:23.000
But I'm showing you in general how it works for this is a computer vision task

01:23.000 --> 01:26.000
of images segmentation.

01:26.000 --> 01:29.000
So the labels come from a street map.

01:29.000 --> 01:31.000
They are in a Victoria format.

01:31.000 --> 01:33.000
And they need to be transformed to a binary mask,

01:33.000 --> 01:36.000
where you have other buildings on buildings basically.

01:36.000 --> 01:39.000
So this is a raster, which is compatible with the ground through the data,

01:39.000 --> 01:42.000
which is from the website that was mentioned before.

01:42.000 --> 01:45.000
Then you go through the machine learning model,

01:45.000 --> 01:48.000
while the training validation takes place.

01:48.000 --> 01:51.000
You build up a checkpoint place,

01:51.000 --> 01:55.000
where with which you can run prediction on another area of a city.

01:55.000 --> 01:56.000
Okay?

01:56.000 --> 02:00.000
So this can find tune up a pre-trained model on whichever city you want,

02:00.000 --> 02:03.000
as long as the RGB imagery is available.

02:03.000 --> 02:06.000
And the data from up a street map I show how it works.

02:06.000 --> 02:10.000
So this is an image from a panorama from a place in Russia.

02:10.000 --> 02:13.000
Yeah, this is the website that was mentioned before,

02:13.000 --> 02:16.000
which actually now is done if you try to make it work,

02:16.000 --> 02:18.000
I mean, to do it with them work, but anyways,

02:18.000 --> 02:20.000
that's where the images come from.

02:20.000 --> 02:23.000
This is the area of interest, the training area.

02:23.000 --> 02:25.000
You fetch the data from up a street map.

02:25.000 --> 02:27.000
And if the data is not available there,

02:27.000 --> 02:30.000
you can actually map it yourself,

02:30.000 --> 02:33.000
which I think is great, because you don't label it just for you on your computer.

02:33.000 --> 02:36.000
And it goes straight to a street map, what you do for the labels.

02:36.000 --> 02:39.000
Anyway, so after you run your training,

02:39.000 --> 02:42.000
I've done this for 25 cities for my research.

02:42.000 --> 02:43.000
Okay?

02:43.000 --> 02:45.000
So that's the performance of fair,

02:45.000 --> 02:48.000
because when you run the training,

02:49.000 --> 02:51.000
you get your model for your city.

02:51.000 --> 02:53.000
This is a, sorry.

02:53.000 --> 02:57.000
This is, this is in Kakuma,

02:57.000 --> 03:02.000
in, this is actually refugee company in Kenya.

03:02.000 --> 03:04.000
And when you run the prediction,

03:04.000 --> 03:06.000
you can get fairly good results somewhere,

03:06.000 --> 03:08.000
as well as it works worse.

03:08.000 --> 03:09.000
Okay?

03:09.000 --> 03:13.000
So yeah, for example, these buildings are not great.

03:13.000 --> 03:15.000
So in the audience, like, like you,

03:15.000 --> 03:17.000
there was someone that thought,

03:17.000 --> 03:21.000
wait, this could be a good idea for something.

03:21.000 --> 03:22.000
Okay? I tell you.

03:22.000 --> 03:24.000
So the way it works is that this prediction

03:24.000 --> 03:26.000
doesn't go straight to a street map,

03:26.000 --> 03:28.000
but then it's again feedback.

03:28.000 --> 03:29.000
So what you can do,

03:29.000 --> 03:30.000
this is the botanier you don't see,

03:30.000 --> 03:32.000
which you leave a feedback.

03:32.000 --> 03:33.000
Okay?

03:33.000 --> 03:36.000
So the guy who was watching in the crowd was saying,

03:36.000 --> 03:39.000
like, what if we try to get this feedback

03:39.000 --> 03:42.000
on the AI prediction from a wider audience,

03:42.000 --> 03:44.000
so we can help port an open street map.

03:45.000 --> 03:47.000
So that's the idea of the map,

03:47.000 --> 03:48.000
plus,

03:48.000 --> 03:50.000
while doing that,

03:50.000 --> 03:53.000
what if you create a tool that helps block the bots,

03:53.000 --> 03:54.000
so a capture,

03:54.000 --> 03:58.000
but without improving proprietary maps and software,

03:58.000 --> 04:00.000
or exposing user information to third parties,

04:00.000 --> 04:02.000
so that's the lock part.

04:02.000 --> 04:05.000
And then if you add the matcha t,

04:05.000 --> 04:08.000
so you get the map-cha logo.

04:08.000 --> 04:10.000
Okay? So that's the idea.

04:10.000 --> 04:13.000
The person behind the idea and the name is Giam.

04:14.000 --> 04:17.000
I have connected the two of them

04:17.000 --> 04:19.000
and plucked everything,

04:19.000 --> 04:23.000
so that then Stuart arrived and developed actually the tool.

04:23.000 --> 04:25.000
So that's the few of us.

04:25.000 --> 04:27.000
What are we trying to do?

04:27.000 --> 04:29.000
We ask ourselves, can we build something

04:29.000 --> 04:31.000
that can reliable reliably,

04:31.000 --> 04:33.000
that the difference between our human and the bot,

04:33.000 --> 04:35.000
because that's what the capture does.

04:35.000 --> 04:39.000
Because you are allowed and not allowed into our website,

04:39.000 --> 04:41.000
with the capture.

04:41.000 --> 04:43.000
And can we actually get that a data for us?

04:43.000 --> 04:45.000
Because in those data,

04:45.000 --> 04:47.000
we don't want to put them straight to a specific map.

04:47.000 --> 04:49.000
And so also,

04:49.000 --> 04:53.000
we got good validation data to improve the models themselves.

04:53.000 --> 04:55.000
So yeah, we have Stuart.

04:55.000 --> 04:58.000
It's going to speak about what is developed.

04:58.000 --> 04:59.000
Let me see it.

04:59.000 --> 05:00.000
Yeah.

05:00.000 --> 05:02.000
Great, thanks.

05:02.000 --> 05:05.000
Yeah, so basically what we wanted to do was test this idea.

05:05.000 --> 05:08.000
So where we are with the project is not like a live capture,

05:08.000 --> 05:10.000
that you can install on your website,

05:10.000 --> 05:13.000
we're kind of prototyping and everything on the idea

05:13.000 --> 05:15.000
to see if we can get something that works,

05:15.000 --> 05:17.000
and that we can then, and in the first instance,

05:17.000 --> 05:18.000
use an upper sheet map,

05:18.000 --> 05:20.000
and then add a wider context as well.

05:20.000 --> 05:23.000
So this first iteration of the platform,

05:23.000 --> 05:25.000
you can try it out here.

05:25.000 --> 05:27.000
If everybody wants to give it a go,

05:27.000 --> 05:29.000
it'll give us more data, which is always good.

05:29.000 --> 05:31.000
But basically you land on the website,

05:31.000 --> 05:33.000
it gives you a little bit of a brief introduction.

05:33.000 --> 05:35.000
And then we have actually two different interfaces.

05:35.000 --> 05:37.000
So when we were trying to figure out how to

05:37.000 --> 05:40.000
make this fast and fun and accessible,

05:40.000 --> 05:43.000
we wanted to try a couple of different modes of interaction.

05:43.000 --> 05:45.000
So the first, the second one is the grid one,

05:45.000 --> 05:48.000
and that's kind of more of a traditional kind of capture,

05:48.000 --> 05:49.000
like you would see on Google,

05:49.000 --> 05:52.000
where you see a bunch of images with the buildings

05:52.000 --> 05:54.000
outlined, the outputs of the AI model,

05:54.000 --> 05:57.000
and you're asked basically which ones of these look correct,

05:57.000 --> 06:01.000
which ones look like the AI has properly outlined the building.

06:01.000 --> 06:02.000
And what that allows us to do is,

06:02.000 --> 06:05.000
as to quickly identify places where the AI is gone wrong,

06:05.000 --> 06:09.000
or places where it's completely messed what's there in the first place.

06:09.000 --> 06:11.000
And the second interface is the swipe interface,

06:11.000 --> 06:14.000
which is a little bit more like a dating app,

06:14.000 --> 06:15.000
I won't mention which one,

06:15.000 --> 06:19.000
but it's a quicker interaction where you see a larger version of the image,

06:19.000 --> 06:21.000
and you just kind of swipe left or right to tell us

06:21.000 --> 06:24.000
whether or not you think that the AI has got the answer correct.

06:24.000 --> 06:28.000
And the idea is that we show a mix of images on here,

06:28.000 --> 06:30.000
we have show images that we knew the answer for already,

06:30.000 --> 06:32.000
to verify if somebody is human or not.

06:33.000 --> 06:37.000
The data that's being generated from the AI model is already wrong,

06:37.000 --> 06:40.000
so we think that it's a good candidate for a reading out bot,

06:40.000 --> 06:42.000
so that we need to test that idea.

06:42.000 --> 06:45.000
And then we also show images that we don't know the answer to,

06:45.000 --> 06:47.000
and so we gradually collect multiple people's opinions

06:47.000 --> 06:49.000
or whether that image was correct or not,

06:49.000 --> 06:50.000
even though we don't know the answer,

06:50.000 --> 06:53.000
and then that helps us with the models and retraining.

06:53.000 --> 06:56.000
So they kind of work like this, the swipe interface has buttons at the bottom,

06:56.000 --> 06:59.000
where you can click on them to very quickly,

06:59.000 --> 07:01.000
just say whether one's right or not,

07:01.000 --> 07:02.000
if you don't want to swipe,

07:02.000 --> 07:04.000
or you can click on the main image,

07:04.000 --> 07:07.000
or use your finger in the main image and swipe left and right,

07:07.000 --> 07:10.000
to kind of just tell us whether it's correct or not, which is fun.

07:10.000 --> 07:13.000
The second interface is the grid one,

07:13.000 --> 07:15.000
which I get said is a little bit more traditional capture,

07:15.000 --> 07:17.000
where you just like all the images you think are right,

07:17.000 --> 07:21.000
and then you get another set of images to have a look through.

07:21.000 --> 07:23.000
So the straight off between these two,

07:23.000 --> 07:25.000
the swipe interface gives you a larger image,

07:25.000 --> 07:26.000
so you can see more detail,

07:26.000 --> 07:30.000
if the grid one will get through more images over time,

07:30.000 --> 07:32.000
so it's a little bit faster.

07:32.000 --> 07:34.000
And we wanted to test out how well these worked.

07:34.000 --> 07:36.000
So when you land in the website,

07:36.000 --> 07:38.000
as I'm sure some of you are doing in the room right now,

07:38.000 --> 07:42.000
you'll get randomly assigned either the swipe interface or the grid interface,

07:42.000 --> 07:44.000
so we can do a little bit of an eB latency,

07:44.000 --> 07:47.000
which one is better, which one works in different contexts.

07:47.000 --> 07:49.000
The entire thing is built on open source software,

07:49.000 --> 07:51.000
so the interface itself is just web components,

07:51.000 --> 07:52.000
which if you haven't played with them,

07:52.000 --> 07:55.000
or are really good alternative to things like React,

07:55.000 --> 07:57.000
or spell, or Angular,

07:57.000 --> 07:59.000
they're just built into the browser,

07:59.000 --> 08:02.000
and give you components to build interesting things really quickly,

08:02.000 --> 08:03.000
and they're a browser standard,

08:03.000 --> 08:05.000
so they kind of work everywhere without too much hassle.

08:05.000 --> 08:07.000
And then the backend,

08:07.000 --> 08:09.000
we just set up a little super base database,

08:09.000 --> 08:12.000
which is open source alternative to Firebase,

08:12.000 --> 08:15.000
to kind of just gradually grab the data and store it.

08:15.000 --> 08:17.000
So once we've come to build the fill version of this,

08:17.000 --> 08:19.000
it'll be using a more robust backend,

08:19.000 --> 08:21.000
that'll have callbacks for logging people in,

08:21.000 --> 08:22.000
and things like that,

08:22.000 --> 08:24.000
but for now we're just collecting basic data,

08:24.000 --> 08:27.000
which is a very simple database in the backend.

08:27.000 --> 08:30.000
And so we'll be running this for about three weeks.

08:30.000 --> 08:32.000
We launched it on half January.

08:32.000 --> 08:33.000
Yeah.

08:33.000 --> 08:34.000
Yeah.

08:34.000 --> 08:35.000
We launched it,

08:35.000 --> 08:38.000
initially, and what you say,

08:38.000 --> 08:40.000
we're among friends,

08:40.000 --> 08:43.000
and then I posted a must-do on,

08:43.000 --> 08:47.000
and we got a big skyrocketed with a number of people who tried it.

08:47.000 --> 08:49.000
So we got,

08:49.000 --> 08:52.000
kind of, yeah, seven of people who used it,

08:52.000 --> 08:54.000
and of these,

08:54.000 --> 08:57.000
we got to one of the people who run the survey after.

08:57.000 --> 08:59.000
We could run it filled up,

08:59.000 --> 09:01.000
and we started from 3098 images,

09:01.000 --> 09:03.000
which is what we have in the database,

09:03.000 --> 09:06.000
and then we got more than 20,000 clicks.

09:06.000 --> 09:08.000
So we got all these data.

09:08.000 --> 09:10.000
How do the images look like,

09:10.000 --> 09:12.000
and try to make it clear?

09:12.000 --> 09:14.000
So these data,

09:14.000 --> 09:17.000
where we have already labeled data,

09:17.000 --> 09:20.000
so we can know if persons are correct or not.

09:20.000 --> 09:22.000
So in case,

09:22.000 --> 09:25.000
so this is the case where the building should have been predicted,

09:25.000 --> 09:28.000
and it has been predicted, because we have labels there.

09:28.000 --> 09:31.000
Yeah, we don't have labels, but it has been predicted.

09:31.000 --> 09:34.000
Yeah, it should have been predicted, but it has not.

09:34.000 --> 09:36.000
Basically, so we have labels here,

09:36.000 --> 09:38.000
and yeah, it's a true negative,

09:38.000 --> 09:41.000
so it's a true force, positive and negative.

09:41.000 --> 09:43.000
So yeah, there's not a big index,

09:43.000 --> 09:45.000
and there should be buildings, which is okay.

09:45.000 --> 09:47.000
So when we asked the question,

09:47.000 --> 09:48.000
is there a shape correctly,

09:49.000 --> 09:52.000
which is the question that we put in the map chart,

09:52.000 --> 09:54.000
and it was very hard to find a good question.

09:54.000 --> 09:56.000
If you have an input for that, please come up with something,

09:56.000 --> 10:00.000
because we're never happy with any of the versions that we try.

10:00.000 --> 10:02.000
So when we asked this question,

10:02.000 --> 10:04.000
we expect people to reply like that.

10:04.000 --> 10:08.000
Yes, no, no, yes, for all these cases, okay.

10:08.000 --> 10:11.000
So going through it, so eventually,

10:11.000 --> 10:13.000
so this is the amount of images we had,

10:13.000 --> 10:15.000
mainly true positive,

10:16.000 --> 10:18.000
for positive force negatives.

10:18.000 --> 10:20.000
And this is how the people responded,

10:20.000 --> 10:22.000
like this is what we were expecting them to reply.

10:22.000 --> 10:24.000
And you see that with the force negatives,

10:24.000 --> 10:27.000
we have really little people who agree with the image,

10:27.000 --> 10:30.000
somehow, then we go through that, if we have time.

10:30.000 --> 10:32.000
And yeah, and mainly,

10:32.000 --> 10:36.000
they agree with the true negatives and for positive.

10:36.000 --> 10:39.000
But the point for us in the,

10:39.000 --> 10:42.000
like it on the SNF people are humans or not,

10:42.000 --> 10:46.000
about how much they agree with the image,

10:46.000 --> 10:47.000
with the labels themselves,

10:47.000 --> 10:49.000
but how much people agree among themselves.

10:49.000 --> 10:50.000
So because they are humans,

10:50.000 --> 10:53.000
and they might be more crack than the labels, right.

10:53.000 --> 10:57.000
So anyway, I'm showing you where most of the people agree,

10:57.000 --> 11:01.000
here, this is like 100% of people agree that this

11:01.000 --> 11:04.000
is where correct or false, okay.

11:04.000 --> 11:06.000
And this is where most of the people didn't agree,

11:06.000 --> 11:10.000
like maybe just 1% of the people agree that this was a building somehow.

11:10.000 --> 11:12.000
Yeah, I just showing you what it is, and even just

11:12.000 --> 11:17.000
them, that's what happens, came out from the test.

11:17.000 --> 11:21.000
Then Stuart, that's what he has to say here.

11:21.000 --> 11:25.000
Yeah, so what Anna was just showing was kind of like the overall results.

11:25.000 --> 11:28.000
So for each one of those categories that were interested in identifying

11:28.000 --> 11:31.000
on average how many people got it right.

11:31.000 --> 11:34.000
But we also can look at the distribution of how well

11:34.000 --> 11:36.000
somebody did an image.

11:36.000 --> 11:39.000
So basically what we're showing here is if we take each image,

11:39.000 --> 11:43.000
that image will be shown to roughly about 100 people.

11:43.000 --> 11:45.000
Or so, I'm given the numbers that we have.

11:45.000 --> 11:49.000
And we can then look at the fraction of people who said the right thing,

11:49.000 --> 11:51.000
given what we know about the label.

11:51.000 --> 11:54.000
And so what you see along the bottom here is the fraction of users,

11:54.000 --> 11:57.000
and then the fraction of the images on the histogram as well.

11:57.000 --> 12:01.000
So essentially, the further to the right here we are on this graph,

12:01.000 --> 12:06.000
the more images, the more people agree to the images,

12:06.000 --> 12:09.000
and the further to the left, the less the agreed on images.

12:09.000 --> 12:11.000
So we can see here in the sniping interface,

12:11.000 --> 12:15.000
there's kind of a peak around here where about 60%

12:15.000 --> 12:18.000
70% of users agreed on the label for the image,

12:18.000 --> 12:19.000
and they got that correct.

12:19.000 --> 12:23.000
And so what that means is if we take images from these buckets on the histogram,

12:23.000 --> 12:26.000
those are places where we know the image was relatively easy to classify.

12:26.000 --> 12:29.000
And so we can use that for human verification,

12:29.000 --> 12:31.000
or we can use that to get information on the site.

12:31.000 --> 12:34.000
It's interesting to see how these compare between the sniping interface

12:34.000 --> 12:36.000
and the grid interface.

12:36.000 --> 12:38.000
Well, and this is, and probably the one we're working.

12:38.000 --> 12:39.000
Oh, here we go.

12:39.000 --> 12:40.000
There we go.

12:40.000 --> 12:41.000
There we go.

12:41.000 --> 12:42.000
There.

12:42.000 --> 12:44.000
And the grid interface, the false positives.

12:44.000 --> 12:47.000
People got them almost entirely correct.

12:47.000 --> 12:49.000
Like they got them to the point where we,

12:49.000 --> 12:51.000
everything shoved out against the right hand side,

12:51.000 --> 12:54.000
and it's like basically almost like perfect.

12:54.000 --> 12:57.000
But the true positives have got this much longer tail here.

12:57.000 --> 13:00.000
And so really what we're trying to understand is like the differences between the interfaces,

13:00.000 --> 13:05.000
and what they lead to in terms of trade-offs between the swiping interface and the grid interface.

13:05.000 --> 13:09.000
And this might be due to the fact that the swiping interface is much larger screen size,

13:09.000 --> 13:11.000
so you can see more in the image.

13:11.000 --> 13:13.000
But in the grid interface, you see more images alongside each other,

13:13.000 --> 13:15.000
so you've got some comparison points.

13:15.000 --> 13:17.000
So you can sort of maybe tell the difference between them.

13:17.000 --> 13:20.000
So we're still analyzing this data, we're trying to figure it out.

13:20.000 --> 13:22.000
But the important thing is that we have a bunch of images,

13:22.000 --> 13:26.000
and these context that we can use then for further follow-up,

13:26.000 --> 13:29.000
and also do the kind of like the verification step.

13:29.000 --> 13:31.000
We also had a survey.

13:31.000 --> 13:35.000
If you try this out, you'll see it that kicks in after about 35 images of being classified,

13:35.000 --> 13:38.000
which is about three or four pages of the grid,

13:38.000 --> 13:40.000
or about 35 swipes in the swiping interface.

13:40.000 --> 13:43.000
And we ask people kind of a number of different questions about who they are,

13:43.000 --> 13:46.000
so whether or not they've done belly identification before,

13:46.000 --> 13:49.000
on an open sheet map or somewhere else,

13:49.000 --> 13:52.000
and it turns out that we have a fairly large,

13:52.000 --> 13:54.000
strongly agreed there and agreed.

13:54.000 --> 13:56.000
So we've got quite a specialized audience,

13:56.000 --> 13:59.000
so we're reaching a biased audience through the Master Dawn,

13:59.000 --> 14:03.000
through the places that we put out there in friends,

14:03.000 --> 14:07.000
which makes sense, but it's also not necessarily that bad of a biased audience,

14:07.000 --> 14:10.000
simply because we want to use this first all-no-push sheet map

14:10.000 --> 14:12.000
to like walk into different things there.

14:12.000 --> 14:14.000
So this is actually quite close to the audience.

14:14.000 --> 14:16.000
It's going to be the first user audience for this, which is great.

14:16.000 --> 14:20.000
People say, we ask people if they could easily identify the types of buildings,

14:20.000 --> 14:23.000
and we got some strongly agree, a good chunk of agree,

14:23.000 --> 14:25.000
and then there's kind of a mix of the end here.

14:25.000 --> 14:26.000
There's a hard task.

14:26.000 --> 14:28.000
We know that people find this hard to do it,

14:28.000 --> 14:31.000
and we're going to iterate on the question we ask,

14:31.000 --> 14:33.000
and the kind of things we're asking in there as well.

14:33.000 --> 14:37.000
People ask if they would like more or less in the imagery.

14:37.000 --> 14:39.000
This is on surprising that on the swiping interface,

14:39.000 --> 14:41.000
very few people said they wanted more of them then,

14:41.000 --> 14:43.000
whereas on the grid interface, most people said they wanted more of them then.

14:43.000 --> 14:46.000
So we're going to look to see how we can make the grid interface a little bit clearer,

14:46.000 --> 14:49.000
and then a benefit more of instructions and find this cool.

14:49.000 --> 14:51.000
We were very reassured that people found this cool.

14:51.000 --> 14:54.000
So we're happy to say that people enjoy doing this,

14:54.000 --> 14:56.000
and it's a good idea, which is very nice.

14:56.000 --> 14:59.000
So I'll pass over to Tyler for some of the other feedback.

14:59.000 --> 15:00.000
Yeah, yeah.

15:00.000 --> 15:03.000
So yeah, I was saying we posted this on MasterDone,

15:03.000 --> 15:05.000
on the 15th of January,

15:05.000 --> 15:08.000
and we got several comments there.

15:08.000 --> 15:11.000
Maybe easy to access to this and to the survey.

15:11.000 --> 15:14.000
Yeah, so it was mainly about accessibility.

15:14.000 --> 15:15.000
We just read out the line.

15:15.000 --> 15:18.000
Probably it was, we could do something else.

15:18.000 --> 15:22.000
So somebody was suggesting to put a kind of,

15:23.000 --> 15:26.000
not to fill up, not only the outline,

15:26.000 --> 15:31.000
but to fill up the building as well, the rectangle, whatever.

15:31.000 --> 15:36.000
So people were saying that in terms of accessibility,

15:36.000 --> 15:39.000
we should take into account people with visual impairment,

15:39.000 --> 15:43.000
and that was an idea well as well about translating to other languages,

15:43.000 --> 15:46.000
or give the possibility at least.

15:47.000 --> 15:50.000
So for someone, so I write a left was not clear.

15:50.000 --> 15:53.000
If the direction was very come from where it goes to,

15:53.000 --> 15:58.000
and then, yeah, and the tiles with no outline got a big,

15:58.000 --> 16:02.000
like logical question, like if you ask me,

16:02.000 --> 16:05.000
if there is a, the right outline,

16:05.000 --> 16:07.000
but there's no red right, but what am I doing here,

16:07.000 --> 16:09.000
but for us, because of the way we built it up,

16:09.000 --> 16:11.000
with the true negative for us,

16:11.000 --> 16:13.000
it was a lot of data more.

16:13.000 --> 16:15.000
So once we explained it in the comments,

16:15.000 --> 16:18.000
I think, understood, because they did it correctly,

16:18.000 --> 16:20.000
and then we can add instructions,

16:20.000 --> 16:23.000
which is something we thought about too.

16:23.000 --> 16:26.000
The survey, many people didn't get to it,

16:26.000 --> 16:29.000
and yeah, the zoom level.

16:29.000 --> 16:32.000
People were asking, in the sort of,

16:32.000 --> 16:34.000
we just made it worth in general,

16:34.000 --> 16:36.000
we'd like to change the zoom level,

16:36.000 --> 16:38.000
and people asked, yeah, in which sense,

16:38.000 --> 16:40.000
but yeah, for us, it was obvious like the zoom more in,

16:40.000 --> 16:42.000
but yeah, that's it.

16:42.000 --> 16:45.000
Again, yeah, future work.

16:45.000 --> 16:46.000
There is a lot to do.

16:46.000 --> 16:49.000
I mean, this is just an initial trial.

16:49.000 --> 16:52.000
We would like to improve the images,

16:52.000 --> 16:55.000
first of all, because of the data we had was limited,

16:55.000 --> 16:57.000
and we only had the, always the same tiles,

16:57.000 --> 17:00.000
we could just switch the buildings that were selected in there.

17:00.000 --> 17:03.000
But the idea for us would be to have the,

17:03.000 --> 17:05.000
the image centered on the building,

17:05.000 --> 17:09.000
and also improve the outlines, as we were seen before.

17:10.000 --> 17:12.000
We should introduce the unknowns,

17:12.000 --> 17:14.000
because yeah, that's what we were seen before.

17:14.000 --> 17:17.000
We already have label data, so we can understand

17:17.000 --> 17:19.000
if people agree or not with what we already have,

17:19.000 --> 17:22.000
and eventually we can get new information on the images,

17:22.000 --> 17:24.000
which are not labeled yet,

17:24.000 --> 17:28.000
so basically we're going to show our mix of all these two categories.

17:28.000 --> 17:31.000
Then we'll add instructions, as we said,

17:31.000 --> 17:34.000
and we'll have to have a good thing on how to integrate

17:34.000 --> 17:37.000
the validative buildings into the wider opposite mountata set,

17:37.000 --> 17:39.000
we're thinking of the other two inputs into,

17:39.000 --> 17:42.000
input the results into the map rule add,

17:42.000 --> 17:46.000
of, you know, well, we have no battery,

17:46.000 --> 17:49.000
and also need back to self training.

17:49.000 --> 17:52.000
Like I, ideally, if we have,

17:52.000 --> 17:54.000
there results from people, we can,

17:54.000 --> 17:56.000
where we have my confidence,

17:56.000 --> 18:00.000
we can send it back to for training on the same model,

18:00.000 --> 18:02.000
so that we get better checkpoints,

18:02.000 --> 18:04.000
and we have better prediction,

18:04.000 --> 18:07.000
good, ideally, forever, right?

18:07.000 --> 18:10.000
Yeah, then we should also add a skipper,

18:10.000 --> 18:12.000
a lot of new buttons as well,

18:12.000 --> 18:14.000
which is also in so many captures,

18:14.000 --> 18:16.000
and the translation,

18:16.000 --> 18:18.000
and there also just accessibility

18:18.000 --> 18:20.000
that there was mentioning before.

18:20.000 --> 18:22.000
You want to see something else?

18:22.000 --> 18:25.000
Okay, so that's it, I think.

18:25.000 --> 18:26.000
Thank you.

18:26.000 --> 18:34.000
I mean, again, I guess just to say,

18:34.000 --> 18:36.000
if MD wants to collaborate with us on this,

18:36.000 --> 18:38.000
on the data site, on the eye site,

18:38.000 --> 18:40.000
or on the, this up to itself,

18:40.000 --> 18:41.000
on the coding site,

18:41.000 --> 18:42.000
please feel free to reach out.

18:42.000 --> 18:44.000
We'd love to have more cards of users,

18:44.000 --> 18:45.000
and more people getting involved.

18:45.000 --> 18:46.000
Yes.

18:46.000 --> 18:47.000
Yeah.

18:47.000 --> 18:48.000
Good question.

18:48.000 --> 18:49.000
Yes, sir.

18:49.000 --> 18:51.000
I think your height is data set,

18:51.000 --> 18:53.000
to make sure you find buildings.

18:53.000 --> 18:55.000
So, is the question,

18:56.000 --> 18:58.000
like our word?

18:58.000 --> 18:59.000
Ah, yeah, okay.

18:59.000 --> 19:01.000
The question is, yeah, so the question is,

19:01.000 --> 19:03.000
are we using any height information

19:03.000 --> 19:04.000
when we're identifying the buildings

19:04.000 --> 19:06.000
through lighter and things of that?

19:06.000 --> 19:07.000
Currently, we're not.

19:07.000 --> 19:10.000
This is, we think about this as satellite data,

19:10.000 --> 19:11.000
but it's often not,

19:11.000 --> 19:13.000
it's often drawn slaying over an area

19:13.000 --> 19:15.000
with cameras and looking down.

19:15.000 --> 19:16.000
And for areas where,

19:16.000 --> 19:17.000
the humanitarian,

19:17.000 --> 19:19.000
which street map is kind of like,

19:19.000 --> 19:21.000
needs often you're just getting fairly straight

19:21.000 --> 19:24.000
forward RGB images of the ground.

19:24.000 --> 19:25.000
So, in that context,

19:25.000 --> 19:27.000
we're not using the hay information or lighter,

19:27.000 --> 19:29.000
which is quite expensive and hard to gather.

19:29.000 --> 19:30.000
Yeah, yeah.

19:30.000 --> 19:32.000
Yeah, just so out on these,

19:32.000 --> 19:34.000
how to want to improve the model

19:34.000 --> 19:35.000
to identify not only buildings,

19:35.000 --> 19:37.000
but other features.

19:37.000 --> 19:40.000
And there is a project in the future

19:40.000 --> 19:42.000
about, which should be starting

19:42.000 --> 19:45.000
about using street-level imagery,

19:45.000 --> 19:48.000
which should be integrated as well.

19:48.000 --> 19:51.000
So, yeah, there's already a deal.

19:51.000 --> 19:52.000
Using it in general,

19:52.000 --> 19:55.000
image segmentation tool for other stuff too.

19:55.000 --> 19:57.000
But not lighter, I think,

19:57.000 --> 19:59.000
would be harder to put it together.

19:59.000 --> 20:00.000
Hmm.

20:02.000 --> 20:03.000
Any other questions?

20:03.000 --> 20:04.000
I think we've probably got a lot

20:04.000 --> 20:05.000
about the actual time event?

20:05.000 --> 20:06.000
Yeah, okay.

20:10.000 --> 20:11.000
No, it's also fine.

20:11.000 --> 20:13.000
You're all just busy playing the thing, right?

20:13.000 --> 20:14.000
Yeah.

20:14.000 --> 20:15.000
Yeah.

20:15.000 --> 20:16.000
Yeah, please.

20:17.000 --> 20:18.000
Yeah.

20:18.000 --> 20:21.000
The question is,

20:21.000 --> 20:24.000
have we been compared to the other

20:24.000 --> 20:27.000
captures and kinds of, like,

20:27.000 --> 20:32.000
what kind of possibilities or what sort of

20:32.000 --> 20:34.000
things might actually also just be more like senseable

20:34.000 --> 20:36.000
to tell some of the steps of our work?

20:36.000 --> 20:37.000
Yeah.

20:37.000 --> 20:38.000
I mean, we'd like to think so.

20:38.000 --> 20:39.000
I think it is.

20:39.000 --> 20:40.000
So, the question is,

20:40.000 --> 20:41.000
have we been compared to the

20:41.000 --> 20:42.000
other captures and kind of the usability of

20:42.000 --> 20:43.000
the comparisons and the other ones out there,

20:43.000 --> 20:45.000
like the crazy text with, like,

20:45.000 --> 20:47.560
on the top of which is barely readable or things like that.

20:47.560 --> 20:48.680
And we haven't done that yet.

20:48.680 --> 20:52.840
We'd love to do some user research for that,

20:52.840 --> 20:54.560
which would be really cool.

20:54.560 --> 20:56.480
Do you want to do it?

20:56.480 --> 20:58.520
I've been no extra.

20:58.520 --> 21:00.120
But I think we probably want to,

21:00.120 --> 21:02.520
I think one of the interesting things that

21:02.520 --> 21:04.960
captures is the more popular they get,

21:04.960 --> 21:07.040
the more people are going to try to break them.

21:07.040 --> 21:10.600
And so I think almost like there's a nice idea here

21:10.600 --> 21:14.120
that we keep it will key and for OSM and a few other websites.

21:14.120 --> 21:16.040
And build it up gradually that way,

21:16.040 --> 21:18.360
rather than trying to make this the replacement

21:18.360 --> 21:20.000
for all Google catches out there in the world.

21:20.000 --> 21:22.160
So I don't think I don't work trying to supplant the,

21:22.160 --> 21:24.200
you know, click all the more banks

21:24.200 --> 21:26.320
or click all the crosswalks catches of the world.

21:26.320 --> 21:27.560
We're just trying to create something

21:27.560 --> 21:29.120
that's more appropriate for this context.

21:29.120 --> 21:30.400
And for the OSM community,

21:30.400 --> 21:32.840
means that they're not having to rely on proprietary services

21:32.840 --> 21:34.320
like Google's capture system.

21:34.320 --> 21:36.240
I mean, some of the new captures are not even clicking on things.

21:36.240 --> 21:37.160
They're just click the button.

21:37.160 --> 21:39.800
And apparently the way the work is just the way that you're

21:39.800 --> 21:42.160
mouse moves as it goes towards the button,

21:42.160 --> 21:43.600
tells whether or not you're a person or not.

21:43.600 --> 21:45.400
So bots will just go in a straight line

21:45.400 --> 21:47.400
or kind of do some convoluted thing,

21:47.400 --> 21:49.160
but the random motions of your hand

21:49.160 --> 21:50.880
is you're moving towards the thing,

21:50.880 --> 21:52.520
everything off to tell whether you're not your human.

21:52.520 --> 21:55.000
So it's not even clear that these telecaptors

21:55.000 --> 21:58.400
are necessary anymore, but it's still kind of an interesting thing.

21:58.400 --> 22:00.080
And we can gather good data from it as well,

22:00.080 --> 22:01.120
which is the whole point.

22:01.120 --> 22:02.560
We wouldn't be doing this if it wasn't for the fact

22:02.560 --> 22:04.760
that it could be used to improve models,

22:04.760 --> 22:05.960
which is the ultimate goal.

22:08.600 --> 22:09.440
Cool.

22:13.600 --> 22:16.000
It's going to be extensive work,

22:16.000 --> 22:18.840
so that we can use it all over the world.

22:21.120 --> 22:26.120
I thought, like, not just, just really got nice,

22:26.120 --> 22:29.920
but, yeah, it's a great question.

22:29.920 --> 22:30.920
Sorry, yeah.

22:30.920 --> 22:36.600
Yeah, so the question was, is this going to be

22:36.600 --> 22:39.360
extensible to be used with other types of data?

22:39.360 --> 22:41.960
And all the types of images are kind of tasks?

22:41.960 --> 22:42.680
Yeah, I think so.

22:42.680 --> 22:44.200
I think what's we've got to up and running.

22:44.200 --> 22:46.080
It's fairly easy just to substitute

22:46.080 --> 22:47.560
and the images that are being shown.

22:47.560 --> 22:50.640
And then on the back end, give the label that you expect

22:50.640 --> 22:52.920
for the ones that you know about already to test this out.

22:52.920 --> 22:56.160
So it should be very easy to add another things there

22:56.160 --> 22:57.160
and use the interfaces.

22:57.160 --> 22:59.920
It's all fairly, the software itself

22:59.920 --> 23:01.720
is not particularly complicated.

23:01.720 --> 23:02.840
The thing that's going to be complicatedly

23:02.840 --> 23:05.480
around is us doing the integration with login systems.

23:05.480 --> 23:08.520
But that's for a future, to worry about.

23:08.520 --> 23:10.080
But yeah, no, for sure, the kind of interface

23:10.080 --> 23:11.160
and the data should be extensible.

23:11.160 --> 23:13.120
And we've talked a little bit about other ways.

23:13.120 --> 23:16.160
Other types of data we may want to put in there ourselves.

23:16.160 --> 23:16.760
Yeah.

23:16.760 --> 23:22.160
Is there any much I'd like to do just one type of data

23:22.160 --> 23:27.560
that I want to put the website of, does it?

23:27.560 --> 23:29.800
Do you have lines on each of the next user

23:29.800 --> 23:35.560
that can be, in terms of, what's it, what does it, what's it?

23:35.560 --> 23:38.400
Yeah, so right now, I think, so the question is,

23:38.400 --> 23:40.080
are we planning on doing different data types

23:40.080 --> 23:41.080
than the current version?

23:41.080 --> 23:43.080
Is that a question?

23:43.080 --> 23:49.800
Is it going to be just a one type of application?

23:49.800 --> 23:50.800
Right.

23:50.800 --> 23:54.400
Is it going to be possible for weeks?

23:54.400 --> 23:55.400
Yeah, okay.

23:55.400 --> 23:57.480
So the question is, so if we put this on, say, for example,

23:57.480 --> 23:59.040
Humanitarianism or Pistrate Matt website

23:59.040 --> 24:02.240
to get a walk in there, would we have the same task each time

24:02.240 --> 24:03.680
or would it be different tasks on that website?

24:03.680 --> 24:04.560
I think that's up for debate.

24:04.560 --> 24:06.520
I don't think we've got a good answer to that yet.

24:06.520 --> 24:09.120
But we want to start with this model, because we know

24:09.120 --> 24:13.320
it has really good value and really prove the idea of using that.

24:13.320 --> 24:16.880
And then we might have some more data types in there for sure.

24:16.880 --> 24:19.640
If I understand the question is like, you would

24:19.640 --> 24:22.520
that makes in the same cop chat, images for buildings,

24:22.520 --> 24:26.320
and images for solar detection, that's what you said.

24:26.320 --> 24:29.080
Or in general, yeah, I mean, if you have one task,

24:29.080 --> 24:30.880
specifically, especially because that one question, right?

24:30.880 --> 24:33.480
That's what I guess, from where?

24:33.480 --> 24:36.400
But I guess, like if you came to Monday and came to

24:36.400 --> 24:37.240
and choose, do you make it?

24:37.240 --> 24:38.800
Yeah, yeah, yeah, it couldn't be.

24:38.800 --> 24:40.440
Yes.

24:40.440 --> 24:41.120
Cool.

24:41.120 --> 24:42.680
I think that's about our time.

24:42.680 --> 24:44.560
Well, sir, one more question.

24:44.560 --> 24:45.960
I think that's how to see one of the findings,

24:45.960 --> 24:48.160
you know, we can include this country.

24:48.160 --> 24:51.400
And one guy program, simple web interface,

24:51.400 --> 24:53.600
or SM buildings.

24:53.600 --> 24:54.440
Yeah.

24:54.440 --> 24:59.400
State artillery and the social rescue record

24:59.400 --> 25:02.840
at the building during the area, and he put it

25:02.840 --> 25:06.600
to the original engineering one,

25:06.600 --> 25:09.800
below the social rescue material, the artillery,

25:09.800 --> 25:12.760
though, and users were selecting.

25:12.760 --> 25:13.760
Yeah.

25:13.760 --> 25:18.520
Which is the static, so you can be used to detect

25:18.520 --> 25:21.280
to verify the detection of the strike building

25:21.280 --> 25:24.400
so you know, sir.

25:24.400 --> 25:29.320
Yeah, so the question I guess was, yeah, yeah, totally.

25:29.320 --> 25:34.760
So the question was from a previous example

25:34.760 --> 25:38.280
of hot using new images and all the images

25:38.280 --> 25:40.600
to see which buildings were destroyed and which ones aren't.

25:40.600 --> 25:43.400
Yeah, I think we could build an interface that looks like that.

25:43.400 --> 25:45.720
I will say we're not trying to replace all of what

25:45.720 --> 25:47.120
humanity in a push-to-beb steering,

25:47.120 --> 25:51.240
like this isn't an extra add-on to get a few more views

25:51.240 --> 25:53.920
a few more clicks to do something so different.

25:53.920 --> 25:56.080
Like, hot has got so many amazing tools

25:56.080 --> 26:00.120
that are focused on getting people who are really invested

26:00.120 --> 26:01.960
in this and kind of have a lot of time spending it

26:01.960 --> 26:02.880
integrating with it.

26:02.880 --> 26:05.680
The idea here is like, can we just get a little bit more attention

26:05.680 --> 26:08.360
for people who might not be interested in doing

26:08.360 --> 26:09.160
right to the airport or shoot me up,

26:09.160 --> 26:10.560
but have a few extra bits of time.

26:10.560 --> 26:12.680
But yeah, I think in the future we could extend this

26:12.680 --> 26:15.600
to be like detection of like whether buildings there

26:15.600 --> 26:16.880
or not at the end.

26:16.880 --> 26:18.400
I think the challenge is always going to be

26:18.400 --> 26:22.280
we need it to be both easy to do, quick, fun,

26:23.280 --> 26:25.360
and then also we let bots.

26:25.360 --> 26:27.200
And so that's always the kind of tension

26:27.200 --> 26:28.120
between these two things, right?

26:28.120 --> 26:30.000
So we need to think about that really carefully.

26:30.000 --> 26:30.680
OK, that's great.

26:30.680 --> 26:31.560
Thank you so much, everybody.

26:31.560 --> 26:33.200
I'm for a pleasant pleasant.

26:33.200 --> 26:34.200
Bye now.

26:34.200 --> 26:37.200
APPLAUSE