So, hi. I'm Bart. I've been working for my
believer for the past three years as a maintainer.
Frank, do you want to introduce yourself?
Hi, I'm Frank. I'm on the governing board.
He's not only on the governing board, he's also a maintainer
for Martin, a tile server, so we're in very good company to be
talking about tiles today.
So, this week we announced a map labor tile.
It's a new tile format, and yeah, we're going to smooth into it
for the people that don't know what map labor is.
Maybe don't know what vector tiles are.
We give a quick introduction here.
So, if you're not familiar with map labor,
these are two main libraries.
There are map tool kits. You can use them to add maps to your apps,
either to your web apps or to your native apps.
I've been maintaining it for the former, so map labor native.
Now, map labor started out as a fork when
map box closed source to their client libraries.
And this is good to keep in mind because I think a lot of projects here
had fuzz them. They have been open source from the start.
So, they have their initial creators are still maintaining the library.
So, they have a huge history, right?
And we needed to start without that institutional knowledge.
We needed to figure out how are we going to organize ourselves.
We even needed to dig into the code base to understand how we can extend it.
And we've been doing that over the years.
So, one of the things we did in the beginning
is we set up an organization to help support the software projects
that handles the sponsorships and donations.
We've actually been quite successful getting sponsorships.
Because a lot of companies, they were deeply dependent on the map box libraries
and they wanted to continue using open source.
So, we've been even been able to hire a few people, including myself,
to maintain these libraries.
Now, we also have a board.
Of course, we have a huge community of contributors.
We have some other projects under the map labor umbrella
and they mostly work these subcommunities mostly work independently.
But we do try to get together once every month.
And also, if you're an active contributor,
you can become a voting member to vote on the board.
So, these are the developments over the years.
One of the first milestones was the rewrite of the map labor
geologists and typescript.
The first board and the first sponsors in 2022.
And the graphics back ends, new graphics back ends
from up to the native globe mode, from up to the web,
and the recent milestone of the project is MOT.
So, let's talk about vector tiles.
What are vector tiles and why would you want them?
Now, you're probably familiar with Raster-based maps.
Raster-based maps are basically just a bunch of images
that are stick together on the client,
which means you have a very simple client.
Now, on the other hand, vector-based maps,
you send the raw geometric data to the client
along with some styling information,
how to draw those geometries.
And you have a much more complicated client,
but you get some advantages.
So, one of them is that you have a lot of better UX.
So, you have continuous zoom,
because when you're in between two,
zoom levels, it gets pixelated.
You don't have that with vector-based maps.
And you can do something like over zoom,
where if you zoom in, you can use the data
of the lower zoom level to render that zoom level
to still have a very detailed picture.
And you can do things like runtime styling.
So, you can have a completely different looking map
based on the style that you provide.
And vector maps are,
even a requirement for some use cases.
One of them is navigation.
So, you can imagine if you're navigating,
you want the labels to save a top.
If you turn your car,
you don't want to read upside down
to know what the name of this road
or this point of interest is.
So, vector-based maps are very dominant
in an navigation space.
And these are actually all apps
that use map-libernative.
If you go to the app story,
download the random navigation app,
it's very likely that it's using map-libernative.
Now, let's go back for technical comment.
So, this is a concept of a tile pyramid.
We're relevant for both vector-based maps
and vector-based maps.
So, we need a way to download
the vector data with the right fidelity.
Of course, if we look at the whole world,
we cannot encode house numbers,
because there's a few to many houses in the world
that make that feasible.
Only when you zoom in,
are you getting that data, right?
We, at different zoom levels,
we have different vector data.
Okay.
So, for the past more than 10 years,
we've been using map-box vector tiles,
and it's a binary format,
which is, of course,
much better than using something like geojason
for performance reasons,
and also because you can pack it much smaller.
It has a relatively simple definition,
so it's defined using something called proto-buff.
It's a format to describe binary format.
It has very wide support,
and everyone has their own implementation,
because it's so easy to implement.
And yeah, it serves us really well.
So, why do we need something new?
Well, we just didn't like the name, right?
Okay, now that's not actually it.
The reason is, we can do better.
And this is sort of the marketing pitch,
and there's, of course, always a little bit
of an asterix there,
but we can achieve a three times better performance ratio,
and a three times better decoding performance,
using the latest insights of data
for data compression research,
because we've actually had some new insights
in the past 10 years there.
So, to understand why we can do better,
we first have to look at MVT,
and I'm handing it over to Frank.
So, it's working.
Yeah, it's working.
So, MVT, very simple encoding.
We start off with one big box,
that is one tile, that is one piece of data.
That data is encoded in protobuff.
Okay, it contains, oh, this is not great.
It contains layers.
A layer can be, this is a street,
this is a tree line, this is something else.
That's great, that works.
If we zoom into one layer,
they contain both metadata,
and the key dictionary, a valid dictionary,
and some features.
So, why do we have a key and be a valid dictionary?
Very simple, because that's where we store our strings.
We do string and turning.
So, let's look at why this is sub-optimal.
Well, first of all, this is very inefficient to render.
Well, we have here movement-based geometry.
We have winding order, it's winding order dependent.
So, for example, if I have one geometry disk goes this round,
and then inside it, it goes the other way around,
then we need to cut this out.
This is very inefficient,
and we also need to do a test relation on the client.
IE cutting up into little triangles.
So, this is fundamentally failing what we want to do, fast rendering.
But we also have, we only have string and turning here.
It's like this here, they're great.
And they're not compressed at all.
If you have very long string in your Open Street map,
it's going to be very large here.
That's not ideal,
because we're literally just having a string dictionary.
Okay.
So, TLDR, these were very simple to implement.
This was back then the only way this could be done.
It is very hard to optimize,
because, frankly, these were not designed to be optimally rendered.
They were designed to be simple.
Well, yeah, if your tiles are compressible by three X,
if you, so G-zip at them,
maybe it's not a good tile format.
So, let's enter MBT here, MLT here.
What is MLT?
It is a more efficient column oriented lightweight encoding,
basically like park key.
And what is column oriented?
Currently, we are row oriented.
So, the data is, for example, string,
integer string, integer, string, integer.
And that's up to more.
Let's start a different way, string string string string string string.
And I can roll them up, do fast FST on them,
be more efficient this way,
and have a more lightweight encoding.
Basically, the same thing that we have been doing
in databases for the last 10 years.
Not like we only have been doing this in the last 10 years
because back in the 2000s,
we were still thinking that computers are going to get faster.
So, we can also do more modern things with them.
Like, for example, nested things like Jason
or MLUs, or actually sucking.
We can move the test relation to the server.
Why is that important?
Because we are currently spending when we are rendering.
Most of our time doing that.
We are cutting up your little polygons into triangles.
That is most of the time where we are spent on rendering.
We are not using a protobuff.
We are rolling our own encoding because we think we can do it better.
It is reasonably simple as similar to how parky is doing it.
So, we start off with a size, a version, and then a data.
Object.
We are going to do it.
It is reasonably simple as similar to how parky is doing it.
So, we start off with a size, a version, and then a data.
Object.
And how modern encodings do we are doing this?
That is that we have four, a certain class of things.
For example, a string.
We have many different encodings that are optimal in some use cases.
This is from a research paper.
This is too complex.
If we ship something like this, we are going to get stranded.
Like, not being able to implement all of them.
Having correctness bugs somewhere.
So, that is not something that we can do currently in the future.
But that is the optimal way.
So, what did we do?
We chose the most effective encodings for the different value types.
And packed them in.
For example, for bullions, they can be either raw in a bitvector.
Or they can be run length encoded.
If you have very many bullions.
Or, like, it goes all the way down.
For example, bull doubles.
We have ob, which is the inductive format.
Or for strings.
We have FSST, which is the format from Munich.
Or for geometry.
We are doing vertex space encoding.
And then the integer optimizations, which is fairly efficient.
Like, delta, early environments.
And, or fast people.
And, like, these all are not new things.
These all have been around in the database world for a while.
But, like, the geo world just hasn't picked up on these improvements in the database world until now.
So, things are going to get better.
Things are going to get faster.
It's amazing.
So, where can I learn more about?
Well, thank you already.
I mentioned it.
Marcus Dremel, who invented the format.
Pushed out.
Research paper is actually published now.
There's some more technical talks on YouTube.
And we also published the specification, which you can find on our GitHub.
Now, the elephant on the room is complexity.
Right?
This is a much more complicated format.
And, actually, there was some controversy, which I was very happy about because, of course,
no open source project is complete without some controversy.
So, the tile maker, maintainer, was limiting the fact that this new format is so much more complex to implement.
And, he went so far as to say that only if you're a big corporate, you can really,
feasibly implement this.
So, complexity.
Well, what makes it complexity?
Well, what makes it complex?
Well, not just because you have to implement all these different encodings that Frank mentioned.
But also, you cannot just put the data that you have in a tile, because you need to figure out what's the best way,
what's the best encodings to choose.
And these encodings, they can even differ per tile.
So, you need to use heuristics.
Right?
So, you need to search all the different encodings.
And you need to think about your trade-offs.
For example, you might choose to have slightly bigger tiles because you get better decoding performance.
So, or you might choose not to search this huge encoding space and have a little bit better encoding time.
Now, how do we address this?
Well, I was speaking to my co-maintainer, Harold, and the first thing he said was, it's not that complicated.
That's his first reply to any technical problem, which is, I mean, it's a good mindset to have.
Another is, MVT is not going away, right?
Well, support MVT indefinitely.
And if MVT works for your use case, I mean, I don't want to throw as much shade as MVT has Frank did.
I think it's a great format still.
And another idea is that we can focus a bit more on sharing code.
I mentioned post-GA, I mentioned tip of canoe, I mentioned ductDB, they all have their own MVT implementation.
And maybe we can use this concept called a library to share the implementation between different integrators, right?
That's one idea as well.
All right.
Another question, when can I start using MLT?
And the answer is always, as always, it depends.
And the answer is today, if you're adventurous, because you will need to make your own tiles, right?
Actually, you can use it today, because it's now supported by both Map Libre native and Map Libre GLGS.
And experimental support has been added to planet iron.
It's a very popular software for making tiles, it's Java based.
And if you don't want to make your own tiles, you sit back and we expect the ecosystem of tile distributors, both commercial and non-commercial to catch up, and will eventually offer MLT.
And AWS is actually funding further MLT improvements this year.
So you can imagine this problem of finding the best encodings that will take some more development to really get right.
But there's a lot of upside potential for optimizing tiles making them smaller.
Now, I mentioned Map Libre, this open source project, we're not a big enterprise, and you can actually shape the future of MLT.
We're active, very active on GitHub, GitHub discussions, GitHub issues.
We have a Slack channel.
Every second Wednesday of the month, we have a big meeting for both Map Libre libraries.
And we also have a monthly meeting on MLT specifically.
If you would like to be invited, everyone is welcome to join, just message us.
Send us an email and we'll send you an invite.
Yeah, we also put a monthly newsletter to all the socials.
And you're welcome to follow us there.
Now, at this point, it's good to mention that MLT came to be thanks to collaboration between both,
between academia, open source, and enterprise.
And we've seen this in the past with other Map Libre developments where we work together with enterprise
to deliver some feature.
And we're very thankful for our sponsors.
And the sponsors are actually why I've been able to be a paid maintainer for Map Libre for the past three years.
But I think recently, especially here in Brussels, not that we're here in Brussels, we're starting to think about
some other funding modes as well, maybe apply for CMU fund or grant.
So if you're a talented developer and wants to work on Map Libre, come talk to us,
and maybe we can apply for one of those grants together.
All right.
Then thanks, thank you very much.
Stand for some questions.
Because it's always a bit awkward getting the first question.
We actually have a question of our own.
And it is, why does it even matter to get smaller tiles better the coding performance?
Frank, do you want to answer that one?
Well, because having a pasta map matters to real users,
I can only share anecdotal evidence, but like when our software got, when our,
our map got faster, we got positive feedback.
Great.
I think you are in India last week and in some parts of the world,
people don't have the latest iPhone and the latest Pixel phone.
So there it also matters a lot to have more faster tiles, right?
Faster.
Yeah, also in some parts of the world, we have very shitty internet like Germany.
And what is the answer?
What do you think is the answer?
And what do you think is the answer?
What do you think is the answer?
What are you going for?
Yeah, whatever.
Is this a question?
Thanks.
So again, on LWN, there was an article from the C-standard
maintainers about implementing use case,
specific compression formats or providing a tool kit to implement it.
Did you already into look that?
Was it too late for the specification of this format?
The depth that work is, I was already done with the microphone,
so I don't need to repeat that.
So I think you're referring to the adaptive,
yes, the extended compression where they have multiple lightweight encodings in there.
And that was too new for us to look into,
but also it wouldn't really fix this.
Because like, the Z-standard people don't know,
like, you wouldn't go to margin encoding for your geometries,
which is like 70% of your tile size.
Because that's not generally what you want.
That's something you want in geometry,
but like for like Z-standard, that doesn't make any sense,
because it's only available for geometry.
Like who would even think about putting geometry in this thing?
So that is something which would be interesting,
but I don't think it could get a 3x performance boost.
It could get a small performance boost,
and MVT, but for that it would need to be shipped into the browser,
which is, for example, need Firefox support,
which is, I don't know, we go talk to them later.
Okay, thanks.
Anyone else with a question?
So, hi, thank you.
You mentioned that it would be nice to reuse the implementation
of your new Mupply Versile format.
Are you planning on providing like a reference implementation
or providing these libraries in some form?
So in an ideal world, we have a spec that's clear as day,
and you can just go and implement it.
So the reality is that right now it's sort of implementation defined.
But, I think the Java encoders is the most mature one right now,
but we have something else going on.
I'll miss the rest talk about it.
The plan is to provide a rust library, which then can be very easily wrapped
into other ecosystems, with some of the existing tooling.
And we're going to see if that is the right path.
We currently also have a TypeScript and C++ version.
I'm hoping on the rust version being the best one,
but it's currently not.
That comes to the hair, by the way, maybe.
I know.
So if you go to our GitHub to the documentation of my
libretile, we have an overview there of all the different implementations and
integrations.
And if you're working on an integration or an implementation in your project,
it's not listed there yet, please create a pull request to add it.
So there will definitely be multiple encoders,
and probably something that wraps this rust encoder.
Yeah.
It's going to be fun.
Any more questions?
Great presentation.
Thank you for explaining that all of us to us.
Have you had any signal from GDAL or PostGIS in terms of supporting?
I know you were just talking about getting all the foundation or sort of
encoding libraries in place, but is there any initial signal from them?
We haven't talked to them yet, I think.
But we're hoping to, if someone here is working in that space,
maybe I don't know what come talk to us, because like,
PostGIS people are here and they're standing reasonably near to us.
So we are hoping on that.
Yeah.
I'm not sure what kind of library they would need, because I don't think that,
actually I think you can write PostGIS plugins in Rust now.
That was mentioned.
Well, it's mostly a technical problem at this point.
But if you want to join that one of the MOT meetings,
just email us and we'll see you there.
Yeah?
I think we're done with time.
Yeah, you're having a good time.
Thank you.
Someone in the library had a standout?
Okay.
Thanks for listening to our TED Talk.
Thank you.