Hi, during the talk, we are going to talk about why D.D., my name is Evgeny or Eugen.
So, my name is Evgeny or Eugen. I am developed by D.D., and the talk is expected about
about why D.D., but first of all, I am going to start the talk from very important things about
rumors and our relations with Eugen by D.D., because actually there are two kinds of people.
Some people believe that we, Eugen by D.D., and why D.D., are exactly the same thing.
So, in this case, Frank and I are colleagues. Even more, some people believe that once upon
time we had a bar fight. And I am today, you are going to know the actual truth. And the actual
truth we had never had any bar fight, fortunately. And why D.D., and Eugen by D.D., despite
common characters in our naming, different distributed data bases. And we communicate a lot
with each other. And actually we enjoy talking about data bases, distributed systems,
and getting things done related to benchmarking. So, what D.D., is not just a database.
It's not just a distributed database. It's a platform. Originally, we started as early
p-system. But later, because there was a very high demand, we added support for Kafka-like
topics, servers, or a distributed queue. And recently, we have added support for all
app queries. And there are even more projects based on why D.D., but mostly, we are known as open
source distributed, SQL database. SQL database means that we are relational database.
We support LTP and the app. And we are major system. We have installations with
1000 servers. A partial license is expected. If you have a laptop or phone, feel free to go
to GitHub and start our project. We are strictly consistent. Because we take care about
developers, application developers, and strict consistency is really important for them.
It means that in CUP theorem, between consistency and availability and partition, we
choose consistency when partitioned. And our transactions have a reasonable execution.
We are highly available at full-tolerant as expected. We support nulls for a very big
source. Imagine situation when one availability zone is not available. It crashed.
Maybe a fire of water, something happened in the data center. And also, you lose a
rack. In another availability zone. In this case, WDB is still read right available.
Which makes us a mission critical database. So we have no downtime for maintenance,
etcetera. And fun fact about WDB. We are not just cloud native database. I call
this feature of database bootstrapable. We are actually bootstrapable because
there are some clouds, which are based on WDB. They use WDB to store their native data.
Also, even more, they use WDB to implement their own versions of the elastic block
store. And finally, they provide WDB as a service. And in this case, when you get WDB as a
service, you actually get WDB over WDB over WDB. So it's infinite number of WDBs.
But even more, you might know that inside almost any database, the risk LSM.
Now, imagine number of LSM's over LSM's over LSM's. And everything ends in SSD where
there is another yet LSM. So let's go to main topic. And start from spatial overview of our
architecture. As expected, we have SQL database. So we have tables. And each table has primary key.
Primary key can be consisted of number of columns. And tables are sorted by the primary key.
And we split tables into ranges. And there is a partition responsible for each range.
Also, we use slightly different naming for partitions. We use a name tablet.
Here is our logical architecture. At the bottom, there is distributed storage.
Distribute storage is responsible to store all users metadata and system metadata.
Also, it's responsible for redundancy, replication, it's a major component for consensus.
And there is a tablet layer above distributed storage. In this case, tablets are reliable components.
And they actually implement some sort of database, more like database logic.
All together with distributed transactions layer, tablets, and distributed transaction,
implement ACID distributed transactions. Between tablets, between different tables, et cetera.
But because we are limited in time, today we are going to focus on WDB platform components distributed storage and tablets.
Most important feature of our architecture is that we separate compute and storage right from the beginning.
There are compute nodes, which implement database logic, compute nodes contain tablets,
distributed transaction, query processing, and even GPC.
And storage nodes, they provide the fact that storage.
Our system works on share nothing architecture, commodity hardware, so it's not like spanner.
We don't require any special hardware, and separating compute and storage allows to scale them independently.
And of course, WDB can be run in virtual machines, in containers like Kubernetes or on-band metal.
This room is for cloud native databases, probably I should not say this, but I prefer bare metal.
Because no overhead, no, it's really better, no fake machines.
Here, illustration about WDB cluster.
Usually, each cluster contains multiple databases, and multiple databases, in this case they are called dedicated databases, they share same distributed storage.
In really huge installations, there are thousands of databases running in single cluster and sharing a huge distributed storage.
This is done for efficiency to reduce number of resources, because if you have separate, if you have per database cluster, it's really more expensive.
Because we are cloud native, and even cloud friendly, you can use dedicated database and share this database with multiple serverless databases.
In this case, such a database is called not dedicated, but shared database.
Still storage is still storage is shared between multiple databases.
So let's dive deeper and have a look on distributed storage.
Distributed storage is kind of where the special purpose key value store distributed key value store, and it is used to store in usable blocks.
Bloops are of different size. Bloops can be very small, just one byte, or 10 megabytes.
And tablets, which are located on a second layer, they use distributed storage for two main things purposes.
First of all, because tablets reliable component and you might guess it contains a big edge state machine, it uses distributed storage to write log records.
Or to read ranges of log records.
Also, because as usual, there is log structured merge 3 inside tablet, and tablets use Bloops distributed storage, or Bloops storage, two stores in the long blocks, which are actually parts of LSM3.
Bloops storage is actually an old title for distributed storage.
So I use interchangeably Bloops storage and distributed storage names.
Distributed storage supports multiple density schemes with support to register coding.
It's kind of very special in comparison with other databases.
For example, you can use WDB and install WDB inside single really ability zone, and you can use a regular coding to cut your expenses.
Because in case of a regular coding, you get just 1.5 redundancy, not 3 times redundancy, you don't have to store 3 replicas.
And in case of single will be released zone, that's a nice feature and it works really fast.
Why 1.5? Because as a regular coding, we use a block 4 plus 2.
It means we have 4 parts of Bloops written to the Bloops storage with data and two extra parity blocks.
More classic scheme is the application.
It's the case when you have 3 ability zones or 3 data centers.
And in this case, you get 3 replicas, everything is expected.
But it's designed in a way that you can add more redundancy schemes depending on your needs.
What do I mean by a special purpose key value store?
Key is a table of tablet ID, generation, step, and probably some other things.
It's expected from any distributed system you should have.
Generation, you should have step to solve consensus.
Value is a mutable block.
Put and get method I expected.
And I would like to tell you more about very special method called block.
It's used with tablet ID and generation.
One tablet wants to become a leader.
It uses block method to say to block a distributed storage.
Hey, Bob Storage, I would like to be a leader.
And this is actually a kind of, it's something between consensus and election.
Actually, it's kind of election but of consensus algorithm.
Well, Bob Storage answers.
Okay, tablet, now you are a leader in this generation.
Tablet can do useful things.
It can store data.
Also, another very special thing about our distributed storage is the garbage collection.
It's a barrier based.
So tablets use barriers to collect garbage and they must move this barrier to tell
Blob Storage that it's time to make garbage collection.
Let's have a look on this a bit of storage node.
Each is a bit of storage node, owns devices, block devices,
and manages these devices.
You might see that there is no file system in between device and WIDB.
We don't use file system at all because there are many benefits.
Main benefit is that we can do our own caching instead of virtual file system layer.
Also, here is this special component called padisk.
Actually, padisk is a component which manages device.
And we can run multiple virtual disk on single padisk.
We have a special scheduler which provides which is responsible to make use of device layer between different
WIDisks.
Distributed storage consists of distributed storage groups.
Distributed storage group is a thing which is reliable.
And it consists of unreliable virtual disks.
Virtual group, these groups are managed by Blob Storage Controller.
It's a special purpose tablet, which is responsible for managing these groups.
I think this is the way to think about distributed storage to think about it as distributed rate.
In my opinion, it is the best intuition about this component.
Tabit is located above Blob Storage group.
It uses special component distributed storage proxy, which is with the way about ways to communicate with virtual disks.
And hides complicity of using virtual disks.
Tabit, I attached one or multiple Blob Storage groups.
And this is actually the kind of API above Blob Storage group.
Virtual disk is active component.
For example, when you lose some virtual disk.
And for example, it's a way to sometimes then you get note back.
And virtual disk communicates with others.
It's a peer-to-peer system to get all the changes and to make synchronization.
So in case of failure, virtual disk will be back.
And it will be able to communicate with others in the group to make synchronization.
And important feature of distributed storage group is that distributed storage group is not static.
It's dynamic. You can easily move with disk from one note to another note from one device to another device.
And this is a bit of storage isolates users because easy-to-date database has own Blob Storage groups.
So they are isolated.
Usually they share same physical devices.
But you can depend on your needs. You can easily configure a system to not use same physical devices.
To not share same physical devices between a dedicated database.
So if the device is broken and replaced, it's possible to make the application automatically.
We have a self-heal process which actually is responsible for this kind of replication.
This is a bit of storage is very scalable because you can scale everything independently.
And Blob Storage controller is a single component, but it's very efficient and doesn't require much CPU.
For example, we have cases when we have 10K nodes and everything is okay.
Let's briefly overview the tablets.
Tablets are reliable components and encapsulate stateful logic.
They run and compute nodes.
And if Tablet dies, our window structure is responsible to start Tablet somewhere else.
And to restore it in the same state as it was before dying.
Tablet contains replicated state machine.
And it has LSM. So everything has good expected.
And Tablets are a different types of tablets.
And logic is above RSM and LSM. So it's easy to add new types of tablets.
Tablets communicate with Blob Storage groups via channels.
So channel hides a completely set of Blob Storage groups.
There might be multiple channels and some columns can be stored on HD.
And some columns can be stored on SSD depending on how you use the actual columns.
And here is the final slide.
Describing it actually contains a bug.
Sorry for this. I'm sorry. I forgot to remove extra text.
So there is that data shot tablet.
It's responsible for raw store and raw store-related SQL queries.
We use own WAKEL language, which is very similar to SQL.
But also we are post-rescompotable to almost post-rescompotable.
We have colon shots. They are responsible for colon store.
And there are tablets like Schimshard, which managed all the metadata.
All the Schimshard's very special tablet called Hive, which managed established.
It's responsible to start, stop balance tablets.
We have coordinators, mediators, transaction allocators.
Because WGB is originally inspired by Calvin.
So it's our similarity with Calvin.
And things like class and management system, this view process, etc.
Thank you for your attention. Please ask your questions.
Feel free to hit me up outside conference.
Here's my Twitter.
Feel free to ask any questions.
Subscribe to me. I will be happy to answer.
You mentioned UDB is open source, right?
Yeah, to license.
It's like all stack open source, including blob storage and PDS.
You mentioned that is all that.
Yes, everything is open source.
Including SDKs and all the layers are open source.
Would they be able to run it, say, on AWS Cloud?
Yeah. Okay, thanks.
Anyone else we question for you, Kenny?
How good?
One more.
Hello, again, it was nice.
It was kind of boring start, because I hope you will tell more about bar fights or something like that.
But still, it was kind of impressive.
I just have one small question.
You say about duplication of data, like a couple of algorithms to do this.
What about the duplication?
As far as I remember, you say about key value storage.
We're kind of simple one.
Are you using some kind of duplication?
To the extent of my knowledge, we don't use any duplication.
So, application applications can do the duplication on their own.
Hello, this is my first time hearing of YTB, and thank you so much for the amazing talk,
and I'm walking us through it.
I would like to ask you, would you recommend that I should be considering using this for my next project?
Or what is a good environment to consider?
Could the piece repeat this kind of noise?
I was saying that this is my first time hearing YTB, and I was curious from you.
Should I consider using this at my next project?
Should I consider introducing this to my company?
Or is this something that you expect to be really a community thing?
Well, I think it's really easy to use WDB externally, because we have external clients,
and they use WDB without any issues, and also they checked if they can scale WDB.
And you probably would like to check our blog to see performance numbers to understand if it matches your needs and requirements.
Thank you so much.
Thank you for your question.
Any other questions?
Cool, thanks for giving it.