So, hello. I'm here to speak about data and Kubernetes, state of the search. I'm
Matthias, I'm from Belgium, and I work at Blenscale. To start off, like, what is Kubernetes?
Like, we're in a database room data. We most of us will understand. Kubernetes is an
open source framework for management, deploying, and scaling of containerized applications.
And then containerized applications, it's something that you pack together with all the
dependencies and everything it needs to be lightweight, to be self-contained, and to be
portable between different machines. Because in Kubernetes, you most likely will
separate storage from compute. So, that brings us to stateless versus stateful. In the
stateless world, each request is independent, and it's an isolated transaction. Think
about, like, an XP web request, they don't know about each other. They don't store any
history while a stateful service is something where you want that history to be stored
and to be retrieved. And so, in many cases, you'll have that storage component attached
to your compute. Something about Blenscale, the company I work in, it was found in 2018
as a by-the-founders of Blenscale, where the Vites developers originally, that left Google
when Google decided that the workload at YouTube was running, need to go to a different
product. And then, in 2021, we changed our business strategy to become database service
product, where we basically were running managed Vites in the cloud for people. And since
last year, we also run postpress. And important for us is that everything we do as a database
company runs a Kubernetes. So, as we are in the business of saving people's data in the cloud,
we are expected to store the things. Not everyone runs on the black hole engine, which
is a little nice growth hook. And database can become quite large. So, we are in operation
side of the database. So, we do backups and we do failovers. And as the data grows, that
become all very slow. And when you run in the cloud, you will most likely be running with
the cloud block storage, like EBS in Amazon, or persistent disks in Google, or any other.
And these storage solutions are all network attached, which makes them very slow, high latency
for every right operation and every operation that you need to search for. So, what we
decided to do somewhere last year is we decided to run everything on the ephemeral storage
that the cloud instances offer. It's available both in Amazon and in Google. But it's
not meant to be stateful, because it's a very fast storage. That is used for scratch-based,
like if you have a processing work line that needs to store intermediate states. You
will store them on the instance storage, and then you can continue processing. If you
lose that data, you can just restart your process. Well, if you run a database, you don't
want that. So, this storage is lost when you destroy the machines, so if you shut it down,
that's storage is lost and you cannot get to it anymore. So, we name the product that
we developed from this plan scale metal, because we are running on the ephemeral storage
in the instances in the cloud. And this make your database run really fast, but virtually
unlimited IOPS. There is a limit to the IOPS that these devices can have, but with the
workloads that we have been running, you cannot reach that limitation. We will do a multi-dase
deployment, minimal 3AZs for a production workload to ensure that we can deal with a full
AZ-outage. And for the durability guarantee of your data, we will use my SQL stemizing
replication, or if you're using postgres. We will have at least one synchronous replica in
another AZ. And we require the cross AZ acknowledgement in the storage in the stemizing
to make sure again that we can deal with the full AZ-outage. We make daily full backups
and verification to the object stores on the cloud. So, every backup to restore the previous
backup and then validates that is valid, that you can replicate from it again. And if we
need to do an upgrade to the database or any sort of rolling operation, we will first search
new instances. So, you can keep the durability of your data before we will remove any
old instances. And that was it.