So, welcome. My name is Soryne. I'm the CTO of a small company called OmniConvert. We run a
SAS, which essentially is an experimentation platform. You can run AB tests, overlays like
top-ups, surveys, or with segmentation, and of course performance tracking. For this we use
events, we call them goals. I'm here today to discuss about the migration of our database. We
migrated from an old version of MongoDB towards a new version of TADB. And just to have a context
of our size, we, on average, run like 18, 19k RPM during the day with peaks up to 25k. And
at the moment of the migration we had like 3.5 billion records in Mongo 80% of them were in the
two largest collections. So, today I would like to briefly discuss about our context, why we chose
TADB. And then go through the migration phase, starting from POC up until the actual project completion.
And discuss about the end user perceptions as well. So, as I mentioned, we had a really old MongoDB
it was a Mongo 3.4. We had lots of issues with it, but first and foremost we had terrible
query performance. So, whenever running reporting, it was almost dead slow. And we had no
easy way to scale out, neither in terms of multiplying the numbers of the number of servers to
process nor the capacity to be stored. And of course, being sold, we didn't benefit from security
patching anymore. And we weren't able to do like migration or like an upgrade to a newer version
of Mongo because it was sold and there was a deadlock between the version of the database and the
version of the driver. So, we were obliged to upgrade them at the same time, effectively rendering
it as a migration. So, we decided to take a look at other systems as well. And we had some criteria.
First and foremost, we wanted to solve our issues. So, we have better query performance,
better reporting performance. Obviously, to be highly available, we wanted to solve the horizontal
scalability as well, scaling out. We wanted the budget of $1,500 per month, but just for context,
we're paying like $1,300 per month now. So, this was not really feasible goal. But we learned that.
And very important, we are very, very small teams. So, we cannot afford to have like dedicated
people maintaining the database. So, it was crucial for us to have it deployed and maintained by
developers. I'm not going to get into the details, but basically from that list, TAB was the only
one who picked all the boxes. Sonny presented earlier, the architecture of the TAB, so I'm not going
to go into this to deep, but essentially, there's load balancer, which queries some TAB servers,
which are stateless, so they don't store any data. And there are two types of storage TAB and TAB flash,
sorry. TAB is row oriented while TAB flash is column oriented. And there's the placement drivers
who decide where the data should sit and rebalance data for performance and also size,
whenever scaling out, for example. So, before doing the actual migration, obviously we did the
POC. We wanted to test two things. How fast does it run an individual query and how good does
it scale in terms of loading? Load testing, sorry. I have here a very simple query and obviously
for MongoDB, it was the equivalent run. It was run on one of the largest collections and as you
can see, the difference is absolutely huge between the time running this simple query. This is
the simple query but grouped by day, so we also have some charts showing the variance by day.
And again, TAB basically crushed the MongoDB that we had. So again, this is a MongoDB 3.4,
it's not the latest version. We also wanted to do like a full experiment cache. We have some
predefined periods for which we cache various data like past week, past month, past year,
something like that. And again, we saw more or less the same factor of increase in speed.
And whenever it came to running a custom report on a custom period like runtime query,
so no cache beforehand, we have even bigger of a difference here. So, this was basically a
no brainer to try to switch to TAB and move forward. So, we went on to the stress testing.
Before we scaled out the stress test, we wanted to determine what would be the optimal parameters
for running a single container, so we can then scale this out. And we run the application which
in just data in PHP, I didn't mention that. Surprisingly enough, we found out that running
free workers per node, per container, sorry, was the best efficiency ratio. And we managed to
have a pretty decent response time as well. So, we went on to try and run this at scale,
covering like four to five times the traffic that we had on a daily basis. We ran the test with
Grafana K6. And these are the results from new relics. So, basically, it shows that for a longer period
of time, I think it was an hour here. We run on average of the target for the test and also having
a very low latency, like 15 milliseconds or so, which is basically the same time as we have in
production. So, this was a success. We decided to move on and the plan was pretty clear. We had to
rewrite all reporting queries from MongoDB to my SQL to SQL, sorry, compatible. And then also
the queries which inserted the data and updated some data as well. We decided to have a system that
would run both of the databases in parallel. So, we are sure that we don't lose any data and
the data is correct within TIDB. So, we created the flip-able switch. First, we created some scripts
that migrated the data and this was done. And basically, it was creating insert batches for being
run for them to be run on the import into TIDB. We did this up until a certain time stamp and
then we deployed the feature which essentially has started to write in both databases. And then
we, this is for phase two, we basically covered the gap in between those two time stamps.
For approximately two months, we stayed at phase three which was monitoring for differences and
we ironed out the queries and fixed all the bugs. And once we were confident, we basically
inverted the two databases making the TIDB database, our primary database and Mongo was just
kept for backup purposes. And after one additional month, we dropped that as well. This was like
eight months ago. So, we were very happy with what TIDB has offered this.
A few things that we learned, there are a few flags that can pretty dramatically impact performance.
They're very well documented in the TIDB documentation. So, no issue there.
And another thing is that joins will not go on as expected. So, at first we were so excited
about the performance that we have loosely written some queries to have like multiple joins,
but this did not perform very well at scale. So, we tried to rewrite and succeeded a bit afterwards.
And another thing to mention is the fact that you have a multi master setup. So, whenever
inserting rows, you need to be careful whenever having like a select plus insert combinations.
Another thing to mention is the fact that TI flash, the column oriented storage is largely
flatline in terms of its response time for a query on a large data set while the row
based storage TAKV scales accordingly as expected. I'm very happy to say that this was a successful
project. It was a very bold project done in three people team. And we completed it in less than
six months from start to finish. We had no major issues and we also tested the highly availability
of the system because we changed some servers throughout. So, it really performed very well as
advertised. And also, their support was very, very prompt. Thank you, Daniel.
Thank you.