Okay, so I'm Matthias.
I work at Pincap, coding on TIDB.
Let's see what's at it.
How many nodes would TIDB, by the way?
Okay, that's good.
I'm going to talk a bit about my SQL vector as well as AI.
What's all the boss around that?
How does it work?
So there's a lot of abbreviations and words
into an LLM-Rag AI vector.
How does all this connect?
What can you get out of that?
How does it work in my SQL,
another mySQL compatible databases?
So the first part is just the AI part more or less.
And the second part will be mySQL focused.
So we do have this LLM,
we have semantic search, vector embedding,
vector search, drag.
So what is all this about?
How can you actually use it?
What can you make use of this?
So let's start with just the large language model, the LLM.
So it's AI, generated AI, the statistic model more or less.
It's trained on massive amounts of data.
It works, it takes some input and generates outputs.
So that's where the generated AI comes from.
Once it's trained, it's not up to date.
So it's a static model more or less once released.
It made me specific content because it's trained on specific data.
But it also would never have access to your own content.
For example, you might have an internal database or an internal knowledge base or something like that.
That the LLM has no context.
It's not aware of that.
And it will also generate an answer even if it's not trained on that.
So it might start hallucinating.
So you knew what TIDB is.
Here I downloaded an installed Olamma.
And I run the smallest deep seek version.
I think most of you have heard of the deep seek the last week, basically.
And I asked, what is TIDB?
And it starts quite good.
It's a model that in memory data base, blah, blah, blah,
PyMongo, Mongo library.
No, that's not actually what it is.
So let's go up a bit in the model size.
So this is a distilled model with 1.5 billion per meter, I think.
So let's try with the $7 billion per meter.
And then it starts to answer in Chinese.
And it's quite easy.
So you can ask please reply in English.
And then of course it will reply in English instead.
And since it's likely bigger model and it's this reasoning AI,
it starts to do some reasoning text, blah, blah, blah.
And then it comes up with TIDB is an open source distributed database.
blah, blah, blah, blah, blah, blah, low latency high throughput.
And then it comes to, it's built on top of the bulk language,
and it's rooted in runtime offering un-unified framework SQL
and bulk-based operations.
And how many using the bulk language here, or bulk-based operations?
Well, I actually used the bulk this morning.
So it actually runs bulk-based operations,
because bulk is using TIDB.
So I do think that's where the connection comes
and through distilling and so that's the connection.
But this still doesn't help in explaining what TIDB is.
So we can go to an even bigger model.
So a 32 billion parameters.
That is really good, actually.
And it's not just, that's a lot of reasoning text and so on.
And then it starts to spew out.
It's an open source distributed database,
the sign-tandle-post, transactional and analytic work load
efficiently.
So it's a solution for large scale applications.
And then it has much more about the different components and so on.
So that would be more or less similar to just look out our website
about TIDB.
So it all depends on the model and the elements,
but what I wanted to show you here is that it will start hallucinating
if it doesn't have the context.
If it doesn't know about it, it will still give you an answer.
And it will sound fairly okay.
So then the next tool in this toolbox of AI,
and so on, that would be semantic search.
And that is meaning that search what I mean,
not literally what I say.
So by literally, then you can use a full text search.
If you want search for anything that mentions automobile,
then you can search for automobile and map that.
But often, if someone says automobile, you probably
mean a car, right?
So the semantic search is to take the context
or the mere kind of meaning of your question
and answer that with something that relates to this.
So not just word for word matching.
And how that works is it's usually taking
a multi-dimension vector to represent the meaning
or something that relates to your object that might be text,
but even the image or some other kind of object.
And the important part here is that it uses tailored models
for generating these vectors.
So depending on your area for search and so on,
it's a model that is used that will say,
if it's a good match or not.
And then from those models,
that's where all these vector and vector embeddings comes from.
So vectors, just stored even like in reference table
or a specific vector database or anything like that,
but it can have a foreign key or whatever,
to your actually content.
So you're embedding these generated vectors with the actual object.
And that's how you use for doing the semantic search.
So you have some other kind.
You can have a question.
You want to say, can you find the image similar to this?
Then you would actually create these embedded vectors
through the tail or model,
and then you would compare these vectors.
So the result of the vectors,
if they are closed by some kind of distance,
it would mean that they semantically are similar.
And these vectors, they can have thousands of the dimensions.
So it's not just a single number.
It's quite a lot of data for each and every of these vector embeddings.
This is how it would look.
If you use a MySQL client,
just connecting to TIDB,
it's similar in the other ones.
One particular thing with TIDB is it actually
implicitly converts from a text string containing vectors
directly to a vector of back and forth.
So you don't need to have the translation function.
I will come back to that in the second part of the talk.
So here you can just insert vectors,
and then you can do a search for the vectors.
And here is the important part that you order by the vector distance.
So that would mean that the closest one with shortest distance
would be more similar than the one with a bigger distance.
And that's what we would be called vector search,
finding the closest neighbor vector.
And for small data sets,
then it's perfectly fine to do a brute force.
Computers are super fast.
You can run millions of computation.
And this is simple data actually.
So you can even have the vectorized computation
with the single-instrict multiple data instructions and so on.
But when it comes to larger data sets,
then of course it's thought to be inefficient and take long time
and be expensive.
So that's where an vector index comes into place.
And from a database perspective,
to me it was really weird when I started to play with that.
Because without an index,
I would have the exact order.
I would have the 10 closest one as they were.
But adding an index, I might get just miss one that was closer
or get one that was not as close.
So it's an approximate indexing,
which again in databases for me that work with database.
It needs to be exact, it needs to be transactional.
Otherwise we're doing something wrong.
And here we're doing it approximately.
So it's actually kind of okay, but it's not exact.
This is just a way of running a plane.
Again, tightly-bears an example.
The last line you can see that the access
or the activity is actually an index.
And further under the operating operator in for you can see
that it actually has an A and an index.
So approximate nearest neighbor index.
And that's what it would be using.
And then now when we have these different concepts,
how do we put that all together?
So we have an LLM that will generate an answer.
We have documents that we connected vectors to.
We have models that match these objects to vectors.
And we have the vector search for finding the closest ones.
So those that would be similar as similar as possible.
And then we can use the result for finding these similar objects.
And we can feed that back to the LLM as context
because it might not know anything about the question or the area you had.
And by adding these context, the LLM can actually give you an answer
with your private information about your company's
sensitive documentation.
That's just used internally and so on.
And it can actually give you a well formulated answer
about knowledge that you have that the LLM model did not have.
So that's what called retrieval augmented generation.
And visually you can say that you have a user query
to start with and you want the nice formulated answer.
You create a new vector from the query.
So you have some kind of representation for what the query
semantically would mean.
And you want to search the database,
but the database needs to be populated, of course.
So that's a big area of what a lot of people in AI
does now that how can you take documents or whatever you have
and search for them.
So you have different models for generating these embedded vectors.
Usually if you have a big document and just creates a vector out
of a big document, the matching will be quite poor.
So this is a big area in a lot of trial and error for their specific context
and so on.
But the overall picture looks like this that you have your own data
and you create embeddings for that so you can search and match for the query.
And then the better matches you're feeding that back to the LLM.
And it's as easy actually in the end as what can be called as prompt
and you nearing as well.
So instead of just sending the user query directly to the LLM,
you will end up sending the user query to the LLM as
please answer this query and then you just quote the query.
Based on the context of these documents or these snippets that you got
from the vector search, the similarity search.
And then you push that to the LLM,
and it can generate really nice answer with context from your data.
That the LLM was completely unaware of.
So that's the whole AI part.
I will not go into deep depth and so on more about that.
Then we come to the other part what's implemented in my SQL
and my SQL compatible data basis.
It basically needs two things.
It needs to be able to store the vector,
and it needs to calculate distances between the vectors.
That's all the functionality you're actually requiring from the database.
And there's a lot of specific vector databases and so on.
But if you already have a database,
why wouldn't you be able to serve this from your already existing databases?
So the vector data type,
that's introduced in my SQL 9, I think.
MariaDB has that in 117, I think,
tied to be added in 8, 3.
I think, or something like that, 8, 4.
It's released in LTS version, at least.
It's normally a fixed number of dimensions.
It can be now all implementation I've seen is a vector of 32 bit floats.
It's only useful as a whole vector.
You never use the individual values or anything like that.
And it's specific to the model you generated it from.
So you need to parse it when you're inserting.
And this will be coming back in the rest of the pages,
because there's so many different function names for the same thing.
So I will ask around that.
The type to be can do it implicitly.
It can see, okay, this is probably a vector from your string.
Otherwise, you need to specify that this is the string.
It's a vector, use it as a vector.
You can even, in one, in the different implementation,
maybe use hexadecimal directly or even binary.
Getting it out, if you've got select where you have a vector column,
it might not be quite useful to do that in most places,
because you have a thousands of vectors.
So for application and so on, of course, it makes sense.
But as a user, it's more as useless.
So it might spew out just binary data or hexadecimal data
or already parsed as the string again.
And as you see, a lot of different ways to convert,
or that different function.
The weird part is why was not just cost used.
It's a simple data type.
There's not a specific way to get to form a vector like
in your medical.
So that was a bit weird.
Is that something that really should be standing for it?
There's no as well, standard, yes.
Yeah, but if it was, yeah, it was simply simplified things for everyone.
Yeah, I actually saw the TIDD actually have the implemented cost.
But it's not documented for, I think.
And then this is a core part.
You want to have a way of getting the distance.
Several ways of calculated distance and for vectors,
the two main parts, the ways is the cost in the distance,
would more or less say which angle would be between this,
if you can think of an angle in multi-dimension world.
Or you could even that would be a straight distance from one point to another.
There's a few others as well.
And then yet again, what function?
And what would distance mean without which type of distance,
would that mean?
It's a late thing in MariaDB, for example,
that will use whatever distance the first in vector indexes.
So it's a nice thing, maybe, but it also doesn't.
It can fool you as well, I think.
And did you only look at Maria, TIDDD?
I'll come back to that.
And then there's some vector utility functions for dimension or length of a vector, for example.
And then the core of it, can you speed it up?
Do you actually have a vector index?
Is the vector index transactional?
So you can see it within your current transaction.
Is it maybe a synchronous, so it will try to follow,
but not have the very, very latest and definitely not your current transaction?
Where is it just manually updated?
Can you configure it more than your type?
And then again, a big different syntax.
For creating a vector, I don't think that the syntax is as important as being more compatible
on actually using that in the column.
So that's not an issue, I think.
The different databases, MySQL by default.
It's in 9-0 and forward.
It has the vector data type.
You can specify it just a vector or vector with a number of dimension.
If you don't say anything, it will be by default,
20,000 dimension.
The weird part here is it treats it's like a worker.
So you can have up to that number of dimensions.
We doesn't really make sense because you would have a normally fixed model
that has a fixed set of dimensions,
but you can still insert fewer dimensions.
There's not an error warning for that.
So that might be something you can take back.
One set of function names.
But then you can't actually have a distance.
They document it clearly.
There's a distance function.
And then you read the fine print.
You don't have the distance function in the community edition.
You don't have that in enterprise edition.
You have that in heatwave.
I'm not speaking about that here.
That also means you have no vector index.
So you can only store and retrieve the vectors in my SQL.
But then next session we'll talk about my vector.
So we'll just touch upon that likely here.
It kind of adds the missing stuff for a standard my SQL.
It's actually really easy to build and so on.
But I'll leave that to them.
That is your introduction to it.
It uses the audit plugin.
He query, right?
Ten.
And yes, that's fine.
I like that.
It's an in memory index, I think.
They add the distance function as you see.
And they have a different way of searching.
And I'll leave that to the next presentation instead.
Then you have a MariaDB 11.
They only allow you to specify the vector data type with a specific number of dimensions.
We kind of make sense actually.
They have their own sets of function names.
We have a vector distance without type.
That's nice.
But is it helpful or not?
That's actually an 8118, I think.
They do a vector index.
The implementation of that.
It's an Indian independence.
But they only support my in a DB for now.
And if you do need a MySQL compatible database.
And MariaDB would be similar.
And it will support vector index.
And then type DB.
That's open source.
The scalable database and so on.
It also supports the vector data type.
Either you can set the fixed number of dimension.
Or you can have it unspecified.
It can be useful if you just want to store different ones.
And per row, you would say, which type of model.
And so on.
So in some use cases, it's possible at least.
It supports implicit conversion from text to vectors back and forth.
So it's a bit nice to use by the command liners.
But that's a bit different.
And more or less always an application that would do uses.
It has own sets of function names again.
And it has some other things.
You can actually use comparison directly on vectors.
And do some arithmetics.
And so on that as well.
So you can do that.
If you have those kind of needs for the AI and rate of AI.
And that probably not as useful.
But that's something that was implemented.
Maybe for testing with uncle that.
But it exists at least there.
And then I do have a bit of cost here.
And I'm not sure.
Are there anyone from MariaDB here?
OK.
Then I can also directly at least to my SQL.
Can we please, please, just agree on the function names and so on?
But what about a postgrace or DB2 or a SQL server?
And thank you for joining us.
Yeah.
I did a quick check.
The vector distance.
That's also what Oracle supports and SQL server supports.
I think that the master on it will provide its vector distance and distance on both.
Is it actually a vector distance?
Oh, yeah.
I couldn't use it.
I can try it.
OK.
It was vector distance first.
Can they make it to distance?
Yeah.
But since I didn't talk about open source software here, I couldn't use it.
Yeah.
And then why not just use cost for costing back and forth because it's quite.
Then I'm open for questions.
Yes?
So which type of index are you using a type of DB2?
Yes.
I think both type of DB and my vector is all using HS announced.
So here I'll get some small world navigatable.
And so I tried it to do the method play with it.
But it stores also the full index and the full vector index.
Yeah.
And do that or are you able to not do that?
Because at the end of my index as big as it is even bigger than my data, right?
So as a DB8 like you actually like that.
Yeah.
No.
That's something I'm more or less.
Well, it depends because normally you're not storing the full index.
You're actually storing like you're going down to, I think, MariaDB does in 16 or in 10 or something like that.
So you're compressing the data a bit as well.
I'll leave that to the next talk as well to go into some of the details in implementing implementation.
In TIDB we are implementing it in the component that does columns in the store as well.
So then it will stream the data through a raft.
So it will kept always up to date.
And in that part of the architecture that we're all implemented vector index.
Those are presumably the data store mainly compressed and you like to start from a storage perspective.
Yeah.
But that's with a limited time I couldn't do too much.
In high level they use the same kind of algorithm for the indexing and it's all approximate.
So that's again something I wanted to stress that it's different from a data perspective.
You have to use a type flash to use vector and vector it is.
Yes.
I don't know but to create a feature request and we can look into it because I don't think that's anything that's actually hard to change.
Yeah.
And I think it's a post recess half, half vector, some half.
Yeah.
And this is very, very new.
It's not GA in any of these database what I know.
And like the distance function in postgres and Oracle also uses this less than equal, greater than
variance of that.
And that already is taken in my scale for no safe compare I think.
So that's why it needs to be a full functioning.
There was something else.
Yeah.
But the bottom of the form of the comparison is the use of vector.
I saw some of these outcomes and I didn't expect them to search anything.
So for performance I would check Mark Kallahan that has done a good bench
marking between MariaDB and PGVector for TIDBI only seen internal benchmarks against other specific vector databases and so on, which is comparable.
But it's performance in this case is also how exact it is.
The call back rate so how exact was it as well as raw time performance.
So yeah.
It's still a bit of try with your own for a while.
Yeah.
This is more I level.
So you show a earlier that you create a bindings from data as a vector.
And then you create a bindings when you create you from the questions.
Yeah.
Was it asked to use the same model?
Well, you don't use them that when you're creating the prompt.
Then you already have the document.
Then you already use the semantic search for getting all the context and everything.
So when you are actually sending the question to the LLM, that's not anything with vectors and so on.
It's all about the query as well as more context.
But the query you created as a vector compare.
Yes.
Yeah.
So this should use the same.
Yes.
Definitely otherwise you can then it's not even apples and oranges.
There's a new one.
You always have to do all your documents again.
No, no, no, no.
You use the same model, right?
So you're.
Yeah.
Yes.
So you can all the vectors to done with one model.
It's completely useless if you use another model.
No.
Maybe I shouldn't ask too much.
We can.
Or answer questions together after the next one.
Yes.
Thank you.
The sweet thing.
Can you do something about the fact that these vectors are straight?
This can be done.
There are.
When you you're.
Yeah.
But that's just how you translate.
That's what the cost function would be for, right?
It's not story.
It's all of them are storing 32 bit floats packed.
The box should be set.
The mathematical way.
It's not the screen right?
How would you do that?
You can do that as binary or hex, which is just a bit for the processing of what you
typed.
How would you express that as a string or as binary format?
Well, I don't have to single force.
I have often write it down to 1, 2, 3, and...
OK, so it's the quoting after you are...
You can specify the user, because it's a mobile bit, an implementation.
Yeah.
To make it faster.
But it's not natural.
So if you would create a scratch.
But, I mean, would you actually type in a thousand of numbers?
Or would you have an application that just sends it?
You can read it from you.
They can write a file.
Yeah.
But then you would use an application that sends it a text, and then you don't have any quoting.
Or you would already parse it and send it as floats.
And we are also typed to what the model responds.
So this is the answer.
We'll get this a string, of course.
Then this is the string I received from the bottom.
We'll get the binding.
We'll get that, and then you send it to your database.
And you get it in a string.
And something I just want to say that, if you want to play with this, if you have your own documentation,
we actually released an open source out of flow, which is super easy to create your own AI assistant or
conversant search page.
So we have this, we call it the TIDBAI as a demo more or less.
Where you can use for asking anything about TIDBAI, that will be generated from our documentation.
Yeah.
And here you can choose.
You can, it's a lot of JavaScript and so on, but it's a great website for you.
You tell it which model you would use.
You point your own documentation and so on.
So it's a whole framework for actually doing it.
It makes it super easy to play around with it.
And it might actually be really useful if you have internal documentation or whatever you want
to add an AI functionality to it.
Thank you very much.