OK, so let's start.
It's already after noon, so we can talk about beer.
So I want to present today a talk on how you can use Postgres to create a recommendation
system.
The idea is to use beer as an example, but if you don't like beer, you can easily prepare
something like that for juice, tea at a place, movies, etc.
So I also, the general concept, then you can use it yourself to start your own simple
recommendation system.
So first slide about me, my name is Anjjino Vitskyan from Poland.
I have roughly 12 years of experience as a DBA, mostly Oracle, but since eight years, I'm
also doing Postgres administration, and I'm a database engineer at CERN since 2020.
So yeah, I've also contributed a little bit to the Postgres.
In 2018, I found a memory leak in replication managers, replication manager by EDB.
So there is a, there is proof that I actually contributed a little bit, but yeah, I wouldn't
call myself a contributor.
It was just very simple.
So at CERN, you probably have heard about that.
We are a European organization for nuclear research.
We host, I think, the biggest laboratory in the world.
So here is a short video to show you what we have.
In terms of IT department, we are roughly 400 people, and we are, let's say, I think, like
more than 10 people here from CERN.
So we have a booth, as well, if you want to check it out, I will also mention the booth
at the very end, if you want to know where it is.
And of course, for the amount of data that we are creating at CERN, we need to have databases.
So my team, we are responsible for making sure that the databases that we are offering
are working, essentially, that we have backups, that they are highly available, etc., etc.
So in terms of the databases, we have Oracle, my SQL, Postgres, etc., I think the biggest ones
are Oracle, because we started with Oracle in the 80s.
And then, of course, we try to share the knowledge, one of the concepts behind CERN
is to have the outreach to share the knowledge that we create, some of the software that
we create as open source, hence we are here to share, let's say, the information.
So, as I mentioned, databases at CERN, we have more than 100 Oracle databases since the 80s.
And then since 2011, we have open source alternatives, let's say, so we have 600 instances
of my SQL for 100 of Postgres and 200 of the influx.
This is not going to be covered at this presentation, this is just to present a little bit
what we do.
We later, at the booth, at one point, so if you want to talk a little bit about that,
feel free to check it out.
And just to give you an idea of how big is the size of our environment, we have more than
five petabyte of data and Oracle databases, 150 terabytes of data in the DBOD and roughly
three petabytes of backups, so that's quite a big, quite a big environment.
But this talk is not going to be about that, we are going to talk about PGVector.
So let's start by building a simple Bureau Recommendation System.
But before we do it, I need to have a disclaimer that I'm not here officially, it's not
what we do officially at work, of course, if you want to, if you want to drink that's your
responsibility, I don't want to promote excessive alcohol consumption, so let's just
make mention that, but let's start with vectors.
Let's start with a little bit of theory.
So in AI, a vector and also in mass, the vector is a list of numbers of scholars that represent
a point in the multidimensional space, and if we write it down as mathematically as you can
see here, and is the dimensionality of the vector.
So let's say we have two dimension vectors, so this would be just two scholars.
If it was 300, then we have 300 numbers that we keep, we store as an array.
And then we need to introduce the concept of embedding.
Embedding is essentially it's a numerical representation of an object that we know from the
real board.
And the machine learning and artificial intelligence systems are using this numerical representation
of objects to understand the world around us.
So for example, a bird's nest and a die on them are similar, and they are analogous as
a pair, while the day and night would be the opposite terms, and embedding simply convert
those ideas into numbers, and in order to create embeddings, we need to have a model.
Embedding models are AI machine learning models that we could use to convert from any type
of input.
I will mention it a little bit later into this vector, and the vector is the thing that
we will store in our database.
So we can imagine that we have some input, and it could be something like a movie or a picture
or some text, maybe an audio fragment with some voice.
We use the embedding model to convert it into the embedding, so we will get the vector
so the array of the scholars at the end.
So for example, let's switch to beer, again, we will have a description of the beer with
might be citrusy with a sweet aroma, and then our embedding model will convert it into a vector
so a set of numbers, a set of scholars that we will keep in our database.
So of course, this is a very simplified idea of what's going on.
If you're input is longer, for example, if you want to embed a full document, something
longer, you would cut this, you would cut this text into smaller times, there are many
different ways to do it, so we will not be covering that.
This is just a very simplified version of what the system like that is doing.
So you can see that with different input, we will have different outputs.
So for example, our embedding model will create the vectors based on the input, and probably
we will get different answers, different values for different results.
But the general idea, the general concept is that similar inputs should give a similar
result, similar values in these vectors, and we could somehow calculate the difference
between the values that we get, and we can use it to find similarities in our dataset.
So our recommendation system will be super easy, it will be super simple, it will just
use the similarity of the vectors that are there.
So how do we calculate the similarity?
It's assumed for simply this sake that we have a two dimensional vector, and let's
say that the x axis is sweetness, the y axis is the number of x of our object that we want
to, that we want to, let's say, convert into a vector.
So we have dogs and we have cats.
In our embedding model, they should be somewhat close to each other, they should be similar,
we can calculate the cosine distance between them.
So here, let's just connect from a line from the, our starting point of our space.
So from 0, 0, we calculate the cosine value of the angle alpha, and we can say that it's
quite similar.
In comparison, bananas are also sweet, but they don't have legs, or they have less legs
than the animals.
So they are somewhere else in our space.
So we can calculate the angle again, and we can see easily on this slide that the better
angle is bigger than the alpha.
So the distance is bigger, so similarity is smaller, so we can claim based on this information
that we got, that a dog is more similar to a cat than it is similar to a banana.
This sounds trivial to us, but of course our computer doesn't know by itself the reality
of the world around us, and essentially the same thing will happen in our similarity search,
but the model that I will be using, so the model that I'll present a little bit longer,
a little bit later, will have 384 dimensions.
So essentially, the same calculation will be done for us, but using the 384 dimensions.
And of course there are some caveats, some limitations, so for example, we can also say that
some other animals in our space are also sweet, they have less legs, and it's not, if we calculate
it just in one, if our vector was just too dimensional, we would get very weird results.
So this is why we need to have more vectors, and they will help us to calculate the distances
more realistically, and we'll get back to that a little bit later.
So there are of course some limitations of the similarity when we calculate it, when the
vectors are somewhat limited.
So for example, if you think about it, healthy and unhealthy, very basically intuitively
for us, this is something that's completely on the two opposite sides of how world functions.
If you drink coke, it's probably going to be unhealthy, but drinking water is healthy, so
they are on two ranges of the same scope, but for our model, they might be somewhat similar,
and this is actually the case.
So for example, with the model that I'm using, the healthy and unhealthy are somewhat similar,
healthy and not healthy are even more similar, and this is because both can be put in the
multidimensional space as adjectives that are related to the health status, so in a way there
are quite similar to each other.
So this is for example, the limitation of the model that I'm using.
But for example, I don't get a banana, I definitely less similar, but for example, there
is some similarity, because one and the other one is an object, as opposed to being, I don't
know, a concept, just an idea, not an object in a physical world, and we can also use it
to calculate distances between even full sentences or longer pieces of text.
So for example, I like beer and table partitioning this animation feature of RDBMSs,
is not similar at all, because if you think about it, yes, it's completely different topics.
But for example, I like beer and I like indexes and databases, this is already a little
bit more similar, and I like to index my data, and I like indexes, this is quite a lot
similar.
So with that in mind, we need to make sure that we are aware of this limitation of this,
let's say, simplified idea of the model that we might get some results that could be a
little bit good, not ideal.
But this is the reality of the world that we live in, so the opposite is not well-defined.
What is the, we can have a discussion like, what is the opposite of King?
Do we have any ideas in the room?
Yeah, maybe president, maybe Queen, you know, it's the first thing that came to my mind
was Queen, but also Prince, or maybe a poor man, or maybe a banana.
So it's trying to implement, trying to convert, like multi-dimensional world that we live
in into one dimensional space, trying to put, trying to simplify it into one number, that
is virtually impossible.
That's why we will have these limitations, and for these limitations, depending on the
model that we are using, it will be different results.
So of course, how we choose your model is going to make a big impact.
But how do we handle the vectors in the database?
Now we have a little bit of theory on how it works, how what are we trying to achieve.
So let's have a look.
One of the ideas on how we can do it, it's PGVector.
So PGVector is an extension that was written and developed by Andrew Kane.
I think he did like more than 99.9% comments to the repo.
So it's quite amazing what he has managed to achieve.
But it's an extension to progress that we can install easily to try to do nearest the neighbor
search to try to do indexing.
We can do single precision half precision binary sparse vectors.
So these are different types of vectors that depending on the model that we are using, we
will get different output vectors with different definitions.
So how to start?
We need to build the download extension.
Of course, we need to build it with post-respineries, or we just download the binaries
for me, it was very easy on the market.
It's a part of home-group, so it was just installing a one package.
We need to, of course, our database, we need to create the extension to handle the vector
data.
And then we can just add a column, the extension will give us a new data type, which is called
vector.
And, optionally, here where the three dots are, we can specify the dimension of the vectors
that we're going to keep in it.
But it's not obligatory.
And then, if you want to connect to your database for some of the languages, you need to
add a library.
I think they are available for any languages that are able to connect to post-gres.
So for example, you just need to do a pip install and then import a pgvector Python.
But this is optional, sometimes it's not needed for some of the queries.
It's not even needed.
So how are we going to query our data?
So here, we introduce a new operator, which will calculate the distance of the column
that we created in the previous step, so the embedding column.
And we will calculate the distance from this vector that we are using here as an example.
And there are many operators like that, as I mentioned, different ways to calculate the
distance between two vectors.
So for our use case for our presentation, we will use the cosine distance.
So what I presented on this slide with the cat and dog and banana and the flamingo.
We can also use inner product, euclidean distance.
So there are different ways on how to calculate it since 0.70, you can also use the Manhattan
distance.
There are different ways you should probably check it out when you select your model,
which works the best for you, which one gives you the best results.
And what about the indexes?
So by default, the nearest neighbor search or a search in comparison to your vector will perform an
exact search.
So that's a very slow operation.
If your data set is quite big, it will be a full table scan.
And the PG vector will calculate the distance between what you provided and the data
and your data set.
So this is quite slow if your data set is big.
So there are two types of indexes that you can use, but the indexes will give you approximate results.
The idea is that you want to sacrifice a little bit the quality of your data,
but this results that are faster.
So there are two most popular approaches to doing things in vector lookups.
So this is the hierarchical navigable small world and the IVFF, so inverted file flat.
The general concept of the first one.
So the H and SW is that you put your data on a space.
Here you can see two dimensional representation.
And then randomly you choose only, like, let's say 5% of your data, you put it on a different layer.
And then out of this layer, you also put like 5% on this additional layer.
So when you want to do a lookup, you just traverse through these indexes to find potentially
something that's the closest to your results.
So that's one way.
A second way is try to do kind of like a clustering of your data.
Try to cluster the vectors that are similar to each other.
So you're, again, this is a simplification because here we have a two dimensional space.
You try to create center-roids of grouping to be things together.
And then here with the green color, you can see that if you try to do a lookup,
it will probably try to result from this cluster of information as well as the neighboring
So if you want to learn more about the indexes, there is an amazing tutorial that's linked here
from Alex G, from Neon.
It's a tutorial how to write down your own search using those two algorithms.
But this is out of the, let's say, out of this scope of this presentation,
we will use those indexes, but without diving deeper into the kv and the limitations of those indexes.
There is one thing that you need to know about pgvector and indexes.
So the index lookup search will, will be the first operation that will be done.
And then the filtering will be done on top of the results from your index lookup.
So just because the filtering is applied after the index lookup,
it is possible that in this query you will get less than five rows by the default configuration.
The idea is that when you order by, by default, the pgvector will start with, I think, 40 as a default.
The, as a default set.
If out of your 40 entries that were looked up with this, with this query,
if only one fits your criteria of category, let's say,
then you might get a one result instead of the five that you would actually expect.
So for the, as I said, the candidate list is 40 by default,
but you can also control it as a parameter if you want to do more.
And there is also the possibility to do interactive scans.
So pgvector can understand that, oh, by the way, your limit was five,
but if only manage to get two results, so I'm going to go and do another index lookup,
try to get more data for you and do it several times until you get the full result set of five.
But of course, there is a limit of, it's also controllable with, with a parameter,
how many times should it do the lookup, because you want to essentially get your data out.
But okay, let's, that's enough, theory, let's, let's finally build the system.
So for that, we need to have a data set on beers.
So I have managed to find on Kaggle.com, I found a data set on beer profile and ratings.
So what we took from this data set is only the beer name and the info, which is a column that's
sometimes it consists information on the taste of beer, sometimes it's just some marketing information
about the beer. Sometimes it's some technical details about the beer, the person tells the
I view. So we will see how that goes. Like, I didn't do any data filtering of this data set.
So in our database, we have, I think, 3,300, around 300, 3,000 data bases. And of course,
this was just a CBS V file that that I imported into our database. So then I created the column
of the embedding, which we will start the vectors in and I've specified the dimensionality of the
vector to 384, because this is the dimensionality of the model that we are using.
So how to put it all together? I wrote a simple Python application that connects to our database,
gets the text from the description, puts it through embedding model and then converts it into
the vector and the vector will write it down. And for the embedding model, again, it's up to you
to choose your model. So for example, this model that I'm using is all mini LM from library of
sentence transformers. It's quite a popular model, as you can see, it was downloaded more than
a 3 million times last month, but also what's important is the license. So it's a patchy to zero
license. It's a model that allows you to transform text in English to an embedding. So there is a limitation
for example, this model, it's not a multi-language model, so it will not understand descriptions
of the beer in Polish or French or other languages. So there are multi-language models available.
Depending on the use case, depending on the data set, you should choose that and you potentially
choose something that works best for you, just by doing some trial and error. So how do we do
the embedding? So simply, the easiest way to do an embedding. Of course, one of the options
is to call an API provided by one of the cloud providers. You can use any of the major providers
of the LLMs or OpenAI, but we try to do stuff locally. So the sentence transformer library for
Python will automatically download this model for you. So when you run this code for the first time,
it will actually call an external service that will deliver the file to your laptop, to your computer,
and it will be hosted locally. So this will work no matter if you have internet access or not.
We just instantiate this object. We try to encode this, as you can see, we encode the data
using the model encode. And then we receive the embedding. As I mentioned, it's the model that we are
using is providing the 300 plus vector dimension. So it's going to be an array of 384
scholars as an array. So for our process of embedding the data on the database site, it's super simple.
It's like a column like any other. Just the data type will be this array of vectors. So of course,
we'll do it like set embedding equal this parameter. So we have 30361 beers, which will try to embed.
So doing it locally on my MacBook, it's 43 seconds. Doing it locally, but in four processes,
it's roughly 20 seconds. So for a small data set that's completely acceptable to test
several models. It's probably a good idea to test it before doing it on full scale.
For example, my real data set would be 3 million of beers, then I would have to worry about
a little bit more. But for this use case, this was fairly simple. And this was super easy to
parallelize, because I've copied my very simple code and Python and just asked the tragedy to
parallelize it and gave me an output that was using the multiprocing pool and Python. So it was
fairly quick and easy. And this is essentially the code. So we have a cursor that's connected to
our Postgres. It's outside of the slide because it's pretty standard. So you can see that here we
are downloading the information from from the beer stable. And then for each of them, we do the
encoding. We append everything into the binds array, because we want to do it the nice way with
reusing the statement. So at the very end of our code, we just do execute many. We run the
update SQL, which is as I presented in the previous slide, it's just setting them, adding to this
value that we have calculated. And that's it. Of course, if your data set would be way, very you
would have to split into smaller operations, probably it would be a good idea to distribute it across
some compute modes. But for this case, for a project like that, like a site project, this is
more than nothing. It was super simple to write that code. So how do we look up our information?
We do a simple query from the Postgres. We order by the embedding, as I mentioned, we have this new
operator that will calculate the cosine distance between the input that we are providing. So the
vector that we are providing, and we will limit, because we don't want to sort all of our data set.
We will limit, so in the examples, I think I have a limit set to five examples.
So again, let's have a look a little bit into how our code works for the lookup, because then
previously we've seen how to vectorize our data. Now we will see how to do the lookup.
So as I mentioned, there is this additional library that you can use to import the procedure that's
called register vector. Essentially, the register vector is responsible for making sure that our
input is converted into an array before it's being fed into Postgres, so it's making sure that the
types are compatible. We don't have to worry about the conversions, because it's going to be done
automatically for us. So I have a piece of code that we will hopefully manage to run in a second
and have a live demo of how it works. We have some user input, and then the user input, we are
trying to create an embedding using the model that was loaded before, and then we simply select
the five top beers, and then we will print the beers in an output. The real code that I'm going to
use a little is a little bit more complicated than that, just because I do some fancy, like,
modification of how we print out the results just for a readability, so it's easier to see.
But so far it's quite a simple code, and the full code is 60 lines, so that's nothing major.
So I mentioned the indexes. Without the indexes, we will get the exact results, so as I said,
it's going to be a full scan on the beer stable. You can see that we have 3,360,
the data set is 361, but what I'm doing here, I'm trying to, I have a query that's also trying to
find the beer that's similar to a beer with the ID 2363. So this is just to simplify the
the explain plan, and you can see that without the indexes, the execution time is 15 millisecond,
if we create the hierarchical navigable small world, we can get it down to 3 milliseconds,
and with the IVF flat, the execution time is less than half a second, it's almost around half
a half a millisecond, and you can see that in the explain plan it's visible as a normal index
operation on our table. So this is just the backup slide in case the demo doesn't work,
but our expected behavior of the system is that we type in a prompt, so let's say we are looking
for something that has lemon, and we get some results, so this is not a full text search,
so we are expecting to see other things that are probably relevant to us, that's not necessarily
so we know that lemon is a cypress, so we have like a proposal of a beer that has
cypress and lemon zest, we have something that has lemon juice in it, we have more citrusy
beers and lemon grass aroma, so there are some good recommendations as well, but let's do the
the fun part, let's do a demo, so I don't know if that's visible for people in the back row,
hopefully yes, so let's start with a first one, so it's looking for beers that are herbal
but fruity, so here we can see that we are getting some, the first column will be the distance,
so it's not similarity, it's distance, so the lower the closer we are to have a perfect hit,
let's say, so here we see some proposals, but maybe it will be more visible here, so the next
query that I'm using is berries, and what my code is doing, it's highlighting with the black
with the blue background, the direct hits, let's say, of the words, so we might see that,
you know, the first proposal is here because of the blueberries, raspberries, but the third one
has raspberries, but only one, so singular, but our model knew that it was similar, so the
similarity surge knew that they are somewhat similar, but this is quite simple, this is something
that probably a full-text surge could also handle for us, so let's start to do something a little
bit more complicated, so let's try to find if our database says has a beer that has a taste of
beers, a berries, and caramel, so this is a little bit more complicated, we will not have a
direct hit in terms of string comparison, but here we can see that for example, we have fruit
ale, which has subtle fruit flavor, but actually know hit on the caramel, but for example,
the package hack will have a dark fruit aroma with rich caramel flavor, so this is probably
something that could be of interest to somebody that wanted to have a beer like that, and I have
some other ideas on what we can try, but maybe we have a suggestion from the room, what should we look for?
Yeah, I knew that this would be one of the, so for them doesn't give us that many, let's say
relevant stuff, but something that worked for me at a different conference that we had SQL,
right, because we were talking about Postgres SQL, so the first hit is a DBA, don't mention
the third one, so here we can see that it's a very interesting example, because with the
SQL, our model actually we didn't put the name of the beer into what we are vectoring, so we
are only vectoring this long text, this description, so we can see that our model knows that
DBA is somehow related to SQL, but of course we know that it's like the double barrel area is
not the same DBA that we are thinking of, so I have also met, yes, the first one, the first one,
the first one, the first one is called to beer, oh yeah, yeah, for first them yeah, maybe it
actually knows that yeah, the first one is happening in Belgium and the trap is a Belgian beer,
it's true, yeah, I could probably also import the information about the breweries because it's also in the
it's also in the data set, so it all like the our model is also sometimes hallucinating a bit
giving us results that are not that interesting, so for example, the second result is just
Ibu's of the beer, so it's not really relevant to our case, the third one, the Oracle,
is somewhat relevant, right? But yeah, let's see some other interesting results, I don't know why,
but maybe somebody would try to look for beer that would go very well with his cigarettes.
So here we can see that our model understands that cigarettes are somewhat related to smoke,
so the results that we are getting are smoked, brown ale, smoked, stout, it's probably a different type of
smoke than the cigarette smoke, but still the model is somewhat understanding the realities of the
world that we live in, let's do the Queen, so our model understands that Queen is somewhat similar to King,
so the first result is King Crimson, the second one is not very relevant, but the third one,
actually mentions the Queen here, another interesting example is Poland, so there is only I think
like one or two Polish beers in this data set, I think like this data set was based mostly on the
US market, so the first one is actually a graduate skier type of beer, which is a Polish type of beer,
the second one is also related to Poland, dancing is good ice, so it's also relevant to Poland,
and then I think it didn't have that many Polish results in the data set, so it knew that for example
Czech beer is and Czech is somewhat similar to Poland, so it was trying to come up with an answer that
was somewhat relevant, so you can see that the other ones are related to Czech, but I also mentioned that
the indexes are allowing us to get faster results, so you can see that here like the results were
super simple, but this is just because our data set is super small for the needs, like 3000
rows that's nothing, but we can have an example of how creating an index will change our results,
so let's do a simple search, it will try to look for beers that have the lime in their description,
so potentially something that was lime, citrusy, similar, I will just move on now here to
to another console, so let's create an index, on the beers using the IVV, IVF flat,
and we are trying to create the index on the embering column, and this is an operator that we need
to mention when creating the indexes, that the comparison operator that we will be using is the
cosine operator, so we can go now back to our example and let's look for lime again,
this was supposed to give me different results, so here you can see that the first one didn't
didn't use the index just yet, but the second one you can see that the similarity search took
one tenth of what this one was using, you can see that it's 0.02 seconds and this is 0.002,
so it's 10 times faster, but also the results are a little different, right?
The Govanus Gold was the first hit and that's the first hit on this one that's using the index as
well, but here on the second point we already get a different result, so this is the approximate
result that I was mentioning that when we are using the indexes, the results are let's say they are
not exact, but if our data set is large enough, that's something that we should make sure that
with our system can handle, and just by having a vast amount of data we will let's say get the
good result, so again, a recommendation to drink responsibly, I know it's Belgium, remember that
Belgium beers are quite strong, but what about the real life usage of a system like that?
So let's move a little bit away from the beer example, let's have some text as an input, so for
this case I've put some first phrases of Romeo and Juliet, so in production we would split it into
smaller tanks, so in terms of tanks like Shakespeare's poems we can chunk it into different lines
into different verses, so we would split our text into line by line, and then embed them in an embedding
model using an embedding model we would create the vectors, and we would of course start the vectors
in our database, so with a table like that, when we have somebody that's looking for information
about houses, of course in the context of the dataset that we have, what we can do is we can
embed the query from our user through the same embedding model, and this is important, we need to
use the same embedding model because different models will give us different results, so they are
just a mix of numbers that will not make sense, so we always have to stick with one model that's
to get the same results, the query from the user will give us a vector, and using this vector we can
do a similarity search, and to retrieve some information, so this information, so for example,
let's just query the most interesting part here, most relevant to the houses, so the first phrase,
the two households will probably be the most similar to the, to the information the query from
our user about the houses, so if we feed that relevant information along with the user prompt
into an LLM, we can try, ask it, if you phrase it correctly, this is, we can ask it to be our
assistant, get the information and provide us with a result, so the LLM along with the
internal information will know that the households are, this is just Shakespeare's way of saying
that these are families and not really houses like physical objects, so this will give us the
result that both houses had people of equal status, so a system like that is essentially
capable of trying to answer our questions, in production we would actually add a rerank operation
we would get a lot of information from database and try to run a what is called a rerank operation,
so query a lot of information from the database, we do a rerank to see what's relevant to the
input from our user's potential using a different algorithm or a different model to do that,
and we are selective into what we feed into the LLM, and if we do it like that, we have a rack,
a simple rack model that we could use to have an let's say AI assistant based on our data set,
and with that, I just want to reference the projects that I used in preparation for this talk,
what could be interesting is the right up of Seren's internal knowledge chatbot,
so there is a working team at Seren in the department that's trying to come up with a chatbot
that would be able to scan all of internal documentation that we have and provide people with the
answers, and yeah if you're ever in the Geneva region feel free to give us a visit we have an
amazing public spaces with the science gateway that you can see here on the on the photos where we have
some exhibitions about what we do about physics it's also a nice place if you have kids it's very
interactive kids can play around with a lot of stuff, but here at Boston we also have a booth in the
building yeah I can't spell building F on level two so you can you can see us there we have I
think we still have some swag we also have people that are responsible for some open open source
project at Seren so yeah feel free to come by and that's it thank you
okay do we have any questions in the room yeah
hi thanks for the amazing talk so I have the question regarding indexes I mean are there any
limitations as the number of vector dimension increases are they betting field so I think that the
PG vector has a limitation of dimensions of 4,000 this is the current limitation of the PG vector
but I know that as a workaround there are some algorithms I haven't used that but there are some algorithms
that let you in an intelligent way convert the 8,000 you know 8,000 dimensional vector into a smaller
dimensional value okay thank you yeah maybe if you can yeah just probably not not a perfect
question but can you explain to us not knowing the difference between different kind of distance
calculations when his sum of the other maybe more appropriate than the one that you use today
yes so the question is about the different types of different types of calculating the distances
so it's actually something that I skipped in my demo so for example in my data in in my case
I have used it's very simple to compare the results and in my case as you can see here like
I prepared three different versions of the of the query here so by using different
by using different operators different
different comparator here we can compare different different results so
how you you should choose the distance calculation how you can choose the
the similarity search whether you use cosine,uclidean etc it's depending on your data set
and also the model that you are using so I checked and this is just something that I found online
that the recommended way for this model that I was using was to use the cosine distance so
this is why I was using just that any other question
well then we still have five
right questions yeah the read the is the question oh yes sorry
hi thanks for the talk I have a question about the chunk size or you talked about chunking
the text are there any recommendations for a size limit that should not be exceeded or would
that depend on the transformer used so the question was how to do chunking how to decide on the
chunk size and I cannot give you a simple answer on that just because I didn't investigate it enough
from what I've seen in real life use cases is use should first of all for it so in my case when
I was presenting the Romeo and Juliet part I was just cutting like each line by by each line
which is not necessarily ideal but what you could do for example try to have a little bit of the previous
chunk in your new chunk because you still get some context another idea would be to split
by sentences whenever you have a dot you could split it into different parts so there are many
different objects and I think as it is in machine learning and AI it's mostly we need to test and
see what works best for you because of course depending on your data set depending on the
model that you are using you will get different results and it's hard to find the correct answer
there is like no magical ball that will tell you the best chunk size is you know 250 bytes you know
maybe it's 17 awards or for a different model maybe it's something completely different also
the chunking is something that I think is quite problematic but it impacts your results quite
dramatically yeah yeah I was wondering thank you for the talk I was wondering if you have some
usage at certain of like what you presented yeah so one of the use cases was that I presented
that in the references we have a team that's working on developing an internal chat but that
will give us some kind of information so you can you can ask questions like a like an
assistant that will give you information based on the internal knowledge based articles at
certain so that's one use case I know that another use case that they are looking into is the
experiments they create huge amounts of data which are also like numerical values etc so potentially
by creating our own model we could find similarities and the results that we are getting but I
don't think it's in production stage yet it's just you know it's in the research phase
thanks for a great talk my question is do you foresee especially with the advent of more
like open source LLMs do you foresee the popularity of training your own LLMs on your data
do you foresee that to be like more popular in the future over using rag I think that training
your own model is not going to get like super popular because it's it's expensive so I think what
people are doing right now partially because of the necessity so using a pre-trained model and trying
to train it a little bit more on your specific data this is this is the way to go or trying to
come up with something like like I presented in the rag part this is for at least for now the
way to go just because of the cost because the cost of you know of course now we have the deep
seek you know they claim to reduce the cost dramatically so maybe it will become more and more popular
but I think like for now this is what everyone is doing
consider in your use case at certain how do you plan to handle aggregate questions to this
chartboard I mean when some calculations is required rag is not enough in this case
this I am not sure what's the plan of the team because I'm not directly involved in this project
so that's a very very good question yeah I've put my LinkedIn information so yeah just send it
to me and I can get to the team and I don't get back to you on that any further questions from
room I don't see any hands so I'll ask one myself yeah did you actually find a bear that you
hadn't tried yet which you really liked using this method I found something I don't remember it was
it was I think it was soup or for I found something that was yeah for spiced porter this sounds
super interesting but I haven't tried it yet I haven't found it yet the problem with this data
set is I think it's like 20 years old and it's mostly US breweries so it's it's it's quite
tricky to actually get a hand on that but I think this one stood out a lot the fall beer I would
give it a try it was probably just just to taste it but not have a full pint okay well then
thank you for your talk we will start something to do now and get on