They are bases can play a role in enhancing AI.
Please welcome Bruce.
Great.
Thank you everybody.
I am the closer today for this room.
I'm excited.
I was here two years ago talking about home automation.
Anyone in the session?
Two years ago?
Well, okay.
My thank you very much.
So this talk is completely different.
I love home automation, but not today.
We're going to be talking about AI.
My name is Bruce Lomger.
I live in Philadelphia.
I work for EDB.
I'm on a five-six city trip.
I spoke in Prague earlier in the week.
I'm going to Berlin on Tuesday to speak.
I'll be speaking also in Rihard and Amsterdam the week after.
A lot of fun and then 10 days, 12 days after that.
I'll be in Montreal and I'm doing a three week trip in India.
In March and then maybe go to Hong Kong.
So we'll see a lot of excitement.
But this talk I will warn you.
Can stress your brain.
So just be aware.
This is pretty complicated.
I'll explain why it's complicated a minute.
But this is going to be.
It might be a challenge, but I'm hoping you'll get a lot out of it.
I certainly got a lot out of it writing it.
All right.
So if you've heard my AI talks before you have not heard this one.
Because I just wrote it.
About three months ago, this is the second time I'm giving it again.
I gave it first in Prague.
My talk from 2020 is called Postgres in the Artificial Intelligence landscape.
And there's the URL right there.
In fact, my website is right here, right up here.
And in fact, if you'd like to download this presentation or
61 other presentations with over 100 videos and 600 blogs about Postgres,
that is the place to go.
And there's the QR code for it.
So this is the other presentation I wrote on AI.
It's more talking about machine learning, deep learning, and effectively what we call
discriminative AI.
I'll talk about what that means in a minute.
So again, if you're looking for something a little earlier technology for AI
and still valid technology, you might want to take a look at that talk.
But today, we're going to talk about in the next 50 minutes AI explosion.
Hey, this, the type of AI I'm talking about today has only been around for two years.
Okay?
So we're kind of in the wild, wild west, right?
If you technologies are that new that we deal with, right?
In fact, in fact, it's so new that this week we had news about deep seek out of China,
which is a different way of doing AI and potentially more efficient way.
Now, we can argue whether it's actually unique or not and what ways it's more efficient.
But again, the concept that this is technology is two years old and almost every month
we're getting a seismic shift in the industry kind of interesting.
Okay?
The goal of this talk is to really go down into generative AI,
explain how it works.
Okay?
And again, that's only two-year-old technology.
And then explain how that's used for what we call semantic or vector search.
We're going to talk about how it's used for rag or retrieval augmented generation.
And we'll finally talk about how databases are used for AI.
The fundamental, you can't decide how to use a database for AI unless you understand how AI works.
You can sort of throw it out.
You've got to go down to the roots.
Figure out what's going on.
Where does database make sense?
Where does AI make sense?
So this is a kind of a two-part talk.
Most of it's about AI and toward the end, we bring in databases to side where they make sense.
Okay?
I would love to be taking questions while I talk.
I will try.
But I am also cautious that I've 85 slides in 50 minutes,
so I will try and leave most of the questions to the end.
So AI explosion.
Basically, let me flip this around here along here.
ChatGBT released November 2022.
So again, two years, two months.
Effectively at this point.
It was probably the first tip time where AI was accessible to the public
in a human-like conversation.
All right?
And that's really, I think, what got everyone excited about it.
And I'm going to explain toward the end of the talk,
how some of the big players like Google missed out on this.
In fact, Google's a specific example that really could have led this,
and officially, unfortunately, for then, missed the boat.
This series right here, this YouTube playlist,
is the best AI playlist I have ever seen.
It is made up of six videos.
Each video is probably an hour long.
So again, feel free to do that.
If you download these slides, anything in blue, you can click on.
Okay?
So you don't even need to worry about it.
Download the slides, and some I gave this talk in Prague,
and the person said,
I'm going to be spending quite a lot of time on this slide deck.
Okay?
Because you basically can drill down one of the goals of my slide decks,
as you can drill down into every single slide on most,
and get more detail.
Because obviously, I can only cover so much in my talk.
Okay?
Again, we have some pictures this year I'll be down here about.
About how things have changed and how chatGBT changed.
Okay?
I talked about two types of AI.
Up until 2022, all of the AI that we normally worked with
was what we called discriminative AI or predictive AI.
And again, the talk about postgres and the artificial intelligence landscape,
which I grew in 2020, is exactly about that.
It's exactly about discriminative AI.
Now, discriminative AI is primarily focused on classification.
Okay?
For example, is a credit card charge fraudulent or not?
Yes or no?
Okay?
Is this a picture of a dog at Cat where a lion?
Okay?
Yes or no?
Which one is it?
Recommendations for web pages.
I'd say here, predictive trends, language translation.
Again, down here, difference between generative and predictive AI.
This is AI up until 2022.
Nothing wrong with it.
There's perfect legal legitimate reasons to continue to use it.
I am going to be, that's what my first talk does focus on.
This talk is going to be about the 2022 forward generative part.
Okay?
So generative AI, again, initially coming from open AI, was revolutionary because it changed
the public perception of AI by releasing something that either generative text or images.
And the concept is it generated new content.
It did not classify existing content, it created new content.
And that was the revolutionary part.
And you've got to give opening a credit for coming up with this concept.
So, what are the generative AI use cases?
There's a whole bunch of them.
Chat bots is a big one.
Semantic vector search.
I'll talk about that today.
Summarization of text.
Now, summarization taking a longer text is summarizing it.
That is actually a unique generative use case.
It's not, you need a special engine to do just summarization.
Okay?
Open AI effectively takes several of these different generative options and puts a single interface to it.
But effectively, when you're trying to do some of the different language translation again,
I'm very impressive.
For example, I'm Armenian.
I'm in the United States, so we speak Western Armenian.
Most every web page only translates Eastern Armenian.
Chat GBT will do Western Armenian.
Chaka.
First case, I've been able to actually generate Western Armenian from English and back and forth.
So, that was a shocker to me.
Generating audio, video, images.
This was crazy.
I was on a channel with some homage people.
I can't describe it.
But effectively, this homage guy loves lobster.
So, I asked Chat GBT to do a dolly, which is Chat GBT.
I asked it to open it.
I said, give me a picture of an homage man riding a lobster.
And it actually generated a guy with a hat when the lobster.
And I sent it to the channel and everyone laughed.
So, again, crazy stuff you can do.
Code generation code analysis.
I've actually had some success with this.
So, for example, I wanted to translate some pearl the python.
And I could basically send it to code and it would translate it for me.
New capability.
I sent some python code up.
I asked for recommendations out and prove it.
And it came up with that.
Again, that's a whole new thing.
We're in the wild last.
We don't actually know all the things we can do yet.
Programming language conversion.
And again, this is kind of a cool one.
Dactual language interface.
So, for example, the ability to generate SQL from a user query.
And I'll show you some examples of that as we go forward.
And again, some nice URLs right here.
Okay.
So, to kind of show you the differences.
The discriminative supervised learning requires training data with known outcomes.
Often problem remains specific.
And output is determined at determination or prediction.
Generative outcomes are determined from massive training data.
You're talking, like, terabytes, petabytes of data.
Usually it's general and it outputs new content instead of being predictive.
Okay.
And again, the description of that.
And again, description down here.
The basic issue is that discriminative.
This is a great illustration.
Discriminative is trying to find the boundaries between the various options.
Okay.
Where is generative is effectively generating new tax new image.
Okay.
You understand the difference.
It's kind of the question of whether you're looking at discrete options or whether you're just generating brand new stuff.
This is what generative looks like.
And I'm going to explain why it looks that way in a couple of minutes.
But effectively, instead of looking for boundaries.
It creates what we call embedding vectors or words or parts of words.
And effectively, those embedding vectors are used to generate new content.
And I'm explaining attention blocks and how that works as we go forward.
Any questions?
Okay.
Great.
All right.
Now, here's going to be a little bit of the brain stuff.
Okay.
I was a math teacher.
I measured it.
I took them.
I was a minor in mathematics.
So we're going to walk through some things here just carry along.
First, I want to explain that mathematics allows you to represent concepts that don't exist in the real world.
Mathematics allows you to represent concepts that don't exist in the real world.
And I'm going to explain why that's important as we start to talk about some of these embedding vectors and how AI works.
Because some of the stuff it does really can't be understood in the physical world.
You have to think about it in a mathematical way.
I'll be my best to explain that.
For example, the number of atoms in the universe is 10 to the 80th.
I don't know what that means.
It's so many zeros.
I can't really fathom how big that number is.
All right.
The general you consider number of how many atoms are in the universe.
I kind of remember that number because I'm going to reference it later.
Secondly, infinity.
I don't know how many of you deal with infinity.
But effectively, there's a whole mathematical branch that talks about how infinity is used in various contexts.
We don't know what affinity is.
We can't really represent it in the real world because everything we deal with this finite, right?
But again, infinity has a representation in mathematics.
Third option, derivatives in calculus are trying to deal with infinitesimally small adjustments of a line of excitement.
We have a mathematical representation for that.
But again, the physical concept of an infinitesimally small something we can't really represent.
Same thing with integration.
An infinite sum of infinitesimally small slices.
Again, you can't represent it, but mathematics has no trouble with it.
So another concept mathematical is called duality.
Duality says that we can think of mathematics into the physical sense.
We can think of it in a computer science sense, like in a race sense.
And we all can consider it in an abstract or mathematical sense.
They're all the same thing.
They all represent the same thing.
They're just different ways of looking at a number.
Again, we have a URL here that talks about duality if you're really into that.
You can see why that person said I'm going to be spending a lot of time on these slides because there's a lot of stuff here.
All right, so what I'm going to talk about right now is what we call embedding vectors.
I'm going to explain how we create them.
But that effectively is an embedding vector for five words, six words.
But I only have two dimensions on this screen.
Okay, what I have tried to do is represent the concept of pink using 10 dimensions.
And the way I did it was I flipped the pink right here on that spot on this graph.
And in that graph, there's a graph up in the corner, which has another set of X y axis.
And then from there, there's another one here and there's another one.
So this is the one, this is two, this is three, this is four, this is five, it's 10.
Okay, because each one is two dimensions.
So that's 10 dimensions of pink, right.
Most embedding vectors have between the thousand and 12,000 dimensions.
I personally cannot fathom what 12,000 dimensions looks like.
Okay, because we only live in a three-dimensional space.
Even if I could show this graph in three dimensions, it still only gets me the three.
It's not going to get me to a thousand or 12,000.
So you have to empirically think when I'm looking at an vector, it doesn't have two dimensions.
It has a thousand or 12,000 different values for each dimension.
Okay.
And in fact, yeah, so that check you continues to thousand and 12,000 dimensions.
Again, really big and again, we have a URL right here.
Okay, so how big is that?
Each dimension is four by floating value.
Okay. If we look at the range of a floating point value, which has an exponent, mantissa.
Okay.
We're basically looking at 1,000 to 12,000 dimensions, being 9,000,
pent to the 9,000 versus 10 to the 118,000.
Okay.
So we're talking about how many atoms of the universe?
And in the 80, so 10 to the 80 dimension, atoms of the universe,
and we have each vector representing 10 to the 9,000,
or 10 to the 118,000 possible values.
Okay.
This is, you can see why mathematics is necessary because we're dealing at a scale.
It's just way off the charts for anything we can understand in the physical world.
Okay.
I'm a cool.
Again, I have a URL here to give you some ideas.
Okay.
So notice that each vector is the same length.
That is actually requirement for embedding vectors.
Okay.
Again, some URLs there.
Okay.
Any questions?
Great.
Thank you.
Question three, creating text embeddings.
This is where it gets exciting.
If you weren't excited before yet, this is it.
I've explained that the way you do generative AI is you create these vectors,
which are literally 1,000 to 12,000 dimensions.
Again, huge number of possible values.
But what, how do you create these embeddings?
And I will tell you, because open AI is so new,
it was incredibly difficult to get all this information into this slide deck.
Because we haven't really codified and solidified the knowledge, right?
It's only two years old.
So there's some people in this little piece and some people in that piece and some people
explain this.
So to bring it all together and something understandable,
a bunch of weeks to do.
Thankfully, EDP is a big one day AI.
And they said, yeah, go ahead and do find figure it out.
So it was very, it was nice to be able to just spend time digging into this
and coming out with something that at least excites me.
Because now when I'm talking about AI, I know what I'm talking about.
I'll admit to you, when I study a topic,
I not good on the surface level.
I want to understand all the way down to the bottom.
Like if I'm working on a computer problem,
I want to understand the code, I want to understand the operating system,
how virtual memory works, how the CPU works, the IO subsystem all the way down.
And when you understand that whole stack,
if you have a problem and you're trying to optimize,
you understand what's down there.
So again, I need to understand what was down there at the bottom.
So how do we do?
We've heard the concept of AI training probably for two years now.
What does it mean?
What does it mean when they tell you they're going to do that?
I went to a conference in...
Well, I'll eat, you know, show it.
Southeast Linux Fest, and there was an AI talking to me as the guy.
How do they train these models?
Well, they kind of take some data,
and they put it in like, well, how do they actually do anything?
Yeah, he couldn't really answer.
So I'm going to try and answer that.
Okay.
So the way you do it is you take
you basically create a vector for every English word,
or for every word in whatever language you want,
or if you're using a lot of foreign words,
you use like what they call byte pairs.
It's actually tokens made up of two letter combinations.
So I'm not going to get into that.
I'm just going to pretend we're doing English words,
but of course when you start to talk about proper names,
they're going to be also to different combination letters,
so you kind of got to do both,
or you got to do one of the other.
But let's just assume we're going to create a vector
for every word in the English language.
And that's probably 30 to 50,000 vectors.
Okay, we're going to start to do simple.
I'm just trying to highlight if there's another way to doing it.
Okay.
Then I'm going to assign each vector the same length or magnitude,
or all going to be the same distance from the center.
Okay.
So remember the illustration I had every vector was at the center,
and they all were the same distance.
So that's like kind of a constant that you have to have.
And this is kind of crazy.
I'm going to assign each vector a random direction
in the 1 to 12,000 address space.
Now, why would you do that?
Like why wouldn't you try to help it?
Well, you got 30 to 50,000 words, let's say.
Or maybe more.
Okay.
Do you have a vector space?
It's 10 to the 9,000 or 10 to the 118,000.
Does it matter randomly where you put it?
Probably not.
But you're just swimming in a vector space.
There's just so much room in there that we can't even
fathom how much room is in there.
Okay.
At least that's where we are today.
And I'm not saying someday we may figure out a better way.
Right now, we just randomly throw them in there.
Okay.
Now, this is the key.
This is the key to the section.
What we do once we randomly give them a location.
We adjust the direction of each vector using a mass number of training documents.
For each word, we adjust its vector to be closer to the vectors of surrounding words.
Or, for each word, we adjust the vectors of the surrounding words
to be closer to its vector.
Again, we have a lot of URLs there.
Let me be specific.
Okay.
Here is my example.
This is a very classic example that every AI seems to use.
A clean man woman.
All right.
And there's a specific reason they chose this example.
I think.
But I'm going to work through that.
And again, this is a great URL.
Everything I put in blue is great to me.
Anyway, because I spent a huge amount of time finding these really detailed things.
Okay.
So, let's look example.
Where this is my training data.
The king was a tall man.
That's my training data.
Okay.
I got that off a web page.
I got that off a chat bar.
I got that out of a movie.
I don't know where I got it from.
The king was a tall man.
So, either I'm going to move the closer to king was a tall man.
Or I'm going to move king was a tall man closer to king.
And then the second step, I'm going to move king closer to the was a tall man.
Or I'm going to move the was a tall man closer to king.
Do you see the difference?
Okay.
And I'm going to do that for all the words.
Right?
So, specifically king is getting closer to man.
Now, I'm going to, let's now, here's my other training data.
Here's my new training text.
The king was a tall man.
The queen was a beautiful woman.
They sat together in the throne room of the castle.
Okay.
It's just very, very simple.
I'm going to throw out some of the words that are like minor like the.
Right?
That's not really key for me.
King is as I'm training here.
King is going to get closer to man.
Paul is going to get closer to man and king.
Queen is getting closer to woman.
Beautiful is getting closer to woman and queen.
Queen.
Throne is getting closer to castle.
And they're all getting closer to throne castle because we're spanning sentences as we're
doing this training.
All right.
Again, the example though is going to be closer to king, throne to room.
And now look at throne room.
Now because there's a close together, throne room becomes closer.
And that gets closer to castle.
So it's because there's always adjustment going on.
Okay.
So if we look at this example in our final output, what we want.
And again, this is two dimensions.
We have to think in a thousand dimensions or 12,000 dimensions.
So in one of those 12,000 or 1000 dimensions.
King is going to be close to queen.
In another one of those dimensions, man is going to be close to woman.
And then Throne is going to be close.
And ideally, the distance between man and woman is the same distance at this between king
and queen.
So you can do something like king minus man plus woman gives us queen.
Okay.
And you can start to see the semantic use of AI here.
Because we've trained on so much data.
When we normally do tax search and post graphs, we just have the word king.
We have the word man.
We have the word queen.
We have the word woman.
And we're just looking for specific word matches.
But here, because we're training vectors, each word is getting a thousand parameter adjustment.
The potentially be close to another word it's related to.
So each word that can be related to multiple words independently.
Right?
When we move man closer to woman, we're not necessarily moving it farther away from king.
Catch that.
When we move man closer to woman, we're not moving it necessarily farther away.
Because the king closeness to man is probably on a different dimension than the closeness
to man to woman.
Because we have a thousand or 12,000 dimension.
This is just kind of what a text embedding looks like.
I'm just kind of giving you an example of visually what it was.
Remember we have a thousand squares.
I don't even think there's a thousand on here.
And each one has a floating point number in it.
Right?
So it's pretty powerful.
Okay.
Any questions?
Yes.
Sure.
Yeah.
So the question is, is there difference between the two training methods?
One is word devak and one is a different name.
If you, I'm going to talk about it in a minute.
There is a URL I have which explains which one is better.
So one of them is better for a certain purpose.
The other one's better for a different purpose.
Yeah.
Great question though.
Yeah.
Anyone understood?
Yes sir.
What do I mean by moving closer?
I'm going to show that right here.
Okay.
Great.
Specifically mathematically what moving closer looks like.
It's actually really cool.
Great.
I'm getting through with questions.
I'm excited.
Okay.
Great.
Okay. Oh, sorry.
Samanic vector search.
Okay. Again.
Postgres is supported full text and free search in 2003.
But again, it's word-based.
There's no context.
There's no meaning to the words and there's no relationship between words or how to relate it.
There's no training involved.
You can use synonyms.
But again, it's only ad hoc.
Okay.
But as you can see this ability to do,
Samanic search is really cool.
And you can even mix Samanic and full text search together.
If you want to again, nice URLs here, kind of going over that.
Okay.
So Samanic vector set up.
How do we do it, right?
How do we adjust them?
Great question.
How do we do?
What's difference between the two of them?
So we download what we call pre-trained text and baggage or we create our own.
And then we chunk.
We chunk the size of our settings.
And then we add, okay, hold on.
Let me just make sure.
Maybe I should back.
Okay.
So let me just back up to that question.
You said, what does it mean to bring the vectors close to each other?
Just to, I'm going to be more specific about what you're saying.
What basically happens is that when we're doing training data,
we do what's called a vector product.
The vector product effectively involves us multiplying the two vectors together.
And we get a new vector, which is effectively between the two vectors that we have.
Okay.
But what's really interesting about it is that the way you do the dot product,
it has a tendency to overemphasize the dimension where those two vectors are closest.
So if I have a thousand dimensions, and two, one of those dimensions of those two vectors is closer than the other ones.
When I do the dot product in a thousand dimensions, my new vector pointer has a tendency to overemphasize the vector that's actually more similar.
That's why when you're doing gig and queed and man and woman and man and king and king and queen.
There's different dimensions where a man and woman are closer, different dimensions where king and queen.
And when you do the dot product, it sees king and queen together in some text.
It moves mostly the dimension where they're closer closer, but it doesn't touch as many of the other dimensions.
And that's why you still have queen when you're having king and queen moving together,
it's not necessarily moving queen and woman farther apart.
Kind of crazy, but that's the way vector product works.
Okay, so let's take a look at, it's a great question.
Let's take a look at how we do an actual semantic search.
So we can kind of see what this looks like.
Okay, so effectively what we're going to do is we're going to download some vectors and we pre-trained them.
So we have man, we have a vector for that, we have woman and vector, we have queen and vector.
30,000 words, 50,000 words.
That that data's all been trained and these vectors are all kind of near each other in different dimensions.
Right, remember man and woman are close in one of those thousand dimensions.
Man and king are closer, king and queen are closer and different dimensions.
King and castle are closer and a different dimension.
Right, that's kind of really amazing how it works.
And then we can effectively choose a chunk size and we chunk our query together.
Whatever we're doing, okay.
We find a text embedding vectors for all our words.
We add it to the vectors then we store them in the database.
So what does that affect the way mean?
So we find a text embedding for the vectors from training.
In this case, like open AI or llama from Facebook or open, well actually I don't think.
Deepseek is not open.
And there's a different open ones and some closed ones.
And then what we do is we take the query, we average all the text embeddings for the query that sentence.
And we get one result vector, okay.
And then we search for the closest match.
Right, and I'm going to walk through an exact example of this to give you an idea of exactly what's going on.
So here's a great one.
I assume you wish to index the document using some message.
The sentence says the king was a tall man.
The queen is a beautiful woman.
They sat together in the throne.
Okay.
So we choose the chunk size of being sentences.
So the king was a tall man.
That's six vectors, six embedding vectors.
We're going to average those six together.
We're going to store them in a database.
We're going to take queen is a beautiful woman.
Six vectors, average them together, store them in a database.
Sat together in the throne room.
Ten vectors, average them together, store them in the database.
Okay.
Now, who is the king?
Who is the king?
We take the four words.
We average them together.
And then we say which stored in bedding is closest to who is the king?
All right.
Remember, we're in a thousand dimensions here.
But the way the search works in postgres, you can just say give me the closest vector to this thing.
And turns out the king was a tall man and ends up being hit, which is great.
Okay.
How do we do this?
I won't actually show you some code.
And in fact, if you're really curious, you can download this SQL file and run it yourself.
Okay.
So have fun.
I download PNG vector.
I actually created table for all of my content and I create a table for my embeddings.
Okay.
And then I got that from here.
Okay.
Then I insert the king as a tall man.
Beautiful woman.
They sit in the throne room.
I create an index.
And then I write a python script.
And the python script effectively connects to open AI.
It pulls the embeddings.
And then for every word in the table.
Okay.
Every word in my document.
I insert an vector which average the words in the sense.
All right.
Everyone with me.
Remember I told you they take the word.
The king's a tall man.
I find the vectors for all six words.
I average those vectors using dot product.
And then I store that in the database.
Okay.
So that's exactly what I'm doing here.
And effectively here's the document.
Okay.
And here are the embeddings.
This is only three dimensions.
This goes to 1,000.
Okay.
1100 I think or something like that.
Okay.
And each of the dimensions is four bytes of a float.
And here's my query table.
Effectively what I do is I take a query.
I pass it to open AI.
I ask for the embedding.
And then I search for the closest.
I search for all my embeddings.
And I order them by their closeness.
Okay.
So who is the king?
Who is the king?
And the answer becomes which embedding a vector that I've averaged for my document's closest.
The answer is the king was a tall man.
Okay.
Who is tall?
The king is a tall man.
That.
This is pretty easy.
This is basic.
We could do this with full text search.
Okay.
Who is sure?
Now, short doesn't appear anywhere in our system.
And what's odd is the winner is the king is a tall man.
What's going on here?
Okay.
It turns out the tall and short are kind of related.
So it thinks that's the closest.
Right?
Because look at the numbers point 68.
These are farther away.
Okay.
Who is pretty?
That's cool.
Pretty and beautiful are associated together.
So what knows.
The closest is the queen.
Who is in the palace?
I don't have the word palace in my text.
But it knows the palace and castle are related.
Okay.
So I have.
They sit together in the throne.
Okay.
Where is the chair?
And use the word chair.
Did I?
But it knows that the throne is a chair.
Because of the embeddings that it learned.
Okay.
So they sit together in the throne room.
Is the winner that?
You can see how.
This is a little different.
This quite a different than a full text search.
Because it has understanding of how words are related to each other.
It does not happen when you're just looking at letters.
Okay.
Any questions?
Yes sir.
So the question is, do you need open AI to do that?
In this case you do.
But I could have used llama.
I could have used anything from hugging face.
What a work just fine.
Only reason I did this to make it super simple.
So you don't have to download and embedding.
You don't have to.
Because sometimes they can be like couple gigabytes.
And I'm like, let's just make it simple.
Okay.
I did open AI.
And we couldn't use anything.
In fact, you can host this yourself.
You don't need to use anybody else.
I just used open AI.
Other questions?
Great.
Thank you.
Okay.
Great.
Okay.
So I talked about.
I talked about.
The actor or semantic search.
But I haven't talked about generative AI yet.
Okay.
Now I'm going to talk about generative AI because it builds on these vectors that we created.
Okay.
And what I'm going to talk about is effectively something called transformers.
Transformers are effectively a part of attention blocks.
I want to work with attention blocks.
And I'm going to try and show you how these in text embedding with your trained on billions of documents.
Effectively, allow you to generate text or generate images.
In this case, I'm just going to give you text that's simple enough.
Images would be a different talk.
But I'm going to just talk about text.
Okay.
So, how do we do generative AI?
We did semantic search.
Let's do generative AI.
First, we load what we call an attention block with the text embedding vectors of the words used in the user query.
I'll show you what that looks like in a minute.
Then we adjust the vectors to be closer to previous words.
Remember how we adjusted.
You asked about adjusting the, that we're going to adjust these.
Okay.
And then we're going to repeat the step to several times.
And then what's really weird is once we're done.
The last vector of the attention block is our first word that we're going to give to the user.
And then we run attention block.
Again, we get the second word.
Have you ever used like an AI system and you ask a question.
Have you ever noticed the words come out like one at a time?
Anybody knows that?
That's not my accident.
Because that's the way the attention block is generating it.
It doesn't generate the whole sentence at once.
It generates each word and cycle through the attention block again to get the next word.
And I'm going to show you exactly how that works.
Okay.
Exactly.
You repeat step two to continue for successive words.
Okay.
So, we're going to show something called self-attention blocks.
There is something also called cross-attention blocks.
Cross-attention blocks are used for translation typically.
But where do you self-attention blocks to generate new words?
And again, I'm simplifying what's going on.
I'm not showing you a lot of the complication.
GPUs are key in this.
And again, if you want more details, feel free to look at the URLs.
Okay.
So, again, they can be thousands of dimensions.
Typically, the attention block is only 128 dimensions.
But they don't need a thousand dimensions to do.
Attention block, which is great.
The sum models use what's called multi-shot learning.
I'm not going to cover that.
I'm not going to talk about word position, sentence-sendings, or sentence-constructions.
That's a more detail.
I don't, I can't get into that.
Again, feel free to look down here if you want more detail.
Okay.
So, here's what we're going to walk through.
The question is, what is the capital of France?
All right? We don't know the answer, but again, we want it to tell the answer.
So, what we're going to do is we're going to take the word what.
And we're going to look it up in the text and batting.
All right?
Then we're going to take is the capital of France.
And we're going to look those up in the text and batting.
And we're going to fill in something called attention block.
So, each of these words is literally 128 dimension vector,
which came out of my attention block.
There's something called dimension reduction.
So, the thousand dimensions becomes 128 dimensions.
I'm not going to claim that I can't.
Okay?
So, what is the capital of France?
So, we're going to take what.
And we're going to adjust is to be closer to what?
That's my first step.
Then I'm going to take, and effectively, this is the way we do self-attention block.
We only look backward.
Okay?
So, I'm going to take though.
And I'm going to make it closer to what it is.
Okay?
I'm going to take capital.
I'm going to make it closer to what it is.
I'm going to take of, make it closer to what it is.
What is the capital?
And I'm going to take France.
I'm going to move it closer to what is the capital of.
That's my first attention block.
Okay?
And I'm going to do the question mark.
The item is the way I've done it.
And again, we're going to move each vector closer to the previous words.
We normalize them.
It's not uniform.
Again, this backdrop again happens is I get a word.
And it turns out that when my last run through the attention block, I get a final vector,
the final vector, if I'm going through attention block, maybe 10, 15 times, is the first word.
I'm going to return to the user.
I don't know why.
It doesn't make sense.
But think about it.
What is the capital of France?
If I think of the important words there, there's France and there's capital, probably.
And probably 10 or 15 times, I've moved France and capital, I've moved France closer to capital, right?
Over and over again.
And as France is moving closer to capital, it's getting closer to ours.
Okay?
So effectively, I now have my word and then I keep going.
And I get my next word.
And effectively, word one is that signal.
I flaggle through it again.
I get word two.
And I just keep doing that until I get my result.
My full sentence returned.
Because when you end, when you end the system, when you end the sentence and when you are done reporting,
I can't get into that.
There's a lot of other things going on.
But I'm trying to give you a fundamental understanding of what's happening there.
Okay?
Any questions?
Yes, sir.
So with attention blocks for user to write that, in fact, the way it works is that,
every new query gets a completely new attention block.
Okay?
But as you're working through the query, what's really interesting,
the first word is generated after you're going through nine times.
So once you have that first word to get to the second word,
you don't need to start over again.
You just take that word and you compare it to all the previous blocks.
And that gives you your next word.
So the mathematics of, do I need to run the whole attention block for every word?
No, absolutely not.
And that also is covered in that big video series I talked about and explained why.
But effectively, the first word, once you run that first word,
attention wise through all the previous vectors, the next word actually just comes out.
Crazy?
Yes, it works.
Amazing.
Other questions?
Great question.
I had the exact same question.
Okay.
So let's walk through this.
Let's actually look at some examples because you may not believe me or you may think I'm crazy or whatever.
Okay?
So what is the capital of France?
It tells me the capital of France is powerful.
Okay?
Again, there's a sense that you've primed this with, like, a previous sentence.
Again, normally you would think Paris would be the first word,
but it kind of has a structure of answering a question.
Again, again, I'm not going to get into that.
The capital of France is, I can ask the same way.
I get the same answer.
Where is Paris?
That's kind of interesting.
I get a whole nice description there.
All right?
That cool?
So it knew the difference between, like, the two cases.
Okay?
This is a good one.
Columbus is in Ohio.
Where is Paris?
Again, remember the earlier words reflect change the later words, right?
Remember the extension blocks.
So Paris is in France.
It says, also, there's a city named Paris in the United States,
located in Texas.
So if you're referring to that, please specify.
So it's trying to give you a hint.
You mentioned Ohio, maybe they want some US stuff here.
Now, if I say, I live in Texas where is Paris?
Now, it gets kind of interesting.
It actually starts talking about Paris, Texas first.
Then it talks about Paris France.
You understand how the attention factors are changing what's going to come out at the end.
Any questions?
Yes.
How does it know what to stop?
Okay, so I can't really cover that.
You can specify whether you want a long answer or short answer, actually,
as part of the chat GPT or the chat AI system.
The whole sentence structure is something I'm not covering,
because I didn't understand it would be a whole different talk.
I'm already feel like I'm being aggressive or ambitious.
So you could question, no.
Okay, so here's something called retrieval augmented generation.
If you've ever heard of that, it effectively allows you to pre-fix a query with some information.
All right, now that you've sent my sounds stupid, it might sound useless.
But once I show you some examples, I think it'll be interesting.
So retrieval augmented generation, you can say, where's Paris?
Okay, now if you pre-fix your question with reply briefly,
all of a sudden you get a much shorter response.
Okay?
How about if I do, I can use it with a little program called RAD.
I can say reply briefly, and what effectively happens when I send the information to chat GPT.
I have a special system role, which says pre-fix my query with this description.
That's effectively how open AI works.
Okay?
So, bonuses in Ohio, where's Paris?
Bingo, I get city of Paris.
I live in Texas, where's Paris again?
I get, if you meant different city, Paris, Texas starts to talk about that.
Okay?
Here's kind of cool.
If I'm using the following RAD data for each query,
okay?
So now I'm using some SQL data.
This is a JSON document.
Again, I don't want to spend too much time on it.
I'm running out of time.
But effectively what I'm doing here is I'm giving you some SQL data.
And now I can ask questions from to the chat GPT using my SQL data.
So I can pre-fix all my queries with this.
And I can say, when were the orders made?
I get nose.
It can read this.
Understand it.
And it can give me a human answer to the question.
Okay?
When is the earliest invoice?
It knows that too.
What is the totally invoices?
It knows that.
In fact, it shows me how I computed it.
What is the average invoice amount?
It computes that for me easily.
Okay?
What is, this is great.
Today's April 15th.
What's the 38th payment expected in which invoice they're overdue?
It does that for me.
Cool.
Okay?
What is the totally invoices that are not pencils?
Cool.
Okay?
So you can start to see how I pre-fix my query with some data.
Maybe coming from the database.
I could actually do some cool stuff.
Okay?
Today's April.
What's the next expected order date?
It actually kind of looks at the pattern of orders.
It was in figures out like when the next expected one is, kind of cool.
I can even generate SQL from chatGb3.
I'm cool.
Okay?
I can pull data from the database.
So again, I can even run SQL and have it pull information and automatically query that kind of cool.
Again, and this is what I wanted to kind of alluded to in the beginning.
We're only a couple years old.
Now Google originally invented this.
In 2013, there's a word to VAC paper they could put out.
And that really started it.
And then another paper for Google in 2017 for attention blocks.
But what happened was Google was so focused on text search that they never really saw the idea of pulling data together from different places.
And that's why chatGb3.
They had, they did the groundbreaking research,
they focused so much on revenue generating activities like web search and advertising.
They kind of missed the boat and they're playing catch up, which is kind of surprising.
But that's the way it is.
I'm going to finish with two quotes.
Okay?
I know two minutes over.
First quote.
I'm just going to read it.
Second quote.
I'm going to read it.
Just trying to have both quotes in your head at the same time.
First quote.
Here I go.
If you had said to me in our lifetime, we would have the capabilities like we had today.
Now, chatGb4.
If you ask me, that here I go.
If you explain the kinds of things to chatGb4.
I probably would have told you a year ago.
I don't know if we have this capabilities in our lifetime.
And now we have it today.
So the speed at which this is moving is staggering.
2023.
Okay?
That's one approach.
Second approach.
We are used to the idea that people or entities that can express themselves or manipulate language
are smart.
That is not true.
You can manipulate language and not be smart.
And that's basically what large language models are demonstrating.
2024.
Okay?
Both are true.
I would say.
And that is the end.
I appreciate.
I'm sorry.
I went over time three minutes.
But I do appreciate taking questions.
I am over time.
So this lady will decide when I'm finished.
I will stay here as long as possible.
If anybody has questions.
I very nice presentation.
I really like the last quote.
You say you're giving a presentation in Amsterdam or not.
Right?
When and where is it?
Maybe.
So I'm not presenting an Amsterdam.
I'm just attending the Postgres Meetup in Amsterdam.
So I think I came too late to get on the schedule basically.
So I make it like a lightning talk or something.
It's at JetBrains.
And if you go to meetup.com, you'll see it.
Great.
At some.
Please be quiet at the back.
We're still asking questions here.
All right.
So at some point you showed the chunking process.
So we can chunk by word by sentence.
By paragraph sometimes or the entire article.
That's right.
Is there any strategies or approaches documented so far that
that showed that chunking this way is better than that way?
Yes.
So the slide I talked about chunking has a URL at the bottom,
which talks about the various chunking.
I'm trying to get to it here.
I think it's here.
Here.
Right here.
Guided chunking strategies.
Guided.
Document chunking for AI applications.
That is where you would love.
Yeah.
All right.
Thank you.
Because there's different trade-offs for different chunking.
Yeah.
Great question.
I had the same question.
Yes, nice.
We're getting a little extra time.
Thanks.
So my question is about hallucinations.
So they do.
I get it now a bit better from your explanation.
But could you also give yourself an explanation about how hallucinations happen?
Or they seem to be a feature of the.
Oh, yeah.
That's better.
So it's the questions about hallucinations.
Yeah.
So the reason hallucinations happen.
Is actually related to this.
This, I'm sorry.
This is it.
It's not that hallucinations are something wrong.
It's our expectation that the AI has any idea of what it's saying.
Right?
Any idea at all?
It's our fault.
In a way, it's our fault that we think.
This is fundamentally the issue.
All right.
I'm not saying this is in true as well.
Okay.
But effectively, we're projecting what we think as logical on to something that has no idea what it's saying.
And in fact, I'll try to show you specifically how mathematics generates the effects.
But there's no knowledge of what it is.
The only knowledge is effectively those text embeddings that it learns from other documents.
Right?
So hallucination is something never going to go away because it doesn't know.
Like, if my kids start answering the way some of these, you know, like, you think there's something
that only wrong with them, right?
But again, this is not a child.
This is a mathematical thing.
And it has no idea what it's saying.
They're continuing to kind of fine tune it.
But effectively, it doesn't know when it's never going to know.
I'm going to have to let her control it, guys.
I'm sorry.
Thank you for the talk.
All these things are about metrics, multiplication, or metrics, vector multiplication, etc.
And by digital computers, what about analog computers for vector multiplications, or metric multiplications?
It could be much faster.
Yeah, I have to never, I never got into that.
Only because I was trying to come at this from a database perspective, being a database postgres guy.
I wanted to kind of talk about having understood the foundations where it is postgres fit in.
That's a different space.
I can't, I don't, I never read anything about that.
I'm sorry.
No problem.
I appreciate your coming to the last session.
That was nice.
Thank you.
Yeah, it's just a quick one.
When you showed how to do vector retrieval on postgres, you had one example that was kind of counterintuitive,
where you were asking who is short, and you got the sentence that King is told.
And of course, this is because it's just retrieving information semantically close, right?
Right.
So if you can end up with this result, I'm wondering, like, what are good use cases for this?
Because you're clearly not here, it seems you're.
Undesignated or free Mac, and it knows to come back and say, it's called the unassigned Mac block.
And here's what it is.
So in that case, I couldn't think, I couldn't remember the word for unreserved or, you know, whatever, unassigned.
But Jackie B.T. could see, I talked about Mac ofgresses.
I'm talking about that and it could kind of give me an answer that it found somewhere that was more.
relevant to my question than Google could.
Another case, flying here, I'm getting confused between the ticketing airline,
the co-chair airline, and the operating airline.
And I go to search.
Three of them.
And I get websites about co-chair.
I get websites about operating airline.
But I don't get anything together.
I go to chat to you, B.T.
I said, tell me about ticketing airline, co-chair airline, operating airline.
It gives me a nice three bullet point description of all three of them.
So in the first case, it took what I lied.
Use vectors to find something close to unused.
And mute was unreserved.
That was the term and then gave me the result in the second case.
It brought information from different websites.
One website was about co-chair.
One was about ticketing.
One was about operating.
It's very complicated the way tickets were.
And it could kind of pull that together for me.
I think that's really the value.
You're right.
And it can do stupid stuff, too.
So that's never going to end.
I'm afraid.
Yes, thank you.
I'm sorry.
Yes, sir.
Hello.
So my question basically is that we also using vector in postgres.
And we are using pdvector.
So the question is, is there a plan to have a native vector data type in postgres?
Because we are lost in the space of plugins.
You don't know what the index is.
It was used.
So I'm right.
So, you know, that's a great question.
When we brought full text search into postgres, it actually was a contrib module.
By the Russians.
And then eventually it was added to postgres.
And then once it was integrated, there were some cool stuff we could do.
The beauty of what we have here is that when full text when AI became important,
we had pdvector already.
It was already there.
So we don't like geniuses.
Because you don't have to do anything with postgres.
Just all the instant you've done.
Same thing happened with data warehouse.
Same thing happened with full text search.
I think happened with the cloud.
We'd have to do much because we could just kind of change what we needed.
So with pdvector, I think I like the fact that it's one of different development cycle.
So as because the AI space is so volatile, I kind of like the fact they can release more often
than postgres does.
Right?
And improve it independent and then we, all you have to do is follow the extension.
So that, for I like whether the technology will eventually solidify,
where we want to move it in, that may be a case.
I don't know when that would be.
Great question.
I think this will be the last question.
So these searches seem quite expensive clearly.
I know someone asked for a bite chunking and someone asked when does it stop.
I'm wondering if you have any like quick insights of ways that you can search for.
The ways that you can search that make that less expensive or make it.
Yeah.
So I think when I was studying this before deep seek, which came out this month week.
When I was studying this, it was pretty clear that some of this was overkill.
Right?
You're at 1,000, 12,000 dimensions like the vector space is unbelievably big.
Right?
And a lot of times what happens when you bring a new technology is you don't know what to optimize.
I'm thinking of the early compilers.
Right?
Like we have compilers now that are unbelievably good.
Like you know, I'm a postgres guy and sometimes I'll compile something and I'll look at the assembly and I'll be like.
In a million years, I've never coded the way they did.
Okay?
But the compiler people know like the order how the registers work, what can be pipeline, what can be proud.
I mean it's genius.
Okay?
GCC, all these guys are amazing.
What I think we have now is the space like where compilers were in the 60s.
They're just trying to get code that works, right?
There's no optimization going on.
What I think we'll see in the next couple of years is a sense that maybe we don't need to be doing some of this stuff we're doing.
Okay?
And I think deep seek, although there is concern that some of the training stuff they're doing has used training from other other open AI and other,
that actually is cheating, but in terms of the way they do attention blocks and the way they actually do the query part, there are actually some optimizations.
At least I understand I watched a couple of videos on it.
It seems like some of the optimizations they're doing could be used by everybody.
So I think, although this is quite expensive now, I believe that over time this is going to become much more streamlined.
I think we will have as many issues.
That answers your question.
That's hard to say, but it's fascinating.
Again, the three pieces are the embedding search, the rag, and then the generative part and how databases for me, you know, having understood how all works I can understand where data base fits in that space.
So thank you very much.
Appreciate it for staying for the last session.
Thank you very much.
Thank you so much for the vain commitment presentation.
Thank you very much.