Well, that's what we also realize, and we do details, specifications, papers, generators,
data sets, and all sorts of frameworks to make our benchmarks be easy to implement.
The benchmarks are easy to implement, but they are not easy to get audited.
We have TPC grade benchmark audits, which means that we have certified auditors, and if
you're a vendor, you have to talk to an auditor.
It's usually something like 30 to 50,000 euros.
It's a multi-week process, the auditor, we'll talk to the vendor.
They will compile the specifications sheets, and then they will, of course, rerun the benchmark
and write a full disclosure report out of it.
This is kind of a painful process, but vendors go through it, because this is kind of the
gold standard and the graph database space, and we have been doing audits for five years,
probably 50 results, and never had to retract any of the results.
The most issues I could find was a few typos in old food disclosure reports.
And as you can see, we have grown quite well in the last five years, so we have more
and more members, lots of user community meetings, benchmarks, results, and so on.
So what are the challenges in the graph database space?
Well, there's a big decline.
I don't think that needs to be denied.
The high cycle has moved over to AI technologies, and I think there's a huge amount
of confusion.
I'm trying to have the confusion here a bit by setting these categories, but when vendors
go out saying, we are a graph database, you don't need joins, and we will replace all
relational systems.
I don't think that does anything, anyone any good?
I don't think anyone is seriously saying any more that graph database is very
replace relational systems.
There are many niche use cases, and there are vendors who are basically tied to one
or two of their customers, they overoptimize for that, and that ends up in this very
fragmented space with dozens of clear languages.
You can see the effect of that, so between 2013 and 2022, there was 13 fold increase in
the DB engines ranking for graph databases, but since the summer of 2022, it has been
a decline.
So we are now back to basically two thirds of the peak, and the decline doesn't look
very good.
If you have heard about graph frag, the vendors have heard about this, this is the
life that they are trying to cling on to.
This is something that the vendors think is going to help them, and indeed it produces
a lot of attention, and it's nice graph use case.
Okay, to sum up, graph databases have syntax sugar and optimizations for joins.
If you have 10 joins in your query, and for some reason it doesn't work value in a relational
system, you should try a graph database, but then based on the problem you may want the
MongoDB, the Postgres, or the Teradata of your database, and licensing is difficult, check
to licensees before using the software, and check the performance results.
Thanks a lot.
Thank you very much.
Thanks.
Hello.
Thank you.
Hey.
Yes, I'm going to hand over to Danny Allen, then I agree with you.
If you can do all questions outside or somewhere different, that's a nice question.
All right.
I do not know what's the next talk though, because Danny I was my student, so...
Thank you very much.
I don't think you should see the line.
It's too close, aren't you?
Yes, but it doesn't seem about doing anything.
We take all the questions outside, and he's going to let's be grappling.
Do you want to know who's there?
Yeah, okay.
So you've got a map.
Yeah, so we've got...
Okay.
Normally what we've found is the issue I didn't work.
This one does, so don't try that.
All right.
See that.
How's it going?
It's going to be fine.
Yeah.
If we can move, yeah, it gets to go and move outside for questions.
Please, that'd be great.
Here's your...
Sorry.
No.
At least that is him off.
Horrible.
What's your other name?
So, what's your shop out of here?
Yeah, it might be easier, might it.
Hold that.
So, right.
Let's see if that works.
Okay.
That actually meant a bit closer.
Sorry.
It's really hard, actually.
Yeah.
You know what?
I'll show you something.
I think.
Then quite nice happened there.
No idea what's up what this is going.
Okay.
Sure, what's going on?
I think it's what I think what it is.
It's that goes down like this.
Okay.
That goes in like that.
Exactly.
So then, that goes in like that.
Like that.
And then I can do it like this.
It's just right up here.
Oh, that's lovely.
Yeah.
It should be really lovely.
So, unfortunately, the audio is not great for him.
Yeah.
I've got, I've got a bad job here.
I think I'll miss him up here.
Okay, let's try doing it in here.
Okay.
That should hopefully be okay.
Okay.
So yeah.
So just project.
Yeah.
Yeah, I'm sorry.
Actually, you know what?
I think it's not in the work.
Sorry.
I'll fix this.
So, yes.
You were on the previous door.
So you know what the situation is with the audio.
I'm really sorry about that.
Yeah.
Yeah. I have some overview of it.
No, not overview of it.
I basically took it over.
Overlapping with a guy around me.
I hope it should be there.
No.
You know, it was interesting to see that the interest in graph actually is really growing.
You know, I'm not sure how much of that is because of graph rag and, you know, the kind of thing.
Maybe it's something there.
I don't know.
I just do not know how this works.
On that side.
Let's try it one more time.
Let's see.
Hold this.
Okay. Hopefully.
Okay.
Can you put on your pocket maybe?
Yep.
I'll add a timer.
Okay.
Next session is going to start in five minutes.
What do you mean? This is my uniform.
This is really good.
This is my uniform.
Oh, I have. Thank you.
I have.
Thank you.
Oh, this is a gig.
Thank you very much.
Thank you so much for agreeing to do this.
Like not.
Yeah.
I have some overlapping slides with camera.
Like overlapping topics.
I hope it's good.
I don't know for sure.
At the beginning, we were able to extend because you know with the dark cone.
We didn't want to fit like that.
Yeah.
So it's like they were like really good talks.
I said they were more talk like not from that cone.
In the proposal, but we were like, I don't know.
I've been in both events because it's like, yeah.
Yeah.
If it's kind of like a advertise for them freeing event,
I would find it is different cities so much.
Yeah.
For them freeing.
It was like, it can be, but it's not.
It's a bit too big.
So they were pretty pretty cool actually.
Yeah.
But like, I don't know man.
So yeah, about about that.
Yeah.
It was like very high rated.
I got it.
Thank you for making this.
I'm happy to.
I think it's really nice to share the,
but it's really nice to share the.
Yeah.
It is.
Yeah.
Thank you.
Happy to.
Yeah.
Acid that because our,
Yeah.
Yes.
Yeah.
So we are going to start in one minute, approximately.
We are delighted to have Daniel, the 10-fold.
10-folder?
Holder.
OK.
I'm getting a good exercise in pronouncing the name today.
Who is going to be talking about empowering data analytics, high performance graph queries,
duct DB with duct LGBTQ.
So who doesn't need more ducts in their life?
Right?
I know there are also creatures starting up ahead of you.
Starting in one minute, so please, again, move in because we have people who are coming
and we don't want them to disrupt yourselves.
If you can, squish in and leave people as places who should be joined like that, that's awesome.
So thank you very much, and over to you.
Thank you very much.
All right, can everybody hear me if I speak like this?
For a bigot?
All right.
Well, my name is Daniel Tanvolde.
I'm a PhD student at the CWA database architects group.
And today, I'll talk to you about using duct DB and duct LGBTQ to do your graph analytics.
OK, so first, my topic is very similar to the one that
Gauber described in the previous talk with graph data management.
So there is some overlap, all admit.
But with graph data management, we often refer to highly connected data.
So many to many relationships are very common.
And very often, tables already represent graphs.
So in this case, we have special vertex tables and special edge tables.
And then people want to do graph exploration over this.
And what that means is that there's pattern matching where we want to find a sub-graph.
We want to do path finding where we want to find the shortest or cheapest path between
any two sets of nodes.
But importantly, people still want their filters and their aggregations.
So these three building blocks are important.
So storing a graph in SQL on the right here, we see a property graph.
And the property graph data model extends the normal graph with labels.
So we have a person label, follows, lives in and city.
And every node in edge can have properties in the form of key value pairs.
So storing this in a relational system is really not the issue.
For every label, we create a different table.
And the properties are the columns within these tables.
All right, nothing special going on here.
But now we're tasked with the following prompt of counting the number of people
Bob indirectly follows, who live in the city, Utrecht.
And Gabber gave it away a bit in the last slide.
But this is what you would have to write in the SQL of today,
SQL 999, where the recursive operator was introduced.
And you have to recursively explore your pulse and keep track of them yourself using an array.
So from a user perspective, this is not nice to write, not nice to read.
I would argue that there may be some of you out there who like to read this.
And from a system perspective, importantly, it's really difficult to optimize these styles of queries.
So what did people do?
They moved towards graph database systems, which had a arguably nicer query language.
So the most popular out there is Neo4j that introduced a visual graph syntax that
Cypher has.
But you also have Amazon Neptune that supports both Sparkle and Gremlin.
You have Genesis Graph with Gremlin.
You have Oracle LFPGX, which supports PGQL, Type Graph with GSQL, or Nebula Graph with NGQL.
And the government mentioned a couple more that I couldn't list.
So the landscape is very scattered.
Now as part of the SQL 2023 standard, yes, there is a standard you can buy it for 221 Swiss
France.
If you really want to, but I don't recommend it because it's not really a light read.
You can buy it, but yeah, SQL property graph queries was introduced.
SQL PGQ.
And this enables you to create a property graph layer over your already existing tables.
And then you can use the visual graph syntax to do your pattern matching and path finding
more easily.
So SQL PGQ is read only style queries.
You cannot do updates or delete with this syntax.
And that's part of the GQL, the graph query language.
And before someone refers me to this XKCD of yes, there's 14 competing standards.
Let's create another one and now there's 15.
Yes, that can be the case, but I would argue that this is part of SQL.
So if you don't like it, you don't have to use it.
You don't have to switch all your data to a new system.
So I think this is more integrated.
This is actually being already implemented in a couple of relational systems.
Oracle is really going in on this.
Dr. B with Dr. PGQ.
There's Google that supports in Spanner.
And actually they edited to the Google SQL.
So also other systems such as BigQuery could support it in the future.
And I know that Postgres has ongoing work with Peter Eisentras.
So it is being implemented.
Now, how would you create a property graph in SQL PGQ?
This is the first step you do.
We create a property graph called social network.
And here we explicitly define what are our vertex tables, in this case,
person and city, and what are the edge tables.
So for the edge table of follows, we go from a source person
to a destination person.
And here we specify the keys that we're going to use to join on.
The joints are very important in our use case, in the analytical space.
And if you define your primary key form key relationship already when you created the table,
you don't necessarily need to repeat them here, because I can deduce them in some cases.
Now we got back to the same query, count the number of people in directly follows,
living in the city of Utrecht.
And so maybe you remember that 3 times 6 plus 3 times 8,
you can also represent as 3 times 6 plus 8.
And this way, you have one less repeated value, in this case.
So we need to use less memory, and operating on this compressed value is actually very efficient,
because we only have one value instead of many duplicates.
So that to be used mostly, hashed joints, which first builds a hash table on the destination key
of the follows relationship in this case, and then it probes it with the source key of the intermediate result.
So how the implementation of the join is inductively, it was slightly changed with this master project,
which is actually very impressive, because the join operator is not easy to work on.
It's a core component of Dr. B.
So before when you wanted to insert a value that had a collision, in this case K2 will have a collision
with K1, then Dr. B would still insert it into this chain.
So you would kind of get collisions, well, you would get non-duplicate values,
or duplicate values in this chain.
But now with this new change, we actually insert it into the next slot.
So you get a unique keys per chain, and this is nice, because, oh wait, yeah, this is nice,
because now we have pure chains, and we don't need to go over all the keys in this chain to check
if there's a matching key. We know that if the first key doesn't match, we know that all
other keys in that chain don't match. And then there's a little optimization, which is the salt,
which is the upper 16 or so bits, because we don't actually need them for the pointer,
that can be used to do a fast filter on these keys. So we benchmark this on TPCH,
which is the industry standard for analytical systems. And we actually saw a big improvement on
especially high scale factors, where there's longer chains, so you don't need to traverse these
long chains, and the filtering on the joints with the salt optimization. And this was actually
merged as part of that to be 1.1.0, and that improved the joint performance quite nicely.
But how does this relate to factorization? Well, this pure chains, you can actually see them as
a factorized result, because now if the pointer in the hash table, we can store that,
and then for every value of E, we can point to this pointer, and then we don't need to repeat
the actual values that are in the chain. So this removes any redundancy from the probe side,
and it removes redundancy from the bill side. And this is actually something we learned from the
3D hash joint paper, which is something very similar, but not quite like this, but this gave
us the idea for this factorization. And additionally, if you have an aggregation on the same key,
arguably maybe a bit specific, but if you have an aggregation on the same key,
then you can actually use this factorization to do the aggregation very efficiently.
So we have an additional data structure where we can store the result of the aggregation for every
key in the chain. So for instance, if we want to get the length of the chain, in this case,
we only have to compute it for the first time we come across this pointer, which is the length
is 2, and then the next time we come across this pointer, we already compute it to value. So we don't
need to traverse the chain again. We only have to look up the value. And this gives a really
nice performance improvement. So the baseline or the vectorization with the pure chains
already gave a bit of a performance improvement about 1.25x. But with the caching, it's actually
17 times faster. And this is maybe a very specific case, but it does show a very nice optimization.
So there's more work that we did that is published as part of the side of the 2025
conference that was work with Paul and me and Peter Bunks. So we also looked at how you can
adaptively use this factorization technique because you don't always want to trigger this factorization
because it does give a lot of over it. But there's more details in the paper.
Okay, I see I'm quite quick with the presentation, but if you want to try out that pjq,
it's really easy to get started. And still that pjq from community load that pjq and you're good to go.
There's a QR code that leads to the website that has more examples of SQL pjq queries and more
documentation. And we're also on get up, I would appreciate a star. And yeah, to conclude,
I introduce dcq, which enable SQL pjq queries within dcb. We outperform Neo4j and crucial to be
for analytical pattern matching. And I showed how you efficiently handling of path finding and ongoing
research work. Thank you very much.