Thank you very much and thanks everyone for coming.
That's amazing to see the full house here in the search Devroom and excited to be here
at first them 2026 and on this session we'll talk about open search and search innovations
with open search v3.
I'm Dutant Horvets.
I'm open search ambassador under the Linux Foundation.
I'm also a sincere ambassador and then my day Joe by Miss Senior open source advocate at AWS.
And I'll be co-speaking with Ashwatt Senior Search Engine architect for open search at AWS.
So for those who don't open search it's essentially a platform for AI powered search,
observability and analytics.
On the search side it provides the full suite from lexical search to a dense vector search,
we have the interesting talk now hybrid search, multi-model search, agentex search.
So the full end to end search capabilities.
It's Apache 2.0 open source license.
It's a top-level project under the Linux Foundation.
It started back in 2021 as a fork of elastic search and kabana.
After these projects were released from Apache 2 to non-open source licensing scheme.
So the community wanted to keep them open, AWS stepped up and initiated the fork.
And then fast forward to 2024, AWS transferred the project to the Linux Foundation.
So it's owned by the Linux Foundation or any vendor.
And there's the open search software foundation under the LF to foster open governance and open collaboration.
And I'm really happy to say that many companies have already joined the foundation.
And you see some examples here, Uber, SAP, NetApp, IBM and others.
And are taking part in studying the future of this project.
This is an asset code base, 140 reports under the GitHub org, 1.6 billion downloads to date.
It's a super active project, more than 3,000 active contributors from more than 400 different organizations.
It actually, I think, I checked the LFX leaderboard.
It's I think the 12th top active project under the LF amongst the nearly 1000 projects.
So pretty impressive.
And the biggest news in the past year is being that after 3 years of 2.x,
we finally reached version 3, the next major release.
And this major release has foundational changes.
Also Apache Lucine moving to version 10.
We have architectural changes, such as the reader write a separation to separate between the search and the indexing workloads.
Streaming architecture, pool based ingestion, native MCP support.
Many new commands on the PPL, the piped processing language, the query language we have as part of the project.
And support out of the box for Apache calcite as the engine and much much more.
So on this session, we'd like to share with you some of the highlights of V3.
And I would like to hand it over to us to start showing us this stuff.
Go ahead, that's what.
Thank you.
Okay.
So, I'll start, thank you.
The passing code is the neural spaces with an open search.
I like to call this as a budget-friendly semantic search.
And if you have used the dense vectors, you will probably agree with me.
The neural space model that is there in hacking phase, pre-trained by open search,
is one of the most popular, most downloaded space model.
But the catch with the neural spaces has always been the scalability.
Beyond 10 million documents.
The latency is...
Yeah.
So, beyond 10 million documents, the latency is quite high.
To an extent where I start recommending dense vectors.
But at that point, we are no longer in the budget-friendly semantic search.
So, with this seismic algorithm implementation,
we are able to push past 10 million ceiling.
We are able to go to billion with the B.
And this is by being able to do approximate sparse retrieval.
So, with the new data structure, the forward index that actually has the dot-hyd to the
first pass vector mapping.
And then we have the inverted index, which probably most of you understand,
but with some slight changes, where the posting list goes through a pruning process.
It only retains that top weighted dot IDs.
So, for example, let's say the term, dot, exist in dot 1, dot 2, dot 5.
And each document that term, dot exists with different weights.
So, pruning essentially removes those dots that have lower weights.
And then what we do is the group similar documents, forming a cluster.
And for every cluster, as summary vector is created.
So, this is what you see in block 1, block 2.
So, it's basically the cluster with the summary.
And now that the index is created, during search time, the query vector,
queries against the summary vector.
If the score is too low, the cluster is simply skipped.
If the score is decent enough, then the cluster is examined.
And for the full final score, we go to the forward index, test the full
speed to do the final score calculation.
So, this is how it works.
I mean just to summarize.
So, ingestion, then pruning happen, pruning process happens.
And then similar documents are grouped together, forming a cluster.
And every cluster gets a summary.
Lock 1, block 2, search happens.
When the search happens, we query the summary vector.
If the score is too low, the cluster is skipped.
If the score is big enough, large enough, then we examine the cluster.
And the cluster is examined.
The full vector is gotten from the forward index, and the final scoring is time.
So, this is how we are able to do approximate sparse retrieval.
And this is how we are also able to scale beyond 10 million ceiling that we have in the past.
Yeah.
So, give it a shot.
Give it a try.
Let us know how it goes.
You know the drill.
Yeah.
So, we three GTE is the Sata Infran3 pre-trained model that I was talking about.
The most popular, most downloaded hugging case model.
Yeah.
And like I said, right.
So, the benchmark that we did over 1.3 billion dot set.
The benchmark that we did.
Look at the box here.
The latency with the BM25 is baseline.
We were able to get four times faster than BM25.
And about 11 times faster than the traditional neural sparse.
Yeah.
And the line about tells you the recall, which is a slight drop in recall.
Has a such architect, such practitioner for me, a slight drop in recall, with such a benchmark
latency that I get.
It's a tremendous trade-off for me.
I'm not sure what you.
Yeah.
So, yeah.
Please try.
Give it a shot.
Yeah.
So, I will move on to a new topic.
System generated such pipeline.
But in order for me to explain this one, you need to know that such pipelines exist with open
search.
And such pipelines are very similar to the ingest pipelines.
A bunch of processors for data transformation that you put together.
But on the set site, right.
The only difference is the processors of different types.
It could be request or response processor or a face processor.
To give you some tangible examples, the normalization processor for normalizing the score
between lexical and cane and search.
And they cross-encorer for re-ranking and the semantic highlighter.
Yeah.
So, these are different processors that you put together to create a search pipeline.
But the challenge is not the challenge, but the problem is always that you have to create
them in advance, define them in advance, register them in advance before you are
able to use them.
Right.
So, as the name suggests, the system generated such pipeline.
It attempts to eliminate the need of having to create them in advance.
So, to give you an example, if you are implementing e-commerce search, let's say you want to
diversify your search results.
And you want to have MMR, maximal, marginal relevancy.
Now, you don't have to create these search pipelines in advance.
You simply have to mention it in a such class.
And the search pipeline is automatically created for you on the fly.
So, it removes the operational overhead that you have.
But the only setting is it is not available for all the search processes out there yet.
Yeah.
So, you still have to create the search pipelines.
And for anyone who is new, there is a steep learning curve because you need to use the
APIs to create the search pipelines.
Yeah.
So, once again to reduce the learning curve to cut out the learning curve, we unfor those who like
the click-click create approach.
So, we have this.
So, you have, as you see here, right.
So, different templates for hybrid search, multi-modal search, rag.
And also agent search.
Yeah.
So, you have different templates which you can use.
And you can also use the custom search if you want to, you know, go beyond the basics that we have.
And it would look something like this.
This is a full-blown screenshot.
But they saved me.
So, it goes like this.
So, you create the ingest pipeline.
Right.
You can add a bunch of processes into it.
I have the ML inference processor that can convert text to embeddings.
And then, of course, you can add more processes.
This does an example.
Yeah.
And then, we have the search pipeline where I have the same ML inference processor.
I'm working to the text to embeddings.
Yeah.
And this is how you define what the processor should be doing.
So, I then, it allows me to pick what model I want.
Which field should I look at to convert the text.
And once I convert the text to embeddings, where should I put it to.
So, basically, you can configure the entire processor with the UI.
Yeah.
Not just that.
You can also test.
Simulates the entire flow in here before you start deploying it.
Yeah.
So, it's a very nice utility, a developer tool.
Yeah.
But let's say you are at this point.
You have created the search pipeline.
Everything is good.
You have deployed the next thing that you need to know is.
It's the search pipeline working at all.
Yeah.
So, enter search relevancy workbench and ask some tools.
I would say, especially for your relevancy engineers, your data scientists.
Yeah.
So, as it shows here, right.
You have four different things that you can do.
You can compare single query.
You can like side-by-side compare.
I bought the search results.
You can also do other volume using query set.
You can also do search evaluation, which is basically all the.
Good search quality metrics.
The NDCGs and the positions.
Yeah.
And the hybrid search optimizer, which is slightly different from the rest of it.
So, if you have used hybrid search, regardless of the vector database, you let's say you combine
Mexico search and can and search.
Right.
So, you need to.
There's a lot of parameters, hyper parameters that you need to tune.
So, the normalization technique is that this core or min max or
equilibrium.
And then you have the combination technique.
Is it arithmetic mean or geometric mean or harmonic mean?
Yeah.
And then lastly, you also have the combination.
Distribution, which is.
Legs equal to cannon.
Is it 0.1.9.2.8?
Yeah.
So, you, you then compute the combinations that you would arrive at.
That's way too many combinations.
Then how would you know, this combination works the best for you.
This is what the hybrid search optimizer does.
I will show it in the demo, which is what I'm going to do right now.
So, this is the new open search dashboard.
This is not just the new look and feel, but it also lets you add different open search clusters
to the same open search UI.
So, this we go past the one dashboard to one backend.
Now you are able to connect to different backends, open search clusters.
And you can also create work spaces, a classic multi-tenancy use case, right?
And then you create a workspace.
It could also be of different types of observability or security analytics search
depending upon it.
The menu changes.
And of course, if you want all the menus in the world like me,
I go to the analytics.
I get all the menus in the world, exactly.
So, first thing I'm going to show you is I'm going to show you the agent search.
And I'm going to compare that with the lexical search.
So, for this, I go to the search relevancy work pens, single query comparison.
Yeah?
I have to type the query.
Of course, I'm not going to do that now.
I have it in here.
And what you see here is I'm going to compare agent search with the multi-metge lexical search
that you probably are used to.
I'm going to use a local cluster for both.
I'm going to use the e-commerce index for both.
But of course, on this side, I'm going to use the agentic pipeline plot.
This is a search pipeline that I created.
Yeah?
And I'm going to search for, let's say, Nike.
Yeah?
I'm going to hit a search.
I have a very nice comparison here.
There are three overlaps, seven unique results on each side.
I'm going to change this to title.
Make it a little bit bigger.
So, the results are quite nice on both sides.
But, of course, nobody is going to search for Nike.
Nobody is going to search for jeans, right?
So, they're going to be a bit more explicit, right?
And I want to make a typo here, right?
Nike black.
Okay.
Nike black shoes.
Oh, one more unintentional typo.
But, yeah, agentic search is able to handle it, right?
So, if you now look at the results, right?
This is no longer Nike.
I mean, it is still shoes.
Yeah?
But, it is no longer Nike.
On here, I have much more as a end user.
I find this one much more relevant compared to the other one.
Yeah?
Now, let's continue the journey.
Let's say under $50.
Yeah?
So, now, I no longer get shoes.
I get cookies.
I get whatnot.
Yeah?
Once again, on this side, I have much more relevant results.
Of course, I show socks here.
Also, you know why that is happening.
But, also, $50, you get what you pay for, right?
With this inflation.
So, I'll show you what happens under the hood.
There's no magic.
Yeah?
So, for this, we go to the AI search flow page.
Remember, this is the one I showed you.
And this is the one I use agentic search for creating this workflow.
This is the one that I created.
And the agent I have.
This is the agent that I created.
So, let's run the search first before I start explaining it.
And this is exactly the same search that I did.
And this is exactly what is causing the magic, right?
So, agentic search is able to break down the user query.
Understand the user query.
Break it down into, let's say, 90 is a brand.
Shoes are category.
Black is a color.
And price is a range.
Yeah?
And this is how.
And this is the lexical search.
There is no gain.
And there is no sparsist.
There's nothing.
There's just lexical search.
Just a powerful rewriting of the query, right?
I get the raw results.
The exact same results just to make sure that I'm not cheating.
Yeah?
So, now we go to the raw response, right?
And we see in the category for the first result, right?
So, clothing shoes, jewelry and athletic socks.
No one there.
This is actually showing a socks showing up in the shoe.
Because there's a lot of cataloging problem.
Not an agentic search problem, yeah?
You also get the rewritten query here.
But who's going to rewrite it?
Which is what is happening on this side.
So, we have this query planning tool.
This is the tool for this agent.
And as it says here, right?
It is basically converting the natural language into open search DSL query.
By using an LL.
This is the LLM that I'm using.
Yeah?
Of course, I have another agent.
Because this is open-source platform.
So, we also have a agent that is actually running in Olamma
on my tiny EC2 instance CPU, yeah?
A 7 billion parameter.
I'm saying all these things because it's super slow.
Yeah?
But just to prove a point, this is going to give me the exact query.
If it doesn't fail me.
But the bottom line is you are able to, you know?
You don't need a super special cloud 4.5 model.
You are able to do with a model that is just under 5 gigabytes.
Yeah?
So, you get the same exact query, same exact result.
Yeah?
So, and of course, one last thing I want to say is you can also,
if you want to be a bit more deterministic.
Yeah?
Do not want to LLM to hallucinate.
You can also use such templates.
Yeah?
You can take a bit more control.
I think I'm taking more time than I may be running a bit late.
But I also want to show you these things that I was talking about earlier.
Right?
So, we did a single query comparison.
There is also a possibility to do a volume test with query set comparison.
Right?
Which is what I did here.
Right?
With 150 queries, you get all the ranking similarity metrics here.
There is obviously a bug in here, which is fixed in the next release,
which is what Jordan is going to talk about.
But you basically can, you know, what you did at a single query.
Right?
Can be seen at a volume here.
Right?
But I like to actually show this one, the such event, right?
Because for this one, if you go back, such evaluation, right?
This is also volume as I'm going to pass a query here.
This is just a field at query, right?
But what I also am passing here is the judgment list.
Because for such quality metrics for NDCG and for precision, you need judgment list.
So, open search allows you to add either implicit or explicit judgments.
Explicit, you just have to find it somehow.
Implicit, you can use a project called user behavior inside,
with an open search.
It allows you to collect click metrics.
Imagine Google Analytics, but such specific.
So, take a look at the project.
It's quite interesting, at least for me, as a such engineer.
But go, yeah.
The most nice thing is for your data scientists,
to be able to look at it at a very thick step back,
have a holistic picture.
We paint a nice little dashboard for it.
I'm not going to go over what is here,
but just to make a point that you can install the dashboards,
and you get this nice stuff.
And lastly, the hyper search optimizer,
I told you the different combinations for the hyper parameters, right?
So, when you actually run it,
you can actually see, T-Towels is a such query.
It ran for 0.8.2,
including an harmonic mean, 0.5.5,
including a geometric mean, so on and so forth.
You get the idea, right?
So, it ran for every single query,
every single combination of that, right?
And then it tells me, let's go to the dashboards.
Once again, it tells me,
which combination is the best combination,
best hyper parameter combination for me?
For my data set, with the judgment list that I am giving,
which is basically the arithmetic mean,
equilibrium 0.1 point, whatever, I'm not able to see it.
So, this is the power of this such relevancy workbench,
try it out, and I think I will have to change the screen.
So, thank you so much.
I ask for a given round of applause.
Come on. Here's the great job.
And the demo goes with with us.
So, we saw, we'll show you the QR code.
So, you can actually run the demo yourself using the dev tools,
API, or of course, through the open-setch dashboards UI,
if it's more comfortable for you.
So, you can do this, and it doesn't connect to the screen,
external screen.
And next up, we will show us some more search highlights in V3.
Go ahead.
Yeah. Thanks, Jordan.
So, I think we are running out of time,
but I want to tell you this one, right?
So, Jordan was talking about the PPL, the pipe processing language.
So, this is more or less a kind of a new programming,
not programming, but query language.
So, what we have is also a natural language ability
to convert the natural language into PPL query.
Yeah. So, as you see here in this nice little animation, right?
So, you don't have to, you know, for using it,
you don't have to know the query, get go, right?
You can learn it along the way.
But for more advanced users, oh, there's also nice little dashboards
that is automatically created when you use the natural language to PPL.
But for most more advanced users, yeah,
you can also create the complicated queries.
I will leave that there. I'm not going to say what kind of complicated queries.
But there are also new commands that you can use,
like Ragex.
You can also use a Grok pattern.
You can also use ML for camis and randomback forests.
There's a lot of new commands that are possible with PPL.
But the most striking thing for me is the ability to join.
So, join is a new command that is released in PPL.
Something that is long standing often asked.
Now, you are able to, you know, join multiple indices in one,
one one one place, yeah.
Oh, sub query is also possible.
I want to go with the rest of the stuff here,
but I'm going to pass it over to.
Thank you.
So, I want to talk a bit about performance,
because, you know, we're all about performance
as being a point of emphasis for the open search community since its origins.
A lot of users actually choose open search for its ability to ingest
and manage and explore massive amounts of data,
but with massive amounts of data comes obviously,
a premium of the performance.
So, we strive to build the most performant search engine out there,
and which is obvious since version one.
And this is why I brought to this plot from one.
One.x to three.
So, three.x, you can see.
And actually, the gains have been so big that we build a logarithmic y-axis.
So, when you see the graph going down,
it's actually means that it's been decreasing exponentially.
And, in fact, one that is across average across all query types,
which is the big five workload that you probably know the benchmark,
that's the dotted line in the middle,
is improved 10 times from one.3 to three.x.
So, pretty nice things to see.
And by the way, it's not just about the query latency.
We also see significant performance in the ingestion,
in the storage, in throughput, in the transport.
The dozens of new features in version three that contribute
to these performance booths.
So, we're running out of time.
So, I won't go over all of these,
but just to give you some highlights.
First, the GPU acceleration.
We only know here in the search dev room that vector operations
are heavy, especially the distance calculations and the K&N.
But, on the other hand, they lend themselves for parallel computation
with GPUs.
So, as part of the project, we've introduced
a ability to leverage Nvidia architecture,
the CVS Cagra algorithm,
which shows pretty impressive results.
You can see 9.3x improvement there on the indexing speed.
Cost by nearly four times.
That's on the GPU acceleration.
Another cool thing is,
we've seen over-ficed,
because we know that we've seen,
many of us know we've seen,
it's memory-efficient, but doesn't scale very well.
And on the other hand,
scales very nicely, but then again,
you need to load the entire
Agenus W graph into memory.
So, we came up with this clever thing of
Lucine over-ficed.
It gives the best of both worlds.
Try it out, pretty new,
learned for the CV.
The initial test show impressive results.
GRPC, with Protobuff,
again, a very good way to,
for faster, more efficient data transport
and data processing compared to the JSON and rest.
And especially with the
significantly heavier operations,
like bulk indexing, K&N,
search and all of these.
And also, it enables
use of platforms such as Apache arrow and arrow fly.
So, arrow, I don't know,
the great talk yesterday for those
attending databases there,
but essentially,
it eliminates the serialization and
serialization overhead.
And also, it allows for streaming architecture
with processing aggregations,
because it can allow to deliver
partial results to boost complex aggregations.
I took mentioned briefly before about
the reader write a separation,
so that you can separate the search and the indexing
workloads to avoid the congestion and get
more predictable performance.
And many more, again,
I won't have time to go over all of these.
So, I want in the,
to say a word about what's coming next.
The opposite project builds on an eight week
release train, so there's always
good stuff coming around the corner.
And in the coming month,
we're going to release 3.5.
And I wanted to just give you a taste of
some of the innovations in search and
observability.
It is the ones that I particularly like.
So, you know,
but just 3.3,
before last release,
we've added skip lists for
date histograms,
for those who don't know.
So, with 3.5,
we're going to add skip list support for
aggregations.
So, for analytics use cases,
this offers alternative for the
multi-range traversal,
that's much more efficient for
certain frequently used query types.
For example, if you have a query that
filters on one field and then
searches on another,
things like that,
we really see at least two times faster
and so on.
Another one is that the next release
will introduce APM,
a support application,
performance monitoring with
support for
a permit years,
for a metric data,
open telemetry,
as the instrumentation standard,
so APM support really cool
think coming next.
And last but not least,
it 3.5,
we're going to introduce
agent observability for
root cause analysis.
So, that's pretty cool,
not having your agents as
black boxes,
we'll be able to analyze the
performance,
and refine our prompts,
based on that.
So, it's going to be released
as experimentally,
we want folks to start getting their hands on
and give us feedback on the
stand.
It's all new to all of us,
of course.
So, to get a sense of this is
the right direction as the community.
And the old 3.5,
there's a lot more,
to talk about decoupling from
Lucene and opening up for
composable,
pluggable architecture to introduce
more engines,
and more format,
for example,
per k,
as a support,
and much more.
So, do stay tuned,
and where is that,
where it is,
of course,
the GitHub repo,
and GitHub org with
140 repos,
actually,
a very active Slack channel,
instance,
with thousands of participants,
with our forums,
and there's also the tags,
technical advisory groups.
I'm honored to be a lead for
the observability tag.
We have a monthly calls open for
everyone,
and Slack channel for that.
So, if you're into that,
do join.
We have us,
groups around the world,
around 19 countries,
I think, around the world.
And also,
the open source content events
around the world.
I just came back last month
from OpenSedgeConjapan.
That's the bonsai tree here.
And we're going to have
openSedgeCon Europe,
assuming most of you are
based in Europe,
in April,
in Prague.
So, you're all invited to join,
by the way.
It has a special track for a patchy
loose scene and search.
So, even if you're not
specifically with OpenSedge,
but the broader search
community,
we welcome your participation,
also CFP for the following
ones,
check it out.
And check out the QR code,
all the links are there
in one page,
including some other stuff,
like roadmap,
like the playgrounds
that you can play online,
and so on.
And of course,
Ash Wattenme,
are here,
happy to answer any questions
after the talk.
So, after the conference,
so we put the QR code,
so you can contact us
also online,
if we don't get to chat here.
So, feel free to contact
we'd love to hear your
question,
you feedback about this talk,
and anything in between.
I'm Dotaan,
this is Ash Wattenme.
Thank you very much for listening.
Thank you.
Do you have 30 seconds?