So, last year, I spent some time interviewing a lot of developers who had just started building
with AI and specifically started playing around with using large language models.
And from those conversations, there was like a few different themes that came up.
And you're in a picture.
Ah, what are you?
It's around here that you can hear that you can hear that.
And yeah, there was kind of a few different themes that came up.
Like, one of them was some developers,
was, sometimes they'd find a really cool open source project on GitHub,
but then the first thing that would come up as soon as they tried to start using it,
was they'd have to enter that open AI API key.
That was really annoying, especially if they didn't want to spend money,
and they wanted to keep that data private.
Another thing that came up, the second one you see there,
and this one was probably the one that came up the most,
it's just this feeling and sense of overwhelmed,
I think Stephen touched upon it as well earlier in the space.
And just how many different tools, how many different frameworks,
how many different models you have to navigate when you're starting out.
And not only is the difficulty come from having to pick the right things,
but then even if you pick certain tools, certain frameworks, certain models,
there were a lot of interoperability issues.
And sometimes some of these projects are quite new,
and a new release comes out, and it breaks everything.
And that was causing a lot of headaches for developers just starting out.
And then the third one that came up a little bit as well,
is just like this feeling of pressure,
and this need to kind of be able to experiment and prototype quickly.
And in a lot of that comes down to this space,
being and moving very quickly.
So what a lot of developers kind of wanted was to feel like they could easily experiment,
and swap out different, swap out data, swap out models,
swap out tools, and that needs to be.
So with Blueprints, we wanted to start trying to address some of these pain points
that were coming up with coming up for developers,
and really trying to make sure that using open source tools and open source models
is a default for developers.
So what are Blueprints?
So Blueprints are essentially customizable workflows
that allow developers to prototype AI applications using open source models and tools.
When we're building Blueprints, there's kind of generally five principles
that we're trying to stick to.
The first one is that we only use open source models and tools,
obviously the code itself is open source.
Secondly, we're trying to make them really customizable,
so that you can adapt the Blueprint to your needs with your own data for your own use case.
Thirdly, we're trying to make them extensible.
So we see the Blueprint basically as the foundation code that you can build on top of,
and extend further, if your needs.
Fourthly, we kind of want to build up a collection of different Blueprints
that show you how to do things differently.
And as part of that, we want to have some standardization across different Blueprints.
So we're trying to build them in a consistent way,
and we think that'll make them easier to leverage for people,
especially people starting out in AI and starting to use algorithms.
And then finally, I think this is the most important one.
We want us to be community driven.
So we've started building our own Blueprints that we've put out there,
but really we want the community to also contribute and build some,
as well, so that we can build up this collection of Blueprints
to really show people how you can start with open source models and open source tools.
So, AQR code, maybe not as many as Stephen, coming up.
Oh, there is.
So to kind of like kickstart this project,
we've released something recently in a beta.
It's called the Blueprints Hub.
So please feel free to scan this QR code or take you to the Hub.
Basically, this is going to be our website for collecting all the different Blueprints
and kind of showcasing them in a user-friendly way.
So people can kind of get started using them and testing them nice and easily.
For now, this is a beta, so we've only got a couple so far.
But over the next month or two, we'll be adding a lot more,
and hopefully it's going to showcase some of the communities Blueprints as well.
So now I'm going to hand over to David,
he's going to talk us through on the first couple of Blueprints we made.
Hello.
So, the first one that we built is called Documentuposcast.
As the name states, it's combatsa.commentuposcast.
The idea is basically that pretty much everyone of you
have probably heard about Notebook LLM by Google,
and it has a postcard feature.
So the question was, can we actually replicate this with open source tools?
We did some research, find some implementations,
that didn't fulfill the details that we, the bullet points that we said before,
either they require an API call to some model or whatever.
So we give it a try.
I don't have a QR code.
I trust in your ability to type.
Yeah.
Sir.
Start Documentuposcast.
Try again.
I guess, image loading, not super optimized here.
So here is just a screenshot of the GitHub rhythm.
So if you go there, you are going to see the same.
It's basically to state that there are three options to run it,
depending on who you want to pay the hardware.
And you can choose to Google to pay the hardware,
and you go to a Google collaboratory and run the Blueprint there.
It's probably the easiest option, and the one where you can customize,
try to tweak the prompt, try a different number of speakers,
try a different model, wherever you want.
Then there's also high emphasis spaces, if you want high emphasis to pay,
and then there's GitHub, if you want Microsoft to pay.
If you want to run locally, there is a pipeline package that you can install,
and try directly if not you can clone the GitHub repository.
And the next one is a working progress in the school.
A structure QA, and this might not be as obvious title as the previous one.
The idea was basically that I was working on the podcast.
I liked board games.
It was Christmas.
I was playing with my family, and the literary told me,
why don't you do something useful?
That can answer questions about the board game rules,
because we're having a little bit of a discussion about the rule of a game.
So a structure QA, the idea is that many of the question answering approaches out there,
assume that the world is absolutely chaos.
No one has the document well structured.
Well, in my experience, for example, in board games, people actually think about
a structure in the document into section, putting some reasonable section names,
so you can navigate them.
And this is also true, I believe, for many technical documentation and other places.
So we wanted to give it a try, whether what of the existing approaches work the best.
Yeah.
So the first thing we did, we collected a small benchmark.
We run different solutions.
So the MENA is probably the largest, has the largest context window out there,
and is the most easy to use for free.
So as a benchmark of what you can achieve with open source, we run the benchmark on the MENA.
These are the results, and then in comparison, we run it against QA and 2.57 billion.
I actually was trying to run the benchmark against DeepCick just today,
and over the weekend, but it keeps freaking out about trying to answer the question thinking,
instead of just creating the document, like reasoning the, so it didn't work.
So yeah, this is a working progress.
We are trying to implement an alternative solutions to the long context,
that is not feasible locally and to the rough implementations.
Here I choose Raga Tool, because it's such a cool name that I have to use that.
So I guess I'll just end my thing.
These are kind of the first couple that we worked on.
We're going to be working on some more, but we'd love to hear suggestions from you on what we think,
what you think we should work on next.
If you've seen something where maybe there's a good close source solution,
or something that works well with a good close source model,
but if you want to see an open source alternative,
please feel free to submit an idea here,
or if you think you can contribute to this project and build something yourself,
feels of scan this QA code and submit something.
We'd love to work with you.
That's it.
Thank you.
Okay, we have questions.
And we don't call the we have questions on the chat.
No, I don't ask questions now.
This is the room first.
Should I put you?
I'll put you.
Quick question.
Do you are you expected in the person who submitted the question
to be a developer?
Oh, I got to talk a lot louder about me too.
So do you expect the person submitting the question to be a developer?
Or is this open to designers and open to anyone?
Yeah, literally anyone.
If you've got something you think would be cool to work on,
we'd love to hear about it.
And then we can chat about your idea,
and you don't have to develop it,
maybe we can find someone else too.
Team, do we have questions on the chat?
No, we're done.
Thank you very much.
Thank you.
Again, if you leave the room, do so with that door or that door?
The...