So, Sergey, now we'll talk about Lama Dieter.
Yeah, hi, guys.
I don't know how many of you heard about Lama Dieter so far, but please do not miss like
there's Luming Dieter, that Mozilla is developing, so Lama Dieter, that's what we are
building.
So, we kind of started it a little bit early, but there is still a funny project with
this defined in names that are really closed.
Maybe we'll join one day sometime, but not at this moment.
So, I will bring a more details about that.
My name is Sergey.
I work for a company called CyberGizer.
We are building software and now doing a lot of funny stuff with AI in open source.
And I will bring a couple of tools that we recently developing together with the AI
Foundry.
So, if you haven't joined, check out the AFoundry.org.
We have a Discord, we have a lot of good reading stuff, so yeah, that's something that
you can find useful.
So, the first thing about the LLM as a judge concept, I don't know how many of you heard
about LLM as a judge.
Okay, some of you, let me give you a little introduction.
So, LLM as a judge is when you use a model to assert the data or your source code
or whatever.
In this case, the target could be another model.
So, you can use one model to assert prompts and quality of the another model.
For example, using OpenAI to assert local Lama's running with Lama CPP.
So, two building blocks that you might need for this one.
So, the first project called Neko API.
This is open source, API that is compatible with OpenAI.
But at the same time, it opens interface to run local models.
Do not change your production code.
So, if you have an openAI, for example, then you can easily switch your development
stuff using, let's say, Lama CPP.
And in this case, interface of the application is going to stay the same.
There is a few more use cases, but for building LLM as a judge, we're going to use this one.
And the second one, which is Lama Gator, that's the tool that you can use to store all your prompts.
And use kind of as a arena for your LLM.
So, what does this thing do?
So, in Lama Gator, you can add all the models that you have local and remote.
For example, for this demo, I have OpenAI model, which is GPD 3.5.
And using Neko API, I have a local model running inside of the Docker.
And that is small one, which is small.
So, I have a prompt that needs to calculate distance to the moon.
And I've created an assertion to use LLM as a judge.
So, what does it do?
It asks local, small LLM, what's the distance to the moon.
And instead me doing regular expressions or parsing the result, I say,
OK, now go and ask OpenAI to validate that.
So, in this case, I ask OpenAI return true.
If it's close, which is again, close, not exactly.
So, it might be a little bit different.
So, which is close to the right answer.
And I can give it like 10% of a kind of proximity.
So, let's see how it goes.
So, in this case, I have this prompt and I have that assertion.
And I can create a test run.
In this test run, I choose assertion, which is LLM as a judge.
And I choose the model that I would like to test out, which is a small,
using Neko API.
So, creating test run, giving it like, and here we go.
So, we have it passed and small give me exact number.
And as validated, I can run.
And that's it. Thank you very much.
Then I'm in Flastock.
Then I'm in Flastock. That's impossible.