So now, Martin, it was supposed to be with Paul, but Paul couldn't make it, so Martin
just present the presentation about the Kubernetes AI building themselves.
Thank you very much.
Hello, everybody.
Remember now, I'm like a lecturer up here, if I see you walk and don't, I got a name and
shame you.
No, it's only a joke.
All right.
So we, AI folks, businesses are out there and are trying to leverage the advantage
with AI.
I wanted to key aspects that is being able to build better AI applications so that they
can get the return on investment in AI.
Because AI costs money to run your models, to leverage them, to train, and
make it, et cetera, and for that to be used, they need to see the return on investment.
So my name is Martin Hickey.
I am an open source developer and I work over at IBM, and I'm going to be walking
through, because we've a five minute talk, I'm going to concentrate on two small tenants
for the moment, and you can take it from there and check them out yourself afterwards.
So the two tenants I'm going to look at is, for so on I'm going to look at taking a model
off-the-shelf open model and how do you tune it for your own domain data.
So I'm not looking at your general, general model, and you can ask a different questions
that's been trained on the internet and stuff.
But what about if you need to build an application for your customer or for yourself, and
you need the domain specific on it?
The second one day is, how do we take models and actually use them for to execute complex
workflows?
So start and off with the tune in aspect.
So when we take a model, whatever model that's out there, the open models, it's going
to be trained from everything that's on the internet, and also any synthetic data after
that, because believe it or not, we've used up all the data on the internet at this
stage, okay?
But the important part to this is that if we're going to use it for customers or for
ourselves, we're going to need to put domain data into that, and that's called tuning.
Now you're going to turn around to me and say, rag, what about rag?
Yes.
They're not mutually exclusive.
You can tune your model with domain data, and you can also use rag as well.
And generally, you'll probably use rag for data that changes quite a lot.
You'll put in your domain specific knowledge, and data that's updating quite regularly,
then you're going to use rag, because there's going to be an overhead car in the cross
into a vector database and pull them back to data.
Now the problem is, I suppose, two kind of complications with tuning models are open
models is.
Number one, trying to contribute into a model today is difficult with no model.
So what happens is, instead of building a new version of model, we build a new variant.
Hands up here who will see in the many different variants of lama out there.
Yeah.
So there's a lama for this, a lama for that, and for the different domains.
And that's no one's fault.
It's just the way it is.
The second one is around when you try to tune the model, you need to have this high barrier
of entry, which is having AI knowledge or DPI knowledge.
Wouldn't it be nice to just tune the model with the data you have and not have to worry
about all the complications that go with it.
And that's where in stroke lab comes in.
So what in stroke lab does for you, it's a workflow which allows you to build, to put your data on top of the base model and tune that to create a new version in the model.
So your base model doesn't change.
The data you're putting in or the knowledge you're adding to it is, as I say, data or knowledge.
It's like PDFs, it's marked on, et cetera.
So you don't need a deep knowledge of our understanding of AI for that.
Also, when the new version of the base model comes out, you can train it again because your data is all stored in the taxonomy, which is essentially a binary tree of data.
And every time you go to bring out the new version, you just tune the model again using that.
The other aspect of the data is, you can contribute your knowledge out into the community because it's an open fiber and community that's out there.
Okay?
Now, you may say, okay, we've got private information, we're not putting that out there.
But that's okay too, because you can take this workflow, bring it inside, have your own taxonomy that you build up, and then build versions of your model as you go along.
Our tune your versions of your model.
The second tenant that I mentioned is around using your model, be a large language model, a small language model, whatever.
For something more than user prompts are for just, you know, chat or whatever else.
Look, it's great to ask it to do a poem or do something else or ask it a few questions, but let's make it a bit more powerful, okay?
The way to help you this, it is the agentic frameworks.
What the agentic framework is essentially, it's making your large language model or whatever type of version of model you have to be the brain of the particular application.
So the way I kind of look at it with an agent is, it's like a wrapper around which your model in there.
And then you've a series of flaws or actions you can put in there.
Five minutes left and a five minutes talk, this is good one, oh sorry, there's one minute.
Okay, I'm not allowed to do those jokes, okay?
Sorry, you have to make it, you have to lighten up a lightening talk.
I'm sorry about that, it's very bad joke.
Anyway, so what you want to do then is you're going to put in a series of tools or functions around that, to give them the large language model, more power or more driving it.
So the way I look at it is, is imagine if you have, you want to use this model to be, to provide you with the best route to some location, be it home or somewhere you have to go.
So instead of just saying, right, give me the best route, wouldn't it be great if the model could force the ball, ask Google, Google maps to say, what's traffic like an location at the moment?
But in addition, maybe you could go out and ask the weather, what kind of weather is there at the moment?
But while you're doing that, couldn't it always ask if you have any emergency service APIs in your area or in your country?
So imagine be able to take all that information together to make a decision to give you back.
And that's the key with the gente that I look at and put in your agents and use your model as a brain for it.
So why, to be, B-A-I-R-I-M-B framework?
First of all, it's open source.
But out of the box, it's production ready.
So what do I mean by that?
So what I mean by that is, it's ready for the different errors and real world situations.
Because no longer know where we just ask in the model to do something.
We're asking the model to call external systems to get information back.
And as we all know, if you've ever written an external system, there's always an error.
Something goes down.
And being able to handle that and being able to troubleshoot it and look at the monitoring and the telemetry, that is key.
The second of all is the tools that are on the, what kind of tools are there?
And in this framework, it's got a lot of tools.
For example, if you want to do searches, if you want to do query to SQL, if you want to look at a weather or traffic, it's in there.
And it's building as it goes along.
And there's also some agents out at the box that you can use for text summarization or different things.
The monitoring as well is, you know, if you want to build your agents go and code it away from it.
But if you're not into coding and you want to know code, you can also have that true true to you.
So to just finish up, I put the two links up here because basically we haven't talked about that much.
But check it out, see what you think and give it a go.
Thank you very much indeed.
Thank you.