Hello, I'm going to be talking about how you can do use AI, I think in a very nice use case
to automate production fixes using open source tools and AI models, and so hopefully you'll
get an idea, I'm going to do the demos, so you can see how you can do it today with all open
source.
I am Carlos, I am a principal scientist at Adobe Experience Manager, this is a content management
system, and I've been contributing for a long time to open source, and people made me feel
very old, this weekend, there's a lot of young kids here, and also part of the Google
developer experts program, so before we start, who's here, here about Arco in general,
Arco rollouts, some people, okay, well I'm not going to ask about Kubernetes, and who has
who's doing canary deployment or some sort of, just a few people, and who believes AI is
going to make or lives easier, okay, who the lives AI is a fail, and this is going to die,
so, okay, I think you're going to be in trouble, but that's my opinion, but we'll see,
so progressive delivery, you have probably heard this term today or before, and this
is just a name to group that groups all this, deployment strategies that try to do progressive
changes to your production or whatever, so you don't break everything at once, so a new
versions get deployed, but they don't replace the system versions, it just goes progressively
changing your infrastructure, your application, something like that, and then the interesting
thing here is that you're evaluating this life production traffic, so you don't have, you
don't rely on unit test integration test, but you rely on actual production data to decide
whether your rollout is being successful or not. I'm all enough to remember when cross-strike
2,5 of the internet, I went up first, so it is all the affected people with Windows,
so we don't care, right? So, one of the interesting things, and I recommend everybody
to read the root cost analysis of all these big outages, like Amazon, Microsoft, whatever,
it's very interesting, I think it was here, yes, it says, now the new model, because they
they play something, it broke everybody at once, and the new model is we're going to do
rings, and every ring is going to be deployed, and then we're going to check if those
rings are going fine and so on, so basically they're going to switch to a canary deployment
of some configuration scripts that before they were doing just all as well, all at once.
The people are getting a lot of interest on this, because they realize that otherwise
they're breaking the internet. I'm going to skip this slide, actually, because you are
already know all this stuff, and I thought directly to our rollouts, our rollouts provides
these advanced deployment techniques, and it makes it really easy to use them on Kubernetes,
so if you can do blue ring, canary, canary analysis, experimentation, experimentation
you have in heravit is very cool that you can say, okay, I want to run this for two hours,
five hours, impar a little, and then I can see what's the, if this work doesn't work,
and then I can kill it afterwards, like things like, we use it for, or we're looking at
using it for Java braids, okay, we want to run two paths, or 10 paths in production,
and one polling experimentation, we're saying, okay, let's run this with Java 2125 and
see if it works, and we just need to run it for a few hours to make sure, and then look
at the logs and see if there was any issues there. The way our rollouts works, you have
the rollout controller, and you have the, some objects in Kubernetes, the way you get
traffic is through the ingress, as usual with Kubernetes and the service, and you're
going to have two replicatsets on Kubernetes, one is the stable, whatever you're running
today, one is the canary, is basically your new version of the image or configuration,
and then the analysis run object is an instantiation of the analysis template, the analysis
template is what says, what a good analysis is, what a good canary is. If the typical
case is go to permissions, get this metric, if 500 errors is over 5% or 1% or whatever,
it's a bad, it's a bad canary, so the analysis run checks that, and that's going to
is going to decide whether the canary keeps moving forward or the canary is rolled by.
I'm here in the analysis template, you can decide, I want to do it 1%, 5% and 20% whatever
sort of analysis you want to do, you define it on an analysis template, and you have a bunch
of plugins, metric plugins, the permissions, you can run just a Kubernetes job that will
decide whether your thing is, there canary is good or bad, and the one I added here is
one to do it with AI analysis with an LLM. So, we are now going, then the demo I'm going
to show you is, you have the source code, you deploy, you do the canary, you do the canary
analysis, now an LLM is going to decide whether this canary is good, and then it's going
to promote it as usual, or if it's bad, and then it's going to roll it back. So, our
integral talks to this plugin, and this plugin will do, and this is an agent using
a2A, that we'll look at your Kubernetes with whatever tools you want to create. I have
an example agent that does the list pods, get pods, get logs, get metrics, get events,
synchronaries, and so on, and that's get from into the LLM, and then LLM will tell you
exactly what the problem is, for us we. Now bear with me for a second, because what happens
after this, I don't want to have to go a look at what the problem is, what I'm doing
is my plugin will do, besides the rollback of the canary, it will create a GitHub issue, it
will assign an agent, an AIA code agent, that we will generate my pull request, and the
pull request for me, that I can only have to approve, and then I go back, or I can just say,
I mean, this will take a bit more work, but you can say, automatically immerse it, and
then you get the full loop, absolutely, ultimately, and this works, 500% of it. So, that's
what we're going to end doing. So, we have Argo here running, I put this on the top,
because probably otherwise on the back of your note, I'm going to say it, okay. So, I have
my green, my, my application is showing green, is responding with green color, right?
If I, I only have one pod here for the sake of the demo to not bother too much, but if I
say blue, if I do an upgrade to blue, I'm running a new canary with a new pod that is going
to be blue, and here we should be, uh, so many of we'll see some blue dots, which means my, the,
the traffic is going to my new version. So, and the canary is going to do the analysis, and
is going to decide whether this new blue implementation is going to be fine or not, and let's
see why it's not getting anything yet. So, the analysis is running here, so we should be getting
some blue, why it's not getting some blue, obviously, always the demos are not working.
That is blue, I might run them right now. Okay, something is wrong, because I'm not getting any blue.
Yeah. All right. Yeah. Let's see, nothing like the back in life.
The other pod is running and going to traffic.
Yeah. Let's go back to green.
I have a preview, also here, that is also not working. So, basically, the two services are
not ritual for some reason. Let's see, okay.
The canary is not working.
Why is this not? There is.
For some reason. So, now this table is the other way around, the table is the blue, the canary is the green,
and eventually it's all going to be green if nothing breaks again.
And we'll take a look at the analysis. Analysis, round, I'll make it a bit bigger.
Yeah, it's the number of requests you're going to be blue and green. So, now it's all green.
So, it's all green and this is a successful analysis. I can go and look at the previous one,
two minutes ago. Okay, can describe the object. And on the analysis, round, I have Gemini
to five flash analysis. It tells me, basically, my instructions are, tell me, look at this, tell
me if it's, if I should promote it or not, how confident you are, and it gives me some, I think.
So, I'm going to read it here. Confidence 95, promote it through remediation,
not remediation steps are required as the canary, the plymia perish healthy.
Not what goes for a failure, the canary, the plyming is operating as expected.
I have a multimodal analysis here, because I'm also running Gemma.
So, I'm running this against Gemini and Gemma. Gemma is running an IGPU inside this Google
Kubernetes cluster, and that's not working that well yet, but it's doing both at the same time.
So, Gemini is telling me, the stable parts show green responses, the canary pulse show blue responses,
as per the instructions that you can also in the analysis template, you can say,
give excellent instructions, and my instructions is ignore color changes, because sometimes
it's like, oh, it's red, red looks bad. I will roll white. So, as per the instructions,
color changes out to be, no, blah, blah, blah. So, everything is fine.
And I'm going to do now a bad one. We'll see, hopefully, how this works.
So, this new version is going to respond with random colors. Yes?
Can I ask the question, why is your system doesn't show the programs that you actually sold by yourself?
I didn't solve anything, it just started working here. It doesn't solve it, I'll tell you now.
Let me, so now I'm getting random colors, which is what I want it, and is going to do the analysis,
and the analysis is already failed, the grade it, and it's going to roll it back automatically, this is what
Argadas. Now, why the previous thing didn't work, the previous thing was an issue with the routing.
It was not an issue with a new part, not working or not working, so there was something in the routing
that for some reason failed, and after I deployed again, it started working, I don't know what to happen,
that it happened before, trust me. Other things happened before, but that one, no.
Only, yeah, the Argadas, the way Argadas works is you have all parts, you have new parts,
and you can look at anything you want there, but it's not going out to check if your routing
is working, it's not going out through the internet or anything like that. It's just all version,
new versions, and you compare, you cannot, it's not that you can check the new version, but you can
also compare both versions. One thing we are doing is I don't care, for instance, you could say
500 errors are bad, but if the old version also had 500 errors, what do you do? So what we then
doing is I don't care if I have 500 errors as long as they are not 5% more than the previous version.
You can do these things playing with both versions at the same time. So you saw this thing failed,
it was rolled back, the analysis failed. Yes. Now, I'm going here to my repo,
and one minute ago, my system created an issue, the analysis says that stable part is consistently
serving requests with green responses, healthy blah blah blah. The canary part is frequently
experiencing panic-servant amount of error. So it went autonomously, check the logs,
describe the parts, whatever, I don't care. So these panics are critical application errors
and not merely aesthetic differences. What do I do with this issue? This issue is already very good.
Like I can go here and tell somebody, hey, the canary broke, here is why it broke.
But the next step is I tell my AI plugin when it creates the issue, it labels it with
Jules. Jules is a Google bot for writing code, you can use Copilot, you can use
what is the other one the code. Basically, once you have an issue, it's very easy to
head one of these coding agents to just work on it. So Jules is on it, and Jules is doing
its magic here, and it's going whatever. It has the instructions, it gets the logs, it has the
logs, it's working somewhere, and it's figuring out what to do, and it is, I think it's already,
yeah, it's run the full test rates, test pass is long everything, right? And it should,
so this was two minutes ago, in two minutes I have a PR, and let's go to my pull requests.
Well, it's not there yet, but the previous one I can show you, it's going to show up in a bit,
and it's, so it created this commit removes the intentional panic, so it figured out
how to intentionally put it in, because like, yeah, you're not that dumb, just, you intentionally
put this thing in there, so, and it just removed the whole thing. Well, it went, I put it here,
if just right to access a great panic, because I'm accessing an index that doesn't exist,
anyway, like, this is all, it should remove everything, right? So, and it's very cool, like,
Jules is reporting for the duty, and you can talk to it, and then I also have Gemini Colorsist,
another one, to review the PR that Jules wrote. So, it's telling me, yes,
results are critical issue eliminating at deliberately introduce panic and clean up the, and you see,
I'm Bob's. So, now, if I go back here, is it not yet? Not yet, okay, so it's still working there somewhere.
All the steps completed, ready for submission. So, in a few minutes, I have a PR to fix my issue.
I can just merge it, and this whole process starts again. And, okay, let's see, this one,
is it here, there it is, open now, and again, this is, every time you get a different description,
or whatever, that's a mother, a removing the intentional panic in the sort of range,
blah, blah, blah, blah, blah. Let's see, what changes sometimes it makes different changes,
this time may the same changes before, and, and that's it, I don't have to do anything.
So, rolling, just to wrap it up, so rolling out changes to all the uses at once is risky,
as you've seen when things, but things happens, can I roll out on feature flags? I didn't even
went through this, but feature flags, can I roll out all these features, allow you to do this
in a safer way, and you can use AI to automate all these things. So, I, for one, welcome,
or new robot overlords, and you have all the source codes, our rollouts, the metric plug-in
that uses LLMs to do the analysis, and rollouts demo, that's this demo, and the coronavirus,
say it in, you can write your on the agent, but let me also show you, I was going to show you
when something good happens, I think it's, was it here? So, with Jim, let's see, I had a lot here,
it goes and tries to run a bunch of tools, so you can see how the LLMs things, it goes,
I execute into these pods, I execute into get logs, get events, the back pod, get logs again,
get events again, and so on. So, you see how it's trying to figure out, because basically,
I'm not, and you can write more complex prompts, I'm giving it more information. I didn't bother
too much, it just keeps going, okay, I'm going to check out the logs, check the events, check
these, check that, and then it's going to do it by itself, okay, so any, any questions, yes?
How do you think, is it ready for the production? No, I mean, you can, there's nothing preventing
you from running this, because the worst thing is going to happen, it's not going to roll out
something, but that's, that's something that is fairly new, I, I brought it in September,
so, because I think this is the, this is a good way to get it, because then you, you don't
need to take any risk, the worst thing that can happen is that either you promote something that
is bad, sure, that will be bad, or roll back something that is good, that's not a big problem, yeah?
Yes? Question, how you connect, how do you decide what matters to provide and what matters
to not provide to the existing, what I mean, for example, as you told before, you just read the logs
of the shirts and pots, but can you read the logs of other elements and see the possible
news? Okay, so, so the question is, how do I define what matters to provide and what not?
With the agentic part, I don't provide that, you can use ARGRA allows without AI, and you can
say, go to premises, get the matrices and what not, in your agent, in my example, agent, I don't
do, I only do list pots, get pots, get events and so on, but you could have a tool for your
LLM that says, get metrics, and but then you have to say, okay, get metrics, what it means,
it means going to this premises server and then you can get the metrics, but you would have
to define how the LLM can go on fetched metrics, then a LLM will then tell you, I want to call
this tool, get metrics, and that your agent is going to do and make those calls and give the
results to the LLM, so you could also have metrics embedded into the LLM reasoning.
You can connect as many of the sources, there's no limit other than how the reasoning
of the LLM, if you also give too much information to agents, at that point, you would go into
a multi-agent system where you would have multiple agents collaborating and talking to each other,
so you may be you have an agent to form metrics, you have an agent for Kubernetes, you have an
agent for something else, and they would talk to each other to figure out to get data from one another.
Okay, thank you.