So, have everyone, today I'm going to talk about enhancing delivery using a community
of gateway API and SEO, excuse me, this is fine now, so yeah, communities gateway API and
SEO and how we use these tools for internal deployments and digital ocean, a good
sure fans like how many of us use communities at work, gateway API, SEO or their sort
of stuff, some folks know this stuff already, bit about me, such in I work as software engineer
distribution, million platform engineering team and that's our student mascot, so I mean,
we have couple of semis or here is someone once, plus choice.
So, it was a recurrent, recurrent stream throughout the entirety of the, that continuous
deployment and continuous integration is a really hard problem and the reason it is hard
is because there are a lot of tools that work here, like lots of software components that work
in tandem with you each other and the growing up of any of the component might lead to
a degradation of the entirety of the deployment pipeline and eventually might lead to an incident
or degradation in Massey and SLOs, so it's a tough problem, there are a lot of solution
but some work, some kind of work and in cases of normal loadout, they don't provide a lot
of productions against some failures, especially in cases of false positive in which like
failure of a container or a readiness check, don't indicate a failure of the entirety of pipeline
and so you don't have a lot of production in those cases because it doesn't provide you
a safe framework to roll back to a bit interlude of how we do deployments at this solution,
we use software called DOCC, it's not an acronym, I think it was an acronym at some point
but we now agree it's not an acronym at this point which, and it's just an abstraction
on top of communities with all the same defaults battery included, defaults provided and
it does not include all those features for communities but it provides like 99% of the features
that make sense for our internal users, so it's like a size, it's just capital notice,
this is my system, I mean I've for declarative deployments that we do additional solution,
a bit architecture of DOCC, simple client server architecture, we have a DOCC, we have a DOCC
client that talks to a server, the server talks to a QBPI server, we have a bunch of controllers
that interact with the QBPI server, we have a we use QKDL but it's mostly for the platform team
to have administrative tasks because users don't usually care about the low-level details,
but we do provide some red access, if someone wants, and one fact DOCC server is deployed
on top of DOCC, so it's kind of like a chicken acting, very simple DOCC manifest, it's
declarative, like I said, we use JSON instead of ML, we got a batch for ML at certain
point, fairly intuitive, we have name names, this is scale, container to define the container
that will be deployed, maintainer for which team, so that team can be rolled back to in case
of a failure, all the jobs really intuitive, really simple to use, getting back to normal
workouts, we have, we'll take a quick look of our normal workouts work and what's the problem
with those approaches in the end, we'll use a simple engine x deployment to track this, so we
have a we used to have a 123 image, we converted it to 124, applied that manifest, the update
starts rolling out, so initially we just have all the traffic pointy get 123, but as time goes on
as soon as we apply the manifest, the newer versions will start coming up, so you see here's the
124 application that came up and the traffic will be reverted to that one and the older version
will be destroyed, and the same thing goes for all of these, it looks like they are in place upgrade,
but the cycle is always the newer version gets created, the traffic is get redirected and the
older one is finished off, and going on we can see some of this problem with this, first thing
first is like there is no way to control the percentage or the time for the traffic to be rolled
over, so in case of large deployments it can, it can take and in case of small deployments is
like couple of seconds, but essentially no one has control over it, it depends on a lot of
factor none of us is in our control, and second most important, if you see the last image like in
case of bugs, there are no previously deployments left back to roll back to, so if there is a fault
in the newer rollouts you are kind of out of luck because you cannot roll back to anything,
so the solution is progressive rollouts, progressive rollouts is like simple
minimal approach of deploying traffic in a periodic way to the newer versions, so that in enhanced
reliability and gives you a bit more confidence on your application deployment, and the user
experiences much more better due to this because user are much more confident on what they are rolling
out, and you get you can test out on a sample of subset of the newer rollouts before
committing to the entire rollout. Among a plethora of rollout methods that are present like
blue green A-B testing, can rollouts really deployed for most of our use cases, canary and A-B testing,
it because it may be more sense, but there is also a scope for improvement as in when we see
there is an actual demand for it, so we will start by looking at how we use canary deployments
and how we created canary deployments using all these two that I mentioned before, so if you
remember out the manifest in it will also section to define the deployments strategy that will
be will employ, so for in this case it is a canary deployment with the configuration, so
5% of the traffic will be rolled out every one or two seconds or two minutes, pretty simple
let's see the reaction, so as soon as we do that they will proxy that comes in between,
and that's a load balancing proxy that talks to the DOC supply and to rather than
the service is directly talking to the client and then the service is doing something changes on
based on those clients, so it's the task of the load balancing proxy to divide the traffic
into as per the configurations into the rollouts, so we can see the of this benefit of this approach,
we have a lot of control over the traffic, the time that we need to
when we need to load to the newer issues and then in the case of any failure, we obviously have
some traffic still flowing onto the older deployments so that it can be limited back in case
of failure, so this obviously gives a lot of confidence moving forward, to achieve this we use
these open source tools, SEO on my proxy get way PI and let's zoom in a bit and see how this
work in synergy and how we use them to apply this, so first thing first, SEO is a service mesh,
service meshes are so failures that essentially separate out the networking part out of your
applications and have a separate layer for ingress or egress traffic or in between,
given in between your services and it provides excuse me, it provides encryption of the
traffic splitting and all that all that cool stuff and it's just essentially used as a
control plane to manage all your proxies, so we use SEO to manage our load balancing
proxy, so SEO provides all the configurations how in which architecture of proxies will be
deployed, what will be the configuration of the proxies, the certificate management, security,
every kind of stuff is provided by SEO to those load balancing proxies and this proxy as we will
see are just on-white proxies underneath the garb shocking, so what is on-white proxies, it's
open source high performance, Agents service proxies, a proxy design to failure to
communication between services, so essentially intercepts and manage all the traffic that comes out
that comes from ingress traffic, maybe in between the services or egress traffic, so it's all
controlled by NY proxy and on-white is controlled by SEO, so on-white acts as a control plane and
sorry, SEO acts as a control plane and on-white acts as a data plane for our architecture,
and yeah, here we have it, so underneath the garb, it was on-white proxy all along managed by
SEO, and to glue this all together, we have gate-papies which are open source Kubernetes native
libraries that provide the consistent framework and interfaces to manage all these
networking-related properties and objects in any community native software application,
it was the predecessor of ingress API, I think it was the try to name it ingress 2.0, but that
did not encapsulate the entirety of what the CPS stands for, there is a lot more that gate will
be a provider that ingress does not, and if any one of us has use ingresses, they know they provide
a lot of pain just because different providers have very different objects, very different way
of doing things, so for example, if you are using NJX, this will that will have a very different
objects or a very different way of managing the traffic than SEO or linkity or any of those
service measures, so gate-vapies try to ease all that pain and provide the consistent framework
to manage the entirety, so you can quickly switch off between different providers without
wearing too much that functionality will break, and another benefit is because its
Kubernetes native, so you can create it as a native community's object and not have to
import or get a third-party software for doing that, so it's all very much in the community
is equal system, yeah it provides a couple of objects, but the two I highlighted are kind of
important to us, gate-vaped object, that encompasses the service mesh in NJX SEO, and how
that service mesh will be, what will the behavior of that service mesh, so it will be
different for SEO, it will be different for linkity, but essentially everyone, every one of these
service mesh will implement a gate-vaped object, so there is you don't have to worry about a
feature-breaking in case you sit between different service meshes, and it also very it also
spits out the roles, it has a role-based architecture, so you get ways something that
cluster admins are very about and HTTP out here which is also a gateway API component,
something that provides traffic management on application level details, so it's not the job,
obviously these two have two collaborate together, but it's not the onus on application
manages to maintain SEO cluster, manage all the configurations, all that stuff, so the kind
role kind of separates out and it's very much less friction in doing work, so gate-vaped, so gate-vaped
provides us a way for managing the traffic from English traffic, and then that English traffic
for a per application basis, it's spitted out into different services depending on what
on the kind of application that we wanted to redirect it to, and so yeah using all these tools
we were able to get a very stable progressive role out, but moving on we also have a bit
testing, sometimes you need to validate your stuff before we actually release it kind of like a
smoke test to validate all those results and smoke a bit testing helps in those cases, so and
very similar to how we use can read requirements, whereas you just point out to a deployment strategy
that we are using a bit testing, provided a bit of configurations, so enable headers will
enable header based routing, as we'll see in the next slide, and pause before roll out is because
these two a bit testing and can we can work in tandem with each other, and so you can define
can be along with a bit testing, and just say that you should pause before roll out and not start
transferring traffic to the newer version, because we need to test before we do any sort of traffic
roll out, and for this two, HTTP out provides a very easy film for using that, so we can just use
synthetic traffic via headers, so the HTTP out reads out the HTTP headers and just
rolls out the synthetic traffic to whatever the application the traffic needs to go on, so
in here we have version 128, that will be rolled out to only 123 instances and 124 instances will
be rolled out to the 124 instances, without any traffic being actually rolled out to any of the
new instances, and also let's talk about automatic rollbacks, because this is also a really
powerful paradigm and kinds of tides not to everything we discussed here, I love this web tool from
XKCD if you like XKCD like me, but it kind of scoresome like automation might be a bit
padded, but in the end if use properly, it really saves us some sleep and not a lot of worry
about, something can go wrong and we have no control over it, and this is what automatic rollbacks
just for us, so this is what it looks like in the OCC, we have failed conditions in which we
define PromQL query, so here we have just defined that the amount of 500 error code that I get
out of that particular application should not exceed 5% of traffic, and the OCC scrapes
Victorian metrics or permit is like anything you can use every fixed amount of time,
in the new rollouts it tries to see if this is true, if this is true and if we do not get a
time series based on this, we have found a failure, something has gone wrong, and it will automatically
roll back to the previous table version and stop sending traffic to the Norwegian or together,
so this is really powerful paradigm because we do not need to buy too much about how things
will go on, and something more to fail than we can easily, without any manual intervention
rollout to the previous instance, and kind of something just all up, we have seen of this benefit
of all of these, every time we see instance rolled back automatically, that is the potential
incident that we saved from ever happening, and this is all very powerful paradigm, like we can use it,
it does not need to be very specific to just deployments, it can be very useful in the CICD pipelines
and all that stuff, so really we are obviously seeing a lot of benefit surface, and we also have
some improvements that we think we can implement ourselves as time goes on, but that was mostly
what I have to talk about, some references which are basically the project pages, you can check
them out, I use X-keleter to use all those diagram, this is open source, really good tool,
and yeah, I have time for some questions, if you have some questions, questions.
So, you can sort of automate it a lot, it's like you're using a bit of system behind it,
it's obviously in your class, you most of the time wouldn't, for a case in the default
computation, so maybe there's a big question, you won't all the right things, this is the
deployment instead of creating you and when rolling out, right? So, we would not really have both
set this one to interesting, as an area example, and set this one to four, how do you work
around that, do you have a different type of system that manages that for? So, we don't use
like Argo all this stuff, so essentially we scrape a Victorian matrix or promethias every end
number, like whatever the configuration is beyond that, so every one minute at least we scrape
that matrix, we verify a very date that we get a time series made out out of that matrix,
right, and if that works, then this condition is essentially false, right, we do not have an
area, as soon as there is an issue in that upper direct Victorian matrix instance, we know that
we have had some sort of error because they essentially the data is not generating there, so we do not
have it, okay, it seems my question was a little bit many other different places, okay, for example,
what I meant was the moment of the roll out before the route even comes into play, okay,
when you start the realizing that you actually have two seconds in a way for the gateway, the
unit route 5% to 1, 95% to the other one, right, basically my question is how do you know
that you have some services instead of operating with one cell phone?
Oh, so you really use tagging, so this app, like we use tag, we tag every of the services that
is using, can we deployments and we keep track of those applications for specific purposes,
so app demo is essentially what we use, yeah, yeah, those kind of need to have these
labels for them to deploy, any more questions?
Yeah, it is an internal tool but it's not like very much different from communities,
it's just like we have provided certain sample, like certain sanity tools over it, so rather than
exposing our users to what is a CRD, we just say like this is a service you can just use that,
so you don't have to know about the internal stuff, it's not doing anything really fancy
about from just exposing a couple of APIs to users.
It's not, at certain point we thought we should open source it, we still are kind of working on it,
but there is not like really commitment on top of it, but yeah.
I mean it's very much related to our internals, what makes sense for us at our company, it might not
make sense for us at their own, but it's very much opinionated for us, for our use cases.
I'd like to mention that HTTP is a generated by the load balancing, also I can talk to you
about a container, the app itself, where does it come from?
So this expression we provide like the HTTP request total, oh, so I mean, the load balancing itself,
that's an iterative order, yeah, I mean, you can play with the group ideas, so, yeah,
yeah, I mean, these are standard like you can have promising metrics,
sort of most of these like this one is, I think, one provided by promising, but you can also have custom
metrics, so the script is for somewhere, where does it script from? Is it the application itself?
I mean, yeah, the proxy, the proxy has all the information from the application, so it gets the
all these traffic information from that application, and so we're to the website, from it is
out of 20 metrics, that's the violation of this expression, a lot of certain kind of indication
of, or does it like it to let you know that, yes, we do this automatically, so it is basically
for automatic rule back, so as soon as we hit this condition for a certain amount of time,
this is an indication that we are essentially not getting the data we want, and
our indication that we should roll back this. So let's, like, let's continue now with the
hey, we're all this back, yeah, I mean, yeah, there is something that we develop, which was developed
after I created this, so it will essentially do a slack notification to you as soon as it goes back
for service target, yeah, no, so it's, it's an internal tool, you can use it, I mean, it's a tool,
you can use, what's the, what I'm forgetting, but the IDP is kind of an IDP, like you take it,
you take the UCC for, we have the UCC service deployed, you can use the client to talk to those
servers, and those will deploy, you can also use it in the pipeline as well, if you have a CICD pipeline,
as I get a publication.
Yeah, I mean, in case of a failure, we just roll out the data, so we have to, yeah, yeah, yeah,
so like 5% that percent, if you take it, I'm looking for some person.
Yeah, I mean, in case of a failure, we just roll out the entire traffic to the,
yeah, so if there is no failure, then this is, like, can really, can be based deployments, so
it is all this kind of user alongside can deployments, so 5% 10%, you see some sort of
error at 15, going to 15%, you know, this is time for a roll back to 100% to the stable version.
So, then I'll come up to you, so yeah, there is a stable version, which is the older version,
and the latest version is where we're trying to go, in case of a failure, we go to the stable version.
So, so now, I mean, you have to, I guess, have to update the days to use our routes,
and don't say that this is managed by that dog server, you mentioned in the beginning,
stable application during the updates to these NFS, or is the automatic time,
like, changing of the percentage over time, something that is already provided by the gateway,
yeah, that's what I mean. So, it's the traffic splitting, this kind of thing is,
any specific to HTTP route, it's provided by gateway.
Actually, because you don't know what time, you have to keep updating that, so my question is just,
like, this old time of days, this is managed by that server, you mentioned, like, the dog server?
Yeah, yeah, I mean, yeah, so it has a server that keeps updating the Kubernetes.
The server talks to the gateway API, so that provides all the information to that API,
that now is the state of, now it's the state of this deployment is this, whatever that state is,
and you need to update. So, the community is like, you have a state that declarative,
you need to have a state, and you need to achieve it.
Yeah, but who of this desired state is what I'm saying, like, you have something updating to the desired state,
because if you're going green, it's much percent of the time. So, whether it is, like,
these over time updates, these are, this is managed by your thing, or is this something
built into the, into the, I don't know, whatever API, gateway API, you know, is console where you're using,
I don't know, is this over time thing of part of the API?
It's part of gateway API. So, gateway API allows, so, and my proxy works,
to set the traffic from that, from the server, right? And it's in front of the, all the applications,
and that manages the splitting of traffic, right? So, essentially, it's on my proxy
doing the splitting of the traffic, and the gateway API managing those steps as time goes on.
Did I answer question?
So, the big thing, I don't know, can you have days to, the percentage, so, and it has been,
it has been external system, gateway API, doesn't, there's a, yeah, the server, so it's a thing,
let's go to what else. What tells the gateway API, okay? I mean, it's five minutes,
I've found out failures, and more, the next percentage, the big question is there.
Okay. I'm kind of not sure on that, like,
the big question is about what portions scheduling that update is that, is that, is that
the, is that the OCC, is that the OCC doing all the stuff, like, it's continuously telling,
that it's time to do a five percent increase. Now, the manifest needs to do five
percent increase every time, so it's basically the OCC server.
Okay. All right. Okay.
Okay. The stack of public open thing, and what is, in turn, all the things, okay?
Okay. I think I, I think I, I think I, I think I, I think I, I think I, I think I, sorry,
sorry for looking for you anymore.
All right. I think we can, yeah, then. Thank you. Thank you all for coming.