So, welcome, you can hear me, okay. So, welcome, I'm David Bernhardt, and the creator of CDVs
and member of the CDVs and groups, the topic of today of this station, at least. I'm open
source, contributors, since several years, maybe some of you already use one of my tools.
No, okay, no, not problem. So, the current story, starting 2023, at the FinTech
scale up, my team work on platforms. The purpose of the platform is to accelerate software
development for software delivery, but also a machine learning model delivery. So, at the end
of the years, at the end of the years, we ask our customers, what do you need for the next
years to prepare the roadmap? And as part of this roadmap, with one of the feature
is having more visibility on deployment, being able to see which version it's at which stage,
UIT stage, production, maybe development, and to have visibility on the history. This
request also comes from product managers, just to know when they create 3 ports, hey, what
is the current version of the product? So, for our my evaluation, I search for solution,
I found none. I was confirmed by some discussion on the CD foundation channel, but also
by a streamer, so we know when streamer says something, it's often true. And we saw the solution
is often build yourself on the CD foundation, it's what's what is mentioned. So, we
developed the cost, this is the value, two, it's was low priority, not a ship solution.
So, that in 20, 24, my department was acquired by another company before we start working
on this topic, it was not probably too bad still present. So, I decided to live in the
company and start working full-time on this open-source solution. But as I live the company,
I also need money, spoiler, money is not yet here, that maybe with your help and the help
of the first them, thank you and the devor, then you can upload them for the work, maybe
it could change. So, the problem is, often you start with an idea, a bag, it's there
up, you create a ticket, start cutting, testing, building, CI, it's fine, you publish to your
package manager, you deploy into the first one, you testing this one, if it's okay,
you promote to the other one, there's multiple way to promote, maybe you publish to another
artifact repository, Xterra Xterra, and this workflow exclude the case of library where
you stop and you're injecting to another project. But to achieve this goals, we use a
ton of tools. Each with its own dashboard, multiple dashboard, one dashboard for Jenkins,
maybe one dashboard for GitHub, one for August City, no unified view, no view of the flow
across all those tools. So, it's currently the many issue, what Hansen mentioned, how
can I correct version of the staff between each stage, what is the deployment frequency,
it could be extracted, that's what is the plumber, sorry, what is the deployment frequency,
at each deployment, it's start to become harder. So, what is the goal? The goal is to integrate
as to have an solution that's integrating to your existing workflow, not to force you to switch
to another tools in which you will have to design everything and it will create. It's
very important to observe and to be able to see what is my software that every life cycle,
in the past that also in the currently. And when I have this capability to observe, maybe
I could use this capability to react and to drive the deployment and not maybe to use a
big orchestrator, that could block external. So, as part of my evaluation, I also joined
the City Foundation and start to contribute to City events. Why City events? So, it's part
of the City Foundation, that it's part of the Linux Foundation project. Multi-proversion
has already been released, it still is the road at it. The last version was end of last year.
Don't you see it? There is a fixed that is coming, maybe this week or next week.
And I bet on this standardization, that in 2023, City events at some
limitation, minimal tool adoption. You understand? It's long. So, no production, deployment,
use it, I guess I know, but specification is not stable. And why for an existing tool,
switching to do this work in progress solution, where each one already has its own tool,
so it's not pragmatic. But we hope, and as part of it, we could make it a better future.
So, I decide to go it. So, what is City events? City events is mainly a set of gistonshimmer
to specify every part of a software-aligned cycle. It's composed of three blocks. The context
writes information about the event itself, an ID for the event, the timestamp when this event
arrives and the type. The type is important because the type defines the content of the subject,
the second part, the important part. The top level field and the other subject are always the same.
The ID and the content and there is an optional source, the type will not talk about it.
And the content of the content of the content is defined by the type, so it's why it's important.
As you can see, the type is namespace, so it means that we have diversity event for normalized events,
and it's followed by what we name the subject in this case, it's service, followed by the predicate,
in this case, deployed, and the version of the type. We also have the spec version used,
and as City event want to be extensible, we are allowed to have another optional part,
name custom data, in which you can put whatever you want. It is important because it's a place
where you can put information not available as part of the subject, at least today. And it's also
extensible because you can create your own type, just on Tuesday, there are 3D events and follow the same pattern.
In the context, there is also some optional fields, like links and charity to help correlate
events between each. So if we back to the previous slide, that this time we subject and predicate,
you can see that we have tickets, change, branch, incident, desk as run, test tree trends,
and for each we have some predicate, I can say action.
Another of them. So with this event, this definition of events,
I start to work on CDVs. So what you have in your company today? You have some tools,
using your software development at the bottom, and some of them already influence CDVs,
but maybe not the latest version of CDVs, maybe each tool can use different version of CDVs.
At the top, you have your enterprise dashboard team, a tool, maybe used by your platform engineer,
the DevOps, that maybe by CDVs, maybe by data analytics. So, and maybe you use several of them
and your system of failure. And as I mentioned, the first goal is to have visibility about
what is running. So we start from the dashboard path, oops, to fast. So to be able to show
data, we need to store data. So for the start, I go with
a pass best fuel, because it's flexible and I can test several approaches with it.
That time pretty happy. I enable timescale DB, because it's even times time period events,
so it's a lot of features and more. But the real core part is the collector.
The purpose of the collector will be to retrieve information from CDVs providers,
and non-CDA then providers. And as we have this collector available to push that
out with the database, we can also use it to send CD events to a third-party system.
Maybe a workflow tool, maybe a QC system and at the other side of the QC system, you have another
collector, you can create a set of topology, because you want to aggregate from different
collectors. It's the run, you want to integrate some tools.
So CDVs inside CDVs. CDVs is like a netale to extract from long load. That menu is extract,
transform load is part of the source. So it can take information from
values sources, HTTP webbook, that also HTTP server sent event,
file system or bucket, Kafka is already support, that most of the tools, you want to integrate,
use a webbook, use HTTP today, so it's this is part. And at the outside of the source,
we have CD event object pushing to the Q and outside of the Q, the rotastic to a bunch of
sink, sink could be log, could be file, could be the database, could be HTTP, could be Kafka,
and more. The collector is provided as a CLI, as a Docker image, as a Kubernetes,
but also as a GitHub action. Why? Because this CLI, you have some kind of
a feature include kill command to push the events, you can do the transformation and send
CDVs or Kafka client directly, and it's also at all YouTube to use it locally,
without have to deploy. Inside the source, as I mentioned, we have the extract part,
the set of transformers, a chain of transformers, and the loader, the purpose of the loader is
to validate that it's a correct CD event that can be pushed to the Q. Each transformer
can split this load, log, create new events, do a bunch of stuff. There are some
upcode transformers that may need, you write transformers using a language, a name, vector,
remap, language, language created by, and pushing by that a dog is used for your further
current as log system aggregator. It is really profound and my repository included events marked
to compare the value solution. And the extractor can be used from a producer that
pushed that a like waybook, but you can also grab information by putting that a example,
looking at the bucket and say, hey, there is a new report, there is a new artifacts around the
bucket, extra. So input can be file, file metadata, JSON, JSON, L, CVS, and next week, XML.
I will explain why XML after. And the database is pretty simple.
In the CD event language, this database is also known as evidence store. It is simple because
I want to allow enterprise to own the data and maybe to extend this schema as they want.
So to ingest that you call a store procedure. And what is done by this storage procedure is
up to you, the default one, the provided one, just store into a table that works like a data
edge. Where we have the rogism and some extracted metadata, the timestamp and the type,
the subject and the predicate. And if you want, you can extend it with new view,
materialized view, maybe compression, retention policy, etc, whatever your database and your
knowledge. What? And on the dashboard, we have some chemitrikes, extra extra.
My favorite and what I started, it is custom dashboards. It is used to track artifact, artifact,
version across the line. So each vertical line is one version of an artifact.
Its original line is a stage, published, maybe tested, deployed as a part of a service
into an environment, etc. And with this information on the table on the other side,
you have access to the frequency of the event, the adverages duration from the point of
promotion between each stage and the latest version at this stage.
So it's a, I'm pretty proud of it, but I'm determined to an interesting, I have to
create it because I didn't find one. Originally, I planned to make a demo that I bypassed the demo
and capture directly the material. So in prediction, maybe I have a single line for an artifact
that is, I don't manage, I just apply it on my first door. It's good. That's sometimes,
I have other bugs because transformer needs to configure for transformation. In this case,
it's a conflict between cube watch events because I watch directly Kubernetes to know what is
deployed and I go city. The naming convention is not well aligned, so the graphics.
And this one is a mix of all the information pretty unreliable. But you can create also dashboard
about your pipelines, test pipeline, CI pipeline, etc. So you have another view of the time,
the queue, the duration, the failure, and you create, you can create. If you want to get started,
I also provide a Docker compose where you can play with it, with this Docker compose, there is
already database, a collector, and a graph and integrative. And inside graph and others, there is a page,
where you have random data and few forms to serve service deploy service events and incident.
So it could be exactly like if you provide this application for your customer to say,
this version is deployed. I have this incident in prediction. And information is used to annotate
the graph below. So what works? I bet for us on that, a part of it, I expected to,
it will succeed at some point in time. I start simple, I guess I use some common tools,
and keeping the relevant, hello to extern, create your view, etc. So it follows,
and maybe later I will create, extract me, create, etc. That you are free to do it.
Don't use information. And because it's modular, user can keep what you want.
If you want, just to use the collector to collect data and storing accounts or another database,
it's fine. If you use all sorts of database, it's fine. In fact, if you change the database,
dashboard should have to be aligned. So we use a can customize. There is a bunch of stuff that
didn't work. The adoption, not very far. It's also the purpose of the collector to help
you can use it without adoption. That by using it without adoption, we lost part of the information.
And a big part is how to correlate information. Because contrary to this with it,
trusting where you have a trust ID and it's propagated inside the process, and you have the
trust ID outside, and then you can correlate. It is not possible. So you have to create some rules
to do it. So the challenge, energy and space, correlation, dashboard audience, maybe you have
devops, c-levels, developers that want it, target, etc. The documentation and the marketing
that I work on it. So we are building on an expected energy and space. So there are some
problems and some problems that we need your help. On this, on CD events, we need your feedback,
and CDVs also need your feedback, and if you're happy, you're money.
So what is coming for the roadmap for CDVs? So more dashboard is dedicated to test. It will
come in for Q1. It's also why XML will be used as input to be able to read directly,
like example, the unit 3 ports, and to integrate this information, more source, more that dashboard,
from your external and integration of all feedbacks. And if it's go well, maybe using also CDVs,
and especially the collector, to trigger your workflow, and to start to go to the event CICD.
So building on an emerging standard is risky that I expect to be rewarding,
and someone has to go first. CDVs and CDVs are you, the doesn't, the doesn't fix all the
problems that you can start to use them today, because CDVs also take care of,
maybe transforming all CDVs into new format of the CDVs with a transformer.
So we are looking for user contribution, star feedbacks, comments including it,
why you think it's useless for you, and what is missing, and what to improve. Thank you.
We have two minutes, maybe two quick questions or one more. Thanks for the presentation,
and happy to see that there's some little thoughts about CICD. I have just one question about
the standard information and the semantics of our motion. Do you have the opportunity to talk
to the office elements in your practice? Yes, so a recurrent questions. Do we work with open
telemetry, CICD groups? No. To be clear, often say we discuss, it's maybe we have just,
I don't take, I'm not part of it. There are some discussion at point. We look at what they
are working on, and we work on another approach. Our goal is to have some thing more
particles and using other side. But we use them as also as inspiration. It's attributes,
they define, etc. as part of what I start to using system data just to look at it. Thank you.