Thank you very much, Wilson.
You can hear me at the back of K.
This kind of volume, I'm glad to...
I'm just saying we should start every talk like that.
Just launching sweets at people.
That's awesome.
I take that away.
That's my takeaway from Boston.
But yeah, documenting you event architectures with Open API and 18KPI.
That's what we're going to be speaking about.
Before we get started, it's just myself.
My name is Dave Boone.
I'm a core maintainer of Open Source Project called Event Cartelog, OK?
So I left AWS this last year now.
It's just focused on Open Source Free Software full-time.
Now, of course, it doesn't pay the bills.
So I did some kind of consultancy stuff on the side.
Fun fact, I was at NDC a couple of days ago, and the speakers did it there during the talk.
And the someone says to me, wow, is this a picture of you?
And I looked and I said, go, this is what full-time open source does to you.
I'm two kids.
So yeah, previously, it was eight to two years.
I was a developer, I became from an architectures.
It's an architecture type that I love.
I actually worked at 18 KPI.
I were Lucas and fan over there for just under a year, doing some open source stuff, which is cool.
And I realized that's almost almost coming up to 20 years,
doing startups and enterprises, and oh my god, I can't believe that.
But yeah, I left open source.
I'm in the right company and the right thing.
The right conference is actually my first time at Pholstum,
so I come here yesterday, absolutely overwhelmed,
just like turning up and being like, oh my god.
There's absolutely awesome to be here talking to you all.
As these three things we're going to cover today, 25 minutes,
that's going to be quite cool run through.
The device between open API and 18 KPI and whites there,
can we use both of them together, better together?
Can we also, like, how does event catalog fit into this?
Why is it a thing and maybe something you can help with?
So, and then at the end, I've got loads of resources and slides you can dive into.
So, a few people don't know actually, I actually spent eight years,
eight to nine years actually working with mere cats, but even though not.
So is anyone work with mere cats before?
Yeah, no one, okay.
I felt like this is kind of how I feel about it, obviously.
But unfortunately, it's not these type of mere cats.
I actually used to work for a company in the UK, but this kind of mere cat,
which is like an electronic weird mere cat.
So as a company in the UK, I could compare the market.
We sold a lot of insurance as to people.
Quite a comparison website, right?
I was there for eight years, I joined at the time.
Thought works were consulting for two years,
and I kind of felt I hit the jackpot there because they taught me a shit load
about software engineering, which is really, really cool.
At the time we were doing a lot of this legacy system,
and a big moral of application.
This was in 2013 and whatever it was.
It was just the way we built applications, right?
And it thought works, Martin found everyone coming bored,
and they said, yes, we got to do guess what, micro services.
Okay, so we're like, yes, you know,
it's like CD driven development at this point in time.
So we'll do some micro services,
split this core into certain things and scale it.
I just actually took a five-year project.
I was there for the most of that.
And there's an interesting project.
And what it tends to look like is something like this.
These split things up, that kind of makes sense.
And we were doing that.
I was mainly focused here on the right, which is the UI kind of elements.
React was a thing, Angular 2 was a thing, back then.
So I was mainly kind of focused over here,
but working very much in hand-in-hand partnerships with people doing APIs,
right, people building micro services.
Now, the architecture type that tends to be the trend there,
but still kind of is, I guess, in some ways,
is this kind of synchronous communication.
Okay, so we did a lot of synchronous communication,
synchronous chaining in the architecture.
So you'd have, like, the client sends a request to the server,
the server goes to the next service and the other service.
And, yeah, you get a response back.
So please get me something here with a response back.
And this is kind of what that kind of looks like, okay?
And now you scale this over five years.
Now there are questions that start to emerge from
people in the organization, right?
And these questions tend to be like these, right?
So like, what APIs do we have in this organization?
And if you guys are free for whatever,
it's just building APIs because it's so easy to do that.
How can I use them or version them?
Was that versioning strategy looked like in our organization?
And then, finally, who's even consuming these APIs?
What about telemetry and all this kind of stuff, okay?
Now, luckily for us, there is something that can help.
Okay, this is back in 2011.
Swagger come out, okay?
Swagger open API specification.
So hands up if you're using open API.
Yeah, most people, okay?
Open API is now generally a great standard for building APIs.
Most people use open APIs to do this.
And it works very well for us.
So Lucas is going to kill me because now I'm showing Yama on a screen.
Times up, okay?
Yeah.
But yeah, we have machine readable code.
Yama all that we can look at represents something.
When we have standards in these, this kind of sway,
we can also build tools and visual representations of things.
Everyone loves it, fantastic.
This is great.
So I'm going to extend where we kind of API first.
People will come back into that in two seconds.
What tends to happen is this.
Yeah, so every organization, every team here is like, yes,
we'll have an open API specification.
Thumbs up, great.
If you want to learn about our API,
I will point you to the API specification.
And everyone's like, yes, this is awesome.
This is working wonders.
To the point, right, that if you wanted to introduce an API,
you cannot do it without a specification in an organization.
And this is where we are today in most enterprises that you'll see out there.
Okay, most enterprises will turn around.
If you go to any large scale, say, no, you cannot introduce an API.
Where is the business case? Where is the specification?
So open API is very well established over the last 15 odd years.
Now fast forward to where we are today.
And yeah, API is so interesting.
Okay, so this is Stripe Documentation Stripe.
Had like, maybe some awesome docs.
So that last 5 or 6 years, they tend to be the lead in this docs space.
These docs tend to bind to some kind of API specification.
We can actually use these docs and check out what's going on
and make pull requests, get requests, post requests, and everything else like that.
So there's maturity happening in this space.
But there was a problem in another space, right, history,
repeating itself in a venture of in architectures, right.
That everyone I speak to when I was at AWS large enterprise large customers
is that governance with event architectures is always an after for.
Okay, and it's an interesting thing when you start to understand this
and dive deeper into this topic, is that the adoption, why is that?
It's because the adoption of an architecture is growing.
Now, this is an interesting thing because the venture of an architectures isn't anything new.
You can go back 25 odd years as an enterprise integration pattern book.
Okay, if you go pass back before then, white papers about using messages
in Linux and low-level kernels and all this stuff, messages aren't new.
So why is it the rage and the in-thing or whatever that might look like?
And realistically, just because a venture of an architectures is just more accessible to everyone.
Okay, so like as sure, we have Google Cloud doing the Pub sub AWS,
where I was at we had like SNS, SQS, Event Bridge,
Confluent, massive people in the Kafka space.
We've got solid and gravity doing API management.
There's a lot of VC money and money being pumped into this.
We've got Cloud Flair, by the way, just coming up from the ranks at the moment,
absolutely in everyone's top food in this space, which is very interesting.
Cloud Flair doing things like durable functions with ingest and restate in temporal
which are startups in this space.
Durable functions give us a nice patterns we can use with events.
Actually, saying, here's my code, and I'm now going to do orchestration in the code.
So when an event comes in, this bit is triggered a year later,
this bit triggers is in the same durable execution in the same server,
as function as an example, which was impossible before.
So there are things happening there, there are things happening in the space.
And the problem is we get like this as easy as much easier to publish a message
onto a broker onto a channel onto a queue.
Okay, you can get SDK installed today within two minutes into a cloud
and you're publishing messages.
This is great, this is awesome.
So when we start building our architectures, this is kind of what it looks like.
We have producers and we have messages and we have channels,
this is queues and what kind of stuff and then we have consumers
putting this information off.
Now when you dive deeper into a message type, you know,
there are things like commands and queries and events.
So a command will be a representative intent.
A command is, please do something, no, like,
I cannot do that here as a response.
Okay, so commands, you might have seen this in kind of rest APIs
and things like that.
Query is the same kind of thing, please get me something,
here it is, an event is a multiple fact,
something in your system has happened.
So an example here is my dog.
You can't really see it here, but my dog's tongue is kind of hanging out
as my dog is weird, but my dog was saying,
like, I want a toy.
Like, I can take that command and reject it and say,
no, you cannot have a toy or yes,
you can eat another toy that I have to purchase.
So what toys are available,
so I start to look in these kind of queries, okay?
So in your architecture, you have these types of messages.
Now what tends to happen is developers and architect
as we build a venture of architectures,
as we go to whiteboards and we draw boxes
because we're loosely coupled, right?
So we start doing this fantastic.
And as someone pears into the room and says,
I like this, this is really cool.
I'm going to add some cues and other channels.
Now we're kind of reliable,
and this thing's looking fantastic.
And then the other thing comes along,
says, we want a piece of this.
We want to continue this.
And we're all like, yes,
we've got loosely coupled distributed system here.
And we look at it and we go build it,
and it's great.
And what happens is, with event architectures as this,
as you get that value over time,
okay?
We have this perception of value that we have this bump in value.
I've just produced my first message.
I've got my first consumer.
Awesome.
Look how cool this is.
Let's keep going.
And that value kind of goes up.
You know, we've got producers and consumers
were adding things and more producers and consumers.
Now at this point,
something interesting happens in this journey that you go on,
is that the value tends to plateau somewhere in the middle.
Now why is that?
Because there's the complexity on to it.
And what you'll see in most people,
most of the times is this.
Is that complexity at some point,
at some scale,
just rapidly goes up with a distributed kind of big ball of mud.
Why is that?
Because assumptions are made as like a standards,
like a governance, lack of documentation.
Okay?
I want you to talk.
I've got this at the end.
But I want you to talk in January,
which is fantastic talk about software complexity,
by Peter Van Hardenberg.
And he says like complexity is not linked to size,
but when we start to interact with things.
And this is the essence of complexity when it comes to event architectures.
It's because we design something like this
and whiteboards in our teams.
That is loosely coupled.
But when we implement it,
it is a completely different story.
And there are things that happen.
There are big balls of mud that you will come across.
It's not if it's when.
I cannot state the amount of people and conversations I have
in the open source community that experience the same problems.
These problems, some of these problems,
lack of standards, assumptions,
ownership, documentation, governance.
So guess what?
There are questions that start to emerge
when you start building event architectures.
These questions are,
well, what events do we have?
Well, okay, so what's the answer to that today?
Here's a GitHub repo.
Go look.
Okay.
What are the payloads of the events?
I told you, here's a GitHub repo.
Go look, right?
So you say, okay, I'm going to make a change in a poor place.
I'm going to add something or change something in a schema.
By the way, who am I about to break?
Well, we don't know.
The reason we don't know is because
we are told as we build a venture of
an architectures that producers do not know about consumers.
This is a whole point where building distributed systems
and we're building as wonderful things.
That producers don't know about consumers,
which is an interesting thing.
And this causes chaos in my opinion.
So on the right, on the left,
we have open API, one to one.
It's quite simple things to think about
what kind of response world.
On the other hand, we have one to end.
Think of a spiderweb now.
Producers are coming and they're going,
how can we manage all of this?
Now, luckily for us,
there are specifications and those are
standards up that that can help.
And this is ACENCAPI.
And so for those who are here earlier,
Luke has done a talk on ACENCAPI.
But for those who do not know what it is
and just recently joined us again,
some YAML,
because I don't show YAML.
That's really bad actually,
it's black.
Yeah, you can't read that.
But anyway,
ACENCAPI,
I just took this from the web, you know.
It's a specification allows you
to document channels messages,
different bindings,
protocol agnostic.
You can actually have an ACENCAPI specification
in your project from now
from now so you can actually go into a service
and see what messages are sent,
what messages are received,
and how that all works.
And so no longer just going through the code
and working all this out.
Why I recommend ACENCAPI personally,
because as standardization,
standardizations allow us to have
machine readable things,
machine readable things that allow us to build
developer tools and ecosystem around this.
As I mentioned,
there is protocol agnostic.
So you can use anything you want,
SNS, SQS, Kafka,
RubberMQ, MQTT,
it doesn't matter.
So it goes with what you're working.
There's tooling, it brings discoverability,
and documentation to schemas.
So again, like OpenAPI,
we can do stuff and render it into
some nice views and visualizations and representations,
and most people like to read something like this.
Some like to read like that,
but we can get both of the same.
So the question is,
when you're building event architectures,
which one do you use?
Do you use OpenAPI or ACENCAPI?
And it's not really a question of,
which one should I use?
The question is, well,
what is your architecture under the hood
and what kind of messaging
palins are you using?
So I said earlier,
most of the palins that I was previously doing
years ago,
was more synchronous kind of requests,
whereas we have this synchronous chain going
on and on and coming back.
And something like you might see some commands
and queries over here.
But in other cases,
you'll see ACENCAPI,
like,
if I order a pizza,
I'm not going to wait around
and wait until I get a response,
things happen,
and when I order things off,
Amazon,
things are happening.
So what tends to happen nowadays,
again,
with companies I'm working with,
in the open source,
things,
and speaking to a lot of people,
is people have a mixture of both.
Okay?
So here's an open API,
specification,
we have a rest API
to do whatever else.
Here's a message based,
kind of application,
and some people have both within their application.
But I think there is,
like,
it's taken another step further.
I think there are problems with
specifications,
okay?
So on the left,
we have standards,
tooling,
schemas,
implementation details,
which is fantastic.
But on the left,
I think we're still missing a whole lot of pieces,
okay?
And I do a lot of,
like, a work with a demand of design,
people are there in,
dive into demand of design.
But I think there is,
a lot missing,
things like semantic meaning,
there's a schema of an order that has been placed.
What does this mean in our organization?
Why have the fields changed?
What about versioning,
and lack of ownership?
Who owns these things?
Okay?
And lots of value on different,
like kind of personas,
understanding these schemas.
It's completely different from,
like a business analyst,
or a product owner looking at events,
versus something else.
So what can we do?
And this is kind of where I focus a lot of my time,
and energy at the moment.
There's a free,
an open source tool called Event Catalog.
Okay?
So is anyone heard of this?
Just curious?
So a few people,
which is really cool.
So this is,
not funded,
it's MIT open source,
this is what I do for the laws
and for life at the moment.
But how it works,
right?
So we create an Event Catalog.
I'm a super geek when it comes to documentation.
I just love it.
I love docs as code,
and all this kind of stuff.
And the value it can add.
We can add Mark down.
We can add somatic meanings,
NDX components.
We can create beautiful visual representations
of information in your architecture.
And then we can use Git as a source of truth
and create pull requests,
whatever else to change this and deploy it.
So what does this look like?
What do you get out of the box is something like this?
So now teams can actually start to understand
what services we have,
what events do we have,
what are the schemas,
what are the brokers,
what are the versioning.
Actually apply some of these to the domains.
But you know,
we actually start to use some of the components
and DX components and start to build
a representation of our architecture
that people all can hopefully understand.
So facing KPI and Open API
can actually use these,
to actually feel event catalog.
And actually feel documentation
and reasoning for your organizations.
Download schemas,
view things,
actually when you're versioning your schemas,
we can actually do something interesting here.
Versioning is an interesting thing
because why has this name been added to the schema
to the payload,
to the event, to the API?
Where has it come from?
Why is it like that?
The people that make these changes
are probably long gone.
Okay?
New people are coming on board.
Why does this thing look like that?
All this history is lost.
So what happens if we can just add
some kind of change log to this stuff?
Or to make change logs in this area?
And this is again,
what event catalog can do?
By the very visual person,
I like visualizing things,
but why can't we just have visual representations
of schemas,
of data, of queries, and commands,
and events, and channels?
And we can, you know,
here is a service in the middle
and on the left and right,
we have queries and events
that are being pushed into this service.
So this is what this service accepts.
On the right, we use in Kafka
to represent something else.
We can use standards
and specifications
to build these visualizations in these tools.
So you can take your 18KPI document
and actually visualize
some of this help to help other visual learners
die deeper.
And again, as I said,
many teams and many people
are like mixing matching things.
So why can't your teams just have their own
event catalog, as an example,
or their own spectrum,
just feel the documentation?
And I think there's huge value in documentation.
I created this project three years ago
when I was working at 18KPI
as a side project, okay?
And then I joined AWS.
I was there for a couple of years.
And this project is just slowly getting traction
because people are out there
in this like, a vendor and shitshow.
And it's like, how can we get value
out of what we already have?
People are leaning into these open source projects
to help them.
So with that, you know,
I'm just one dude with an idea.
And as I said, it's not funding.
I'll just do this because I enjoy it.
If you're interested in any of that,
you know, I recommend come check us out.
We've got like almost 1,000 people on Discord now.
20,000 people create catalogs in that 60 contributors.
If you're interested in helping out in that space
or you're a geek like me in documentation
and things like that,
I recommend just come check out the project.
Because, you know, taking a step back,
the only reason I can do this
and invest a lot of time in this
because what is the alternative?
The alternative literally is this.
I haven't seen anyone else kind of avoid
this complexity that comes with event architectures.
The only thing we can really do is manage it
and push it down.
And we can do that with open API
and A-Sync API.
And I believe we can't have standards in tooling.
And this is where I'm at with event catalog
and hopefully event catalog might be able to help you
in some of that.
So to summarize everything I spoke about here,
I said at the start, you know,
the open API specification is a standard now.
You'll go out there in many enterprises
and organizations and many people are using this
to build stuff.
Events of an architectures is completely different.
We draw systems like this.
We decoupled real resilient or whatever else.
There will be a point in time.
That complexity will come in
and you've ended up in some big fall of mud.
All right?
Because it's not really a one-to-one mapping
between rest APIs.
It's like this spider web that we're trying to control.
So how can we control it?
I highly recommend if you haven't already,
check out A-Sync API.
It's a fantastic specification,
fantastic community behind it.
So I recommend just diving deeper.
If you want to go further and look at other specifications
out there, there are things happening.
Cloud events, specifications is a fantastic one
that mainly driven by folks at Microsoft.
Cloud events is like here is an envelope
and here is a message inside the envelope.
Now the envelope is standard for everyone.
Okay, why create envelopes all the time.
So I would recommend looking back
if you're using event architectures.
There's a lot of those specifications,
which is fantastic all about workflows and things
like this.
And then we've got X registry down there,
which again is more around
discoverability of your event architectures.
Again driven by Microsoft folks.
So there are things happening,
and it's really cool space to be in.
Personally, I feel like we could do more
than just specifications for our organizations.
I think we can actually have business value there.
There's a lot of like stuff we can take
from domain different design and actually combine the two.
And that's it.
That's what I do.
I'm trying to do with event cataloging
if you're interested, come hang out.
Last year I created a book.
This is free.
I've been saws thing.
I've done loads of whatever.
There's 120 pages on everything.
Event driven architectures.
You can just go and help yourself.
And then here at the end,
everything I spoke about.
This is resources from another talk
but everything is pretty much in there.
So if you want to dive deeper in your own time
or free, you can hang out with me on X.
I'm not sure who uses X anymore.
So I'm an OG.
OG handle here on blue sky.
I haven't changed that yet.
And then LinkedIn or whatever.
And I'm here for the rest of the days
if you're seeing me walking around
that awesome.
And I hope there's some value in that for you today.
So thanks for spending your time with me.