Okay. So, Ricardo, we'll explain how to use of intermediary in the situation of a package
of a tool in Python, of course. I think that will be a really interesting. Please.
Thank you. So, hi, everyone. I'm Ricardo Mallocetti. I work in absolutely the elastic
where I maintain open-ended stuff. And also, I'm a maintainer for the upstream open-ended
Python, Python 3B projects. I'm this talk. I want to show you, like, I'll open-ended
Python implementation works. And so, we see, I'll be able to automatically recognize
an install instrumentation libraries with open-delamity bootstrap. Then we'll discover
our open-ended instrument, is able to leverage the Python site module to enable
automating instrumentation of your applications. We'll then see, like, our end-to-point
used to discover components in different packages. And later, we'll see, like, our
instrumentation libraries are able to patch for particle. And you're already at, like, a
spoilers on talks ago. But I'll start, like, giving people new to open-delamity and
the visibility, like, a bit more context on this thing. So, how many of you already
knows, like, open-delametry, at least the word, okay, okay, many hands. And how many of you
are already using open-delametry? Okay, still a bit cool. Okay, so, observability is the ability
to understand the internal state of a system by examining its outputs. And this means, like,
being able to understand what a software system is doing, by looking at a telemetry data,
it exports. And this telemetry data, usually, is like, it's about the problem programs as a
cushion, like, in terms of states, and also about the communication between components.
And the open-delametry is an open source and vendor-neutral observability framework, providing
both a specification and a lot of implementation different languages. I noted to create a
manage-based telemetry data. And this data could be, like, traces for tracing as a cushion
of the software parts. So, calls into and from your Python application, also think, like,
database calls, HTTP calls, RPC calls, and stuff like that, but also, like, internal function calls.
And then we have metrics to capture some measurement at runtime. For example, like, in memory usage,
or, like, the number of requests and the time spent saving them and stuff like that.
And then we have lots, but can be, like, some structured or unstructured text with a timestamp.
And in open telemetry, this free concept traces metrics and logs are called signals.
There is another signal in development for profiling data. And in practice, with observability,
you can do something, but look like this. You can see, like, distributed transaction between
different services. In the picture, we have free services. The one I read is the main one.
And we can select calls. And also, like, it's communication with the one in blue,
the middle of the screen. And with the one you read in the, at the end of the screen.
So, the idea, like, to see, like, the whole life cycle of requests and insider application.
OK, this was enough for introduction. Now, let's look at some open-demory Python specific stuff.
So, the open-demory Python project consists of two different repositories. One is named open-demory Python.
And that one is called the open-demory Python country. But open-demory Python contains, like, a few different
package. The main one are the open-demory API. But defines the interface,
exposes to users. And this is not tied for specific implementation. And then we have the SDK,
by the same implementation on the API we've seen before. And then we have another important project.
Package is the semantic convention package, by contents, contains all the article definition
that have been populated by these semantic libraries. This semantic convention defined
practice what data you get in traces logs and metrics. And the content of this package is generated
from a specific cache. And then we have the other repo, that is open-demory Python trip.
By contents, all the instrumentation libraries, when it contains also the open-demory instrumentation
package and the open-demory to destroy package. Each instrumentation library,
specific to a Python package, is a different package itself. And that sits inside the instrumentation
libraries. We also have an instrumentation generic directory for a generic specific instrumentation.
And then we have the open-demory instrumentation package, but the implements what we call
like auto instrumentation or zero-cond instrumentation. But it's the ability to instrument application
without touching the application code. Finally, we have the open-demory to destroy package,
like contains like basic default configuration, like deciding to configure like where to
support the data. And also the support in protocols. And please note that the project
likes a lot on environment variables for configuration. And so the destroy uses environment
levels a lot of suits. Enough for the instrument introduction. Let's look a bit of the
thing's work. So for a demo, we use like a very basic flask application. This is just an
endpoint mounted on the root part. But it is returning as thing, all the worst thing. So I guess
you are familiar with something like that. So first we're going to stop like a virtual environment
activated and install flask. And we're just installing an open telemetry destroy because it
depends on the API SDK and instrumentation packages. So we get everything we need for the just
for demo. So the first tool we'll take a look at is the open-demory bootstrap. And open-demory
bootstrap is a CLI tool from the open-demory instrumentation package, but is able to list
or install an instrumentation library available for your project dependencies. The first thing
but here requires is an updated list of some special libraries and the library instrument.
And this is generated at each full request. So every time an instrumentation library is added,
the open telemetry Python-con3 repository is getting updated. A bunch of instrumentation
libraries depend only on the standard library and so they're always list as available to install.
And in practice, this is how the file containing the demo thing between the Python packages
on one side and the instrumentation libraries on the other side looks like. So for each instrumentation
library, that has some dependencies. Open telemetry bootstrap searches the models you have installed
in your virtual environment. For a distribution of the dependencies dependency, that has a
much in version. If it finds a library, but can be instrumented, it lists the instrumentation
library as available to install. In the case, you're asking the tool to install this library.
It also calls a course pip check to check for any version conflict. And in specific case,
for the flash library, we see that we're looking for a flashed version of a bigger equal than
one.to. And if it is found, it will like list or install the open telemetry instrumentation
flask version at a package at version 0.50 m0. And there, you have the default instrumentation,
but there's the instrumentation, but does not have any dependencies. And so this one are stored
only by default. Now, let's try this tool against our virtual environment. Mine, this is the output
of the list common, even if calling the install action, because the output of the action.
A lot bigger and doesn't fit in the slide. So, like we can see, we have the common packages,
but that does not have any requirements, like I said in Cairo and the WISGi,
open LDS and the special packages. But we also have the open telemetry instrumentation flask,
because in the sounds like the go, we installed flask, but is already at one. And later,
installed in our virtual environment. Okay, now let's talk at the next tool,
but is open telemetry instrument, but is the one that remains of the instrumentation.
So, our instrumentation is the ability to set up a telemetry like tracing matrix
log is metering without touching the code. So, the tool is like a wrapper, but wraps your application.
So, question, resilience. Oh, many of you already know the set customized module.
Okay, one pair, two persons. Cool. So, you'll learn something new, hopefully.
Okay, by default, if you don't know it, like a set module of your Python installation,
tries to load from your Python part, a model named set customize, and open them to Python,
auto instrumentation uses like leverage system in order to initialize itself.
If you want to learn more, there's the documentation, the standard library documentation.
And a quick example to demonstrate how the set customized work. First, we'll tell the Python
temperature to print the word string, and it works. And then, we're doing the same,
but adding the current directory as the first element of the Python part variable.
Yeah, unfortunately, the library changes, and we see another string printed.
And this work, because in the current part, there is a five module named the set customize,
a five name set customized.pi, but contains the print. And so, yeah, yeah, this is like magic.
So, I open the letter instrument, words by like wrapping application.
We've the open the letter instrument command. And this command, first thing is puts
the part of the open parameter Python set customize module as a first entry Python part.
And also, that's like a bunch of things under root. First thing is it loads the
alters, but loads the distra, the open parameter is the package, by default. And also,
the alters, the loads the configuration, but are usually provided by the distra. And we'll see
our later. And the alters, the loads, the instrumentations. And I'll explicitly disable the
random variable again. All distraimmentation founds are enabled and loaded by the fold.
And all these alters are using like Python at importance in order to work. And we'll see later
what we are. So, if we run our Flask application wrapped by open parameter instrument
and do a request, we'll see like atrace like this, because the installed Flask instrumentation
library has been loaded and it's working. Please note that here I'm configuring an
open that instrument to the drop matrix analogs and print the traces to the console. And so,
we'll see something like that. This is like useful only for debugging or developing.
And if we look a bit into the output, we'll see like the attributes key. And this key is
in values like the depends on what the semantic convention packages and specification define.
And like, not important to look at specific keys, but this, like, follow more or less the
HTTP semantic convention. Now, let's take a look at 10 different points. Do you already know
anyone already knows what are the entry points in Python? Okay, a bunch of hands. Cool.
Okay, so I enter points are mechanism where packages use like to provide the discoverability
over some very cold. And this is the way they're open-termitri Python project implement plugins.
And this project, and open the open-termitri visthro package. And also in other packages,
we configure entry points in the pyproject.com all metadata file. This is an example of
the configurator entry points. Like, we have a group. In this case, open-termitri configuration
greater. And every group has a like a bunch of couples of name and controversies. In this case,
we have a configurator name with the value of the open-termitri configurator class inside the
open-termitri visthro package. And in practice, like, other code, but we need to pop with the open-termitri
configurator group. Just called the entry points function. We're importing it from the open-termitri
utility, imported method data model. But the re-entry mutation is from, importantly,
under-score method data models. So it's a different package, not the one in the standard library,
because it changed a lot between Python version. I mean, it's a part like white range of Python
versions. So we're just looping over the all the entry points it's finding. And we're calling
this is like a bit hard to read, but like we are loading the entry point. And then we are creating
an instance of the open-termitri configurator class. We are calling the configurator method of the
instance. And, you know, just for debugging, we are planning like, but we're loaded with code.
So, let's begin. We are seeing today, that is like monkey patching or patching for particle.
You know, how many of you were already there and seen the pair's term? Okay, so a lot.
So, I can say before you, that if you use any tracing or instrumentation libraries or something
or anything like that, you're already monkey patching your code in production. So sorry.
So, monkey patching is a technique used to dynamically change the behavior of some code at
runtime. And it used by most of the instrumentation libraries in the open-termitri Python project
to wrap other classics, methods or function. Like for example, imagine we are still
renting like a function. For each code, we want to know of course the function parameters,
and also what it returns. And the instrumenter code, my ear loops, for example,
like so libraries like request, I guess. We can use, we can use, but most of the time,
like libraries does not provide and you can, and you can do. And, thanks to Python,
many being a dynamic language, we can monkey patch the functions and methods again.
Instead of calling the original function, we can have like our wrapper, it called before,
and after the original code. And yeah, this wrapper function is usually like the one
I will create traces, matrix, and maybe sometimes ever, sometimes. So, like we're using like
most of the time, the RAPT library by Graham Dubb, Dumpeton. And, you know, my
person suggesting is whatever you need to monkey patch, they create something just as
RAPT because it's also provided of tools. Yeah, great. And we see like a couple of examples,
on how to use like a couple of different way to instrument things. For example, is using like
toy instrumentation library, what it created for another talk, and this is
something like a couple of methods from the Azgirf library. Azgirf is an implementation of the Azgir
protocol that is used by Django, at least also I think also all the Azgir servers.
So, this specific instrumentation, the Azgirf instrumentals, implement like a class, but is
in editing from the basis to mentor class, but is provided by open standard Python. And
it implements like at least a couple of methods. First one is an underscore instrument,
but is the method that is like applying the actual monkey patching. And it's using like function
from RAPT called RAP function RAPA. RAPA, where yeah, first parameter is the function we want to
the model we want to look into. Second parameter is the actual in this case I think is a function
we want to RAP. And the third parameter is our function we provide as a RAPA, and we see it on
the next slide. And then we have also like an underscore instrument,
method, but of course I'm RAPA, but it is an IPR also from open standard Python, but like
removal RAPA and the rest day to the original code. And yeah, so our RAPA code,
you know the important bits here, it's the first parameter, well after self, because it's like
developed function, so the original function, and so we can call it and get it's output.
And this example we are doing something I think a bit silly, but is like printing, like sending
an event, adding an event to a trace of the function, adding its stack trace, and this is
like not like the usual way you do for tracing, because it's maybe like a lot of data if you
call the function a lot, but it was like down for again research purposes, because like I was
divided by the bagging, how many times a single application was going from a single to single
and the other way around. So again, like last line is returning what the original function returns.
Then like another example of how we can use RAPA to RAPA things, we are using the object proxy,
but it is a class, but you can use to wrap classes instead of functional methods, and the usage
is back like you replace the original value with instance of this class passing the original class
you want to wrap as a first parameter, and this is from Maria called this is from Maria in
its documentation, for the botto call library, but it's like what botto free library used to connect
to AWS stuff uses under the root, and the idea here is that like we are like hooking into the
the under I termited, because this class is just like an iterator where you get like events
from an API, and so we are able to like consume them with iterator, but also
process things, and so we are able to call our own code, and the color doesn't see the difference,
because we are yielding the original value. Okay, so we used was the last topic
conclusions, to conclude like we are seeing like a bunch of tricks, they are open to having the
pattern project users to work under the root, and maybe like for the user you may be like
now it's more clear what you are doing under the root, and we are seeing like how we are able to
automatically recognize and install instrumentation libraries, we will open the telemetry boost up
command, how the site and site customize modules can be used to run our code before the application
one, and our interpoints are used to discover components in different packages, and how you can
use RAPT to patch for a party code. If you are interested in the open telemetry Python project,
we are always looking for help, I live some links, like the open telemetry IO, which is the open telemetry
website, the two repos, so the telemetry Python and open telemetry Python on Twitter, and then we have
a select channel on the CNCF as lucky instance, and we also do like a weekly call to discuss topics,
yeah, last one is my master dynamic, if you want to get in touch, if you have time, I'm glad
you take any questions, thanks. Thank you very much. I heard a microphone do you have a question?
No, yes.
Thank you. Thank you very much for this talk. I'm sorry to continue, so guys,
guys leaving, please love it more silent, please. Thank you.
Thank you very much for your talk.
I have been using production your tool to instrument. I'm sorry, I can't hear it.
That would be really a little bit louder. Does it work now?
Yeah, thanks. Thank you very much for your talk. So I've been using the bootstrap and
holding instrumentation in production for a while, and I've noticed that there's a little bit
of results in terms of which dependencies it pulls in. One example can be, I'm not using
AWS as a cloud, but some other dependencies are pulling in S3 of libraries because of dependency
leakage. Is there any way for me to run the command in a way that I would be able to understand
which dependencies are pulled why, like the interconnection between which open telemetry
dependencies are pulled and why are they pulled? Okay, so you're asking, if you're able
to discover which instrumentation library gets installed, depending on your packages you have already
installed, your dependencies? Which package is pulling the dependency? Yeah, like,
ever seen on this slide, you can grow like, oh, you mean dependencies of this
instrumentation? Yeah, exactly. So I have a dependency. The bootstrap is pulling in lambda hours.
Okay, so like, if you need to see like the dependencies of each instrumentation, you can look at the
by project.com all five. If you want to look like which packages this instrumentation
wraps, you can look inside the Python 3B repo, the mod is called boostrap and the score
gen and where is the map I've shown before, where you can see like the libraries and the instrumentation.
And we'll later guess. Thank you, may you can talk later.