Hello everyone, thank you for attending this session.
This will be a bit interactive and very useful for anyone who is working on containers,
regardless of local or some other sources available.
So I will get to start it.
Very short interval about me, my name is Usman, I work at your final in the
airplane team.
I am actually working very closely with the open source community on their
final or other areas on GitHub or Slack and yeah, I do a lot of open source
stuff.
So you can find more about me also if you scan this QR code or so.
And yeah, I am actually a Linux user and while working with the communities for
your final end doctor, I learned a lot of things and I am just going to explain one topic
which I really like, not life.
So by we are today, so let's go back in the history.
So back in the days, we have like this simple architecture of application, like you
have one server and everything is there and people use to access through it.
We have devices like laptop, PC and then smartphone and for you or the customer or the
client, you name it like it works fine and they were happy and everyone was happy and
they are like, can go home and relax, right?
But then they were problems and they identified that they need a better solution because
putting everything on a single page is not good and high on resources as well like back
in the days we used virtual machine.
So the evolution of containerization started and Docker was the pioneer for this and they
solved this problem like if you have multiple services running, you can use like Docker or
Docker Compos or Docker run or Docker Compos to do this is kind of a stuff and they are all
isolated but you can still make your web application up and running without having too much
problems and then this micro services application ideology also follows in Kubernetes as
well because deep down like Kubernetes big thing but deep down there is the part of
containerization so you have to use like a container based image and the concepts also
flow there.
So then the question is like how do you observe your applications to there, right?
So you say like, hey I'm using Docker in my organization, me and my team we are happy,
we have four applications replicas they are up and running and then if something goes
bad or the service is degraded it's broken but it's active so it's degraded then what you do
or how you know if thing goes wrong you say okay application for is not working why not
spin up more applications right that will solve the problem yes it will but I used still
having the complete view maybe not right those application will also fall at some time
and you were like hmm what should we do? How do we find these problems because then the thing
get out of hand that your customers start screaming or your team isn't in the firing mode
and you're trying to solve the problem without knowing what the problems are you don't know
the root cause and yeah you start screaming in your building or office or from the customers
and then you finally like oh my god we need to do something right so what is the solution?
Let's talk about a little bit about the foundations of observability it's a fancy word but
it is very important to understand so I will explain this that it's basically a journey not a destination
because whether you create applications in any environment you make it on the production it will
break at some time because you will do a customization of code also so it will break there is this
big chance but you will fix it and you will try to solve or find out what the root cause is
doing the root cause analysis and after that it will happen again but it's like a never ending
cycle and you need understanding of observability and the three plus are like logs,
metrics and traces so this is very important to know at least on the ground level what you are
looking into it for your application just simply looking one or two of it may or may not solve all
the problems so you need better understanding like how they are working together and then the
question come is like finding a complete observability stacks so let's talk a little bit about
graphana so I would say graphana use case in the market in general is known as a visualization tool
it does not solve the problem it's a tool which gives you that visualization experience right
it is available on all platforms and you can connect the source of data to graphana and it will
give you that in a nice visual format and the question is like how you can make your own observability
stack right because okay you know graphana raise your hand if you have used graphana
very nice a lot of it so you probably know that this visualization layer this is the dashboard
which is connected to your data sources of any kind right and that is coming from your environment
but here at the bottom we have this composable open source stack and you can use any of your
we call it like LGTM looks good to me and you can read like it's we have for logs visualization
the main graphana tracing and metric is for either memory or promise yes but you can use like
open telemetry nowadays or you can use any tool of your GIS it will still work and with that you can
do more and more you can observe the area which you are looking for either is a performance testing
or infrastructure observability and so on then since we are talking about container so the next
tool is known as C-adviler how many of know about C-adviler okay a few nice very nice so basically
C-adviler known as container advisor basically it's main job is to give you that observability
or monitoring experience for your container because the common mistake which everyone thinks like
hey we can get the metrics via Docker engine but Docker engine give you the the information only
of the Docker itself not the container itself while the C-adviler gives you much more deep down
what's going on in that particular container how many process the number of memory they are
using and so on so this is a big difference and this difference will come handy and we will have
a live demo of it as well let's go in metrics or demo path so I think this is big enough for now
so one thing which we can also do is that Docker has a built in command called Docker steps
and okay yeah so it can give you also that live information of the container how they are working
internally and this is like a runtime but the problem is that this is only visible on your screen
but you cannot share it with others so there's this limitation within the core Docker path so
to overcome this limitation we can use C-adviler so C-adviler comes with this integration
integrated web UI so if we do very quickly so this is the C-adviler default web UI and as you can
see here it has a Docker container path and if I just say like I just want to even visualize
how the gyrphana container is doing so I can get all these live data here and view it and
understand how the performance is going on and the one limitation here is as well because since
it is limited so it does not store anything right so you need a solution and the best way here is
to use the prometheus so if you hook up the C-adviler data into prometheus and then visualize in
gyrphana then you can have the data which is stored in a particular place you can view the
historical data as well and the live data as well so in gyrphana this is how it will look like
so now you have a complete overview of all your containers so you can see here that they are
telling you all the information and this data is coming from C-adviler so you can see here for example
this graph tells you like how much currently I believe this is the CPU usage is it's done so currently
C-adviler is using us bit and as the other services which are listed here so this is the very
useful and a productive way to get more information about your containers because you will have
some application which may behave and you do not know what's going on so you can get this
visualization layer then the next question is about logs obviously logs is very important as
someone who is also been in the Linux world I love you viewing logs and they tell you the more
information like what happened something crash or out of memory and so on and yeah in the IP world
we say like customer may live but logs don't live so logs are very important and and for logs
I think that for me the default choice is to go for Loki it's again an open source but the cool thing
about Loki is that it can accept from everything so you can use Loki if you want to visualize
log for Linux for database for Kubernetes something on Windows which I'm not very fond of but yes there
it is so yeah and it has also an API so you can use Loki with a called functions with API and
you it in a JSON protocol format and in Loki you need an agent an agent can be of your choice
independent your own freedom so you can use like for example the promptail is or Defana
Alois from Defana but you can use fluent bed logist as and you have probably heard of like
ELK stacks or Splunk or Elastic and they also use Loki as a top layer for the visualization so
you can build your own choice of tool with Loki there is no problem and Loki what it does it does
the indexing of the timestamp not the data or the information in the log line so it also saved
a lot of this space which is very vital when it comes to logs on the cloud style they are big so
it has like one key advantage that is good to know and to view the demo so
here I have logs as well so now if I so I'm using Loki with promptail with Defana Alois and
I can view all the logs so you can see like here we have a very nice visualization layer and it's
already shows like something was bad so it is highlighted in red good our ingredients unknowns are
always in like gray so you can get all the information in a single place so there's also one part
which is tracing you can use tracing as well to get this since it is due to time limitation
I'm just sending this link here you can take a screenshot or at the end you will get all the
content I will share the code so but with this demo which is I link here what you can do basically
yeah so you can get the whole experience so not only you can just use for logs and matrix you can
use tracing and profiling as well so once you have an application you can get all of this data in a
single place and you can again choose the tool of your choice you can use promethias or
your ELK stack whatever and just use your fun as a visualization layer it works fine so this is
what the whole picture looks like and the final take of this for me will be like the OSS part
because I love working on OSS and since there are many tools available on the open source platforms
so they're try to use it community is a very backbone because most of these tools are
come from the community side and also the new features which they're where we get up also
and yeah and the last but most important is like you want a layer a complete stack of everything
you don't want like too so many application which are scattered around and you have to connect one
place to another so you try to use a tool which can give you all thing in a single place so that is
very important and will be saving you a lot of time this is the QR code you can scan it because
I link all the documentation links and also that MLTP repo which you can use at home it's basically
a fun way to test out how doctor works so yeah and yeah that's pretty much it I have
us also just want to say that we have a griffana booth here as well in the cable so ask questions
get some swag or just chat or what you are working on griffana we will love to know your feedback
so yeah any questions
okay then thank you thank you