All right, next time we have Nadia talking about highly available at blocking private
DNS setup in Kubernetes.
But I've been told you do not need to know Kubernetes to have fun in this talk.
Exactly.
All right.
Good luck.
Okay.
Hello, everybody.
Welcome to this talk.
It's this on.
It's this okay.
Okay.
Better now.
Good.
Still okay now.
I don't want to look down.
Good.
So welcome.
Welcome again to this talk.
If I have said, I say, Captain made Peter said that this is like a very very long title.
Uh, explicitly engineered to catch your attention, like if I were to talk about this with friends,
I would rather call this like a cool over engineered DNS setup.
So I think that's that's like a more faithful representation of reality.
Cool.
So as again, my name is Nadia.
You have my contact team for in my website in the in the footer in there.
So let's get into the into the agenda.
So we have a lot of things to talk today.
I have not rehearsed this.
I have no idea if I will have time for demos.
I will do my best.
But in an nutshell, first of all,
you may have noticed that there is some sort of elephant in this room after you have seen the title.
So we will address that.
Then I will present you my wish list for the DNS setup.
And we will talk about that.
Then I will briefly comment a talk about Kubernetes have it again.
No prior experience needed, but I will comment why Kubernetes and how does Kubernetes help with this.
And then we will get into the DNS god details.
We will talk about how we do the SAP with DNS mask.
There is a connection here between the SAP and DNS trust me.
And then we will talk about more DNS stuff like what's my upstream server,
which is the in a script proxy about the status DNS,
which does cool stuff with DNS and Kubernetes.
And then more how we are going to tie everything together with DNS mask.
Okay, so first of all, the elephant in this room.
Is this talk about Python?
No, it is not.
Why didn't you use Python?
Will be like the second in the front in this room.
Well, because I didn't want to.
I mean, Python is great software.
If you are happy with Python, that's amazing.
It's Python is amazing.
If you have a use case that the Python solves, then go ahead.
That's great.
I chose to go the hardware because I think it's more fun.
And because I think you learn stuff doing that.
And you can also do some things that are harder to do with out of the box solutions.
So we won't talk about Python anymore.
Whoops, not black, forward.
Okay, we used this.
So when I was like tried to come together with this DNS setup for my network,
I was thinking, what kind of a stuff do I want to have in this network, right?
So first one, and not sure if it was important, but certainly a quite important requirement is to block ads.
Why?
Because they are they are exploitative.
They are bad for you.
They are bad for your mental health, and they are a malware vector.
So let's get rid of those.
Second thing, it has to be fast.
Like time to resolve.
That's what I meant by TTR here is like a huge factor in perceived speed.
You open up a browser, you go to a URL.
First thing you need to do, resolve a name.
Maybe one, if you are lucky, maybe ten, if you are not so lucky.
So the time it takes to resolve that is like a, again, a major factor in how the speed of your network is perceived by users.
Third thing, it has to be private.
If you use your, usually use your ASP DNS,
like the DNS, that cheap with your ASP router.
These are like dubious or best.
Hopefully they are fast, not guaranteed, but they are definitely not private.
And some of them do not have far use of stuff with domain names that do not exist.
These are bad.
Okay, so what if I point to 888?
Well, I mean, I will point to 888 in particular.
But doing this and encrypt this, I will send dubious.
I don't feel comfortable addressing an encrypted traffic through my network.
That can be trivial amounted by anyone.
I don't know if my ASP is doing this.
I hope it isn't, but I don't want to try it.
Okay, more things, be flexible.
So that's kind of a, kind of a given, but they want to be able to maybe unblock this name.
That is in a block list somewhere, but I do want to use it.
I want to be able to integrate with the ASP because that's like quite nice.
You can jump ping a horse name and that's just works.
And I want it to integrate maybe with all the name servers.
We will see how that works.
Like maybe a cafe specific name server for a niche application.
So the flexibility is like kind of kind of important.
And maybe the most important thing on this, on my requirement list, is to be reliable.
If your home DNA is now, there is no internet.
People with geology.
People who live with you will geology.
Hey, internet is now.
Did you mess up with DNA as again?
Yeah.
I used to get these questions quite a lot.
Like one-to-month.
Since I deployed this thing, zero.
Not a single time.
And I am very proud of this.
Not a single time.
Any people living in my house have told me.
Did you mess up with DNA as again?
It just works.
And I can update and restart and do stuff and everything just works.
That is amazing.
And that is something that Kubernetes really helps with.
Okay.
Oh, infrared.
Let's go.
Spoiler.
Don't look at this very much.
We will look at it again.
But you may see that there are Kubernetes pods.
There are nodes.
There is coordinates.
We are not talking about coordinates.
So you can ignore it.
There is other devices that is devices that are outside the Kubernetes network.
There is DNS mask.
Lots of stuff.
Don't look at this too much.
We will go to that later.
Okay.
So let's get into the Kubernetes stuff.
Why Kubernetes?
Why would you run your DNA setup inside Kubernetes?
Because Kubernetes is quite nice for a lot of stuff.
And I can't believe it here.
If I can't go in order to say that Kubernetes makes failover and high availability.
That's what I mean by H.I.
For free.
Mostly.
For free.
It means in cognitive load.
Not in terms of cache.
So that means that you can upgrade your DNA server.
Which you should be doing.
That you can make changes to your infrastructure.
And nobody will gel at you for messing up with DNS.
So that's kind of kind of cool.
And again, it works for both containers and nodes.
So that means that you can take the DNS.
The containers are being DNS down.
Or the whole node.
Serving.
Hosting the container that serves the DNS down.
And nobody should notice.
The second thing that I really like is GitHub's free.
If you are not in the cloud native Kubernetes scene.
GitHub means that you put your stuff in Git.
And you have something that makes what is in Git a reality.
So this means that the config files for your services.
In this case for the DNS services.
How they interconnect.
How they relate to each other.
That will be versioned as text in Git.
That is amazing.
That is prizes.
Like, do you make a mistake?
Oh.
I will think about the mistake later.
But DNS will be up like very soon.
That means that it is kind of self-documenting also.
So if you forget and if you make a setup like over in general like this,
you will forget.
At least I have forgotten.
Then you just go to the Git repo.
You see the config files.
They put each other.
It is so nice.
It's observability.
That's what the only of the pronouns stands for.
And you can like Kubernetes is very friendly.
You just drop an annotation here and export it in there.
And you have metrics.
And most importantly, it was already there.
Like, to answer every time I talk about doing something with Kubernetes,
people ask me, should I set up Kubernetes for this?
No.
You should not.
It is too much effort to set up a whole Kubernetes cluster for one thing.
But if you have many things.
Like maybe too much for one.
Maybe too much for two.
For three.
Still too much.
You have ten things.
I would recommend you to start thinking about it.
Because then you will start having everything in it.
Everything managed the same way.
So if it is already there, why not use it?
Okay.
How does Kubernetes help us in the technical goal of detail?
So there is this thing called Metal LV, which I really love.
This is like an amazing project.
It's like, I mean, 25% of what may Kubernetes great for me is this thing.
Metal LV is layer 3 failover.
And that's what is going to be unlocking our reliability here.
And also highly, highly availability.
How does it work?
Metal LV is some sort of ARP proxy.
ARP is like DNS for IP addresses.
If that makes sense.
So what this does is that when the client ask our DNS server,
which we configured to be in other S1015,
which computer, which MAC address is 1015,
the nodes that have DNS pods will gossip between each other.
And some sort of agree on which is going to be 1015 for the next period of time.
So one of them will say, hey, I am 1015 for sure.
So then the client will say, okay, can you resolve this name for me?
And the node with the pod that has a DNS server will say, hey, deal you up.
If this node goes away, if this pod goes away from the node,
then metal LV will notice.
And the next time somebody asks for 1015, it will be a different node,
answering.
So that's how you get this failover and highly availability.
And this is how you upgrade the machine without breaking DNS for everybody,
which is quite amazing.
Okay, more stuff.
I could pause and do some demos here and there,
but I'm not sure how well we'll be doing with time.
So if there is time, I will do the, we can play with the terminal in the end.
Let's talk about something else for now.
So we got this, we got metal LV to be in the high level,
the high level availability and phylogosida for us.
So what about DSCP?
So DSCP and DNS are some sort of related,
if you want them to.
The cool thing about, about relating them is that then,
when a host connects to an algorithm, say, hey,
I am, I don't know, TC1, whatever.
Then the DSCP server can remember that.
And then when you ask the DSCP server, if it also is post DNS,
okay, who is TC1?
The DSCP server will remember, say, hey,
TC1 is on this dynamically, like,
in this address from the DSCP pool, which I gave, TC1.
So that's, that's kind of nice.
And DNS mask, which we will be talking extensively about today,
that's, that's this for you.
And I thought, okay, why not?
I need a DSCP server, and I already know DNS mask.
So it also does DNS for me, why not?
There are some papercats, I'm not doing that good on time,
so, well, that's a matter.
DNScript proxy is my third tool in this tool belt.
I think it is, it is very nice software.
DNS, privacy, DNScript proxy is going to solve
our privacy concerns, our conflicting relationship
with our, with our, with our ISP.
DNScript proxy presents itself as a DNS server,
so you can ask DNS questions to it for 53,
very normal.
It can be in result count.
No, nothing client specific is needed
and, unlike, for example, DOH.
DNS server is the PS.
But on the other side of that, of that, for 53 facade,
is going to be quite complex machinery
that fetches a list of well-known groups.
That fetches a list of well-known resolvers
that do use DNScript and will proxy stuff.
We'll proxy your queries to them.
And it is quite a smart.
It will try many of them.
It will sort them by latency,
and it will favor the ones that are like, like closer to you.
So it is in our self-balancing.
This list of upstream resolvers is also self-refression.
This goes to the in GitHub.
It is signed with a known public key by the,
by the author, so it is quite nice.
This is like a good strong contributor to DNS being fast
and definitely the key thing that makes it secure.
It also does DNS server is the PS, by the way, in this list.
I'm actually unsure if that ends up in the real world being faster
than the in a script.
I have no idea.
But for our purposes, as a client does not matter.
As long as it is simply.
Okay.
Final tool in our tool belt.
When I when I mention in my wish list that I wanted this to be
flexible and to connect to other servers,
you know, I have a Kubernetes cluster.
And in that Kubernetes clusters, there are some services
that I want to be able to reach from the outside.
As it usually happens.
Right?
Those services in that Kubernetes cluster have host names,
and it will be quite nice.
For me, to not have to replicate the host names that are on my Kubernetes cluster,
tool at the inner server that is outside by hand.
Right?
So there is this project, which I co-author called state,
there is a nest missing here.
Stateless DNS, which is like a hand chart.
If you know Kubernetes, you know what this is.
If you don't know, that's a matter.
It's like a package that bundles these two things together.
External DNS is a tool that will connect to your Kubernetes cluster.
No, look what host names your Kubernetes cluster is exposing,
and then will register those somewhere else.
This somewhere else is power DNS.
So you can have in this package power DNS is included as a small,
you know, sidecar, like it is, it isn't that.
And it configures both things so that external DNS looks at your Kubernetes cluster
say, okay, the host names that they need to expose as this, this,
and this register stands in power in power DNS, right?
So then this power DNS inside your cluster automatically is able to resolve the host names,
that your cluster exposes to the outside, which is, which is pretty nice.
And then we will tie this together.
This is like, if you want to, if you want to look this up,
this is the GitHub repo where this package, this, this hand chart exists.
Don't, don't ask about the name of the office and inside your,
dot, dot, dot, dot.
Cool, recap, pause, where are we?
So we have, metal dig, we talk about it in the, in the, in the beginning.
It's going to make our whole thing highly available, not picture here.
We got external DNS, which is, by a, a style of DNS as we just talk.
We got DNS script proxy, which is our upstream, and here there are many boxes,
because we're going to want to have like more than one of these.
And we also have DNS mask, DHCP, which is our DHCP server,
and it's also serving names for our, for our local network, right?
So now we should kind of, these are like three very independent names,
service that don't know about each other.
So we should kind of mask them together, right?
How are we going to do this with DNS mask again?
Because, you know, it's, it's so nice.
I love the, so much, so much.
So DNS mask, you probably know about this.
It has been around for quite a while.
I, I remember in my, my unidays, I will pride myself about knowing
the man page of this thing by heart.
It is no longer the case.
I don't think it was ever the case, but never the less.
This thing is, it's, it's like so nice.
It is so flexible, it is so efficient, and it can do so many things.
All of them reasonably well, which is, which is where.
So how does this work?
DNS mask will be configured to route our queries, to, got it,
to the different, to a different name service, right?
We'll see, we'll see how we do that in, in the next slide.
It can also be, be fed denialist.
That's, that's how we're going to do our, our blocking of bath hosts.
And then, yeah, again, it can load balance, and it can cache the query.
So that's, that's, also, we'll also continue to, towards pithing.
Okay.
How does this look?
The intimidating is like, okay.
So, the, glossy over this, this part here,
well, this is like, don't propagate cluster names outside of the cluster.
Cluster will be using DNS mask as well.
So, this, this should not go outside, this is a local.
This is our routing thing.
And the car here is like the name of my local network.
So, everything that is like, in the car,
the LED goes to the DSIP server.
Everything that is in the terabox.mo, the main,
that's like the domain, the main domain, that services in my clusters use.
Those external services, we're managed, we'll install DNS.
Those go to 10, 14.
This is like the state-led DNS server.
Those are, by the way, matter will be addresses.
So, these are like load balance.
Who knows what this is?
Pop quiz.
Use application DNS.net, show hands.
Who knows what this is?
Nobody.
One person, two, okay, three, four, okay, amazing.
So, this is like a can of the main,
that modzilla made up to the best of analytics.
If your network rejects this domain,
then Firefox will not default to DNS over HTTPS.
This is quite nice.
Like, if you have an over-engineer DNS network,
you want your clients to use your over-engineer DNS network, right?
It will be a pity to do this world for nothing.
So, if you black hole this domain,
then Firefox will not default to DNS over HTTPS,
which is quite nice.
I don't know how I came across this thing, but I'm glad to be.
Okay, this is like our DNS script,
proxy service, our upstream, we want DNS maps to use it.
If it is not up,
a couple of reasons, it may be not up,
then I fall back to $2.9,
and we do it in a strict order.
This is another quite nice thing of DNS mask.
The DNS mask, by default, doesn't do retrace on its own.
It relies on clients.
So, you may tough experience this,
where you query the DNS server, it times out,
and it takes like five seconds for the client to try again.
That's awful, I don't want that.
So, I tell DNS mask, KB, slightly unpolite,
and retrace on your own.
We are load balancing this across so many servers,
so hopefully operators won't hate me for this.
And then we know how some cashing and whatnot,
and then we include our bad blocks,
our bad hostlies.
Okay, so there is something missing in this complex file.
I'm not, I'm not challenging, well, before that.
How do we do the ad blocket thing?
So, can the DNS mask, is this built-in into the DNS mask?
No, it is not, but it doesn't have to be,
because the best configuration language is bash.
So, yes.
Exactly.
So, it doesn't matter if the DNS mask does not support this.
We can coerce it to do it.
So, I have this tiny vasescape here,
like it's a four, five liner, five liner,
that will fetch like my, this well-known,
there are many of these things online.
This is just one.
And it will like, okay, right, this to a file,
and then tell DNS mask to reload it,
and then if you succeed it, wait for eight hours,
because there is probably no need to refresh that list,
more often than that.
So, there is that.
I mean, it's, it's cool to be able to do stuff like this.
Okay, so in that conflict file,
there was something missing.
It's, if you, you're scuba and enter surveyed.
You may know that you have a Kubernetes pod.
How does a Kubernetes pod resolve names?
Well, it has a resolve confine, right?
Where is that resolve confine coming from?
Well, it comes from the host, right?
But we want the host to use a Kubernetes pod
to resolve hosts, right?
So, if we, if we do not tell DNS mask to not look at this file,
DNS mask will try to recourse to itself,
which is not very nice.
And it will be a very different IP address.
So, DNS mask is has no chance to notice this is happening.
And the, fun things will happen.
So, in, in that conflict file, you have to use no resolve.
So, DNS mask doesn't load resolve conf, automatically.
Fun stuff.
Okay, so this is our full picture again.
Where, where, we?
We have Kubernetes pod.
We'll use coordinates.
We don't care about this.
Coordinators will use DNS mask to DNS.
This is like a load balancing,
caching, not blocking thingy.
Many pods, highly available, tend to,
tend to, as well as be.
Otherwise, we'll also use this.
We prevent recursion of, of hell,
with, with this magic, no resolve flag.
We also have other devices outside of the network.
Since this is provided by,
the, this IP address is reachable from the world.
Well, outside of communities.
When you may argue with us the world.
And, so, all, other devices in the network will use this.
And DNS mask is configured to route things to,
to different places, right?
So, hopefully that,
we'll make sense and we'll,
you make of,
see how we have full fill,
stuff in our, in our, in our wish list.
Now, there is one thing, another elephant in the room,
so to speak, which is,
you, you might wonder,
isn't there, like, like a problem here.
If your, if your Kubernetes cluster is using itself
to resolve names.
How do you pull the images, right?
How, how do you don't load the NS mask?
If you depend on the NS mask to resolve,
you know, Docker.io,
or hopefully not, or GSER.io, or whatever.
Well, is this a problem, yes?
Does this set up bootstrap called,
no, absolutely not.
If I wipe everything and create the cluster from scratch,
it will,
not bootstrap.
It will not self heal.
That's in matter,
I will say no, like,
you know, there is no perfect system,
all, all systems, all,
every, very clever piece of infrastructure
that you try to figure out.
We'll have some problems.
I think that's, that's hardly, hardly available.
This thing will not bootstrap,
but I, this is like not a problem for me.
I will say why later.
But first, I've, I've run in this setup
in my network, I think,
three, maybe four years.
This situation, where the,
where the cluster cannot start,
it has, it has happened to me once.
As I was telling you,
the number of times that,
a more, as less of a engineer set up in Kubernetes,
has failed my network is,
we used to be once a month.
So that, that's an improvement by, by my books.
Why is not a problem often?
Because now it's will cache container images, obviously.
You should use, if you use image,
pull policy always,
you should go to jail.
This is bad.
Don't do this ever.
This is kind of a crime.
Not the, not the, not the good kind,
not the big aid to crime kind,
the bad kind.
Don't do that.
And the redundancy and clever affinity,
also helps a lot.
So I haven't show lots of jamming here,
but the key important thing,
that is DNS mask and DNS mask proxy.
This, like, tolerate the control pane,
and will, at least one replica of this,
will be in the control pane.
So that way, if the control pane is up,
at least we have something.
We are, we are functional.
And then everything else can start from there.
And, and again, yeah,
the, all nodes will, will,
eventually, as pods move a bit around,
all nodes, or most nodes,
should have a local copy of this image.
So that, that will be okay.
In the event, this is not the case.
Well, you get your hands on the keyboard,
you go to a control pane,
you edit resolve comp,
you put quiet nine in there,
and when everything stabilizes, you revert.
It's not the end of the world.
It takes two minutes.
I promise.
There is this thing I would like to try,
I call, speaking, that, or a spagal, I guess.
It's, it's not, it's not the,
not the German world,
it's a finished world,
a Swedish world, actually,
which is able to distribute container images across nodes.
In a classic, this is quite nice.
I don't use it because it doesn't support cryo,
but I think it will also, it will also help with this.
Okay.
So take a ways.
What, what, have a line?
What I, what I think you should,
you should get out of this room with,
with within your head.
DNS is very composable.
It is, it is, it is,
it should surprise nobody in this room,
but it is, it's quite nice protocol.
And, you can make cool stuff.
You can compose, you can beligate.
You can suck men.
You can, like, divide and conquer your problem.
Without doing complex stuff,
I am scared of a, a, a, a,
a, a, a, a, a, a, a, you don't need a,
a, a, a, a, a, just beligate names to places.
Quite nice.
Kubernetes is, is like a nice,
a, a, a, a, a system is,
you already know a bit about it.
If you already have some exposure to it,
I, I, I, I recommend playing with, with this,
with this, with this cool stuff, like,
highly available and failover.
It's like, really, again, game changer.
Self-documented, good versions for a structure is like a, like, a treasure.
Like, it's, it's, it's, I cannot, when I was like,
22 years old, everything I wrote on a keyboard,
live forever in my head.
This is no longer the case.
Absolutely, no.
If I may, if I had made this thing,
and not write it down as Yamil,
four months after,
I would remember nothing.
So, having everything in,
in a, a, a, a, a, a, a, a, a, a, a, a, a, a, a, a, a, a, a, a,
like, really, really nice.
And directed to this, your,
the space that your brain has is finite.
If you are able to reuse knowledge,
that's like a superpower.
Reuse import knowledge for me,
means putting stuff on Kubernetes,
because it is what I, what I already know.
I recommend you figure how that pattern works for you.
It may not be Kubernetes.
It may be something else,
but if you manage to optimize your brain to
remember the stuff that you can reuse for other things,
that's like a research for success in my opinion.
And this is all.
Thank you so much.
Thank you.