All right, so let's start good morning everyone, this and like me, my name is Harin, I'm
part of my school engineering team, my main focus is working on security features of my
school server and live my school client and today I'm going to talk about how you can
extend my school servers functionality by implementing components. I'll show you small demos
about some toy components that I have written which is available on GitHub which helps you
check the password whether it appears at any database breach or not. So with that, let's start.
I'll just skip over. So small agenda, the overview of my
skill component architecture followed by two specific use cases, the password breach
check component which utilizes an already existing service that my
skill provides and then one more refinement on how you can achieve component
changing or broadcasting a service called to multiple components and some small
library to help you back. We'll be with that. At the end, we'll see a couple of points
which may help you write your own component as well.
Okay, so my skill component infrastructure is actually inspired by service oriented architecture.
It has a registry which allows producer and consumer to talk to each other. The good part
about registry is that it's type agnostic. So it's not tied to a specific type. It can
actually work on any service name and then producer and consumer can talk to each other.
Very easy to reuse modular code. So you know, easy to maintain. One of the core thing
about this infrastructure that is that it has strong dependency tracking. So when you're
loading a component, it makes sure that all the dependent services or all the dependent
implementation that the component uses are also available. If they are not, your component
loading will fail and once the component is loaded and defines all the dependency, those
dependencies are logged in till the time the component is again uninstalled. Also, we
design it in such a way that it's not strictly dependent on my skill server. So in theory,
if any binary implements a small self, self-contained library called live-man chassis,
it can load any component using that and start using the functionality. So it really provides
good useability. There are no versions in components. No versions. No versioning.
So if you, you have a be careful when you're writing your services. If you need one more
service is 2 years later, that got to be a new service, not a version 2 of the same one.
Yeah. All right. Okay. So four major modules within a component architecture services,
which are essentially containing API definition, street of it as header files, which defines
how your API is will look like. Examples are event tracking services, password validation
services, and there are 200 plus services in the server. So you can take a look in source code.
A component is something that implements one or more services. So a component can have
n number of services implemented. It can actually define that I as a component implements
service 1, 2, 3, 4, and then it's essentially a shared library. So it can be loaded by something
called dynamic loader. So in very simple terms dynamic loader will do a deal open, get the content
from the shared library, and then interface it with the registry, which is the final part of
the server. It essentially stores the kind of a map of implementation and actual function reference.
And it utilizes it whenever a consumer of the service comes in and says, hey, I want
service password validation. Okay. Let me go and look. If I find the implementation name
password validation, I'll return the reference along with the reference count incremented. So that
that is the component in question is not an installed while someone is using it.
So this is the, this is basically the heart of component architecture in my scale. Now,
registry and dynamic loader together forms live main chassis. Remember in one of the previous
I said that we want to make it as modular possible. So if you use live main chassis, which is
static library within any binary, you should be here theoretically, you should be able to load
components. That's what mySQLD binary does. It embeds a shared this static library within
itself. And with the help of this static library, it can load multiple types of component.
So different services provided by different shared library is loaded alongside the server using
live main chassis. Not just this, you can also have multiple implementation of same service
loaded in parallel. So think of it as if, if let's say you have more than one implementation
of a connection event tracking, you can actually have all of them loaded. And then you can choose
whether to use one or first or second or third so on and so forth. So like registry provide
that kind of infrastructure. And once this is all done, modules within executable or
in fact other components can use any other component service using the registry. So server doesn't
come into picture. The only thing that server does is to make sure that live main chassis is present
and then every consumer will go to registry, identify the reference at once and directly talk to
the producer of that component implementation. At the moment, in mySQL server, we have now 200
plus services. So any functionality that is extending mySQL, mySQL server, we are using component
architecture now. So we have more than 200 services, you can take a look at mySQL's component
services space. You will find a lot of header files. A single header file can contain multiple
service implementation. So what's out for that? And since we are continuously including new
features, we are also adding to this list every now and then. So last I checked yesterday,
we were somewhere around 240. So yeah, I mean, as and when we add new features, this count will increase.
What this means for you is that all these services or all these hooks are already available in
the server and server already emits this event. All you need is an implementation which listens
to this or implement this services and make something useful out of it. In no particular order,
I just, I'm just listing some of the services which I think are really good, really cool,
provides this very good access. Take a look at few of them like table access. You can now
access any SQL table from component or you can actually execute any SQL command through component.
Right, so that's all about the overview of component infrastructure. Now, once it's there,
let's take a look how you can use it to extend functionality which is not available out of the box.
So for this part, I'm going to take example of password validation service. So server
implement, so provides this service and there are two APIs within this service. The first
API is what is used when you are creating a user or altering an existing user's password.
It goes to this component and checks whether that password satisfies certain criteria or not.
My skills are also provides a default implementation called component-related password and that
that implementation essentially checks. Okay, does the password have N number of characters? Do you
have mixed case characters, special cases, numbers, so on and so forth? It also can support a local
dictionary based search saying that okay, I have a dictionary of weak password. If I see any
password within that dictionary, I will straight away reject it. Apart from this, there is one more
API which essentially provide the information about password strength, so it gives you score out
of 100, 0 being, it's really bad password. You shouldn't be using that and 100 being, okay,
I haven't seen that password anywhere before and it satisfied all the criteria, so yeah, you can
go ahead and use it. This is the part that we are going to focus on today in next couple of slides.
Right, how many of you have heard of have I been born? Great, how many of you have searched
your password on have I been born and found that it's actually listed there? Yeah, good, all right,
so yeah, so okay, so this will help. So this is, so have I been born is one of the largest
database of breached password. It basically captures password which appears in various breaches.
Continuously updated has millions and millions of password, real world password, that's the main
part, these are not made of password, these are real words. So and it provides an API, so you can of
course, you can go to haliband.com and then search it and it will show you yes, your password has been
appeared in so-and-so breached blah, blah, blah, but it also provides an API to search your password
programmatically. So if you provide a shavan prefix, just the first five characters of shavan
hash of your password, it will send a range of password where this prefix appears. It doesn't mean
that your password is necessarily there. It just says that if I see this prefix, here is a list of
password that I have which happened to have this prefix and then you can, you can then compare with
the suffix and see if your password actually appeared and also it returns account, we'll see that
in the next slide. On an average you will have more than 400 password for any shavan five
character prefix that you throw to this API. So it's really safe, it's not like by using shavan
you are exposing your password to potential problems. We are not sending the complete shavan hash,
it's just accepts the first five character. So what I did was it's a small component which I wrote,
it's compatible with the latest mySQL 9.0, but again it's not production code, so play with it,
throw around it, do it what you want, but yeah, be careful if you want to use it in production
scenario. So whenever user supplies a password, in this case I actually chose a password which will
pass mySQL's validate password criteria because it has a location lowercase characters, it has
numbers, it's a special character, more than eight characters, everything. You take this password
supplied to the component and it will search for this password using the prefix and have
I been pawned and then again this being really really bad password, have I been pawned will return
with yeah, I have seen this and again this is not just the only reply we will get we will get 400
plus password prefixes along with account that this password has appeared 1118 time
on all the databases that I have seen so far on password bridges shouldn't really use this password
and based on that the plugin or sorry the component will reply saying okay you can't use this
password even if it satisfied some of the criteria of you know how it should be composed etc.
Right so let's take a look at the demo. So what I have done is this is a mySQL instance and
this basically doesn't have any password validation component installed. So if you create a user
it will allow very lousy password like me cd. Now once you install the component and
validate that yeah it is actually present you can try and create user or alter an existing user
password with weak password something like this right so once you do that it will it will immediately
stop you from using the password e fgh because again it's really really weak.
If you further go and check server's log it will also log information about that this password
has actually appeared so many times it doesn't give you the actual shawan hash it doesn't log
the entire hash but it tells you it okay it has appeared 454 times as per the records of her
I've been pawned. I know that this is not logging user name so it's not you know super helpful
but we are again limited by what's provided by server as of now. Right so let's let's now go back
and let's see if if we can do something about this okay we can't use e fgh can we use a variation
of word password for example um that variation will have special characters it will have sorry
yeah then you sure okay right so what I just did was I used a variation of word password which
replaces a with at the red sign o with zero so on and so forth. Turns out that's even more popular
so that appears for what more than 600000 times so the component will not let you use that.
So I can't I can't put the user name information that okay this password was used by
user herine or her user herine was being created with such a password and this is why I
didn't it because the components API is an yeah the API doesn't take user name as argument it simply takes
of incoming password as argument so ultimately user will get there that password I mean you know
the password operation failed or the create user operation failed and then they can take a look at the
would you can extend the component if you want it's possible yeah but the API changes and it
doesn't version so you have to come with another another name for a component which is almost the same
as the previous one with a V2 or something. No no it's a different service it's a different service
implementation that's okay but there's the end of the result is you you change your adjusting the
API some idea it's a make it. No no I'm not I'm not saying that I'm going to change the component
so we say PIs I'm saying there are other APIs available really to error log which may provide
more information than this so within my component I will implement one more service which will
let me find this information from server and then displayed but again probably you can do that
and extend this component and then contribute to that yeah right so okay moving forward so now
I'll use a password which is not available or which is not present in any of the
breach database of the password and it successfully actually works you can create a user with that
password you can also use that password for an existing user basically you can change existing
user's password it works because go to the API bin porn provide the prefix returns the result
but the suffix doesn't match so yeah so that's that's pretty much it about this component
but that's just the first part now what we have seen is that if you have this component it can
help you check that passwords status online but what if I want to have both
component related password as well as the password breach check working simultaneously so not only
I will check for the weak password online I will also put the constraint of how your password should
be you know created should it have certain length criteria character criteria zone and so forth
so so for that what you need is you have one service you have multiple implementation and each
implementation gives you different different functionality or different use case and then
you need to somehow broadcast this call to all this implementation and one by one get the result back
whenever you encounter a failure stop it requires something called an anchor component
not an official name I mean I made it up so my call it anchor component because
it is aware that there can be multiple implementation of the service that I provide I should use
some mechanism to broadcast this call to all other components get the result back aggregate them
and see whether I should reply success of failure to my skill server in turn it uses something called
a reference caching service this is another service that exists right now within my skill
component infrastructure right so what I did was I write a small static library just a couple
of files it it basically is a type agnostic library that lets you broadcast your service called
to multiple component implementation yes very simple API you initialize the initialize it
and then there is a broadcast call which lets you it will find all the implementation of a given
service type and then return the handled back to the caller as a call back and then the caller can
decide what to do so if you see here this is what the caller will do this is my call back
in in password breach component that I have return so with each for each service it will get a
call back it can call the validate which is the password service and if this service returns error
it will return an error we will see this in action soon all right so this is the same demo
we reached to the stage where both this users were able to use a password which didn't appear
in have I been points later base now what we are going to do is we already have password breach
check component we are now also going to install my skill validate password component which
comes with my skill packages right and this comes with certain criteria like you know 8 characters
mixed case number special character so on and so forth once you install this let's try to create
a password which possibly does not satisfy the criteria see this is the same password that we use
before if you you see is the same password that was earlier okay because it was just checked by
validate of password breach component now it is being checked even by validate password component
and this doesn't have any special character it has a per case lower case it has a number but it
doesn't have any special character and that's one of the criteria put forward by this component so
this call basically goes fast through password breach check which checks online comes back says
hey is there any other components that implements the same service yes let me go back and ask that
service if that service says yes I will allow the password if it says no I will block the operation here
if you go to air log I don't think we will see any additional error because this was just
it I mean the valid the password breach check did not really throw an error for those times
so now we will try a password which satisfies criteria of
validate password component but not the password breach check it is the second scenario we have
already seen this password will be rejected here this was rejected here it's again rejected here
so it's this is just to showcase that the validation happens against both the component it's
it's not like that we are validating only against a single component and if you go back and see
the error log you will see that yeah one more message appears that okay you're trying to use the
password which has appeared in the breach you can't really use the password now what what we'll do
now is we'll tweak the password little bit we'll add you'll change it so instead of first
ms I have added dollar sign which now make sure that this password passes both the components
criteria it is not appearing in any breach and it also contains all the mandatory characters
that are you know enforced by validate password and as you do that it passes and you know you're
able to use that password you can create a user or you can change existing users password using that
so yeah that's that's pretty much it about the demo the final piece of the puzzle writing your
own component these are some of the things that I have learned through my experience with component
and just you know would be good to share if you're writing your own component follow an example
I would say component related password is a very good example it's very small but it has all the
right ingredient that you would need it has various services that it uses so you know how to
specify dependencies and how to use them in your component and it also provides certain services
more than one service actually so you can you can take a look you can copy paste and then play
around with this stuff there are extensive macros provided by component infrastructure header files
please use them that they will make your life very very easy defining various mandatory parts of
this component in fra this macros are very useful I mean it's like 10 lines of code is can
encapsulate it with just one macros so please use it if you're writing your own service and
this will involve hacking into server adding certain services here and there make sure that you
use only basic types so integer floats characters so and so forth if you have a complex object
that you want to use or return or pass around between your APIs use them as a opaque pointer
so only your component should be operating on that you should not expose that through service
APIs one of the very important point is how long you want to keep handle of or keep a
reference of another service so if you define an explicit dependency component is such
will by default hold all dependency references for the lifespan of your component that is one way
but if you are dynamically taking reference to any service be careful it's expensive so doing it
few times is okay but if you are doing it hundreds and thousands of time that will put pressure on
the entire live-manjasi infrastructure on the so there is a dedicated call to acquire a service
and it's always recommended that you acquire a service or you already hold the service
reference before you call any of the API of the service so once you do an acquire behind this
in live-manjasi will increment a reference make sure that that component is not an installed
and this is really about it keep it for the long time that means that other component cannot
go away till the time your component releases the reference so it's kind of striking the balance
right and if you so okay so component first you start state less if you want to maintain a state
pass your own opaque pointer but if you are doing that make sure that you have a destructive API otherwise
you will leak memory very easily so that's about it this are some of the references you can
you can take a look at the doxygen documentation of server it provides a nice summary of all the
components that we have it also gives you an example of how you can write your own component
and the last two is the implementation that I have used in this presentation so that's it thank you
we have time for questions how questions long last one first my place I have this issue
I have to ask you to ask your plug-ins is still around plug-ins you say want to go away
you will be replacing your plug-ins with the components so that people can use this
we are already doing that yeah so slowly all right I'll write about it but we can not
straight away remove plug-ins or the API is because we may break things left and right so right
now we are deprecating things that okay so validate pass what is one prime example the plug-ins
service is now deprecated probably we will remove it sometime in future but it's very it would be very
ugly to remove it right away because just because we have the component in place because if somebody
else is using it there might be an issue sure sure sure sure sure sure
I'm not sure I understand if you can please repeat I'm not sure I understand if you can
please repeat I'm not sure I understand if you can please repeat I'm not sure if you are running
just serving an environment but you don't have to have access to that yes yes you can
okay thank you thank you