Hello everybody here at FosterMet for Lighting Talks. I want to introduce to you
Claus Ellic and he's talking about tweeting bit-differentiance indibendent of
their origin. If you're from a warm welcome and enjoy the talk.
Okay, thank you very much. As a title suggests, I want to look into how built-in
finishes are treated, especially in turning a build system, and why I believe that it's
important to have that independent of where the definition is coming from. And of
course, there's all implementing free and open software. And since it's a lightning
talk that's jumped right in an example, I brought you a make file, not because I believe
make files are a particular close to the independent of where the definition is, but
because a lot of people can read make files, I hope you understand what this example
is all about. In fact, make files are pretty, the opposite of what I think. So make
files as you have targets, use files, and they're identified by a pass into the file
system. And then you say, well, this file better be newer than some of those other
files. And if not, then there's a command run and hopefully it will have the right side
effect. And that it's all about side effects is something you notice. If you look at all
those phone targets that don't even create a file that just there for side effects. Still,
make files are used to build software. So there is a computational content to that, and
let's have a look at how that might look like. I actually just look like in the build
tool I'm working on. So we have this first object file. It's a file, and we say, well,
it is the output of running some command. It's actually the output at location food at
oh. The interesting thing is, obviously, what is behind that hash there, and it's an
action. So what is an action given by, well, you run some commands. It's a make file.
You probably have a string that indicates string, and then pass it to a shell. You specify
some environment variables, definitely pass. Make files just use the ambient
pass on the location, but to get it well defined, you specify something, either something
very generic, as here in that example, or something very specific to the installation, like
a next pass or so. The interesting thing is, so, now, how do we talk about files? I
say, I don't want to say, well, take the file from the end of the file system. And there
I use that it is a committed source file, and now what I speak to the one entirely, use
it so we can use the git plot by the endifier to identify a file, and then we define it
by its content rather than, oh, it happens to be there on my disk. Okay, the next thing is,
well, it's got some interesting, if you look at this library. Well, of course, it's also
some output of some action, but if you look at that action, well, we also have the command,
we expect one file to be created, and there's an input, well, we have the output of the previous
action, and that is the hash user scene, so that way we can encode the whole action graph
into these identifiers, and have an description in the, well, have a description, first of all,
independent of where on my disk, the source files were, but also with that, I get an action
graph, and having all this input output very explicit, I'm independent of running the command
in the same working directory, I can have a separate directory if I could all the files in
where they belong to and get out the expected outputs, and that way I can also use remote execution
or whatever, distributed over many machines, and maybe share cache with other ones working the same
thing, but the more interesting thing if you look at targets, so make file, we have files,
but if you look at that, make file, you see, well, there are two libraries, two and bar,
and if you look at bar, some of the inputs is also seems to be that food and HPP, so there
is a dependency, well, hidden in the, in terms of that make file, and I guess that's why the
template make first got out of fashion, and you want to make that structure explicit, so you write
something like that, saying, well, I have food, it is a library, it has health, it has sources,
say, so it's bar, but bar depends on food, and I have an application of binary,
that depends on bar, and only implicitly then depends on food, so I should mention a bit
about the syntax here, this reference to the built-root, saying it's a library, I'm referencing
to a built-root because there's a built-root, I can't know any programming language on earth,
so I want that delegated to a separate proof set, and since this toy example,
and I talk is not the only C++ project that should be built by the tool, the C++ specific knowledge
is, of course, in a separate logical repository, and that syntax is kind of a, this,
like way of writing some type, the first-character, first MTC says, which kind of addressing a rule
it is in this case rule in a separate repository, that repository is called rules from that project,
that you find a directory called CC, and that you find a rule description file,
and from that you better take the entry library. Okay, so what does a rule do? It takes all the inputs,
and then gives a description of the output, and you've seen already how you can talk about files
without having to build them, and if you look at who we get something, well we get
a primary artifact, which is a library, it is created by some action, and here you see that the rules
are a bit more elaborate than what I've shown on the introductory examples,
because you won't maybe the compiler as an input implicit to a tool tennis well, so you
have to move everything to a sub directory to make space for the compiler being an input of
that action that's why the output pass is in a sub directory, and now it doesn't look as ridiculous
in the pass that well we take the output called full and put it as position full, and you also have
the header file, because to use the library need the header file, and the only other useful information
in library that doesn't depend on anything else is, well the linking, if you want to link
against this library, that is what you should at your linker command plan. It's because more interesting
if you now take the library bar that depends on full, we still get a primary artifact, which is the
library, we still get a header file, but we get the additional information that, well, if you want to
compile against that library, you better include that header file for as well, and the same,
if you want to link, well, you don't only need our primary artifact bar, you also need that other
library, which will depend upon, and the linker command plan says well at first bar, then full,
so that the open libraries simply from bar are found in full, and that is something where we get
away from the way we defined the library, so that is all the information we need, so we no longer
have to reflect that it's coming from a library that actually depends on another library,
we can depend on that description, that is the only thing that a consuming target gets,
and by including it here we basically can forget about the build switch, we have all the
information we need, and there's a second thing we can do, here we have the script of files,
still in, as an instruction on how to build them, we can actually do that, and then we get this,
see how the edges have changed, now it's more concrete, or actually what we have replaced is all
the description of an action by the result of running that action, so that is now basically a package,
you can use that, we have some controversial bestow where you can store the files, and that's
also why it's no harm of having the files duplicate in everyone depending on it, because the only
copying is the hash saying, well that is where we refer to, and yeah, so that's why it's
basically a package, and it's just a small projection, so how when can we replace such a library by
it's kind of a package version, well if the repository hasn't changed, so we have to have a look
at repositories, so first this is again the build description, if you say, well this by
something that should be reusable, then I just rename that entry, but I didn't change the definition
though, it's still the same thing, and then as a new by I say, well basically take with bar,
and make it reusable, and I have to provide some information, which I haven't really talked about,
there's a configuration that you kind of debug with, et cetera, and I have to specify which part of
the configuration should be taken to account, the rest is from one away, that way I can immediately
know what is relevant, and now I have to talk about repositories, this is literally what I
added to this example, it's saying, well here you find take the current working directory,
but actually it's a Git repository, take the head commit, and the same is for the rules, I have,
when I talk about rules, I mean that repository in some parallel paths, and again it's a Git repository,
take the Git commit repository to be that, I could have also specified some some branch or whatever,
in some external repository instead of copying it here, in any case what I get is this,
I'm saying okay, you got a repository, and the root is just that gets tree identifier, and in
case you don't know what the tree identifier looks, there's some repository where you can look it up,
and that's actually what this already gives me, I don't need a check out, I'm building against
the repository with these three identifiers, because I create a fresh directory for action anyway,
and now the first time I'm showing the tool I'm working on, which is called just those,
so you call the repository tool, is to ask it to analyze the target, well you get
precisely what I've shown in the earlier slides, so that five mentioned there's just the
repository script for showing you, the interesting thing is saying well I went to an
target that is meant for caching, and here's the cache key, that's why I increase the lock limit,
what does it contain? Well the target inside that repository and the configuration,
and a description of the repository, again the interesting thing is what comes from
that good plot identifier, and now you see the graph, so the basic message is,
you only have the trees, because the tree of a source repository determines the source
tree completely, no matter where you start that thing or where you got that repository from,
so that is a key, and the other thing is, I only looked at the definition of the
naming, so I could also, in order to make it a canonical key, rename the repository names
by taking the entry point zero, and then enumerating some canonical reversal,
though that's why the bindings are they are changed, and now if I build something that uses
that library, well you get some also you compute some built actions, but as a side effect,
you evaluate the package, so if you now look at the same library again, you see the other
definition of the package, all the action graph is gone, and by gone I really need gone,
if I build it, it says no actions at all, and in fact it's not only that they are all cached,
but the analysis now says well you have a package definition of that, you can use it from there,
of course, that requires to use it that this repository, the graph is shown doesn't change,
so it's mainly useful for someone from an external repository, logically one,
could be several sub-directory, consumes it library, and there is one more thing,
I've shown you in detail all the keys, so it's basically the get tree,
of course you would never specify dependency or I want that get tree, but the way you specify
the something like oh I want that archive or that release or that commit or that
take of some get project or something like that, and there's of course a functional dependency
from the commit ID to the tree ID, or from this is the hash of an archive, what is the tree,
if you unpack that archive, so I can build a service that answers that questions, and I mean that
service of course needs to know all the sources, but whenever you use a project you should have
a backup copy of this upstream source is anyway, even for the third party projects, and from that
information I can compute the tree at the cache kit that I've just shown you, and the next
thing is that contains all the information to build it, so I can have a service that works with
the remote execution service I might use anyway to share computation, not only to boost
you to buy it, but also to share it with others working on the same project, and in that way
I can really, I'm in the situation that I can take the project I'm working on to check out,
do my changes built, and all the dependencies I don't even have to fetch days day on my service,
and I have all the benefits as if I were bootstrapping all the dependencies without having to
clutter these things around, because even at the end, the only things are bootstrapping
a particular compiler that can as well say on the remote execution together with service backing
up the sources. Yeah, so with path to the data sort, I would be, so that is basically the
message I want to say, and it is, so what is it? Take a message, first of all, definitions
are an interesting object. In their own right, you can take them, you can serialize them,
you can send them over the wire, you can sort them some content and I submit a store,
and if you focus on housing that you find rather than at which place they're defined,
that has some benefits, for example, the seamest position between building and package building,
you've seen that, oh, I just replaced that one hash by another hash and the meaning of that hash,
and of course as I said, it's all free and open, it's all software. You can download
from GitHub and depending what you use, you might also find it in your favorite package repository.
Thank you.
Okay, thank you for the talk.