Yeah, next up, we have Zobia.
Hey everyone.
Can you hear me well enough?
Yeah.
So my name is Xavier.
I'm a programmer and songwriter.
I'm based here in Brussels.
And I'm going to be presenting a software project called
it's a prototype of a music production software that tries to bridge the gap between
life-coding and conventional dollars.
The stock is not going to be a product pitch.
Power is mainly a research prototype at this point.
And while it is open source, I wouldn't recommend you actually try it out.
Instead, I kind of want to sell you on the ideas behind it and how I think they fit
into the free software movements.
I'm hoping to make you maybe curious or inspired.
So to explain how I started this project, I'll give you an example from Odor.
Let's say I have two different effects in Odor.
And I want to sync up their settings.
For example, one node that I can turn and it changes the EQ on two different tracks.
It's something that you can do in some of the dots.
I haven't found a way to do it in Odor.
To me, it feels conceptually simple.
It's basically, I declare some states for my knob.
And then I use that state as the value for two different EQs.
And then, you know, slap a GUI on top of it.
How difficult can it be?
But in reality, this requires a really good understanding.
I think it's a good understanding of the codebase of Odor, which is really big, by the way.
And not only that, you can't just slap a GUI on top of something.
It's like you have to kind of understand how it fits into the program,
how to configure the settings of the knob and everything.
So even though the feature seems simple, it's not something that I can do quickly.
For example, what I take a break from mixing a song that I'm working on.
To me, this represents a mismatch between the ideals of free software and reality.
We have the essential freedoms and they're important.
But why they guarantee that we may change software that we'd use.
It doesn't mean that we actually can change it.
And I find that in practice, software doesn't really want to be changed.
Unless you're like an actual contributor to the program you're using.
And we have it pretty good.
I assume most people in this room are programmers like me.
Like most users, like to most users of the ideas of free software, even less accessible.
And I think one of the big problems is a lack of composability.
A tool is composable when it's made from simple parts that can be combined together into complex things.
For example, the UNIX shell is composable.
Simple tools that do one thing well, and then you pipe them together to accomplish complex tasks.
Another great example is EMAX.
When you're using EMAX, basically it involves learning and using a little bit of E-list.
And this creates a sort of gradient between using and programming EMAX.
And you can walk this gradient based on your needs and your mastery of E-list.
And in comparison, most GUI applications have poor composability.
There are good reasons for that.
Making GUI is composable is difficult.
In some ways, it's still an open problem.
And composable interfaces tend to be more complex and less beginner-friendly.
But the consequence is that instead of a gradient like EMAX, we have a cliff.
And so sometime ago, I set out to make my own DAW with composability in mind.
And to do that, I borrowed ideas from two different fields.
The first is live coding.
And if you don't know, live coding is the form of live music, where a performer writes a program in a domain-specific language.
And then the program is continuously executed and produces sound.
And because live coding is based on programming, it's inherently composable.
Typically, live coding is used in electronic music to make procedural loop-based music.
The part my project, it's based on live coding, but the goal is a bit different because it targets the same use cases as a traditional DAW.
So things like recording, arranging, and mixing tracks.
Let's look at an actual example.
How do I do this?
So let's pray that the sound actually works.
And shout at me if it's too loud.
So basically, I have this source code here on the left.
And it's the source code for this music that we can hear on the right.
And I'm going to show you how I can, so this is supposed to be a DAW.
Imagine that it's a DAW.
And I'm going to show you how I can add a track to this DAW.
And so because it's a programming language, a track is going to be a data structure.
I'm going to copy some code to go faster.
So this is a track.
And the important part is this list of clips that I have here.
And so there's a single clip in my track.
It's this wave file.
I have a start and end that represents how the clip is trimmed.
And then I have a location on the track.
And I'm going to go ahead and set it at the beginning of the track.
Let's rewind and let's listen it.
Right, we count here anything because this is just a data structure.
I actually need to sample from it.
So I'm going to call this function here called track read buffer at.
And basically, it reads a buffer of audio from this track at a given location.
And then I'm going to mix that buffer into the master buffer,
which is the one that we're actually listening to.
And this is just like a vector addition.
I'm like adding all the samples in those two buffers.
And let's see what this sounds like.
Now I can hear my track.
Okay.
Now maybe I want to change the volume of the track.
And for this, I'm going to call this buff scale function,
which multiplies every sample in a buffer with.
By a constant, let's actually maybe set it to minus 10.
So we can hear better.
Oh yeah, it had to compile.
It's a programming language.
Yeah, there you go.
And then another thing.
Yeah.
If you want to reply and effects,
well, effects and like pretty much every behavior in part are functions.
And so if I want to do a low pass favor, for example,
I'm doing a function call here.
Let's set it to maybe 200 hertz and a queue of 0.7.
And let's listen to it.
We can barely hear it.
But yeah, you can hear the low pass filter, hopefully.
Yeah, yeah.
And that's basically it.
And if you're kind of wondering how this works,
like this is this application on the right.
It's like language server for the programming language
that I have here on the left.
So I can actually go to like the definition of things and so on.
Okay.
Let's,
oh, that's my notes.
Let's continue.
Yes.
So now as you can plainly see,
I'm changing a song this way is miserable.
And I think this is why live coding doesn't usually target this use case.
But I think the properties of this medium are still valuable.
I'm using a programming language.
It's during complete.
I can sort of do anything.
It's making easy things really difficult.
But it's not making anything impossible.
And something that I also really like about this is like all the data in my
file is there in its raw form.
I can see it.
I can understand it.
I can change it.
You saw me like copy pasting stuff.
I get like all the nice things from having this text based medium.
A lot of the features that I would need in a regular DAW.
I don't need to implement them directly in this because I have them in my text
at it or in some way.
And all the features of my DAW like the effects and everything are functions.
I can compose them and I can go to the source.
I can make my own version.
But we have the usability issue.
And so to address it, I borrowed a second idea which is bi-directional programming.
It's a concept from the fields of programming language theory and human computer interaction.
In bi-directional programming, you write a program that has some sort of outputs.
But you can also change the output of the program by changing it directly.
And this gets reflected back into the source code of the program that generated that output.
A seminal work in this space is sketch and sketch.
It's a vector graphics tool.
And basically, you have code here on the left.
And when this code is executed, it generates the drawing on the right.
And then you can actually change this drawing directly like an inkscape.
And it essentially runs the program backwards to change back the source code,
like really cool project.
In part implemented, the like simple version of this.
Basically, I have these, what I call these visual macros,
their special forms that wrap a value in the programming language
and generates a GUI for that value specifically.
And I'm going to show you a few of those.
Let's go.
Okay, so now I'm going to, where's my mouse here?
Now I'm going to reveal this window here.
And you can see a bunch of these visual macros that I talked about.
So if I click on this one, I can see this checkbox here is connected to that piece of code there.
I have a slider here.
It's connected to that piece of code there.
And to give you an example, I'm going to change this frequency that I have here in the effect that I added earlier.
And make it into a slider so that I can actually adjust the frequency as I'm playing.
I'm going to arrange it from 100Hz to 10,000Hz.
And so I have the slider that appeared here.
Now if I start playing.
Yeah, you can sort of here the, you can hopefully hear the,
like frequency of the loop as well as changing.
And you'll notice that when I change this slider, it changes the source code.
And like a slider like this, it's a pretty common thing that you'll find in a lot of life-coding systems.
It's very useful to have.
Another macro that I really like is the checkbox.
I find it to be very versatile.
So for example, a lot of this music, like all of this music here, I recorded myself.
We have like some MIDI stuff.
We have some audio stuff.
And I recorded it with a click track, which we can listen like this.
And if I uncheck the box, we don't have the click track anymore.
And I check it again.
And then we hear it again.
And the way this works is very simple.
In the source code, here I have the definition in use of my click track.
And then it's wrapped in an if statement.
And the condition for the if is the value of the checkbox.
That's it.
And I use it also to like record on tracks.
I can use it to mute a track or to enable an effect.
Okay, thank you.
And then the final, which is that I'm going to show you the final macro.
Then I'm going to show you is this track macro.
So I returned to the track I had at first.
I wrap it in this track macro, I save.
And now we have this track that appears here.
And this is the sample that I did at the start of the first demo.
And I can move it around.
I can resize it.
And yeah, that's kind of it.
And if I don't like what I did, I can undo the changes in my text editor.
And there are like the things I'll back to to where they were.
And yeah, that's it for the second demo.
So this track editor is like the final thing that kind of makes this into a sort of usable though.
Like I've used it to make actual music.
But it's also maybe my least favorite macro because it kind of goes against some of the ideas in this project.
Because it's not really composable.
It's just this monolithic entity.
I think that better ways to do it.
This is what I have now.
Okay.
I really understood.
I argued that adding a small feature to auto was a lot of work for someone who's only a user of auto.
How is Pau any better?
I think it improves in two ways.
First, this idea of walking the gradient that I mentioned earlier.
Using Pau and extending Pau are no distinct actions.
They exist on a continuum.
If you know how to do one, you sort of know how to do the other.
And then another thing that makes that is better is how changing the GUI in Pau is less complicated than in auto.
Like don't get me wrong.
Overall, auto has a better GUI.
It's more powerful.
It's like tax more stuff into it.
But it's more complex to extend.
In Pau, the GUI is more modular and more composable.
If you're familiar with immediate mode GUI's, it really feels like writing an immediate mode GUI for the song that I'm working on.
Pau doesn't like meet all my ideals of like maliable music production tool that I would like to have.
But I think it's a step in the right direction, and I think there's a lot more to explore in this space.
So I'm going to wrap up now.
Why did I come to first them to show you a prototype of an obstacle doll that I don't think you should use?
Well, as I said, I'm hoping to make you think about how we make software.
And I think we the developers of free software, and particularly free creative software like does, have an opportunity to think about this sort of stuff in different ways.
And to build it differently.
As I explained at the start, I think the essential freedoms of free software are not enough to uphold these principles we value.
We should strive to make tools that users can actually own and shape to fit their needs.
This is particularly true of creative software.
Making art has always involved some degree of toolmaking.
I think of painters who are like tweaking their own paint recipes.
I also don't think we will see this sort of innovation from commercial actors or like big commercial actors.
If you're in the business of selling creative software to a mass market, you can't make anything that's too weird.
Whereas I think in this community, we have the freedom to make something different.
And I think like if you're interested in creating new music production software, I suggest you try and make it weirder.
And it doesn't have to be great.
Creative software does not have to be great to be a value.
It mainly has to be different and interesting in its own right.
Because art is not about efficiency, it's not about correctness, it's about aesthetics.
And aesthetics is shaped in part by the tools that we use.
Tools that are different, invite their users to make art that is different.
And tools that can be shaped by their users, invite them to craft their own direction.
Thank you very much.
Thank you.
Amazing presentation.
Really, really, it's very interesting.
There are any questions.
In the meanwhile, it's just in the room as well.
No, I mean, it's just the next speaker.
Okay, so if you can start preparing.
So the first question was here.
I don't know if our door is scriptable or not, but he's okay.
So yeah.
Okay, indeed the question was if it was thinkable to kind of integrate these project into an existing project to have the best of the two worlds.
And for instance, I also think into LMMS.
I don't know if you know LMMS is another sequencer, simpler.
So maybe a good starting point for this kind of integration.
If it's not scriptable, that as well, I don't know.
But the idea is to take the best of the words.
Yeah, so the bird question was, is it like possible to integrate this into an existing tool?
What is it?
I don't know.
If you just put it into the MMS.
So I think it is possible.
Yeah, yeah.
And if you've used the door like a lot more tweakable in a lot of ways.
This project was more like, as I said, it's more of a research project.
And I think I wanted to explore some specific ideas.
I do think probably the, you might get more value by trying to like extend something that exists already.
The advantage of doing something green field is that you don't, you are not limited by the thing that already exists.
You can sort of do whatever you want.
And there are drawbacks to this, but there's also a lot of value in that.
Thank you for the presentation.
I have a very basic question.
What does FTL stand for?
Oh, it's just the name of the programming language in the, it's just called FTL.
It's called futile.
It's a reference to a song that I like.
That's it.
So it's also available as a standalone programming.
Yeah, but I mean the programming language is tightly coupled into the project.
Like, it's just, I made this project without, I'm also programming language nerd.
And so I was like, oh, it's an opportunity to make a programming language.
So I did.
Yeah.
Thank you.
I'm also available for questions afterwards.
It's also time to get everybody.
It is also easy to write, effect inside or only use the effect that is ready from the,
So everything that you heard, like the, the drum and everything,
I wrote everything in the programming language of the thing.
For example, if you want to create an equalizer, for example,
I can code it to the integralizer or I can only use the function of equalizer.
No, so the equalizer that we heard, I wrote myself.
The, like, not the equalizer, but the, like, the low path filter,
which is sort of an equalizer if you rethink about it.
They're making me run for the, to the, to the back lines, so just a second.
Okay, so the question, because this is very, very interesting.
I think it would fit very well in the modular approach,
where you connect things to other things.
Yeah. So do you see any technical problems
into turning this into an audio plugin?
Um, no, I don't think so.
Like, here, here I'm writing buffers of audio outputs,
but like, I could connect them, like, I'm, this is running on jacks,
so I can connect it to auto if I want.
And I could also, like, output MIDI if I wanted to, I, like,
the drums I recorded with a MIDI pad, so I can input MIDI,
I can also output MIDI, so I could connect it into other things.
Okay, because sounds would do more than, yes, thank you.
There was, yeah, one last question, and then we'll move to just.
I mean, I haven't jogged much lately, so this is good for me.
I'm sorry, there's something that is just begging,
and I might have missed something.
You have bi-directional programming,
and it seems to me that AI is the best bi-directional programming ever.
So, I'm expecting this question, and I'm not going to know that for that.
So, I don't feel totally sold on the state of AI as it is right now,
and I'm not going to talk about, like, the ethical aspects of it and anything,
but the thing is, like, AI works great as, like, to generate code,
but it's also good to be able to understand it.
I think here, the idea is, like, if the medium of my, like,
like, program Moldo is this programming language,
and I can use an LM to generate the source code.
I can get a lot of value through this approach as well,
and so I can have an LM to generate some source code.
It also generates the GUI for it, and then I can use the GUI instead of like this.
I think, I don't want to talk more about LM's,
but there might be some ideas there, for sure.
Thank you very much.
Thank you.