Hello everyone, and thank you for being here in this talk, we'll talk about new pipe and
how it was for at it to sail fish or us.
So first of all, who am I, I'm Fabio Germán Asio, it's also known as Staipox Online.
I'm the maintainer of new one of the maintainers of new pipe, and also I created an assistant
for Android called Dicho, which is available on Android.
I'm a member of the new pipe association, and also for local hack space.
I'm currently studying in ETH, in Zurich, and yeah, you can find my details there.
Then we have David.
Thank you very much.
Hi, I'm David, Lauren and Jones.
I am not a new pipe developer, but I am a member of the self-ish OS community,
and I have the privilege of meeting some of the new pipe dev team last year at
them.
Try it out, and we're so taken by it that I was really keen to see whether it would work on
self-ish OS.
So, after Fabio has talked about a new pipe protocol a little bit more about
putting it to self-ish OS.
Yeah, I briefly started with a quick introduction of what new pipe is in case some of you
don't know about it, so it's basically an Android app that lets you watch YouTube videos,
or those videos from other streaming platforms.
Yeah, it's completely open-source.
It's available on Android.
You can use it also as a way to get more privacy when watching YouTube videos.
For example, because the official YouTube app tracks and everything.
So, I will go through some of its features.
So, here, number one, you can obviously search for stuff.
For videos, you get videos, playlists, channels, then you say that you tap on a video.
Number two, you will get a video with a bunch of information about it, such as comments,
description, a bunch of actions you can do.
Number three, you get recommended videos and related videos.
And finally, number four, the important thing is you actually get to play the video.
And the player is one of the most important parts of the app.
Since it's very complicated and it allows you to play stuff in the background,
or even in a pop-up that goes other other apps.
You can build a queue of multiple songs to listen to and so on.
Then onto other features, we have the possibility to download stuff.
So, download music, download videos, download subtitles.
Then one interesting thing is that you can book my playlists.
And this might seem simple, like it's just a link to a remote playlist.
But we actually have another feature which is you can build playlists out of videos that are stored locally in the app.
And this works not just for YouTube, but for all of the services that are supported by new pipe.
And you can build playlists that contain videos from multiple services.
So you can have some YouTube videos together with some peer-tube videos and so on.
And the number three you can see how the playlists you are looks like.
Then moving on to more multi-service features.
We have subscriptions which also just work across all services together.
We have a unified feed for all of your subscriptions.
So basically what new pipe does is it keeps these subscriptions locally.
And it makes a network request for each of the subscriptions every time you reload the feed.
And it just aggregates everything together.
And again, this works not just for YouTube, but for all channels you subscribe to from every service.
And yeah, finally number three you can see that new pipe actually supports YouTube, SoundCloud, MediaCC.
All possible peer-tube instances and Bandcamp.
How does this multi-service support work go?
Because you might understand that it could be high to implement it in the absence.
Every service has a totally different interface for fetching stuff for how it needs to be handled.
So what we did to make sure that things work at simple and clear interface is that we developed a completely separate library called new pipe extractor.
Which doesn't rely on any of the Android APIs.
It's fully like it's plain Java.
You can use it in any Java project actually.
And what this library does is it provides unified interface for all of the services.
And then internally, it does all of the things that need to be done in order to get data from some specific service.
For example, YouTube has to make some requests in some way SoundCloud has to make you have to make requests in some other way and so on.
And how do these requests actually look like?
So for example, say that we want to search for something on YouTube.
So we write the query in the search bar.
And if we were going to open the accessibility, so the developer tools in Firefox or in any other browser,
we would see that the browser made a request.
And that the request number three contains a bunch of JSON information.
And yeah, what the extractor does is it basically makes the same requests that the web browser would do.
And then extract this information to show it nicely in the app.
It's a bit harder for videos because YouTube openskates stuff and so on.
But yeah, this is basically basically it.
On a completely separate topic, you know, when you have an app that is used by a lot of people who have all different needs.
They come from different backgrounds, they want to do different stuff with the app.
Sometimes it's hard, it's hard to come up with UI that works for everyone.
You don't have to read the text here, it's just an example.
But the point is that sometimes you have to make high decisions to build your eyes in a way that tries to meet as many needs as possible.
And this is an example of what we did a few couple of years ago.
So we had to redesign the action buttons under the videos because there were too little space and we need to add more.
And so we came up with three possible designs and then had people vote.
And yeah, we have to do many of these kind of say surveys.
It's not actually a survey, but yeah, many of these decisions.
For example, now we are working on refactor in your pipe to improve it UI, which has started to feel a bit old.
So we want to update it to newer material design, new material design guidelines and also better like code under the hood.
And so for example, in number one, we have the new long press menu, which is another example of trying to fit together as many needs as possible while making the interface.
Still accessible for people that just downloaded the app for the first time and don't know anything about it.
So we have the new long press menu, which allows you to do a bunch of actions when you long press on videos or long press on channels or long press on playlist.
You have icons, you have text previously, we only used to have some text in the middle of the screen and was high to read the high to add new options to little space now.
It's better hopefully. The number two we have improved the comments UI, it's not a separate page anymore, it's a proper pop up from the bottom.
And finally, number three, we have a new player for the application, which will hopefully replace the current one and fix many of the related bugs that sometimes pop up in the player we have right now, which is.
It has been called the spaghetti monster of doom.
So yeah, thanks to Shabby, who should be in this room for making that player.
And yeah, with this, I will switch over to David.
Thank you very much.
Okay, I'll just put myself up because I might need a pace.
I think that works great.
Yeah, thanks very much for a brilliant introduction to new pipeline explanation of the features barrier.
So as I mentioned before, I tried new pipeline last year at Fosden because I was basically given this flyer and tried it out on Android.
And as you've just seen it, it has a lot of really brilliant functionality.
And I'm a selfish OS user, so I was really keen to see whether or not that functionality could be ported on to selfish OS.
Oh, not to see it. Sorry, no, no, it's okay, it's okay.
So selfish OS, if you're not familiar with it, is a Linux based operating system for mobiles.
That has its own user interface, it is GNU slash Linux.
So it has a sort of a GNU user space.
So it's a little bit different from Android and quite a bunch of different ways.
One of the ways it's different is that it is called self-ish OS.
Everything is nautically themed.
And so the name change from new pipe to self-ish OS as it went through that transition.
Okay, now it's good. Cool.
So one of the big difficulties that are getting new pipe to run on self-ish OS is that new self-ish OS doesn't have a Java runtime.
And of course, Android apps are written in Java or something that compiles down to Java bytecode.
So you need a JVM in order to run it.
And without that, of course, that's a problem.
The good news is that a member of the selfish community developed something called sailing to coffee,
which is a way of compiling down Java source code to native executable to run on self-ish OS,
using something called the Grow VM native image compiler.
So it compiles it down to native code and then you can run it without the need for a JVM.
And so that was what I thought would be neat to try with new pipe to see whether or not that would work.
And then the functionality that then you get an executable library and that functionality can be exposed using Java native interface.
And you can then link that to a C++ or C or some other native executable and execute the code in the Java as if it were just like a normal executable.
There are some restrictions with using Grow VM.
For example, you can't use reflection, which you would normally do with Java and certain things like that.
So that adds some complexity, but in theory it should be possible.
But there are some other restrictions as well, because you also don't have the Android libraries.
So it's user interface libraries, for example, on your system. So you need to get around that.
And one of the really nice things about new pipe is that it is split into this extractor that Fabio explained before that deal with all of the back end services.
It deals with interacting with YouTube and SoundCloud and Bankamp and all of the others.
And then the user interface code, which is actually, it's not just separate, it's in a separate repository.
So they're very clearly defined, very clearly separated.
So the idea then is to expose the new pipe extractor code through using this small shim that gives us a Java native interface to some C++ code, which then uses a cute user interface, which is what self-ishores typically uses for its app integrations.
And then you get a nice self-ishores like experience in the user front end, but you get all of the nice functionality that new pipe provides from the back end.
And for people to use self-ish, that's really nice because they get something that they're familiar with in terms of user interface.
And of course native applications are something that self-ish users generally encourage because it helps the ecosystem.
So this is another diagram that kind of shows how the extractor and the UI works on Android.
So this is the Android approach where you have the extractor that deals with the back end and the user interface that is also written in Java.
That's some Kotlin on the user interface side, and they interact with each other using Java calls.
So to convert that to using Grow VM, we end up with something that looks like this.
I want to shout out to Jerome, who has done all of the images for this.
Thanks so much for the great images.
And the way it works on self-ishores is that we compile the extractor down into a Java into an executable library, a native executable library, with this very small shim wrapper that converts all of the data types into JSON on one side.
And then we have some C++ code on the other that converts all the JSON back into C++ structures on the other side, and then the two can interact with each other essentially using JSON.
And then you get this QML side to do with the user interface and the back end.
So the whole effort of going through the process of converting it took in my estimation about six weeks of full time development time that was split across 200 days.
And I want to shout out to all of the people that followed that took part in that process with me. I really appreciate it to your support.
So it was quite a lot of work to get things to the place where it works.
But in practice, I actually only touch the surface of the new pipe functionality.
You saw all of the stuff going on in new pipe. There's a lot more that could be integrated.
But one of the things that I was really pleased to discover is that the Java compiled down to native code was really performant.
Much more performant than I was expecting. I thought I would end up with something that was slow and hard to deal with.
But it actually works really well, really nicely, even though there's not this JVM in the back end.
And you can see in the images that on the left-hand side is the Android interface.
And you can see a lot of the Android functionality shown there in terms of viewing videos on the right-hand side as the self-issue and OS interface.
And you can see that they are different. They have interfaces that match the platform that they run on.
But you can also see similarities hopefully in functionality across the two as well.
So some of the drawbacks and benefits of doing this approach of porting Android apps to self-issue S.
In terms of benefits, well, the real big benefit is that the new pipe development team do all of the hard work, right?
That's the main benefit. So self-issue S users gain from all of this fantastic work that this team of devs do.
And they can just pull in that, we can just pull in that capability and make use of it.
And that's actually really important especially for new pipe because Google changes the back end.
The code needs updating to support it. It's a lot of work to keep the two in sync.
It's a lot of effort to keep the two in sync.
And so it means that you don't have to duplicate all of that effort. That's really important.
So that shared functionality is really nice.
On the other hand, there's no compromise in terms of the look and feel.
The Android app looks like an Android app. It's self-issue out, looks like a self-issue OS app.
And users get what they expect on the device that they're using.
And in this practice I mentioned before, the performance of compiling down the code to a native executable is actually
End up being really performance and surprisingly good in practice.
So it actually works really well. There is some work to be done to implement the shims, but in general it actually works very nicely.
So those are the benefits. There are of course drawbacks and the biggest drawback is that you have to have this really clear functional
separation between the back end code and the front end code.
And you pipe happens to have that. And when I started this project, I didn't know that it was just a really lucky
care instance that it worked out that way. If it hadn't done it, it would have been a bit of a disaster.
And not all Android apps, although it is good practice, are separated in this way.
They often have joined tighter coupling between the two.
So it only really works when you have that sort of situation.
Another downside is that growl VM that does the compilation,
will only compile to R64, so that is 64 bit ARM and X86 binary code.
It will not specifically compile down to ARM 7HL 32 bit ARM code.
And there are a lot of selfish OS devices out there, which are still running on 32 bit ARM chips.
So unfortunately, as it stands, it's just not possible to apply this to 32 bit devices, which of course is very sad.
It's a really unfortunate. It would be really nice to have the app work on those devices too.
And then finally, there were some small patches needed to the extractor code to get it to work,
because of the fact that, for example, reflection doesn't work with JARP-Pro VM.
And thick, who did the selling to coffee implementation, actually push some changes upstream to
new pipe, the new pipe repository, which the team accepted, and that worked really well.
So that was really nice to see. So that was a good symbiosis.
Yeah, so in general, it's a process that seems to work really well,
but when you have this good separation.
All right, so I'm just going to hand back now to Fabio to wrap things up.
It's actually mine now.
So yeah, thanks David for implementing the Save Usure as a port of the app,
that was very interesting to hear.
So to give a few information to conclude the talk, I would like to let you know that
new pipe is not only a project, now it's also an association.
And this association doesn't only want to promote new pipe itself,
but it actually wants to promote more libre free access to media,
like software that lets you access media more freely,
such as applications, protocols, and so on.
So five with only supported new pipe actually, because that's our baby, let's say,
but we want to be a bit more helpful to the community in general.
So if you have a project that has something to do with free media,
and you think you would need some help, for example,
in handling bureaucracy hiring or donations,
you feel free to ask us, will I have already doing this for a new pipe?
And if you want to join the association or contribute to new pipe,
contribute code to new pipe for each free to join us.
With this, I want to thank all of the contributors that have worked on new pipe throughout this last ten years,
because new pipe is ten years old since a couple of months or something.
Like a hardful, deep, hard thank you to all of the contributors.
And yeah, you can find further links to our talk, to like the slides,
and to everything we talked about in this talk by scanning the QR code on the right.
And I think David wants to thank, again, the person that created all of the nice graphics.
Yeah, thank you. Thank you, Fabier. Great.
So thanks to Jerome Haymans, who did these amazing graphics.
And Jerome actually, I think, live, live, drew the Devrim last year.
So if you have a look on the message on, you can find all of the talks from last year, all of these in this format.
So really, thanks. Thanks so much.
Yeah, we're happy to take questions now.
Thank you.
Yeah, thank you.
Would this also find a good starting point for, for example, going to touch our postmark to last?
So regarding, yeah, so, can this approach be used for pointing your pipe to also wound the touch or other Linux devices?
So there are already a couple of ports of new pipe to Linux.
One of them uses an abstraction layer that was built specifically around new pipe, which basically takes all of the android API calls and turns them into Linux API calls.
It doesn't work very well, but it kind of works.
And this is just something that I use created and they put it on flat hub.
So you can already download it for going to touch, I think.
But don't expect to get a good experience out of it yet.
And then another thing you can do is, well, you can just run it through way through it, or I don't remember the name, but like the android emulation software.
We can talk about this outside, if you want.
Yeah.
How does the new pipe association consider the extraction of certain media from big platforms?
And by as does that be considered at all?
Whether new pipe association has considered the extraction of social media like Twitter, like Twitter.
Okay, I see. So I think to this extent, there were other projects doing that in the past, but they have all been DMCA at some point or something like that.
No, not DMCA, so the season disease did.
Yeah, whether new pipe has had trouble with YouTube.
Well, not really so far, which is fortunate, I don't know.
So I mean, we have problems every say once a year when YouTube breaks changes the way the website works and then the up stops working and we have to push a hot fix release just like three days ago.
But yeah, generally there's nothing too bad happening on the YouTube site where it was, for example, if YouTube were to start pushing for DRM because it's illegal to break DRM.
And but so far, yeah, technically what we do is legal, so the hasn't haven't been backlash.
Yeah.
So the question was when will it work on landscape, it's a good question.
So I guess you're not a disinterested party here, right?
Because you developed the Voila ports of selfish rest for the Voila tablet, right?
Yeah, there's a really good question. Not this week is the answer.
But you have submitted an issue, I think, right? Someone submitted an issue, at least.
So say the plan is for it to happen eventually, yeah.
I mean, this week.
Right.
The person, they're in the back.
Yeah.
So I have a question on, you know, Charles, but I'm just out of who you're seeing.
So I think, I think then, so I guess there were two big reasons.
Oh, sorry, I big one, thank you, yes.
So the question was there are multiple Java version machines that are able to run.
So why choose Growl VM specifically for this?
And that's a really good question.
I think there were two reasons. The first one was that thick, who I mentioned, had already
developed this process that worked on selfish OS to build these binaries down into
Executals books that would run and link against C++ really easily.
So that was, that was main reason was that infrastructure was already there.
And the second reason I suppose was that it feels, it feels more comfortable to have a native
resource.
It's quite a heavy addition to the system where it's actually Growl VM is quite light, relatively
speaking, there's my experience.
So you actually get quite a nice nimble.
They're actually really big binaries, but apart from that, they're quite nice and
contained.
So I felt that was nice.
Yeah.
I think one of the last questions in the middle of the time.
Yeah.
Now, how were you supposed to develop a certificate situation?
What do you get to develop a certificate for Android?
Yeah.
We are definitely aware of that problem.
So as far as we understand, the terms of service, of the Goething Program,
are not Google's terms of service.
So technically, they might be different and they might allow us to get very
fine, but it's still very bad that we are aware of the problem considering possibilities.
I don't really have a better answer right now, but yeah.
Thank you everyone for being here.
We'll be outside and also to other members of the team.
We'll be outside if you want to speak with us.
And yeah.
Thank you for your time.
Thank you.