All right. Hello, everyone. I'm Martin and I'm going to talk a little bit about my tool chain,
and I'll be in Mnjw. So first off for a little context, what is Mnjw? It's a weird acronym that stands for
minimalistic new for Windows. Most of you may hear Mnjw, probably think of GCC on Windows or GCC
for Windows, but a traditional Mnjw tool chain usually consists of GCC. It has been new to us,
and you have the Mnjw component. Where the Mnjw component is what provides the Windows headers
and libraries. So this is a standalone re-implementation of the whole Windows SDK, which makes that
freely redistributable, which makes things much much easier and much more open source use.
And these days the Mnjw component is essentially the Mnjw-W64 fork. And also the point out,
Mnjw is building totally normal native Windows executables. It's not about emulating
in POSIX, which is what Sigwin does. So what does Lnvm do in this context? When you're targeting
Windows, Lnvm can emulate both Mnjw and GCC depending on what you pass as the target argument.
So with Lnvm, you can emulate what GCC does there. So then what is Lnvm, Mnjw?
Well, it's a Nnjw tool chain where we swept out all of the GCC components to their Lnvm counterparts.
So instead of the GCC, you can have Clang. Instead of the GNU linker, you can have Lnvd. Instead of
the GCC, you can have compiler RT and Libanwind. Instead of LibsDDC++, you have Libs4++, and then you have
LnvsDbugger. So it's a complete tool chain set up from scratch only using the Lnvm component.
It's not all the GNU parts. And if you're contemplating this today, it doesn't sound too hard or
weird to do. But back many years ago, when I started doing this, it wasn't quite a straightforward.
Then for the Y, why would you do this? Well, if you want the target arm, now I'm 64,
you might want to do this. This tool chain also gave its support for the Microsoft BDB debug format.
It gives you sanitizers, which is nice to have. It gives you C++ 11 threads without using the
WinP threads library. It gives you Windows native TLS. Instead of the emulated TLS, you get working
weeks, symbols. And for other parts, you get one single tool chain that lets you cross compile
and target all of the four architectures that Windows runs at the moment instead of having to have
four different tool chains for this. And then from the developer perspective, it's built on a
modern code base, which is quite easy to work on, easy to fix new things on. These were the
selling points of this tool chain in say 2019. To be completely honest, though, GCC is catching up
on many of these fronts. So for example, for arm 64 support, there is support that in progress.
It's not completely done, I'm sure, but it's getting there. There is work on PDB. There is,
I have seen some patches somewhere for sanitizers. I don't know if they're being upstream,
but they do exist in some foreign somewhere. There is C++ 11 threads without WinP threads,
since a couple of releases. There's been patches for native TLS on x86, merged a couple of months ago,
so I'm, but you don't get the single tool chain targeting on four architectures.
So the goal of the LVM in GW2 chain is that you, it's supposed to be dropping compatible
for any existing project that is ready to be built in in GW2 context. Both that you can have
the source is supposed to work as it used to, but also making it compatible with the build tools.
And the build system, whatever they expect from in GW2, kind of tool chain.
So I'm going to talk about how this project got started, which is the more interesting part.
So I have a background in multimedia, in FFNPEG and BLC and radar project. I have been
porting FFNPEG to weird mobile platforms since 2006, soon been on Windows CE and these kind of things.
And in 2014, BLC had the desire to run on Windows Phone.
And the problem is that BLC is very heavily tied to being built with the NW2 chains,
we'll see for Windows. They're building over 103rd party libraries, most of them at the time
built with AutoCone of AutoMake and so on, and you primarily cross-compiled this from
from Unix. And in upstream GCC, and there was no support for anything else than X86,
if you want to target Windows. They tried to have a GCC maintainer involved in this,
and there were some patches, but it didn't really go anywhere. So I tried to help them out,
and I came up with a proof concept. So you can wrap them as VCCL.exe compiler, wrap it in a shell strip,
a GCC style command by an argument, and then invoke CL.exe internally. I gave them a proof
of a concept that worked for the most trivial library, and they took it on, and made it actually
work for them. So they did ship Windows, VSE for Windows Phone. With this, it was most certainly
not pleasurable to use, but it was possible to build things with it. So they still had some
desire. They wanted to have a proper Mingi W2 chain to target arm, and they got,
got aware of the fact that LVM did support generating code for Windows on arm, and LVM did support
targeting Mingi W. If you had the next existing Mingi W2 chain. But they were missing the linker,
they were missing building all of the runtime libraries, they were building the import libraries
that you need, and tying it all together. So they had involved a guy called Martin Malone in
around 2015, and I got involved in this also in 2016. Then as a complete separate track, in 2017,
there were lots of rumors about Windows coming to arm 64. If you downloaded the Windows SDK from
Microsoft, it did show that they were import libraries for arm 64, and it had a handful of
executables that you can't run anywhere because this thing doesn't exist. But then people from the
wine project, a guy called Andrey, had the support to wine for running arm 64 binaries with wine
on Linux on arm 64. So from the dozen or so of binaries that were out there in the SDK,
most of them crashed. But maybe one or two of them actually did successfully do something.
So I tried this out, tried to play with it, and tried to look at what's failing in the ones
that are crashing, whether crashing in printf. And looking a bit more on it, I realized that printf
seems to be using any function with a variable argument calling convention, it's using a different
calling convention than the regular one on Linux. So to solve this issue, I realized that well,
first I did a number of misguided attempts, can we hack around this with some inline assembly,
or whatever you can't really. You do need to have compiler support to implement the different
calling convention. And as I happened to have LVM checked out, and was somewhat familiar with it,
tried to have a look at how hard could it be. Well, and after noon of gripping around,
gets you somewhere. So I got some of the calls to printf working. So, and take this on word a
couple of months, I had mostly working wine environments where you can run executables,
but Windows are norm 64, which does not exist, except for these dozen or so executables that
happened to live in the Windows SDK. So we're having the test environment. There was some initial
commits to LVM and client, but people from Qualcomm for generating the Qualcomm 64 objects files.
You didn't have anything of the rest of the ecosystem, but you could generate like trivial
objects files. And the Windows SDK contained the import libraries, but the MSVC tool chain itself
didn't have a compiler. It didn't have the rest of the MSVC C run time startup libraries, and so on,
that you need to actually do something. But you had like some parts of it, so I tried to play around
with it, and tried to link things with any support to the LLD linker for this. As it turned out,
the MSVC link.exe linker that already existed actually had support for the Qualcomm 64 objects file format.
So with that, you could kind of reverse engineer and figure out what it's supposed to be doing
with these relocations and implementing the same thing in LLD. So with this, this progress
a bit more and managed to produce trivial, small executables that I could run in my emulator.
And I followed this into the overall effort of LV together with the LW, so I had to support
this also to the MNJWSDK. And that's a small side quest added support for targeting the
universal C run time as well in MNJW. So by the end of 2017, I had got a seemingly working tool chain
with this. Both figured out how to get the Liban wine C++, like C++ and so on, to properly
composite of us in general for X86 and so on, and also have a mostly working on 64 tool chain
targeting the emulator. And the main GW linker interface for LLD by Martell was finally merged,
which meant that you could actually build this tool chain without any extra patches on top of
LLDM, which is a key thing. So I didn't really set out to maintain my own tool chain distribution.
That's a lot of work and I don't want to do that. But building all of this was kind of non-trivial.
So I needed to have a reference example of how we are supposed to do it. And initially this was
just a branch on video lens, Dr. Billbot's repo, but at the end of 2017, I graduated
to split it up to a separate repo on GitHub called LLDM in GW. So I had mostly working to watch
in targeting ARM 64, but only running in an emulator. Well, I contacted my friends in video
and asking them, can they talk to Microsoft and get me something? And they managed to get me a
prototype ARM 64 device. I could try out my existing tool chain on the real OS,
prototype device, did it work? Of course not. Who am I kidding? But the thing is it wasn't really
far off, it was actually quite close. So it turns out that you get that dialogue if it refuses to run
your binary. It refuses to run your binary if you don't have the dynamic base flagset. And that's
just one single bit to say that, yes, I am okay with the OS loading me at a different address.
Set that it takes a little while to figure out, but once you do it, it's like a couple of lines
that code to fix. Also, simple binaries worked, bigger ones crashed, it turns out that on Windows,
when you're allocating stack, you need to probe it. So if you have a function that say allocates
12 kilobytes of stack, you need to touch it, one page at a time, to let the OS actually allocate the
page for you. You don't notice this when you're running on wine on Linux, but when you're running on
the real thing, you do. So then I learned what stack probeing is and have the implement that. And then
it also turned out that set jump was broken, because I didn't have the proper SEH on winding
info, which they require. So instead, I did a trivial rehabilitation of these just by dumping
registers, restoring registers. So with this, with the real device at hand in hand, I made
a working build of VLC with lots of hack in the process, of course, in a couple of weeks.
Then we fast forward a couple of months, and Microsoft had their build conference, where they were
unveiling Windows for ARM 64 for developers, and they were showcasing how do you port
apps to it? You do it by recompiling with the latest Visual Studio, but they also showed something else.
The market thinking, the open VPN driver, it looks simple, I just saw like 5 C file, I saw like
5 header file, that sounds like really simple. What about a more complicated app? So with each
star to VLC, VLC, I don't know if you know is one of the most commonly used software Windows,
it's a media player, it's open source, and it's extremely popular, we know that based on our
usage. So we restart the VLC, and we were like, hey, are you interested in porting your app,
your desktop app, ARM 64? And so just like these tools, the main developer who we will see,
he was like, yes, we are in, we are keen, let's try this out. And so we shipped them a couple of
devices, right? We loaded them up with Windows, we gave them preview tools, and in shocking
the short order of time, they had this up and running. In fact, we have folks from VLC in the
room, I don't know if you go here, maybe there is, there's you go, so if you, he's from VLC,
so if you do actually have questions on what they did, you should reach out to him.
Again, just to be labelled at the point, when I asked John that, he's like, how much code
did he have to change? His answer? Zero. He changed, zero lines of code. His biggest problem was,
again, we'll see, has his own home-brew way of building things. And so, ingesting the latest
Visual Studio Preview compiler into his build system, that was his biggest challenge.
Changing code, zero. He didn't have to change a single line of code. Everything just worked.
Yeah, so that wasn't quite true.
But, then again, sure, Shampa, this didn't change any lines of code. I did. And quite a lot.
I had like a stack of 60 to 70 patches on top of VLC to get this building. But, the point is,
I wasn't using MSVC. And, I mean, the point is that I rather rebuild a complete tool
change from stretch than trying to adapt the automated auto-convill system to use MSVC.
Anyway, going on from there, I mean, that was how things got started. After that, it's just been
a bunch of years of cleaning things up and so on. I'm just going to mention a couple of all fun hacks
that we did during the startup. So, when I started doing this, we didn't, LLVM didn't have
a tool for treating executables or objects files for PECoff. So, to fix this, I built new binutals
and to there, strip-util. But, it doesn't recognize R64 binaries. So, what I did, I did a small
wrapper that changes the machine flag saying that, yes, this is an X86 executable. Please strip it
for me and then I'll change it back to the real one. It works. I mean, a couple months later,
yeah, I did implement the real thing in LLVM, but you can get quite far with hacks. For the resource
compiler, it didn't have integrated pre-processing, which meant, again, a shell script wrapper,
that integrate parsing the command line options during the pre-processing and so on.
What's a hacks at the beginning, much less hacks these days, to these days, it's quite
straightforward. For other parts, there's been quite a lot of work on implementing quite
minted W specific features. Of them, the author import features, probably the biggest one,
which means that you can reference data symbols in a different DLL without explicitly
declaring them as DLL import. And this is maybe the most minted W specific feature that you have,
which before having this, this was the biggest worker for porting or building lots of existing
minted W projects with this tool chain. At the start, I was wondering, shouldn't you just
advocate for people actually adding DLL import for everything? But it turns out that if you want to
link the simple first standard library as a DLL with the titanium C++ API that you use from
in GW, then you do need to have this feature. So the project is matured. We've gotten rid of
most of the hacks in the build process, but these days, since we've added support for ARM 64EC,
which is our fifth architecture, we had to add back some hacks around that. We're not initially
only provided cross compilers from Linux and MacOS, but these days also running on Windows.
So it's works for any case anywhere. And to prove the concept, the M62 package manager
have also added client 64 and client ARM 64 environment, which is in environment where you have
playing NLLD as the system tool chain. Exactly like mine. So these days, I do releases by building
everything on the free GitHub Actions. I do nightly builds every day, every night. With the latest LLVM,
like this Ninja W, if you want to test things out, you can always grab that. And I do release every
time there's a new release of LLVM, which is essentially every other week almost all year round.
All right, so the question is what we use for the C library. So one of the cornerstones
in how Ninja W works is that we are building things to use existing Microsoft C runtime on Windows,
which is also a cornerstone in how you get things GPL compatible because that's like
even though it's a close source library, it shipped as part of the OS. So GPL is okay with linking
to it. So that's the main design in Ninja W that we're linking against the existing one.
Yeah? Do you need that reach of consoles or do you use convenience?
So the question is whether I build Ninja W from source or if I use pre-built. I build everything from source
because if you want to switch, if you want to target something as like ARM 64 or whatever,
you need to have it, you need to build everything from scratch. You can't reuse anything pre-built.
And that's also part of the porting process. So instead of just going to say I want to build things
for ARM 64, you try to build things with a new tool chain for X86, where things are supposed to
be working in the same way. And then once you have that working, then you switch over to a different
architecture. Okay. And the back? The question is why is it still supported?
Yes, I do my local nightly build the same testing with wine on both X86 and ARM 64.
And the ARM 64 support in wine used to be quite hacking. There were a lot of quite fundamental
issues around that, but all those fundamental issues have been solved in the last couple of years.
So also things are really good these days. One final question?
So the question is, if need the WS permissible license, and yes, it's mostly permissible license,
I would say, so some I can't really say for sure, but most of the components have something like
a public domain or similar, there's an all over them.