Hello everyone, welcome. My name is Peter, I'm a computer scientist and software engineer from
Norway. You have my handles and my website, I'm a master-down handle over on the right
side. If you want to get in touch, that's my nickname on pretty much all of the various
platforms where you can find me. What I'm going to talk about today is NIM and C and I have
subtitled the talk, reaching the stars by standing on the shoulders of giants. So there are
a lot of languages created every day and just a quick show of hands, like how many here
have tried their hand at making their own language? Okay, that's decent decent chunk.
So you can create your own language and you can have all of these nice features, but as soon
as you get one user, you get tricky questions like, how do I calculate and MD5 somewhere,
hopefully I have shot 256 on? How do I download files from the internet? How do I do this?
How do I do that? At that point, you need to start looking into creating libraries and obviously
you don't want to create them all yourself. A lot of languages in sort of to bootstrap
the environment of libraries and packages will allow you to use libraries from a different
language. So for example, closure and F-sharp will allow you to use Java and C-sharp packages
because they're built on the respective virtual machines. TypeScript, closure script, they
have both compile or transpiled down to JavaScript, so they will allow you to use JavaScript
functions and SIG and NIN and a lot of other including probably all of these will also allow
you to use C language or C libraries. But just quickly before we get going, what is NIN?
We got a small introduction, but NIN is a compile language, creates tiny binaries that you can
just send through an email and the other person can run it in their system. It's a statically
type language, so all type information has to be known on compile time, but it has type inference,
so for example, some in count, NIN knows that those are integers, same for the line and the
force statement, the lines iterator, only ever returns strings, so line has to be a string.
It also has a lot of like clever types, so you can have for example an ID stored as a distinct
integer, so you can't take an ID and time it by two, and you can write a procedure that only
takes IDs and won't accept other integers, even though there are stored as an integer.
It also importantly for this project, it has a flexible macro system. So NIN macros take a piece of code
and it outputs a piece of code. So you get an abstract syntax tree,
parsed of the code you gave it, and you can return a new tree doing whatever you want the code to do.
And this is a quote, or I thought it was a quote, by the creator of NIN, the speed of C, E
of Python, and the flexibility of Pearl. I've talked to him since and he says C never said it.
And now I've been quoted as saying that, so yeah. But yeah, for this project, it's important to know
that NIN sort of compiles to C, so as you saw, NIN will create C code, that C code goes into the C compiler
and outcomes binary. This of course means that we get very easy foreign function interface,
the only thing that NIN needs to do to allow you to call the C function is just to put that C function
into the C file it creates. NIN also has hookable automatic memory management. So it has a GC,
you can enable a GC, but by default it uses an automatic memory management system, which isn't technically
a GC, it's not stopped the world, it's not a lot of those issues and it's hookable. So if you
import a C library, you can say, well okay, when I create this type called this function and when
the type goes out of scope and it's not reachable by anything anymore, call this structure function.
So that way we can wrap C libraries in a way that makes them super easy to use in our NIN project
and feels more like a NIN library. But I said compiles to C, he said transpiles to C,
this is an example of some more horrible C code generated by NIN. This is just the inner
part of this loop. NIN uses C more as an intermediate representation. You're not intended to look
at this code. There was a question about debugging, it will generate line directives and stuff
like that. So if you open it in GDB, you will actually see the NIN code and I'll see code, you can
step through the NIN code. But if you want, you can disable that and step through the C code instead.
I'm like if this horrible mess, it's doable. So yeah, it's more, it creates efficient C code
that the C compiler is able to optimize very well, but it's not meant for human consumption.
So the foundations are for doing FFI and NIN. We have a logic that C file just contains
a single function. In our calculated.NIN file, we have compile logic.c. So we need to tell NIN
how to find the C code. So it could be a dynamic library, it could be a static library. Or in this
case, we just say compile list file, it will be available during runtime. Then we'd say, okay,
you have a function add to integers and it is imported from C. So we just tell NIN trust me,
there's a function called add to integers. Don't worry about where it comes from, it will be available.
And then we can just call that in our program. We don't need actually the one. It's main module
thing, but this is quite a simple C file. The libraries that you want to use from C are probably
a bit more complex than this. So writing all of these gets super tedious, super fast. And if you
spend a lot of time, you write your nice wrapper, a new version comes out and they've suddenly
like added a field to a type here or change the function or stuff like that. Then you need to remember
go over all the types and put everything back in yours. You need to do the whole manual process
all over again. And that is where we try to automate stuff. So this projects that I've created is
called tooth arc, it basically creates all of those definitions automatically from C header files.
And this isn't a new thing. NIN has had C wrappers since pretty much day one.
The first one was created by the creator of NIN himself and that tries to read the header files itself
and understand everything in output NIN code. Unfortunately, C is notoriously hard to parse and
understand and the whole pre-processed sort of thing doesn't exactly make it easier.
So it sort of fails in a couple things. Then another project came along called I think was named
drop or yeah, which is the tree-citter algorithm, which is great for parsing C syntax,
but not for parsing C semantics. So it's meant for like syntax highlighting and understanding
like the structure of C, but it doesn't really tell you much about what's actually going on
in the code, like what's the type as this thing. So it's still only to understand like the
semantics of C when you're using tree-citter. My approach is to use live-clank. So live-clank
the C language fontan or LVM, because what's better at understanding C, then it's E compiler.
Luckily for me, playing also has a library version dynamic library that you can just import
and now all of a sudden you have a C compiler within your own program.
So the way it works, you import the library. We still need to tell
them how to find the library. In this case, I pass a link or flag to just add this to my
and aesthetic library to my build. And then I say import from C, I give it the path where my header
files live, if they are installed system-wide, you don't need this. Then you give it the header
file you want, and then I've created a bunch of like small niceties, small rules that you can apply
to in this case rename the MAPM type to MAPM internal. So MAPM is just an arbitrary position
math library from C. So what happens when you do this, the NIM file goes into the NIM compiler,
the NIM compiler calls out to the food archive library, but the food archive library is basically
just one big macro. So everything runs on compile time. And on compile time, we can't pull in dynamic
libraries because the compiler won't let you do that. So it has a helper program called up there.
So a food archive calls the upper program, which reads the C header files, and then the upper
program spits out basically a big JSON file containing all the functions, all the types, all of that,
which then foods art parses the JSON code and generates NIM code. Then the NIM compiler only has a bunch
of NIM code. So it compiles the NIM code to C, and the C compiler comes in, pulls in the same C sources,
and you don't need to compile with a client. So you can you can do this step with client,
and that's step with whichever C compiler you want. And then that spits out a binary, which then
can load it in NIM file B or whatever you want. So the code, the import C statement from before,
on computer compile time, it basically gets turned into this. This is the output from food art,
just a bunch of definitions. And all of a sudden you can just import the library where we put
that code, and you can use it just like you would in C. But we're not using NIM to write C code.
I don't want to write C style code in my NIM project. So with a couple of tweaks, wrapping things
in nice NIM features, adding the automatic memory management stuff, and also in this case,
hooking into the error function in the library to pull out. It has a stupid thing, but
if it throws an error, it quits the program. It calls the quit function. It's quite annoying.
So what I did, I hooked into that function, turned that into an exception, and then you end up with
code looking like this. And this looks like NIM code. To here, like this, does a call to init
map them feeds this into the map function. When this is no longer needed, then it will feed that
object automatically by calling the C function. And all of that just works as you would want.
The extra wrapping here is about 500 lines of code. A lot of that is just duplication,
because I need to do the error correction for every function, or the error handling for every function.
But the good thing is that the whole C to NIM conversion is done automatically. That's handled
for me. So I don't need to sort of ever revisit that part. The only thing I need to care about
is my sort of NIMification of the project. And if the map and library work to update, which
it probably won't, because the author unfortunately passed away, and it was one one-man project.
But if it updates, then I can run the code, I can compile again, and it says, well,
this function has now changed, or the type, one of the type size, and then we'll just know that.
It won't try to allocate a smaller size for an object or do something, don't like that.
But of course, this has a couple of limitations. The good things first, handles pretty much any
code. Macros, you can do like thoughts slash configure on the project and get the header files
and everything in order. You can do C defines and all of that. So all of that pretty much just works.
I'm wrapped stuff like GDK, I've wrapped X-Line Wies, I've wrapped. I used this to wrap a project
that wasn't intended to be used as a library to create a dynamic library to link into it.
And it all just works. It's also a very automated process. So, as you saw, you just do the
import C. It just works. A lot of the earlier solutions would sort of, if it didn't understand
something it failed. Truthart has a lot cleaner fallbacks. If you want to override something,
if you want to improve on the wrapping, you can do that much more easily.
Sort of the middle things. It's agnostic to the C linking. You need to specify how these things
are actually getting into the final binary, which is good because it's flexible, but you also don't
get any help from Fuitart for doing that stuff. It's also recently portable to C++.
Leb playing reads C++ code just fine, but I haven't had time to do the conversion yet.
So, it doesn't currently support C++, but there's no technical reason for that.
Apart from time being finite. It's then like rate on function style macros. For those of you who know,
the pre-processer and C is basically just like a copy-paste machine.
So, a function style macros doesn't need to be complete. You can have like half a statement in
a macro and then the other half comes when you compile. So, they don't have type information. They don't have
yeah. Even Leb playing doesn't really know how to deal with those. If anyone has any good ideas
for us all of this, I would be open to ideas. I know C does something very similar in how they
are wrapped. C code. I tried to look at their solution for this and it's still just basically guessing.
And it only wraps the C code. As we saw, we only get like the C output and then we have to
nameify it ourselves. Yeah. And that's it.
So, yeah, have three minutes for questions? Yeah, I have three minutes.
If C bugger goes a so much trouble, maybe I'll try to see after. So, the question is if C macros
calls so much trouble, why not wait until or why not pre-process the C code and then
read the output? The problem is that a lot of libraries will use C functions down macros to apply
or to give you default arguments. So, a function that takes a lot of arguments, they might have a C
macro that you're supposed to call, which fills out extra arguments. To still need some of the
functions style macros in your output. Yep.
Finding a library, a macro component. Yeah, so I think the question is with all of these pre-processors
stuff. Yes, yes. So, when you wrap this code, you need to give it all the same defines that you
would for building the binary. And yeah.
So, my question, do you have questions? Yeah.
Would it be more correct than, say, more safe to analyze the library directly as we all get
the same ones, instead of trying to cross the other five glitches by the commission and I
I'm not sure if I
times up. If you, yeah, I'll be, I'll be down here soon.