Okay, everybody, my name is Konovat.
I work on the Swift team at Apple, I focus on primarily concurrency in server use cases.
You may remember me from my Java life, in which I worked on Akka or the reactive streams
spec.
And yeah, it's great to be back and working a little bit with the JVM.
So I will introduce Swift a little bit, a quick show of guys who is aware of Swift,
who has used Swift.
Okay, not so bad.
So Swift is an open source language, works in many platforms, anywhere from Linux, Windows,
to embedded systems, in firmware, anything between also wasm, which I don't have on
this line, slide, but I just remembered.
And it's a natively compiled language, it has an explicit goal to feel familiar to people
coming from other ecosystems, including Java.
It has a memory safe by default, and has pretty cool features like concurrency safety, verified
that compile plan as well.
Since day one, Swift got an amazing C-eaten objective, C interrupt story, which is how it
managed to slowly replace objective C on Apple platforms.
And we since then added an amazing C++ interrupt, which is really cool, and today we're
adding Java to that list.
So why do we even care about this?
But I'm merely because of memory safety, so why do we care about memory safety?
Well, memory safety vulnerabilities of a number one security vulnerability.
So if we want to reduce those, it would be great to adopt memory safe language as across
the whole stack.
And while Java is a good solution to that, you know, Java's memory safe, it's fine,
but Java programs very often end up calling into native code at some point, and when
you lose the safety.
So it's our opinion that whenever you're writing native code, Swift should be the tool
you should be using for that.
So in order to facilitate that, we need to be supporting incremental adoption of Swift,
and what also means, infertility with many languages, and making it both easy and efficient
to do that.
I call it dark early on in the main track, how they told me just about that.
So we decade of moving more and more of an ecosystem into Swift and memory safety.
So, you know, that was in the past already, but you can check the video.
And where do we care about this?
A great example is, of course, servers.
So there's a lot of Java services out there, and we're increasing number of native code
in libraries written in Swift, and we would like to be able to call them both efficiently
and nicely.
The other situation is that it's interesting for Java and native intro up is Android.
The common thing that happens is people want to have a shared library that is the core
of their apps, and maybe interesting algorithms, and they want to use it in the server,
on vermobile apps, and anywhere already.
So we think again, Swift is a good language to do that.
Golds and directions here, this is a small distinction on the primarily written in some language,
where you can be primarily written in Swift, and maybe you need to call it Java library.
And we do support that.
We have some Swift macros and JNI wrappers, and you can kind of do that, but that's not
what this talk is about.
The second situation is when you have a primarily written in Java application, and you
want to call it native code as the previous talk, gracefully explained with the new APIs
that we just got with FFM.
So this is what I will be talking about today.
So Java and Swift, of course, have many differences, but you'll be surprised to find that
there are actually a bit similar when you look deeper.
Of course, Java is all about classes and objects, but it has primitives, and Valhalla
is bringing value types to the JVM as well.
Swift on the other hand has classes, but it's very much focused on rich value types.
So again, something that the slogan of Valhalla that calls like a class works like an
int, very much applies to Swift's value types.
Of course, both are memory managed, so we have a GC here in the JVM and automatic reference
counting.
Yes, but it is a difference, but for the most part of a developer's day, it's a very similar
user experience.
And the generics, of course, have a slight difference as well, so the JVM is type of
type of area at one time, and Swift has generics, but with fully reified types, that
will be an interesting thing to dig into.
There's smaller differences as well, so Swift has errors with your values versus exceptions
with stack traces and the threading model, versus thread versus a single weight in actors,
a bit different, but again, something we can work with.
So a typical application, you will have some Swift sources, but either are from a library,
or it's your Swift sources, and you will have some Java sources, but we'll want to use
this native code.
So we will build a dynamic library, depending on the platform you want to run it on, and
here's the new things.
We will have a tool that will generate Java sources that will make calling this native
library easy.
This is where we will be using the FFMAPIs for function interfaces to call down to this dynamic
library.
We will also have to generate a bit of Swift code, which is a bit unfortunate because
with a plain C-style language, you wouldn't need to do that, and I would explain in depth
why we have to do this bridging layer.
At the end of the day, you put it in a jar, or in your ship for dynamic library separately,
and you can just call these functions.
So we do have a tool that is called JXTrack Swift.
The name is very much inspired by the JXTrackStool, which is something from the JDK that
takes C header files and turns it into these Java wrappers that you can use to call
these C functions.
So our tool is very much inspired on that, looks more or less like this, where instead
of accepting C header files, we take Swift sources, and again, generate the Java code
that it makes it possible and easy to call into these native functions.
The tool is split up in a few phases, or some analysis phase, so Swift is a very explicit
language.
For example, you can have extensions on types located in other modules or sources, so we
have to do a little bit of analysis to figure out what's type has, what's which method.
And now later on, of course, this is the source generation step.
Today we use the foreign function in memory APIs.
We could have a source generation step that would use the legacy, you know, JNI APIs for
platforms that just don't have the FAM yet.
And we do generate these Swift function as well.
And so how does it look like when we extract some source?
Let's say we want to extract a Swift class.
Swift class is imported as this, we're about and we implement this with KeepObject.
We use this primarily to know that Aha, this object has to be referenced counted because
we will try to do the right thing on the Java side as well as we import these types.
When we extract a struct, we implement this with value, which again is a signal for us
but okay, we will be copying this Swift value around rather than reference counting.
Next we will be generating all of this for in foreign function descriptors.
The previous talk very much explained, so we generate those, so we don't have to write,
you know, the function signatures by your hand, by hand.
And we finally generate the actual methods that Java developers will call, which kind
of mirror the native shapes of the methods will be calling.
An interesting thing here is, at the bottom one, you can see that these types can have
other imported types, so I'm passing a my Swift struct in the Java side of things.
And yes, this will be wrapping a Swift struct and we can just call a collection of Swift libraries
without doing weird dances, we can just use them more and it all fits together.
So I did say that we are using Swift sources, I feel that I have to explain the
fact that for Swift, it is typical to use Swift sources as packet distribution mechanism.
So when you publish a package, it's a get repository, you get for sources, so we always
have resources available.
But if you don't have the sources available, for example, you're using an operating system
provided library or just close source code, the same tool will actually work the same way
because Swift does have something alike header files, which is the Swift interface files,
which are basically Swift source files, but except having extra bodies and implementations,
it just could out without the bodies.
So the same tool works the same way with those source or operating system provided library
libraries.
Okay, let's dive in and we will start from an interesting point which is initializers
because they will lead us to some interesting discoveries I want to talk about today.
So let's say I want to extract from this Swift source file, so it's an initializer, it takes
a length and it takes a capacity, the capacity is another value type and I want to import
that into Java.
So of course we generate the function descriptor as you can see here, it will return a Swift
pointer which is the initialized object and we accept the length and we accept the capacity
as a pointer to that struct and oh wait what's that?
We didn't see that in the Swift signature, it's another parameter here that is the Swift type.
So somehow we will have to find out at runtime the Swift type metadata that we have to pass
to this native function initializer to be able to call it.
So how can we find that?
Luckily we can just ask Swift to give us the type.
So on the Java side we generate a little call that will basically down call into Swift using
a well known function name and on the Swift side it's very simple because we just write
the type and we just return it.
I'll talk about the details of these functions a bit later, but basically I'm just returning
the value out of Swift back into Java.
This should feel familiar because on the JVM we have a class object and the type metadata
is exactly what it's.
It's very equivalent of a class object, it describes, hey this type is this and that.
The type is immortal so I don't have to worry about retaining it or anything like that,
I can just return it like that, let's find it.
We've installed that on the Java side and we're ready to call initialize it directly,
very cool.
This way I can literally do a down call and I get back a Swift instance but I now have
a reference to in the Java side.
What about value types which is I think we've spent most of the talk because they're
more interesting and value types are interesting because of course on the Swift side
they will be stack allocated, that's the reason you want to have value types, they can be
stack allocated and they're nice and efficient to pass around.
So if we were to just return the same way as we did with the class, point to that would
be pointing to a stack which could very easily be not valid anymore as I use it in Java.
So instead what we do is we create a small allocation of your exact right size that the
Swift initializer will fill in with the data of the struct.
Now the question becomes, well I don't know the size of the struct in Java, how do I find
out?
And we have an analysis phase so maybe I can do that, maybe I can use the analysis to
just do the math and figure out the size of the struct.
We can do that, if that's an integer that's two eight bytes then okay I know that my Swift
class is a class, it's a pointer again eight bytes and then it's a struct I have to look
at that struct and recursively keep descending to figure out the size of that and eventually
I can find that out.
I could also ask the Swift content to just tell me, so the Swift runtime, the Swift API called
memory layout, you can give it a type and ask these kinds of questions so you get the
size, the stride, the stride is just the size plus any necessary padding to align it properly.
And so we could ask Swift as well.
But before we do that I want to explain why we have to do that.
So what is the size of this struct?
It's the same struct but now we're value and object types are generic, I know nothing
about them and more interestingly when they could be reference types they could be value
types.
So the size of this struct at runtime really depends on how exactly it was used and I don't
know by looking at this type declaration at all.
So I don't know how to generate this initializer.
So another cool thing about that is you may know if you've looked at a Valhalla which
is we can produce tightly packed arrays and other structures using these kinds of mechanisms.
But again we have to know the exact type at runtime at moments we're trying to initialize
the thing.
So how could we do that?
We already have the type metadata and okay if the type metadata doesn't actually have
this information but we can calculate a simple offset and jump to something called
the value witness table.
So the value witness table is something that contains a lot of interesting both functions
and values that we'll be using to work with value types.
Specifically how to initialize how to copy how to destroy these values and great we
have the size and all the data we need to do our initial allocation for.
How can I use that?
So that is actually a C struct so I can just describe it using the memory layout, struct layout.
I literally just layout okay this is this field, this is that field I know this is an integer
this user addresses and as I'm done with that I can get a Valhalla at the appropriate
offset for a given value witness table because I have that I can get the size of any swift
metadata that I press.
So now we're able to not only just get oh I got the size we can go a bit nicer and get
the entire memory layout description based on these information.
So I get the size, write or calculate the padding and I can return a struct layout that
it has the proper name that describes the swift type name and today at least it's going
to look a little bit like this from the previous perspective it's you know native memory
layout and it's a blob.
At least it's the right size of blob so we're happy because we can initialize and copy
it using the value witness functions but maybe we could do that a little bit better in
the future today we don't have that metadata we could do that and then we could describe
exactly oh I know this is an int and this is an int and we could do the absolute correct
metadata and then even do direct access to the offsets rather than going to rather than
doing down course.
Okay so we can check that if I got that right I can implement for initializer for
a value type at all before because I just passed the value layout as we do for arena allocation
and then we initialize that using the initializer.
As you can see we can just print that on the JavaScript and it always has the right amount
of bytes we need to be passing around.
It's really cool we just taught Java to understand the layouts of native Swift objects.
Now let's actually call some methods on things and this will be also interesting.
Let's have a again my struct and I want to call a method called capacity on it.
Capacity is another value type so first thing first I describe the function the function
is a member function so we need to pass this the self as we call it in Swift.
Let's say I pass it as the first parameter here and when we describe the rest of the function
it's going to return again indirectly because it's a value type as we talked before
in initializers I do the dance with getting the value layout, allocate that do the
down core and they return I wrap it again with the appropriate wrapper type.
Now how do I make this call?
As as any of our non-seal language would today which is I have to generate a small
function that we'll be calling using the C calling convention and we make sure using
the C declarative use on the Swift side that okay this has to use the C calling convention
and you know don't use features that C wouldn't understand like generic and you know other
things that Swift has.
This function doesn't really do much except cast the things to appropriate types and
down core into the actual Swift function using the Swift convention because it Swift
is calling Swift.
Now in an ideal word we wouldn't do any of that and just call directly right with interaction
costs as both build complexity I have to generate these things build times I have to compile
them and of course performance because the yet another interaction in the way that would
be cool if we could call directly and you know maybe to explain even more so why do we
even need this interaction well Swift actually has its own calling convention and
that's because it's you know optimized to patterns and invocation invocations that
it often does and one of those optimizations in this API is that self must be passed in
register so it's not like passed in a parameter it must be passed in a register but Swift
has a stable API so we can easily find out where we should be passing it so the
API is stable on some platforms it's well described you can check it out here so specifically
on this register on x86 in this one cool so can we fix the JDK to use the Swift calling
convention so I can get my magical dreamland version of directly calling yes and no I believe
the code is very well structured and we have an opportunity to do that it would be here in
the color range of a specific platform or maybe the calling sequence builder which would you
know we would maybe tell it hey this parameter is special so that's a good news the bad news
is these types are JDK internal so that's not something we did today but it's certainly
possibility and other one times like dot net runtime inversion 9 have actually explored that
a bit and that's a PR that implements parts of Swift's calling convention again to be able
to call directly rather than from some sea shims and finally a little bit of object lifetime
discussion because object lifetimes and memory safety is kind of the core of our goal here right
we want to have this integration have a nice and safe so how do we do that but before I say how
I need to explain what's really destroying a value means in Swiftland because it's actually a
little bit different than on the JVM so again value types may have a value type code container
it contains a number core primitive a person and the person is a reference type so if I create
this container and then assign it to it because we're a reference content you know the ways with works
this will be then this will call destroy on this value type from our good friend the value
within stable this will then go through all the fields and call destroy on them or if it's a
reference type eventually just release a retain come release on it if that gets zero we have to run
we didn't the initializer and we're gonna run we didn't the initializer right now and noted
some later point like it would be we finalize some of the JVM so developers expect this to work
like this so what do we do to make it work with this we actually got inspired from the arena's API
and we offer something called a Swift Arena in the you know support library that comes with all of
us and yes it's a segment allocator same as arena's in the JDK but we also when we do initializers
of return types we register them with the Swift Arena now we register them of course to manage
for lifetimes so as I have registered for example in a Swift Arena of confined style which is
again inspired by open JDK arena at the end of this scope we will call destroy on all the types
that were registered with it again that is why we'll do the right thing here the same pattern
applies for reference types you can just register them create them in this scope and we will just
reference comes down on them today if we read if we don't read reference comes zero here we assert
but I think you could totally see valid use cases for maybe you want to assert if it didn't get
released and destroyed at the end of the scope or maybe you do actually want to you know just keep
this style for maintaining reference counting so this is something we can do and we also try to
guard against programmer errors so on the Java side you could make a mistake and unsafe he escaped
a value created during this scope and you try to escape it outside of it so of course after
this scope this value would have been destroyed it would be pointing its tail of the marine so on
the Java side as we do with the choice we mark them so we don't try to down core into memory
that was already the allocated again or then the name of safety so we don't the bad things here
now we do have an auto arena as well again inspired by open gdk where we use phantom references
and a cleaner thread to clean up these objects eventually I think it's great for prototyping if you
want to show you know evry Java or swive developer hey please give us a go think that's a good one to
start and there may be other situations where you want to rely just on the gc to these things as well of
course it's less predictable than so for swive developers this would be a bit can be a bit surprising
as you can see this is a very deep integration we're not just saying hey you can call
simple functions now we're shooting for integration of both a programming model and 2020 you
would safety as well as you know making it a pleasant thing to use and with the end goal of being
able to import really most if not all swift code we have laying around so if you walk away from
this talk I'd like to remember I'd like you to remember one thing but on goal is here to have memory
safety all the way we don't have to lose memory safety as we cross into native land and if we
collaborate from both sides with the AVM and the native language you know enforcing each
other's patterns then we can pull that off swift and if a family PI's really worked well together
I didn't show super crazy advanced cases but we can integrate them very deeply even with you know
more in generics and use cases I didn't show today and you know value types are the core of swift
and we can actually deal with them pretty well in Java which has been great so there's ongoing
work on direct-track swift if you're interested in you know integration between languages like that
please let us know and whatever like upcoming next steps we can look into of course performance
if you're interested in performance let's say never ending story here of course we'd like to
challenge a little bit of the generics because we want to support optionals and connections
and then we will have to handle generics as well as swift closures and you know passing a
closure to a swift function and then calling that on the swift side we can implement using up
course into the AVM as well and you know express swift closures as function interfaces and you know
those other calling convention differences for example for errors or async code and last but not least
integration so that's all I had if you have questions please check me outside later on and thank you very much
you