Hi everyone, so I am here to present my talk on high performance Jupiter Notebooks with Zaspir.
About me, I am an open source contributor, I have been contributing since 2016 and this is my
second first demo talk, last one was in J. Ruby and these are my GitHub and X URLs.
So as you all might have used Jupiter in your work and you might have found certain
problems related to the performance, like a lot of times you might see the sessions crashing,
the Notebook becomes unresponsive, especially if you are using it on a Jupiter hub instance
and you have a sign very lesser RAM or less CPU, so like and the kernel dies and sometimes
you will use your work, so yeah this was me like I was facing such issues, so I thought
that can I make it better and so the project Zaspir was born and also it handles other
problems like scalability issues and it has a better user experience and in terms of
packaging Zaspir is a single binary just 8MB of size, so yeah it is very like it
shifts small and it also has much better performance compared to Jupiter lab, so I will explain
this in more, so yeah in terms of performance Zaspir uses 500% less CPU than Jupiter lab
and around 4000% less RAM, plus it is much faster than Jupiter lab that the responses are
instantaneous, it has ultra-low latency and the kernel start-up time is also very fast and it
has higher throughput and yeah you can hand much more kernels than Jupiter lab, so if you are
like running an HPC cluster and you are using a kernel gateway, it can easily handle around
100 kernels whereas Jupiter has just around 64 kernels, so yeah there are proper benchmarks
that I will show you and that you can run on your machine and you can verify this, so this is
how a Jupiter lab architecture looks like, so we have this in the center, the Jupiter server
and on the right most part we have this Python kernels or Jupiter kernels, this kernels can
be either in IPython or Ruby or Julia or R kernels, the center part is Jupiter server like
Jupiter lab, so whenever and the backend is built into a nano, a nano is something like
it helps you run asynchronous core routines, plus we have this the front end which is built in
domino and react, so yeah and how as per the first from there is like not much changes, we wrote
the backend in go, go back in and we have a custom react front end here, very light fit, so yeah
and rest of these things are like the Jupiter kernels is the same from what Jupiter lab does it
and we have a 0MQ, 0MQ is like a communication library like on if you want to communicate on
two sockets, you use 0MQ and for the message not being lost, 0MQ provides something called a
message queue, so if a client or a server goes down the messages are not lost, so that makes 0MQ
really cool and while Jupiter lab uses piZMQ python base 0MQ, we have a go native 0MQ,
so it's not connected to the C bindings directly written in go, whereas piZMQ uses C base
UMQ within it, so let's go into the benchmarks, so if you see the RAM uses, so how I've done
benchmarks is, let's say whenever you open a Jupiter lab screen and you create, you launch an
ipython notebook and at that point of time a notebook opens up but in the backend, the Jupiter
server creates launches a Jupiter kernel, so this is a different process and this process,
the server and the Jupiter kernel, they communicate to each other via 0MQ sockets, so to benchmark
this under stress, what I do is I try to create a lot of notebooks, for example there is 0MQ
2040, 60800 on the x-axis and we try to bombard it with messages, so like if you have let's say 15
notebooks open, so there would be 50 small ipython processes open, now you have to communicate with them,
so what I try to do is I add max, I do a 10 messages per second to per kernel, so that means
if you have 50 kernels open, I am bombarding it with 500 messages per second and we see that how many
kernels die and whether the Jupiter server will crash or not, so this will try to do it,
as you can see in terms of CPU users we are at 0.8% and Jupiter is in blue color, whereas
as per the blue color, so it is at 2.7, so as we increase the number of kernels, so for higher
number of kernels you can see that the blue line it goes around till 12.5% but in terms of
Jupiter it goes up and then you can see after 64 it goes down, this is because basically because
the Jupiter server was crashing, like a lot of kernels went down and that's why you see the
losing performance but if like this use keep kept going up, so yeah and Jasper on the other hand
no kernels will lost, so yeah in terms of memory users you can see that there is in terms of
Jasper there is a flat line, so we are always near 20.2mb, but for Jupiter you can see it keeps
growing, like at the last one you can see that it crossed even 1000mb of RAM and at that point
even some of the kernels crashed, so yeah, so how do we do this? Let's say in Python we have
something called cooperative single threaded road routines, Coruetines, so let's say if we have
something an eye-oblocking thing, like in Jupiter, Jupiter server most of the thing is like we
tried to communicate with the kernel, the kernel does the CPU thing but like CPU bound task,
but the server Jupiter server is mostly responsible for, receiving the message from the client,
getting the message to the kernel and applying back, so in that case is you try to run most of
these messages as asynchronous Coruetines, so these Coruetines are like there and in a like
advance in the event loop executor, like you submit that to the event loop, but this event loop,
like even though you have created a new thread, it will run in the same on the same code,
like the GIL doesn't allow Python to run it on a separate thread, so that's why you start
getting this performance drops, whereas in terms of go we are using preemptive lightweight green thread,
so you can all use all your CPU course advance, so this is the major reason for why we are
getting such a amazing performance boost, scheduling is done in go via this go runtime scheduler,
via the violin asyncio we are using the event loop, now we have this blocking behavior in
go routines, but asynchronous IO, this must not block and they are like they need to be cooperative,
and so you have some done blocking, you must use a weight or futures and then we have to wait
for these messages, so yeah this is how we are getting the performance boost, now what are the
benefits, you get much improved responsiveness in Zaspir, it's very lightweight, 8MBs of binary
package that you can just run and for interfaces it has a much better cost efficiency and better
scalability, yes before this I like to show you this the benchmark report like it's not trust
me bro, you can just run it on your machines and the entire code for benchmarking is here and
I did it on my Mac mini M4 with 16MBs of RAM, so these are like different plots for like
RPS, like 10 messages per kernel, two kernels, four kernels, then you could be doing now
then I do it for 100 kernels and 10 RPS, then I go for 100 RPS, like 100 messages being sent to
each kernel, so two kernels, four kernels, yeah and there is also like why sometimes Zaspir crashes
and why Jupiter crashes, and yeah apart from that this is the demo, like this is how when you
load a Zaspir instance it looks like this, I create a Python 3 project, I run LS,
122, you can run all kind of Jupiter kernels, like C base, C plus plus base, Julia base,
just follow the docs on our website and let's say I had to show a faster kernel startup time
I just click on this, all the cells will run, you can see like this is much faster than Jupiter,
and yeah that's all, acknowledgments to fast United and zero the for supporting this project,
I worked on this for one year full time and these are also very helpful to me in funding this project
and yeah thanks questions, right time for the question, it may be even through, we're doing
well on time, so in terms of authentication you want me to tell you how we do authentication
in Jupiter servers, so if like the Jupiter kernel is a process you don't need authentication there,
if you have to put an authentication you have to put it on top of the Zaspir server and if you
run it by the Zaspir CLI with a more minus protected is equal to true then you will have a log in
screen there, you log in and then you can go ahead, okay you can, so right now we have allowed something
like access token like you set up randomly and it's on random you just compare it and then you
can share it with like how Jupiter does it and for more like but your custom ways you have to improve
the code and then you can learn it, I have a quick question as well, how is this received in the Jupiter
community because any feedback from them, there are like two Jupiter contributors here and yeah
like the community was very supportive and I also like acknowledge them because it's not like
this work is based on the top of Jupiter wire protocol and I just didn't invent something new
but I just implemented it in a new way and like I received some good feedback from the
containers and like they're really helpful, okay excellent, thanks a lot