Hello everyone, my name is Hendrick. I work at CSE, which is the Finnish National Supercomputing
Center. And I'm going to speak to you about the your HPC Federation platform where I currently
now function as the technical coordinator for the whole solution. Now for those who are
now very aware of the ecosystem and Starbucks waiting what is the your HPCJU and what are we
actually federating? I'm going to give brief overview of the challenges the user are facing, which
then drive the design of the service. I'm going to talk about the architecture, the components,
and highlight the open source aspects of the project of service. So for those who do not know
the your HPCJU is a joint initiative between EU, European countries and some other actors.
But essentially there are entity which then co-funds really large machines in collaboration with
European countries. The most important part is whoever is managing the infrastructure, the
supercomputer, that's a hosting entity. I'm going to talk about them a lot and you're going to see
the abbreviation HE multiple times in the presentation. And so for example on the right we have
Leonardo in Italy, they're the hosting entity would be Chinika, as many GPUs and BSC to the left
with modern Ostrom 5. The users of the system as with most national supercube computing infrastructure
is academic researchers, research institutes, public authorities, and industry. This is kind of a
maybe a more novel point on the HPC side where a lot of universities that's just for research
about 20% of the compute capacity for the systems is kind of reserved for SME usage.
Okay, so the actual systems we are targeting with this federation which is going to be accessible
by the platform. So we have around 12 classic supercomputer, big Linux boxes, large parallel
file system, loads of local users, a nightmare for anyone who actually gets a hiccup when you say
that we have 3,000 untrusted users with shell access on my machine and that's my status quo.
Then after the classic supercomputers, they are these AI factories which are essentially just
AI optimized supercomputers with then a bunch of added functionality. They do not exist yet.
About seven have been announced at the end of 2024, but probably a lot of GPU, but then more features
around data management, next talk previously illustrated, some of the issues, and then more cloud-like
features, let's see what actually emerges. Then for more experimental scientific usage,
which will also federate are the various procurement quantum computers for European researchers
to test out different architectures and technologies to see which one is going to come out on top.
If you want more information, you can go to the UHBCJUs page where they have a nice info on
what's up and running and what's up and comment. Okay, so the current issue if you are a researcher
or industry or something and you want to use your HPCJU resources is that each of these now currently
12 different HPC sites have completely different identities, projects, and onboarding methods.
So this means that if you're granted compute access for the computer in Italy,
okay, you register an account, you send a copy of your passport, you're filling of a bunch of stuff,
you get an project, you add people there. Okay, then you need to do some other research. Now,
you got resources on Lumia, which is in Finland. Again, register completely different process.
It does not know anything about you. So then when people start using multiple of these machines,
it's really a hassle to keep redoing this and then especially managing large projects.
For example, some of the largest projects, or Lumia, they have 150 people in the project,
then you might have a 50 people more in Italy and you don't have a central way of managing that.
We're seeing an increasing amount of new user groups, which are not familiar with classical HPC,
the Linux command line, or then are accustomed to a different level of abstraction for their compute.
And of course, there is a great interest then to make sure they can effectively utilize the compute power.
Not just dump them in the shell and say, have a nice day, because that's not how Europe is going to win,
or at least participate in the AI race. Growing heterogeneity of the compute is a headache for
everyone who is not in love with the hardware. They just want to do the science, but then we
a center keeps switching out. Okay, now we bought AMD, yep, sorry, it's way worse the software,
but you're going to have to now do three months of extra work to continue your analysis.
And again, to the topic of the previous talk, compute is way easier to move than data.
So all of the time, one of the challenges when you want to utilize more resources on another system,
or you're just allocated resources on a different system, it's not putting your code.
It's if you need to move your data there, and this is for some science areas, this is more of a problem,
climate science, high energy physics, or something. If you suddenly want to move a petabyte,
there's something, it's just a snap of the fingers. Okay, so do the actual solution, now that you know
what's going on in the ecosystem. The Federation platform has a key few functionalities.
The first one is one federated identity with single sign-on. This in practice would mean that a
end user would use their probably home institution university provided authentication to then
identify themselves to all the services provided by all the machines. Then a single point,
one stop shop for resource allocation, management, and monitoring. So if you're a professor
and you have multiple projects, multiple allocation and multiple machines, you have a single point of
doing that. Direct access, you'd like SSH certificates. I will not cover these in details,
but essentially again, making sure there is the same way of accessing all the systems. So you don't
need to teach an end user, then different ways that we manage SSH keys on this particular system.
That's more of that. Okay, I want to compute there. Okay, it's going to be the exact same. It just
tell me where you want to go. And then for the kind of less low-level compute oriented user
communities, high-level features, like web-based access for Jupyter notebooks, following up
what your AS simulation is doing, browsing some stuff or just simple shell access.
A federated software catalog, which is based on easy, more on that later, but leveling the
feel that you can expect some sort of basic catering of software and all the different system. Again,
not having to start from scratch every time. And then advanced features for workflows and data
transfers. So again, for people not very familiar with the ecosystem. Okay, I want to compute
that in that, and I'm going to need that data. Please go and execute my code for me. For the timeline,
so we started work now in January a few weeks ago. And the plan is that this will be ready on
nine systems by Q1226. So you can look forward to that. And unless I mess up my job.
And then just as information, it's a consortia of five partners where CSC where work is coordinating,
other partners are Nordenette, Geont, IT4I, Tartu, and University of Gent.
Okay, now I discuss the features. The high-level, really high-level picture you can keep in
your head is very safe platform, which we build, which provides all the fancy features and services,
which I just described to you. And now the end user then either interactively, click at the clicks
in their web browser. On that platform or then fetches API keys to do the more fancy stuff,
the more programmatic stuff, or is allowed then to fetch SSH credentials to go via the command
or not removing any of the low-level stuff that's still going to work just be more streamlined.
Now the secret source of the whole solution is of course the integration
to the actual hosting sites. This talk does not cover that. I hope I can give that talk at some
other point. But essentially, there will be a varying amount of glue making sure that data
about users and allocation go in the correct direction, so that user then actually has something
to touch on the system. Not drawn out, but of course, I mean data transfer is not going to take
around trip via our platform, which is probably going to be physically located in Northern Europe.
That would be very slow. Data transfer and other things which do not need to go via the platform
go directly from system to system, or then from the user to the system.
So the architecture, if we're looking now, this is the central platform. So the thing in the
middle and the previous picture. So essentially, it's a very modular and flexible architecture
where separate components are tightly glued together with the SSO and single identity,
but they're responsible for some particular feature of the system.
And this is, I mean, we're heavily leveraging a lot of open source components, which I'm going
to show next, and then to be able to accommodate all the future systems. Now we know how nine systems
look, but there's three systems coming online, seven AI factors, which we do not know what the
text that will look like, and eight quantum computers where people have now even decided what
they interfaces. So we're going to need a high degree of being able then to adapt to whatever
somebody decides to build to be able to plug it into this. So there's the core platform, which we
manage, and then the, again, glue components, the hosting entity components, which are responsible
for connecting our various components. Perfect. Okay, so now I'm going to flash a bunch of components,
some of these are maybe familiar for you, some are not, but I've left the links there. So the AI
is going to leverage John's My Access ID, and then a few other components, which are still being built.
Allocation is based on Valdor, or then you might have heard the term Puri, which is a kind of
deployment of the Valdor software. Open on demand is probably the HPC technology, which is most
broadly used out of these ones, it comes from the states, and that would be responsible for the
interactive click at the click at the, work flow is based on Lexus and hippie. Reporting is your
basic standard tech stack. There's Grafana, there's a saying that there's open search.
We have a help desk, because, again, this is we're not just building a software product,
we're actually producing a service, which means we'll have users, which will lead help.
And then the software catalog, as I briefly mentioned, is going to be based on easy-built and easy
via a certain VMFS. So most of this is open source, you can go click of this up, you can go
download them, some of them are fairly complex to run, so you can just do an install and
accept, expect them to be up and running, but if you're willing to put in the time, most of
this thing, so you can run them yourself. A lot of them are already running separately on various systems.
Good. Okay, I'm going to give a extremely brief overview of the main components.
Okay, so the AI, as I said, that you utilize as my access ID for those not in the know,
that is essentially super simplified, it's an AI proxy with a discovery service and account for
registry. What that means is that when a user logs in, they can search for their home institution,
for example, there's about 3,000 hits for something named University of Blah,
then they are presented with that, for example, that university's logging screen, and okay,
now they are registered. Now, my XID knows that there is some unique ID which identifies with
this identity provider, then there's fancy things for linking if you have multiple identity providers
mapping to the same actual natural person and so on. For those who are doing the technical integration,
essentially some or open IDC works out of the box. And this is going to provide a SSH certificate
authority for then the access. So users log in, get a short live SSH certificate, which is then
trusted at the target sites, you can log in. 24 hours later, you have to reauthenticate and get
another key to continue working. Perfect. Okay, let's have to speed up. Okay, allocations,
this was based on valor and SSH had a unified place where you can view your project, your project
memberships, and to what systems the project grant you access and how much compute hours you have left.
The actual granting of the project is not directly part of our platform, that's the responsibility
of another component being procured, your HPC peer review platform, which should be online somewhere
around summer, if I recall correctly. Workflow is kind of your entry point for doing easy to use
managed workflow for selecting, like I have an application, it needs this input. I can now
graphically connect these and it will take care of running it then on a single system or then
over multiple systems. It also supports then this kind of data staging aspect that, okay, if your
data is on system X will move it somehow to system Y depending on how much functionality the
various hosting entities provide for moving data. Do they have irons for staging? Perfect,
this is going to be really quick. Do you only provide SCP through a login node? It'll work,
but you'll have to spend a bit more time in the coffee room waiting for us to transfer the data.
And this is able then to target, I mean, slow them, other batch gaps, or then Kubernetes,
which is most likely going to be more relevant with the AI factories.
Okay, open on the band interactive, you can get test stops, jupy notebook, a shell, for those
who have used open on demand, we're not going to use the vanilla open on demand. We're going to
develop functionality to move it further away from the system. Currently open on demand will
essentially sit in your HPC center. We don't want to do that. That's too invasive. So we're going to
do several contributions to be able to move it further away from the system, which might also
be useful elsewhere. There will be health desk and reporting will be able to show you how much
resources you are consuming and hopefully how much resources, some of your postdocs burnt on
something stupid and then you can go and not run the door. The software catalog, there are multiple
people here on the room who know easy way better than me, but essentially we're going to use easy
to provide a pseudo unified stack with at least some of the software and compilers being that
the user does not need to care specifically what architecture they're running on, are they're running
on AMD or are they running on Nvidia? Is this okay, I want to use PyTorch, module load PyTorch.
Easy fixes that. Okay, so last two minutes on the open source aspect, this would not be possible
without open source. If you consider the scope of the functionality and the implicit understanding
the implemented technologies have of user requirements, starting from scratch and compiling that
into something sensible would be an extremely large job. Federation, of course, requires a degree
of trust that, hey, we're our stuff works. With open source that, okay, somebody else is running it,
you can have a look at it, you can go and check it out. It's much easier than here is my binary
blob, please install it so that stuff works. And then also on the promised features, because
now I've shown you a bunch of stuff. I'm promising you very many things which can be done.
If this was a close source solution, which wasn't used anywhere and you can
look into it, you would probably think that he's trying to, I mean bullshit us. There's no way,
but now you know kind of, yes, that is an open source component. It works there. I understand why it
works. This could also work. And users would probably rather integrate into less things,
the more things. And then of course, we need to do heavy modifications to be able to integrate.
I mean, glue everything together. If this was close source, not a chance.
And third seconds for less thing. Yes. So hopefully the benefit for the rest of the community
would be that we try to upstream what makes sense. Everything does not make sense, but we have
fortunately the ability to continuously work with the upstream and proposed features,
instead of doing this involved, which is sometimes the case with these kind of projects,
when there are some IPR-related restrictions. For a lot of projects, the integration
is the hard part. So I'm hoping that when we work out the integration with 20 plus systems,
we're able to upstream or those change or at least document that this is things you can encounter
during the integration. And then the final last point, of course, is it's a large user-based
for the project. You get bug testing's feedback for certain projects. It's also a very good
measure of impact. Hey, we built such a good project that they've dumped it and opened it
to 3,000 European scientists. Okay, thanks. Questions?
Thank you, Henry. Any question? Yes. All right. Is this something that it's only meant for the
big players in the HPC ecosystem, or can we expect to have small clusters, university clusters,
also left in people FBA? Yes. So the question was that if this is intended for the big players,
in the long run, this is meant also to facilitate that that if you want to connect smaller,
national or regional things to this, that will be possible.
You're mentioning that you're going to use certificate authorities, because everybody said
you have to use M&A even for SSH, you see people on it. But the thing that keeps on popping up
on this Marsite is there are a lot of people actually SSH into the cluster with spits,
and the shortlet certificates are an issue. You already have a solution to that, and I'm going
to try to protect that network connection framework. So the question was that how the certificate
lifetime of the SSH connections affect certain user workloads. So yes, some of the things you
will be able to do the workflow stuff, and the hope is that we would get rid of these
infinitely lifetime on secure keys somewhere, then the Federation platform tries to be very
agnostic from site policy. So then it's more up to the hosting entity, the how long do you want
your certificate to be. So if somebody decides we only want to give our HPC users 48 hours,
fine, we'll dump out 48 hours, or then if, yeah, we'll find with three months, good.
Personally, I would like to see a kind of dynamic shift where the longer your SSH certificate life
time is, the more restrictions I add to it. So okay, you can get six months, buying and
restrict the IP, I'm going to turn off port forwarding X11 forwarding. You want everything, all the
fancy stuff. Okay, I'm just going to give you 48 hours. That's what I would like to see us
going towards. It's a good compromise in my opinion.
More questions.
Yeah. So I'm also a plan of again, we'll get healing in this layer. So we can maybe decide
if we can say my code, if the cheapest of most convenient site and copy the data,
if you're previous to when the copy of those is our convenient way to do it.
Yes, so that's an explicit, sorry. Yeah, the question was, is there going to be any kind of
automated scheduling features built into the Federation platform? That's explicit
requirement for the Federation platform. A flushed on one of the slides, I had a very, very
but yes, there will be a smart scheduler, both available in the workflow and available as a service,
which is, okay, please tell me what system should I run on if I'm time to solution or energy
to solution or you have some other specific requirements. Usually, probably, police tell me which
system is the least used, because that's probably going to get you the quickest answer.
So the question was how we manage the scheduling and the staging in of data.
This is a layer sitting on top of all the clusters and slurming everything. So most likely it
would be that, okay, we stage in the data and when that's done, then you would start scheduling
the jobs. If you have very, if the cost of failure when data is now available is small,
then you could try to cheat it out that, okay, data is probably going to, I mean, be available in
hour and the queue is at least two hours, let's take a small risk. But then if you're scheduling
half the machine, then you're probably going to take the conservative approach.
So the question is, how we ensure that the S8 certificate maps to a local user? How that essentially works
is that the platform when there is a new user will exchange information with the target side.
That, hey, there is a user with the following attributes. Please create necessary local accounts
and then inform us of the unix account. At that point, in the certificate, that will contain
the unique My Access ID identifier, which the hosting entity would have in there principles mapping
that this unique identifier is allowed for this unix user. And then we've know that, so then they are
let in.
George, yes, this transfer, in terms of what you have to point it to technology yet, for data transfers.
So, yeah, the question was the technology for data transfers, the answer is whatever is available
at the sites. So we're not introducing or imposing any additional, we work what's
available at the various, some have IRONs, some of S3, some have just SCP.
All right, the record up here, one final take, where does both of this is, as well as getting
university, is in all of that, several of these files will be highlighted. So if you want to work
on open-source software, on the biggest supercomputers in Europe, this could be very interesting for you,
looking for new jobs. I'm hiring.