Hello, this is our talk about things that are coming together for Fortran Tooling, and
this talk is about how me and my colleague experienced working with Fortran.
And when I'm talking about me and my colleague, well, this is Peter.
We are both working on a scientific computing institute in Darmstadt, Germany, and he is
doing tooling for profiling and instrumentation.
It's a researching way to put your measurement probes about how to figure out which parts
of your code take how much time, making that efficient with low overhead, and he also
does apply performance engineering mostly in a space and aerospace safety context, and
done a master's thesis in collaboration with ESA.
And I am just doing C++ compiler stuff, but I also have fun doing that.
If it does analysis, transformation, be it source to source or IR to IR, or optimizations
with the limited optimisation knowledge that I have, I'm at least very much interested in it.
And basically, I'm using everything that is claimed to lean or IR passes in my day-to-day
business.
And day-to-day business means that at our institute, we have developed a lot of different
tools that do a lot of different things, and they fall into exactly those categories.
So metasyG and the cage, so call graph generation, is what it sounds like, it generates
call graphs, we will go into that a little later.
But there are also Alpaca, which is a program difference, analysis tool, and mini-apx is
a mini-ap extraction tool, or proxy-ap extraction tool, which I presented last year at the
HBC Devroom.
And there are also two other tools that I want to highlight.
One is Pyra, which was developed by JP Lair, the men within impressive mustache, and
the second to last row.
And a copy, which was developed by a colleague of mine, Sebastian Kreuzer, who also brought
you x-rays capabilities to now instrument shared libraries.
So if you make use of that, he's the guy to thank for it.
And now that we have all these tools, all of those tools were developed in C++, 4C++
with C and C++, idiosyncrasies in mind.
When I say all the tools for this talk, those two are the ones that are most irrelevant.
So on the right side, you see a call graph, I think most of you are pretty familiar with
how it looks.
It's basically how do functions call each other, and what are the possible call paths
through a program.
This is basically what our library meta-seaches, a meta-cograph library.
Thus, it allows you to specify a call graph, and additionally attach arbitrary meta-data
tool, the notes, or the edges, so that you can keep track of things that are interesting
to you, your use case, your program, and cache the call graph generation tool is the IR-based
usage tool of this library.
There's also a source code based usage tool for that.
And Pyra is the performance instrumentation and refinement automation tool.
This basically tries to automate his job by inserting the appropriate instrumentation
and timing calls into a program, trying to minimize the overhead, but trying to maximize
the amount of information you get about the timing and runtime behavior of your program.
The exact ways how Pyra works will be explained later by Peter, but Pyra is basically trying
to figure out that these three functions that are marked in a darker gray are the ones
that make up the kernels of your program, so the part of your program where most of the
computation time is spent.
And both of these tools are technically not dependent on CNC++.
So every time we talk about our tools and what they are doing, usually in an HPC context,
we're always getting asked, well, are you able to do Fortran?
And usually we say, well, Fortran is a little bit different than C, and they're different
front end, and no we can't.
But let's give it a shot, right?
One of the first things first, neither of us really does know Fortran very well.
And luckily, we are not the only ones.
This is an excerpt screenshot from the code that he found during his master thesis working
at the European Space Agency, where they just copied some Fortran code from original source
and treated it as a black box, which is basically summing up our combined Fortran experience.
And a second thing, second, neither of us really worked with Fortran too, and MLIR, which
make up a whole lot of the Fortran front end driver.
But we know IR, and you can get flying you to emit IR.
So we skip all the scary things that we don't know, and we lie on the things that we actually
have some experience about.
So the idea was, you take flying you, you ask it nicely to emit some IR, then you take the IR
call graph generation tool that is obviously language-agnostic and will work out of the box.
Then you have your analysis done, and then you can use our tools, namely Pyra, to do
your analysis, and hotspot detection, and whatever, profit.
There's no way this could ever go wrong.
And when a colleague again came for us from Arhan and asked, well, we have this U-Lish KKR
K loop code that might be interesting to you for your hotspot analysis, we gave it a shot,
and we just loaded the available flying U driver that we had flying around and gave it a shot.
And the program didn't even configure.
We weren't even able to ask flying you to provide IR, because during the configure stamp,
the program checked for the compiler identification, and flying U is too new for the build
system to recognize the compiler identification.
But not to worry, one afternoon, maybe two afternoon's later, we hacked flying U's compiler
identification and the appropriate flanks inside the build system, and we could get it to configured.
Right.
So, after you configure, you compile, but our compile also failed.
The reason for that was an unsupported intrinsic.
And now this project started in January 2024, and we were relying on LVM-16, because again,
that was what we had lying around, and we started looking into what is this intrinsic,
do we need to implement it, and some helpful person in the LVM community pushed a fix for
that exact intrinsic two weeks earlier to the marine branch.
So, instead of taking what we had lying around, we compiled LVM from source or flying
and everything that belongs to this whole tool changer, flying uses, and we got rid of our intrinsic
error, and we got LVM IR.
This was another few afternoon's to get things working, but now we have IR.
We at least compile to IR, and we take our IR and move it to tools.
Well you remember that we had to build the newest version of LVM, and Cage, as it was originally
designed to generate call graphs for C, C++, made use of type pointers for some of its
analysis.
So, mainly VTable analysis to figure out, are we actually calling something that is a function
type, or are we calling into a struct, which then is usually a VTable.
But your new struct doesn't make use of VTables in the same way that C++ does, so we were able
to just strip out most of our logic of our beloved call graph generation tool, which left
us with a very bare bones call graph generation tool, with basically no other analysis than
just providing you a bare call graph, but now you have call graphs for fatran codes.
And now that my work with LVM IR, getting things to compile, and having tooling done,
I was able to hand over to my colleagues, so he can do the actual pyro work and the instrumentation
work.
Great, all right, thank you.
To give you some background first on what do you know how the measurements are working
basically, is both pyro and also some of the other measurement tools rely upon Scorpie
for the actual measurements, which is a profiling and tracing library coming mostly out of
our usually interesting, I think Scorpie relies on instrumentation for its measurements.
There are a couple of options how this instrumentation can work.
You can use F instrument functions and related flags, which gives you some, but not complete
controller, but what's actually being instrumented, there's also a GCC plugin that's being
shipped with Scorpie that kind of hooks into the compiler and serves the measurement points.
And it's not released yet, but we checked a couple of days ago, and it's by now released
candidate, so Scorpie 9 will include a LVM instrumentation plugin finally, so it basically
does the similar thing to what did you see plugin does, hooks into the compiling process
and inserts these measurement probes.
As we did, we did some debugging figured out Scorpie is not working with the other versions
of Scorpie, but the fix has been pushed, so you built Scorpie from scratch,
and then you try it, and then the compilation fails again, because the open MPI
fought around 90 compiler effort needs to be aware of the kinds of flags that are supported
by the actual compiler, and it meets flags that are not supported by fling you.
So one more diversion, basically.
Yeah, well, as I've said, so open MPI needs to be aware of the compiler fixed supported by the
actual underline compiler, and it didn't do that until a few versions ago, so we tried to build
open MPI for one four, which didn't work, but it happens to work that if you use
basically one of the newest open MPI versions that this version you can build with fling for the
four-trend parts, then it knows the right flags, and it actually manages to compile an open
MPI fault-trend program. Well, we then had to recompile a Scorpie again from scratch, because it's
kind of bound to the compiler version and open MPI version that you use, but after that, we're
actually able to run to unparallel. To give you some background here, I talked about instrumentation,
and I guess most of you are aware, but instrumentation is one of the two major techniques besides
sampling to generate data about a performance run. Basically, it's based on inserting measurement
points into the application. There are a couple of variants of instrumentation. You can do instrumentation
at different points in the process. You could do it manually, you could do it in compiling stuff,
but we're talking here about compiling instrumentation on a function level, so no loop instrumentation
or anything. Instrumentation is able to produce very precise and reliable measurements, but kind
of introduces the risk of very, very high overheads. So if you instrument every function in an
program, you can get over as up to a thousand X or something or even worse, which can be the
kind of ruins your results. So what you need to do is you need to find a balance between
no instrumentation and full instrumentation with full instrumentation giving you a lot of data,
but you can basically do 30 data right away, and no instrumentation with, well, there's no overhead,
but no useful information either. What performance analysts usually do to kind of to find
this balance is what we call the builder and analyze cycle. So they start with a rough initial
overview measurement and then they iteratively stepwise work their way and deepen the instrumentation
further until they get some instrumentation configuration that is useful to them that actually
gives useful information about the application. And as Tim already kind of announced the idea of
pirates to automate this exact refinement workflow. So pirates start by just looking at static
information in the metadata g core graph and then it builds some initial instrumentation
runs the program and then combines the dynamic and static information to iteratively refine the
instrumentation until it reaches to some configuration that yields useful profile hopefully.
So that's what you did. And after these several afternoons of debugging, we actually got it
there was one one more thing. So Tim already talked about it. There was some problems configuring
there was some problems building, well of course there was some problems linking as well.
So they turns out that the Suley Kakaar code is kind of designed to work with the Intel MKL
library and it did refuse to work with open blast or something. So we kind of hack the build system again
to kind of link the MKL things into the fling you built which in the end worked and we got it
working and actually pirates able to successfully instrument the binary and find the the relevant
hotspot which in this case is some some kind of matrix inversion which calls into into MKL here.
All right to some things up. So as it turns out you can use LVM IR based tools to do for
trenching with fling you now. It's a little bit hacky sometimes. So from time to time you need to move
to your potentially unreleased version still but that will sort itself out over time naturally.
Then system is always a bit tricky so you need might need to spend some time hacking your
range of it and by now you still might need to compile things from Suley as they're not released again.
But it is it is possible and we we demonstrated that using using this ULIC mini app.
So as Tim said right we're not photron people not even fling developers but basically from
our perspective which is looking in from the outside. It really looks like things are coming together
for fling and for fling tooling as well and I thought some people might
appreciate that some things are moving forward. To conclude this talk just let me give you a quick
kind of outlook into what's next for us. So as Tim said at the moment we mostly rely on LVM IR
to do our tooling some things also on source code level. So we think about trying to kind of move
that to forward run as well. So fling you does have a kind of source code level
plug-in interface under development. It seems to us that it's pretty much in early stages of
development at the moment and you're basically discouraged from using it as of now but as soon as
that's to some mature state we're definitely going to take a look at it and we're always in the
process of refining the heuristics behind pire on the other tools to kind of come up with better
implementation configurations with left with less overhead. I'm actually working on a project called
flip right now which is kind of tailored instrumentation to kind of generate just the information
that you need to generate a good profile and we're also taking the looking at dynamic instrumentation
which is kind of instrumentation that does not need rebuilds in between iterations so you can just
turn off and turn on the instrumentation at one time which is something this LVM x-ray does
and we in the process most of our code expression is doing that to do that in his copy tool.
All right thank you very much for your attention and we're looking for any any questions. Thank you.
All right so the question is what's the the killer application for instrumentation based
profiting against something based profiting well it's kind of always so you do have both options
and there's some upsides and downsides for both so the the main thing is that kind of gives
instrumentation the edge over sampling in some use cases is that it's reliable you won't miss anything
so with sampling you if you depending on your sampling interval of course you can't be sure that you're
not missing anything any smaller functions which is something you especially if you're
tracing right and then you also want to see smaller functions and that's something you can do with
instrumentation you kind of you you start using an instrumentation to do profile and as soon as you're
you're you like the instrumentation configuration you use it to trace this but the part is right
sampling does similar things with a different approach you have any plans to look at
really if you want to suggest you see if you do K is considered a for trial so the question was
if we try to look into things that are not proxy or mini apps but CP2K was the exact application
dimension which is known to be a for triangular so we are definitely interested in getting things to
work for more complex codes we looked into mixed codes like cleverly which you see and for
trying combine to do things which also had interesting first results but short answer short is
we are still scared of large and big for foreign codes because if things go wrong neither of us
really will know what is going wrong so while it would be of course interesting there are a lot of
things inside our pipeline that would then have a realistic chance of failing and the ideas to
work are way slowly up to more and more complex codes and not jump from this is a small proxy application
mini app which we have a domain expert on which we can ask nicely to help us figure out what's going
on to this is a known compiler killer are you interested in having your application work on that
yes we are but we'll give it a little time so the question was what about other tools like
flank tidy compared to like clank tidy if I get this correct you're asking about whether we
are looking into helping developers write better better fortune code with like tools like
flank tidy in my experience which is very limited when it comes to fortune. Fortran tooling support
is very bare bones so while there is a lot of things to do writing a Fortran base tool that
helps you write syntax and tries to find obvious errors would again require us to actually know
what's going on in Fortran so sadly again the answer is I really see the use case for a tool
like flank tidy and I hope that someone with more Fortran experience and knowledge than I will
have a go at it because you can start doing it there are compiler plugins available for the
flank compiler now again the IPI is under heavy development and it says do not use for production
use case on the documentation but you can start now and you might be ready once the IPI is
able to just release flank tidy nice use case currently not an hour old map sorry
thank you very much
you