Yeah, thank you for coming to this talk as well.
I'll just keep going through the slight chaos, but I'm Connor Erd, I'm from University
College London and I'm here to talk about unit testing in Fortran, so I'll try and go over
a bit of just my experience in his few court examples and how we can kind of automate it
a little bit. So, starting off with just a bit of intro about Fortran, kind of. So, it's
not working. Is it on at all? It's on, okay. Right. Hopefully it's fine. Yeah, yeah. So, for
some colleagues I've got who work in Fortran, I've got a list of a few projects that
they're aware of and they've worked on and what I'm trying to get across here is that it's
still kind of relevant. The context that I work in is academia a lot of time and a lot of the projects
work in things like fluid dynamics or weather simulations, which you can see from the
UK Met Office being quite a relevant code, which is in Fortran, and we can also see what I'm
trying to get across is this kind of popularity. We've got projects with quite a lot of GitHub
stars which are still Fortran, but something else that I'm trying to get across is that
these projects, even though they're popular, or relatively important, they often don't have
unit tests, which you can see I've got a kind of ticking cross system here, so they might have some
integration tests, but individual units functions often aren't tested in that way, giving
that extra confidence that you think you might want for these kind of codes. So, why might that be
why might it be difficult? And again, this is experienced from academia, so it might not map
to someone in industry. So, in my experience, it's often legacy code, and what I mean by that is
there's often if you perhaps bad practices, like a lot of global state being used,
which makes it quite hard to unit test, because you've got to manage that state and make sure
your tests are isolated and not interfering with the state that might affect the different
tests. There's often a lack of units to test, quite bad wording there for confusion, but
sort of a single function might be writing to file systems, it might be doing multiple
responsibilities, and it's difficult to write that into a single nice unit test. So, that makes
a difficult, and then often you've got lock into, like, specific compilers of dependencies,
this might be because you start this project 10, 20, who knows how long ago, and things haven't
developed with the rate that Fortune has developed, so you're locked in and I can limit the
environment you can work in and the different libraries that you can connect to.
And then, another point why it's difficult to test Fortune sometimes is that you get a lot of
custom testing setups, so people don't necessarily use an open-source framework, they might
develop something themselves, and that kind of means that the community, yeah, sure, the community
won't benefit from sort of open-source contributions to a framework instead, they all have to
start from the scratch again and develop their own framework. So, that limits someone thinking,
oh, I might start unit testing, they don't know how to do that necessarily, which links to a
lack of know-how, and again in academia, this is often the developers might be a PhD student,
who already has a lot of things to be thinking about rather than learning this new unit testing
way of doing things. And then, it's unclear which frameworks to choose, so what tools, and with
that, we can go into what are the available frameworks or tools. To try and figure this out,
you might do a little online search, and you might come across a few websites, one being the
Fortune Wiki, which for a while has been sort of go to page for Fortune, and they have this page
on that website, which is unit testing frameworks, but there's quite a large list here, and there's
no real suggestion of which you should choose, how to use it, or why, so that can be quite confusing.
If you go to the more up-to-date, perhaps more maintained website, Fortune Online, which is
kind of like a new community developing slightly more modern Fortune tools, they have packages
which they list, and on this page, they've got a big sort of large bin to handle things in,
which is error handling, blogging, documentation, and testing, and again, this is a large list,
and again, they're not all testing, and there's no real direction of what to choose,
so it can be quite difficult. So with that, what have I chosen? So I have gone with something called
peer-funit, and the reason for that is, there was a previous experience in our department,
so I could get help when I was learning how to use it. It's open-stars, and it's open-source,
sorry, and it is still being developed, which is more than you can say for a lot of the ones
on the previous list, and it appears to be the most fully featured in my opinion. The most
importantly, it supports unit testing, MPI, parallelized code, which is very important for
Tran, because that's just a very common thing that you all come across. So what is peer-funit?
It's a bit of an overview, it was developed in NASA. It is open-source, as I said, it's on
GitHub, and they do accept contributions, that's the link, which there will be a QR code for this
slide later. It's available under a NASA open-source license, which I believe is similar to
MIT or BSD, but there are some differences, so be aware of that. It uses a pre-processor file,
so it takes a specific peer-funit format and converted to Fortran, but it is very similar to Fortran,
we'll see an example later, and it is trying to follow in that words J unit, and I think essentially
it uses directives, which like annotations in J unit, I think, and they say essentially label
the code sections for the pre-processor to understand what it needs to do. And then one last point,
is it uses a lot of slightly more modern features, you often need more recent versions of
compilers to get it to work. Okay, so a little bit of syntax, so this is a very simple
relatively.pf file, which if you know Fortran, it is very similar to Fortran,
straightly only real difference, are these the decorators, the directives, so a few things to point
out is you can import pfunit first, it's funit because it's the serial version of the library,
and then you label your test function, so in this case this is a subroutine, which is just testing
something silly, I've written, which just doubles the numbers, doubles the integer, and then you
can assert within that function using one of these directives, so that's just checking your actual
output equals two, and you can do a nice message in case of a failure, so that's sort of what
writing a test would look like. So I'm going to fill on the pre-processor, it's written in Python,
as I said, convert stop pf to fnit, and if you build with make or see make, it will do this
automatically, once you've got it in your build system setup, and we'll see how we do that in a moment,
the directives for you can label a custom type, so you can parameterize tests, which is obviously
very common in other frameworks and is kind of needed in my opinion for a testing framework, which is
going to be worthwhile using, you can label test subroutines with the app test, as I've just said,
and you can specify various assertions beyond the assert equal, which I've shown, however the
documentation for this is not the best, which I'll show in a moment, but I do have a poor
request to try and improve it, but what I mean is this is the documentation it looks like they're
all documented, but if you scroll down there's no information, so there are definitely some issues
with this, but they are trying it's a small team. Okay, so now we can look at how you integrate
with make and see make, is an example from make, so this would be just within your test directory,
you'll have to compile the source code as well separately, but first off you include your pre-built
pfunit information, so I'm passing the path to it here, but you can do it in a bit of a nicer way,
and then you include this dot mk file, that gives you access to these flags here,
which are needed to build, and then it also gives you access to this function, make pfunit test,
and then you name your test, here I've just called it tests, and then you use these specific
variables to link up to that function, so you define your dot pfiles, you define the source objects,
so your source code that you want to test, it looks a bit complex, because I'm just excluding
the main program, because in Fortran, when you're doing these tests you can only have one
sort of entry point, so I just exclude the source main, and then I pass in those test flags
as I just defined, and something or not is that this strange syntax of test and disguise important,
it is the name of the test, if I change that, I have to change the name of the variables,
and that's how it picks it up, and then this last bit of boilerplate, which is an unfortunately
I have to do, is to convert the F-90 files that the pre-processer will create and convert them into
object files, but once you've done that it then picks everything else up, that seems to be the
only bit of thing missing from the function, so now a CMake example, it's quite similar, we include
pfunit using find package, like we do with make, and then we get access to a function,
add pfunit ctest, we name our function, we define the source files, the test sources these are,
so this is the dot pfiles again, and then we link to our source code that we want to test,
again, excluding the main, and that's all you need to do for those, so then how can we now use this
in a CI pipeline, for example, an automated, in this case with GitHub Actions, so I'm using
containerization, most specifically Docker, so within a Dockerfile you would install and build any
dependencies for your project, build the source and the test codes, and then run the test, thank you,
something that can help is by wrapping, slow changing dependencies in a parent image, for example,
pfunit itself, you could build that once and then just pull that parent image and have that at the
start, we'll see an example of the Dockerfile, but something that can make it difficult is if you've got
proprietary or software which doesn't link with say new compilers or things very well,
so I'm having a problem in a current project that we rely on a licensed library which mixes
on able to use it in Docker files, that's a little problem, but there is an example of me doing
this in this repository, and once if someone is interested, you can have a look. Okay, so an example,
Dockerfile, we start with that parent image, so I might have prebuilt pfunit in here, we copy the
source code, including the tests, we then build the CMake, and to get it to work with pfunit,
we specify the CMake prefix path and then that would point towards our installed version of pfunit,
and we just build everything goes well, and you've got a test, and then you just run c-test,
and because this is the Docker image and if those tests fail, the image won't be built,
then I know that if the image is successfully built, my tests have passed, so then using this
in a GitHub actions workflow, for example, if I wanted to do this on a push to main,
or the default branch, a little thing is I need some permissions, because I'm storing the image
in the GitHub repository, the GitHub image repository, I can't remember what it's called,
but you need those to access that, and then you just run Docker build, and as long as you pass
the path to the Dockerfile, if it's not the default name, then this goes green, your test is the
past, so it's just a small example of how to do that. Some future work that I am going to be doing
is a workshop on how to unit test fart run, which we're going to more detail, so this is going to be
in the style of something called software capantry, which people may be familiar with, but it's
kind of an open-source example, especially in academia of how to write lessons and workshops,
but there is an available website for the software capantry, and yet the workshops on the
16th and 18th of February, if you happen to be in a London, which is where I'm from University
College of London, sign up, it's free, the material written so far is available at that lesson,
I've also developed a repository of exercises, which will meant to complement the lessons, so you
can do those in the workshop, and I'm hoping to expand to including automated testing, which I haven't yet,
lintes, so there's some fart run specific linting tools, which have been developed, which are
quite good, some documentation generation, including the ones specifically for fart run, which is
fart, but yeah, some conclusions, so fart run, I think it's still relevant, however, it does
lack some unit tests in a lot of popular projects, I recommend that it will use pfunit, again mainly
for the MPI compatibility, pfunit uses the pre-processor, assertions are provided for various things,
it integrates with make and see make, and you can automate your testing in a CI pipeline if you've
got not to proprietary dependencies and things like that, yep, and just some acknowledgements,
I am not a pfunit developer, so I want to point out the people who are, Tom Clown in Mac Thompson
specifically, and these slides I learnt how to do them because of party runny, who I work with, but
if you're interested in how they're made, and there's the QR code for my slides, so thank you
very quick question, is one there, yes, I think that's, if anything, the main problem yes,
so the question is, fart run developers, I guess in that your opinion, don't know,
are often a bit older, and it's a difficult to persuade them to do unit testing, I believe that was
the question, yes I found it is, slightly, although not everyone, not everyone's the same,
some very keen to give it a go, and this is why I'm doing things like this talk, the workshop,
some trying to ensure that it is possible, and that you can do it, and that it is worthwhile,
rather than just saying, oh I do an if statement in my code, so therefore I know it works,
because I've checked the value as it's running, which doesn't make sense, but that's the kind of
arguments you get, so yes, I do have those problems for it. Yeah. Thank you.