Okay. Thank you. So, thanks everybody for being here. What seems to me to be extremely
late on Saturday night. I'm going to talk about research software, sustainability and
RSEs, and kind of in the context of one project, and with a few kind of other things
along the way. And I'll mention the second author, Ben, is also here in the audience, and so
if I tough questions, I'll see if he can answer them. Okay. So, first, just to start, the
project that I am part of that inspired this is called parcel, which sometimes stands for
parallel scripting library, but not really officially. But it's a parallel programming and
Python library, and it has the idea that you have apps that define opportunities for
parallelism, and they're basically decorators on Python functions, or on bash apps, external
applications. And these apps then return futures rather than values, and the fact that you're
doing that, then let's them return immediately, and so you can do them in parallel, and
you can have them run remotely, in some cases. And in the fact that you can do this, then
let the runtime basically figure out what concurrency is needed, what is ready to run, what has
to wait because it's waiting for something else to run first, things like that. And so
this lets you not have to specify a lot of the parallelism by hand, but just the dependencies
can calculate this automatically. And this also, at least in theory, is independent of where it
is running, because the code doesn't say anything about where this happens, we say where things
are going to happen in a separate object that's not related to the compute parts of things. So,
anyhow, so that's just a little bit about parcel, if you're interested, you can try it,
there's a binder instance, so you don't have to download anything. Okay, so we wrote this,
the software that does this, and we found out that it's being used by a bunch of different kinds
of people. So, one group is individual researchers and large research consortia, and an industry
to some extent, and across a lot of different scientific and other domains. And so some examples
of what parcels been used for was to produce an interconnected simulated sky survey in preparation
for an astronomical telescope that's coming online soon. And some biomedical applications looking
at, well, basically looking at genomes and looking at molecular structures. So, it is a bunch
of different things that can do. It basically tries to do things, or it's most useful when you're
trying to do something that's at a very large scale. It would be hard to do on a single computer.
And so, we also then have external stakeholders. And so, these are, in some cases, they're the
direct users, like the astronomers, the astronomy project, for example. In some cases, they're
platform developers. So, we have different projects that are using parcel as a component,
and then they provide that platform to their end users. And so, parcel is not directly used by
their end users. And a couple of examples for this is QCRC hive that's working in molecular
sciences and in globalist compute. There are also a cyber infrastructure, or E infrastructure,
or E research infrastructure providers. So, national labs in the U.S. like Argonne and Nurse,
for example, support parcel on their systems. And so, they are part of our stakeholders.
There's also linked contributors. So, because we're building a, we're providing a library
that's useful in a lot of different cases. Other people that are providing their own library,
sometimes see wanting to work with us as a way of increasing the impact of what they're doing
and trying to share work to do something bigger. And so, there's a project called WorkQ that was
developed at University of Notre Dame. And, and they basically have integrated this in some sense
into part of parcel, which makes people that are using parcel able to use workQ and adds to
workQ's usage as well. And then there's also funders. And so, for us, national funders in the
U.S., NSF and DOE, Chan Zuckerberg Initiative, as a philanthropic funder, and then collaborating
projects like the astronomy project that gives us a bit of money to support their own needs
as well. So, I've talked about parcel, but let me kind of do a little bit of the history for a second.
This was initially supported by an NSF Award, a five-year award that we got out of a one-year
no-cost extension for them, and from 2016 to 2022. In 2020, we released version one, and we have now
moved on to weekly releases since then, rather than version releases in the same way,
semantic versioning. The focus since that version one has mostly been on maintenance rather than
adding new features, so we do a little bit of different stuff now than we did initially.
And I'll talk more about that as we go on. We had a funded development team that was somewhere
between two and four people or FTs per year when we were doing this building initially.
And currently, under the funding that we have now, we are basically supporting about one FTA
year of maintenance and development in about half an FT a year of community management.
And I'll just point out that there's kind of a few different metrics that you can look at,
and our metrics are going up, and that's good. So we have lots of downloads, lots of website
visitors, lots of users, which are these you believe are actually indicative of anything,
I don't know, but we show them to our funders and they're happy. Okay, so sustainability,
then, if we want to talk about that part, the way that I kind of define sustainability is that we
want to balance resources and work. And so the resources that we have are grants,
as I've talked about external funding from projects that use parcel, a trail said,
volunteer effort from groups that develop tools that use parcel, where we're not really
directly involved. In some cases, in other cases, we are, and then companies that use parcel
in their services. And then, on the other side, we have the work. And so the funded parcel team
is doing this core work, which includes managing the community and reviewing code contributions
and fixing bugs and supporting users and developing new features, although not so many of those,
at this point, and releasing new versions of the software. And so, if we're sustainable,
basically, these columns match up. But one of the challenges is that the provided in the volunteer
resources can add features to parcel, and they can support some use cases, like somebody
can support their own use case pretty easily, but they're not as likely to support somebody else's
use case. And so, the challenge is that those provided in volunteer resources aren't currently
sufficient or aligned well enough to fully support what we need over multiple years. And so,
we don't think, at least at our scale, that we could survive without any funding just based on volunteers.
And what we've tried to do, basically, let me see if I can, can I do this as well?
Yeah, okay. We basically had this period where we were getting this initial funding. And we're doing
lots of development, a fair amount of user support, increasing maintenance was increasing,
not so much research and development, and the number of users was slowly increasing.
And what we kind of pitched is that we got into this period where we started ideally wanting to do
less of these things, less development, less maintenance, less user support,
or at least have less resources going into those things and do them more efficiently,
but have some R&D that continues and have the number of users grow.
And we're hoping that we can get into some kind of a sustainable period where we don't need a huge
number of resources to do the maintenance part of this, and we can find those resources.
And that's the question, as can we, can we actually get to there?
So in terms of actually then doing this, we see that there are projects that are successful
and we'd like to be able to follow their model. And so Astropy in astronomy is one that's,
I think, is a good model. There's a challenge that it's much bigger than parcel, and I'm not
actually sure if that's a qualitative difference as well as a quantitative difference. I suspect it is,
but we'll see. And YT is another project that we also were looking at. And so then we will work
and are working on community and governance and funding streams and innovation and training and
outreach and engagement, trying to improve things to get to that point. We are trying to work with
other related communities, so there's lots of other people working on software sustainability,
the SSI and the UK, research software alliance and others. And so we want to kind of share
work with them as well as the workflow community, because if we can bring together more community,
then we have less work that we have to do ourselves and ideally becomes more sustainable.
We also, in our proposal, said we were going to capture endocuments and share sustainability lessons.
We've done a horrible job of this. And it's partly because what we said we were going to
do is to track everybody's time on all of their activities and be able to report on what we did
to increase our sustainability and it's really hard to track time at the level that we wanted.
And we could never find any good tools and we tried to do things manually and then people forgot
to do it and then we try to remind ourselves every month and it just, it was a mess. So I think we're
going to end up kind of failing on this unless you count this talk is as that and then maybe we're
successful. Okay. And the other thing that we're also trying to do is to reduce technical debt and this
is what I would say has been a lot of what Ben has been working on. So if we can make the code simpler
and make it easier to maintain than we don't actually need as many resources to continue maintaining it.
So all right. So let me kind of go on and talk a little bit about how we've done this and where we are.
So we kind of think of the project as having these stages and I think this is probably general
for a bunch of other projects as well where we had an initial concept testing and an initial
development kind of proof of concept effectively. We then started growing a little bit doing some
testing with with initial users and continuing development to move past the initial stage.
And then we got to expanded usage and support and still a little bit of development but not really
anywhere near as much. And then we think we're hopefully getting into this point where we're in
community usage and some sustained maintenance and supports but not very much development at this point.
Okay. So these four things are kind of not exactly completely distinct from each other but we feel
like we've kind of progressed through these different stages. And so how we did this in some ways is that
this partial started as an idea in 2016 based on a previous project that was probably 10 or 15 years old
where we'd done a lot of work and a lot of thinking but that code was all getting very old
and it was written in some bad ways and it used a language that didn't really make any sense anymore.
And so we basically said to ourselves if we were doing the same thing from 15 years ago we were doing it
today what would we do differently and we thought well let's try that and see if we can do that and see how it compares.
So effectively a simple tool or language in a runtime for fast easy scripting on big machines.
And we realized we should do this in Python today that they would make much more sense than what we were doing.
One person basically did the initial exploration and proof of concept and proved that this worked.
And in this one person had about four people that were kind of managing and helping them which I guess worked.
And they built the initial usable system and they were the main developer and they did things the way that they wanted
because they were the main developer.
Once we had a second developer that became active the two developers then needed to agree on and define
and document processes and that actually was kind of an interesting process because they didn't have the same things in mind.
Yeah so kind of going from one to two is a remarkably huge change.
And then as we moved to this more open community process again two to four of TA year of developer like
73 contributors at the time but we finished this initial work in 2022.
These processes became more important because all these community members had to know what they were going to try to do and how they were going to be able to fit in.
And so these processes include things like how do we make design and architecture decisions, what coding style do we use,
when what testing is sufficient, what documentation is sufficient, how do we actually engage with and support users,
what properties do contributions and changes need to have to become acceptable.
How do contributions and changes actually get accepted then, how do we encourage and develop contributors,
how do we mix CS research as well as software product development which kind of have completely different goals often.
And then things like who writes papers and who's listed as co-authors on those papers.
And so all of these things we had initial answers for and all the answers I would say changed over the life of the project.
And so the thing that we're trying to do then is to be consistent not about what the processes are but be consistent about documenting the processes.
Okay, and so then to look at these two different things then the developer work has changed so so as I've said before,
we are currently looking at maintenance and how reach and support that's a lot different than earlier in the project and so we need different kinds of contributions now.
And the development activity then includes maintaining the code base, adding more tests, responding to issues, supporting development on different resources.
As the community has grown, the number of use cases has grown and the range of challenges has grown and part of this is because every HPC system is unique in some way.
And so we don't have, so we have to kind of worry about different configurations, different operating systems, different hardware.
And then also reviewing contributed code and so this is led us to develop minimum requirements on that code starting ideally with the pre-coding discussion and then talking about plans for future maintenance and what's if we accept this what's going to happen to it, what are you committing to do.
So internally then we have different developer roles over time so we had initially this research developer research programmer focusing on doing things quickly what the software can do more than who could actually use it.
This person really is essential to do new research software projects do the initial device development testing, but they could take shortcuts that are going to hurt the project's later sustainability.
And so this is really important in this initial development and maybe is looking at new features and later stages and then we get into a software developer at a later stage and this is a person who really focuses on developing professional class research software.
They focus on the software itself and it's users rather than new ideas they want to explore or something else.
They're dedicated to making the software as useful as possible, making it clean and relatively beautiful, increasing simplicity and compatibility and future maintainability and reducing technical debt.
And it's important to have this kind of a person involved in everything except maybe the first stage because these processes might actually impede the development to the the initial thinking that we need to do.
And then there's also a user developer.
So this is somebody who's main job is as a scientist or a disciplinary researcher and they maybe are adding some features relevant to their own work.
They're focusing on their own usage, their writing code somewhere else and so they may not be writing code in the same style that we want which can be a challenge.
They can take shortcuts that harm the project's future sustainability, but they're important to have because they're the users and they bring in new ideas as well.
And then finally as the collaborating developer and just because I'm running a little bit short on time, I'm not going to go into this in great detail.
Other to say that this is somebody whose whose main job is developing their own software for their own project and somehow that has to be integrated into ours and so there can be challenges in terms of differing styles or differing code contributions.
And if we want to be sustainable we really need to define the interface to this kind of person as well as we can because again this is the the resources that we're not paying for.
But are coming in to help the project be sustained.
Okay, community work also changes so in addition to the developers there's members of the community that do other stuff.
And this is stuff like answering user questions which actually happens sometimes down to managing social media which never happens.
So the range of these different things are all the things that need to happen from how likely it is somebody external to the core team is going to do them to how unlikely it is.
Somebody external from the core team is going to do them, but we need to do all of them so the challenge then as I said before is that volunteers and other people can do some of these but they can't do all of these and so we need still some some way of doing some of these other ones.
And specifically or particularly one one way of thinking about this is I'm sorry that Abby isn't here who gave one of the first couple of talks in this in this room earlier so I kind of borrowed this from her.
Is that we think of community as having different stages from discovery to contact to participation and all the way up to leadership hopefully.
And at each stage there's a way that people get involved and there's kind of ways that we can try to promote them and the community manager I would say one of their main job is to kind of push people up along this path as they can and to make it easy for people to move along this path and again that's something that somebody has to do it doesn't just happen automatically.
Okay, the last the last thing yeah the last thing I want to say is kind of a new topic is num focus so num focus is a.
An open source software foundation kind of an umbrella foundation we joined num focus in September of last year.
So we are a fiscally sponsored project under num focus now and the process for doing this involved first we applied and then we are accepted and then we needed to sign a financial sponsorship agreement.
The core work in parcel was developed at University Chicago and University of Illinois and so they own the the copyright on most of the the code because those of us working for either of those don't actually own our copyright because our employer does.
University of Illinois agreed to transfer the ownership of its IP to us so that we could transfer it to num focus relatively quickly a couple of months.
Took them a little bit of time to understand what we're trying to do but once we had a call and they understood it they were fine with it.
University of Chicago took about nine months first they didn't understand it.
Then they needed to have lawyers involved and then those lawyers didn't understand it and then they needed to have more lawyers involved.
And and then they didn't understand it and then so anyhow so there were lots of discussions and the lawyers didn't actually use email or calendaring they went to do everything over the phone which was very weird.
So but eventually they did agree to this and so and so we see being part of num focus as part of our sustainability plan.
In particular because ownership of the project in a neutral place we think encourages others to move up in leadership and governance.
It's not something where these two universities are going to be in charge forever other people that want to come in and have a say in what happens or more than welcome to and.
And we think that being in num focus actually helps do this I'll just mention quickly there is a funding devroom tomorrow and HPSF the high performance software foundation is going to be giving a talk there which is another one of these foundations so if you're not familiar with this that will be an interesting talk I think to hear.
And the other thing is the num focus actually gives us some mechanism to hire staff outside the US one needed and to contract for specific work items to the best available person regardless of their affiliation.
So we don't have to have everything go through the universities we can move money around more easily and we can move money outside of a particular country as well which is relatively important.
So where we are now there's some good news which is good community growth and that to do at least in part to the community manager lots of contributors lots of users I think all this is quite happy.
The code is gotten better thanks to our core maintainer at least in part.
We're moving more to plugins to reduce what the core code has to do removing old code that isn't used or doesn't work or things like that and having more and better tests and I think this is all very important.
But on the less good news side we actually are not really sure how we're going to sustain the community manager and the core developer and I feel very awkward saying this with Ben in the room sorry.
But but again this is core community and maintenance work is really harder and possible for for us to rely fully on volunteers to do.
And so at least for a project of our size I think if we are as at like 10 times the size we'd be in a different situation.
But where we are I think this is a challenge okay so finally so lessons so sometimes some of the choices we made are just choices and we made them for the sake of having made a choice.
And and once you make a choice then it can be hard to change them but it's important to I think consider these changes regularly and particularly when you get new developers coming on becoming part of the core team that's a good opportunity to think about this.
Going from one to two developers was a big step and again it's not opportunity to think about changes.
Research software sustainability is a hard problem and there are no simple answers.
The existence of these different types of developers and their utility during the different phases.
I think was interesting in this emerge during the process and I would be interested if this matches other people's experience and if anybody's in computer science research.
I think there's some interesting research we could do to find out if this does match other processes.
Again yeah let's piece and then finally the the boundaries between the types of people on their roles are kind of fuzzy so different people can have different roles this isn't a statement about people to statement about roles so.
Okay and then I'll just close with having some acknowledgments up for a second as we end all right so thank you.
Thank you.
One question the most motivated guess it is you.
Yeah so question yes I'm trying to yes.
Okay so the question is about the the possible role industry can play I think I can imagine industry playing actually a pretty significant role.
The challenge I think that we have is that what we're doing is potentially relevant to some industry and and we do have one kind of industrial partner that we work pretty closely with and they have been.
helping with some volunteer time from their staff that actually have done some nice things.
If we I think if we had done a better job or if we will do a better job of selling to industry.
I think that that would actually help and and again that the HPSF model that that's like that other foundation is much more focused on things related industry but they're building code.
That computer vendors rely on to sell their computers and so there's there's a much stronger incentive we're providing am a library that people can use but it.
It's not the only library we think it's probably the best in some situations but in other situations there's probably a different one that might be better so it's.
We don't have that kind of I don't know and somebody's I don't know that I would say we're the killer app for industry in general we in some particular problem places we are.
But as a general statement I don't think I can say that and so that makes it hard to make that case I think to industry but but that's part of what we're actually another piece of why we're interested in.
We're interested in joining them focus is because of the industry connections and we've had some discussions already so it it may work I don't know.
Thank you very much.
Okay thank you.