Okay, thanks very much.
Thanks very much, thanks for coming.
It's been really interesting, I think, being in this room today.
And also, I was here in the early last year.
And there's a lot of different aspects of funding of open source.
And I think this aspect of how research software gets funded is a little bit different
than some of the others.
But it's probably just my personal view, because I think probably everybody is going to say
that the way their software is funded is different than all the others.
But I'm going to go ahead and start talking about this research software,
specifically.
I just do want to mention also that there's two co-authors, Eric,
Johnson and Michelle Barker, neither of them were here,
but they were contributors to the work overall.
So just to start off, it's worth introducing research software,
because it's a part of open source.
And sometimes it's not open source, but mostly it is.
There are a group of us that in 2021 end up finishing a long discussion,
where we tried to define what research software meant.
And we published a report on the process with a bunch of different examples.
And basically what we said is that research software is source code files,
or them scripts, workflows, et cetera, created during the research process
or for research purpose.
And people that are doing research are also using other software.
They're using things like operating systems and libraries,
and their software has lots of dependencies.
And these are things that are used for research,
but they weren't created during or with a research intent.
And so we would say that these are software in research,
as opposed to research software.
And this varies between disciplines.
So if I am a computational astrophysicist,
and I'm modeling black holes,
the compiler that I used to compile my C code,
for example, is not research software.
It's software in research.
But if I'm a computer scientist,
and I'm trying to build the next generation of compilers,
that same compiler might be research software.
So it's a little bit of a fuzzy definition,
unfortunately, we've never been able to come up with a crisp one.
And so we could argue about this.
If anybody's interested in research software,
I'm happy to talk about this,
and you can tell me why this is a bad definition,
and I'm won't necessarily disagree with you.
But it's not clear that there's a better one.
And it would say in talks where I discuss this and other people discuss this,
there's often a desire to have a community effort
to try to come up with a better definition.
And I'd be happy if that happened.
It's not helpful that it's going to do anything.
It'll be much different.
So if we now have research software,
and if we talk about this from the point of view of the researcher,
research software is usually developed by a researcher who needs it.
They do it because they need it in order to solve their research problems.
It's scratching their own edge.
And maybe that's the end,
is that they develop their software for themselves,
and they solve their problem, and nothing else happens.
But maybe they also archive this for reproducibility,
so people in somewhere else in a few months and 10 years
can do the same research and understand where these results came from.
And maybe that's the end.
Or maybe that researcher then says,
okay, I've solved this problem, but now I have another problem.
And I have to modify this software to be able to solve that next problem.
I have to put in new physics.
I have to put in new method of looking at a new biomedical scanner or something else.
Maybe that's the end.
Or maybe the software is actually going to be used by other people,
which is really in some ways what we think is a community would like,
not necessarily with the person that builds it like.
And maybe that's the end.
And if we do that, then ideally we get a community that forms around that software.
We get other people that are using it.
They're suggesting features.
They're creating issues.
They're finding bugs.
Maybe they're even contributing to it with four quests.
Something like that.
Maybe they're just going out and telling other people that they should use it
and that they should work together to build it.
In this situation, there's often somebody that takes on responsibility for maintaining this software.
And I know this is kind of obvious open source.
But in research, it's kind of an odd thing that happens because
it's somebody whose goal is to do research.
And now they're not going to do research instead they're going to try to maintain their software.
And so it certainly happens.
But sometimes it happens consciously.
Sometimes it happens kind of just along the way.
Ideally it's done intentionally by somebody that's experienced and knows how this works.
And they're choosing to do this at the expense of their other activities.
Because they see this is the best thing for the community or the best thing for themselves.
And this then ideally leads to reusable community research software.
And this is, I think, where we would often like to get to.
From the point of view of somebody that's really a developer that's not a researcher,
but somebody that sees a common research problem.
And they likely have some experience within or next to other researchers that have this problem.
They develop a tool that addresses it.
This is a project that I've worked on that's a workflow system named partial that Ben Clifford
and the back is our maintainer for this is kind of an example of this.
The developers then work with researchers to test and improve this software.
They try to form a community or to develop some other method for sustainability.
And this is, again, usually done intentionally by somebody experienced.
And this is somebody that could do other things, but they feel like supporting this area of research and building this tool that's going to enable that is important.
And again, this ideally is then going to lead to reusable community research software of everything goes well.
So from the point of view of a funder, and I should say here, this is a funder who's interested in supporting the research primarily.
So this is often a government agency, it can be a philanthropy.
It can be a business if that research software is of interest to that business.
It isn't often so much, but it certainly is in some cases in the financial industry, for example.
They see multiple researchers that are working in similar areas that often would benefit from sharing software and having reusable community software.
They don't want to support multiple people who are doing research in the same area.
Each of whom is going to write very similar software for that research.
They would much rather see that happen in a community because it's cost effective for them.
And they should be supporting researchers then who choose to make their software into reusable community software.
And they should be supporting developers who want to create reusable community software.
And so what I'm going to be talking about is this situation, right?
We're aiming to get to reusable community software.
I'm not going to talk about the vast amount of research software that's built by one person for their own research, and that's the end.
It never gets shared.
Or it gets archive.
That's important, but it's not really the focus that I have.
So if we then look at the funding processes, we have the fact that resources are needed to maintain software.
That's again, I think, obvious, everybody from even from all the talks that we've had already in this session.
Those resources can come from users.
And that's clearly the case when you have commercial software or the users are paying the developers effectively for the software.
Or in theory it can come from other models.
It can come from people that are contributing their time, people that are contributing small amounts of money, grant funding.
These things are all more difficult, but they can happen.
I'm sorry, let me go on.
It can come from the developers.
So somebody who's getting funded to do the research, put some of their research time into the software, and they need this software in order to do their research.
They keep doing this.
So it's their choice of how to do this.
Or it could be that in institutions a university or a national lab or a research center,
sees the software something that's really important to them and it's part of their reputation.
And so they are going to put some money into supporting it as well.
And that happens occasionally, but not very often.
And then it can come from a third party, a funding organization that can be a government or a philanthropy or industry.
And again, this is the part that I'm going to focus on is where we've got this third party activity that's bringing in the resources for the research software.
And so today we have multiple grant making organizations.
Just the last talk was another one that I wasn't aware of, not necessarily for research software,
but potentially maybe there's a research software piece that could come out of that as well.
And there's multiple projects that are seeking grants.
There's probably, I don't know, thousands, tens of thousands.
I'm not sure how many.
Maybe even hundreds of thousands of research projects that would like to have better support than they have.
So right now, each project has to be aware of which organizations exist that could support their project and write proposals to them.
And again, this isn't specific to research software, but it is the main thing that happens within research software.
These projects and also have to be aware of all the policies of those potential funders, including rules that they may have against duplicate submissions.
And against getting multiple awards for the same work.
So they have to create different proposals that are somewhat different from each other in order to avoid having these issues, even though they're really looking for funding just to do one thing often.
And so this is difficult in time consuming for projects, and it's not really very efficient at all.
Funding organizations, which I'm just calling funders, very often, decide which submissions they're going to support.
And the means that they use for this is typically peer review.
They bring together a bunch of peers, either software developers or software users, and they get them to give opinions of what's the quality of this proposal, what's the importance of it.
And again, this is often a, within a peer review process, but sometimes it happens in a different way, particularly from philanthropies.
Some funders, and this is particularly the case for philanthropies, will approach projects directly.
So the funder says, I'm funding work in computational chemistry.
It's very clear to me that this chemistry package is being used by a bunch of my researchers, I'm going to go out and reach out to the developers and ask them what they need to make sure that that package stays around.
And that's really nice when it happens, but again, is somewhat rare.
And it's also good for well-known and well-connected projects, and it's bad for other projects, because they don't really have a way of getting into that funding at all.
And so overall, there's no coordination that happens of all these different funders, and this isn't efficient for anybody.
And then, in addition, projects have dependencies, and the funder decisions often don't account for these dependencies.
So they fund the application that somebody uses, but they're not aware that that application actually is dependent on a bunch of other packages that aren't research software and they're not being supported.
Even though, if any of them went away, there would be a lot of problems.
Okay, from the point of view of projects, so research software projects typically begin under a fixed duration grant.
In the US, this is usually three to five years, maybe two years.
Depends on where you are, but there's usually some duration after which the funding's going to end.
And the project then has to find resources for development and maintenance when the grant ends.
And sometimes the grant asks the project to give some idea about how they're going to do this at the beginning and to start thinking about it from the beginning, which is certainly better than not thinking about it until you've run out of money.
But both happen.
And this becomes an ongoing process for most software projects.
They have grant funding that's coming in for first app.
They often have some volunteers, some collaborators, some other people.
And they need to do a bunch of work, and that work includes developing and maintaining the software itself, doing the documentation, and testing.
And then a bunch of different community work aspects of community work like user support, supporting community development, doing outreach, things like that.
Small projects have times when there's a grant, and they can do all of these things, in times where there's no grant, and they really have difficulty doing any of these things.
In large projects, often can have some staff who are dedicated to sustainability, and because they have those people, they can stay in these smooth periods more often, and be more likely in these grant periods.
There's been a bunch of work that's happened around software project life cycles.
And so I just wanted to point out one recent paper that I was on that Yoyo Houdi led, and she's a person who's little out of this work.
That proposes a life cycle model for research software projects that has the idea that projects have challenge events, and a challenge event could be a loss of funding.
It could be the initial developer steps down, it could be something else.
And then there's a bunch of different things that can happen.
I just want to mention that there are kind of academic studies of how this process works, and what the issues are, and what things we need to be concerned about.
So from the point of view of the funding organization again, the funders have to understand what's needed by the research communities that they support.
So again, if I have a funder, I'll say the Chan Zuckerberg initiative that's funding a lot of biomedical work, they need to understand what software their users are using and how do they make sure that that software is still going to be available.
And in fact, CCI in 2024, had a talk where they were saying a bunch of the questions that they have again as a funding organization are what tools are most frequently used by scientists in a given field.
How does the use of open source compare to proprietary software?
Are there emerging new tools that are replacing legacy ones to help them again understand decisions they could be making?
What's the prevalent programming language? Should they be supporting the language community? For example, if there's one that's really important to them.
What tools should be part of a student's computational curriculum? Can they help make that happen?
And what software project should they prioritize as critical infrastructure for science?
Again, not thinking necessarily about just what the scientists are using, but what are the dependencies as well? If there's one dependency that's being used by all the applications, and that's probably important for them to be supporting.
Okay, and then we get into multiple funding organizations.
So multiple funding organizations are working in the same spaces often, and they have alignment challenges.
So sometimes they have divergent priorities within a particular space, sometimes they have different strategic goals, different geographic foci.
They often have timeline mismatches. One is giving out two year grants, one is giving out five year grants.
The decision making process for one may start in September and have to have awards made in December for January. The other one may be something in a different cycle entirely.
They also have issues about risk of version when they try to work together.
So multiple funders may be are concerned that they're going to lose control over their funding. If somebody else is also involved in that process, they also might be concerned that they're not going to have the same credit that they would.
But if they work together with somebody else, the one funder might say another funder is actually as a nice mechanism. They could give them some of their money and help the community.
But then would they be recognized by those projects and by the community, so maybe they're not going to do that.
Maybe there's going to be some harm to the reputation if one funder does something, and that's going to basically look bad for the other funder.
And then there's also issues about relationship and trust. The people that are actually doing this because the funders in some sense are people, in addition to being organizations.
These people have to have to have relationships with other people and those relationships have to exist for long enough that they build up trust that they can work together and they agree on what they're going to do.
There's also operational complexity. So how decision making happens may be one funder says we need to use peer review and our peer reviewers have to be in our country and another one says something completely different.
So how do they actually work together? One funder says I need to have an annual report that includes these things. Another one says I need to have a report at the end of the project that includes something completely different.
So what's the project going to do to meet those different requirements?
Again, maybe they have different requirements on budgets and operations.
They also have legal and compliance issues. So it could be that two different funders want to work together, but they can't because of government rules.
That say this money can't go out of this country to this place or something else.
And there may just be not the right legal frameworks to actually do this.
And then there's also resource constraints.
So people that are in funding organizations typically are working quite hard on what they're doing and it's challenging for them to take time away from what they're doing to try to explore how they could do things in a better way.
So there are a bunch of different types of funding programs. I guess I should have mentioned before on the bottom of the slides.
There's a URL on the URL as to the slides and there are also links from the fast and page for this talk.
And both of those also then have links to the paper that this is based on and the paper went through some review and we're in the process of updating it.
But a lot of this I'm just going to highlight that actually in more detail that's in the paper itself.
So there's different kinds of funding organizations again government that can be local, that can be national, that can be multinational industry and philanthropy.
Funders can work in a single discipline, they can work across disciplines, and they can work at different software stages in terms of what they want to support.
So someone to support prototypes and new software, someone to support enhancement and maintenance of existing software, someone to support infrastructure and platforms, and then there's other categories as well.
And then finally there's different mechanisms. So the thing that happens most often is that standalone grants are made to do the software work in research projects.
But sometimes there are people that are doing research and they're along the way, they're building some software and then they can ask for a supplement to make that software public or to start to build a community around it as well.
So there's a lot of different options here.
If we look then at how different funding organizations can coordinate there are some examples of this that have worked.
I just want to mention a few of them here.
So there's an activity that the Chan Zuckerberg Initiative, the Cavly Foundation and the Welcome Trust did together that was called a central open source software for science.
Three different philanthropies that used separate funding but used a coordinated proposal and review mechanism.
So this was a nice example that supported a lot of projects in biomedicine and underneath biomedicine over I think six grant cycles over about four years.
There's the Gordon and Betty Moore Foundation and the footprint coalition and Schmidt futures had a call that was called low cost tools for science.
Where they used a centralized decision process and they basically put all the funding together and then distributed it once as opposed to the first one were each organization picked their own things that they wanted to fund.
There's the kissed era consortium which is a European project that exists and does fund research software as well.
European Commission clearly in terms of the horizon projects is doing this in a very centralized way.
And then the Belmont Forum which has mostly been working in climate and earth science but not entirely has a way basically a mechanism for different funders to come together for particular call.
Where they each put in their own funding but they coordinate on how the decisions are made.
So so we do have examples of this happening we don't have a huge number of them.
In terms then of the coordination the decision making so any coordination between the funders is great shared knowledge between the funders can lead to better decisions and better support.
And if they can make this more active this can actually make the decisions even better.
All the decision making that I've talked about so far is funder driven and I'm actually going to go a little bit different in a different path now which I think is a little bit different than anybody that I've heard talk about here so far.
So so this option of top down funder driven is that the funders are deciding maybe using some some peer review or some internal knowledge.
But there are alternatives and those alternatives include user driven decision making and software project decision making which I'm going to call kind of bottom up in middle.
So if we imagine user driven funding decisions the idea here is that the funders want to support research.
The research projects that they're funding rely on research software the funders want to support this research software.
In order to implement this we could use something like DRIPS which I don't know how many people are familiar with DRIPS.
Okay absolutely no one that's exciting.
So you're going to learn something from this if not from me.
So DRIPS is a way that users can basically register their usage and that kind of gives tokens and those tokens and can be turned into actual funding is a simple way of saying this.
So basically the projects that the funders funding would say what software they're using they would divide up some virtual currency across those.
The funder would then look and see which projects have the most virtual currency and those are the ones that declare the the most research projects want.
And they would use that to make the funding decisions rather than using peer review or something else.
Right let the community of users decide rather than rather than having the funder decide.
The software project driven decision.
The idea here is that research software exists in different areas.
It's a chemistry, astronomy, mechanical engineering and also in tool areas like workflows or databases or other things.
And if we have projects that are working there you could imagine that these projects get together and they collectively understand their state.
In computational chemistry and molecular dynamics for example there's probably six or seven kind of key projects at this point.
And they could say my projects in fairly good shape, this project really needs some work in this area, this project needs some work in a different area.
If we had, let's say $10 million, how would we divide that up among ourselves to be most effective?
And let that community decide what the right thing to do is and then tell the funder.
We've tried to do this a little bit at a very low level in something that's coming out of the US Department of Energy,
which supported something called the XSKL Computing Project that developed and maintained a lot of science software for XSKL Computing for about seven or eight or nine years.
The Department of Energy DOE more recently created something called the Consortium Pre the Advancement of Scientific Software,
with six member organizations to support that software that came out of that original bits of funding.
And so each of these member organizations has a focus, some area, and they have an internal process by which they decide how to divide up that funding across that area in order to support the projects.
Today the Department of Energy is supporting the overall piece, but I could imagine that those high level organizations could get together and try to make a funding recommendation as well.
Okay, so getting towards the end, let me kind of recap the problem.
Research software underlies contemporary scholarship. There's almost no scholarship that happens today without software.
The funding pillars that support it are not well coordinated and they're not aligned to maximize efficiency and impact.
We have multiple agencies and multiple organizations that provide grants, but they rarely coordinate and this creates duplicate of efforts for both them as well as for the projects that have to keep applying for these different funds.
And the central issue then is really that there's a mismatch between the software projects ongoing resource needs in the limited time grant cycles that happen.
And this leads to funding gaps in maintenance bottlenecks and gaps in support for the tools themselves.
So what I've said then is that we could have new frameworks for more efficient coordinated funding that draw across organization communication and infrastructure sharing and these things are starting to emerge in a few places, but again not a huge number.
The collaborative structures that exist show that this actually can work and the multiple funders can work together and reduce inefficiencies.
And so pooled resources and co-funded calls and community different allocations can help identify software tools that have product applicability and critical dependencies across fields.
And again, these new funding models and procedures have emerged and they can distribute resources and help make decisions that are somewhat better than what we've done more recently.
Pure review is the thing that exists today mostly, but there's been a little bit of work in a few different areas about trying to imagine user driven ideas and project driven ideas to actually distribute this funding.
So and then finally, if we did have a mix of processes that might be the right way to actually go forward.
And so the fact that we have some of these now is good and it's kind of a time for funders to try more of them and a time for projects to try more of them as well.
So just to conclude, if you're interested in this, there are other groups that are looking at this, the research software alliance, which is an organization I co-founded about six or seven years ago.
It has something called a software funders forum that's looking at this.
The global research council, which is government funders has a multilateral engagement working group.
And so if you're working with any government funders, you can encourage them to join this group and look at different options.
Bringing these philanthropic government funders together would help research software projects gain consistent support to be able to evolve from prototypes to sustainable infrastructure.
And if you're able to get to this kind of alignment, we'd be able to reduce duplication.
Strengthen the software ecosystem for better resource allocation and ensure the tools that are the vital tools for knowledge creation are robust, widely accessible and continuously improved.
And I would be happy to take any questions. Thank you all for listening.
Thank you for the talk.
I love the idea of cooperation at both from an efficiency and efficiency up and down.
Do you have any concerns that as institutions cooperate more that some built-in biases will exclude projects?
And do you think that there's a model where the operational complexities and the things that are not decision making should be separated from the decision making processes so that it sort of avoids those problems?
So I think there certainly is the possibility for bias.
And I think where we are right now where we've tested a few of these things at very minimal scale, let's us experiment and see if that bias exists or not.
I've been at University of Illinois for 10 years and I was at the U.S. National Science Foundation for four years before that as a funder and so I kind of have some knowledge of different ways that funders can experiment and can try to understand these things.
And so I think those five examples that I gave in some ways are experiments and the funders then look at those and say kind of what's happened.
We did what we wanted were their biases so I think it definitely is an issue but I don't think it's an insurmountable issue that we can't learn about.
I'm not sure about the second part of that question that you asked.
I don't think I have a good answer for that so I'm just going to I'll just go on. I think it's a good question though.
Thank you for this very nice talk which point a lot of question and issues.
I'm saying this myself I write some code I know to write some code and I can include code writing in my grant but as a scientist I don't how to do anything else like write it documentation packaging code and managing a community for that matter.
So is there a ground dedicated specifically to filling these gaps that I cannot fill myself.
Yeah, so that's a great question. So there's two different things I would mention one of them is almost at the same time tomorrow in the open research.
I'm going to be giving a talk on the research software engineering movement which is kind of a group of people that have come together that are trying to do what you've said.
They're often people with software engineering experience but who also understand research that worked with researchers to try to improve the software that they're writing and help turn it into good community software.
So we have people that are developing with these skills. The started in the UK and expanded to a lot of the world and there's probably something on the order of 10,000 people at this point that would call themselves research software engineers primarily Europe and North America and Australia but some other countries as well.
So the people exist. Some funders recognize this and have specific calls.
The first call that I put up the Chan's Leckerberg lead call was exactly for this.
NSF in the US has done this. The UK, some of the research councils there have done this. So it happens in DFG and Germany has done this quite well.
So it happens in some places it doesn't happen in others. Again, this is part of the case that we're trying to make is to show funders where this does work and that they should be using these models to help their own researchers as well.
Yes. Thank you for the talk. This is kind of a lazy question. Is there a central place to look for funding calls or is it divided out by academic areas?
So there's not a good answer to that at this point. So in this top organization, Risa, the research software alliance.
We tried to kind of crowdsource this and build a, to build what basically was a Google sheet that you could search through.
It's very hard to do anything like this and to maintain it because all these grants, all these proposed opportunities are coming and then they end and then they may or may not get renewed in the year.
Because it's really hard to keep the stuff up to date unfortunately.
I'm also some of the projects I work with are working with NUMFocus, which is umbrella 501C3 charity that helps some of these research projects.
And there's a person there who tries to keep a list of the ones that are of interest to the NUMFocus projects. So I don't think that there is a single answer unfortunately.
Researchers generally kind of know what's going on in their country, but they don't necessarily know what's going on outside and.
And we've tried to work on that a little bit, but it's it's difficult.
Thank you so much for this great overview and great round of applause.
Thank you.