Hi, so, yeah, in research, it is actually common, right, that the paper builds on
previous work, so this can be done with numbers from tables or getting model parameters
or reference data set. Despite now, all the open source initiatives and open access
nowadays, reuse is still hard and requires, but it is required to have the results such
that they can be made reproducible. So, what often happens in practice is that we still
have to extract values from tables or get the data from the figures, especially in the modeling
fields to then implement or implement analysis because they are not there or only half
shared. So, it is not because the other researchers don't want to do that, but it is
because the original research is not shared in a reusable form. And even when the data
and the code are shared, reproducing the results requires to reconstruct the environment
that the original authors have at the time. So, the problem we face isn't that we don't
share. The problem is that we publish research as static artifacts. So, thank you very much
for the organized, great organization and opportunity to present my name in Andreas. And
I want to talk in the next few minutes about how we can turn research outputs into executable
and reusable project using open source infrastructure and RAP as one concrete exam. So,
to start and very hard today compared to 10, 20 years ago, where most of the information
was contained in an article. It has changed a lot with open data becoming widespread and
thanks to repositories such as senodo or fixture, the data is accessible, but also because
of funder requirements and new policies. The same is true for code, sharing code on platforms
like it lab is now common practice in many fields. And yet when people actually try to run
published analysis results still don't reproduce. So, this is not a story about no data or
no code. It's about the fact that data and code is not accessible. And even if it's accessible,
that's not enough to reuse and reuse the research. So, let's see why this still happens and
very breaks down. But first, maybe to bring everyone on the same page, there is repeatability.
And this means that the original researcher can rerun the analysis and generate the same results
repeatedly using some data and some code of locally. Next, we have the reproducibility.
This is often used in a live computational reproducibility. When we are talking about some
analysis workflows and pipelines, here for example, researcher A, share state and code with
researcher B. And then in researcher B's environment, we try to reproduce the same results.
Then there's the replication or reuse. So, here we share some code with researcher B and
researcher B tries them to extend it with new data. So, we say a result is generalizable.
If this is possible with new data. And this last step is actually what researchers actually
want. But this takes a lot of time and effort to achieve. So, still today, researcher A
B is able to do that. So, it takes time to publish fair. The concept is around since 10 years
now and the data should be findable, accessible, introproble, and reusable. But so, what happened
is that we have now the policy expectation that if we publish a manuscript with the data and
delete the eLeCode to a data repository, science magically becomes reproducibly. It's true
that such data is then findable and accessible. But not by default, introproble, for example,
proprietary file formats were used at the time. And reusable only in the sense that we have
now access to it and someone can use it. But the researcher experience is still that
methods description, for example, are scattered between the manuscript supporting information.
And the DUI to a data repository just leads to one folder with a bunch of files, which
is also true for code, but there at least we have a history how the stuff evolved.
But then the questions we need to leave for a researcher rise like, how do I rerun this?
What environment is needed and to reproduce it? So, we are missing the context and
execute the ability. But research artifacts they don't just exist, they evolve.
Guided by a research question, they evolve in different cycles. So, data follows one cycle,
for example, from planning, acquisition, or processing to a one point archiving. On the other
side, we have code that follows a different life cycle from development, testing, publication,
and also maintenance. But these two life cycles are managed by different tools, different
systems, and often out different people, such as collaborators or maintainers of third-party
libraries. So, what we are missing is a mechanism that binds a specific version of the data with
a specific version of the code, under a specific execution environment in the current
system. So, now even if both are open, they don't form this unit. This may now sound like
a publication problem, but it's not because it fails much earlier, inside the lab.
Many of you may have seen such messy folder like this semester project, and their data
code, lab nodes, and different versions of the report are just something to one folder with
your parent structure. But this is not necessarily bad practice. This happens if the tools don't
capture the context. Similarly, in larger research projects, for example, this lab tries to
study how protein acts, regulates the cell cycle. So, now the whole team works on the same
project, it's exactly the same data, same tools. But here we have, for example, a student
working on a mac laptop, a recent Python version, and creates some results. But then it
starts to differ, not because the student made a mistake, but because assumption about the
environment are implicit. So, if the student leaves at one point, and it's a postdoc tries
to reproduce the study, but now runs on a Linux system with slightly older Python version,
suddenly results slightly different. Then, say, a PI tries to help on a Windows system with
an even older version, and just gets there. So, the question is then, is it now the Python version
that causes you the problem or the pre-processing steps run manually that no one captured
to sort this out on a long term. It's very difficult, time consuming, in a completely inefficient.
And, these differences, they all happen and slowly accumulate long before the papers written.
And even if you have the in the paper, it doesn't fix the problem, it just freezes the problem
into a snapshot of the PDF. So, we've seen, I mean, while sharing data, code, alone does not automatically
give for a possibility, but I also want to stress the point that reproducibility should never be the end
code. It's the minimum requirement, and questions like, can I rerun this? And does do the result match
the paper? They should be in certain ways, if a clear yes. Upon publication. However, a recent study that
looked at around 200 random selected paper from the journal science found that they could only
get the data of 44% of it, and then only reproduce the result of roughly one quarter.
So, this is reset, as it is today, and is extremely far from the researcher and the
new research community bonds, to reuse the work and ask new questions.
So, when data and code of the study are reproducible and give the same result, we can then finally
work on extension, for example, developed new analysis methods, use such data for benchmarking, for example,
of new algorithms, and that extend work, bringing in new data, reuse their analysis,
flows, and address new questions from a different angle. So, with open source code, the environment
is assumed to be obvious, but it's not, it's really the key missing dependency. From this
execution stack, we see that code runs on some infrastructures or a laptop that then runs
some operating system that comes with some libraries, and then on top, you have your packages
of code that you write for your research to process some data. Traditionally, this stack is implicit.
It lives in people's laptop and disappears of the publication, and is relevant, and almost never
published. What are the other options? So, bare metal is not portable, but systems can be
virtualized, however, this is also heavy. So, then there are containers, and they made the environment
portable and executable, and have been used in research projects, but are very complex to set up.
And finally, binder used this idea to make repository executable, but sadly, it's stateless.
But long-lived research involve data set in access control, and needs to preserve this over a long period of time.
So, we've seen how binder can help now to make an environment to take it into account,
but we haven't talked about how data is managed. So, it lives within a folder, even if it has DOI,
but they don't have identity and code, however, produced. On the other side, we have a research
management system, where data sets are treated as first class object, and it has persisted
and identifiers, metadata, explicit provenance, and this turns a collection of data into reusable
research objects. So, in our work, we use the open source, RMS tool called open piece.
It's used in many institutions and scales to terabytes of data, and is specifically designed
for academic and lab environments. So, here we see laboratory information system.
We have some methods and protocols, and then you have a tool section with clear APIs,
how to get the data in and how to get it out, publish it to open research repositories.
Then here we have the lab notebook section, we see raw data, an example of microscopy data,
where we can describe all the details, and more importantly, link to the parents,
so like which materials, chemicals, and so on have been used, what was the goal, and then link it to analysis.
These relationships can be searched, so you see which chemical,
let's to which modification of the biological strain, and then this strain is imaged on the microscope
and leads to the data, and then eventually which analysis produces a result.
But the system has no execution context.
So that's where RAP comes in and it reaches the data, and code life cycle that we saw earlier
to generate an open shareable unit of research around the question.
So we need to have the data by reference, the code in version control, and we need the compute context.
The data in our system lives in an RDS like open piece, and the environment and the code is on the version control.
So RAP then takes this and builds a project where one can do the whole management,
you can collaborate on it, and you can share this whole environment.
So quickly on the system architecture, a user just interface through a web browser with it,
and the environment is built on a Kubernetes system by taking the repository, building a Docker image,
mounting the data directly into it, and then has means to publish it on different repositories.
So in practice, this looks right. We assume the data to be either registered or generated by the lab automatically in the argument system.
You just specify which is your database and what data sets to use.
Then you specify your environments of which dependency which packages,
but you can already specify what kind of analysis you want to do, but this you can also do later.
And then you have an example, a project like this, so the environment lives on a bander folder,
the data set in RAP folder, like the data sets, where you keep track of it,
and the rest is completely free for you to manage and organize and depending the needs of your fields and projects.
One year you reach a milestone, you can take the whole thing, freeze it, and register it to the argument system,
and eventually publish it to an open repository, where others can use and run it.
So here, if you use the interaction with the TI, so you can either build it from scratch from the gift or reuse it through OpenBeeshare 25.
You can define the computational resources you need, memory RAM, start, stop delete your project,
you can mount new data anytime and it's immediately available within your project.
You can upload intermediate data, if necessary, or save the whole thing, and you can share it within your institution or with the world.
So you have a container that contains the data, the code, and the environment and runs anywhere.
So here to case studies, this is from the field of archaeology, the first registered data in the RMS system,
then added the R packages that the study developed as a kit submodule to make it available, adopt the path to an Rmarkdown script,
and we're able to reduce the results.
Here, another example in the field of neuroscience, there they had data code in a kit repository, but the data they fetched from data bases.
So this we could just use, but for example, didn't specify the execution context, so we had to find the right path in version and the library,
but once we saw it to this out, we were basically able to rerun all their analysis script and generate the results.
So these are two examples from two different fields where you can now reuse that system, trace everything back and build on it.
Now, RP adapts to the researcher and not the other way around.
So this is important because we all know, with time, we have our favorite tools, and we are less open to adopt.
So we used open source, and took it to lab interface that has a huge community and supports many languages for analysis.
Computational scientists prefer our studio, but this is also accessible, engineering engineers like MATLAB, despite its proprietary, but even that runs on the platform.
Or if you're coding, you may like visual studio code or similar with a debugger, but then if you really like GUIs and like to click around, you can have a full Linux desktop environment with your
Libro office or only office and here we see, for example, some data mounted into it.
So once we have an RP instance running and the user defined their environments with the data and tools, the user experience for the user is almost like on the laptop.
So it can be installed on local infrastructure, such as lab server or institutional infrastructure.
Small labs can manage it themselves, but also for the IT department.
It's low maintenance and scale the RGMS system to use data, handled long-term preservation, enable cheap tape storage, or provide this high-performing computing resources.
All this technical complexities hidden away from the users, when interacting with the system and requiring more computational power.
Even after the export, it contains everything necessary to run anywhere and it integrates well with the established tools and services such as LATCH.
So once we have the environment specified here, an example on a institutional server, we can bundle it and now this is how it would look on the local computer,
and we get exactly the same visualization, same tools, and the user just needs to run it.
Thanks to the rich open source ecosystem in TupiT arrived, we can use it in many disciplines.
Here, engineering, 3D design, PCB design for electronics, in our group we work a lot with large image data sets, terabytes in size.
To segment, we can do that, use deep learning for segmenting images.
Maybe not everyone is fluent in coding, so we can link it to LLMs, ask for help.
For example, Gemini, hey, can you give me an example in Python, how to count the cells on my images?
And yes, it speeds out some code and found the 47 cell on that image.
It can even use in molecular biology to make it more reproducible, because there's still a lot of stuff is done manually.
Here we load all the strains, the sequences and so on from the database, can simulate the processes long before, for example, using automated systems to then construct the strains in silicon.
And finally, at the end of every research project, there's a report summarizing the work.
Here, I show you, overleaf, a latex editor, where we can start a project, sync it as a git sub module into RP, right for example, an introduction, and sync it back, and use synergies with such existing tools, for example, to publish it to a preprint server or to the publisher.
So RP not only unifies data code, but it also includes the written documentation, making the publication an integral portable part of it, so it runs everywhere in future.
So what's next? We have plans to integrate it with the European Open Science Cloud.
We plan to engage with the communities to allow mounting, not downloading reference data from senor to for example, and use LLM assistance to lower the adoption barrier even more.
This work is the result of a close collaboration between software engineer, domain scientist, and the scientific infrastructure team at ETH, which I want to specifically thank and highlight here.
And the early adopters in our team, who built now their workflows on this system, and the funders to make it possible to develop this platform as open source infrastructure.
So to conclude, RP is one example how this can work, but the broader point is not RP, so research should really be something that we can run, inspect, and extend anytime.
So that's how we move away from reproducibility as the exception to reuse as the default.
The cure codes link to the manuscript or the demo if you like to install it locally on your infrastructure or laptop.
Thanks for the attention. I'm happy to take questions now.
Thank you.
Yes.
My question is about the runtime. Sometimes when I want to reproduce some experiments in physics, sometimes there are repositories very old.
And some packages don't even exist anymore to download these data through for this in our view.
So the question was, in physics, there are sometimes really old projects without data packages.
So obviously we cannot, there's no magic right if the package is gone it's gone right.
So but that's why we kind of emphasize to build something like or use something like RP because then it's never gone because once you export this bundle it's all there.
So you have the system that kind of recreates a computation environment and it includes all the libraries, even if they're long gone on the repositories or by the maintainers.
But to basically find all the ones it's impossible, right?
So when you use RP it creates a package by all this.
Yes.
So someone looks at it 15 years later.
Yes.
So it handles the code, the data, the environment potentially the documentation.
So everything that was really related to this research.
All the dependency everything, but there are different export options.
You can also decide to share a lightweight, right?
If you have terabytes of data, then we have a mechanism that you basically only bundled the environment and your analysis code and the data is then downloaded from, as in order for example.
Because otherwise these bundles, it's basically a zip file, get very large.
But for smaller meat size project, you can really bundle it, you have one big thing that you download.
You press start and in the browser it looks exactly as what I showed and you can run.
It's as someone stopped there.
Yes.
The code, the project of the new page.
How from 2DL and we actually have we run the code sheet project.
You talk to a reproducibility.
Yes.
You've already got a publication of earlier.
You talk to somebody else.
Is there something in here?
There's a little process.
This one read.
What does it take on that kind of process?
Of course, or intense manual labor involved?
How do you compare that to what you basically have here?
So the question is if I'm aware of code.
Check.
So no, I'm not.
So then I cannot comment on how it compares.
But any initiative, right?
I don't tell you now everyone should use RAP, but what I'm saying as a field in the open research community
we should move towards tools that kind of enable us like reproducible research.
And it's really not just putting the code somewhere.
But we have to have the environment.
We have to have the data.
And as we heard before, we need also the packages because it is common that some lose the maintenance
or they just get deleted.
So if we want to make and preserve this work of research for far future, we need to bundle it everything.
Here we can do it.
Maybe if you're too, as well.
Then that would be great.
Otherwise maybe you want to integrate it.
Yes.
So there's your compiled fund where there's an includes sort of the things you've got for
file because of course the issue is sort of sort of a finished document image.
I plan repository and I run.
It's like an act.
It's actually using apps.
So what is in underlying repositories?
There is changing even in this.
I type Python 3.10.
So we got kind of 1.2.
I think it's a data.
It's a different data.
So are you outputting like the SHA256 check-up that results when you build a doctor image?
So the question was if we have basically the image and if we have some kind of check sum to check if I go
the correct if everything is still available.
So yeah, so basically what we share or publish as OCE to OCE containers like Tokrap is the full image.
So that's everything.
But we can also publish the whole container to the node.
It's then it's just the tip file, right?
It's on the check thing.
Maybe you want that?
Yes, please.
No, it's different.
So the question was if this is related to RENCO.
It's called RENCO.
It's exactly.
So RENCO is also an initiative developed by the Swiss data science center.
And so this is fully part and based.
And there are also just, you know, a user or no, from this side.
But if I understand the correct list, what you do is you have kind of the system and you can put into the system like you can put in files and like everything.
And once it's in the system, it's kind of track.
But I don't think that they have some possibilities to have the whole container to have.
Like a computer environment really in it.
So it's always like on demand, right?
So you need to have access to the packages to the repositories and so.
Yeah.
Thanks.
Thank you.