Next up is Thomas on research data line which will be for P.C.
Yes, okay.
I hope you understand that.
We're not going to stop.
Okay, so idea for this.
I'm not going to go.
I'm going to go in a search.
I'm 15 years ago.
We had some technical report on that.
And we also did some case.
That was I was in the system for a minute in the letter.
But it was really easy.
I'm going to explain this for you.
And I'm going to show you the structure.
You're going to work on easy.
That you may be using the device.
Please.
We can do this.
So right.
This.
I'm going to go.
So.
And.
So.
But actually the research of the one that has some kind of creative help, so I don't
know, it's brilliant.
And I can also just, for the most fun, you know, fun stuff.
Search there.
So, about there's more products, so I think this is such an instrument.
Not really, they're not as familiar as it is.
So, if you go out, look at that.
If you're not going to go out, you're still a, I don't know, whatever it is.
You're in the post of Professor, you rarely find anything about research there.
And I would guess it's the same as all the side values you use.
You have to write the statement that you've done.
You never really get anything that you've done.
Also, it's very often fun that any organization, because it has some training,
and some policies, but not much more than that.
And of course, of course, it was just fun to what they do,
and you really do the many more to satisfy the funders,
and how they just require this.
But there's actually more products through researches.
So, they have to deal with many different IT systems.
They have to deal with all that products.
They need to see, now, they just remain in different sorts of areas.
How does it develop for the also happening?
Please, intergenity of the architectures.
And then, so whatever architectures or gentlemen,
in search of an instrument, it kind of has to manage the goods,
the goods, the data, and such that this is no longer a project.
Moving data, and then, also, when the urban system,
I can actually proceed to this.
Very difficult storage space, and from the right,
it's important for this.
It doesn't make any noise, either.
So, now, if you can do the picture, you can do it immediately.
So, you can buy it, but then, it's not necessarily to have a third of all,
but it's very inclusive.
You can use many different kinds of things.
You can just have to do anything about it.
You don't have to install it, and you can do that.
And the picture was had this all in different IT systems,
because you have to do it, and they were all in system.
Also, I mean, I mean, the tools and actions you know,
something that goes very quickly in architecture.
You have the benefits available for that.
But the question is, how do we have the mentioned data?
So, the question also, how do we get this?
You want this data?
You can access data, and you can use it.
Now, you don't have to do that.
You have to insert this.
You want to avoid all these new data,
but they don't manage to operate.
And, yeah, so, why you would like to use easy,
because everyone's already used it.
No, no, no.
And, as I think easy to watch the races,
what is it?
As an easy example,
there is a thing of it that's a three-dimensional
site to fix up these conditions.
I mean, you know, for the two-dimensional techniques,
you can use it, or use it as a site to fix up this condition.
You have to use it as a single product.
And, you have to use it from the same method.
You can use it from any system.
And, if you can use it, you can use it,
and get a maximum source of the data,
and you can use it from your device.
So, we don't want to do that at all.
The problem is that you use our site there.
My business system again again.
Now, if you want to look together,
there's very good business.
So, I'm attempting to use it as a three-dimensional,
or if you want to use it as a business.
But also, it allows us to put it right on it.
Okay.
So, at least the structure values in the left,
actually, when you see this before.
So, we don't have to have a source of the S,
or something like this, and we can use it in the middle,
and then change the system to not work it.
Sorry?
Okay, now.
Okay, I'm sorry.
And then, in the middle,
they have to come for the business.
We use again to for that.
And so, that allows us to, at the top layer,
we don't have to care about all these different Linux distributions.
And here, the file estimates,
the last I want to mention is that
there, we use the S to distribute the software across the world.
It's a proven solution that scares very well,
and you can get quickly access to it.
So, once we add the software package on our sample server,
it comes within minutes of the day that everybody.
So, that's very convenient.
Okay.
So, here's a little bit different way of showing this.
And so, of course, we don't want to extend the CNN software there,
because then you would have to do this in many packages.
It's much better if you do this here in the comfort layer.
So, where we have to sort of again, to layer.
So, the assumption is that all, let's say,
already made it all, but most of the data access
goes through the comfort layer, and then to the data file system.
And so, we have, example, open functions,
read, write, close, and so on.
And these are all part of GNIPC.
So, if we do something with that, then change there,
apply to all the software package.
So, how can we change it?
So, we could of course, because we build GNIPC from source,
we could directly change GNIPC.
So, that's more of the advantages,
really transparent for you,
and they don't need to change anything.
But, also, the problem is that it's always on,
whether you want to need it or not,
and they also may have some on the consequences.
And, yeah, you also need to change what
you already should within you.
So, maybe that's also what you don't want.
And there's a way could be that you read functions,
and then you use the pre-load.
Of course, the advantage of that word would be that,
to keep the default GNIPC,
and it could be easily switched on off by users.
The disadvantages we have to sort of adjust every run for this.
So, and you kind of make this very transparent.
And, yeah, and of course, it can also, again,
create conflicts, if you set the slow release,
because then we'll apply to all codes,
we run, maybe they don't, maybe they've taken all the codes
that the programs we execute,
that are not part of eerie,
but we're going to come from systems like swear or command,
and so on.
For prototype, we do this,
grabbing a functions and use the pre-load.
So, now some ideas what we could actually wrap
as to what we could do with that.
So, we could log information for codes,
and it's like open access.
So, then later on, once we have a log,
we can post-process that and create data flow grounds.
We could also enable that we use remote data directly.
So, we don't have to manually download them,
and then open them, and use them,
but we can directly use them.
And we can also define some kind of virtual data infrastructure.
We just have to say, okay,
so this data will be belongs to this,
what I want to use,
some kind of view over the namespace,
and then some runtime and choose
that the data is there as accessible and drawn.
Okay, so now,
we just want to go through this idea and show what is needed to do,
it's an example for open.
So, if we register,
we can just print something and do a log,
and then we have to call the actual open function.
So, it's not so difficult.
The compilation is also not so difficult,
and then to use this,
we can just use any pre-load,
and then, in fact,
some of the cat may be not very scientifically interesting,
but yeah.
And then, what we get is we get some log lines,
and we can have many information in there,
and then, basically, what then we have,
as we get something like, okay,
there was a program,
it produced some image,
and then, maybe,
this image later,
we used to make a publication,
and then, of course,
we put connect,
sort of,
these two programs,
and we'll wire the ed files.
Okay, there are many different use cases,
what we can do,
so we can, sort of,
automatically,
we have the image and the end,
we can start from the data source,
and get a full buffer of what we did.
Yeah.
And of course,
if we do this,
let's say,
not this already,
we have,
the workflow,
we can also generate the workflow of that,
and then we can use that,
and copy the workflow,
managements of,
like,
next close,
make an all this,
yeah,
and yeah.
Another interesting use case,
could be that,
maybe if many data files we wanted to analyze,
but we don't know if we actually use them all,
so we could check if we have already used all these data files.
Okay,
so,
go back in time,
because if we do a lot of times them,
we could check what did we do on Monday,
because,
maybe on Monday,
I had a good idea,
and we learned to check this easily,
and we don't have to lock this manually.
There are some ideas,
when I worked on this,
so it's been a negative sort of spin,
it was filling up,
start times of programs that need a lot of libraries,
and we could also use it for this,
and of course,
and we also interesting is that we could actually use it also,
for easy itself,
when we build software,
it's very important that we only build,
again,
software that is already in ease,
when the software layer,
or in the compatibility layer,
not elsewhere,
because that could then make problems,
when we ship it,
and maybe the software we use on the build system,
it's not very user.
Okay,
and then, of course,
we could also use this to pre-load the cache,
to improve startup times.
Another idea is sort of to,
in a novel,
access to remote data directly,
so,
pattern would be usually that you find something interesting,
you download it,
and then you start using it,
and to go of that would be,
of course,
that you sort of avoid this manual download set.
And as a benefit of that,
it would also be that,
you make very explicit,
where it's got data from.
In the work event,
you see that,
where it got data from.
And the nice thing is also,
when data is associated with these,
persistent alignifiers,
you can get metadata from it,
and then,
sort of enrich the description of data flow.
So, it's just an example of how this could look like,
but again, for open,
we could check,
okay,
it's just a URL.
The person,
you have there,
then download it,
and then,
do the,
the actual open list,
sort of the local copy.
And for the user,
this looks,
totally transparent.
And,
so again,
if you compile this,
it's,
much more difficult,
and then you sort of,
maybe have some option,
that they can say,
okay,
so there's some space,
they can make a copy of this.
And then,
we have sort of,
now here,
a little command,
we say,
okay,
so it's really,
I run,
and then,
just a standard command,
this URL looks very long,
yeah.
And,
so what we could think of,
is that to improve this image,
that we actually specify the,
the UI,
so for them,
you have to improve it,
the implementation,
but this could also work.
Okay,
so this actually works surprisingly well,
although not for all commands,
but I tested this,
but how it didn't work,
and all at the moment,
only works for red excess,
but also works for bright.
So,
for bright,
what you would need,
is some kind of API,
the background,
that sort of supports that you actually write in.
So,
and you probably would do the upload,
when you,
the file is closed.
Okay,
maybe I skipped the rest of that.
So,
then sort of the last idea was that,
you defined some kind of virtual data infrastructure.
And what you could do is that you,
maybe say you define some namespace,
and maybe some labor,
so it identifies that you want to use.
And,
the searcher would then,
only give it a name,
and register some of these data files,
or maybe even the words,
whether data resides.
And then,
sometimes,
runtime would figure out where the files actually stored.
This is already local,
or this is remote,
and it has to be downloaded.
And then,
because many things,
or all these basic,
based on HTTP,
we can also redirect
to different source,
and we can do much more of that.
And so,
what we did,
so,
in the first step,
we created some commands,
they can create some of these namespace,
you can list them,
and you can upload files.
And,
so,
there we also sometimes,
two,
you know,
they can manage all those,
and then,
you can get a link,
and,
to then,
just sort of back under the data,
and then,
you can just say head,
and then you see sort of the output of that.
And,
but this really is just basic proof of concept.
So, what's the status?
We have some,
kind of prototype implementation.
We have some,
command line tool,
with the eye,
which,
you can use the run things,
also,
to manipulate this,
virtual data infrastructure,
and to add files to that,
and it's all,
it's open source,
it's a processor,
it's built-in source in less than a minute,
it's very easy,
it could be shipped with easy,
it's not shipped easy,
but could be,
in principle,
and we have some,
kind of,
server infrastructure for front end,
and back end,
so that the author's nice,
is to use it,
with one piece of one line,
okay.
So,
outlooks of what we could do
is the,
with these data flowgraster,
we could actually export it,
through tools like next flows,
make,
make nice,
with the actions of this work,
and so that you can create
the picture with light,
and then you can edit to your publication,
we could of course,
could intercept more,
functions and yeah. So what I learned is the next talk will have something
audible throughout for engines and they also use Iorots. So interfacing this
was as a back-end quickly very good because then the problem is a little bit
that how do we access sort of the data file system and the remote system.
And I think this could be done very well with this what the next speaker
probably tell about. Okay summary so working with the search that is not
too easy. We have two told with a couple of today a lot of data to create
this data work progress and then for effortlessly accessing remote data we have
some prototype and with that. So let's say if this vision becomes fully implemented
then I think the researchers they can focus on working the
data and don't have to some tools take care of all the logistics for the data.
Okay some resources if you want to look up more about the original reporter
and some case study so of course easy if you want to don't know what easy
you can look it up. So if you have a slacker you can either join everything
in this open source or to most of the development of the user is funded by
your HPC instead of excellent, excellent, smart, extra and all the codes for this talk.
Thank you Thomas. You have plenty for a couple of questions.
Do you see a lot of things? Yeah. So yeah so if you're right, okay so the
question is if I expect that we need locking for writing the model, right, so if
there could be multiple vertices I think there need to be some there need to be some
synchronization among the writers.
I don't really know. Okay so the question is if sort of might provide a bit of
coordinate, synchronize, right. I don't know, I would expect you need to have an
like this. So I have there was some work like maybe 25 years ago on implementing
a sort of kind of virtual, this is which has memory and so there were lots of development
of different ways to synchronize it with different consistency models and maybe one could
look into this to see if there are supposed to be questions, how to implement this efficiently
and thank you.
I have one maybe used to go on. Why has anyone 12 this year?
No, it's a stupid idea, I don't know. It seems like a simple concept. I don't know.
Does anyone have an answer?
So the question is if all the data is lost, those into some kind of lots work, some
notebook or so, where it would keep normally nodes in a let or so.
Oh yes, you could do this. So actually I would think that maybe if you just want to focus
on the fun part, yeah, not on all this logging and keeping nodes, one could use it, maybe
if I can. I think that we can do something like this with IOS because I don't
have such a lot of post-expression systems and we can probably use the metadata IOS to
bring it to the logs, certain data and choose because the metadata IOS actually has the
data.
So the comment is that one could probably use IOS because they have stored information
database and then, so I can watch the objects that are, yeah.
Yeah, I don't think that's the data, but I don't think that's the data.
I don't think that's the data, but I don't think that's the data.
I don't think that's the data, but I don't think that's the data, but I don't think
that's the data.
Okay, so I here quickly that if you use IOS back end, you just could complain that
the work also has to bring it in and get it out, so if you have a layer on top of that
that sort of makes the transparent that you don't have to do this, it just works like
this normal program that just helps in using IOS. But then, okay, the problem is maybe
a little bit, if you have very large data, that's how to handle this. And if you do
to do this then, it's an open call, that could be maybe not a best call, but you could
automatically do it with just a weak call and just say, okay, if they know that the data
in IOS in a database, you can access it, you can maybe just use the weak call and all
of right doors.
Finally, yeah, just to come and keep you mentioned in the last part of the other similar
efforts, I believe that I don't want to stay there, just because people are learning
this, something from the system, or bring it to the platform for analysis that
that lets you get that right.
And what's this? I don't want to stay turn around with you.
That's from the TSCS?
No, sweet thing to say is, sweet data sends it out, okay, what's the name of the
rank call?
One detail, okay.
Okay.
All right, yeah, just wrap up, thank you so much.