Okay, hi everyone, I am Pratikshaya and thank you for joining this session especially
my talk and today in my presentation I will be highlighting the open source component
that we have to offer in the Copernicus Data Space ecosystem.
Within the presentation, how we will start from the definition of remote sense again
Earth observation which in a geospatial session might not be that much relevant but still
I thought it would be nice to relate to the Earth observation because prior to this presentation
I have seen most of you been talking about different geospatial locations but not exactly
on the focus with the Earth observation itself so I thought it would be nice to start with
that and then I will introduce you the Copernicus Data Space ecosystem with the different components
that we have to offer including data API and application but with the focus to the open source
tools that I will maybe conclude with highlighting on how do we support community and how
community can support us. Starting with the definition of remote sensing just to connect
among ourselves naturally when we want to observe anything we do that with our eye so remote sensing
is something similar in there there is sensors placed at a remote location and it tried to
observe something in a distance that is exactly what remote sensing is how ever with sensor they
have an added value that they can see beyond what human eye cannot so there is a range of
spectrum which they can identify and we cannot so ensure this can be defined as the remote sensing
process where it tried to detect something being placed at a certain location to understand
his characteristics and behavior and connecting that with Earth observation Earth observation is
simply a remote sensing process where different satellites or the aircraft placed at a higher
location is trying to observe our Earth whether it be above below or on the surface of the Earth
and according later try to tell us about the physical characteristics or different related
phenomenon that is going on the surface and these kind of information can be very helpful in
special temporal global maps or as someone said earlier maybe some pretty maps or some useful
maps that can be used in making decisions or various other research and planning purpose these can
also be very helpful to monitor different human activities at all scales and if I talk about the
different data sets as well as what can be used with this data sets it can be whether if you want
to simply monitor the ground or monitor the Earth surface like in this case the global land cover
map that is getting popular in order to address the food security and other reasons so that could
be very helpful with the Earth observation data set or even to monitor the water bodies not just
not just in Alaska but maybe small rivers or lakes that is also possible with the Earth observation
data set you can also check the or you can also analyze or observe the temporal variation
that is going throughout the year though we talk about the global warming and everything but there
should be a pattern that we want to observe through the satellite data that's possible with this
this Earth observation technique there is also different kinds of data that would be rather
or optical data sets based on which you cannot only not only analyze the
analyze what is going on the Earth surface but what's going below for example in case of
maybe Earthquake or landslide itself they use rather base images in order to analyze what is happening
in the different phenomena is related so those are the those are the few application of remote sensing
data sets however in past it's like not in past even in like every day their database of data are
being recorded and it has always been kind of a complication on how do we store this data or
how to manage this data one task as a geospatial injury that always come to my table used to
be like okay data set available out there but how do I access it and suppose I want to get
even the optical data as well as the rather data but they both are located at a two different
location so do I have to take every thing do I have to take the steps in a different manner
so that has always been a kind of a discussion so in there came the Copernicus data space
ecosystem where we are trying to centralize the data at one place with a set of different tools
that could be used to directly access the data there process it there and also analyze it
using the different visualization tool that's available that is in a summary what is Copernicus
data space ecosystem so again if I have to highlight it is a user centric platform which
allows you to access terabytes of earth observation data set it gives you tools to
access and process them and also platform to visualize them and within the CDAC there are different
components starting with the Sentinel data set I assume most of you have heard the terminology
Copernicus so Copernicus provides you with this Sentinel data set and the Copernicus data space
ecosystem will be the authoritative source where you will find the first hand data set and there
is the Copernicus contributing mission data set as well you will find now within this platform
and there will be additional EOD data sets that would be available maybe for European scale
or for global scale it will depend for example the recently developed is a world cover map you can
find this data set as well in the single in the same single platform and with this you can access
maybe directly with the website or using different streamline data access and API tool or even
there are different catalog APIs that supported within the ecosystem one example could be the stack
API itself there is Copernicus browser which allows you to even visualize like how does this data
look like or how does this data look over time so you can directly go and visualize and this I
think for those who are not yet sure on how to get started with handling this data set
Copernicus browser could be a good point to get yourself started then there are several on
mode code repository that we provide along with a cloud computing capacity there is certain
amount of cloud computing capacity available for everyone who want to use this data set
and there are online code labs and interface and there is also a concept called federation
and user identity service suppose you have a set of data set which you want to share with global
audience you can reach out to the Copernicus data space team and accordingly share it with a wide
audience that's where it comes into the federation concept like for Belgium there is a there is
the entity called Terra scope which provides you with the different data set more focused with
Belgium so that is that is one of the federation part of the Copernicus data space ecosystem
similarly if you have your own data set that you prepare and you want to provide then that could
also be part of it and it it it whole makes as an open ecosystem within the within the ecosystem itself
so that was the different components that I talked about what we have to offer in Copernicus
data space ecosystem most of them being open source however there is one very interesting
component and which is the open you to just simplify the definition of what exactly is open
you open you is a is a is an open source API in the form of source code which you can use not only
to access the data but also you can use it directly to perform analyzes or do some processing
with this satellite data on the cloud itself and there are few pillars that is always used
in defining what exactly does open you does so as I said earlier that there used to be different
platforms that used to provide you with different satellite data and it used to be always
complicated like how how many accounts do I create or how many platforms should I visit in order
to fetch the data so you know with open you it becomes easier you can just use your one account
and no matter which back in the data is saved you can access this additionally I can also say
that if your data is in a stack form and it is hosted somewhere then using open you you can also
access that data if it is not already the satellite data is not already there in the ecosystem
but there is some raster data that you want to use within your workflow then you can access that
using open you that would be a simple data access and processing workflow and no matter what kind of
workflow you develop using open you they will be scalable and efficient for processing
since they open you is developed using open source code and and supports the open
community so we have also we can also already say that it supports fair and open science principle
and however one more thing I would like to highlight with open science and fair principle of open
you or sports is the code that you have developed they are independent of underlying technology
whether you develop using r or python or even javascript so open you you can if it supports
whichever library you want to use and it is independent of the underlying technologies
and the work that you repeat the work that you prepare so you can reuse and share it with
the global audience I will later show you how you can do that if you develop it using open you
workflow then it just has to be a matter you will just be provided with a name space which you can
access with any different APIs how open you workflow look like is here I am trying to connect to
the company because data space back back in so I just connect using authenticate OIDC and then I
just provide what is the data that I want to access what is the temporal extent what is the special
extent and in this case I was trying to get Sentinel 1 which is a rather data so I am just trying
to get to to polarization band out of it and the next step I just showed a very simple
process used here which is already built in open you minimize a minimum time because we have
asked for a data over a temporal range so we just want to minimize reading it to one time frame
that is why I just use one simple process in here however there could be a scenario that okay
there is there is a very complex workflow and the processes are not already there in the open you
then what you can do is use this concept called user defined function which are basically python scripts
you sorry which are basically the python script that you can bring it to the open you and
make use it within your workflow it does not have to be that all the processes are in there and at
then you just write the yeah you just get the result when you get the result since I am sorry
I just missed this part that open you uses the concept called data cube because we have to deal
with a large amount of data over different time frame so it could be multi spectral data as well
multi temporal data so we use the concept of data cube and the data cube is executed as the
end of the whole workflow you can also do it in the intermediate step however I am showing here in the
example the last step so that is how it would look like and I think I wanted to show it in a
demo but before that I mentioned earlier that you can share your code how I just wanted to
highlight it already here we have something called open you algorithm plaza where if you have
your workflow that is developed it should include the open you component in it you can already
publishize it here so that why everyone or wide audience can use it one example is
as this pi u gpr which was recently published by one of the user which is basically which is
basically a python machine learning work library that try to predict the biophysical
trait using the Gaussian regression process so similarly if you have it does not have to be very
complex it can be very simple like you can already say few examples of band math calculation
that is done here it can be as simple as that to very complex workflow you can already share it
within here and yeah it will be used by a wide audience that is the idea of shareability
that we support in open you and I just quickly wanted to show you a demo in the Jupyter lab
environment that is offered to everyone in Copernicus data space ecosystem so you can access it
you have three different flavors to choose from when using the when using the Jupyter lab environment
and you have our and python kernel installed so I'm not sure how many here would be interested in
using our kernel hopefully yes with the python kernel so I will show an example with the python itself
let me quickly I just wanted to show you how does it look like in the the workflow
I hope I can
it has not come
already
you open display like in your settings see if I can see the extra
or what I think it's fine
you might have a button you can press like F8 to switch display
okay okay so this is the Jupyter environment if you register yourself in the Copernicus
ecosystem you will get this you will get access to this for free and then you will
you can choose from any of the kernel that you want to use they separately open your kernel provided
for you as well so the idea of different kernel with different name is that we have already installed
few libraries that you might need that's the idea and in addition to that there are some sample
examples provided in here so just go to open you there are few to get started for example if I go
inside one of this how it looks like is basically as I said earlier that you just have to authenticate
yourself provide the different parameters that you want to and in this case yeah using the process
that was already built in open you but there are can be cases where I want to do more and open
you does not have this process so in this case I have showed in the form of string which is not an
ideal form I can understand that so you can just do you can just write in your python file
and what you have to do is just import it from there using the from UDF command of open you
and just you get the result how it looks like at the end is this there is very cloudy image
so this is just a sample example of what you can do with Sentinel two data just downloading
and visualizing the RGB image but similarly you can do with the different raster data set you have
available as I said earlier if you have your very high resolution data available in stack
to evaluate in stack compliant format then you can directly fetch it using open you
before the analysis on it it can be as simple as just visualizing it to as complex as maybe
running a machine learning model or using the train model to get the inference
and in order to save I think I have an example I wanted to show on how how UDP can UDP which is the
user defined process the one that I showed you earlier was user defined function which are
the function that you define user defined process are which you want to save as a process for others
to use or for yourself to use it later so all you have to do is define the input parameters
in which format you want like in this case for temporal I gave this key method okay it should
be temporal interval and array and for a special extent in bounding box or maybe simple special
extent with suppose both bounding box as well as the feature collection and then you have to
define the workflow as it was but at this time pass in the parameter not the actual value and
at then you just save it with a different name that you want to give it will be a sign with
it will be the ID can be any if you want however we request it to be unique as much as possible
however the user ID will use the ID that is associated with your account the name space itself will
be unique so that will help in identifying the process that you have created and you just share
you will get a link that is what you share in the open your algorithm plaza so that was
quick demo as well
I am back to the action but before that let me do that I wanted to also show you how you can contribute
if we have everything provided in GitHub so it is an open source course so if you want to maybe
if you already have an idea okay this feature might be a good one to have in open you as well you can
already come and maybe create issues or pull because anything like that however there is another
repository called community example where we try to create example maybe basic to very complex one
for users to have an idea on what they can do with the remote sensing data set and also with
open you you can take a reference out of this and could be useful in some of your application like
in here there is also an example of how to load your stack the data that is 17 stack you can
directly load in open you that that there is an example how to do that there is how to do large
scale processing to do multi-backend processing and also you can find all the range of different
application examples in here and you can also if you already have your use case you can directly
come and create a pull request if it is if it is valid and I think that it will surely be
accepted and merged so I think that was more or less I had to say so do do do do
maybe coming back to the presentation
I think my time is also almost over
so yeah that was about community support and when using Copernicus data specifically if you
run ran into any issue then please feel free to post the issue in the forum or create ticket
however it is encouraged mostly to post it in forum because they should that you ran into someone
else might also have the same problem so it is always helpful in that scenario and to summarize
again about the Copernicus data space ecosystem it is a free public platform where you can share
your data you can use the data set that's already there use the tools that's provided to
you or you can also share the tools that you have developed that was more or less all thank you for
your attention on the app of every round so yeah I also have my team member from the Copernicus
data space ecosystem here as well as the OpenU core team so if you have any in-depth
question as well please feel free to raise yes please yes it everything happens in the cloud
yes yes that's as well
I don't think we ourselves it is not publicly shown for sure the user defined function even the
UDP you're just posting the URL you don't show the whole core and everything is handled as the
process wrap in adjacent format so yeah your UDP UDF it won't be publicly announced I don't know
from the back inside yeah I am to that you have to tell to be that in existence will be able to
use the spot system that's ready to run in and watch so if you are running the tasks on the
set of that data you're fighting for the global learning of service if you need to be in a
speed up especially during the course and then what's the speed up maybe this was can I mine
yes please yeah for three years ago this data would be offered free to download for almost
everybody on ESA's side-up platform and also in national archive such as Francis Peps program
both of which were decommissioned rather recently and moving towards this cloud-based platform
on which it is much more difficult to export the data out to process on a local machine you have
you kind of have to use the you kind of to use the given platform and given machines given virtual
machines by by the by the service is there any why why is such why can I know the
export of data regarding the question of like yeah there was I have and everything that allowed
you to download the data but now why cannot you download it locally I don't yet but not
no longer in high volumes I think it should be possible in high volume as well with regards to
open you it is more on processing so it is not only about downloading the data but with regards
to downloading high volume that also should be possible unless you go beyond yeah beyond too much
that should not yeah they certainly limit because everyone around the world we have given certain
quotas so that's the thing and Tyher is now the Copernicus browser that's there so I think the
feature that was there in Tyher we still preserve now that's there and I think it should be
possible but yeah how about the tooling that was also deprecated around the same time the Python interfaces
to snap with regards to snap I don't know how it would be related with this one yes I'm sorry
other things like you are the data cube based on xrd data cube they are to some extent
but yeah the data cube that is there handle in xr this one is slightly different in the part
there is not only multi-specure but also temporal and yeah in cluster yeah I think it is divided accordingly
unless I don't want to add something
okay so thank you
thank you