Okay. Good morning, everyone. My name is David or David. I'm a research data scientist
at the Alan Cheering Institute and today I'll be talking about explainable forecasting
from big weather data, rapid and sustainable solutions.
So, data-driven weather prediction has progressed substantially over the past few years.
And the reason is essentially that traditional physics-based forecasting is accurate but
is actually computationally very expensive. And a lot of open source data has been made
available over the past few years and as a result, many data-driven models purely data-driven
models have come out. And some of these data-driven models can actually outperform these
physics-based models for many standard forecast scores.
Most of these data-driven models use a deep learning architecture and they employ different
approaches. For example, vision transformers, neural operators or graph neural networks
or typical approaches. For example, Google, DeepMind graphcast uses graph neural networks.
And a lot of these models are trained on the publicly available error 5 data set, which
is the European Center for Medium Range Weather Forecasting and waspherick-rean analysis. And these
are measurements of the global climate covering the period from the mid-20th century until
today. And they contain hourly estimates of many variables, such as temperature, wind
speed, precipitation, etc. On a lot longer grid at the multiple pressure levels, so at
multiple altitude. And it's a huge data set. The whole thing has hundreds of terabytes,
but just to give you an idea, a slice at a single pressure level of 500 hectopascals on
the finest resolution of 0.25 degrees for a single variable temperature. And for a time
range of 20 years, has a size of 730 gigabytes. And here I'm showing a slice of temperature
at that pressure level 500 on the first of January 2020. So this is roughly what the data
set looks like.
So this was actually a side project that we did. We're developing a graph neural network
model with the UK's Met Office as well, but we had a pause in the project and we decided
to try a different approach. And we wanted to develop an inexpensive data-driven forecast model
that could serve as a baseline for comparison, so sort of a benchmark. Having a similar
role to persistence forecasting and climatology. So these are two metrics that are used to
compare data-driven models against, which are very simple, persistence forecasting essentially
assumes that the weather forecast remains static for the whole of the forecast period.
And climatology essentially assumes that the forecast is the same as the historical average
for that particular data in time. So those are very simple models that are used to compare,
that are used as benchmarks to compare data-driven models against. And we also wanted to develop
a method that could deepen or understanding of the underlying physics of the problem. And that's
why we came across dynamic mode decomposition. So dynamic mode decomposition is a purely
data-driven model. It's computationally efficient. It's explainable because the modes and eigenvalues,
I'll explain a bit more about that later, that come out of this are interpretable and they explain
the underlying physics of the model that you're of the data set that you're trying to model.
And they can also approximate a nonlinear dynamic called system through a linear approximation.
So the team, most of us were based on the Alan Turing Institute, we also had a very good collaboration
with Nathan Kuts from the University of Washington who has been working on DMD for many years now,
and with a PhD candidate from the Barcelona Supercomputing Center, Bennett, who's working on
lower-order modeling for fluid dynamics. So a bit of background about DMD, this looks a bit
scary, but it's actually very simple. So actually DMD seeks the leading spectral decomposition,
so the eigenvalues and eigenvectors of the best fit linear operator A that relates to snapshot
matrices in time. So imagine here my matrix matrix X was my error five slides. And along the rows,
I would have different spatial locations on the lat long grid and different columns, I would have
different snapshots over time, organized in time. And then I make a matrix X-prime which is just
this matrix shifted in time by one snapshot. And my assumption here is that X-prime can be expressed
as a linear function of X. So X-prime equals A times X. And A is this best fit linear operator.
And DMD essentially seeks for an eigenvalue decomposition of A, and seeks this
decomposition by expressing X as a multiplication of the eigenvectors by the amplitudes and
the matrix of eigenvalue exponentials, which inform about the time evolution of the problem.
So this is essentially DMD and a nutshell. It's essentially doing, assuming a linear relationship
doing a linear regression problem. And then calculating the eigenvalue decomposition of A,
and expressing X as a, as a run car reconstruction through this equation at the bottom of this
slide. And this has connections with Kutman theory, which essentially says that you can
express and nonlinear dynamical system through an infinitely dimensional linear system.
Here A would be our approximation of the Kutman operator, which in this case would be a finite
dimensional operator. And DMD has the advantages of connecting the SPD. So single value decomposition
for spatial dimensionality reduction, as well as the fast Fourier transform for temporal frequency
identification. So it's doing both things at the same time. So it's reducing the spatial dimensionality
through SPD. And finding the dominant temporal frequencies through FFT. And here you can see a
typical example of DMD being applied to a classical problem in fluid dynamics, which is a cylinder
in a cross flow. So imagine I had different snapshots of the full field behind the cylinder.
And I applied DMD to this. I would solve this linear regression problem. And then if I did the eigenvalue
decomposition of A, what I would get are these dynamic modes, which in form of the spatial behavior
of these vibrations that compose the problem. And these dynamic modes, spatial modes,
would also be accompanied by these temporal oscillations. And I could also, so I could use these results
to better understand the problem, but also for forecasting, because I could just project
the data onto a new time vector into the future. And I could predict the future state of the problem.
Now DMD is very good, but it's actually very sensitive to the presence of noise, which is
always the case in real world data sets. So the approach we're using is actually optimized DMD,
which is based on the algorithm proposed by Askerman Kütz in 2018.
And essentially, optimized DMD is a post-processing step of the exact DMD algorithm. It seeks
an nonlinear optimization of DMD through a variable projection algorithm. So actually,
I don't know if you can see my cursor here. Yes, so it's actually solving,
optimized DMD is solving this exponential fit in problem directly. And by doing this,
it's much less sensitive to the presence of noise and avoids much of the bias of exact DMD.
So here, the approach to solving of DMD is that you first compute the exact DMD,
and then you solve this equation in a post-processing step. This is actually quite computationally intensive,
especially if X is very large. So what you can actually do is compute the SVD of X to produce
a rank R reconstruction of X, and solve this optimization in a low rank approximation of X.
So this is what we're doing, because it's much easier to solve. So this is essentially called
approximate op DMD, because I'm solving the optimized DMD algorithm in the rank R approximation of my matrix.
Right, so we've been using the PIDMD package, which is a publicly available Python package
for performing DMD, and this optimized DMD algorithm is implemented in the Bob DMD class.
Now, in order to make this work for error five, we've had to modify the algorithm slightly,
and we've incorporated a new method called FIT icon for a fit economy, essentially,
which allows you to do a much cheaper DMD fit. So just to give you an idea,
this is the old fit method that was in the algorithm, and this becomes intractable if X is very
large, which was our case. So we were getting out of memory kills on a HPC trying to do this.
However, with this new fit icon method that we've implemented, we can actually solve this problem
on a laptop. By using this trick of solving the problem in the low rank approximation of X that I showed
in the previous slide. Okay, so yes, we can apply of DMD to the rank R approximation of X,
but we still need to compute the SVD of X, and actually computing the SVD of a very large matrix
is very computationally intensive. So how about building separate DMD models for much smaller
subsamples of X, and combining these DMD models to produce a single forecast. So instead of trying
to solve everything at once by doing the SVD on X, we produce different subsamples with an each
subsample, we capture a range of the time frequencies and spatial frequencies that are present in
the dataset. So with the DMD model applied to each subsample, we only capture some of the dynamics,
but then we can combine all of these models together to produce a single forecast. So this is the
approach we've gone for. So here I could summarize X by producing three different subsamples.
So in X1 for instance, my snapshots would be one hour apart from each other, and the whole duration
of the observation would be one month. I could then produce a matrix X2, where with a time delta
one day between snapshots and a duration observation of two years, and a next three matrix
with a delta time of 15 days between snapshots and a duration of 20 years. So neither of these three
subsamples solve the full range of the dynamics, but together they do, that's the trick here,
and X1, X2, and X3 are much smaller than X. So we tried this in a toy dataset, so this is one
one-d problem, so over time, so imagine this was a rope oscillating over time. There are three
different vibrations in bed and on top of each other with some Gaussian noise, and so this is my
problem essentially, and I try I produce three different subsamples with different observation
durations and delta periods between snapshots. So again neither of these three subsamples is
able to capture the full dynamics, but hopefully by applying a DMD to each one of these separately
and combining those DMD models I can produce an accurate forecast. And it's actually, this actually
works. So if I produce a DMD model for X1, X2, and X3 separately, I can then cherry pick the DMD
modes from each of these models and produce a single model for forecasting. And as you can see,
I can get a very accurate forecast, which is very similar to the ground truth. You can check
this out and get hubbed if you're interested, it's in a public repo. Right, so what are our preliminary
results? I should say that we've literally finished the code base very recently and we've just
tried playing playing with the error five data set a few days ago. So this is I'm showing temperature
really mean square error for different pressure levels. So trying to predict temperature over a period
of 10 days, the first 10 days of January 2020, with a DMD model trained on December 2019 only
with a delta time of one hour. So as you can see for a pressure level of 100 hectopascal, so that's
quite high up in the in the atmosphere that's around 16 kilometers above sea level. And that is
doing is doing okay, it's definitely doing better than persistence, especially at higher lead times,
and it's sort of doing the same as climatology, sometimes doing better, sometimes doing worse.
Now when I move to a pressure level of 500 hectopascals, my DMD becomes, my DMD prediction becomes
quite rubbish. The reason for it is that as you move closer to the Earth's surface,
the field becomes much more turbulent. So there are many more scales to resolve and DMD is not doing
such a good job here. Either because I'm not used to get enough low rank approximation of error
five, or because I'm not using enough data, for instance, maybe December 2019 is not enough
in order to capture the first 10 days of January. So what I tried here is this approach that I mentioned
in one of my previous slides, which is combining 2 DMD models to produce a single forecast.
So on the left hand side, I have the DMD forecast with data trained in December 2019 only.
And here I've combined two different DMD models together, one trained in December 2019 with a delta
time of one hour, and another DMD model trained on two years worth of data with a delta time of one
day between snapshots. And as you can see, my prediction becomes considerably better
using this approach. Again, this is super fast to compete because the SVDs of these matrices are
you can do them on a HPC very quickly. And it's a very cheap method essentially. And here I'm
still doing worse than climatology, but again I haven't included data for further back essentially.
I could have included, for instance, data for the past 20 years with a delta time of 15 days
between snapshots to try and improve this model further. Right. The next thing that DMD is
good at, apart from forecasting, is explainability. So the results that come out of DMD are
actually explainable. You can understand them. You can plot them. So here on the left hand side,
I'm showing the DMD spatial mode number four. And this is what it looks like if you plot it on a
left long grid and it shows your regions where in this case the temperature field is linearly correlated.
I'm how this patterns evolve over space. And on the right hand side, I'm plotting the temporal
evolution of this mode. So I'm not a meteorologist and I'm not very good at interpreting what this
means, but essentially this would hopefully summarize a particular weather pattern that could help you
better understand the underlying physics and could also help you, for instance, have you're trying
to produce a deep learning model? You could use this as an input feature to your model.
Right. And this is my last slide. This is the package we've developed in Python. It's called
DynamoDera. And it's publicly available on GitHub. It's current capabilities are error five
downloading and slicing from the cloud, data pre-processing, the singular value decomposition,
either standard or through a randomized algorithm. We also have integrated data version control,
which we found to be very useful because there are various hyper parameters that you can tune.
For example, the low-ranked reconstruction, how many DMD modes you're using, how many pressure
levels you're testing. So actually using data version control has been an amazing addition.
We're currently working on the integrated application of DMD using PythonD. But the key thing here is
that actually you can just do the SVD on the HPC. And with this new fit icon that we've developed,
you can actually run the DMD step on your laptop. It's very quick.
Future work includes adding post-processing capabilities and parallelized SVD using
Pylon, Pylode or Modeling, which is this GitHub package that is this Python package and GitHub
that then it's from the Barcelona Supercomputing and Sender is working on, which can actually
do parallelize SVD on huge matrices in a very short time. And of course, contributions are welcome.
That's all. Thank you very much.
Questions for Dali?
Well, speaking question, I'm here for Dali for the Barcelona Supercomputing.
Yeah, that's a very good question. It's not straightforward to do that.
Oh, yeah, sorry. So the question was how do you choose how to subsample the data in order to produce
these separate DMD models that you can then build a DMD model and then you produce a single
forecast from those DMD models. So it's not straightforward to choose that subsampling and actually
it's a lot of trial and error. You sort of have to ensure that you capture the full range of dynamics.
Well, yeah, that's so if there's overlap in the dynamics, that's not a problem
because essentially what you would get is a repeated DMD mode on separate subsamples. So if that's
the case, you only choose one DMD mode if it appears twice across two different subsamples.
But it's better to be cautious than not to be cautious in this sense. So you choose a subsampling
where you know there's going to be overlap of the dynamics across the subsamples.
More questions. Yeah, in the back, I think very much for a very good tool.
Are you sending your any comparisons against the baseline of temperature? Yeah.
Right. So the question was, how does it compare how does DMD perform for other variables such
as precipitation for instance? I haven't tried on precipitation yet. That's the answer.
I've only tried on temperature and wind speed and I got similar similar performance for temperature
and wind speed. I don't know about precipitation.
More questions? Yeah.
I'm not very far in constant combination, right? But since you are basically putting a data
on vector in the locality in your grid somehow encoded in that board, I think, in just matching
arbitrary data without knowing any data without knowing any of the locality of the data.
So the question is, is there locality information in the matrix action which I do DMD essentially?
So as long as the location remains constant across the different columns,
when you do DMD you're essentially doing SVD across space. So as long as the spatial location
remains the same across different columns, you're capturing the linear correlation that exists
between spatial locations. How much is data do you need to derive this problem?
I was thinking, well, we have a lot of them, scale, forecast, weather, and few months of things.
So that's a good question again. Sorry. So the question was how much data do I need in order to
develop a good DMD model? And that's something to be answered. So error five goes all the way
back to the 50s. So I guess the more data you use the better. Although that might always be the case
because you're making it harder for the optimizer to match your X matrix. So my gut feeling is
that 20 years worth of data is enough. Now we also want to capture things such as global warming.
And for that, DMD might actually not be the best possible model out there because that's more
of a moving average. And DMD is not great at capturing trends. It does capture trends
by fitting a very slow sinusoid, but it's essentially trying to fit sinusoid to everything.
More questions?
Right, that's also a very good question that I have to test out. I've tried to, sorry, sorry.
Yeah, the question is how far into the future can I forecast before my forecast just becomes rubbish?
So that's something that I need to test out at scale. I've tried two months and it behaves
relatively the same for those two months. And the reason is that is capturing these oscillations,
these periodic oscillations that repeat themselves constantly. So if you compare this against
for instance, graphcast is designed to essentially forecast the first few days. For instance,
in the plot that I showed of January 2020, graphcast would probably do well in the first 15 days of
January. And then it would become rubbish. Whereas DMD does consistently well over a longer,
much longer period. It doesn't do as well as graphcast, of course, in those first few days of the
forecast, but it roughly stays the same. The quality of the forecast roughly stays the same.
Right. So the question is how would it deal with non-linear weather events? And I would say non-periodic weather events, such as storms,
that are sort of stochastic, probably not well on those. Because again, if there is no periodicity,
if it's not a repeating pattern, DMD is not great at that.
It could be used for really long forecasting, and the methodology, like for those of the year,
for the two juniors of our future, like eight to the end of the course years, or all the other.
Yeah. So the question was whether DMD could be used for weather forecasting, long-range weather forecasting,
essentially. And I think the answer is similar to what I gave before.
So yes, because it can capture these oscillations that happen periodically in the global climate,
it's stable at producing those, and it would produce a better quality forecast in that sense,
that graph neural network models that essentially are trained to predict
the next 10 days, for instance, of a particular variable. Yeah.
Okay, that's enough up here. Thank you. Thank you.