Okay, awesome. Good afternoon, everyone. I'm glad everyone has made it to the
3 o'clock on Fuzzdom. My name is Vasim Musa. I'm the CTO of Open Cities Lab, a non-profit
organization working out of South Africa, predominantly in the public sector, and it's
a civic tech organization, generally working with cities around Africa and specifically
in South Africa. So today I'll talk a little bit about open source approaches to a secure
data exchange in South Africa's DPI. South Africa is going through a large digital transformation
as many other African countries and South American countries as well. So I'm hoping to
share our last 16 months of learning to build DPI, and yeah, the journey is so far.
So firstly, open cities labs vision a city where every resident regardless of income, identity,
or location can access essential services they need with dignity and ease. And it's very
important that the beginning part is a city, because generally this is where citizens interact
with local government. And I think just two sessions ago in Nicholas was mentioning
local government initiatives for open source. And DPI often, in often cases, is working
at a very high level, generally at a national level, so to see service delivery impact
with DPI is hard to track, but it's important. So what is DPI? It's a set of foundational
digital systems that forms the backbone of modern societies. DPI enables secure and seamless
interactions between people, businesses and governments. This is from UNDP. How I like to
think of it is very much like roads. So just how roads are physical infrastructure,
both to provide access to other infrastructure. So we use roads to go to hospitals, police
stations, private sector, businesses, and so on. But in itself, it's also an infrastructure
that needs to be built, secured, and transport various people throughout. On the next
two slides, I've got a bit of personas about the current state of affairs in South Africa
and how people access digital services. So we got the example here of Tandy, where, you know,
maybe in the EU, it's a little bit different, but speaking to some people, I know they
are some similarities. So incomplete information about where to access services. Oftentimes
in South Africa, we have to visit physical locations to apply for certain things. Tandy
must submit physical copies for verification. So, you know, just interacting with government
needs physical copies of certain things. Oftentimes, they don't meet our needs in terms of
services, and there's large amounts of delays, long cues, and limited payment options.
So that's the current state of affairs. This was pre-digital transformation. It's been just
over a year and there's a lot of national initiatives kind of building alongside to help
speed this along. South Africa in general is highly digitized from the IT perspective. So
there are a lot of great systems built for government. However, as you'll see, a little bit later
in the talk, most of it is isolated in silos within the departments, and, you know, speaking
and listening to open forum, summer yesterday, and many of the other EU cases, it seems like
it's not just the African problem. It's across the world. So there's three core components to
DPR. You have digital identity, digital payments, and the backbone of all of it is the data
exchange. These then power all your digital services. So a very simple example, one could think
about is renewing the passport. To renew your passport, you have to provide an ID. That
check has to go against some interdepartmental verification. In South Africa, we have home affairs,
department of home affairs that does the check, and that generally goes through the data exchange.
Then some sorts of payment is needed to then, you know, renew your passport, and you get
the renewal process done. A key part here is not only digital IDs, but we were looking at
verifiable credentials holistically. So we were thinking about how you would have a health card,
a digital driving license, and not just digital ID. So that's just one example in this particular case.
The focus of my focus is going to be on the exchange layer. It's something not seen to us
generally as citizens. It's usually interdepartmental accessing of data, and extremely important.
In fact, I think the most critical layer for DPI. But firstly, what is the data exchange component
trying to solve? In South African context, data currently exist in isolated silos, and this
rarely hamper streamline research evidence based policy making, government operations, and ultimately
at local government service delivery. So the data exchange is critical. In fact, I would say
it's the most critical part of DPI. It connects governments, scales nationally, and enable services
for millions. And still, at conferences, I think even the last two or three times, I've got a
few people just saying, oh, you're building a middleware, or an API gateway. But I'm hoping to
give a bit more context in this talk. So yeah, over the last 16 months, we did a bit of research
in the first six months around the data exchanges than around the world, and what technology we
should use, and we even started piloting in the last six or seven months with government agencies
in South Africa. The challenge was set out for us in back in 2023. So we were approached by the
World Bank to do a feasibility study on what the data exchange for an African country would need
in this particular case, South Africa. This was done alongside the leading department,
national treasury in South Africa. And national treasury is like, I don't know what's the
ministry equivalent here, but they had our finance, they do our budgets, and they set the
economic policy of the country. And what we needed to do was make use of integrated
data source from various sources for different purposes. So exchange of data in real-time,
bulk data transfer is still, you know, one of the main ways the government agencies move
data is almost in large flat files and they're moving data. We even wanted to look at
open data access and ultimately how all of that leads to evidence-based decision making.
What we did in the design journey, so the initial four months, we did a needs assessment,
meeting the government departments, the major ones that were identified to share data and exchange
data. We did a good practice report and research. I link to that at the end of the
presentation where you'll be able to see it. A high level architecture of what needs to be done
to do a data exchange at scale for a country. And what we did not only in terms of research,
but we actually prototyped software and government through use cases. So it wasn't theoretical,
we actually did some real work and we used open source to do that.
Okay, so initially it was called the integrated data lake project and we were given the brief
around a data lake. It was a good starting point. So what is a data lake? Just an easy
technical solution for structured unstructured and even semi-structured data. Just a place to
move a lot of data into, which generally to some extent made a bit of sense for a starting point.
However, we'll soon see it's very limited and it wasn't effective in this particular exchange
for an entire country. So the data lake started with three initial architectures. We were looking
at centralized, federated and hybrid or federated, centralized system. We moved all the data into
similar to where treasury is storing the data and do a lot of the research,
querying of the data in one place. This is technically challenging. There's a lot of data,
you know, hundreds of terabytes of data actually been moved in national government. And not only
would it be technically challenging, even from a mandate and policy perspective, you know, are they
able to actually hold all that data and how's it? And in South Africa, we have the personal
protection act for identity called Poppy. And that would kind of break some of the rules around
that, especially with like your revenue service, sharing data with certain other services.
You know, it's quite an extensive legal landscape to navigate from a data exchange.
Then we looked at federated. So can we put a data lake solution or data exchange solution
in each department? And this made a lot of sense because kind of that's how our government operates
where departments have various IT solutions and various data maturity. So some of our
our systems like for revenue is actually quite advanced. And then there are other departments
which are much further behind in terms of IT and infrastructure. So what the mixture of both
on the good end really good high quality systems ready for integration and on the other end
almost nothing available under-capacitated, we said a hybrid approach is more likely needed in the
short term with the federated approach in the long term. So you would need a central system where
those departments or ministries that didn't have capacity can work with and operate in.
And then those that can federate of the bet have the IT systems can start by federation.
So there are advantages of the federated and hybrid model in hand-stator control. So
the department still maintain ownership of all the data. It's only access to the data,
improve scalability, no single source of failure as well.
Some of the challenges and something we're still grappling with today is
even infederated, it's hard to ensure all departments adhere to a certain standard of exchanging
data. So it is a tough challenge and there are some solutions going around, but it's still there.
Also the technical complexities you're dealing with different ministries, different departments,
with various authorities, and each one coming in with different levels of understanding of
what data sharing actually means data exchange. And it's almost like a quick pro quo.
Like you give me some data, I'll maybe give you some if it's valuable to me.
So not only from a mandate perspective, but you're actually kind of motivate why you'd want
access to the data, even from a department sharing data with another department's perspective.
We then looked at case studies. So we said okay as the technical solution, the data lake makes a lot
of sense, but it's not encompassing and not broad enough. So we looked at a number of other countries
that already implemented data exchanges and because we've been working in the open source space
for the last decade, we were looking for open source solutions. Very high on the list came Rwanda's
Erembo platform, India Stack, Brazil and UK. Brazil and UK was more around digitizing
government information and not about the data exchange itself. They did have an open data platform
which was particularly useful. India Stack was actually very potent, but not all of it was open
and it was extremely hard to find good documentation on. So something we really struggled with
off the bat and we even engaged with them to certain extent and just found it didn't fit the solution
entirely. Now coming on to the next one and I think this might not come by surprise because throughout
today I've just heard this particular software mention a number of times. We looked at Estonia's
X-Rode. So just by definition it's a secure decentralized data exchange platform almost
fits the spec too perfectly which enables interoperability across public and private sector.
And one of the nice things about X-Rode is it's essentially managed but allows
federated exchange of data. So the registry sits essentially, but departments can share
data directly. So once you become a member as I'll explain in a bit you can share data directly
with each other without the central operator needing to come into effect and that's particularly
powerful in my opinion and fits the use case exactly. Now when motivating for X-Rode we have to look
at the underlying open source software. So firstly the license what does it provide from a
feature perspective and secondly what is the ecosystem for X-Rode? So an X-Rode ecosystem refers to
one X-Rode instance where members are part of the registry. And this is very important because
just by setting up X-Rode you don't automatically have a data exchange solution. You need members
to come on. They need to send these departments need to sign legal agreements,
NDAs, or MOUs and we have to do a lot of legal work which I won't touch on in this particular
presentation but there was a large legal undertaking to understand what the mandate of each department
was and how they could access data and how that could be enforced through security protocols
in the X-Rode ecosystem. Now the X-Rode operator controls who is allowed to join the community
and defines regulation and practices in the ecosystem in this particular case this was treasury.
So why we chose X-Rode? The software is actively maintained with the strong community we spoke to the
NIS Foundation, Estonia, Iceland and Finland are all using it in production for over a decade,
Estonia for over two decades. So it rarely had the data exchange in production for a number of years.
Over and above that we were able to engage Petarianist team very actively and is the CTO of the
NIS Foundation. I think it was earlier today we have been mentioned the model of a member
organizations from countries coming together to create a sustainable open source community and
NIS Foundation is one of them where Iceland, Finland, Iceland and Estonia have all come together to
back abroad as a software being built. It also complies with all the relevant security standards,
its interoperability is a design cornerstone and it has extensive documentation.
I think one of the best I've seen from an open source perspective.
This is a quote I found on LinkedIn recently coming shortly and I think it's one of the challenges
I've seen with open source and it's from Arturo Morante. It says too often open source solutions
are expected to be implemented by an in-house team where in practice lacks the mandate or resource to
sustain them. So moving from I have a proprietary solution and I am the only one who can implement it
to I have an open source solution and you are the only one who can implement it is progress but
it's still not enough. Not all the time departments can maintain this infrastructure themselves,
know how to operate it and they need to be implementation partners who know the local context
to come in and assist. So I think this is really nice and from what I understand in first
them just looking around it's it's a big thing. Not only how we build open source communities
but how we implement the open source solutions. So the bottom line x-road creates a trusted
transport network organizations maintain their own secure entry points. Very similar to chain
stations, you know exactly who's coming in and going out. Data travel is standardized
secure containers essential authority maintains order without controlling the data. So as you'll
see in a bit all the central server does is have a registry of the members and that's in
force through public key infrastructure and everyone can reach everyone else using the same system.
So you plug in an API once it's available to every single other member organization or let me
rather say it's visible. It's not accessible it's visible to every other member organization.
Technical architecture deep dive and what were our final recommendations. So we said for sure
federated data exchange for a country makes a lot of sense, especially in South Africa's context.
It was the way both from mandate and a technical perspective to make sure it's federated.
And something really nice about that is you then maintain data sovereignty so each department
still holds all the data and you you are ensuring that it's a cross government data exchange
is actually just secure. Something about the South African government is most of the IT infrastructure
is 90% on-prem. So you have very, very few systems that are actually in the cloud.
This is maybe both the positive and negative to some extent because it's hard to scale by
procuring servers through in government, especially because you need budget and capacity and so on.
Yeah, so the components mainly included data owner and data consumer layers, which I'll show
in the technical architecture, cross-cutting standards and policies for compliance and legislative
requirements. We didn't touch on that but there was a large undertaking from the lawyers perspective
to make sure that even something from a technical perspective we were suggesting was allowed and
was going to be put into legislation. Yeah and let's dive into it. So on the left-hand side and
the right-hand side we have the data provider internal environment. So in the case of renewing
a passport this could be like home affairs. This is already existing environments where for example
department of home affairs has servers ready to be used. However they would in our case be
missing the API layer or the API layer to share data with external entities and you've got
the data consumer environment on the other side. This could be like your state security agencies
so those giving out grants who want to check against your population registry and that's an
example of interdepartmental sharing of data and this is where X-Road kicks in. X-Road has
essential server and this has the memberships of all those in the ecosystem so home affairs
would be onboarded onto the central server all your other departments would be a member as well.
And the entry points or the encapsulation for security in X-Road is very aptly called
security server. It's the entry point and exit point for all data packages going in and out of the
system. This is extremely important because this is your train station, your entry, your barrier.
So whatever is coming in through your, you know exactly who's coming in, who's got access
and who should be accessing what? So down to very granular and fine-grained policies for your
APIs. So for example on the population registry if you wanted to allow only certain access by
certain government agencies you can do that through the security server. It does mean that has to be
high availability. It doesn't necessarily mean it needs to be very large servers. So in the case
of Estonia we saw they had I think only about three or four servers on the population registry
feeding almost entire country which is crazy and that's because it's not doing any creating of
some sort is just the gateway access point to go through and get the data that's happening over here.
Data cataloging part is a cool feature of X-Road. So every member in the ecosystem can find out
what is happening in the security service in the network. So if a department's brought on board
they can find out every API that's been made available by government. They can't access it.
They still have to go through all your normal membership or MOUs for memorandums of understanding,
go through all the legal, go through all the technical onboarding to become and get access to that
particular API. But the cataloging part is really cool because it at least allows transparency
to show what's out there from an organization. So in the example of you bringing on department
of transportation if they only have one API for drivers license but they're missing cars,
they're missing all your other important key details. You know that's something very transparent
in the network. So the secure part comes in here. So as part of the entire ecosystem
trust services or the public key infrastructure is extremely important. This could either be
third party or built by government and what it means is you need a certification authority
and you need a timestamping authority. A certification authority assigns TLS certificates to
your security service and that's checked against the registry in the central server and that
allows you to become a member. So they will assign a TLS certificate to your security server and
that will be your authentication certificate. You get a second certificate for signing and that's
for all outgoing and incoming requests of data. timestamping authority is just to make sure
something or every exchange of data is packaged and stamped you know at a point in time. So it's
not on your server time or you know as an example I think Estonia mentioned they were requested
from government in a court case to check against a transaction in the population registry
and they had the timestamp of the exact time it went out and which department created that
and it was for some particular lawsuit. So it's very important that these two audit controls are in
place and like I mentioned it doesn't necessarily have to be built by government it would be
really cool. In our case one of the other departments is assisting with this in Europe's case they
might already be some PKI available at a national government level. Okay so I think that's it.
I just wanted I just wanted to end on one last note for first them thanks to all the EU colleagues
and the amazing open source community we have out here I think we leveraging it it's a global
community not just the EU approach and I'm glad to be part of it.