We're switching a bit to the topic now. We're back in 2020. It's on OI here. At least not
we're back in 2023 and we're talking about digital services act, which is, well, new legislation,
but that kicked in, at the end of 2023. We are specifically focusing on a package that
me and Luca, which is over there, developed to analyze the transparency database of the digital
services act. A quick overview of digital services act is a legal framework that basically promotes
transparency online for the platforms and especially for a very large platforms. It has
many layers of transparency and let's say rights that have been given to the users of online
platforms. For example, now we have to do, there is the obligation to have transparent terms and
condition, which are clear to read that explains and risk assess the algorithm that
the platform uses, such as the recommended system. Platform platforms have to explain the content
moderation policies, they apply to the user content. Then there are additional consumer protection
rights. For example, there cannot be targeting advertisement for miners. There is a mechanism now,
which is mandatory for platforms for users to report illegal or incompatible content online.
And then there are new transparency and that access provisions, which are going to focus on.
For example, now, every time you go to an online shop, there is the obligation to share the
selling information, which was not mandatory before, quite interestingly, and you can have also
the details of the advertisement that is proposed to you by the platform. There are many transparency
provisions. I will just quickly show you them. There are some transparency reports, which are
beyond what reports about the content moderation activities of the platforms. The transparency
database, which I present later in details, then, as we said, the terms and condition that we are
tracking in collaboration with the time open terms archive. So we are tracking the changes that
platform applies to them, how they treat the user data, how they present content to the user
and then there are advertisement libraries, so every platform, every large platform,
has to present the user, the repository, where all the information about all the advertisement
that has been run on the platform can be freely seen by everyone at anything. And then there are
others, which are very technical. I won't go into details, just know that for researcher, there is
a new data axis provision, meaning that the vector researchers can get access to close data or
private data from the companies under very strict conditions, but it is an unprecedented measure
to scrutinize the activities of the platform. What does the provision do for example,
the transparency reports gave for the first time another view of the content moderation
human resources that platforms are allocating to moderate the content online, so we have a
breakdown by language. We have the accuracy of the content moderation activities, etc. And so far,
we had three rounds of these reports, and one is due next spring. And then we focus on the transparency
database. The transparency database collects the anonymized version of all the content moderation
decision that the platform takes against the user content, so the data life cycle is the user
of course creates content on the platform, and then you have the platform that either by
proactive decision or by not is from a user takes down or moderates the content, and then the
platform under the DSA is obliged to notify the user on why and the causes and the reasons
that caused the content to be moderated. And then it sends it to the user and an anonymized version
to the transparency database. And what it looks like, it's like a very big JSON with all the
information about the statement of reasons to content moderation, specifically finding the category,
the automation that was used in the process, the pretext outlining, for example, the legal
grounds, etc. And then once it's in database, you currently have three ways to look at this data.
There is a website search that we offer, which is a kind of real time, but it's very limited in
the scope and the only covers the last six months of data. And then there is an online dashboard
that gives us an aggregate view of the data, but it's very limited in functionalities if you
want to go deeper in the analysis. And then there are these daily dams, which are basically the
daily volume of statements received by the database in a CSV damper, which are very big and
requires a lot of pre and post processing. So our package basically focuses on this part of the
of the pipeline and tries to optimize and streamline this streaming, this kind of analysis.
So the content of the database is quite big, and our package cannot do like a very
miracle, for example, now we are after about 25 billions of statements in the database.
And as you can see, the biggest share of it is by one specific player. So even if you want to
analyze them, you still need the package, you still need a very big machine, let's say,
with a lot of throughputs, if you want to analyze the daily dams. And even the aggregated view of it
by the categorical views, by the categorical variables, it's kind of two gigabyte in the end.
So if we just remove the bigger player, you see that we have a breakdown of the content, which is
like more heterogeneous. And so that said, that was just to say that this database is quite big,
if you also account, for example, for the free text data that are in. So the coming back to our
package, it's a package that can install in different ways. We provide the different
venues. There is a Python package that you can directly install. We also provide out of the box,
well, Docker container image that is exposing different ways to interact with it. I will show
it in the last one. One of these is the best boarding capabilities. And we also offer, of course,
interactive online documentation. As said, there are three ways to interact with the package,
if you, for example, run the container images. There is an API interface, which offers a standard
dyes-fast API interactive interface to try it out, the different queries that you can perform,
which are basically the download, the filtering and the aggregation of the data that are
found in database. The same functionalities are applied and mirrored by a common line interface,
which is easily configured with some configuration file. And then you have an interactive way.
Just to say that we will be in the workshop later. So if you want more details or you want a
small demo, you can stay and we will be happy to provide one. So coming back to the third
way to interact, there is also a dashboard link built on superset, the Apache dashboard link system,
framework. And we just show some of the possible solutions that are like breakdown that you might
be interested in. For example, I'm sorry, but the default font of superset is quite small,
I have to say, but you can have, for example, a breakdown, very easily, of course, the platform,
that meets the contents. For example, you have a TikTok, Amazon, Pinterest, Facebook, etc.
And the category of the content that they were moderating. So for example, for most of the platform,
this is just a scope of platform service, which is kind of a part two for them. And then you have
other categories. You can also have breakdowns of in a time series or like compare the manual
or automated content moderation from different platforms. So you can see like daily patterns
and where the people are where platform are using automated or not content moderation.
There are other breakdowns that I can show you later in the workshop, just to mention that
there is a flourishing community about this in the research, in the community, in academic research
community. And there will be an update of the database, late in July 2025, which mainly
we didn't produce content identifier for illegal products that are moderated. These are our
coordinates, if you are interested in and I said stay around for the workshop and the
panel if you want to more information about this. Thank you very much.
I don't think there is time for questions.
Yes. I forgot to mention that I'm in Rico from the European Commission. I
welcome, did you connect in the DSC enforcement team as a data scientist?