Okay, hi everyone, I'll be to be here, it will be a quick presentation, only 10 minutes,
and I will try to both present a new tool and make a quick demo.
So I'm a reception engineer from the end of the 70s and I'm in team in the south of Paris
in Crest and we are developing from one year a tool to accelerate text invitation for social
sense.
So this is the basic plot, the context is following, we have a sheet load of pixelal data for social
sense and you can imagine ranging from newspapers to social media to speech to text
the copies and it's come in range from a hundred to millions of documents regarding the
kind of research topics.
And there is this hype on AI, the new tools that are just emerging.
So we have a lot of new horizons with Fondesha model and LLM to embed in our searches.
And on the other side such terms and a lot of fields, not all the researchers are developing
using code.
So there is a deal and we have also an example today of the need of tools to expand the
possibilities of those new evolution for research.
So this is the basic context and the tool I'm just going to showcase today, it's called
active trigger, together, trigger the temptation on text and it is an open source web application
for text verification.
So the rationale is following first to be able to collaborate on text annotation because
it seems one to work together on it.
Second is to accelerate the annotation on specific elements and the tool integrate a loop
of active learning, which is basically selecting specific elements to accelerate the annotation
of specific patterns in the data set and the last point is to scale a few annotation
under the thousands to the world data set which can be a millioner elements.
And it's based on the observations that even with we are the lot of zero shot models
right now, there is still something relevant that even the complex tasks, a world-trained
to provide model can be called in certain events of past humanitarian and so there is
a lot of space to replace human and titles with well-trained supervised models to annotate
the data.
So basically it's the rationale of the tool.
I will get back to that while I've done the demo, but since there's a clue what it looks like.
Just to give you a very quick technical review, I don't want to deeper into it.
The backend is developed like Panoptic, I think, in Python and CEST API for architecture
client, several organizations.
The front end is a collaboration from this open room with Paul Girard, so it's something
which was born here also somehow with React and also Python client to interact with
the application and also we are using one specific family of models which kind of are all
models for comparing to the new ones which are Bird Models and Bird Models as a one developed
and trained by the end of 2020 and they are having a comeback because just a few months
a few weeks before a new version just dropped, which is modern birth, which extends the
contextual window, so it is still, it's all stuff that's still relevant for social science and
for text analysis.
The tool is based on three main values, first to specific to task, so we don't intend
to be a platform to all the text analysis.
The idea is to integrate it's possible, the best practices in our field, so in sociology,
political science and so forth, how to undertake correctly, by that I said, to help people
to follow the best practices and it's still a research tool, so at the end it needs to be simple,
simple enough to be modified for each team to add elements that they need for doing the research.
So this is a trade-off, but we'll try to create them.
Just to give a few elements of the current mainstone, so we have 10 beta testing, so
currently we have research chains that are starting using it for the research, so currently
we don't need to have, so please join if you want and to give you a just an idea that
is scheduled, we are starting to integrate a better interface to integrate the recall for
external model, like in phase, open-eye model to use the research in size application,
we are trying to better to adapt the management of rights and we aim for stable
classical version by the mid-June in a Docker-wise version. Okay, so I want to jump to a quick demo
for the five minutes that's left, do we need, do I need the classifier right now, maybe not,
maybe yes, so we need all to choose the next talk to attend to, so why not use our preferences to
train classifier on the first-dimensional test set, so we have all the abstract as a conference,
it's easy to do the nerd, it's not a real social science test, of course, but you can translate it on
the no-activity like trying to analyze the way journalists are talking about specific topic and
then train in classifier to do so, so how to proceed and if you want to do it, please join
on the interface, so we need the first-dimensional set and you can take it, but it's already online,
it's for them, for them, for the access, and then what's the next step, first you create a project,
then you decide how do you want to code, so you create a label, you want to use, and if you are
sort of in a time a codebook to describe how do you want to code and this is the huge parcel
result project, and then you start to annotate, first randomly, you just take a few elements,
say, okay I want to add on to this one, not to this one, and so forth, and at some point you can decide
to achieve learning, so trying to predict the next label for the element, and so to increase
the number of elements from one class and the other, once you have enough, you're going to train
the model, it's good enough, so if it predicts, if it predicts the well enough, what you want,
so the F1, for instance, is high enough, you say, okay, I'm done with that, and you can just
generalize the prediction on the rule of the set, so let's do it very quickly, so you can go to the
project, if you were able to connect with the same interface, and I have already created the project,
but if you want, you can create it this way, but in that a set, you have a very simple interface,
you can prepare a project like adding a label, I already created them, so I would attend,
I won't attend, and it's interesting, you create features like you create to transform the text
in a vector that they're based to help the model for the active learning book,
then you can explore and the main part is to annotate, so now it's, you know, the humanity's
part, so you need to read and say, okay, is something I want to attend, now it's not very interesting,
interested, do you want to attend to this one, oh yes, it's nice, okay, so at the moment I just
annotate, I can train a simple model based on the 62 elements I annotated, so I can train this one,
if I do that, I have a first loop of prediction based on what I've already annotated to
predict the next one, so I can have an information about, is it something that I can predict,
and it helps me to look for a specific elements or to loop to those, while not very easy to predict,
with a high entropy, so you can select to focus on those, was the most difficult to predict
as a model and then to improve your train that is, this is basically principle, okay, so once you have
enough, you can just switch to train a model, and then you can find a bad model, so bad model is
kind of one-and-rad parameters, so it's five-and-rad megabytes to one gigabytes, you can use the name,
test, or for them, let's try to the last modern model, you can go for all the elements,
and the point is you need GPU for fine tuning those models, so you need 20 gigabytes of GPU
if you want to do it, so that's why we're running them on our servers, and so you can train
you predict, okay, I just move on, so I can finish, and then once you've trained what,
21, you can look at it, you can see the parameters, but you can also predict scores,
and for this one I have a F1, about around, so that seven is not very good, I should
annotate more elements, but now I can iterate and start to have a better model and then,
I can just extend as a prediction on the work of us here, just computing right now,
and export the prediction here, I already did that, so that's prediction, and then I'll look
on the wall for them presentation to see which one I will attend to in the next session,
so I'm done, thank you, thank you, I need to look for the probabilities because I'm not sure
I predict always, it's good enough to pick up one, I will stay here because I'm trying to support the team.
It depends on how much can we handle things like transcripts, interview data, and the answer is,
what do you want to do with that? If your ID is to detect a specific pattern of
in the interview and you know what you want to do and you can write a code book, it works very well,
just have to prepare you better state, like maybe the whole interview is too big to enter
the conceptual window of a bad model, so did I do it by paragraphs and then start to annotate it
and deploy it on the wall that has it, so it will work as well, it's basically designed also to do
this kind of little, you know, stuff of creating some specific annotators for finding a pattern of
an instability or speaking about such a topic in the desert, but the idea is really to extend your own
you know, view and way of doing the data on the specific criteria for doing what you want to do.
Where is the training happening? Where is the training happening? Right now the server is
in our laboratory, so we have a few GPU, at the end you will be able to install it whenever you want,
so for the moment this is in the service of Paris inside laboratory.
Thank you.