All right folks, we are ready to get started again.
Carol Chen is here to present, get your docs in a row with ducklings.
Let's give it a hand to Carol.
Thank you, Daniel.
Hi everyone.
Thanks for being here today.
I think this is my 10th awesome perhaps.
I've been coming almost year or less since 2013.
But always learning something new from people and new development in projects.
And new faces, old faces.
A lot of people, I'm based in Finland, a lot of people who live in Finland.
I only see them once a year here in Brussels.
Yeah, I'm good to be back.
Anyway, today I'm going to talk about this project called Darkling.
It's a open source project, obviously.
And it's about processing, digesting, parsing documentation, keeping the meaning of it.
And consistently, you know, like being able to kind of do that with different kind of documentation,
document formats.
And this in a row really reliably organized way is kind of my take on it when I was coming out with the proposal for this talk.
So my name is Carol Chen.
I'm from Red Hat.
This is an open source project.
It's actually part of the Linux Foundation.
And I'm just, you know, part of the community.
I really like this project.
And I would like to share that with people.
Before I start, I would say I'm not a technical writer.
I'm not a documentation subject matter expert.
You know, like people like Daniel and other people in this room talking here.
It has a lot more experience in a lot of the things.
So apologies in advance if I get some terminology or concepts wrong.
And feel free to give me feedback on like the Fediver's Macedon.
I'm a Macedon or Matrix.
And you know, LinkedIn, if you want to, LinkedIn there.
So you can also send me a PDF or write me a feedback in a document format.
A DocX or HTML.
Why?
Because docling can help me process parts and make sense of information from different document formats.
So here's the doc in project.
The key things is it can parse different formats.
Like I just mentioned.
And one of the key things is the advanced parsing of PDFs.
We talk a lot about, you know, the different ways to represent information and documentation.
A lot of it is about how it looks visually.
Like mark down mark up.
Whatever.
You're looking at this headers.
There's like paragraphs, sections, tables, diagrams and stuff like that.
PDF is one of those formats that can encapsulate and reproduce reliably across different platforms.
You know, I made a copy of this presentation in PDF just in case I couldn't present it on my own laptop.
And we have to use some other laptop.
So a lot of times, there's a lot of information.
But it's great for humans to read it, to digest it, to understand it.
But machines, you know, that's why it's really probably quite challenging to parse PDFs.
So doclin has this whole process pipeline to make use of certain AI models to be able to do a very thorough and accurate parsing of PDFs.
But besides that, there's also like, since the project started, this was not like right away from the start.
It's a support for things like images and audio files with OCR and ASR.
So there's different pipelines for the different file formats.
So it's multi-model.
And while the key concepts for the blockchain is this unified, expressive doclin document representation format,
where it captures no matter what kind of input you give it, to be able to capture and preserve the meaning behind
the document. So that, especially because when you want to translate it or convert it to some other format, that meaning of the document is not lost.
A lot of times when you do like conversion or parsing, like it's one to one, maybe like PDF to HTML or PDF to mark down.
But, you know, sometimes going through the process from one to the other, you lose some information data.
So by converting that into like an intermediate unified format, it's able to do that and you can then use that for outputting to different other different formats.
So I'll get into a little bit of that more in the coming slides.
Another key thing is the local execution.
A lot of trans, I'll say, convert this, I've used myself.
You know, you have to kind of upload the file somewhere to some cloud.
And I guess it's fine if you're working on documentation for an open source project, most of the things are in the open.
But sometimes you might just want to work with something that's a little bit more sensitive, or, you know, before a project release,
there's something that you don't want things to kind of get leaked or anything like that.
And, you know, can you trust all the different cloud services or SaaS out there, right?
So, without being, you can download everything on your computer, on your laptop or on-prem and be able to execute that locally.
So, yeah, I'll get into some of these later on.
And it's a Python package.
So, and you can run, you just pick and install a dockling.
And there's also a CLI version, of course, you can also include it as a Python library, another app.
So, first, very versatile.
And let's look at more about the project.
I'll try to help promote it.
And it started, I think, and near the end of 2024.
So, more than a year ago, slightly more than a year ago, around, right here, you know,
November, 24, it started big job in how to say popularity.
And it was like, for a while, trending on GitHub.
And I know the dockling team would love it.
If I say this, please, you know, like and subscribe.
I mean, like and follow the GitHub repose.
And, you know, lots of popularity and lots of contributors.
It's not just about using it, right?
People find it useful and want to add stuff to it.
So, we have a very active community as well behind this project.
And, yeah, I was just looking at Python.
The down numbers are also pretty amazing.
So, take a look at GitHub.com slash stopping project.
And there's a lot of, of course, repose under that organization.
Now, let's take a step back and imagine that you are reading a paper,
comparing the English language and the Finnish language.
I don't know why you want to do that, but let's just go with that.
So, we know that the English language has, sorry, the English alphabet has 26 letters, right?
The Finnish has 29.
So, if you, you know, know the language that's probably not very surprising,
there's this idea of pengram, which is like the shortest sentence you can form
with that contains all the letters in the alphabet.
And I think also many of us know that, so the paper goes right.
So, an example of a short sentence that contains all the 26 letters in the English alphabet
is, quote, the quick brown fox jumps over page 6, horizontal line, footnote.
Oh, here's a study about how kind of change may impact the speed of brown foxes.
Seven, the lady dog, and quote.
Wait a minute.
That was quite a short sentence, and I'm well sure it contains other 26 letters,
but you won't quote me on that.
You won't quote the paper like that, right?
Of course, it's also a fake scenario, because that's not such paper yet.
And if I ever get to write it, if I get my finish, you know, improved enough to write that.
I will make sure to have that page break over the lazy dog.
But there are actually real cases of stuff like that happening, you know, right?
Now, this is actually a paper from the 50s.
Some 1959 article, yeah, this is there.
And it talks about, I have no idea what it talks about, but that's this part.
It's actually in two columns, that says, vegetative, electron,
multiple copy, okay, whatever, right?
But it's actually not, you know, in the same paragraph, just, you know, half a century later,
a bunch of research scientific papers actually quote the study and say,
oh, where it transform an infrared spectroscopy, vegetative,
electron, microscopy, blah, blah, blah, blah, okay?
Now, that's not the only one.
If you search like in the Google scholar site, that's like,
site of 112 papers.
So, we can see how parsing wrong information from documents can have an average effect.
That may be a bit more extreme, extreme case.
And sure, you know, probably most parses now can and do something like a two column thing.
Oh, we've been out, I don't know, because when you have images and graphs,
it also can cause more confusion.
When we use top thing on the same paper, we see that it accurately checks and
understands the layout, at the two columns.
And, you know, parsed it correctly, like what happens to the vegetative cell wall,
when the source release blah, blah, blah, and then, you know, effects by means of
electron, microscopy, blah, blah, blah.
So, again, having the right understanding of the PDF document,
made a big difference.
It's one thing to get kind of the meaning wrong is another just kind of losing content all
together.
A lot of like basic parses probably, you know, like the quick brown fox case,
you know, put the header and the fooder and all those fun information together
with the context that I mean the main part body of the paper.
So, the meaning is kind of included or kind of diluted by this kind of undesired
page header, headers, tables become like a list of numbers that has no relation
to each other.
You don't understand what this list of numbers mean.
The whole image is missing and line wraps.
It's just like chunks of lines, broken and there's no wrap.
So, you know, like when you, again, like parse those lines, you lose that context
and you may like, you know, break them out in the wrong places and stuff like that.
So, talking to the rescue.
Markdown.
I love Markdown.
It's one of those formats that's easily understood whether it's like, yeah,
actually formatting thing, the kind of the raw part as well as, of course,
you can then make very kind of clear and structured output.
So, in that sense, Markdown is usually not the problematic one, right?
But, again, like I was saying, PDFs tend to be one of those formats that,
even though it's not easy to kind of create or whatever,
but it does have to capture that visual representation with diagrams and tables and stuff like that.
But then once you have that to kind of go reverse, it's really difficult.
So, when talking first started, they created this PDF pipeline to kind of address that more specifically.
Like Markdown's HTML, those are a little bit easier.
They have a kind of a problematic pipeline and kind of a bit more straightforward parsing,
but PDFs.
First, you have to use an OCR to kind of scan a whole page.
Find out where's the tags, where's the pictures, information, whatever.
And there's actually two key parts of the process here, a layer analysis.
So, again, like the column flow, right?
Is it two columns, three columns?
You know, when you have a line under that diagram,
it's that part of the paragraph below, or the image above, they kind of thing.
The table structure, tables are simple yet, not really,
because you can have, like, multi columns that, you know,
spans across multiple columns and rows.
You can have tables within the table and so on.
So, this is actually two small models that they use to kind of do this job specifically.
Like, instead of throwing the document into a large LLM, you know,
two B7B, whatever, right?
Sure, they can probably do a very decent kind of general,
parsing of that, but maybe it's like 70-80% accurate.
If, maybe some simple PDFs, and that's fine,
and, you know, you're happy with the results,
but for more kind of specific use cases, you want models
that design to do that specific task.
So, these are, they kind of models, it's small, tiny models,
but if I remember correctly, like 40, 42, and million, rather than billion,
you know, so downloads on your laptop, you can process everything locally,
like I said, and because they are trained to just do,
what one to just do layout analysis and other just to do table structure.
So, they are very good at that task.
They're good at nothing else, but those two, those specific tasks,
and be able to do that.
Later on, the team also developed this VLM pipeline,
which is a slightly bigger model, like 258 million parameters,
but still, small compared to most LLMs or Foundation models out there.
So, this VLM pipeline takes the step further, and do all this,
there is design to do all these steps in one go,
so you don't have to kind of go through the whole thing,
and be more efficient in the understanding and the parsing of the PDF document.
It does take more resources, but it also produces more accuracy,
more accurate results, and you know, if you can get something back,
that's like 92, 95% accurate, sure, it's never 100%,
but you spent a lot less time, we always want to double check the work,
and we don't 100% trust the AI model.
So, but you have a model say, you can save a lot of time
in that kind of follow-up checking work compared to general models.
So, again, this is the kind of the intermediate document format
that I talked about, and then, which then can be exported to different,
oh, my goodness, I'm taking too much line.
Okay, so when I saw, when I first heard about this document document format,
I was like, you know, this is like, yeah, another one.
But the key thing is, like I said, you know, you want something
that you able to have almost lost this representation of that information data,
so that you can be able to parse it out to,
walk down to, I think that's an issue in my own and different ones,
and one of the interesting things that
I'm adopting wanted to solve was parsing for
AM models to train, to fine tune, for rag, and stuff like that.
So, I'll talk about chunking a little bit.
So, these are the input formats.
I think the list is growing every time this,
and you really say something new supported as well as the output formats.
And there's a thing, right?
It's an open source project, and it's able,
if there's some formats that you care about,
the ability, because of the standard document format,
you can then, you know, write your own kind of input for two or
that, and then also output.
So, the update list is probably on this link,
but this was when I grabbed it like maybe a week or two ago.
So, it's just a lot of 10 code details about the document format.
Again, you can see from the link, but basically it expresses
a lot of the things that a lot of times it's not easy to capture
with just simple, like, how to say.
Like, you preserve that, okay, we know this is a text,
we know this is a table, and how it relates to things.
So, and also the hierarchy,
we know H1 is under, sorry, H2 is under H1,
and so on. So, a lot of parsing may not be able to capture that.
So, it does that, and then, again, like I said,
the APIs to build the document from scratch with different formats.
Like I said, it does a CLI, you can use that,
but I also wanted to introduce this document project,
actually done by somebody in my team, in Red Hat,
because we were playing wrong with Doppling,
and I was like, it would be nice to have like a simple UI,
GUI interface that we can, you know, try things out,
instead of trying to check check all the different CLI parameters
and configurations.
So, if you go to ducking, that's UI.org,
there is the, you can find how to use this.
And I kept this non-sensical thing here,
because I fell asleep at 2am, updating the slides.
And I don't know if it was my hand or my face was on the laptop,
and I thought, well, this is artifact for Boston.
Let's, I'll, I'll feed these to ducking and see what it says about that.
But, that's for next time.
I was just chatting with my teammates yesterday.
He said, that's the latest release, which, of course, I have to install,
and it supports internationalization.
So, you can see, like, now it has this,
I think German and French, yeah.
So, and I think a couple of others.
So, let's see, we can just run the,
actually, I already ran the demo,
because it takes a couple of minutes.
So, I wasn't going to, like, let it run,
but it is live running on my laptop.
And I also tested this,
without, right now, it's connected to the Boston Wi-Fi,
but I tested it without, just in case it was working.
And again, just to prove that it works locally,
you don't need to, it connection after you download all the models.
So, you can perform everything on your machine.
So, this converted, I had, like,
some free, sensible, ebook that just downloaded,
in PDF format, extracted images from it,
understood all the tables.
That was in the PDF and chunking.
Right.
Most AI models, when they take input to, you know,
do rag or do whatever,
the context window and, you know,
it's limited number of tokens.
There's kind of the most primitive chunking,
we'll just say, okay, 500 tokens.
I'm just going to count 500 tokens,
but then you lose meaning,
and you lose context when you do that.
So, here you can see,
you actually did hundreds of something chunks,
and each one is like, I want to show,
let me be this here,
like, you know, heading.
It's clear, like, this heading is abstract.
This one is one introduction to getting started.
It's able to,
and because,
docking understands format,
it creates blocks of information.
So, you understand that it's able to
group that as a chunk.
And if it's like,
within the context window limit,
you will just use that.
If not, you'll find a appropriate way to,
kind of chunk that into smaller blocks,
but still be able to kind of keep that,
cementing information.
And then I just, again, want to quickly show,
like,
docking versus,
this is docking.
It's able to, like, you know,
pass the index with page numbers accurately.
This was,
I can remember, I used,
market down on some other parser.
It's like, you know,
block,
numbers, block, numbers.
What's chapter one?
I have no idea.
Here, you can see,
about chapter one,
it keeps all the, you know,
heading and information,
and be able to output that accordingly.
So,
again, this is,
this was from,
I just used the,
the startling UI to be able to do that.
So, making it easy for you.
You don't have to remember,
the, you know,
all this CLI references.
You can access that from here,
right from the setting.
You can enable,
OCR.
You can select one.
You can even switch OCR,
and that if it's not installed,
you install the automatically,
except for test drag,
which is since system white install,
so you can do that.
Then add options,
which, again,
you don't have to remember all the,
all the different parameters.
You can just,
turn them on and off here.
And then there's even documentation,
built right in,
MK docs,
perfect markdown,
very easy.
And I've one minute left.
So,
let's see what's my slides.
So, feel free to,
check it out about chunking.
As I clearly,
like I said,
it's about, you know,
splitting it in the right way,
it preserves meaning and context,
and,
be able to understand layout
and reduce hallucination for the models.
All right.
So,
I will upload all these slides,
maybe I'll remove that chunk of nonsense letters,
but I'll upload this to pre-talk,
so you'll be able to see,
get all these links for both document and document.
And that's it.
The document team,
by the way, is from IBM Research in Zurich.
You know, this is not the full list.
We have them with, you know,
of course, a lot of these images
and the docling logo is from them,
but the docling author is from Red Hat David.
Hi, David.
Thank you so much for all your help.
If you're watching this,
amazing stuff.
So, thank you.
If you have any questions,
like I said,
feel free to look for me as sidebat.
And that's my time.
Thank you very much.
Thank you.