All right, welcome everybody, we're going to hear about Stethethes again, and connecting
the front line with some beautiful seats today.
So, give it up for the next one, Mr. George.
Hi everybody.
I'm Patrick Donnelly.
I've been working on Stethethes for almost nine years now.
Previously, the Stethethes team lead.
Now, just tackling larger projects.
Thank you, Shankar.
It's now the team lead.
All right.
Yeah, hello.
So, my name is Contedational.
I also worked for IBM since quite some time.
I was working for Redhead before that.
And working on some of the code since very long time now.
I think since 2022, 2004, I think was so very long time.
Yeah.
And at IBM, we look into providing us a piece of part for Stethethes, which is a relatively new feature.
I've just been provided as a tech review for Stethethes, zero.
And of course, we are looking for a lot of performance improvements and feature addition
and support.
So, there's a whole bunch of stuff to do.
And one of the most important things we realized is really the problem of case sensitivity.
And this is what our talk is all about.
So, I hope you all can familiar with what the core problem of case sensitivity is.
Actually, it's meant by that.
Very easy in post-exfiles systems.
You have, for example, if you take the term, a SFFFS.
Okay.
And this may be said that example because it's all uppercase.
But typically, it's written like with a capital C and then with lower case EPH and that capital F and S.
And if you have, if you just imagine all the possible case-folding types of that name, then you can have all the different files and a unique file system, which are all completely independent.
They have nothing to do with each other.
Well, in the Windows world, they're all considered to be the same file.
And that's, of course, a fundamental problem of interoperability and of compatibility.
And for that reason, we are taking this work up.
So, first of all, Samba is not really a SFFFS.
It's kind of an independent project.
Obviously, it's really just running on top of SFF and exposing the file system with Windows semantics,
to Windows clients or clients that are expecting Windows semantics.
So, you would imagine that maybe with that additional layer on top, you have a way to kind of negotiate that behavior on the server side.
So, is the server kind of maybe possible to run in the case-sensitive mode or in the case-insensitive mode.
And actually, in the old S&B1 protocol, you see a network trace here.
You have a negotiate protocol packet being sent from the cloud to the server.
Then the server can reply, yes, I do support that.
There was actually a case-sensitive bit that would indicate whether path names are case-less or case-sensitive.
But, as you all familiar, S&B1 is really deprecated, not used anymore.
And if you look now at the follow-up protocol, S&B2, and all the, according protocol specification,
I mentioned them here and the SFF's, MSS&B2.
In brackets, they're kind of the industry standard definition of control the Microsoft.
You will find this bit still mentioned, this S&B2 case-sensitive bit.
But, also, you have a footnote there saying this bit is ignored by Windows system,
which always handle path names as case-insensitive.
So, in that case, you can, of course, look at the file system implementation that you have in Windows,
or whatever, not the file system that is exposed to the network, but just the local file system.
And they actually also have a flag that indicates a case-sensitive behavior.
They have one for file-cancetive file case-sensitive search, and one for preserving names.
And then, if you look in an other document in the FSCC, no FSA document, it was actually,
you will see that actually all the file systems that they have, like RIFS, NTFS,
fed, X fed, and so forth, they all are case-preserving.
And the majority of the ones that you would encounter on a kind of a production machine,
particularly anti-FS case-sensitive search is always set.
So, to some perspective on the protocol level up,
it's basically, which means we have no way to kind of negotiate a case-sensitive behavior at the SMB protocol level.
That set, of course, somebody itself has really much more flexibility,
like the Windows system, and in particular it has to address this problem in the first phase.
So, somebody came up in the past with a set of configuration options that you can set in the SMB.com file.
That would control the server behavior, whether it is case-sensitive,
whether it will preserve the case, and there's also third setting for default case.
And if you go to the documentation of some of the main page,
there's an eye section about this specific behavior.
And the default case setting has been designed,
so that you can basically eliminate all these costly case-forwarding operations,
but just telling, okay, I have the assumption that all the files in this directory
will be either uppercase or lower case.
And if you then turn the other notes, like set case-sensitive to yes,
then preserve case to no, then you can really avoid this problem altogether,
of course, at the cost that all the content in these directories will need to match that specific description.
So, it can be a use case imagined where this is appropriate,
and really a meaningful way to avoid these case-forwarding operations,
but of course, for many other scenarios, like whatever default share that will just be written with all kinds of clients.
This is not applicable.
So, if you look a little bit deeper at Samba, Samba has its own module stack,
the FS modules, it's called, so that allows file system specific modules to be created
to control specific aspects of the file system.
And there has been one addition being made a very long time ago already,
which is the VFS get real file name call.
And if that module is implementing that call, it's basically just getting a request for a file name,
it could be really in any case.
And the function will then return the properly-cased file name on the disk,
really in the file system.
So, that Samba has an easy way to really completely avoid all these look-up operations,
and just can, okay, I'm looking for CFFS, like the file name,
regardless in what case, and then the file system can report back,
okay, actually it is with capital C or all capital letters or all lower case letters,
or whatever.
So, there's a whole bunch of VFS modules that actually do implement this specific API call
for GPFS, there's an implementation for cluster, even as well.
But we don't have anything like this for CFFS right now.
Then inside of the Samba file server, there's another capability function that will basically indicate
what kind of properties they'll be exported files system, really has.
There's a lot of knowledge already in place, which actually will be the vehicle for our
implementation.
I'm going to talk about.
And of course, there are some special cases, like if you followed a focus talk about the
unique extensions earlier today, he was talking about also posics path names,
so that basically if you have posics extensions negotiated in the SMe2 world,
that existed even in the SMe1 protocol, then you can also completely avoid all these
case operations, but just whatever assuming both client and both server are running on
posics systems and are following posics semantics.
And then you can avoid all these lookup operations as well.
And there's a bunch of clients available that support that the kernel is a big client or
the S&B client utility, there might be might be others as well.
And also there's of course the server support in Samba available.
But again, this is really a special case, which is really mostly addressing posics
two posics communication over S&B protocol.
So to really go down what really the flow of operations really looks like in these two examples,
when you have a file system that is supporting case sensitivity case
and sensitive lookups or if you haven't one, if you just imagine there's an operation
someone wants to open a file.
So there's just a Windows client trying to open a file with a file,
and in this case it's file name with a capital F and with a capital N.
So the Samba server will receive that request that will then actually look for
exactly that formatted string and the file system if it exists,
if it exists, if this is fine, it will just open it and whatever return to the caller.
If it does exist, it actually has to open the directory and iterate over the entire contents of the directory
in order to find really the exact matched file name because they could be in the same directory
file also called file name, but with a lower case F or with a capital lower case N or something like that.
So you can imagine this multiple scenarios where you have really a long sequence of calls
and really only done in order to find the appropriate file.
While you have a file system that does support case and sensitive lookups,
the flow of control is much shorter that we will have the same incoming request for file name
and that specific case, if it exists it will just be opened and we are done.
So obviously much shorter and we can avoid all these full directory traversal operations for this specific operation.
And one colleague of mine actually did it test run really just an experiment by
untarring the Linux kernel sources on an S&B share over LipsFFS and he counted,
he did an analysis of what kind of system calls are called and he identified that
140,000, 204 retail operations are called just really because of that specific operation.
And the entire time it took for the untarring to complete was really spent in these operations for
30.3% of the whole time.
So really an enormous impact on the file system performance obviously.
And then he repeated the same test with really these settings that I mentioned earlier,
which basically just assumed in this case he decided that the default case would be lower
but basically eliminating all the needs for these additional lookups.
And the execution time really went down by almost a third.
So just really with this simple configuration setting.
Then of course this is something which we can't, whenever built in a production system or something
this has been really just a test.
So we definitely need to address the problem really at the blue layer which is connecting
the S&B at the Seth World and with it I handed over two petitions.
So let's move on to talking about case instability in S&BFS.
Before we begin I'm going to bring up a slide we just saw in the last talk.
For those who are not familiar with SethFS and just joining us.
SethFS is a POSIX district file system.
It's been around since about 2006 during Sagewall's PhD thesis work.
It's the UC Santa Cruz.
It is the original use case for Seth's rados distributed object store which was also
developed at the same time.
SethFS did something somewhat novel in the beginning by
charting metadata and data into separate pools and having metadata servers
act as a basically a cache and authoritative access point for all metadata in the set file system.
And clients are able to interact directly with the data pool doing
reads and writes with them to go through the MDS so long as they have appropriate access.
Collectively the clients in the MDS collaboratively maintain the distributed cache of the metadata.
And sometimes the client's authoritative for what the cache state is for a file,
but generally the MDS is.
The MDS officially writes out metadata changes to journals.
It distributes metadata and exchanges metadata with other MDSs in the background.
And it hands out rights to the clients as part of the distributed cache in the forms of capabilities
which you may have heard about before.
So jumping right into it.
Directory entries.
So something we don't really think about very often especially within the context of file systems,
but it's a little part of of directories that holds metadata we don't often think about.
Here is the POSIX definition of a directory entry on the right.
It holds the I node number.
And you get this structure when you do a read their call on a directory.
It holds the I node for that particular directory entry.
Some record length and offset information.
The type of the directory entry which won't change because I know don't change type.
So it can be like another directory or a file which can be helpful to cut out system calls.
If you only carry it into look for directories, for instance.
And then the directory name which in Linux is limited to 256 characters seems to be a common choice among POSIX file systems.
Now, while ago we had the observation that it would be useful to add another bit of metadata to the directory entry.
And that is this new field alternate name and it's just an opaque vector, a byte vector that we can stuff whatever we wanted to.
The MDS does not actually care what's in this opaque structure.
It only puts the data that it's been given by the client in that alternate name.
So how do we actually use this and why is it exist?
So the first use case for alternate name was actually encryption.
A project that was worked on a few years ago was to plug in the kernel library FS script into the CFFS kernel driver.
And the idea there was that the client would be able to encrypt a directory tree and including the data, of course,
and also the file names in the directory names.
And the MDS has no idea or any other entity that recovers that file system through whatever means would not be able to decrypt and know what those directory entry names are or what the file data is.
So you just bring your own key and you can encrypt an entire file system tree on CFFS.
So what that would look like is like the client's trying to manipulate file.text.
And it sends a file crate with the encrypted directory entry names to the MDS.
And that's what the MDS stores in the directory.
Now before we wouldn't have this alternate name field, we would just store the encrypted name.
And then the I node number for the file.
The problem is that we encountered was, well, we're going to encrypt a name.
It's going to output binary data.
Many of those, the characters in that binary data are not actually valid file names.
So we have to encode it.
And when you encode it, the file name size increases.
Well, if I give a valid long file name to FS script, the encoded name may be larger than the maximum size directory entry.
So we had to deal with that.
And the way we do it is we just put the if the file name is too long.
We put it in this alternate name field in the directory in the directory entry.
So now we can recover the long file name and decrypt it without having it,
without overflowing the directory entry maximum length.
Name maximum length.
So the observation was that we could also use this alternate name functionality for a similar purpose within,
for the handling case folding in SEPFS.
And the idea here is we can have all the clients agree on how to transform a directory entry name,
such that it's no longer has case in it.
We case folded it.
And then we store the actual directory entry name with the case in it.
What I call the case full name in the alternate name field so that it can be recovered later.
Like when Samba asks what's the actual file name.
Or if I'm doing a reader operation on the directory I want to know what the file name is,
complete with the case that was used when the file was created.
Now the nice thing about this is that the MDS doesn't actually care at all about the alternate name.
It's just storing what the client says the alternate name is,
then look at it.
All the cares about are the path names and those would be the case folded names that are actually used to name the file.
Also the client doesn't really care much either about the the the name and the alternate name.
And only actually needs to unwrap that name for a reader call for posics only API.
That's the only time an application learns the the name of a directory entry is so reader.
All other times the client is just using the case folded name.
So whenever we send the creates RPC to the MDS it's going to attach the alternate name to the MDS stores it.
And then when a future look up comes in or a reader to the MDS it collects that alternate name from the from the MDS and reinterpreted as it's as needed.
So as a concrete example we're studying this path.
We're going to send the client's going to case fold that to lower case home.
It's going to do a look up operation on the root OX I know one with lower case homes and that off to the MDS then the S finds it in its table.
The alternate name is capital H home that's the real case case full name of of that directory sends it back to the client.
It doesn't matter for the path traversal on the client side because it's not trying to recover what the real name is.
Then we're going to look up Patrick we case folded to lower case Patrick send that look up call to the MDS.
It discovers the alternate name is capital P Patrick stores that doesn't need it right now it's going to continue with look up and then search for file that text again case folded.
And find out that the alternate name that was used for when the file was created was capital F file and then the extension was capital's text.
Again, not needed for a look up operation it just stores it in its cache.
For a reader the application is going to come to the to the client the mount and say I want to read their home Patrick it's going to.
Do a reader on this I note after does the path discovery on it it's going to get this table from the MDS and it's going to transfer it here and then the trick is it's going to use this alternate name as what it's going to pass back to the client.
That's the only time the alternate name is is actually used and presented to the application.
So how do we set up a case and stability in the in CEPFS we use a new virtual extended attribute suite of virtual exatters.
They include Seth dot dirt our case sensitive Seth dot dirt our normalization Seth dot dirt our encoding.
And then Seth dot dirt our charm app which is just a read only view of what the charm app is and the charm app looks like this.
It's just a JSON output and you can see that for for this example directory the directory the case sensitivity is false so it's a case in sensitive directory.
We have a certain normalization setting that we'll get into and then it's a UTF8 encoded directory entry names.
The requirements to modify or set the charm app on our directory is it must be empty and it must not be part of a snapshot.
And that's important because I can't just suddenly mark a directory case sensitive because there could be like a bunch of files which conflict with each other if they were properly folded.
So it has to be done when the directory is created.
And the idea there would be that it would be used for somba shares upfront.
I want to use this directory tree for somba.
I'm going to mark it upfront that it's going to be in sensitive.
So the first charm app we'll talk about is Seth dot dirt our normalization.
There are four normalizations that you have to choose from which come from unicode standards.
These are supported by boost which we're using as a in the boost local library to actually implement these normalization routines.
The default normalization is NFD form d canonical decomposition.
And the way that looks is I'm going to just set the normalization for a directory to be NFD.
And then if I create files, for example, that there how do I say this?
Who is it?
Okay.
I don't know any German.
Sorry.
But I love this word.
So when it's normalized, it's going to translate the you with the umlaut into a you.
A regular you like a English you.
And then the umlaut gets separated out as a separate unicode character.
And then this capital B, which is pronounced like two S's.
It gets turned into this this unicode character is zero zero DF.
And that's how the NFD transformation is done on that.
Normalization is not optional for handling case sensitive directories.
And the reason for that is it's very easy to construct two.
Two directory entries which are rendered exactly the same.
On our on a screen, but they are actually the the bite encoding is different.
So the normalization is there to help with the um, collating that properly.
So the set dot dot case sensitive.
We're similar.
We're going to set it zero to mark the directory in sensitive.
And now we're just running the the case folding.
The other standardized unicode case folding algorithm on the directory name.
And then you can see like the capital G gets turned into a lower case G.
And the nice thing about this case folding table that in unicode is this locale independent.
So it doesn't matter, you know, what the locale of the client is.
And again normalization is required.
Set dot dot during coding.
This is really just to give us room for changing things in the future.
If we want to support other encoding types.
It's actually a complicated thing to change because if you switch to for example,
for example, the UTF 16, then you can have nulls occurring in directory names.
And that that is not a no no um for for a lot of the the code that we already have.
Because it assumes null terminated names.
So that's just there for future proofing the the API.
Um, there's an equivalent sub volume API, which we would expect to be used within the context of CFCSI.
With some by exports.
It works exactly the same as setting the x-atters.
And then finally we have client access guards to prevent in compatible clients.
And the main one would be kernel clients from interacting with the case sensitive case
and sensitive directory because we don't have an implementation yet for that.
But there's a client feature bit that now protects it.
So the MDS will not allow an incompatible client to create files,
create links, et cetera.
But unlink an arm does okay.
So you could mount a kernel client and just nuke a directory if you're an admin.
And you want to do that for some reason.
Yeah, so here's an example.
We're going to set the case in sensitive directory for step of Esther.
We're going to, um, here we're just getting it to have a look.
It's it's a case in sensitive.
The normalization's NFD, the default and the encodings UTF8.
We're going to create this file.
We LS it.
We see that it actually, uh, we get the case full name back when we do the reader.
And then we're going to tell the MDS to dump the cache for that particular address.
So we can just have a look at it.
And here we're just, um, finding that particular directory entry,
ending with SSEN.
And the reason I did that is because it's been case folded and normalized.
So now instead of this, where it's, it's loosened.
But, um, I can actually can't type it correctly on my terminal with the normalization.
And then you can see this is what it looks like, uh, for the, on the MDS site.
And then similarly, uh, if we wanted to, uh, base 64 decode the alternate name,
we would see that, uh, we get back the correct name.
So some closing thoughts, the alternate name, metadata,
turned out to have some more use cases.
I think it's, uh, pretty interesting that that was the case and we found another one so quickly.
And it worked out very well. It's very, very performant.
Um, and now SOMONS have festival and joy, uh,
efficient case and sensitive directory trees.
That, thank you and questions.
Thank you.
Yep, please.
Are any plans for, like, converting existing data to use the new case and sanctitude?
Any plans to convert existing file system trees to be case and sensitive?
Uh, no, we would, you'd have to copy it.
Um, there's no plans to, to change an existing directory tree.
Time's up. Okay.
We're happy to take questions else.