Yeah, hello. One question upfront, who was here in this talk last year? Okay, only very
few people, that's good, because I believe in some slides I copied from last year, just
to give an intro into the topic. So, overall, what's SME3 Unix extensions? Essentially,
what we do want to do is, in drastic words, kill NFS. And that's a very great. I know
this won't happen until I retire, but what can you do? So, why? So, in my world, we have
two dominant file system protocols, the one is NFS, and which is native and historically
native to the Linux, or Unix, or Solaris, or AIX, or whatever. And that's SMB, which comes
from the Windows world. And NFS has really native support in all Unix's, in all Linux's.
It's pretty simple to set up initially. You just edit some files in EDC export, or add some
lines in EDC exports, and it just works. So, you don't have to think about authentication.
You don't have to think about security, nothing it just works, which is very, very handy
for simple setups that you just want to quickly set up and so on. It is reasonably compatible
with what Windows application expect. It's not 100% local coherent. One particularly bad
example is how to delete open files. I hold a file open, I delete it, and in a local file
system like X4, it just disappears from the namespace. The data is still there, it will
still be hailed, held in NFS, NFS has to play tricks and rename that to dot NFS, and then
some long number, and just for the file to still be accessible, while it's still open on
the client. In plain NFS metadata cashing is particularly bad. So, one example is the file M time.
There are applications which depend on the last modification time to be accurate, and this
just doesn't work over NFS just because always if you want to do a stat, the M time can
always change, and you don't need to do cashing, and there is no coherent cashing mechanism
at all. Then, file locking doesn't work at all, for reasonably okay. I can't say it doesn't
really work at all, but it has very bad corner cases. There's a huge discussion about, or there's
a huge section in the NFS before IRFCs, how to make locking work, how does the server notice
when a client has gone? How does a client notice when a server has gone? How do we retain
locking state? How do we establish locking state if both are back? A lot of work. Next point,
there is no file security, and in general, I mean, yes, you can make it secure with courberals,
but I have yet to come across a scalable or scaling large deployment where you have courberals
NFS, and people are happy with it. That combination is the piece that makes it difficult to achieve.
SMB, on the other hand, also similar goal, just share files and directories, and very similar
goal from like 10 miles above, with some differences, it's secure by default. In any sense that
you have RF, authenticated connections, you can easily just enable transport encryption,
transport signing is mandatory anyway for a few release now of the SMB protocol, so
in the default installation, it's much more secure from my point of view than NFS is.
When you have a decent mechanism for cash coherency, I know that NFS we fought out something,
added direct releases, added leases, and they call it delegations, but it's not implemented everywhere,
so SMB has all this really embedded into the core protocol and everybody just implemented.
Locking works in the protocol as such, and this is because Microsoft has to be compatible with local
applications, and you can say a lot of bad things about Microsoft, but they are really good at
one thing, which is application compatibility to the core OS. This means they have to make lock
in work, and it just works across the protocol. There might always be problems, of course,
with implementations, but the protocol provides good locking, and yeah, so it was just everywhere,
and if you just read a little whatever intro, it's not really much more complicated than setting up
ETC exports, adding a line there, and the server will just work. Of course, you have to take into account
this secure by default, you have to take care about users and groups, but yeah, this is a downside
of NFS I would believe. So, from in my world NFS is pretty arcane, and I come from the SMB world,
of course, so take this with a grain of salt. We want to make SMB a competitor to NFS. What does
that mean? There is, of course, the Linux client, by the French, that where you can mount SMB
shares, but they have severe limitations when using these clients against native SMB without these
extensions. In particular, in this is what I'm going to talk about, we have fine-name handling,
we have POSIX metadata, and we have special files. You want to run your SSH-added agent,
you want your SSH agent on your client to open a socket where your home directory is on SIFS,
or, yeah, SMB mount.SIFS, it's still called mount.SIFS. When you have your home directory on
SMB, you want your SSH agent to open a unix domain socket, and that has to live on SMB.
And this is something plain SMB doesn't do properly right now. How do we do this?
The SMB's two protocol is really open. It was designed with extensibility in mind.
The first thing we do is we call it negotiate protocol. Client tells the server,
hey, I have these capabilities. As the first packet, completely unauthenticated.
First packet is, okay, I want to speak this version, I have these extensions and so on.
Please server tell me what you can do. As a client we offer the ability, hey, we are speaking
POSIX extensions with a negotiate context. That's what it's called technically.
Server tells us, yes, I know. I know the extensions and it answers properly. If it doesn't do that,
it just ignores this request and we can now figure out, yes, the server can do
POSIX extensions. What would that activate?
That would activate that when we deal with file names, we are privileged or we can send a
so-called POSIX create context. So, server tells us, yes, I know POSIX. And then when we open
a file, when we start a file essentially, when we look up a file name, we can tell the server,
yes, for this file name, please handle it POSIX like. Which in particular means,
it's case sensitive, because that's one of the main differences between Linux or within NFS,
between NFS and SMB. SMB is case in sensitive. So, you can only have one file test or text.
In NFS, you can have a gazillion file test or text with different upper case lower case
combination. And we can tell on a per call basis, yes, for this file name, please treat it in the
POSIX way. That's long implemented. Then you want to query POSIX metadata. What is it?
If you look at the statuses call, it's structs.net, there are things that POSIX knows a file that
NTFS doesn't natively know. Yes, we have owning group, we have owner and so on. We have a lot of
things in common. For example, one thing we don't have is number of links, number of heart links.
The POSIX permission bits, yes, you could probably hide them in whatever anacle is style of
but the POSIX permission bits are particularly hard to get. So, we added them to a so-called POSIX
information class. I'll talk about that in a minute. We have all this documented as an protocol extension.
Go there. That's just a pure document. That's not code. Go there. And this is, of course, open
for discussion and extensions. Coming back to this, SSH agent. And SSH agent wants to open
or debug server, debug session server. SSH agent wants to create a socket in my home directory.
It can't right now. And there are other cases here. For example, if you want a discless boot,
you also need block devices and character devices. And so on. All the, you want to represent
properly all these file types beyond regular files and directories. Of course, Samba,
since inception has done files and directories, obvious. I mean, otherwise you wouldn't be a file server.
But what do we do about the others? Block devices, character devices, and so on.
What's the block device? What's the character device? Essentially, in the Unix Word,
this is a link to hardware on the local system. I open Dev SDA and I get direct access
to the blocks on my scuzzi disc or whatever is behind Dev SDA with these days.
So if I now present a block device over a network protocol, there's essentially two choices
that you could make in theory. I could say, okay, I'll open Dev SDA over a network protocol.
And the question is, who interprets this as a device? And Dev SDA might still be a reason to say,
okay, I opened Dev SDA over a network protocol and I want to look at the server side
hard drive. Could be reasons for it. I think it's not a good idea, but there could be reasons for that.
The example I gave there, Dev now, you don't want to send the garbage, you want to send to Dev
now over the network for the server to just throw it away. So the idea here is that if a character
or block device is presented over the network, it's only an indication for the client system
to look at the local devices it has. That's the idea.
FIFA's and sockets, what is that? Essentially, that's an interprocess communication mechanism
for processes local on a system. Here as well, you could argue that, okay, I want to open a socket
on the client and I want to talk to the server process. You could argue that. This is in general
not what we want to do, because again, we want to have the SSH agent. I don't want the SSH agent
to run over SMB on the server. I want to look at it locally. So my home directory needs sockets
to be presented like local sockets. Same for FIFA's. We already do handle sockets as a cross,
cross node network interprocess communication mechanism. There's a special share in the Windows
work called IPC Dollar in the process communication. Dollar means not visible. We do that for DCRPC,
printing, remote registry, a whole bunch of others. We have patches pending for the Windows search
protocol, which is not DCRPC but raw pipes. So we already do that in the process communication
mechanism over the network for special cases for this share for the general home directory you don't want this.
Then the last one, some links. There's, I mean, really heated discussions and even the rust
community was hit by a simling race. Everybody was hit by a simling crisis. So the question is,
are some links a good idea overall? I mean, they are there. You could argue and know you don't need
some links. You don't want some links just make them go away. It won't happen. We all know that
the only thing is the server should not bother. And if you look at the NFS RFCs, symbolic links
are not a server thing in the NFS specification. It's only a client thing. Somebody historically
has treated simling's server side. And I will show example, an example later on. We fixed that.
Okay. Locking character devices, sockets, fifos, special types of files. So we could extend the
protocol with all these things. There is a native Windows mechanism called repass points.
That interestingly enough, the Windows based NFS server also uses to represent sockets,
sockets and all the other ones. The NFS server on Windows uses repass points. What repass point?
Essentially, it's a mechanism to extend functionality of the NFS file system. What does it mean?
You take a file and essentially you just create a special extended attribute on that file on
NTFS. When this extended attribute exists, what happens at open time is, the native NTFS
implementation sees, okay, this extended attribute exists. This extended attribute then points
at some DLL on Windows, dynamic link library, shared object, Linux, and when this extended
attribute exists, essentially some function is called to take care of this whatever special
functionality for this file. Essentially, you can intercept open calls. That's what the repass
point is, and this is what they used here. They have a special error message. So you set
a repass point, extended attribute, pointing at some DLL and that DLL is stressed out there.
When trying to open that file, all you get is repass tag not handled, which is fair, because I told
the system, hey, use this whatever hierarchical storage management mechanism to open that file,
but whatever HSM software was installed, repass tag not handled anymore.
And so this is, as I said, a general mechanism to extend NTFS functionality with some magic.
MSFCC, that's one of these several thousand lines of document. Google for this in brackets MSFCC,
and that's one of the documents that Microsoft have published. They define a few dozen repass
tags, which is special, extended attributes. This repass tag does that. This repass tag does this
and so on. They have to define some of these. Some of them, or most of them, are not meaningful
over the wire, which means we don't document, we don't want to document them.
The Windows NFS server just does exactly this. It uses repass points to represent
special files, like Firefox, sockets, and all the others. So when Windows NFS client comes in and does a
make-not or a socket call, what the Windows NFS server does is it creates a file, it attaches a
data name it here, special defined repass tag NFS, one of these special extended attributes,
and as content of this extended attribute, they have the major and minor device number if it's
a block device. This is what they do, and they define subtypes of this extended attribute,
just look up this spec, I mean it's pretty obvious. They have NFS siblings. Interestingly,
Windows properties shows just L for all of these, which means it believes the Windows
explorer application. Beliefs whenever there's one of these repass tags, it's a native Windows
signalling, but that's just a GUI glitch I would say. Okay. So now we have established that
we want to, when a Samba server lists slash dev, you will find all sides of special devices,
if you do it, L is minus L on slash dev. The very first column is the file type.
If you look at the read here, system call, which is get the entities days, but if you look at the read
here documentation, all you get, when you application on Linux does a read here call, all you
get, it's a name essentially. There's more, of course, but all you get is the name.
And if L is minus L, wants to attach nice colors to sockets and nice whatever, there's a pin
for pipes, it has to stat every time. For each of the name, 10 minutes, thank you. For each of the
name, which means round flip to the server, because the NFS read your call only gave names.
This is why they implemented read your plus, which already in that first run gives
the attributes, the relevant attributes. SMB does exactly the same. When you do a read your call
over SMB, you can ask so called info levels. So this is essentially a read your plus plus,
with roughly maybe 10 or 15 different info levels defined with different level of detail.
And what we did is we added another info level for projects. And if you listen to all of these
SMB specific Unix extension talks, there was a hot debate, what style of similar videos,
what a story, what style of repass points we use. There's not only the NFS server, there's other
applications, there's Windows subsystem for Linux using its own repass points, and there are
downside and upsides for each of these specific ways to represent repass, represent special files.
And what we settled upon is, we have extended our struct info level. We have extended
an info level for structs that. And this is the first thing I want to show you here just
upon wire, short trace. So, pause it's read here. Can everybody read that?
large enough. So, going down, going down, going down, find requests. I mean that's, yeah,
that's the way to ask for read the information. And this is a directory with a lot of files in
there. And we could probably extend wire, short to show the name here. But what we do, what we find here
is, it is a name called a file called FIFO. And that's what we think, what's missing in the summary.
So, I just didn't realize in a directory with a FIFO called FIFO. And I mean, that's really pretty
detail. We represented as repass tag NFS. But the point is, the shift client with just repass tag NFS
should have asked what's the content of the repass tag. To find out, is it a socket, is it a FIFO,
is it a block device? And this is something that got an a hunt, hot debate. And the way we just
solve this is, we already have pause-expermissions. And if you look at normal pause-expermissions,
I mean, there's a few bits in the 32 bit mask left out of those nine. And we just defined,
okay, we have space here. The old tactic, okay, don't really extend the protocol, just use some
unused bits. And this is kind of, no, that's one thing that's new since last year.
That LLs minus L is just as efficient as Retier plus NFS. That's one of the things
that are new. And which are, I think, a good way to do things.
Okay. Simnings. And I don't have too much time left. We have three ways, or I think there
was even one more way. We have three ways to repress in Simnings. One is, Windows subsystem
for links, it finds its own Simnings. And if S has yet another version, and then Windows and T has
Simnings. I mean, Windows does have Simnings. They also have hard links, but that's another story.
They have Simnings in NTFS. And we settled on, I already passed Tag Simnings,
the native Windows way, for one specific reason. So LS had Simnings, races are
present all over the place. They have been present in the Simnings server side. We solved those.
But we want to get rid of Simnings handling in the Simnings server completely.
What do we do? When we hit a Simnings, we just tell the client, I had to stop on Simnings
and let the client deal with it. And there's a very nice error message, and I want to show you
that as well. No. Okay, what did I do? I have a directory, and there's a sub directory called A.
This one here. There's a sub directory of that called B. And within that sub directory, there's a
Simnings, that points at B. B, CDC, extra text. What did I do? I try to open the file A slash
D slash C slash X dot text. Oh, that was it. Crab. Here we go.
And this is the error. So you'll see here, I try to open A slash D slash D and so on. That's
what I try to open this here. I hit the Simnings, and what I want to point out here is this
really nice error message, and I like actually like that, which essentially tells me,
if I could now click correctly. Okay, rep unpassed past it. What does it mean?
It gives me this error message when trying to open that file gives me the Simnings target. Remember,
I pointed D or at B. It directly in the error in the return gives me the target. It also tells
me by a this field where the Simnings is in those path components, which means that the client
can easily just from this answer, resolve that Simnings. Doesn't have to go back to the server.
It can directly say, okay, the ultimate target file is this all information there.
One thing around fab, Simnings can be completely on the client. The server doesn't have to solve
Simnings. Okay, I'm a bit in a hurry. Long run in compute jobs. I said NFS secure, SMB secure,
you need user accounts, a long run in compute job. Failure on that. Google for
Daniel Cobras's talk with the GSS proxy. There's some first, I believe, first time talk, I don't know.
Google for Daniel Cobras's GSS proxy talk brilliant works today. That problem solved.
Current status. Our server side is done for all these special files.
Server side returns stop on Simnings and has the client follow the Simnings. So we can represent
Simnings. We can create Simnings. We can properly handle client-created Simnings with this.
Linux 613 has positive special files and Simnings handling already there. Of course,
backspending try to test it. The big missing piece is axles and that's another kind of
warms. Probably that's a talk for next year. How would you like the act of?
So from a server side, I would say pretty much done except for actals. Now we need client integration
client. Clients doing the Simnings handling properly. Our lib SMB client has on that's pending work.
That's it.
One minute. One minute. The question was for extended attributes.
The NFS, sorry, the SMB protocol natively has extended attributes support. So that's
mainly a client thing to actually make use of those calls.
Server side is implemented. Yes.
How about the different checks with the TFS and WSL whatever? Are you implementing them all and
using them at runtime depending on which files to say which brand of windows we have or how
is this going to work on? How are you working? Can you say again?
You showed the different checkpoints for WSL and the TFS whatever the NFS checks.
Isn't them all depending on what's available in the runtime as well?
Question was about these different tags. Do we use them all?
Depending on what the client does. So the client is free to set the WSL reposted.
And the way we do it and this is a slight ascript ascript. If a client does this to us, it just
creates an empty file and sets a special extended attribute. That's what it does. And we don't
care about those contents. This talk is about how do we represent things we find on this?
When some of our finds a native socket on this, how do we represent those? If you list
slash dev, how do you represent these things? And our decision is go with NFS.
But the client is completely free to do anything.
Time's up. I'm here.