Discussion:
Interoperable junctions on Linux
(too old to reply)
Chuck Lever
2013-04-23 15:42:33 UTC
Permalink
Hi Simo-

Thanks for taking the time to read through my post.
Hi-
I led a discussion on Friday at the Linux Storage and Filesystem
Summit on how to store {DFS, FedFS} junctions in Linux filesystems.
I'd like to summarize the discussion, then ask a few follow-up
questions. I apologize in advance for the length.
FedFS is to NFS as Microsoft DFS is to SMB/CIFS. FedFS uses NFS
referrals to glue together a file namespace out of separate shares,
starting with a root share that may contain nothing but referrals.
For more on FedFS, start with RFC 5716.
The physical object that stores referral location target information
is called a junction. It can contain an actual list of locations, or
it can contain the DN of an LDAP record where the location list is
maintained.
+ Samba uses a symlink for this purpose. The contents of the link
represent the location information passed out to CIFS clients.
+ FedFS uses an extended attribute on a directory for this purpose.
The xattr contains XML which encodes location information passed out
to NFS clients.
The reasons for this difference are simply historical.
Linux is often used as a multi-protocol file service platform, where
the same physical data is presented to clients via both NFS and CIFS.
For this reason, we think there would be value in using the same
physical representation for both NFSD and Samba so that a single
junction object on our exported filesystems can trigger a referral for
both NFS and CIFS.
Samba has been around much longer, and DFS support is actually
deployed. FedFS is newer and still experimental. Thus we decided to
change FedFS to use a symlink instead of a directory. Samba will
still use the regular contents of the symlink, and FedFS will store
its metadata in an extended attribute attached to the symlink.
There was a rough consensus to prefer JSON over XML in the FedFS
xattr, though there are still some who dislike both. I'm open to
suggestion, since we're now essentially replacing the existing FedFS
junction format and can change it to anything we like.
Can you give an example or refernce of what is stored in this XML/JSON
blob ? Why do you need structured data there ?
An "NFS basic junction" stores the location list in place. Each item in a location list contains a number of pieces of data, including: a server hostname, an export pathname (which is a list of path components), and a number of integer and boolean settings that help clients sort which replica of this data they should mount.

A full explanation of this data is in RFC 5661, section 11.10. This data is returned to an NFS client when it encounters one of these objects. The client can redirect its requests to one of the servers and exports listed in the returned data.

A "FedFS junction" stores a reference to a location list stored in LDAP. The LDAP server's hostname and port number and the UUID of a FedFS Fileset Name record are stored in the junction. The Fileset Name record has children, each of which constitute a location (see above).

An explanation of this data is in an IETF draft:

http://datatracker.ietf.org/doc/draft-ietf-nfsv4-federated-fs-protocol/

See chapter 4 for an overview of the schema used for these lists. An NFS fileserver converts the LDAP records into an fs_locations4 or fs_locations_info4 attribute for NFS clients. Other protocols use a different representation for communicating this list to clients.
Today FedFS junctions can contain either a location list or an LDAP
DN. One option for FedFS is to support only the LDAP DN junction
type, and have a (possibly local) LDAP service available to store the
location information. The FedFS junction xattr would then always
contain an LDAP URL. Storing complex data types (a list containing
pathnames, hostnames, integers, and other values) would then be up to
LDAP.
Having to install a whole LDAP server as a pre-requisite seem very heavy
handed.
True. Today, the LDAP/NSDB pieces are optional if an admin wants to support only "NFS basic junctions," for just this reason.

However there are certain benefits to allowing location lists to be managed via LDAP, rather than being specified at junction creation. Junctions can share the same location list, for example. A filesystem migration can update a central location list once, rather than having to find every junction that references the migrated filesystem.

In addition, storing these lists in a publicly available LDAP service means that any fileserver, anywhere, can access the lists.

If we are really wily, maybe a small single-purpose daemon can be constructed from a minimal LDAP server implementation (or from scratch), and it can listen on its own port or only for loopback requests.
We will have to discuss a conjunction of administrative interfaces at
some later point. However, we should clarify how our junction
management tools behave now that each junction can have metadata it
did not have before.
nfsref add - if no symlink exists, create it (what contents should
it have?)
- add an extended attribute
nfsref remove - remove the extended attribute, leave the symlink
- can we remove the symlink if its contents are
meaningless?
Why should we leave a symlink ? Don't we expect to remove junctions for
all protocols ?
add - if a symlink already exists, replace its contents, preserving
xattrs
remove - if a FedFS extended attribute exists, leave the symlink
(what contents should it have?)
Why should we leave a symlink ? Don't we expect to remove junctions for
all protocols ?
The difficulty I have is how we are going to conjoin the administrative tools that manage junctions. I imagine that for some time, the tool used for managing DFS junctions will be unaware of FedFS junction content, and vice versa.
What I do not get is why are you trying to use the same mechanism (a
symlink) but then treat them as independent and separate entities ?
What is the aim ?
From your premise I thought you wanted to allow parallel functionality,
ie a DFS created in samba would be seen as a junction for nfs and
vice-versa, but the latter points seem to not allow that ?
FedFS junctions can list both NFS and SMB (and other types). The SMB parts are not defined by the IETF, since SMB is a proprietary protocol controlled by Microsoft.

One way to have DFS and FedFS information in the same filesystem object is to have one object that can contain both. The tools then have to be designed not to step on each other. Eventually we figure out how to make this seamless.

I think you are suggesting we ignore this problem for now, and just have the tools pretend the other protocol does not exist, while still allowing the possibility of storing both types of metadata in the same filesystem object. That may be an easy way to get started.
The FedFS extended attribute is called "trusted.junction.nfs". Should
we rename it?
Shouldn't it be namespaced and use something like trusted.nfs.junction ?
That is probably a matter of taste. I'm not especially attached to the current form. But originally we had the junction information split into several attributes in the "trusted.junction" namespace.
Also why a xattr in the trusted namespace ? What are the security
considerations that warrants a trusted attribute rather than a normal
one ? (Links to RFCs or other docs are just fine)
This is another historical design decision. If there is consensus that we don't need to protect junction metadata from unintended or malicious local changes, then we can put these in another namespace. However, without strong security here, redirecting network clients to another server and export can be hijacked, sending remote users to who knows where. Is it enough simply to insist that junctions be owned by root?
Note that CAP_SYS_ADMIN capabilities are required to access the xattr.
Will that be a problem for Samba administration tools?
It should not be a problem for Samba, as we can retain capabilities if
needed, and we already handle data in xattrs (not for DFS symlinks).
--
Chuck Lever
chuck[dot]lever[at]oracle[dot]com
Myklebust, Trond
2013-04-23 15:51:31 UTC
Permalink
Post by Chuck Lever
Also why a xattr in the trusted namespace ? What are the security
considerations that warrants a trusted attribute rather than a normal
one ? (Links to RFCs or other docs are just fine)
This is another historical design decision. If there is consensus that we don't need to protect junction metadata from unintended or malicious local changes, then we can put these in another namespace. However, without strong security here, redirecting network clients to another server and export can be hijacked, sending remote users to who knows where. Is it enough simply to insist that junctions be owned by root?
Junctions resolve into mountpoints on clients. Allowing arbitrary users
to change the junction parameters basically means giving them the
ability to control the namespace on clients. They can for instance
redirect an application from a trusted server onto an untrusted one.

I therefore strongly recommend that we ensure the creation, deletion and
modification of a junction remains a privileged operation on the server.
--
Trond Myklebust
Linux NFS client maintainer

NetApp
Trond.Myklebust at netapp.com
www.netapp.com
Myklebust, Trond
2013-04-23 16:24:08 UTC
Permalink
-----Original Message-----
From: Simo Sorce [mailto:simo at redhat.com]
Sent: Tuesday, April 23, 2013 12:20 PM
To: Myklebust, Trond
Cc: Chuck Lever; samba-technical at lists.samba.org; fedfs-utils Developers;
Linux NFS Mailing List
Subject: Re: Interoperable junctions on Linux
Post by Myklebust, Trond
Post by Chuck Lever
Also why a xattr in the trusted namespace ? What are the security
considerations that warrants a trusted attribute rather than a
normal one ? (Links to RFCs or other docs are just fine)
This is another historical design decision. If there is consensus that we
don't need to protect junction metadata from unintended or malicious local
changes, then we can put these in another namespace. However, without
strong security here, redirecting network clients to another server and
export can be hijacked, sending remote users to who knows where. Is it
enough simply to insist that junctions be owned by root?
Post by Myklebust, Trond
Junctions resolve into mountpoints on clients. Allowing arbitrary
users to change the junction parameters basically means giving them
the ability to control the namespace on clients. They can for instance
redirect an application from a trusted server onto an untrusted one.
I therefore strongly recommend that we ensure the creation, deletion
and modification of a junction remains a privileged operation on the server.
Is it not sufficient to make sure the symlink is owned by root ?
How do you check that atomically with the getxattr?

Trond
Myklebust, Trond
2013-04-23 16:30:27 UTC
Permalink
-----Original Message-----
From: Simo Sorce [mailto:simo at redhat.com]
Sent: Tuesday, April 23, 2013 12:29 PM
To: Myklebust, Trond
Cc: Chuck Lever; samba-technical at lists.samba.org; fedfs-utils Developers;
Linux NFS Mailing List
Subject: RE: Interoperable junctions on Linux
Post by Myklebust, Trond
-----Original Message-----
From: Simo Sorce [mailto:simo at redhat.com]
Sent: Tuesday, April 23, 2013 12:20 PM
To: Myklebust, Trond
Cc: Chuck Lever; samba-technical at lists.samba.org; fedfs-utils
Developers; Linux NFS Mailing List
Subject: Re: Interoperable junctions on Linux
Post by Myklebust, Trond
Post by Chuck Lever
Also why a xattr in the trusted namespace ? What are the
security considerations that warrants a trusted attribute
rather than a normal one ? (Links to RFCs or other docs are
just fine)
This is another historical design decision. If there is consensus that we
don't need to protect junction metadata from unintended or malicious
local changes, then we can put these in another namespace. However,
without strong security here, redirecting network clients to another
server and export can be hijacked, sending remote users to who knows
where. Is it enough simply to insist that junctions be owned by root?
Post by Myklebust, Trond
Junctions resolve into mountpoints on clients. Allowing arbitrary
users to change the junction parameters basically means giving
them the ability to control the namespace on clients. They can for
instance redirect an application from a trusted server onto an untrusted
one.
Post by Myklebust, Trond
Post by Myklebust, Trond
I therefore strongly recommend that we ensure the creation,
deletion and modification of a junction remains a privileged operation
on the server.
Post by Myklebust, Trond
Is it not sufficient to make sure the symlink is owned by root ?
How do you check that atomically with the getxattr?
Using fgetxattr() after an open and a fstat() ?
So, 3 operations in order to do the same thing you can do with just 1 if you choose a trusted xattr?

Trond
Simo Sorce
2013-04-23 16:28:43 UTC
Permalink
Post by Myklebust, Trond
-----Original Message-----
From: Simo Sorce [mailto:simo at redhat.com]
Sent: Tuesday, April 23, 2013 12:20 PM
To: Myklebust, Trond
Cc: Chuck Lever; samba-technical at lists.samba.org; fedfs-utils Developers;
Linux NFS Mailing List
Subject: Re: Interoperable junctions on Linux
Post by Myklebust, Trond
Post by Chuck Lever
Also why a xattr in the trusted namespace ? What are the security
considerations that warrants a trusted attribute rather than a
normal one ? (Links to RFCs or other docs are just fine)
This is another historical design decision. If there is consensus that we
don't need to protect junction metadata from unintended or malicious local
changes, then we can put these in another namespace. However, without
strong security here, redirecting network clients to another server and
export can be hijacked, sending remote users to who knows where. Is it
enough simply to insist that junctions be owned by root?
Post by Myklebust, Trond
Junctions resolve into mountpoints on clients. Allowing arbitrary
users to change the junction parameters basically means giving them
the ability to control the namespace on clients. They can for instance
redirect an application from a trusted server onto an untrusted one.
I therefore strongly recommend that we ensure the creation, deletion
and modification of a junction remains a privileged operation on the server.
Is it not sufficient to make sure the symlink is owned by root ?
How do you check that atomically with the getxattr?
Using fgetxattr() after an open and a fstat() ?

Simo.
--
Simo Sorce * Red Hat, Inc * New York
Simo Sorce
2013-04-23 16:19:32 UTC
Permalink
Post by Myklebust, Trond
Post by Chuck Lever
Also why a xattr in the trusted namespace ? What are the security
considerations that warrants a trusted attribute rather than a normal
one ? (Links to RFCs or other docs are just fine)
This is another historical design decision. If there is consensus that we don't need to protect junction metadata from unintended or malicious local changes, then we can put these in another namespace. However, without strong security here, redirecting network clients to another server and export can be hijacked, sending remote users to who knows where. Is it enough simply to insist that junctions be owned by root?
Junctions resolve into mountpoints on clients. Allowing arbitrary users
to change the junction parameters basically means giving them the
ability to control the namespace on clients. They can for instance
redirect an application from a trusted server onto an untrusted one.
I therefore strongly recommend that we ensure the creation, deletion and
modification of a junction remains a privileged operation on the server.
Is it not sufficient to make sure the symlink is owned by root ?

Simo.
--
Simo Sorce * Red Hat, Inc * New York
Chuck Lever
2013-04-23 18:11:31 UTC
Permalink
Post by Chuck Lever
Can you give an example or refernce of what is stored in this XML/JSON
blob ? Why do you need structured data there ?
An "NFS basic junction" stores the location list in place. Each item
in a location list contains a number of pieces of data, including: a
server hostname, an export pathname (which is a list of path
components), and a number of integer and boolean settings that help
clients sort which replica of this data they should mount.
A full explanation of this data is in RFC 5661, section 11.10. This
data is returned to an NFS client when it encounters one of these
objects. The client can redirect its requests to one of the servers
and exports listed in the returned data.
A "FedFS junction" stores a reference to a location list stored in
LDAP. The LDAP server's hostname and port number and the UUID of a
FedFS Fileset Name record are stored in the junction. The Fileset
Name record has children, each of which constitute a location (see
above).
I do not see mention of LDAP URIs, andyou seem to speak in the singular
when mention 'the LDAP server' does it mean you hve no way to specify a
pool of LDAP servers for HA ?
An "NSDB name" (used in a junction) is meant to represent a pool of replicated LDAP server resources, or just one LDAP server. Note that specifying IP addresses in FedFS Fileset Names is specifically disallowed for this reason.

A FedFS Fileset Name can indeed be represented by an LDAP URI. See the operational examples in the FedFS NSDB draft.
Post by Chuck Lever
http://datatracker.ietf.org/doc/draft-ietf-nfsv4-federated-fs-protocol/
See chapter 4 for an overview of the schema used for these lists. An
NFS fileserver converts the LDAP records into an fs_locations4 or
fs_locations_info4 attribute for NFS clients. Other protocols use a
different representation for communicating this list to clients.
To be honest the constrains in this document on the LDAP DIT, seem to
indicate it will be possible to use this stuff primarily only with a
dedicated LDAP server, Dictating how the rootDSE/namingContext should
look like is a quite strong demand.
Why do you need a full namingContext for FedFS ?
Why a subtree is not sufficient ?
The document does recommend using a dedicated LDAP service for an NSDB, but does not require it.

One or more of the namingContext records on an NSDB has a new attribute (fedfsNceDN) that points to the FedFS DIT. So it is entirely possible to use an existing LDAP server for storing FedFS records, simply by adding that attribute to the namingContext under which the FedFS subtree resides.

The goal is to allow automated discovery of the location of the FedFS DIT using a DN advertised in the LDAP server's rootDSE. NSDB clients thus need know only the name of the NSDB service.

By default, NSDBs that my tools set up store the FedFS DIT under the LDAP server's domain controller suffix, rather than using the special "o=fedfs" DN. By convention the FedFS domain name matches the domain controller suffix, but operation doesn't depend on this.

It is not quite as restrictive as you think, but it is very fiddly to set up on industrial strength LDAP servers like OpenLDAP or 389-ds.
On the security side, you recommend RPCSEC_GSS for NFS, but then TLS for
LDAP, why not use SASL/GSSAPI for LDAP as well so you need a single set
of credentials ?
There are two classes of tasks involving NSDBs: administrative, where an administrator is available to provide credential material; and fileserver, which is an unattended access performed simply to retrieve location lists.

The purpose of TLS is to allow fileservers to authenticate NSDBs without requiring user intervention. A certificate chain for each NSDB is provisioned on fileservers (using the FedFS ADMIN RPC protocol). The fileservers use them with START_TLS each time a connection is established to that NSDB.

SASL/GSSAPI can of course be used when performing administrative operations on an NSDB, and that is the plan for the Linux FedFS implementation. As you observed, an administrator can then use the same credentials for an RPCGSS-protected FedFS ADMIN service and her NSDB.
Post by Chuck Lever
Today FedFS junctions can contain either a location list or an LDAP
DN. One option for FedFS is to support only the LDAP DN junction
type, and have a (possibly local) LDAP service available to store the
location information. The FedFS junction xattr would then always
contain an LDAP URL. Storing complex data types (a list containing
pathnames, hostnames, integers, and other values) would then be up to
LDAP.
Having to install a whole LDAP server as a pre-requisite seem very heavy
handed.
True. Today, the LDAP/NSDB pieces are optional if an admin wants to
support only "NFS basic junctions," for just this reason.
However there are certain benefits to allowing location lists to be
managed via LDAP, rather than being specified at junction creation.
Junctions can share the same location list, for example. A filesystem
migration can update a central location list once, rather than having
to find every junction that references the migrated filesystem.
Understood.
Post by Chuck Lever
In addition, storing these lists in a publicly available LDAP service
means that any fileserver, anywhere, can access the lists.
If we are really wily, maybe a small single-purpose daemon can be
constructed from a minimal LDAP server implementation (or from
scratch), and it can listen on its own port or only for loopback
requests.
I think in most cases this is what will actually happen, but you do not
need to use a special purpose built server, you can use an existing LDAP
server simply specially configured for your needs. It will cause
administrative overhead to handle this infrastructure though.
Don't I know it.

To simplify initial configuration, I've implemented a script that installs OpenLDAP and performs enough configuration to provide a stand-alone NSDB, with or without TLS. I already have a number of C tools that can set up the rootDSE and add Fileset Name records on either 389-ds or OpenLDAP servers.

The idea is to allow both types of NSDB (stand-alone, and as part of an existing LDAP service), but that kind of generality adds a lot of complexity to the set up process and documentation. Going with just one or the other makes life much easier.
[..]
Post by Chuck Lever
Why should we leave a symlink ? Don't we expect to remove junctions for
all protocols ?
The difficulty I have is how we are going to conjoin the
administrative tools that manage junctions. I imagine that for some
time, the tool used for managing DFS junctions will be unaware of
FedFS junction content, and vice versa.
Yes, for some time one may end up wiping out the other.
Post by Chuck Lever
What I do not get is why are you trying to use the same mechanism (a
symlink) but then treat them as independent and separate entities ?
What is the aim ?
From your premise I thought you wanted to allow parallel functionality,
ie a DFS created in samba would be seen as a junction for nfs and
vice-versa, but the latter points seem to not allow that ?
FedFS junctions can list both NFS and SMB (and other types). The SMB
parts are not defined by the IETF, since SMB is a proprietary protocol
controlled by Microsoft.
One way to have DFS and FedFS information in the same filesystem
object is to have one object that can contain both. The tools then
have to be designed not to step on each other. Eventually we figure
out how to make this seamless.
I think you are suggesting we ignore this problem for now, and just
have the tools pretend the other protocol does not exist, while still
allowing the possibility of storing both types of metadata in the same
filesystem object. That may be an easy way to get started.
I thought this was what you were proposing actually. With my
'integrator' hat on I would rather quickly define common tools that can
handle both, and have the old tools return loud warnings (were possible)
if you try to use them.
The "old" FedFS tools in this case are available because they are one of two independent reference implementations required for the IETF standardization process. However, I think solving the problem of integrating FedFS ADMIN with an existing DFS implementation on a multi-protocol fileserver is going to be very valuable implementation experience that can be fed back into the IETF standards process.

I'm not confident we can fit rpc.fedfsd (Linux's implementation of the FedFS RPC ADMIN service) into this model, so we may end up abandoning its use for managing FedFS junctions, leaving it only to provisioning NSDB certificates.
In the Samba case the basic tool is 'ln', not sure we can do much about
it :-) But we can certainly patch the RPC code that allows handling via
SMB/RPC although I am not quite sure how to populate all the data the
SMB world has no concept of, perhaps using 'good defaults' ...
Exactly the problem.

My thought was to define a place-holder value for both protocols that can be used by our existing tools, for now. No xattr for symlinks created on behalf of DFS, and maybe an empty symlink for junctions created by rpc.fedfsd or the nfsref command.
--
Chuck Lever
chuck[dot]lever[at]oracle[dot]com
Chuck Lever
2013-04-23 19:52:27 UTC
Permalink
Post by Chuck Lever
I do not see mention of LDAP URIs, andyou seem to speak in the singular
when mention 'the LDAP server' does it mean you hve no way to specify a
pool of LDAP servers for HA ?
An "NSDB name" (used in a junction) is meant to represent a pool of
replicated LDAP server resources, or just one LDAP server. Note that
specifying IP addresses in FedFS Fileset Names is specifically
disallowed for this reason.
A FedFS Fileset Name can indeed be represented by an LDAP URI. See
the operational examples in the FedFS NSDB draft.
Sorry I found that after some more searching, the document is vast and I
missed it at first glance.
Post by Chuck Lever
Post by Chuck Lever
http://datatracker.ietf.org/doc/draft-ietf-nfsv4-federated-fs-protocol/
See chapter 4 for an overview of the schema used for these lists. An
NFS fileserver converts the LDAP records into an fs_locations4 or
fs_locations_info4 attribute for NFS clients. Other protocols use a
different representation for communicating this list to clients.
To be honest the constrains in this document on the LDAP DIT, seem to
indicate it will be possible to use this stuff primarily only with a
dedicated LDAP server, Dictating how the rootDSE/namingContext should
look like is a quite strong demand.
Why do you need a full namingContext for FedFS ?
Why a subtree is not sufficient ?
The document does recommend using a dedicated LDAP service for an
NSDB, but does not require it.
Ah I didn't realize you recommend a dedicated LDAP server.
Post by Chuck Lever
One or more of the namingContext records on an NSDB has a new
attribute (fedfsNceDN) that points to the FedFS DIT. So it is
entirely possible to use an existing LDAP server for storing FedFS
records, simply by adding that attribute to the namingContext under
which the FedFS subtree resides.
That 'simply' is not so simple, you may not be allowed to do that.
In which case, you set up your own LDAP server, where you _are_ allowed to do it.
Post by Chuck Lever
The goal is to allow automated discovery of the location of the FedFS
DIT using a DN advertised in the LDAP server's rootDSE. NSDB clients
thus need know only the name of the NSDB service.
You can do the same w/o forcing the attribute to be on the base suffix,
all you need to do is a subtree search with
'&(objctclass=fedfsNsdbContainerInfo)' as part of the filter.
Given this objectclass is used as the container entry main objectclass
it also means the fedfsNceDN would be redundant and could be removed
entirely.
Would your search filter be able to find a fedfsNsdbContainerInfo objectclass in a namingContext _and_ under it? The search base is determined first by this search, then the NSDB client can search for a particular Fileset Name under that base DN.

Although I have been working to simplify this mechanism as specified in the NSDB document, I don't remember the specific reason we require namingContext changes (I inherited this work from the original authors). I agree that a subtree search would be simpler.

But we may be too far along in the standards process to address this concern in this iteration of the protocol. If we can't fix this protocol specification, it's certainly possible to construct our own implementation without fedfsNceDn that does not interoperate, and then document our implementation in a new protocol specification.
Post by Chuck Lever
By default, NSDBs that my tools set up store the FedFS DIT under the
LDAP server's domain controller suffix, rather than using the special
"o=fedfs" DN. By convention the FedFS domain name matches the domain
controller suffix, but operation doesn't depend on this.
It is not quite as restrictive as you think, but it is very fiddly to
set up on industrial strength LDAP servers like OpenLDAP or 389-ds.
Well I have some knowledge in this field, and I do not find it very
fiddly, but I may be biased as I have been working for the past 7 years
to make LDAP+Kerberos simple to manage within the FreeIPA project.
Your requirement to set an objectclass on the base suffix is something I
find particularly unappealing, and no other tool that I know of requires
this (because it is unnecessary).
I presented the NSDB schema to both OpenLDAP and 389-ds developers last year. Neither group suggested that fiddling with an LDAP server's rootDSE was onerous (although they didn't like the way we were doing it, and that has since been changed).

If you can provide back up documentation for your root suffix issue, I can put this in front of the Working Group. "It's particularly unappealing" is not going to fly ;-) Can you help me formulate a proper problem statement (off list, of course)?
Post by Chuck Lever
On the security side, you recommend RPCSEC_GSS for NFS, but then TLS
for
LDAP, why not use SASL/GSSAPI for LDAP as well so you need a single
set
of credentials ?
There are two classes of tasks involving NSDBs: administrative, where
an administrator is available to provide credential material; and
fileserver, which is an unattended access performed simply to retrieve
location lists.
The purpose of TLS is to allow fileservers to authenticate NSDBs
without requiring user intervention. A certificate chain for each
NSDB is provisioned on fileservers (using the FedFS ADMIN RPC
protocol). The fileservers use them with START_TLS each time a
connection is established to that NSDB.
SASL/GSSAPI can of course be used when performing administrative
operations on an NSDB, and that is the plan for the Linux FedFS
implementation. As you observed, an administrator can then use the
same credentials for an RPCGSS-protected FedFS ADMIN service and her
NSDB.
SASL/GSSAPI can also be used by the file server given the use of
RPCSEC_GSS requires you to have a keytab there. This means I have to
care only to drop a keytab and I get mutual authentication and
encryption of the channel to LDAP w/o having to care to get a x509
certificate and configure the server to trust it directly or drop a
suitable CA Certificate.
I find that properly configuring SSL is a lot more difficult than simply
dropping a keytab in place (especially given you alredy need to do it).
Basically in most setups I encounter for real SSL is effectively
"disabled" and by that I mean that peers are configured to ignore
untrusted certificates, also certificates expire and the whole lifecycle
of using SSL certificates *properly* is a lot more complex than using
GSSAPI with a simple keytab file.
Having just implemented TLS support in my NSDB client, I can attest to the challenges.
So I strongly recommend you put language in your RFC allowing
SASL/GSSAPI as a mechanism for securing communication to the LDAP server
for both administrative and simple retrieval, you will do a favor to the
implementer allowing conformance w/o having to jump though hoops.
We have discussed SASL/GSSAPI with LDAP server implementors, and it is already on the radar. It should be possible to introduce with another FEDFS_SEC connection security type, but the plan was to introduce SASL/GSSAPI in the next version of the NSDB protocol.

Again, documentation references would help me formulate a problem statement to address this at this late date, and we can still take the approach of building something that doesn't interoperate for now if we think this is a strong requirement.
[Sorry for the OT, but this really is something you should get right]
Post by Chuck Lever
Post by Chuck Lever
If we are really wily, maybe a small single-purpose daemon can be
constructed from a minimal LDAP server implementation (or from
scratch), and it can listen on its own port or only for loopback
requests.
I think in most cases this is what will actually happen, but you do not
need to use a special purpose built server, you can use an existing LDAP
server simply specially configured for your needs. It will cause
administrative overhead to handle this infrastructure though.
Don't I know it.
To simplify initial configuration, I've implemented a script that
installs OpenLDAP and performs enough configuration to provide a
stand-alone NSDB, with or without TLS. I already have a number of C
tools that can set up the rootDSE and add Fileset Name records on
either 389-ds or OpenLDAP servers.
The idea is to allow both types of NSDB (stand-alone, and as part of
an existing LDAP service), but that kind of generality adds a lot of
complexity to the set up process and documentation. Going with just
one or the other makes life much easier.
I think you should allow the broadest possibilities of course, which is
why I am picking on things like allowing SSAL/GSSAPI explicitly in the
RFC language.
Because there are so many moving parts to FedFS, I have striven to make set up as simple as possible. Just because the spec says you can do something, doesn't mean it is wise to do it, or that you should offer it as an option, nor is it appropriate for every implementation.

Practically speaking, implementation resources are limited. I have to pick the features that offer the most bang for the buck. I can't afford to implement everything and see which features are appealing. (And besides, I thought the idea of open source was to let people scratch their own itches -- if someone wants a feature, they have the opportunity to add it themselves).
Whether people will integrate into existing LDAP server or
not remains to be seen, if we can avoid the need to add an objectlass on
the root suffix I see that we can easily add this a standard feature for
FreeIPA as well (we already provide automount data for example) and
provide management tools in our framework around it.
Does FreeIPA manage DFS metadata as well?
Post by Chuck Lever
Post by Chuck Lever
I think you are suggesting we ignore this problem for now, and just
have the tools pretend the other protocol does not exist, while still
allowing the possibility of storing both types of metadata in the same
filesystem object. That may be an easy way to get started.
I thought this was what you were proposing actually. With my
'integrator' hat on I would rather quickly define common tools that can
handle both, and have the old tools return loud warnings (were possible)
if you try to use them.
The "old" FedFS tools in this case are available because they are one
of two independent reference implementations required for the IETF
standardization process. However, I think solving the problem of
integrating FedFS ADMIN with an existing DFS implementation on a
multi-protocol fileserver is going to be very valuable implementation
experience that can be fed back into the IETF standards process.
I'm not confident we can fit rpc.fedfsd (Linux's implementation of the
FedFS RPC ADMIN service) into this model, so we may end up abandoning
its use for managing FedFS junctions, leaving it only to provisioning
NSDB certificates.
In the Samba case the basic tool is 'ln', not sure we can do much about
it :-) But we can certainly patch the RPC code that allows handling via
SMB/RPC although I am not quite sure how to populate all the data the
SMB world has no concept of, perhaps using 'good defaults' ...
Exactly the problem.
My thought was to define a place-holder value for both protocols that
can be used by our existing tools, for now. No xattr for symlinks
created on behalf of DFS, and maybe an empty symlink for junctions
created by rpc.fedfsd or the nfsref command.
Sound like a good first start to me.
--
Chuck Lever
chuck[dot]lever[at]oracle[dot]com
Simo Sorce
2013-04-23 20:33:06 UTC
Permalink
Post by Chuck Lever
Post by Chuck Lever
One or more of the namingContext records on an NSDB has a new
attribute (fedfsNceDN) that points to the FedFS DIT. So it is
entirely possible to use an existing LDAP server for storing FedFS
records, simply by adding that attribute to the namingContext under
which the FedFS subtree resides.
That 'simply' is not so simple, you may not be allowed to do that.
In which case, you set up your own LDAP server, where you _are_ allowed to do it.
Of course, if forced that is an option, but I would rather see to it
that we do not add barriers to adoption that force people to deploy
additional LDAP servers where possible.
Post by Chuck Lever
Post by Chuck Lever
The goal is to allow automated discovery of the location of the FedFS
DIT using a DN advertised in the LDAP server's rootDSE. NSDB clients
thus need know only the name of the NSDB service.
You can do the same w/o forcing the attribute to be on the base suffix,
all you need to do is a subtree search with
'&(objctclass=fedfsNsdbContainerInfo)' as part of the filter.
Given this objectclass is used as the container entry main objectclass
it also means the fedfsNceDN would be redundant and could be removed
entirely.
Would your search filter be able to find a fedfsNsdbContainerInfo
objectclass in a namingContext _and_ under it? The search base is
determined first by this search, then the NSDB client can search for a
particular Fileset Name under that base DN.
That's what subtree searches are for, if your search has scope subtree
you'll find *any* entry that has that objectclass in the specific
namingContext you are using for the search.
Post by Chuck Lever
Although I have been working to simplify this mechanism as specified
in the NSDB document, I don't remember the specific reason we require
namingContext changes (I inherited this work from the original
authors). I agree that a subtree search would be simpler.
It is, it would be nice if you'd allow that.
Post by Chuck Lever
But we may be too far along in the standards process to address this
concern in this iteration of the protocol. If we can't fix this
protocol specification, it's certainly possible to construct our own
implementation without fedfsNceDn that does not interoperate, and then
document our implementation in a new protocol specification.
Well the change is not huge, you could allow both with minor language
changes. Knowing there is an interoperability issue and not addressing
it when it is simple to do so is not really friendly to implementers.
Post by Chuck Lever
Post by Chuck Lever
By default, NSDBs that my tools set up store the FedFS DIT under the
LDAP server's domain controller suffix, rather than using the special
"o=fedfs" DN. By convention the FedFS domain name matches the domain
controller suffix, but operation doesn't depend on this.
It is not quite as restrictive as you think, but it is very fiddly to
set up on industrial strength LDAP servers like OpenLDAP or 389-ds.
Well I have some knowledge in this field, and I do not find it very
fiddly, but I may be biased as I have been working for the past 7 years
to make LDAP+Kerberos simple to manage within the FreeIPA project.
Your requirement to set an objectclass on the base suffix is something I
find particularly unappealing, and no other tool that I know of requires
this (because it is unnecessary).
I presented the NSDB schema to both OpenLDAP and 389-ds developers
last year. Neither group suggested that fiddling with an LDAP
server's rootDSE was onerous (although they didn't like the way we
were doing it, and that has since been changed).
I have the integrator perspective, which differ from the core developers
perspective at times.
Post by Chuck Lever
If you can provide back up documentation for your root suffix issue, I
can put this in front of the Working Group. "It's particularly
unappealing" is not going to fly ;-) Can you help me formulate a
proper problem statement (off list, of course)?
I am not sure what documentation you require, what you ask *can* be done
to an LDAP server, it's not like your are doing something forbidden. It
is just not what all other extensions do. In FreeIPA I would be
uncomfortable adding that stuff in the root suffix, doesn't mean I will
Veto it, but I might see if I can get away not following the standard in
that regard if we come to integrate this as a default feature.

For Active Directory admins (should somoene want to use AD for this
information) changing the root suffix may also be very unappealing (if
possible at all), although they will find extending the schema
problematic already.

It is a matter of best practices and 'acceptability' more than anything.
It is more acceptable as a solution if you do not have this particular
constraint, which is technically unnecessary so I am not sure why the
RFC process should be dead set on mandating it. Surely you can relax
some MUST here and there ?
Post by Chuck Lever
SASL/GSSAPI can also be used by the file server given the use of
RPCSEC_GSS requires you to have a keytab there. This means I have to
care only to drop a keytab and I get mutual authentication and
encryption of the channel to LDAP w/o having to care to get a x509
certificate and configure the server to trust it directly or drop a
suitable CA Certificate.
I find that properly configuring SSL is a lot more difficult than simply
dropping a keytab in place (especially given you alredy need to do it).
Basically in most setups I encounter for real SSL is effectively
"disabled" and by that I mean that peers are configured to ignore
untrusted certificates, also certificates expire and the whole lifecycle
of using SSL certificates *properly* is a lot more complex than using
GSSAPI with a simple keytab file.
Having just implemented TLS support in my NSDB client, I can attest to the challenges.
So I strongly recommend you put language in your RFC allowing
SASL/GSSAPI as a mechanism for securing communication to the LDAP server
for both administrative and simple retrieval, you will do a favor to the
implementer allowing conformance w/o having to jump though hoops.
We have discussed SASL/GSSAPI with LDAP server implementors, and it is
already on the radar. It should be possible to introduce with another
FEDFS_SEC connection security type, but the plan was to introduce
SASL/GSSAPI in the next version of the NSDB protocol.
Ok, if it is on your radar I am fine. Not sure why you say NSDB
'protocol' I am assuming this is the LDAP protocol, or are you planning
on deviating from LDAP ??
Post by Chuck Lever
Again, documentation references would help me formulate a problem
statement to address this at this late date, and we can still take the
approach of building something that doesn't interoperate for now if we
think this is a strong requirement.
What sort of documentation ?

SASL/GSSAPI is a standard SASL mechanism and SASL is a
standard ,echanism for LDAP binds, so all the references should be easy.

Using SASL/GSSAPI would make it easier for some directory services that
are tightly integrated with Kerberos.
In my case, the FreeIPA project uses SASl/GSSAPI with LDAP, although we
do configure and support also STRAT_TLS by default.
In the Active Directory case SSL is not enabled by default (and it is
hard and unconvenient to configure), while SASL/GSSAPI is.

So SASL/GSSAPI would be an easier route for adoption in some circles.
Post by Chuck Lever
I think you should allow the broadest possibilities of course, which is
why I am picking on things like allowing SSAL/GSSAPI explicitly in the
RFC language.
Because there are so many moving parts to FedFS, I have striven to
make set up as simple as possible. Just because the spec says you can
do something, doesn't mean it is wise to do it, or that you should
offer it as an option, nor is it appropriate for every implementation.
I think we have al;ready determined SASL/GSSAPI is actually simpler than
TLS, so if you are striving for simplicity... :)
Post by Chuck Lever
Practically speaking, implementation resources are limited. I have to
pick the features that offer the most bang for the buck. I can't
afford to implement everything and see which features are appealing.
(And besides, I thought the idea of open source was to let people
scratch their own itches -- if someone wants a feature, they have the
opportunity to add it themselves).
Not sure what there is to implement, if you are using a LDAP library in
your client, switching on SASL/GSSAPI is a very few lines of code if
any.
Post by Chuck Lever
Whether people will integrate into existing LDAP server or
not remains to be seen, if we can avoid the need to add an objectlass on
the root suffix I see that we can easily add this a standard feature for
FreeIPA as well (we already provide automount data for example) and
provide management tools in our framework around it.
Does FreeIPA manage DFS metadata as well?
No, DFS data requires Active Directory in the CIFS worls, and freeIPA is
not an AD implementation. FreeIPA is focused on Linux clients though so
FedFS junctions may be appealing to support.

Simo.
--
Simo Sorce * Red Hat, Inc * New York
Simo Sorce
2013-04-23 18:37:42 UTC
Permalink
Post by Chuck Lever
I do not see mention of LDAP URIs, andyou seem to speak in the singular
when mention 'the LDAP server' does it mean you hve no way to specify a
pool of LDAP servers for HA ?
An "NSDB name" (used in a junction) is meant to represent a pool of
replicated LDAP server resources, or just one LDAP server. Note that
specifying IP addresses in FedFS Fileset Names is specifically
disallowed for this reason.
A FedFS Fileset Name can indeed be represented by an LDAP URI. See
the operational examples in the FedFS NSDB draft.
Sorry I found that after some more searching, the document is vast and I
missed it at first glance.
Post by Chuck Lever
Post by Chuck Lever
http://datatracker.ietf.org/doc/draft-ietf-nfsv4-federated-fs-protocol/
See chapter 4 for an overview of the schema used for these lists. An
NFS fileserver converts the LDAP records into an fs_locations4 or
fs_locations_info4 attribute for NFS clients. Other protocols use a
different representation for communicating this list to clients.
To be honest the constrains in this document on the LDAP DIT, seem to
indicate it will be possible to use this stuff primarily only with a
dedicated LDAP server, Dictating how the rootDSE/namingContext should
look like is a quite strong demand.
Why do you need a full namingContext for FedFS ?
Why a subtree is not sufficient ?
The document does recommend using a dedicated LDAP service for an
NSDB, but does not require it.
Ah I didn't realize you recommend a dedicated LDAP server.
Post by Chuck Lever
One or more of the namingContext records on an NSDB has a new
attribute (fedfsNceDN) that points to the FedFS DIT. So it is
entirely possible to use an existing LDAP server for storing FedFS
records, simply by adding that attribute to the namingContext under
which the FedFS subtree resides.
That 'simply' is not so simple, you may not be allowed to do that.
Post by Chuck Lever
The goal is to allow automated discovery of the location of the FedFS
DIT using a DN advertised in the LDAP server's rootDSE. NSDB clients
thus need know only the name of the NSDB service.
You can do the same w/o forcing the attribute to be on the base suffix,
all you need to do is a subtree search with
'&(objctclass=fedfsNsdbContainerInfo)' as part of the filter.
Given this objectclass is used as the container entry main objectclass
it also means the fedfsNceDN would be redundant and could be removed
entirely.
Post by Chuck Lever
By default, NSDBs that my tools set up store the FedFS DIT under the
LDAP server's domain controller suffix, rather than using the special
"o=fedfs" DN. By convention the FedFS domain name matches the domain
controller suffix, but operation doesn't depend on this.
It is not quite as restrictive as you think, but it is very fiddly to
set up on industrial strength LDAP servers like OpenLDAP or 389-ds.
Well I have some knowledge in this field, and I do not find it very
fiddly, but I may be biased as I have been working for the past 7 years
to make LDAP+Kerberos simple to manage within the FreeIPA project.
Your requirement to set an objectclass on the base suffix is something I
find particularly unappealing, and no other tool that I know of requires
this (because it is unnecessary).
Post by Chuck Lever
On the security side, you recommend RPCSEC_GSS for NFS, but then TLS
for
LDAP, why not use SASL/GSSAPI for LDAP as well so you need a single
set
of credentials ?
There are two classes of tasks involving NSDBs: administrative, where
an administrator is available to provide credential material; and
fileserver, which is an unattended access performed simply to retrieve
location lists.
The purpose of TLS is to allow fileservers to authenticate NSDBs
without requiring user intervention. A certificate chain for each
NSDB is provisioned on fileservers (using the FedFS ADMIN RPC
protocol). The fileservers use them with START_TLS each time a
connection is established to that NSDB.
SASL/GSSAPI can of course be used when performing administrative
operations on an NSDB, and that is the plan for the Linux FedFS
implementation. As you observed, an administrator can then use the
same credentials for an RPCGSS-protected FedFS ADMIN service and her
NSDB.
SASL/GSSAPI can also be used by the file server given the use of
RPCSEC_GSS requires you to have a keytab there. This means I have to
care only to drop a keytab and I get mutual authentication and
encryption of the channel to LDAP w/o having to care to get a x509
certificate and configure the server to trust it directly or drop a
suitable CA Certificate.

I find that properly configuring SSL is a lot more difficult than simply
dropping a keytab in place (especially given you alredy need to do it).

Basically in most setups I encounter for real SSL is effectively
"disabled" and by that I mean that peers are configured to ignore
untrusted certificates, also certificates expire and the whole lifecycle
of using SSL certificates *properly* is a lot more complex than using
GSSAPI with a simple keytab file.

So I strongly recommend you put language in your RFC allowing
SASL/GSSAPI as a mechanism for securing communication to the LDAP server
for both administrative and simple retrieval, you will do a favor to the
implementer allowing conformance w/o having to jump though hoops.


[Sorry for the OT, but this really is something you should get right]
Post by Chuck Lever
Post by Chuck Lever
If we are really wily, maybe a small single-purpose daemon can be
constructed from a minimal LDAP server implementation (or from
scratch), and it can listen on its own port or only for loopback
requests.
I think in most cases this is what will actually happen, but you do not
need to use a special purpose built server, you can use an existing LDAP
server simply specially configured for your needs. It will cause
administrative overhead to handle this infrastructure though.
Don't I know it.
To simplify initial configuration, I've implemented a script that
installs OpenLDAP and performs enough configuration to provide a
stand-alone NSDB, with or without TLS. I already have a number of C
tools that can set up the rootDSE and add Fileset Name records on
either 389-ds or OpenLDAP servers.
The idea is to allow both types of NSDB (stand-alone, and as part of
an existing LDAP service), but that kind of generality adds a lot of
complexity to the set up process and documentation. Going with just
one or the other makes life much easier.
I think you should allow the broadest possibilities of course, which is
why I am picking on things like allowing SSAL/GSSAPI explicitly in the
RFC language. Whether people will integrate into existing LDAP server or
not remains to be seen, if we can avoid the need to add an objectlass on
the root suffix I see that we can easily add this a standard feature for
FreeIPA as well (we already provide automount data for example) and
provide management tools in our framework around it.
Post by Chuck Lever
Post by Chuck Lever
I think you are suggesting we ignore this problem for now, and just
have the tools pretend the other protocol does not exist, while still
allowing the possibility of storing both types of metadata in the same
filesystem object. That may be an easy way to get started.
I thought this was what you were proposing actually. With my
'integrator' hat on I would rather quickly define common tools that can
handle both, and have the old tools return loud warnings (were possible)
if you try to use them.
The "old" FedFS tools in this case are available because they are one
of two independent reference implementations required for the IETF
standardization process. However, I think solving the problem of
integrating FedFS ADMIN with an existing DFS implementation on a
multi-protocol fileserver is going to be very valuable implementation
experience that can be fed back into the IETF standards process.
I'm not confident we can fit rpc.fedfsd (Linux's implementation of the
FedFS RPC ADMIN service) into this model, so we may end up abandoning
its use for managing FedFS junctions, leaving it only to provisioning
NSDB certificates.
In the Samba case the basic tool is 'ln', not sure we can do much about
it :-) But we can certainly patch the RPC code that allows handling via
SMB/RPC although I am not quite sure how to populate all the data the
SMB world has no concept of, perhaps using 'good defaults' ...
Exactly the problem.
My thought was to define a place-holder value for both protocols that
can be used by our existing tools, for now. No xattr for symlinks
created on behalf of DFS, and maybe an empty symlink for junctions
created by rpc.fedfsd or the nfsref command.
Sound like a good first start to me.

Simo.
--
Simo Sorce * Red Hat, Inc * New York
Chuck Lever
2013-04-22 20:39:45 UTC
Permalink
Hi-

I led a discussion on Friday at the Linux Storage and Filesystem Summit on how to store {DFS, FedFS} junctions in Linux filesystems. I'd like to summarize the discussion, then ask a few follow-up questions. I apologize in advance for the length.

FedFS is to NFS as Microsoft DFS is to SMB/CIFS. FedFS uses NFS referrals to glue together a file namespace out of separate shares, starting with a root share that may contain nothing but referrals. For more on FedFS, start with RFC 5716.

The physical object that stores referral location target information is called a junction. It can contain an actual list of locations, or it can contain the DN of an LDAP record where the location list is maintained.

+ Samba uses a symlink for this purpose. The contents of the link represent the location information passed out to CIFS clients.

+ FedFS uses an extended attribute on a directory for this purpose. The xattr contains XML which encodes location information passed out to NFS clients.

The reasons for this difference are simply historical.

Linux is often used as a multi-protocol file service platform, where the same physical data is presented to clients via both NFS and CIFS. For this reason, we think there would be value in using the same physical representation for both NFSD and Samba so that a single junction object on our exported filesystems can trigger a referral for both NFS and CIFS.

Samba has been around much longer, and DFS support is actually deployed. FedFS is newer and still experimental. Thus we decided to change FedFS to use a symlink instead of a directory. Samba will still use the regular contents of the symlink, and FedFS will store its metadata in an extended attribute attached to the symlink.

There was a rough consensus to prefer JSON over XML in the FedFS xattr, though there are still some who dislike both. I'm open to suggestion, since we're now essentially replacing the existing FedFS junction format and can change it to anything we like.

Today FedFS junctions can contain either a location list or an LDAP DN. One option for FedFS is to support only the LDAP DN junction type, and have a (possibly local) LDAP service available to store the location information. The FedFS junction xattr would then always contain an LDAP URL. Storing complex data types (a list containing pathnames, hostnames, integers, and other values) would then be up to LDAP.

We will have to discuss a conjunction of administrative interfaces at some later point. However, we should clarify how our junction management tools behave now that each junction can have metadata it did not have before.

FedFS:

nfsref add - if no symlink exists, create it (what contents should it have?)
- add an extended attribute

nfsref remove - remove the extended attribute, leave the symlink
- can we remove the symlink if its contents are meaningless?


Samba:

add - if a symlink already exists, replace its contents, preserving xattrs

remove - if a FedFS extended attribute exists, leave the symlink (what contents should it have?)


The FedFS extended attribute is called "trusted.junction.nfs". Should we rename it?

Note that CAP_SYS_ADMIN capabilities are required to access the xattr. Will that be a problem for Samba administration tools?
--
Chuck Lever
chuck[dot]lever[at]oracle[dot]com
Simo Sorce
2013-04-23 14:51:14 UTC
Permalink
Hi-
I led a discussion on Friday at the Linux Storage and Filesystem
Summit on how to store {DFS, FedFS} junctions in Linux filesystems.
I'd like to summarize the discussion, then ask a few follow-up
questions. I apologize in advance for the length.
FedFS is to NFS as Microsoft DFS is to SMB/CIFS. FedFS uses NFS
referrals to glue together a file namespace out of separate shares,
starting with a root share that may contain nothing but referrals.
For more on FedFS, start with RFC 5716.
The physical object that stores referral location target information
is called a junction. It can contain an actual list of locations, or
it can contain the DN of an LDAP record where the location list is
maintained.
+ Samba uses a symlink for this purpose. The contents of the link
represent the location information passed out to CIFS clients.
+ FedFS uses an extended attribute on a directory for this purpose.
The xattr contains XML which encodes location information passed out
to NFS clients.
The reasons for this difference are simply historical.
Linux is often used as a multi-protocol file service platform, where
the same physical data is presented to clients via both NFS and CIFS.
For this reason, we think there would be value in using the same
physical representation for both NFSD and Samba so that a single
junction object on our exported filesystems can trigger a referral for
both NFS and CIFS.
Samba has been around much longer, and DFS support is actually
deployed. FedFS is newer and still experimental. Thus we decided to
change FedFS to use a symlink instead of a directory. Samba will
still use the regular contents of the symlink, and FedFS will store
its metadata in an extended attribute attached to the symlink.
There was a rough consensus to prefer JSON over XML in the FedFS
xattr, though there are still some who dislike both. I'm open to
suggestion, since we're now essentially replacing the existing FedFS
junction format and can change it to anything we like.
Can you give an example or refernce of what is stored in this XML/JSON
blob ? Why do you need structured data there ?
Today FedFS junctions can contain either a location list or an LDAP
DN. One option for FedFS is to support only the LDAP DN junction
type, and have a (possibly local) LDAP service available to store the
location information. The FedFS junction xattr would then always
contain an LDAP URL. Storing complex data types (a list containing
pathnames, hostnames, integers, and other values) would then be up to
LDAP.
Having to install a whole LDAP server as a pre-requisite seem very heavy
handed.
We will have to discuss a conjunction of administrative interfaces at
some later point. However, we should clarify how our junction
management tools behave now that each junction can have metadata it
did not have before.
nfsref add - if no symlink exists, create it (what contents should it have?)
- add an extended attribute
nfsref remove - remove the extended attribute, leave the symlink
- can we remove the symlink if its contents are meaningless?
Why should we leave a symlink ? Don't we expect to remove junctions for
all protocols ?
add - if a symlink already exists, replace its contents, preserving xattrs
remove - if a FedFS extended attribute exists, leave the symlink
(what contents should it have?)
Why should we leave a symlink ? Don't we expect to remove junctions for
all protocols ?


What I do not get is why are you trying to use the same mechanism (a
symlink) but then treat them as independent and separate entities ?
What is the aim ?
From your premise I thought you wanted to allow parallel functionality,
ie a DFS created in samba would be seen as a junction for nfs and
vice-versa, but the latter points seem to not allow that ?
The FedFS extended attribute is called "trusted.junction.nfs". Should we rename it?
Shouldn't it be namespaced and use something like trusted.nfs.junction ?
Also why a xattr in the trusted namespace ? What are the security
considerations that warrants a trusted attribute rather than a normal
one ? (Links to RFCs or other docs are just fine)
Note that CAP_SYS_ADMIN capabilities are required to access the xattr.
Will that be a problem for Samba administration tools?
It should not be a problem for Samba, as we can retain capabilities if
needed, and we already handle data in xattrs (not for DFS symlinks).

Simo.
--
Simo Sorce * Red Hat, Inc * New York
Simo Sorce
2013-04-23 16:54:29 UTC
Permalink
Post by Chuck Lever
Can you give an example or refernce of what is stored in this XML/JSON
blob ? Why do you need structured data there ?
An "NFS basic junction" stores the location list in place. Each item
in a location list contains a number of pieces of data, including: a
server hostname, an export pathname (which is a list of path
components), and a number of integer and boolean settings that help
clients sort which replica of this data they should mount.
A full explanation of this data is in RFC 5661, section 11.10. This
data is returned to an NFS client when it encounters one of these
objects. The client can redirect its requests to one of the servers
and exports listed in the returned data.
A "FedFS junction" stores a reference to a location list stored in
LDAP. The LDAP server's hostname and port number and the UUID of a
FedFS Fileset Name record are stored in the junction. The Fileset
Name record has children, each of which constitute a location (see
above).
I do not see mention of LDAP URIs, andyou seem to speak in the singular
when mention 'the LDAP server' does it mean you hve no way to specify a
pool of LDAP servers for HA ?
Post by Chuck Lever
http://datatracker.ietf.org/doc/draft-ietf-nfsv4-federated-fs-protocol/
See chapter 4 for an overview of the schema used for these lists. An
NFS fileserver converts the LDAP records into an fs_locations4 or
fs_locations_info4 attribute for NFS clients. Other protocols use a
different representation for communicating this list to clients.
To be honest the constrains in this document on the LDAP DIT, seem to
indicate it will be possible to use this stuff primarily only with a
dedicated LDAP server, Dictating how the rootDSE/namingContext should
look like is a quite strong demand.

Why do you need a full namingContext for FedFS ?
Why a subtree is not sufficient ?


On the security side, you recommend RPCSEC_GSS for NFS, but then TLS for
LDAP, why not use SASL/GSSAPI for LDAP as well so you need a single set
of credentials ?
Post by Chuck Lever
Today FedFS junctions can contain either a location list or an LDAP
DN. One option for FedFS is to support only the LDAP DN junction
type, and have a (possibly local) LDAP service available to store the
location information. The FedFS junction xattr would then always
contain an LDAP URL. Storing complex data types (a list containing
pathnames, hostnames, integers, and other values) would then be up to
LDAP.
Having to install a whole LDAP server as a pre-requisite seem very heavy
handed.
True. Today, the LDAP/NSDB pieces are optional if an admin wants to
support only "NFS basic junctions," for just this reason.
However there are certain benefits to allowing location lists to be
managed via LDAP, rather than being specified at junction creation.
Junctions can share the same location list, for example. A filesystem
migration can update a central location list once, rather than having
to find every junction that references the migrated filesystem.
Understood.
Post by Chuck Lever
In addition, storing these lists in a publicly available LDAP service
means that any fileserver, anywhere, can access the lists.
If we are really wily, maybe a small single-purpose daemon can be
constructed from a minimal LDAP server implementation (or from
scratch), and it can listen on its own port or only for loopback
requests.
I think in most cases this is what will actually happen, but you do not
need to use a special purpose built server, you can use an existing LDAP
server simply specially configured for your needs. It will cause
administrative overhead to handle this infrastructure though.

[..]
Post by Chuck Lever
Why should we leave a symlink ? Don't we expect to remove junctions for
all protocols ?
The difficulty I have is how we are going to conjoin the
administrative tools that manage junctions. I imagine that for some
time, the tool used for managing DFS junctions will be unaware of
FedFS junction content, and vice versa.
Yes, for some time one may end up wiping out the other.
Post by Chuck Lever
What I do not get is why are you trying to use the same mechanism (a
symlink) but then treat them as independent and separate entities ?
What is the aim ?
From your premise I thought you wanted to allow parallel functionality,
ie a DFS created in samba would be seen as a junction for nfs and
vice-versa, but the latter points seem to not allow that ?
FedFS junctions can list both NFS and SMB (and other types). The SMB
parts are not defined by the IETF, since SMB is a proprietary protocol
controlled by Microsoft.
One way to have DFS and FedFS information in the same filesystem
object is to have one object that can contain both. The tools then
have to be designed not to step on each other. Eventually we figure
out how to make this seamless.
I think you are suggesting we ignore this problem for now, and just
have the tools pretend the other protocol does not exist, while still
allowing the possibility of storing both types of metadata in the same
filesystem object. That may be an easy way to get started.
I thought this was what you were proposing actually. With my
'integrator' hat on I would rather quickly define common tools that can
handle both, and have the old tools return loud warnings (were possible)
if you try to use them.

In the Samba case the basic tool is 'ln', not sure we can do much about
it :-) But we can certainly patch the RPC code that allows handling via
SMB/RPC although I am not quite sure how to populate all the data the
SMB world has no concept of, perhaps using 'good defaults' ...

Simo.
--
Simo Sorce * Red Hat, Inc * New York
Matt W. Benjamin
2013-04-23 18:43:01 UTC
Permalink
Well I have some knowledge in this field, and I do not find it very
fiddly, but I may be biased as I have been working for the past 7
years
to make LDAP+Kerberos simple to manage within the FreeIPA project.
Your requirement to set an objectclass on the base suffix is something
I
find particularly unappealing, and no other tool that I know of
requires
this (because it is unnecessary).
Yes. Thank you.
I think you should allow the broadest possibilities of course, which
is
why I am picking on things like allowing SSAL/GSSAPI explicitly in
the
RFC language. Whether people will integrate into existing LDAP server
or
not remains to be seen, if we can avoid the need to add an objectlass
on
the root suffix I see that we can easily add this a standard feature
for
FreeIPA as well (we already provide automount data for example) and
provide management tools in our framework around it.
I noted this point on a FedFS concall 2 years ago. I dont understand why
not -permit- easy integration with existing LDAP infrastructure.
--
Matt Benjamin
The Linux Box
206 South Fifth Ave. Suite 150
Ann Arbor, MI 48104

http://linuxbox.com

tel. 734-761-4689
fax. 734-769-8938
cel. 734-216-5309
Continue reading on narkive:
Loading...