[Federated-fs] federated-fs conf call today

[Ellard, Daniel]

There will be another federated-fs conference call this afternoon, at
4pm EST, dial-in number 1-888-765-3653 # 6206056.

The original goal was to talk about the draft of the admin protocol this
afternoon, but I'm still writing the draft, and there are a bunch of
things that I don't feel like we've discussed enough to say the draft is
strong yet.  So let's talk about some of those things...

To refresh everyone's memory, the admin protocol is the protocol that a
federation member administrator uses to make federation-oriented
requests to an individual server.  I would like to propose some
important changes to the current protocol, which I now believe is broken
(although I admit I originally argued for it).

In the current protocol (as seen in the earlier drafts) some of these
requests are to change the state of a server in some way (e.g., create a
junction) while others are only to query the state of a server (e.g.,
find the FSN that should be used to refer to a given path).  The former
class of requests are meant to be performed by the administrator on
servers under its administration (i.e., within the same federation
member) while the latter may be used more widely.  There might be
limitations on who can query servers in the system -- that's another
topic for discussion -- but for now let's assume that anyone can query
any server they can reach.

The first (and perhaps messiest) operation the admin performs in the
current, proposed protocol, is creating junctions to stitch together the
namespace.  In the worst case, we're starting from a blank slate, and
the target of the link does not already exist in the namespace.  We need
some way to identify it, and the current notion is that such filesets
are identified initially by a concrete address, via the following
protocol:

The "source" admin wishes to create a junction from a fileset under its
control to a fileset that it can reach, using ordinary NFSv4, as
hostname:path.  (there may be additional parameters, but this is the
minimal case)

1. The admin sends a query to host hostname, asking it for the FSN of
the fileset rooted at hostname:path.

2. Host hostname may refuse, or the path may be invalid, etc.  If it is
valid, and the hostname knows the FSN, it returns the FSN.  Otherwise,
if the host wishes for the protocol to succeed:

	2a.  Host hostname creates an FSN for hostname:path by choosing
an NSDB location, creating an FsnUuid (or having the NSDB location
choose one), and registering the resulting FSN with the NSDB.

	2b.  If the fileset is replicated, then the host registers all
other instances of the same fileset as FSLs for that FSN.

	2c.  The host notes the new FSN as belonging to that
hostname:path, and returns it to the admin.

I think that this protocol is problematic in several ways, primarily in
steps 2b and 2c.  The problem is that the host must somehow know the FSN
of the filesets it exports (if they already exist) and the host must
also have some way of finding all the replicas of any fileset it hosts.
The final problem is that there is a potential for a race condition in
step 2a: if this protocol is executed concurrently on two replicas of
the fileset, we may potentially end up in a state where the same fileset
has two FSNs -- and no method to tell them apart or banish one.  There
are ways to extend the protocol in order to add more centralized
control, but I think the result would be awkward and complicated.

The fundamental problem is that we are asking the NFS servers to
shoulder some of the burden of managing the global namespace.  I think
this should all be pushed back onto the NSDB.

So here's my new proposal: junctions can be created ONLY to FSNs, not by
reverse-resolving FSLs.  Filesets MUST be registered as FSNs before they
can be the target of a junction.  (The exact procedure for accomplishing
this may be host-dependent.)  This means that the target host is never
involved in creating a junction, so there is no need for what
effectively was server-to-server communication via the admin.

-Dan



-- 
Daniel Ellard, Ph.D.
Advanced Technology Group, Network Appliance, Inc.