[Federated-fs] Conf call 6/5/2008

[Everhart, Craig]

I'm not sure we're communicating.  In the DNS case, there isn't a "root
server" that's backed by a real file server somewhere; there's just the
DNS.  I'm not presupposing that there is some root fileset anywhere.
I'm just describing a convention by which any agent can construct a
cache on an implicit "global file system root", one that results in the
same answer for the same question anywhere.  This seems valuable in
itself.  If one were to amend the convention so that the DNS SRV record
were to point to some NSDB that would be queried with a
conventionally-defined FSN, that would work too.  It's just a
DNSname-to-filesystem mapping.  But the fact that it's globally uniform
strikes me as valuable.

It's true that you need to ask for something in order to see anything--a
domain name, in that formulation.  And feel free to invent whatever
convention you want for finding "the local whatever" that you're looking
for.  I'm still hoping that some convention can emerge, but that hope is
ebbing.

I still don't know what "my own pseudo-root for the namespace" would be,
or for that matter, why Joe would be interested in Sam's "my own
pseudo-root".

All this does indeed make me worry about what the various NSDBs need to
communicate with each other.  In the formulation in the drafts,
communication isn't even necessary between them.

I get the feeling that I need a brain implant to understand what you're
looking for.

		Craig

> -----Original Message-----
> From: Ellard, Daniel 
> Sent: Wednesday, June 11, 2008 9:24 AM
> To: Everhart, Craig; 'Paul Lemahieu'; 'Robert Thurlow'
> Cc: 'federated-fs at sdsc.edu'
> Subject: RE: [Federated-fs] Conf call 6/5/2008
> 
> I think the difference is that in the DNS case, you need to 
> ask for something (perhaps the name of some root server, or 
> perhaps some sort of wildcarded name for "whatever root 
> server I should use").  In the NSDB case, once you find an 
> NSDB (which might be done via the same sort of DNS lookup), 
> you can ask it "if I wanted to create my own pseudo-root for 
> the namespace, what would I need to do?"  The difference is 
> more about what's in the response than what's in the question.
> 
> I can even imagine clients asking this...  And that could 
> lead to a world without root servers.
> 
> The questions still remain:
> 
> - how constrained can we make the pseudo-root?  Can we make 
> it something as simple as the "/afs" directory?
> 
> - how consistent does each NSDB's view of the world need to be?
> 
> - how do we achieve this level of consistency?
> 
> I suspect that it's easier to keep the NSDB's notion of a 
> (highly constrained) pseudo-root in synch than to replicate 
> root filesets across the federation (not "easy", but "easier")
> 
> -Dan
> 
> -----Original Message-----
> From: Everhart, Craig
> Sent: Tuesday, June 10, 2008 11:11 PM
> To: Paul Lemahieu; Robert Thurlow
> Cc: federated-fs at sdsc.edu; Ellard, Daniel
> Subject: RE: [Federated-fs] Conf call 6/5/2008
> 
> Is this "top of tree" thought independent of the DNS-based 
> lookup?  Why wouldn't that simply *be* the top level, leading 
> to, as you say, real file systems?  Is this an intermediate level?
> 
> I can read all your text substituting "DNS" for "NSDB" and 
> get a working (and specified and nearly existing) result.  
> What am I missing?  Do I need to invent yet another 
> replicated global service?
> 
> 		Craig 
> 
> > -----Original Message-----
> > From: Paul Lemahieu [mailto:LeMahieu_Paul at emc.com]
> > Sent: Tuesday, June 10, 2008 7:53 PM
> > To: Robert Thurlow
> > Cc: Everhart, Craig; federated-fs at sdsc.edu; Ellard, Daniel
> > Subject: Re: [Federated-fs] Conf call 6/5/2008
> > 
> > Robert,
> > 
> > This is very much the motivation. It's a couple of key things:
> > 
> >     * A standard to facilitate the administration of a federated 
> > global namespace
> >     * The ability to federate different file servers so they all 
> > expose the same global namespace
> > 
> > A few things about how I would see this:
> > 
> >     * I always thought of this as a "top-of-tree" namespace. 
> > In other words, at some point the global namespace ends and you hit 
> > real file systems. and the global namespace ends.
> > Perhaps it is possible to manage junction points at arbitrary 
> > locations in the global tree, I just hadn't really considered it.
> >     * Changes to the global namespace are made by 
> administrators, and 
> > made via the NSDB. The global namespace is not frequently changing.
> > Participating file servers reflect those changes later in a
> > loosely- coupled manner.
> >     * This does not invalidate the existing federated-fs work. It 
> > would be essentially an additional database in the NSDB, 
> mapping paths 
> > to FSNs.
> >     * It is assumed that the NSDB takes care of replicating 
> this data.
> > 
> > The big difference from what you describe below and my view 
> is whether 
> > we have an NSDB that reflects the namespace created on the 
> file server 
> > (the query model you describe below), or whether the NSDB is 
> > authoritative for the namespace and the file servers reflect that 
> > namespace configuration (my description, where the NSDB is 
> > authoritative).
> > 
> > --Paul
> > 
> > On 2008-Jun-09, at 15:36, Robert Thurlow wrote:
> > > Everhart, Craig wrote:
> > >> I *totally* agree with Dan's bias on this.  It's a 
> surprise to me 
> > >> that others thought that there was a "namespace" that existed 
> > >> independently of the file systems that make it up.
> > >>
> > >> What is the relationship between path data that exists both in a 
> > >> fileset (in a file server--Dan's #1 case) and in the NSDB
> > and (as in
> > >> Paul's addendum to Dan's #3) all the instances of the 
> parent path 
> > >> data that are subsidiary filesets?  Is there some
> > authoritative copy
> > >> with the others just hints?  What is the replication protocol by 
> > >> which path data is propagated?  How consistent does it 
> have to be?
> > >>
> > >> If we were stumbling thinking about the constraints on fileset
> > >> replication and consistency, why would this be a simple 
> answer?   
> > >> Right
> > >> now it's an unmotivated, underspecified, and unconstrained
> > additional
> > >> criterion for any implementation.  (And them's its good
> > points...:-})
> > >>
> > >> If others (Paul? Renu?) feel like this needs to be 
> changed, could 
> > >> they do it with a more fleshed-out proposal?
> > >
> > > I don't want to keep a copy of all of the mapping data (with or 
> > > without authority) in the NSDB, either.  But I do wonder if the 
> > > motivation is something like this:
> > >
> > > I want an admin application that can show me the whole
> > namespace.  I
> > > want to be able to browse it like a Google map, going up/down, 
> > > left/right in the global directory tree.  I'd like to see
> > which nodes
> > > are junctions, and to be able to click on them and see 
> the details 
> > > about their replicas.  In the fullness of time, I'd like to
> > be able to
> > > click on a point in the namespace and select an option to 
> make that 
> > > point a separate, replicated filesystem, and to adjust the 
> > > characteristics of existing filesystems.
> > >
> > > Now, if all we have is a way to drill down from the top of the 
> > > namespace for any particular path, this could be bloody
> > hard.  Having
> > > more information would help.  An alternate question is,
> > where could we
> > > get more info?
> > >
> > > One suggestion is that the protocol permit a query to an 
> NFS server 
> > > participating in the namespace, to list the junctions in a
> > particular
> > > filesystem.  If we could start at the top, find the Nth level 
> > > filesystems and ask them what junctions point outside of the 
> > > filesystem, we could enumerate the namespace far quicker. 
>  We would 
> > > still have to send packets all over the place - "find the 
> root" DNS 
> > > queries, NSDB lookups, NFS accesses and whatever we use to
> > enumerate
> > > junctions in a filesystem - but I have always expected that.
> > >
> > > Does this shed light or kick up dust? :-)
> > >
> > > Rob T
> > >
> > 
> > 
>