[Federated-fs] Conf call 6/5/2008

[Paul Lemahieu]

As for why, it's to have a global namespace that spans heterogeneous  
file servers, and have a common configuration/management for that. A  
Linux box or AIX or a NAS vendor could all be exposing the same /fedfs/ 
home, and handing out referrals to users.

--Paul

On 2008-Jun-11, at 10:23, Everhart, Craig wrote:

> OK: why?  Also, what do we do with the hard questions (synchronizing  
> multiple data sources, doing permissions correctly)?
>
>
> ________________________________
>
> 	From: LeMahieu, Paul [mailto:LeMahieu_Paul at emc.com]
> 	Sent: Wednesday, June 11, 2008 1:19 PM
> 	To: Everhart, Craig; Robert Thurlow
> 	Cc: Ellard, Daniel; federated-fs at sdsc.edu
> 	Subject: Re: [Federated-fs] Conf call 6/5/2008
> 	
> 	
> 	Yes, it's independent of anything with DNS. When I say "top-of- 
> tree", I'm talking about storing all the configuration of the  
> namespace (tens of thousands of junction points and their paths in  
> the logical namespace). For example, if there is /fedfs/home/bob,  
> there is an entry mapping /fedfs/home/bob to bob's share on some  
> physical file server. We'd be storing a pseudo file system  
> representing the top-of tree in the NSDB.
> 	
> 	--Paul
> 	
> 	
> 	On 08/6/10 20:11, "Everhart, Craig" <Craig.Everhart at netapp.com>  
> wrote:
> 	
> 	
>
> 		Is this "top of tree" thought independent of the DNS-based  
> lookup?  Why
> 		wouldn't that simply *be* the top level, leading to, as you say,  
> real
> 		file systems?  Is this an intermediate level?
> 		
> 		I can read all your text substituting "DNS" for "NSDB" and get a  
> working
> 		(and specified and nearly existing) result.  What am I missing?   
> Do I
> 		need to invent yet another replicated global service?
> 		
> 		                Craig
> 		
> 		> -----Original Message-----
> 		> From: Paul Lemahieu [mailto:LeMahieu_Paul at emc.com]
> 		> Sent: Tuesday, June 10, 2008 7:53 PM
> 		> To: Robert Thurlow
> 		> Cc: Everhart, Craig; federated-fs at sdsc.edu; Ellard, Daniel
> 		> Subject: Re: [Federated-fs] Conf call 6/5/2008
> 		>
> 		> Robert,
> 		>
> 		> This is very much the motivation. It's a couple of key things:
> 		>
> 		>     * A standard to facilitate the administration of a
> 		> federated global namespace
> 		>     * The ability to federate different file servers so they
> 		> all expose the same global namespace
> 		>
> 		> A few things about how I would see this:
> 		>
> 		>     * I always thought of this as a "top-of-tree" namespace.
> 		> In other words, at some point the global namespace ends and
> 		> you hit real file systems. and the global namespace ends.
> 		> Perhaps it is possible to manage junction points at arbitrary
> 		> locations in the global tree, I just hadn't really considered it.
> 		>     * Changes to the global namespace are made by
> 		> administrators, and made via the NSDB. The global namespace
> 		> is not frequently changing.
> 		> Participating file servers reflect those changes later in a
> 		> loosely- coupled manner.
> 		>     * This does not invalidate the existing federated-fs
> 		> work. It would be essentially an additional database in the
> 		> NSDB, mapping paths to FSNs.
> 		>     * It is assumed that the NSDB takes care of replicating this  
> data.
> 		>
> 		> The big difference from what you describe below and my view
> 		> is whether we have an NSDB that reflects the namespace
> 		> created on the file server (the query model you describe
> 		> below), or whether the NSDB is authoritative for the
> 		> namespace and the file servers reflect that namespace
> 		> configuration (my description, where the NSDB is authoritative).
> 		>
> 		> --Paul
> 		>
> 		> On 2008-Jun-09, at 15:36, Robert Thurlow wrote:
> 		> > Everhart, Craig wrote:
> 		> >> I *totally* agree with Dan's bias on this.  It's a surprise  
> to me
> 		> >> that others thought that there was a "namespace" that existed
> 		> >> independently of the file systems that make it up.
> 		> >>
> 		> >> What is the relationship between path data that exists both  
> in a
> 		> >> fileset (in a file server--Dan's #1 case) and in the NSDB
> 		> and (as in
> 		> >> Paul's addendum to Dan's #3) all the instances of the parent  
> path
> 		> >> data that are subsidiary filesets?  Is there some
> 		> authoritative copy
> 		> >> with the others just hints?  What is the replication protocol  
> by
> 		> >> which path data is propagated?  How consistent does it have  
> to be?
> 		> >>
> 		> >> If we were stumbling thinking about the constraints on fileset
> 		> >> replication and consistency, why would this be a simple answer?
> 		> >> Right
> 		> >> now it's an unmotivated, underspecified, and unconstrained
> 		> additional
> 		> >> criterion for any implementation.  (And them's its good
> 		> points...:-})
> 		> >>
> 		> >> If others (Paul? Renu?) feel like this needs to be changed,  
> could
> 		> >> they do it with a more fleshed-out proposal?
> 		> >
> 		> > I don't want to keep a copy of all of the mapping data (with or
> 		> > without authority) in the NSDB, either.  But I do wonder if the
> 		> > motivation is something like this:
> 		> >
> 		> > I want an admin application that can show me the whole
> 		> namespace.  I
> 		> > want to be able to browse it like a Google map, going up/down,
> 		> > left/right in the global directory tree.  I'd like to see
> 		> which nodes
> 		> > are junctions, and to be able to click on them and see the  
> details
> 		> > about their replicas.  In the fullness of time, I'd like to
> 		> be able to
> 		> > click on a point in the namespace and select an option to make  
> that
> 		> > point a separate, replicated filesystem, and to adjust the
> 		> > characteristics of existing filesystems.
> 		> >
> 		> > Now, if all we have is a way to drill down from the top of the
> 		> > namespace for any particular path, this could be bloody
> 		> hard.  Having
> 		> > more information would help.  An alternate question is,
> 		> where could we
> 		> > get more info?
> 		> >
> 		> > One suggestion is that the protocol permit a query to an NFS  
> server
> 		> > participating in the namespace, to list the junctions in a
> 		> particular
> 		> > filesystem.  If we could start at the top, find the Nth level
> 		> > filesystems and ask them what junctions point outside of the
> 		> > filesystem, we could enumerate the namespace far quicker.  We  
> would
> 		> > still have to send packets all over the place - "find the  
> root" DNS
> 		> > queries, NSDB lookups, NFS accesses and whatever we use to
> 		> enumerate
> 		> > junctions in a filesystem - but I have always expected that.
> 		> >
> 		> > Does this shed light or kick up dust? :-)
> 		> >
> 		> > Rob T
> 		> >
> 		>
> 		>
> 		
> 		
> 		
>