Avati, Amar,
Amar, Anand S and myself had a discussion on this comment and here is an answer to your queries the way I see it. Let me know if I am missing something here.
(this is not a NFS Ganesha requirement, FYI. As Ganesha will only do a single lookup or preserve a single object handle per filesystem object in its cache)
Currently a glfs_object is an opaque pointer to an object (it is a _handle_ to the object). The object itself contains a ref'd inode, which is the actual pointer to the object.
1) The similarity and differences of object handles to fds
The intention of multiple object handles is in lines with multiple fd's per file, an application using the library is free to lookup (and/or create (and its equivalents)) and acquire as many object handles as it wants for a particular object, and can hence determine the lifetime of each such object in its view. So in essence one thread can have an object handle to perform, say attribute related operations, whereas another thread has the same object looked up to perform IO.
Where the object handles depart from the notion of fds is when an unlink is performed. As POSIX defines that open fds are still _open_ for activities on the file, the life of an fd and the actual object that it points to is till the fd is closed. In the case of object handles though, the moment any handle is used to unlink the object (which BTW is done using the parent object handle and the name of the child), all handles pointing to the object are still valid pointers, but operations on then will result in ENOENT, as the actual object has since been unlinked and removed by the underlying filesystem.
The departure from fds is considered valid in my perspective, as the handle points to an object, which has since been removed, and so there is no semantics here that needs it to be preserved for further operations as there is a reference to it held.
So in essence for each time an object handle is returned by the API, it has to be closed for its life to end. Additionally if the object that it points to is removed from the underlying system, the handle is pointing to an entry that does not exist any longer and returns ENOENT on operations using the same.
2) The issue/benefit of having the same object handle irrespective of looking it up multiple times
If we have an 1-1 relationship of object handles (i.e struct glfs_object) to inodes, then the caller gets the same pointer to the handle. Hence having multiple handles as per the caller, boils down to giving out ref counted glfs_object(s) for the same inode.
Other than the memory footprint, this will still not make the object live past it's unlink time. The pointer handed out will be still valid till the last ref count is removed (i.e the object handle closed), at which point the object handle can be destroyed.
So again, as many handles were handed out for the same inode, they have to be closed, etc.
3) Graph switches
In the case of graph switches, handles that are used in operations post the switch, get refreshed with an inode from the new graph, if we have an N:1 object to inode relationship.
In the case of 1:1 this is done once, but is there some multi thread safety that needs to be in place? I think this is already in place from the glfs_resolve_inode implementation as suggested earlier, but good to check.
4) Renames
In the case of renames, the inode remains the same, hence all handed out object handles still are valid and will operate on the right object per se.
5) unlinks and recreation of the same _named_ object in the background
Example being, application gets an handle for an object, say named "a.txt", and in the background (or via another application/client) this is deleted and recreated.
This will return ENOENT as the GFID would have changed for the previously held object to the new one, even though the names are the same. This seems like the right behaviour, and does not change in the case of a 1:1 of an N:1 object handle to inode mapping.
So bottom line, I see the object handles like an fd with the noted difference above. Having them in a 1:1 relationship or as a N:1 relationship does not seem to be an issue from what I understand, what am I missing here?
Shyam
From: "Anand Avati" <address@hidden>
To: "Shyamsundar Ranganathan" <address@hidden>
Cc: "Gluster Devel" <address@hidden>
Sent: Monday, September 30, 2013 10:35:05 AM
Subject: Re: RFC/Review: libgfapi object handle based extensions
I see a pretty core issue - lifecycle management of 'struct glfs_object'. What is the structure representing? When is it created? When is it destroyed? How does it relate to inode_t?
Looks like for every lookup() we are creating a new glfs_object, even if the looked up inode was already looked up before (in the cache) and had a glfs_object created for it in the recent past.
We need a stronger relationship between the two with a clearer relationship. It is probably necessary for a glfs_object to represent mulitple inode_t's at different points in time depending on graph switches, but for a given inode_t we need only one glfs_object. We definitely must NOT have a new glfs_object per lookup call.
Avati