Re: [Gnu-arch-users] Re: [BUG] FEATURE PLANS: revlib locking

[Jan Hudec]

On Fri, Jun 04, 2004 at 15:41:13 +0100, Andrew Suffield wrote:
> On Fri, Jun 04, 2004 at 02:38:50PM +0200, Jan Hudec wrote:
> > No. But it uses the underlaying syscalls, that have the properties and
> > the server has no reason to break it.
> 
> It's not <syscall is made on client> <syscall completes on
> server>. It's <syscall is made on client> <...network...> <syscall is
> performed on server> <...network...> <syscall completes on client>.

Sure I *DO* *KNOW* *THIS*. The "underlaying syscall" is the <syscall is
performed on server>. That and only that.

> They don't behave anything like the same way, because the
> <...network...> stages aren't reliable.

"Just aren't reliable" is insufficient qualification. They are
unreliable in that a message might be lost without indication. They are
reliable in that it does not create or change messages.

Now, if arrival of message 'link x to y' always causes syscall 'link
x to y' to execute on server and if that syscall has the property of
failing if and only if (sans permanent errors) the target does exist,
then the schema proposed guarantees, that the client that got the lock
knows it, unless it crashes.

The schema proposed is:
    1) Client creates a file with unique name on server. Retries as
       required.
    2) Client requests to link this file to the lockfile name. Retries
       until it gets a reply.
    3) Client IGNORES the reply, unless it is a permanent error
       (permissions, wrong path).
    4) Client stat's the temporary file it created in step 1. Retries as
       required.
    5) 
       - If the reply is 2, the lock is taken. Enters critical section.
       - If the reply is 1, someone else has the lock. Wait a short
         while and repeat from step 2.

> > > > Now the only way for the NFS server to break this would be to do the
> > > > same useless work -- create the link, then check if it should do it and
> > > > delete it again if it shouldnt. In addition to being a bug, it's SO much
> > > > extra work (when you can just do the syscall and see), that I don't
> > > > think any NFS server breaks this.
> > > 
> > > You think wrong. Here's a specific counterexample along the lines you
> > > describe:
> > > 
> > > The server creates the link, sends the RPC reply saying that the link
> > > was created, and the reply gives an EHOSTUNREACH message. Just to make
> > 
> > But I do not care for the return value of the link request! I just drop
> > it on the floor!!!
> 
> Then your locking protocol is unsafe; two clients can think they've
> got the lock at the same time. Really, you're doing the "naive
> student" thing; this discussion is older than *me*. You're making all
> the classic mistakes.

No, they CAN'T. It's impossible for both to get link count 2 on their
temporary files, since there is only one entry to link...

Perhaps I am doing classic mistakes. I am, however, not doing the
mistakes you tell me I am doing.
 
> > > things even more fun, you now have a link and two clients which think
> > > it's not their lock. You can spend weeks chasing the corner cases, but

No. I don't. The link points to either client's temporary file. When
that client reads it's link-count, it will see it owns the lock!

There is a problem if a client crashes with the lock already granted.
Well, this problem is there for all lockfile schemas. I am not arguing
about THAT.

> > No. The clients check link-count on their respective temporary files.
> > Since it will be a link to one of them, one of the clients will,
> > eventualy, get reply saying 2. That client knows it has the lock and
> > goes ahead. The other equaly well knows the resource is not available.
> 
> Now you've assumed that the temporary files were created
> atomically. You might as well have just used that as your lock state;
> it would be just as safe (ie, not at all, since file creation isn't
> atomic on NFS).

No. I am assuming anything about atomicity! I am just assuming, that the
client requesting lock uses unique name for that file. 

> The safe protocol for hardlink-based locking is *this*:
> 
> Create a randomly named temporary file.

Yes, that was in the original proposal! And all the time, I am saying
I do just that.

> Hardlink the lock file to the temporary file.

Still the same.

> If the link succeeds and the link count on the lock file is 2, you
> have the lock. Otherwise you missed: delete the lock file and your
> temporary file, wait a random interval, and try again.

Still the same.

> Which performs like a dog. And that's the only one which works. No
> amount of trying to vary it will discover anything new about NFS.

AND IT IS EXACTLY THE SCHEMA THAT WAS IN THE ORIGINAL POST!

I am arguing that this very schema is correct.

> > Btw: Connections don't win you anything.
> 
> Go away and do not bother me again until you have tried to implement a
> network file system.

Been there, done that!

Connections are nice. They simplify the stub/skeleton a lot. They can
assure ordering, so you can pipeline commands. They can save some data,
because some things can be sent once for many commands. And they also
deal with common transient faults as packet dropped now and then due to
congestion or noisy link.

But when it comes to tolerance to major faults, they don't help much.
You still need to solve the case when the connection breaks. And it's
just as nasty.

> > > > > > And no resolution is perfect.
> > > > > > It simply can't be, because you never know for sure whether the 
> > > > > > client
> > > > > > crashed, or just it's connection died.

"for sure" meant "under any imaginable circumstances". You can guess and
be usualy right.

> > > > > ...actually, you do. That's what NSM gives you, as supported by
> > > > > rpc.statd, which is a required component for fcntl() locks over
> > > > > NFS. It's based on the assumption that clients will get rebooted
> > > > > reasonably quickly.
> > > > 
> > > > ... which may work in few more cases, but does not work generaly.
> > > 
> > > Actually it does work in the general case.
> > 
> > But not universaly.
> 
> Right, only when both server and client are running NLM and NSM,
> otherwise you'll get an error when you try to acquire a lock. Which is
> quite adequete, if your boxes run the relevant services, otherwise
> it's useless.

Even if they are both running NLM and NSM. The assumption that client is
rebooted in a resonable time is mostly true, but not guaranteed to be
true under any imaginable circumstances (excluding circumstances where
it does not matter anymore, like server being destroyed, of course).

-------------------------------------------------------------------------------
                                                 Jan 'Bulb' Hudec 
<address@hidden>

signature.asc
Description: Digital signature