About renameat2() RENAME_EXCHANGE flag and proposal to use it in GNU Coreutils mv

[Juha-Matti Tilli]

Hello,

(I'm not on the list, so please retain my e-mail address in responses.)

Today, I needed an atomic rename or exchange operation for
directories. Of these, for understandable reasons, exchange is easier
to be implemented by the kernel than rename is. I found that recent
Linux has renameat2() RENAME_EXCHANGE flag. I created a command-line
tool around that. One of my coworkers suggested I should do mv -T -b
dir.tmp dir.

I investigated using strace what mv -T -b actually does and it seems
to differ in two ways from my solution:
- The backup is named dir~ (if I recall correctly) and not dir.tmp, so
it's not a true exchange
- The rename operation is not atomic, because it does renameat2()
without RENAME_EXCHANGE to rename dir to dir~ and only then rename()
dir.tmp to dir. So there's a small time window during which dir is not
there.

Now, this could be improved by first doing renameat2() with
RENAME_EXCHANGE to swap dir.tmp and dir, and then afterwards do
renameat2() without RENAME_EXCHANGE to rename the new dir.tmp (the old
dir) into dir~. If done this way on a recent Linux system, there is no
time window during which dir is unreadable. Some details to be worked
out are what to do if dir~ is already there, probably that could be
solved by calling it something else than dir~ such as dir~1, dir~2 or
similar?

Now, I haven't read the source code of GNU Coreutils mv utility, so
I'm not sure if my modification proposal is particularly easy to
implement. At least implementations have to overcome the fact that
renameat2() syscall is exposed only via the syscall header and there
is no glibc wrapper for that. Also, non-Linux and old Linux needs to
be supported as well.

My question is, if I created a patch for improving this mv -T -b
operation to be atomic, would you accept it for the GNU Coreutils
distribution? Of course, the patch would be Linux only, but these days
most Unix-like systems are actually Linux systems. And, the patch
would be naturally created in such a manner that on old Linux systems
or non-Linux systems the RENAME_EXCHANGE is simply not used.

Perhaps mv -T without -b could also be improved...

Another option would be to go to the exchange route and implement a
flag for mv that does an atomic exchange (on recent kernels) or a
non-atomic exchange (on old kernels or non-Linux kernels). I'm pretty
sure such an atomic exchange would be pretty useful. Would you accept
such a patch to GNU coreutils if it's implemented in a manner that
allows it to work also on non-Linux and old Linux? I see the -e
argument is currently unused, so -e would be a pretty good candidate
for atomic exchange command line argument. This might be actually
easier than improving mv -T -b or mv -T. There also could be
--only-if-truly-atomic option that would fail outright if the kernel
doesn't support atomic exchange, without trying to do it in non-atomic
dangerous way that is prone to race conditions.

BR, Juha-Matti