[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Quilt-dev] Another shell re-write of backup-files.
From: |
Jean Delvare |
Subject: |
Re: [Quilt-dev] Another shell re-write of backup-files. |
Date: |
Mon, 21 Mar 2011 13:40:37 +0100 |
User-agent: |
KMail/1.12.4 (Linux/2.6.32.29-0.3-pae; KDE/4.3.5; i686; ; ) |
Hi Kaz,
On Sunday 20 March 2011 07:43:45 pm Kaz Kylheku wrote:
> On Sat, 19 Mar 2011 13:55:05 +0100, Jean Delvare <address@hidden>
>
> wrote:
> > And the fact that the linked
> > copies have their timestamp updated as I work is not a problem at
> > all in practice.
>
> Hi Jan,
>
> But if a widely-included header file is touched in every
> hard-linked tree, you could be looking at long incremental rebuilds.
This is correct. But then again it isn't a big problem in practice, at
least for me / us (Suse). For one thing, the reference tree (from which
others are hard-linked) is typically never used for builds, it's only a
reference to spawn working trees. For another, the touch only impacts
other trees when you pop down to the first patch in the series which
modifies a given file, in several related trees at once. This seems
rather unlikely to happen. Lastly, my typical work session rarely
implies building different trees at the exact same time. In fact, I have
never noticed any unexpected rebuild so far. To be fair, I did not even
realize it could happen until you pointed it out.
Anyway, we don't have any other choice here. Not touching the files
after "quilt pop" would lead to incorrect partial rebuilds.
You may argue that using hard-linked trees is a bad idea because of the
above theoretical issue. Well, if you don't like it, just don't do it,
and let people who like it, do it.
> The reason it works this way is obviously for speed.
> Quilt restores files using "backup-files -r -t".
> Files are restored by hard linking followed by
> a touch. (In my script I chose to implement -r -t
> as an inefficient copy; a decision I will revert
> for the sake of compatibility and speed.)
For completeness, please note that "hard linking followed by a touch" is
really a trick for "move followed by a touch" here, when we don't have
to keep the backup. The "-r -k" combination has a lot in common with "-
r" from a technical perspective, but semantically they are completely
different. In practice, quilt never "-r -k" to the original working
tree; it's always to a separate, temporary workplace. You really
shouldn't think of "-r -k" as "restore and keep the backup" as it is
advertised, but as "duplicate the backup". It might as well have been
made a separate action IMHO, for clarity.
> Link + touch means that a "quilt pop -a" never has
> to copy any files (except maybe in some corner cases),
> making it very fast.
Correct.
> I do not believe that it's due to any intent
> to preserve links. That is a naive interpretation
> of what the program does, because such a requirement
> would be totally nonsensical. Rather, it's all about
> speed.
And I don't share your analysis, sorry. The link preservation is by
intent here, not by accident. The code could be simplified if it were
not the case. And, as I explained before, the use case for hard-linked
trees is in heavy use at the company were quilt was developed
originally. This can't be a coincidence, can it?
> "quilt pop" performs a just-in-case link breaking
> pass in the case when not all the patches are popped.
> This evidently covers some corner cases when it is
> suspected that there may be a link between the backup
> directory at the current patch level, and some
> of the working files.
This is so "evident" that your explanation is wrong ;) It's all about
the original files, not their backups. Their backups may well have hard
links, we even preserved them on purpose originally.
Honestly, I have no idea what case is supposed to be handled by the
just-in-case link breaking (other than "user shot him/herself in the
foot on purpose".) It looks totally paranoid to me, and I'd get rid of
it. But I can only imagine that I'm missing a case were it would
actually have saved someone's work. This decision if beyond the scope of
the C-to-bash conversion anyway.
> When all patches are popped,
> there is no backup directory, so this suspicion doesn't
> exist.
That's correct.
> Note that this just-in-case breaking is done
> in the backup directory, not in the tree:
>
> # code from pop
> patch="$(top_patch)"
> if [ -z "$patch" ]
> then
> printf $"No patches applied\n"
> else
> # Ensure that the files in the topmost patch have a link count
> # of one: This will automatically be the case in the usual
> # situations, but we don't want to risk file corruption in weird
> # corner cases such as files added to a patch but not modified.
> $QUILT_LIB/backup-files -L -s -B $QUILT_PC/$patch/ -
> printf $"Now at patch %s\n" "$(print_patch $patch)"
> fi
>
> The command
>
> backup-files -L -s -B $QUILT_PC/$patch/ -
>
> means "walk the backup directory $QUILT_PC/$patch,
> and replace any file that has a link count > 1
> with a clone of that file."
No, sorry, it doesn't mean that. See for yourself (with the original C
implementation of backup-files, which is the reference):
# ln AUTHORS AUTHORS2
# ls -l AUTHORS AUTHORS2
-rw-r--r-- 2 jdelvare users 1110 2009-09-09 10:18 AUTHORS
-rw-r--r-- 2 jdelvare users 1110 2009-09-09 10:18 AUTHORS2
# lib/backup-files -B backup/ -b -L AUTHORS
Copying AUTHORS
# ls -l AUTHORS AUTHORS2 backup/AUTHORS
-rw-r--r-- 1 jdelvare users 1110 2011-03-21 11:29 AUTHORS
-rw-r--r-- 2 jdelvare users 1110 2009-09-09 10:18 AUTHORS2
-rw-r--r-- 2 jdelvare users 1110 2009-09-09 10:18 backup/AUTHORS
# ln AUTHORS AUTHORS3
# ls -l AUTHORS AUTHORS2 AUTHORS3 backup/AUTHORS
-rw-r--r-- 2 jdelvare users 1110 2011-03-21 11:29 AUTHORS
-rw-r--r-- 2 jdelvare users 1110 2009-09-09 10:18 AUTHORS2
-rw-r--r-- 2 jdelvare users 1110 2011-03-21 11:29 AUTHORS3
-rw-r--r-- 2 jdelvare users 1110 2009-09-09 10:18 backup/AUTHORS
At this point, both the original and the backup files have a hard-linked
copy.
# lib/backup-files -B backup/ -L -s -
# ls -l AUTHORS AUTHORS2 AUTHORS3 backup/AUTHORS
-rw-r--r-- 1 jdelvare users 1110 2011-03-21 11:30 AUTHORS
-rw-r--r-- 2 jdelvare users 1110 2009-09-09 10:18 AUTHORS2
-rw-r--r-- 1 jdelvare users 1110 2011-03-21 11:29 AUTHORS3
-rw-r--r-- 2 jdelvare users 1110 2009-09-09 10:18 backup/AUTHORS
As you can see, backup-files -L broke the link of the original file, and
left the backup file untouched.
I told you twice already, but I can write it a third time: backup-files
semantics are not trivial.
--
Jean Delvare
Suse L3
- Re: [Quilt-dev] Another shell re-write of backup-files., (continued)
- Re: [Quilt-dev] Another shell re-write of backup-files., Jean Delvare, 2011/03/19
- Re: [Quilt-dev] Another shell re-write of backup-files., Kaz Kylheku, 2011/03/19
- Re: [Quilt-dev] Another shell re-write of backup-files., Jean Delvare, 2011/03/18
- Re: [Quilt-dev] Another shell re-write of backup-files., Kaz Kylheku, 2011/03/18
- Re: [Quilt-dev] Another shell re-write of backup-files., Kaz Kylheku, 2011/03/18
- Re: [Quilt-dev] Another shell re-write of backup-files., Jean Delvare, 2011/03/19
- Re: [Quilt-dev] Another shell re-write of backup-files., Kaz Kylheku, 2011/03/20
- Re: [Quilt-dev] Another shell re-write of backup-files.,
Jean Delvare <=
- [Quilt-dev] Re: Link check after pop really needed? (was: Another shell re-write of backup-files), Jean Delvare, 2011/03/21
- [Quilt-dev] Re: Link check after pop really needed? (was: Another shell re-write of backup-files), Andreas Gruenbacher, 2011/03/21
- [Quilt-dev] Re: Link check after pop really needed? (was: Another shell re-write of backup-files), Jean Delvare, 2011/03/21
- Re: [Quilt-dev] Re: Link check after pop really needed? (was: Another shell re-write of backup-files), Jean Delvare, 2011/03/21
- Re: [Quilt-dev] Another shell re-write of backup-files., Kaz Kylheku, 2011/03/21
- Re: [Quilt-dev] Another shell re-write of backup-files., Jean Delvare, 2011/03/25
- Re: [Quilt-dev] Another shell re-write of backup-files., Kaz Kylheku, 2011/03/18
- Re: [Quilt-dev] Another shell re-write of backup-files., Jean Delvare, 2011/03/19