Re: [Gnu-arch-users] back-building pfs updates

[Tom Lord]

    > From: Aaron Bentley <address@hidden>

    > Merging back-building will have two impacts:
    > - we'll want better revision-selection heuristics
    > - most operations will no longer will need to be serial

    > Better heuristics
    > =================
    > We need to know how long a download will take.  Currently we can only
    > count revisions.  In order to estimate download time, we need to know
    > file size.  pfs does not provide this, but most supported protocols can.

    > This is another area in which http blows.  Can we add file size data to
    > the listing files?  Should we add other file data?

This was discussed a while back.   Being behind in my email, I'll
leave it to others to search the archives.  (It was in the
summary-deltas discussion.)

Short answer:  yes -- I'm interested in adding file sizes to the
archive data.   Indeed, I'm intrigued by the idea of adding
"summaries" of file sizes to the archive (so that, looking at just
patch-N, you can get a clue about the total size of changesets back to
some previous revision).



    > Parallel operations
    > ===================
    > These operations could profitably be implemented as parallel operations:

    > - finding cacherevs
    > - calculating total download cost of a patch list
    > - retrieving a patch list
    > - finding the continuations of a list of package-versions

    > This would suggest making parallel versions of
    > file_exists
    > get_file
    > file_size

    > One way of doing this would be to "prime" the pfs.  Priming would mean
    > "I will probably want to do this, so be ready when I do".  The pfs could
    > then start doing the operations into buffers.  When the actual requests
    > came in, they would be satisfied from the buffer, blocking as necessary.

    > pfses that didn't support priming would simply treat priming commands as
    > noops.
    > Priming could be a mode, but considering the bad effects of forgetting
    > to turn the mode off (all operations are noops!), I think it would be
    > better to implement it as an alternate function or function parameter.

    > What do you make of all this?

I'm generally against the idea of parallel operations, at least at
this stage.   It counts as premature optimization at complexity cost
in my book.   It's foolish for local operations and I'd rather see
emphasis on making sure that local operations are the common case.

In other words, in my current thinking, a little 3-line convenience
feature hack for archive-mirror is worth a 300 line revision for
parallel operations.    A less latency-bound archive-mirror at a cost
of 1500 lines is worth 150,000 lines of parallel operation code.

-t