Re: [Duplicity-talk] Following up on an extended attributes question

[edgar . soldin]

hey Arjun,

we are a quite friendly and patient bunch on this list. i'd say go for it, 
every contribution is welcome, time is not a factor :)

..ede/duply.net

On May 29, 2017 8:07:58 PM GMT+02:00, Arjun Krishnan via Duplicity-talk 
<address@hidden> wrote:
>Hi Ken
>
>I'd be happy to work on it, with a couple of caveats: I'm a research
>mathematician by day, and have little professional programming
>experience.
>So as long as people are willing to indulge slow progress and trivial
>questions as I get familiar with the codebase, I'd be willing to give
>it a
>shot!
>
>
>> Currently we use the GNU_FORMAT, which is the default format, so
>there is
>> no pax_header in that format.  For sake of backwards compatibility,
>we'd
>> need to be able to read GNU_FORMAT, and the PAX_FORMAT would need to
>be
>> readable by GNU tar for manual recovery of a corrupted backup. 
>Perhaps
>> it's time to change formats?  I don't know all the tradeoffs we'll
>need to
>> make and what the complexity would be.
>>
>gnu tar appears to support pax_format with `--xattrs` flag. From what
>I've
>seen so far, tarfile happily reads both pax and gnu format
>files, but it neither appends extended attributes when creating files,
>nor
>does it extract files properly and set extended attributes.
>Maybe they didn't want to add a dependency on the xattr module, or
>maybe
>this is for cross platform compatibility.
>
>>
>> Since pax_headers are key/value pairs, we need to track their size
>and be
>> prepared for a value that may exceed our max block size.  I've read
>of
>> cases where key/value pairs exceed the size of the file itself,
>performing
>> the functional equivalent of a dictionary file.  We would have to
>allow for
>> cases where a large value crosses a volume boundary.
>>
>I can see that the blocksize is a diff process parameter,
>and that volsize affects the difftar size. I'm still figuring out
>what's
>being compared during the diff process,
>and how the diffs are being created, so I don't understand how the
>xattrs
>fit in to this process.
>
>Is there a brief technical overview of what duplicity is doing?
>
>Arjun