[Duplicity-talk] Increasing number of asynchronously created temp files?

[Tapio Sokura]

Hello,

I just started using duplicity (0.7.15 from epel) and it seems to be 
working fine. I'm using the asynchronous-upload option to make better 
use of the CPU and network resources. I can see the two temporary gpg 
files of volsize being used.

The stuff I'm backing up varies, there are collections of large files, 
but also some areas of the file system have a lot of small files. When 
the backup process comes across areas of small files, the upload 
"pipeline" stalls. Meaning that the archive file upload is happening 
faster than the archive file creation. Thus changing the backup process 
from being mostly network speed bound to CPU(/disk) bound when it hits 
these small files.

I tried to look for a setting in the duplicity man page for increasing 
the number of temporary archive files to use from two, but couldn't find 
one. Would it be possible to add such a user-tunable feature? This way 
when duplicity is going through sections of the filesystem containing 
large files it could build up a bigger queue of upload work. This queue 
could then keep the upload process better occupied when the archiving 
part runs into those areas of smaller files.

  Tapio