Re: [rdiff-backup-users] How to speed up rdiff-backup

[Carsten Lorenz]

Wiebe Cazemier wrote:

> Carsten Lorenz wrote:
>
>> I started the first backup to an empty directory on a second server.
>> I'm wandering about the low speed of ca. 1.6MB/s
>>
>> Both servers are DL380 with dual PIII 833MHz running debian, python2.3,
>> librsync 0.9.7-1 and rdiff-backup 1.0.0-0.cvs20050819 (thanks to Dean
>> Gaudet for the debian package).
>> They are directly connected by gigabit.
>>
>> I started rdiff-backup this way:
>> rdiff-backup -v5 --terminal-verbosity=4 --ssh-no-compression
>> /mnt/files/ 192.168.0.1::/mnt/raid/backup/fileserver2/
>>
>> Everything looks fine:
>> CPU-usage on the source-server is ca.15% for ssh and ca. 6% for
>> rdiff-backup
>> CPU-usage on the destination is ca. 18% for sshd and ca. 12% for
>> rdiff-backup
>> No swap-space is used.
>>
>> rdiff-backup needs 132s to transfer a 140MB file (1.1MB/s).
>> While rdiff-backup works on the next file scp transfers the same file
>> in 15s (9.3MB/s)!
>>
>> Since we want to backup one tera-byte of data this would last more
>> than a week.
>>
>> Has anyone hints how to speedup rdiff-backup?
>>
>> Carsten
>
>
> What filesystems are your source and destinatin partition? What are
> the mount options? Could you give me the output of "mount" of both
> machines?

I don't think that the file system interferes here.

Both file systems are xfs on raid5 systems.
This is the actual output from mount:

source:
/dev/sde1 on /mnt/files type xfs (rw,noatime,nodiratime,logbufs=8)

destination (4 partitions through LVM2):
/dev/mapper/VG1-LOGVOL1 on /mnt/raid type xfs
(rw,noatime,nodiratime,logbufs=8)

We added the additional options for testing, but they don't change anything.
Both can of cause transfer more than 1.6MB/s. Reading or writing files. ;->
Copying local to an identical raid-system transfers ca. 100MB/s

> Also, when copying with SCP, do you copy to the same target partition
> as you would with rdiff-backup?

Yes and i used the same file and a different file with identical results.

Carsten