Re: [Bug-tar] a multithreaded tar ??

[Joerg Schilling]

"Dr. David Alan Gilbert" <address@hidden> wrote:

> > What do you like to achive here and on which OS?
> > 
> > star runs as two processes since 12 years, an archive I/O process and a 
> > file I/O process.
>
> I've been thinking a multithreaded tar would be nice for traversing
> the directory trees in parallel; if you have a very big directory
> tree with zillions of small files a single threaded tar ends up
> I assume spending most of its time waiting for the OS to stat and open
> files and the disc head to whizz back and forward.
>
> If you had a multithreaded tar then you could have multiple outstanding
> open's and stat's and directory reading with the OS and it could
> have a chance to order them for least head movement.

One reason why star is already faster than other tar implementations is that 
star implements a FIFO between a tar process and an archive process.

A tar archive is a linear concatenation of files. Doing this in parallel 
and keeping in mind that star is already two processes would make things hard
to implement. On OS with slow I/O like Linux, it may make sense to have 
separate threads with fsync()/close() hangig aroung on extract mode, bot it 
may be better to just overhaul the OS...

Jörg

-- 
 EMail:address@hidden (home) Jörg Schilling D-13353 Berlin
       address@hidden                (uni)  
       address@hidden     (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/old/private/ ftp://ftp.berlios.de/pub/schily