Re: [Bug-tar] GNU tar, star and BSD tar speed comparision +script

[Joerg Schilling]

"Jan Psota" <address@hidden> wrote:

> Concatenated output of tcp script (below).
> I replaced 'tar' with 'star' by hand. Have fun :-).

I am sorry, but it seems that your tests cannot be repeated anywhere
as important information (such as the test data) is missing.

-       If you like to do real testing, please provide the test data tree.

-       If you like to make comparable tests, never compare against gtar
        without archive type parameter. GNU tar by default creates the
        non-POSIX GNUTAR archive format that needs less USER CPU time
        to create, make sure to add "-Hustar" to the GNU tar parameters.


-       What compiler has been used to compile the programs?

-       What compiler optimization options have been used to compile
        the programs?

        I did my test on Solaris with SunStudio C and with highest optimization
        for both gtar and star. 

-       Provide data for your platform:

        -       CPU type and clock rate

        -       CPU cache size

        -       Number of CPU cores in the system.

        -       OS name and revision

        -       Amount of RAM in the system

-       Was the system loaded when you did run the tests?

-       How many time did you repeat the tests?

-       What was the standard deviation for the test results?

-       If your time(1) program is really unable to give better precision
        than 0.01 seconds, you cannot do tests that result in less than
        0.5 second in any of the realtime, usertime, systemtime results.

-       If you like to make "dummy" tests, you need to know that GNU tar
        is cheating and has irregular behavior. Never use "/dev/null"
        with GNU tar, use "/dev/zero" instead.

-       If you believe that star is slower with reiserfs, you would need
        to run additional tests with different filesystems. It may be
        that the results are caused by implementation deficits in reiserfs.


> 1. Tested on Athlon 1500MHz, RAM 512MB, disk speed 50MB/s, reiserfs
> best time of 3 repetitions, /tmp/drv, 82M, 4105 files
> program       operation       real    user    system  speed
> bsdtar        create          0.71    0.10    0.60    102253 KB/s
> star  create          1.18    0.02    1.05     61532 KB/s
> gnutar        create          0.67    0.01    0.65    108370 KB/s

These results are beyond reality:

It is obvious that you did make a mistake here. Star needs much less
user CPU time than other tar implementations but any ration that is much 
higher than 1:3 cannot be correct. In addition, if star needs more system
CPU time than GNU tar, there must be a mistake. Star typically uses the same
systemGPU time as GNU tar.

> bsdtar        list            0.05    0.02    0.02   1452160 KB/s
> star  list            0.59    0.02    0.46    123064 KB/s
> gnutar        list            0.22    0.02    0.19    330036 KB/s

A list speed of 1.4 GB/s with bsdtar cannot be correct.

If you get less USER cpu time with GNU tar than with star, your test is not 
correct. Please explain.

Star has for unknown reasons a higher standard deviation for the list results
compared with gtar. Star is definitely not slower in list mode.


> bsdtar        extract         0.74    0.08    0.65     98118 KB/s
> star  extract         1.29    0.03    1.16     56285 KB/s
> gnutar        extract         0.68    0.04    0.64    106776 KB/s

I did a test with gtar vs. star with unpacking linux-2.6.17.3.tar
on tmpfs:

star:
real        0.969
user        0.127
sys         1.153

gtar:
real        1.043
user        0.156
sys         0.887

If you get much different results, please explain why!


> bsdtar        compare         unsupported
> star  compare         1.04    0.22    0.71     69815 KB/s
> gnutar        compare         0.59    0.16    0.43    123064 KB/s

Running a diff test with linux-2.6.17.3.tar with gtar vs. star
gives:

star:
real        0.675
user        0.267
sys         0.717

gtar:
real        0.729
user        0.200
sys         0.522

Note that this was a quad core system!

Star -diff times without FIFO (single process via -no-fifo option):
real        0.597
user        0.152
sys         0.444

is still faster than gtar.

Conclusions:

I cannot believe your tests.

This was all Solaris, but star on Linux is still faster than gtar.
Let me use the "create" test on /usr (1.7 GB) as an example.....

Linux:

time -p tar -cf /dev/zero -Hustar /usr
tar: Removing leading `/' from member names
tar: Removing leading `/' from hard link targets
real 231.81
user 1.93
sys 16.32

time -p star -c f=/dev/zero /usr (gcc compiled with -O)
real 223.05
user 2.48
sys 20.98

time -p star -c f=/dev/zero /usr (gcc compiled with -O4 
-fexpensive-optimizations)
real 224.28
user 2.15
sys 21.51

time -p star -c f=/dev/zero /usr (also the same Linux system but compiled
                                with Sun Studio 12 -fast)
real 220.84
user 1.97
sys 21.30


This was a loaded Linux system (I have no idle one).


As I mentioned before: If you give me serious tests that show star slower
than gtar, I call this a bug and I would fix this.

But make sure that you do not compile the programs with different compilers 
or different optimization levels. For best results, I recommend to compile
all programs with Sun Studio 12 using -fast

For "bsdtar", please provide a download URL


Jörg

-- 
 EMail:address@hidden (home) Jörg Schilling D-13353 Berlin
       address@hidden                (uni)  
       address@hidden     (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/old/private/ ftp://ftp.berlios.de/pub/schily