Re: [Quilt-dev] [patch 3/8] tac is not portable

[Gary V. Vaughan]

Jean Delvare wrote:
> Hi Gary,

Salut Jean!

> Granted we don't care about this specific benchmarking anymore, but the
> discussion seems interesting to me still. Benchmarking code changes is
> something we should do on a regular basis, so sharing good techniques
> sounds like a good idea to me.

Cool.  Likewise :-)

>>Okay, my bad.  Lets make it 500 1ines then.  (That will actually
>>make the difference between the two even smaller btw, as the startup
>>time of perl will be less significant as the processing time
>>increases).
> 
> 
> True, which suggests that we should not have been been testing on the
> largest reasonable value but a relatively small one, say 5.

Maybe.  But that doesn't give an accurate indication of performance in
actual common use.  I guess we should be using 20 or 30 patches in this
case... hypothetically, perl might start much slower, and then be able
to process 10 times as fast as tac, so tac is faster up to say 13 lines
and then perl overtakes it significantly after about 15 lines.

>>>Please compare:
>>>
>>>  time head -n 500 <largefile> | perl -e 'print reverse <>'
>>>
>>>with:
>>>
>>>  time head -n 500 <largefile> | gtac
>>>
>>>This will be a more valid test.
>>
>>Ofcourse this just times 'head', and will be identical for both
>>test cases ;-)
> 
> 
> How can you be so sure that head will take much more time than tac/perl?
> I see no evidence. My tests show that tac takes more time:

No, I mean that with that syntax you are running:

  (time head -n 500 <largefile;) | gtac

not:

  time (head -n 500 < large file | gtac)

[[interesting benchmarks snipped]]

> That being said, I fully agree that bringing head into the picture is
> not needed and should be avoided. Anyway, it's also possible to time the
> common part of the commands we try to compare, and substract the result
> to both candidates' times.

Good point, I hadn't thought of that. D'oh!

> Which only proves one thing: benchmarking on a single system has very
> little value.

ACK.  Especially as I have a PowerPC G5, so the instruction times will
have a totally different profile to a Pentium.

Cheers,
        Gary.
-- 
Gary V. Vaughan      ())_.  address@hidden,gnu.org}
Research Scientist   ( '/   http://tkd.kicks-ass.net
GNU Hacker           / )=   http://www.gnu.org/software/libtool
Technical Author   `(_~)_   http://sources.redhat.com/autobook

signature.asc
Description: OpenPGP digital signature