Re: [Quilt-dev] [patch 0/8] test/run: Cleanups and optimizations

[Andreas Grünbacher]

2014-02-05 Jean Delvare <address@hidden>:
> Hi Andreas,
>
> Le Sunday 02 February 2014 à 23:42 +0100, Andreas Grünbacher a écrit :
>> just a heads up, I wrote a utility called exxe which could eventually
>> replace the run script if a few more features are added. It's written
>> in C, and relatively simple and fast. Maybe we want to switch to that
>> at some point.
>>
>>   http://git.drbd.org/gitweb.cgi?p=exxe.git
>>
>> I originally wrote this for the drbd test suite where we need to run
>> various commands on various cluster nodes, send them stdin input, and
>> capture their stdout, stderr, and exit status. The test suite keeps an
>> ssh connection open to each cluster node; it uses this connection for
>> running commands on the cluster node.
>
> Seems a bit overkill, as remote execution of commands is not something
> we need to test in quilt.

I didn't say it can only do remote command execution, but it can do that as
well. I'm thinking of something pretty close to what the run script does
today.

>> The test suite itself (http://git.drbd.org/gitweb.cgi?p=drbd-test.git)
>> is a bit of a shell hell, but there are enough test cases in exxe
>> itself to easily understand how exxe works without having to
>> understand the drbd test suite.
>
> I can see that the test case format is different from what we use in
> quilt right now. So we'd have to rewrite all 52 test cases. We'd need a
> really good reason for that.

That's really not such a big deal if there are clear advantages.

> I can only think of two:
>
> * Performance, if exxe is much faster than what we have. Do you have
> benchmarks?

Running tests isn't even supported yet; right now, the utility will only run
commands. So of course I don't have any numbers.

> Admittedly running the test suite is the longest part of
> building quilt at the moment, for example it takes 31 seconds on my
> laptop while ./configure && make takes only 1.25 second. OTOH since I
> made it possible to run multiple tests in parallel, the test suite
> completes in 10 seconds on my laptop. That's still very fast compared to
> building most other packages. So performance isn't really an issue at
> the moment. Me trying to optimize the test script was more of a
> theoretical exercise (or a proof of my inability to properly prioritize
> my efforts and spend my time on things that really matter...) In the end
> I think I made the code rather cleaner than significantly faster.

I don't have an idea how much of that time is actually spent in quilt; that
time wouldn't change with a different test driver.

Andreas