octave-maintainers
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: For loop benchmarks between Octave versions


From: Mike Miller
Subject: Re: For loop benchmarks between Octave versions
Date: Mon, 23 Jun 2014 10:46:05 -0400

On Mon, Jun 23, 2014 at 12:35:25 +0100, Carnë Draug wrote:
> On 6 June 2014 17:16, Rik <address@hidden> wrote:
>> 6/6/14
>>
>> All,
>>
>> I followed up one of the recent posts about differing performance between
>> Octave versions.  Alas, I think it is quite true.  We don't yet have 100%
>> test coverage for behavior, but it seems like another wish list item would
>> be a set of benchmarking tests so that we don't accidentally introduce
>> large performance regressions.
>>
>> The test code in question is a double for loop.  We expect for loops to be
>> inefficient and slow, but at least they shouldn't get any worse than they
>> already are.
>
> Just an idea, but would it be possible to have hydra [1] run such
> checks with each commit?

We certainly could run some benchmark test with each build (each hydra
build ≠ each hg commit). The first step would be to commit something
to the repo, a benchmark make target maybe?

Once we have something automated that works, how do we compare it over
time? I don't know that the Octave hydra build is done on the same
(real or virtual) hardware each time, or what other builds or resource
problems it might be contending with at the same time. Even if we
could be sure that the timing output were reliable and comparable and
the hydra build host never changed, is someone going to look at the
logs every day? I only look at them when I get a notification that a
build failed. That doesn't even include test failures.

-- 
mike



reply via email to

[Prev in Thread] Current Thread [Next in Thread]