[Monotone-devel] benchmarks

[Nathaniel Smith]

On Fri, Mar 11, 2005 at 01:40:59PM +0100, Georg-W. Koltermann wrote:
> The one big issue left that I have is performance.  I did a short
> benchmark of several revision control systems with a snapshot of my
> sourcecode, see attached screen shot (it's a gnumeric spreadsheet, I
> didn't know if it was a good idea to attach the data file as I was
> unsure how may of you would have gnumeric available, hence the screen
> shot).  Times are in seconds.  I have highlighted the problem areas.
> 
> You will see that it is not practical to use monotone for this use case
> right now, it just eats up too much developer time.

Indeed, those numbers do seem unreasonable.  Sorry for being slow to
respond on these issues; it's not that I don't like performance bugs,
it's just that there are many things to do, and it's often not clear
how to reproduce these kinds of bugs to work on them...

In light of that last point, would you (or anyone else out there) be
interested in beginning to put together a set of performance tests for
monotone, that could be run automatically and used as benchmarks?  We
have a way to track test coverage:
  http://frances.vorpus.org/~monotone-cov/net.venge.monotone/
and it turns out to be very useful to actually see how things are
going.  Something similar for performance would be very, very useful;
we could actually see progressions and regressions that way...
It would also give developers something concrete to point their
profilers at and work on improving.

Anyone interested?  For a first pass, just starting to gather and
distribute some scripts that get monotone to do something slow, would
be a great start...

-- Nathaniel

-- 
"If you can explain how you do something, then you're very very bad at it."
  -- John Hopfield