Re: [Monotone-devel] Speedup chances

[Markus Schiltknecht]

Hi,

Jack Lloyd wrote:
> On Thu, May 08, 2008 at 09:48:14AM -0700, Eric Anderson wrote:
> > Markus Schiltknecht writes:
> >  > Eric Anderson wrote:
> >  > > Revision: 464e510af4959231ff63352c902c689b0f1687aa
> >  > > Branch: net.venge.monotone.experiment.performance
> >  > 
> >  > Hm.. why didn't any of this get merged into mainline? Looks like there 
> >  > are more good ideas lying around in that branch.
> >
> > IIRC the feeling was that many of the fixes were somewhat ugly and
> > people thought this style of performance improvements didn't really
> > matter that much.  They would rather have had algorithmic performance
> > improvements.  (Consider the pthread hack, 20% performance
> > improvement, grotesque code).

Well, yeah, the pthread hack doesn't look appealing... but other bits 
do. I will try to test some of them again with today's monotone.

> Speaking of 'interesting' optimization ideas with potentially nasty
> results on the code, something I thought about a while back:
>
> Currently calculate_ident() returns the hash value immediately. If
> there was a way to instead return a promise of a hash, that could be
> forced to resolve when the value was actually needed, one could do
> some reasonably interesting optimizations (thread pools (what is the
> 'pthread hack'?

No, the pthread hack seems to provide all (or at least most) pthread 
functions, thus replacing libpthread with noop functions. At least I 
find it rather surprising to see a a performance hit for that, but then 
again, what do I know about the pthread library?

> is that basically this idea?), or a SIMD SHA-1 to do 4
> independent SHA computations in parallel with SSE2/Altivec, probably
> other optimizations are possible when batching is feasible). I never
> wrote any code for it. I also did not investigate if it would actually
> help... perhaps at the point calculate_ident is called, it is too late
> to extract much parallelism b/c the value is needed right away, I
> don't know.

I've just started playing around with some interface to threads (see 
nvm.threaded). I'm pretty familiar with twisted's deferreds, but it's 
pretty hard (or at least ugly) to do something similar in C++.

> Somewhat less exotic, but perhaps the appended patch would improve
> performance a bit? It builds and seems to work on Gentoo/amd64, but I
> don't know how much if at all it would improve performance and don't
> have a good way to measure. Clearly it is somewhat less generic and
> more verbose than the previous code, but if it helps, it helps...

Cool, thanks, I will run that through benchmarks that as well.

Regards

Markus