Re: segfault in gmp p-1 factorer

[Kevin Ryde]

Alexander Kruppa <address@hidden> writes:
>
> But I found out something different. The code that segfaults was built
> with a gmp-mparam.h generated by tuneup, which has 
> 
> #define KARATSUBA_SQR_THRESHOLD     65

Why did the value come out as 65?  Around 30 is about right.

> If I build gmp with the default gmp-mparam.h, which has 
> 
> #define KARATSUBA_SQR_THRESHOLD     34
> 
> the code works without segfault.

Ah yes, if you tune that value it needs to be updated in config.m4
too, for the benefit of the k6 sqr_basecase.asm.  If you do that,
remove mpn/sqr_basecase.lo and rebuild it should be right.

It's vital config.m4 and gmp-mparam.h are the same, since
mpn_sqr_basecase relies on mpn_kara_sqr_n not to call it with a size
bigger than it can handle.  That's not described properly in
tune/README, I'll fix that.  (config.m4 is generated by ./configure
normally, so maybe a re-run of configure should be recommended when
updating thresholds.)