[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Performance test results Guile 3.0.8 / 3.0.9 / 3.0.10
From: |
Dr. Arne Babenhauserheide |
Subject: |
Performance test results Guile 3.0.8 / 3.0.9 / 3.0.10 |
Date: |
Sun, 30 Jun 2024 10:36:49 +0200 |
Hi,
I ran and evaluated ecraven’s r7rs-benchmarks for Guile 3.0.8 to 3.0.10
and found 3.0.10 to be around 3% faster than 3.0.8 and 3.0.9.
I’m actually calculating the geometric mean of the slowdown of different
tests compared to the fastest:
=== Guile--V3.0.10 Geometric Mean slowdown (successful tests / total tests) ===
1.0241580763853801 (55 / 57)
--
=== Guile--V3.0.9 Geometric Mean slowdown (successful tests / total tests) ===
1.0601747419740355 (55 / 57)
--
=== Guile--V3.0.8 Geometric Mean slowdown (successful tests / total tests) ===
1.0568441538253395 (55 / 57)
This was only run once, so take it with a grain of salt. Or better
still: run it yourself and post the result, so we can compare!
To reproduce on Guix:
export PROGRAMS=/tmp # adjust this
cd $PROGRAMS || exit 1
git clone https://github.com/ecraven/r7rs-benchmarks
git clone https://git.savannah.gnu.org/git/guile.git
hg clone https://hg.sr.ht/~arnebab/wisp
export VERSIONS="v3.0.10 v3.0.9 v3.0.8";
cd $PROGRAMS/r7rs-benchmarks;
for i in $VERSIONS; do
(cd $PROGRAMS/guile;
git checkout $i;
guix shell -D guile gperf sed guile -- \
bash -x -c 'make clean; autoreconf -i; ./configure CFLAGS="$CFLAGS
-march=native"; make -j6');
GUILE=$PROGRAMS/guile/meta/guile ./bench guile all;
cat results.Guile >> results.Guile--$i && rm results.Guile;
done
rm all.csv;
for i in $VERSIONS; do
grep -a -h '+!CSVLINE' results.Guile--$i | sed s/guile/guile--$i/g | sed
's/+!CSVLINE!+//' >> all.csv;
done
for i in $VERSIONS; do
$PROGRAMS/wisp/examples/evaluate-r7rs-benchmark.w
$PROGRAMS/r7rs-benchmarks/all.csv guile--$i 2>/dev/null;
done | grep -A2 "Geometric Mean slowdown"
More detailed output (without the grep, and reducing redundant best time
output):
=== Best times ===
(("ack:3:12:2" . 3.026162621)
("array1:1000000:500" . 2.570030061)
("browse:2000" . 4.439979993)
("bv2string:1000:1000:100" . 2.772243723)
("cat:50" . 37.996525162)
("chudnovsky:50:500:50:1000" . 0.13607684)
("compiler:2000" . 2.829384225)
("conform:500" . 6.061795319)
("cpstak:40:20:11:1" . 15.376498663)
("ctak:27:16:8:1" . 29.210273056)
("deriv:10000000" . 6.459855722)
("destruc:600:50:4000" . 3.844580263)
("diviter:1000:1000000" . 3.47571884)
("divrec:1000:1000000" . 5.923372727)
("dynamic:500" . 5.790662373)
("earley:1" . 4.740488712)
("fft:65536:100" . 3.663431697)
("fib:40:5" . 4.166439342)
("fibfp:35.0:10" . 12.587807545)
("gcbench:20:1" . 0.995555936)
("graphs:7:3" . 11.161914468)
("lattice:44:10" . 6.287998907)
("matrix:5:5:2500" . 3.700863832)
("maze:20:7:10000" . 2.330800762)
("mazefun:11:11:10000" . 4.386609981)
("mbrot:75:1000" . 26.094648762)
("mbrotZ:75:1000" . 26.299364033)
("mperm:20:10:2:1" . 7.026928755)
("nboyer:5:1" . 1.970066148)
("nqueens:13:10" . 7.917849131)
("ntakl:40:20:12:1" . 3.363077275)
("nucleic:50" . 6.278230475)
("paraffins:23:10" . 2.959535594)
("parsing:2500" . 5.61743579)
("peval:2000" . 5.08106331)
("pi:50:500:50:100" . 0.082752902)
("pnpoly:1000000" . 16.038586581)
("primes:1000:10000" . 3.248385311)
("puzzle:1000" . 7.033796782)
("quicksort:10000:2500" . 4.122845352)
("ray:50" . 8.519810821)
("read1:2500" . 6.36043769)
("sboyer:5:1" . 1.591647452)
("scheme:100000" . 7.302776431)
("simplex:1000000" . 5.685521242)
("slatex:500" . 10.970135207)
("string:500000:100" . 0.677046739)
("sum1:25" . 5.613144143)
("sum:10000:200000" . 1.710376816)
("sumfp:1000000.0:500" . 22.795371313)
("tail:50" . 12.964143459)
("tak:40:20:11:1" . 2.135131049)
("takl:40:20:12:1" . 3.367369662)
("triangl:22:1:50" . 2.935341486)
("wc:inputs/bib:50" . 12.688892668))
=== Guile--V3.0.10 times ===
(("ack:3:12:2" "3.026162621")
("array1:1000000:500" "2.807721003")
("browse:2000" "4.439979993")
("bv2string:1000:1000:100" "2.772243723")
("cat:50" "38.846298461")
("chudnovsky:50:500:50:1000" "0.147779112")
("compiler:2000" "2.866082384")
("conform:500" "6.061795319")
("cpstak:40:20:11:1" "16.299212988")
("ctak:27:16:8:1" "29.419785896")
("deriv:10000000" "6.459855722")
("destruc:600:50:4000" "3.886062447")
("diviter:1000:1000000" "4.1036992")
("divrec:1000:1000000" "6.254394067")
("dynamic:500" "6.237513291")
("earley:1" "4.752008016")
("equal" "ULIMITKILLED")
("fft:65536:100" "3.663431697")
("fib:40:5" "4.36791794")
("fibc" "ULIMITKILLED")
("fibfp:35.0:10" "12.716975291")
("gcbench:20:1" "0.995555936")
("graphs:7:3" "11.161914468")
("lattice:44:10" "6.420426816")
("matrix:5:5:2500" "3.813956798")
("maze:20:7:10000" "2.330800762")
("mazefun:11:11:10000" "4.953506464")
("mbrot:75:1000" "26.225870661")
("mbrotZ:75:1000" "26.356033154")
("mperm:20:10:2:1" "9.663579251")
("nboyer:5:1" "2.005036256")
("nqueens:13:10" "7.917849131")
("ntakl:40:20:12:1" "3.371785352")
("nucleic:50" "6.327269619")
("paraffins:23:10" "2.959535594")
("parsing:2500" "5.61743579")
("peval:2000" "5.19038936")
("pi:50:500:50:100" "0.08354783")
("pnpoly:1000000" "16.085522527")
("primes:1000:10000" "3.32526491")
("puzzle:1000" "7.033796782")
("quicksort:10000:2500" "4.122845352")
("ray:50" "8.655408113")
("read1:2500" "6.46734249")
("sboyer:5:1" "1.591647452")
("scheme:100000" "7.346666037")
("simplex:1000000" "5.685521242")
("slatex:500" "10.970135207")
("string:500000:100" "0.677046739")
("sum1:25" "5.613144143")
("sum:10000:200000" "1.784586314")
("sumfp:1000000.0:500" "22.798675713")
("tail:50" "13.034485337")
("tak:40:20:11:1" "2.148528019")
("takl:40:20:12:1" "3.367369662")
("triangl:22:1:50" "2.935341486")
("wc:inputs/bib:50" "12.688892668"))
=== Guile--V3.0.10 slowdown ===
(("ack:3:12:2" . 1.0)
("array1:1000000:500" . 1.0924856660655222)
("browse:2000" . 1.0)
("bv2string:1000:1000:100" . 1.0)
("cat:50" . 1.0223645003161987)
("chudnovsky:50:500:50:1000"
.
1.0859975290431494)
("compiler:2000" . 1.0129703695509928)
("conform:500" . 1.0)
("cpstak:40:20:11:1" . 1.0600080906077989)
("ctak:27:16:8:1" . 1.0071725738269661)
("deriv:10000000" . 1.0)
("destruc:600:50:4000" . 1.010789782281104)
("diviter:1000:1000000" . 1.1806763978642187)
("divrec:1000:1000000" . 1.0558839288453237)
("dynamic:500" . 1.0771674964307232)
("earley:1" . 1.002429982370982)
("equal" . #f)
("fft:65536:100" . 1.0)
("fib:40:5" . 1.0483575018047147)
("fibc" . #f)
("fibfp:35.0:10" . 1.0102613378492036)
("gcbench:20:1" . 1.0)
("graphs:7:3" . 1.0)
("lattice:44:10" . 1.0210604217587533)
("matrix:5:5:2500" . 1.0305585320438237)
("maze:20:7:10000" . 1.0)
("mazefun:11:11:10000" . 1.1292333910366854)
("mbrot:75:1000" . 1.0050286899891556)
("mbrotZ:75:1000" . 1.0021547715347372)
("mperm:20:10:2:1" . 1.3752208949213973)
("nboyer:5:1" . 1.0177507278298759)
("nqueens:13:10" . 1.0)
("ntakl:40:20:12:1" . 1.0025893181416712)
("nucleic:50" . 1.0078109818037542)
("paraffins:23:10" . 1.0)
("parsing:2500" . 1.0)
("peval:2000" . 1.0215163723279803)
("pi:50:500:50:100" . 1.0096060437856307)
("pnpoly:1000000" . 1.0029264390451713)
("primes:1000:10000" . 1.0236670196542457)
("puzzle:1000" . 1.0)
("quicksort:10000:2500" . 1.0)
("ray:50" . 1.0159155285074843)
("read1:2500" . 1.0168077741203372)
("sboyer:5:1" . 1.0)
("scheme:100000" . 1.0060099889972929)
("simplex:1000000" . 1.0)
("slatex:500" . 1.0)
("string:500000:100" . 1.0)
("sum1:25" . 1.0)
("sum:10000:200000" . 1.0433878063043156)
("sumfp:1000000.0:500" . 1.0001449592531146)
("tail:50" . 1.005425879328045)
("tak:40:20:11:1" . 1.0062745422611294)
("takl:40:20:12:1" . 1.0)
("triangl:22:1:50" . 1.0)
("wc:inputs/bib:50" . 1.0))
=== Guile--V3.0.10 Geometric Mean slowdown (successful tests / total tests) ===
1.0241580763853801 (55 / 57)
=== Guile--V3.0.9 times ===
(("ack:3:12:2" "3.071804715")
("array1:1000000:500" "2.570030061")
("browse:2000" "5.785597128")
("bv2string:1000:1000:100" "3.123004942")
("cat:50" "38.242498848")
("chudnovsky:50:500:50:1000" "0.13607684")
("compiler:2000" "2.874347126")
("conform:500" "6.175686541")
("cpstak:40:20:11:1" "15.799006881")
("ctak:27:16:8:1" "31.658227849")
("deriv:10000000" "6.741746503")
("destruc:600:50:4000" "3.844580263")
("diviter:1000:1000000" "4.145369749")
("divrec:1000:1000000" "5.923372727")
("dynamic:500" "6.352599595")
("earley:1" "4.740488712")
("equal" "ULIMITKILLED")
("fft:65536:100" "3.940548985")
("fib:40:5" "4.350043701")
("fibc" "ULIMITKILLED")
("fibfp:35.0:10" "12.941261319")
("gcbench:20:1" "1.066529892")
("graphs:7:3" "11.684187274")
("lattice:44:10" "6.37311493")
("matrix:5:5:2500" "3.971696614")
("maze:20:7:10000" "2.385548399")
("mazefun:11:11:10000" "4.434486612")
("mbrot:75:1000" "26.681443377")
("mbrotZ:75:1000" "26.299364033")
("mperm:20:10:2:1" "7.026928755")
("nboyer:5:1" "1.970066148")
("nqueens:13:10" "7.92781873")
("ntakl:40:20:12:1" "3.363077275")
("nucleic:50" "6.278230475")
("paraffins:23:10" "6.824999971")
("parsing:2500" "5.785535478")
("peval:2000" "5.183468797")
("pi:50:500:50:100" "0.083377889")
("pnpoly:1000000" "16.423639967")
("primes:1000:10000" "3.282540743")
("puzzle:1000" "9.290568978")
("quicksort:10000:2500" "5.260418029")
("ray:50" "8.519810821")
("read1:2500" "6.617283917")
("sboyer:5:1" "1.631722978")
("scheme:100000" "7.302776431")
("simplex:1000000" "6.144728481")
("slatex:500" "11.135196346")
("string:500000:100" "0.749629931")
("sum1:25" "5.932732862")
("sum:10000:200000" "1.785624114")
("sumfp:1000000.0:500" "22.795371313")
("tail:50" "13.169771799")
("tak:40:20:11:1" "2.135131049")
("takl:40:20:12:1" "3.392594552")
("triangl:22:1:50" "3.085200044")
("wc:inputs/bib:50" "13.692880703"))
=== Guile--V3.0.9 slowdown ===
(("ack:3:12:2" . 1.0150824987670084)
("array1:1000000:500" . 1.0)
("browse:2000" . 1.303068287947576)
("bv2string:1000:1000:100" . 1.1265261117159)
("cat:50" . 1.006473583701438)
("chudnovsky:50:500:50:1000" . 1.0)
("compiler:2000" . 1.0158914086686124)
("conform:500" . 1.0187883648336031)
("cpstak:40:20:11:1" . 1.027477530955514)
("ctak:27:16:8:1" . 1.083804584377111)
("deriv:10000000" . 1.0436373184063505)
("destruc:600:50:4000" . 1.0)
("diviter:1000:1000000" . 1.1926654426973156)
("divrec:1000:1000000" . 1.0)
("dynamic:500" . 1.0970419592446166)
("earley:1" . 1.0)
("equal" . #f)
("fft:65536:100" . 1.0756441803533372)
("fib:40:5" . 1.0440674503884328)
("fibc" . #f)
("fibfp:35.0:10" . 1.028079057670404)
("gcbench:20:1" . 1.0712907767745958)
("graphs:7:3" . 1.0467906117268053)
("lattice:44:10" . 1.0135362655526619)
("matrix:5:5:2500" . 1.0731809637680287)
("maze:20:7:10000" . 1.023488767419581)
("mazefun:11:11:10000" . 1.0109142666449422)
("mbrot:75:1000" . 1.0224871628030692)
("mbrotZ:75:1000" . 1.0)
("mperm:20:10:2:1" . 1.0)
("nboyer:5:1" . 1.0)
("nqueens:13:10" . 1.0012591296998787)
("ntakl:40:20:12:1" . 1.0)
("nucleic:50" . 1.0)
("paraffins:23:10" . 2.306105047304256)
("parsing:2500" . 1.029924630077525)
("peval:2000" . 1.0201543418674701)
("pi:50:500:50:100" . 1.0075524481304594)
("pnpoly:1000000" . 1.0240079376106714)
("primes:1000:10000" . 1.0105145876273791)
("puzzle:1000" . 1.3208469431154515)
("quicksort:10000:2500" . 1.2759193178197097)
("ray:50" . 1.0)
("read1:2500" . 1.0403818478410405)
("sboyer:5:1" . 1.0251786449000642)
("scheme:100000" . 1.0)
("simplex:1000000" . 1.080767834549232)
("slatex:500" . 1.0150464088076758)
("string:500000:100" . 1.107205585403462)
("sum1:25" . 1.0569357762526996)
("sum:10000:200000" . 1.0439945731818199)
("sumfp:1000000.0:500" . 1.0)
("tail:50" . 1.0158613132175152)
("tak:40:20:11:1" . 1.0)
("takl:40:20:12:1" . 1.0074909773894611)
("triangl:22:1:50" . 1.0510531938838275)
("wc:inputs/bib:50" . 1.0791233767412935))
=== Guile--V3.0.9 Geometric Mean slowdown (successful tests / total tests) ===
1.0601747419740355 (55 / 57)
("/home/arne/eigenes/Programme/guile/libguile/.libs/guile"
"/home/arne/eigenes/Programme/r7rs-benchmarks/all.csv"
"guile--v3.0.8")
=== Best times ===
(("ack:3:12:2" . 3.026162621)
("array1:1000000:500" . 2.570030061)
("browse:2000" . 4.439979993)
("bv2string:1000:1000:100" . 2.772243723)
("cat:50" . 37.996525162)
("chudnovsky:50:500:50:1000" . 0.13607684)
("compiler:2000" . 2.829384225)
("conform:500" . 6.061795319)
("cpstak:40:20:11:1" . 15.376498663)
("ctak:27:16:8:1" . 29.210273056)
("deriv:10000000" . 6.459855722)
("destruc:600:50:4000" . 3.844580263)
("diviter:1000:1000000" . 3.47571884)
("divrec:1000:1000000" . 5.923372727)
("dynamic:500" . 5.790662373)
("earley:1" . 4.740488712)
("fft:65536:100" . 3.663431697)
("fib:40:5" . 4.166439342)
("fibfp:35.0:10" . 12.587807545)
("gcbench:20:1" . 0.995555936)
("graphs:7:3" . 11.161914468)
("lattice:44:10" . 6.287998907)
("matrix:5:5:2500" . 3.700863832)
("maze:20:7:10000" . 2.330800762)
("mazefun:11:11:10000" . 4.386609981)
("mbrot:75:1000" . 26.094648762)
("mbrotZ:75:1000" . 26.299364033)
("mperm:20:10:2:1" . 7.026928755)
("nboyer:5:1" . 1.970066148)
("nqueens:13:10" . 7.917849131)
("ntakl:40:20:12:1" . 3.363077275)
("nucleic:50" . 6.278230475)
("paraffins:23:10" . 2.959535594)
("parsing:2500" . 5.61743579)
("peval:2000" . 5.08106331)
("pi:50:500:50:100" . 0.082752902)
("pnpoly:1000000" . 16.038586581)
("primes:1000:10000" . 3.248385311)
("puzzle:1000" . 7.033796782)
("quicksort:10000:2500" . 4.122845352)
("ray:50" . 8.519810821)
("read1:2500" . 6.36043769)
("sboyer:5:1" . 1.591647452)
("scheme:100000" . 7.302776431)
("simplex:1000000" . 5.685521242)
("slatex:500" . 10.970135207)
("string:500000:100" . 0.677046739)
("sum1:25" . 5.613144143)
("sum:10000:200000" . 1.710376816)
("sumfp:1000000.0:500" . 22.795371313)
("tail:50" . 12.964143459)
("tak:40:20:11:1" . 2.135131049)
("takl:40:20:12:1" . 3.367369662)
("triangl:22:1:50" . 2.935341486)
("wc:inputs/bib:50" . 12.688892668))
=== Guile--V3.0.8 times ===
(("ack:3:12:2" "3.531747768")
("array1:1000000:500" "2.595002313")
("browse:2000" "5.684264109")
("bv2string:1000:1000:100" "3.093703859")
("cat:50" "37.996525162")
("chudnovsky:50:500:50:1000" "0.152678282")
("compiler:2000" "2.829384225")
("conform:500" "6.102615699")
("cpstak:40:20:11:1" "15.376498663")
("ctak:27:16:8:1" "29.210273056")
("deriv:10000000" "6.637096622")
("destruc:600:50:4000" "3.846330763")
("diviter:1000:1000000" "3.47571884")
("divrec:1000:1000000" "6.130485142")
("dynamic:500" "5.790662373")
("earley:1" "5.007613881")
("equal" "ULIMITKILLED")
("fft:65536:100" "4.071625926")
("fib:40:5" "4.166439342")
("fibc" "ULIMITKILLED")
("fibfp:35.0:10" "12.587807545")
("gcbench:20:1" "1.056158715")
("graphs:7:3" "11.603127482")
("lattice:44:10" "6.287998907")
("matrix:5:5:2500" "3.700863832")
("maze:20:7:10000" "2.43246928")
("mazefun:11:11:10000" "4.386609981")
("mbrot:75:1000" "26.094648762")
("mbrotZ:75:1000" "26.916159183")
("mperm:20:10:2:1" "7.111042727")
("nboyer:5:1" "1.99832413")
("nqueens:13:10" "8.029677037")
("ntakl:40:20:12:1" "3.37413541")
("nucleic:50" "6.361936704")
("paraffins:23:10" "7.215236947")
("parsing:2500" "5.76683771")
("peval:2000" "5.08106331")
("pi:50:500:50:100" "0.082752902")
("pnpoly:1000000" "16.038586581")
("primes:1000:10000" "3.248385311")
("puzzle:1000" "8.812818025")
("quicksort:10000:2500" "5.00484214")
("ray:50" "8.552510496")
("read1:2500" "6.36043769")
("sboyer:5:1" "1.617977606")
("scheme:100000" "7.375016772")
("simplex:1000000" "6.110993795")
("slatex:500" "11.277019932")
("string:500000:100" "0.808208386")
("sum1:25" "5.71583455")
("sum:10000:200000" "1.710376816")
("sumfp:1000000.0:500" "22.9880245")
("tail:50" "12.964143459")
("tak:40:20:11:1" "2.241859341")
("takl:40:20:12:1" "3.398021064")
("triangl:22:1:50" "3.250704609")
("wc:inputs/bib:50" "14.38065278"))
=== Guile--V3.0.8 slowdown ===
(("ack:3:12:2" . 1.167071373987472)
("array1:1000000:500" . 1.0097167159166547)
("browse:2000" . 1.2802454330789144)
("bv2string:1000:1000:100" . 1.115956664752452)
("cat:50" . 1.0)
("chudnovsky:50:500:50:1000"
.
1.1220004961902408)
("compiler:2000" . 1.0)
("conform:500" . 1.0067340412950028)
("cpstak:40:20:11:1" . 1.0)
("ctak:27:16:8:1" . 1.0)
("deriv:10000000" . 1.0274372846124689)
("destruc:600:50:4000" . 1.000455316284289)
("diviter:1000:1000000" . 1.0)
("divrec:1000:1000000" . 1.0349652849053272)
("dynamic:500" . 1.0)
("earley:1" . 1.0563497110168838)
("equal" . #f)
("fft:65536:100" . 1.1114240042565204)
("fib:40:5" . 1.0)
("fibc" . #f)
("fibfp:35.0:10" . 1.0)
("gcbench:20:1" . 1.0608733038582374)
("graphs:7:3" . 1.0395284353114251)
("lattice:44:10" . 1.0)
("matrix:5:5:2500" . 1.0)
("maze:20:7:10000" . 1.043619566141192)
("mazefun:11:11:10000" . 1.0)
("mbrot:75:1000" . 1.0)
("mbrotZ:75:1000" . 1.0234528541916852)
("mperm:20:10:2:1" . 1.011970232648246)
("nboyer:5:1" . 1.0143436716724905)
("nqueens:13:10" . 1.0141235206872246)
("ntakl:40:20:12:1" . 1.0032881001819978)
("nucleic:50" . 1.0133327741524174)
("paraffins:23:10" . 2.4379625511609913)
("parsing:2500" . 1.0265961064060511)
("peval:2000" . 1.0)
("pi:50:500:50:100" . 1.0)
("pnpoly:1000000" . 1.0)
("primes:1000:10000" . 1.0)
("puzzle:1000" . 1.2529247429429076)
("quicksort:10000:2500" . 1.2139291466686088)
("ray:50" . 1.0038380752445113)
("read1:2500" . 1.0)
("sboyer:5:1" . 1.0165427048351157)
("scheme:100000" . 1.009892174802633)
("simplex:1000000" . 1.074834396863555)
("slatex:500" . 1.027974561772418)
("string:500000:100" . 1.1937261335808604)
("sum1:25" . 1.018294632096356)
("sum:10000:200000" . 1.0)
("sumfp:1000000.0:500" . 1.008451416928231)
("tail:50" . 1.0)
("tak:40:20:11:1" . 1.0499867640677076)
("takl:40:20:12:1" . 1.0091024761391343)
("triangl:22:1:50" . 1.1074366047371702)
("wc:inputs/bib:50" . 1.133326063689264))
=== Guile--V3.0.8 Geometric Mean slowdown (successful tests / total tests) ===
1.0568441538253395 (55 / 57)
Best wishes,
Arne
--
Unpolitisch sein
heißt politisch sein,
ohne es zu merken.
draketo.de
signature.asc
Description: PGP signature
- Performance test results Guile 3.0.8 / 3.0.9 / 3.0.10,
Dr. Arne Babenhauserheide <=