Re: -warmup

Hi!
I changed code to warmup with number of iterations.

What exactly means 'Baseline (ms)'? Is the shown number the time
for one loop? For all loops together? Please clarify and mention
this on the HTML page.

Clarified that the times are milliseconds for the cumulative time for all iterations.

There seems to be a fundamental math problem in calculating the
percentage numbers. For example, looking at the 'TOTAL' field, the
percental difference between 2.788 and 2.740 is not -6.1% but -1.7%!

it was average of the all percentages but you are right. I have changed it percentage of total time changes.

Looking at the 'Load_Advances (Unscaled)' row, I think that 100%
difference between 0.001 and 0.002 doesn't make any sense. How do
you compute the percentage? Is this based on the cumulative time of
all loops? If so, and you really get such small numbers, there must
be some fine-tuning for high-speed tests (for example, increasing N
for this particular test by a factor of 10, say) to get meaningful
timing values.

it was cumulative time in milliseconds but converted it microseconds as how it was and it seem got better. If any fine-tuning needed since now, i will.

Looking for reply.

On 3 Aug 2023 19:50 +0300, Werner LEMBERG <wl@gnu.org>, wrote:

It is warming up as the given number of seconds with -w flag before
every benchmark test.

There are still differences like 100%.. Also, 1 sec warmup means
(test count)*(font count) 70 secs for the results.

Mhmm, I'm not sure whether a warmup *time span* makes sense. I would
rather have thought that every test would get a certain number of
warmup *loops*. For example, '--warmup 100' means that for a value of
N=50000, the first 100 loops of each test are not taken into account
for timing so that effects of the various processor and memory caches,
the operating system's memory page swapping, etc., etc., doesn't have
too much influence. This should be just a very small fraction of
time, not 70s.

I am thinking of what else can be done and waiting for your test.

Just looking at your most recent HTML page I see some peculiarities.

* What exactly means 'Baseline (ms)'? Is the shown number the time
for one loop? For all loops together? Please clarify and mention
this on the HTML page.

* There seems to be a fundamental math problem in calculating the
percentage numbers. For example, looking at the 'TOTAL' field, the
percental difference between 2.788 and 2.740 is not -6.1% but -1.7%!
What am I missing?

* Looking at the 'Load_Advances (Unscaled)' row, I think that 100%
difference between 0.001 and 0.002 doesn't make any sense. How do
you compute the percentage? Is this based on the cumulative time of
all loops? If so, and you really get such small numbers, there must
be some fine-tuning for high-speed tests (for example, increasing N
for this particular test by a factor of 10, say) to get meaningful
timing values.

Werner

From:	Ahmet Göksu
Subject:	Re: -warmup
Date:	Mon, 7 Aug 2023 15:06:53 +0300

Info	Baseline	Benchmark
Parameters	-c 1000 -w 100	-c 1000 -w 100
Commit ID	d7371720	d7371720
Commit Date	2023-08-03 19:08:57 +0300	2023-08-03 19:08:57 +0300
Branch	GSoC-2023-Ahmet	GSoC-2023-Ahmet

Test	N	* Baseline (µs)	* Benchmark (µs)	Difference (%)
Load	120000	544769	548050	-0.6
Load_Advances (Normal)	120000	472392	483467	-2.3
Load_Advances (Fast)	120000	2818	2804	0.5
Load_Advances (Unscaled)	120000	2774	2875	-3.6
Render	120000	407268	425227	-4.4
Get_Glyph	120000	160786	166644	-3.6
Get_Char_Index	94000	2728	2318	15.0
Iterate CMap	1000	1771	1718	3.0
New_Face	1000	39404	39015	1.0
Embolden	120000	214085	209987	1.9
Stroke	68160 \| 65880	1622171	1618429	0.2
Get_BBox	120000	101134	101693	-0.6
Get_CBox	120000	81055	79277	2.2
New_Face & load glyph(s)	120000	97837	100719	-2.9
TOTAL	2726040	3750992	3782223	0.8

Test	N	* Baseline (µs)	* Benchmark (µs)	Difference (%)
Load	95000	696891	751976	-7.9
Load_Advances (Normal)	95000	614680	740438	-20.5
Load_Advances (Fast)	95000	2292	2519	-9.9
Load_Advances (Unscaled)	95000	2168	2516	-16.1
Render	95000	333506	325103	2.5
Get_Glyph	95000	142596	137808	3.4
Get_Char_Index	94000	2289	3078	-34.5
Iterate CMap	1000	1875	1751	6.6
New_Face	1000	49544	46965	5.2
Embolden	95000	192603	187243	2.8
Stroke	61750 \| 60610	1480048	1471704	0.6
Get_BBox	95000	92079	88674	3.7
Get_CBox	95000	66369	66422	-0.1
New_Face & load glyph(s)	95000	106669	100515	5.8
TOTAL	2214360	3783609	3926712	3.8

Test	N	* Baseline (µs)	* Benchmark (µs)	Difference (%)
Load	95000	849280	869354	-2.4
Load_Advances (Normal)	95000	782796	759969	2.9
Load_Advances (Fast)	95000	2326	2315	0.5
Load_Advances (Unscaled)	95000	2147	2181	-1.6
Render	95000	355258	372212	-4.8
Get_Glyph	95000	142597	145345	-1.9
Get_Char_Index	94000	2515	2298	8.6
Iterate CMap	1000	1757	1724	1.9
New_Face	1000	49602	49108	1.0
Embolden	95000	256686	255087	0.6
Stroke	49305 \| 48640	1544194	1540442	0.2
Get_BBox	95000	105618	108203	-2.4
Get_CBox	95000	67453	70368	-4.3
New_Face & load glyph(s)	95000	126134	124325	1.4
TOTAL	2189945	4288363	4302931	0.3

Re: -warmup

Freetype Benchmark Results

Warning: Baseline and Benchmark have the same commit ID!

Info

Results for Roboto_subset.ttf

Results for Arial_subset.ttf

Results for TimesNewRoman_subset.ttf

Results for Tahoma_subset.ttf

Results for Verdana_subset.ttf

Test	N	* Baseline (µs)	* Benchmark (µs)	Difference (%)
Load	95000	525068	575170	-9.5
Load_Advances (Normal)	95000	447570	455850	-1.8
Load_Advances (Fast)	95000	2323	2268	2.4
Load_Advances (Unscaled)	95000	2239	2169	3.1
Render	95000	311839	312159	-0.1
Get_Glyph	95000	135936	137695	-1.3
Get_Char_Index	94000	2265	2347	-3.6
Iterate CMap	1000	1721	1736	-0.9
New_Face	1000	44266	45632	-3.1
Embolden	95000	202235	196470	2.9
Stroke	69160 \| 67450	1566809	1555649	0.7
Get_BBox	95000	85721	85337	0.4
Get_CBox	95000	63373	62761	1.0
New_Face & load glyph(s)	95000	96532	94596	2.0
TOTAL	2228610	3487897	3529839	1.2

Test	N	* Baseline (µs)	* Benchmark (µs)	Difference (%)
Load	95000	511496	544501	-6.5
Load_Advances (Normal)	95000	417777	433294	-3.7
Load_Advances (Fast)	95000	2352	2244	4.6
Load_Advances (Unscaled)	95000	2148	2194	-2.1
Render	95000	316300	312850	1.1
Get_Glyph	95000	142782	135812	4.9
Get_Char_Index	94000	2274	2411	-6.0
Iterate CMap	1000	1724	1743	-1.1
New_Face	1000	42758	43630	-2.0
Embolden	95000	191345	186361	2.6
Stroke	69160 \| 70585	1567084	1568601	-0.1
Get_BBox	95000	86768	83590	3.7
Get_CBox	95000	63951	64501	-0.9
New_Face & load glyph(s)	95000	95327	92563	2.9
TOTAL	2231745	3444086	3474295	0.9