Re: build system rules & algorithms

[Mike Shal]

On 6/13/09, Paul Smith <address@hidden> wrote:
> On Fri, 2009-06-12 at 22:31 -0400, Eli Zaretskii wrote:
>  > > Date: Thu, 11 Jun 2009 22:27:48 -0400
>  > > From: Mike Shal <address@hidden>
>  > > Cc: address@hidden
>  > >
>  > > Is there a better way to benchmark it to figure out the bottlenecks?
>  >
>  > One way would be to compile and link Make with -pg, invoke Make as you
>  > always do in this project, and then run gprof on the resulting profile
>  > data.  Feel free to share the results here.
>
>
> You can also use valgrind (if you're running on Linux anyway).  I've
>  used it for memory examination but I'm pretty sure there's a profiling
>  tool available for it as well.
>
>  The trick with both of these is that if you have a recursive make you'll
>  have to figure out a way to combine the results; I don't think either of
>  them have that ability natively, which is too bad.  Maybe someone's
>  already done this.  If you have a non-recursive build environment it's
>  significantly simpler of course.

I found this info on using gprof with multiple processes:
http://lkml.org/lkml/2001/9/5/157

I built make-3.81 with -pg per Eli's suggestion, and ran it in my
kernel tree as follows:

GMON_OUT_PREFIX=gmon /path/to/make-3.81/make

This produced 222 gmon.PID files, which (I think?) gprof automatically
combines if you specify them all:

gprof /path/to/make-3.81/make gmon.* > ~/make-profile.out

The make-profile.out file is quite large, and only the top section
looks useful to me (I'm a bit of a gprof-n00b). Here's a sampling:

Each sample counts as 0.01 seconds.
  %   cumulative   self              self     total
 time   seconds   seconds    calls  ms/call  ms/call  name
 22.14      1.22     1.22  4481117     0.00     0.00  find_char_unquote
  6.17      1.56     0.34  9311932     0.00     0.00  hash_find_slot
  6.17      1.90     0.34  2072900     0.00     0.00  end_of_token
  4.54      2.15     0.25  1384814     0.00     0.00  file_hash_1
  4.17      2.38     0.23   318270     0.00     0.00  collapse_continuations
  3.99      2.60     0.22  1885862     0.00     0.00  a_word_hash_1
  3.63      2.80     0.20  1158525     0.00     0.00  directory_hash_1
  3.45      2.99     0.19   913354     0.00     0.00  handle_function
  3.09      3.16     0.17   756187     0.00     0.00  a_word_hash_2

>From what I can tell, find_char_unquote is only used during Makefile
parsing. However, it's confusing to me why it would claim
find_char_unquote accounts for 1.22s, which is 22.14% of the total
time. This would mean the total time is ~5.5s according to gprof, when
in reality it's ~17s. I feel like something is missing here :)

I may try the valgrind route next to see if that can produce any more
meaningful output.

-Mike