Re: [Autoscons-devel] Re: delay dist command

[Zhang Le]

On Fri, Sep 26, 2003 at 09:44:55AM -0500, David Snopek wrote:
> Ok, this was an oversight on my part.  The builder makes a file with
> contents like:
> 
> file1\n
> file2\n
> file3\n
> 
> Doing a "get_contents().split('\n')" will create the the file list:
> 
> [ 'file1', 'file2', 'file3', '' ]
> 
> The last entry ('') will cause trouble.  While this is still untested,
> doing this ...
> 
> pkg.DistExtra(cache.get_contents().split('\n')[:-1])
> 
> . will exclude the the last entry.  There may be other problems, but the
> concept is good.
It works,  but need a little modifications.
Actually,  in the function to create filelist.cache  open(str(target),
'wt') will open a file called "[<SCons.Node.FS.File instance at 0x84a636c>]"!.
I had to specify 'filelist.cache' as filename.
(How to get the filename from SCons.Node.FS.File instance?)

Here's improvements:
before I use cache it takes scons 8 sec to process SConstruct
after filelist.cache is used,  6.3 sec (adding nodes from cache needs
2.3 sec)
if I do not call get_dirfiles(),  4 sec (that means get_dirfiles() needs
1.7 sec)
It seems I got 1.7 sec improvement from cache.
I also tried pickle the list to a file,  but no significant speedup
gained.

It seems the main computation is adding nodes to the building system
(2.3 sec)

Finally, I found a new solution (less elegant):
1. get_dirfiles() is defined as before, but do not use pkg.DistExtra()
2. add a pre_dist hook: AddPreAction(dist_node,   pre_dist)
3. in pre_dist()
    a. create a package-version dir
    b. copy all files in get_dirfiles() to dir
    c. tar uf package-version.tar dir/files
    d. rm -rf dir

Now no CPU power is wasted on adding nodes, I'm satisfied:-)

BTW: I switch to latest CVS and find Autoscons.get_host() is missing
(previously found in libksd/Autoscons).
-- 
                                     Sincerely yours,
                                            Zhang Le