Re: [Chicken-users] Packaging eggs

[Jim Pryor]

Thanks Christian and John for your (quick!) comments.

Yes, this script is part of a larger packaging system, Arch's native
packaging system called pacman. (It attempts also to be usable on other 
distros, but Arch Linux is its home.) It manages binary packages that either 
reside on the distro's repositories, or that you've built yourself, to the same 
format, using a wrapper called makepkg. makepkg reads a packaging script which 
has the format of a bash shell script; the packaging script sets some metadata 
like package name, version number, where to get the source files, dependencies, 
and so on. And it calls the build() function from the packaging script with 
user privileges (and some variables set, as I described); and calls the 
package() function as fakeroot.

We don't use a chroot. One can build inside a chroot, to make
extra sure that the package isn't relying on any undeclared
dependencies, but in that case you'd have:

+------------------------------------
| your real /
| +---------------------------------
| | the chroot, where you've exercised stricter
| | control about what's installed
| | +-------------------------------
| | | the $pkgdir inside the chroot
| | |

And the package() function should still make sure that the relevant
files are installed to (what it sees as) $pkgdir (which is in reality
/chroot/$pkgdir.


I should have said explicitly in my earlier email: when I'm attempting
to build package X, I've always made sure to have previously built AND
TEMPORARILY INSTALLED any packages that X depends on. So as long as my
dependency-extraction is done correctly---and when it isn't, it's
usually obvious---the build shouldn't be failing because any interfaces
are missing. They should all be there, inside /usr/lib/chicken/5, which
is where the egg should think it will also eventually reside. Nothing
else resides in $pkgdir/usr/. For some packages, I need to
explicitly create $pkgdir/usr/lib/chicken/5/ or
$pkgdir/usr/share/chicken/; for most I don't need to do that.

What I'm seeking is the behavior typical of building say a C binary,
where you'd do:
    cd $srcdir && ./configure if necessary
    make PREFIX=/usr
    make DESTDIR=$pkgdir install

I thought that CHICKEN_INSTALL_PREFIX and the -p option should work like
DESTDIR, and so should only be needed at the installation step; and
moreover that they should work the same. However, in my experience
neither of those claims are turning out (reliably) to be true.

Re using salmonella / chicken-install to extract the source or metadata:
I'm only vaguely aware of what's possible here, since as I said I only
use chicken occasionally. I will look into it more closely when I can.

But my initial impressions were:
    * it'd be best never to install anything to my /usr (and etc)
    directories except under control of the Arch package manager
    (pacman). Partly this is because I encounter lots of bugs, both
    in my package-automation scripts and in the eggs themselves,
    and I'd rather keep closer tabs on what's where than is possible
    if I just ran through six or seven abortive attempts to build
    install and extract the metadata from 300-some eggs.
    
    * if that were the only concern, I could work around it by doing
    everything inside a chroot. More work to maintain, but it'd be
    an option. However, there's another reason which moves me even
    more: if the chicken-install system ever writes to /usr and so on,
    then I can't be sure that the packages are built correctly, because
    they might only pass my tests because of the presence of some file
    that did get installed on my machine, but not under control
    of the package manager, so it won't show up on other users'
    machines.

    So the best approach seems to me to be the hygienic one: never
    let chicken write directly to /usr, but only to $pkgdir/usr; and
    rely on pacman to install any properly declared dependencies
    to /usr before we build.

    * Finally, about getting the sources via chicken-install -r:
    The way things are supposed to work with our packaging system
    is that your package script's metadata tells the package manager
    where the source files are, and it will take care of downloading
    them, to the destination you've specified in your config file (or
    via flags when invoking the package manager); it will take
    care of caching them so they don't have to be re-downloaded,
    checking their md5sums (which I calculate and specify in the
    package script), and so on.
    
    Now in especially tricky cases, you can just remove the sources
    from the metadata and control of the package manager, and have
    the build() function itself take care of them. But in that
    case you lose all the integration with the just-mentioned services
    that the package manager provides for source files.

    So that's why I'm currently doing the downloads by specifying
    a henrietta url to get the whole blob, and then unpacking
    the blob in the build() function.

    * Finally, I should say that what I end up posting on the Arch
    User Repo (AUR) is just a tarball of the packaging script (and a
    post-install script). There are no sources or binaries in that
    tarball; the user's own instance of the package manager
    has to download the sources (and verify their md5sums) itself.
    

Hope this additional context is helpful!

-- 
Jim Pryor
address@hidden