[Plash] Packaging system (was: Plash Wiki)

[Mark Seaborn]

Thomas Leonard <address@hidden> wrote:

> On 2/4/07, Mark Seaborn <address@hidden> wrote:
> > The immediate reason for setting this up is to use it for planning new
> > development.  In particular, right now I am working on a package
> > system for installing Debian packages into Plash sandboxes.  There is
> > more information on the wiki:
> > http://plash.beasts.org/wiki/PackageSystem
> 
> Interesting. I hope you're aware of how much time making your own
> package manager / distribution takes (speaking from experience here
> ;-) My ROX developement work really slowed down when I started working
> on Zero Install. I hope the same thing doesn't happen to plash :-(

Yes, there is that risk, but I think package tools are necessary for
Plash to be usable.  Without tools to set up applications to run under
Plash it is too much hassle to be practical.

As you know I have been considering Zero-Install.  (Thanks for
answering my notes from the wiki, by the way, and apologies for taking
a long time to reply to your answer.)  My design for plash-pkg ended
up being different enough from Zero-Install that it did not seem
sensible to take Zero-Install as a starting point because there would
not be much scope for reusing code.  Both halves of the
package-installing process are different:

 * Setting up the environment is different.  plash-pkg populates a
   root directory rather than setting up environment variables.
 * Choosing packages (i.e. following dependencies) works differently.

The second is more out of choice on my part than the first, but both
come from wanting to stick to how the Debian package system works.

That said, there is a lot of overlap between Zero-Install and
plash-pkg in concept.  I'd like to see if the two can converge
somehow.


> > "You have to declare trust for authors/packagers of all dependencies.
> > This does not scale."
> 
> We'll make it scale when we have enough packages ;-) "Trust all keys
> on the Debian keyring" would be a possible policy, for example.

I would rather just trust the key used to sign the debian Packages
file.  This goes back to the debate that Debian had when they
introduced signatures on the Packages file.

> > "If sandboxing were added, the inability to distinguish kinds of trust
> > would be a vulnerability."
> 
> One of the things I like about the plash model is that it doesn't have
> these kinds of trust levels. I don't say "I trust gedit a lot"; I drag a
> file to it to say "I trust gedit to read this file". Why is it necessary
> to assign levels of trust to programs?

To clarify, I meant the sort of trust you're referring to here.  When
I launch gedit I want to be sure that I am getting a program that was
signed by the public key I specified when I installed gedit, and not a
program signed by a public key associated with another program I
installed.  Secure references, rather than levels of trust.

> > "Zero-Install does not provide a way to refer to an immutable package.
> > There is no way to reproduce a configuration."
> 
> This is not true. In fact, 0compile stores a complete record of the
> configuration in a file called "build-environment.xml", so that you can
> reproduce a build exactly using the digests of each dependency (header
> files and build tools). It also stores the versions, in case you want to
> use a different arch but still get it as close as possible. See the
> Pager 1.1 binary for an example.

That sounds good.  It would be handy if that record were referenced by
binary packages.  That would give an audit trail recording the
compilers that built the package, the compilers that built the
compilers, and so on.

Can a similar file be generated by 0launch?  Would it be possible to
split 0launch into steps that can be invoked separately from the
command line:

 1. Take a feed as input.  Follow all the dependencies (fetching further
    feed files as needed).  Output an environment file.
 2. Fetch and unpack package data for the digests in the environment file.
 3. Set up environment variables from environment file and launch program.

It's useful to have an intermediate file so that it can be saved,
checked, modified by hand, etc.

That is the approach I've taken with plash-pkg.  It's partly inspired
by the way git is split up into lots of small utilities, each of which
has a public documented interface.

In plash-pkg step 1 is actually split further into getting the debian
Packages.gz file and choosing dependencies from it.


> > "Zero-Install does not make it easy for a third party to replace one
> > package in a set of dependencies. To do this you would need to copy all
> > the feed files and re-link them."
> 
> Also not true. For example, if someone provides an alternative to
> ROX-Lib (used by many ROX applications), you can add it with:
> 
> $ 0launch --feed http://thirdparty.com/Alternative-ROX-Lib.xml
>
> It gets the program to extend from the <feed-for> element.

My reservation about this is that the package name in <feed-for> is a
feed URL.  You have to specify that you're replacing a particular
package provider's version of a package, rather than implementing an
interface.  For example, with debian packages, to replace libc6 you
provide a package called "libc6".  With Zero-Install you would
(hypothetically) have to override both "http://ubuntu.com/libc6.xml";
and "http://debian.org/libc6.xml";.

If there was a tool for automatically generating Zero-Install feeds
en-masse from a debian repository it would presumably need to make up
feed URLs.  I can imagine there being problems if the same package is
converted twice but given a different feed URL each time.

The reason I'm thinking about that is that my aim with plash-pkg is to
provide the option of getting updated packages directly from a debian
package repository without users having to wait for them to be wrapped
in a new format by a maintainer.

> The current behaviour works well for CVS checkouts, where you want to
> use whichever is more recent: your checkout or the latest release
> (given that you may have forgotten about the checkout weeks ago).

That's not what you want if you've made local changes though.  I like
what Conary does here: it can attempt to merge your changes into the
newer version of the package for you.


> > "Zero-Install doesn't have a tool to import a Debian repository's
> > packages en-masse. You can convert an individual package but you'd have
> > to follow the dependencies yourself and create a feed for each one."
> 
> This is an interesting one. I was recently challenged to produce a
> version of svn 1.4 for Debian/stable. This involved creating Zero
> Install feeds for 9 separate Debian packages! The whole process took
> roughly an hour doing it manually, but it could certainly be automated.

It would definitely need to be automated if you want to pull in all
dependencies to make them explicit.  For example, Evince pulls in
about 180 debian packages, including >100 library packages.

Would the length of LD_LIBRARY_PATH start to become a problem if those
were all packaged with Zero-Install?

Do you find there are many programs that can't be relocated by setting
environment variables?

Cheers,
Mark