[aps-devel] Fwd: GNU package management

[Robert Millan]

On Sat, Nov 30, 2002 at 11:20:24PM +0100, Wolfgang Jaehrling wrote:
> On Sat, Nov 30, 2002 at 03:38:35PM +0100, M. Gerards wrote:
> > > After this works, we can easiely do the Right Thing: We change the
> > > file system to care about problems as well, and tell us about them via
> > > this special node (why do symbolic expressions come to my mind every
> > > time I mention this node? ;-)).  This should not require to much work,
> > > since at this point, we will already have a library that does most of
> > > the work for us, and we can use this library in the file system.
> > 
> > Can you describe this library more detailed? If you mean analysing
> > package dependancy files etc. I would agree.
> 
> Yes, the library would discover conflicts, unresolved dependencies and
> whatever else can go wrong.  The action to perform in the case of such
> a problem would be a callback into the program that uses the library,
> obviously.  This library would not be very large, and we might want to
> have another library using it, which would be libapt-compatible.
> 
> > I have my doubts about a seperate layer. Before I will agree I want to
> > know these things:
> > 
> > - How much will the file lookup process be slowed down?
> 
> We will only know this for sure when we have implemented it.  The
> overhead will be a full RPC (i.e. two non-local IPCs including
> creation plus extraction of the message data) plus additional
> consequences for the implementation; for example, internally looking
> up the node in the second layer.
> 
> I would like to implement it this way and see how it performs; if it
> has a significant impact, we can make this an option for the "custom
> setup" part of the installation: Those who want to use only frontends
> can say that they don't want to use the additional PackageFS layer.
> The default would depend on how much it slows down the system.  And in
> any case, this is (IMO) not a mandatory requirement for the first
> version of GNU, the only thing we should make sure now is that at a
> later point, we are still able to decide which road we want to take.
> 
> > - When a new package is added will unionfs be informed of this
> >   addition? (the filesystem can notify a program of changes, I don't
> >   know the exact possibilities).
> 
> Currently, UnionFS requires that all directory trees it unites are
> given on the command line and they can be modified with fsysopts (or
> at least they should).  In this case, UnionFS would not notice newly
> installed packages.  However, I do not see a reason why we should not
> add an option to UnionFS that makes it unite the _subdirectories_ of
> the directories given at the command line, in which case it would have
> to cache the contents of these to notice additions.  So the short
> answer is that it is no problem to have a new package show up
> immediatelly.
> 
> > - How to deal with filename collisions? (In some cases it might be
> >   usefull to override a part of a package, in some case it isn't)
> 
> I would think that the cases where you want to override a part of a
> package are so rare that we do not need to handle them explicitly;
> rather, let the user do that manually, like this:
> 
> settrans /packages/emacs/default \
>        /hurd/unionfs /some/path/override-emacs /packages/emacs/21.2
> 
> Or can you give me an example of a situation where it is useful to
> have a _package_ actually override a part of another package?
> 
> > It might be usefull to write down all features of packagefs, how
> > they will be implemented with packagefs and how they will be
> > implemented with packagefs+unionfs. Also we have to create a list
> > with all advantages and disadvantages.
> 
> Yes, this is a very good idea, although I would expect the difference
> to be rather small.
> 
> > Personally I don't like to lose flexibility because of development
> > speed, although it would be stupid if we choose the difficult
> > solution if there is no gain :).
> 
> I completely agree.  I tried to come up with a solution that gives us
> reasonable results fast, but does not make it hard (or even
> impossible) to switch to the ideal solution later.  Since I have not
> seen any indication for why my approach would not get us there, I'm
> tempted to think that my approach is a good one, but that's just
> me. :-]
> 
> > I think we should talk more about the design. Not because I disagree
> > (which I don't), but to catch problems before writing it and to come
> > up with a better design.
> 
> Sure, so be it.  In the following paragraphs, I assume UnionFS and
> PackageFS to be seperate servers; most of what I say is also relevant
> for the other case, of course.
> 
> It is clear that we need some kinf of backend store (and it doesn't
> even matter whether that is an ext2 file system or the combination
> Robert proposed).  Which is to say, we will have the raw packages in
> /packages, which might look this way:
> 
> /packages/
>   emacs/
>     20.7/
>     21.2/
>     default -> 21.2
>   libguile12/
>     1.6.0/
>     default -> 1.6.0
>   local/
>     1.0
>     default -> 1.0
> 
> I will say a few words about why I propose the `default' symlink below.
> 
> However, we also need a place where we configure which packages are
> visible and which packages aren't.  The location is arbitrary, thus I
> will just assume /active-packages, knowing very well that it is a
> stupid place. :-)
> 
> This might look as follows:
> 
> /active-packages/
>   problems
>   list/
>     emacs -> /packages/emacs/default
>     libguile12 -> /packages/libguile12/default
>     local -> /packages/local/default
> 
> The node `problems' (which you suggested to turn into a directory; I
> would like to know why) would only exist if there is a PackageFS on
> /active-packages.  If that is the case, the attempt to create a link
> in list/ that points to a package which has a problem (e.g. it
> conflicts with a package that is already linked there) would cause the
> creation of an entry in `problems' that described the problem.  If we
> do not have PackageFS yet, the frontends have to ensure that
> everything is alright, as I said earlier.
> 
> I think that the indirection through the `default' symlink is a good
> idea because everyone who has used Unix will realize what it is about
> immediatelly when he sees it (which is to say, it is trivial to
> understand), and thus the only disadvantage is that it breaks Bash's
> tab-completion when there is only one package.  On the other side, it
> makes some things simpler for advanced users.  I want to give an
> example for this:
> 
> A user might want to have multiple systems, which share most packages,
> but differ in a few important ones, like glibc.  I don't think this is
> even an uncommon situation, especially if I think about our past glibc
> upgrades ;-).  This user could create /active-packages-2/ that differs
> from the first one only in the links to these packages.  By telling
> UnionFS to unite either /active-packages/list/* or
> /active-packages-2/list/*, the user can switch between these two
> systems.  If the links in `list' point to the default versions of the
> packages, the upgrade of a package (and thus modification of the
> default symlink) would make the new version available for both the
> system in /active-packages and the one in /active-packages-2, where
> otherwise it would probably cause a broken link in one of them, if the
> old version gets removed.  I am sure that there are various other
> situations where the default symlink is useful.
> 
> Also, you might have noticed the package called `local' in my example.
> The purpose of that is to provide the place where new files will be
> added, making it a replacement for /local, which then could be a
> symlink to the root, without loosing the advantage of having a place
> that does not get touched by the package management.  Two obvious
> facts about it: First, it should always stay at version 1.0; second,
> the user can decide to switch between different `local's, so that you
> can install a new version of a non-packaged (and non-autoconfified)
> program for testing in its own `local', and depending on whether it
> works install it in your usual `local' or not (sure, the same thing is
> possible on Unix, but not as convenient).
> 
> If you think I'm writing bullshit here, just tell me; I never shared
> these ideas with anyone before, so they might be broken.
> 
> > We should also discuss kitten. I hope kitten can be written by a
> > newcomer. We need developers, such simple translator might make
> > someone enthousiastic about the Hurd :).
> 
> Personally, I think it cannot be done by a usual ``newcomer''.  To
> implement it correctly, a newcomer would need to be a very capable C
> programmer that is able to already focus on the actual functionality
> instead of the framework even if she never used the framework before.
> I don't think I would have been able to write kitten as a newcomer.
> 
> > I spoke with ^SilenT^ about this, he told me he thought about doing
> > this. I discussed writing support for kitten with Wolfgang. I came
> > up with the idea to use regular expressions, that might be a little
> > difficult (or is there a libregexp?). Wolfgang also had some smart
> > ideas, I hope Wolfgang wants to describe them :).
> 
> We have support for regular expressions in glibc, as POSIX defines a C
> API for accessing them.  However, I thought about this and think that
> the patterns we use are so simple that it does not gain anything to
> use regular expressions; as far as I can tell, it will always be of
> the form
> 
>   ^maybe some text (.+) maybe other text$
> 
> where \1 would be the source/target file name.  Or are there patterns
> possible which differ from that?  If not, we could just use something
> like
> 
> settrans foo /hurd/kitten --scan-name '<file>'
>                         --pattern '# BEGIN <file>'
> 
> which is easier to understand not only for those who never used
> regular expressions, but even for those who use them every day.
> 
> Cheers,
> GNU/Wolfgang
>