[Savannah-hackers-public] Re: Setting up Bzr hosting for Savannah

[Sylvain Beucler]

(This is turning into a technical discussion, so let's switch to
sv-hack-pub and involve other interested people, namely Aleix. If
people feel like avoiding further CC:s, please tell :))

On Thu, Jun 05, 2008 at 03:46:28PM +1200, Tim Penhey wrote:
> On Wednesday 04 June 2008 19:51:02 Sylvain Beucler wrote:
> > Hi,
> > 
> > On Tue, Jun 03, 2008 at 10:59:20AM +1200, Tim Penhey wrote:
> > > On Monday 02 June 2008 11:29:46 Sylvain Beucler wrote:
> > > > Hi,
> > > > 
> > > > I was asked a few times what was needed to support Bzr at Savannah. I
> > > > specified that Savane integration was the primary point (the sysadmin
> > > > part is essentially already done).
> > > > 
> > > > Your suggestion is to partially recreate the sysadmin part of Savannah
> > > > (SSH access, etc.) and avoid work on the Savane codebase.
> > > > 
> > > > Since I don't want to spent time helping recreate Savannah, and since
> > > > we can't find anybody for the integration, I'll just write the
> > > > bzr/Savannah integration instead. I can spend some time on it atm and
> > > > it should be ready within a week or two. (Incidentally it's not this
> > > > hard, somebody recently added Mercurial integration.)
> > > 
> > > Sylvain,
> > > 
> > > If there is something specific that I can help with, please get in touch.
> > 
> > Currently there's a couple tasks that you could work on at Savannah.
> > 
> > 
> > Firstly, loggerhead doesn't seem easy to setup for a lot of
> > repositories. We'd be glad to know what you would recommend (what
> > version to use, whether to let it detect repositories or specify them
> > explicitely in the configuration, run it through mod_python or through
> > mod_proxy...). We also wonder whether it supports the following
> > layout:
> > 
> > /srv/bzr/myproject/.bzr/
> > /srv/bzr/myproject/repo1/.bzr/
> > /srv/bzr/myproject/repo2/.bzr/
> > 
> > - that is, with a main repository and subrepositories. I think it
> > better supports subrepositories only.
> 
> I talked with Michael Hudson (resident Loggerhead expert, and CC'ed)
> about this, and the short answer is for any non-trivial loggerhead install,
> you will want to have some custom initial navigation handling within
> loggerhead itself.  This isn't very difficult to write, and we can help with
> that.  What we do need though is a definitive directory / repository layout
> in order to discover the branches.

Hmm, maybe it will be easier to regenerate loggerhead.conf
periodically. This also gives an opportunity to specify
"name"/"description"/"url" for each branch (unless that can be
specified otherwise).

Other problems that I noticed:

- how to easily run it in Apache?

  I currently quickly tried running it as an independent server behind
  mod_proxy. I didn't manage to run it in a directory (such as
  bzr.sv.gnu.org/lh/) instead of the '/' (bzr.sv.gnu.org/)

  Maybe it's easier to run it through mod_python or something.

- version: is it good to use the latest (and old) release, or is they
  some branch or trunk version that is considered better? We'd better
  use a non-patched version to ease the maintenance.

> > Secondly, there's interest in supporting several repositories per
> > directory, with alternate permission models (currently we setup shared
> > repositories, people would appreciate other permissions to allow
> > non-member to contribute, which would rely on ACLs rather than Unix
> > groups). So there's a need for a new web interface in the Savane code
> > to instanciate and configure subrepositories. Since projects are
> > carefully validated at Savannah, non-official (and so non-validated)
> > repositories probably require special display (for example, put them
> > in a "forks" subdirectory and mark them as such in the web interface).
> > More at http://savannah.gnu.org/maintenance/SavaneTasks (there's an
> > entry for this).
> 
> What exactly do you mean with "several repositories per directory"?

Here's the suggested layout:
/srv/bzr/myproject1/.bzr/
/srv/bzr/myproject1/repo1/.bzr/
/srv/bzr/myproject1/repo2/.bzr/
/srv/bzr/myproject1/forks/repo1/.bzr/
/srv/bzr/myproject1/forks/repo2/.bzr/
/srv/bzr/myproject2/.bzr/
...

> It is not possible to enforce your own ACL on top of a Bazaar repository
> right now.  I was talking with Aaron Bentley earlier today about this, and
> he said "Until all operations are possible over the smart server protocol,
> it isn't possible to enforce an arbitrary ACL".  This is due to Bazaar falling
> back to a virtual file system over the smart protocol for operations that
> are not implemented yet.
> 
> There is current work in Bazaar on stacked branches.  This is where a branch
> has its own repository, but also has another branch that it is "stacked upon".
> This way the branch has only the revisions in its repository that are not in
> the stacked upon branch.  This gives the flexability of having efficient 
> storage
> of branches without having to have write access to the stacked upon branch's
> repository.

I was thinking on filesystem-level ACLs (setfacl...), but a simplified
version with either shared or personal branches could be fine in a
first step.

> It seems to me that you would want something like a shared repository for
> the official team for any given project, and allow other users to create their
> own branches from that repository, still be able to host those branches on
> Savannah but to have these branches stored in a different repository to the
> official team's repository.

Yes. This is the 'forks' repository in the layout (I don't really like
the term, maybe 'unofficial' or 'unoff' would be better - suggestions?).

> One question could be "is it sufficient for there to be one official shared
> repository, and one unofficial repository?" or do you feel that each user who
> is not in the official team for the project be able to have their own 
> repository?

AFAICS users wish to have several repositories, so to the least,
people need several official repositories.

Since most projects only have 1 repository/branch, and for simplicity,
there could be 1 official main repository and several official
subrepositories, as shown in the layout above. If there's only 1 repo
per non-member, that's acceptable IMHO.

Aleix started a reflexion on how to specify repositories in the Savane
database at savannah-hackers-public. This should provide the general
idea for the Savane integration.

> What access to these repositories are you offering (http, bzr, bzr+ssh, sftp, 
> ftp)?

bzr+ssh (write), bzr-hpss (anon), and http (anon through firewalls).

-- 
Sylvain