Re: updating many haskell packages

[Troy Sankey]

Quoting Federico Beffa (2017-02-17 03:00:04)
> On Wed, Feb 15, 2017 at 8:31 PM, Troy Sankey <address@hidden> wrote:
> > Quoting Troy Sankey (2017-02-06 16:00:29)
> >> Quoting Federico Beffa (2017-02-06 15:34:47)
> >> > I would consider a discrepancy between a cabal file on Hackage and the
> >> > actual cabal file included in a tar archive a bug.  It may be helpful
> >> > to report it to the author.
> >>
> >> I found this issue on the hackage github:
> >>
> >> https://github.com/haskell/hackage-server/issues/503
> >>
> >> It looks like it's a feature of hackage and/or cabal to allow multiple
> >> revisions of cabal files per tarball release.  IIUC, the `cabal get`
> >> utility takes care of automatically updating the cabal file to the
> >> latest revision.
> >>
> >> I'll have to look more into this later.
> >
> > Indeed the .cabal files can be revised between releases.  Somehow the
> > new revisions get pushed to the all-cabal-files repository [0] with an
> > additional "x-revision" key [1], and Hackage picks up the new cabal
> > file.  The `cabal` and `stack` utilities substitute the .cabal file from
> > the release tarball with the revised .cabal file from Hackage if it is
> > newer.
> >
> > In theory, anything in the .cabal file could be modified, not just
> > dependency bounds.  I think guix should use the updated .cabal files for
> > better consistency with the rest of the haskell ecosystem, and also for
> > less per-package maintenance cost.
> >
> > How can guix adapt?  Should we somehow incorporate the all-cabal-files
> > repository in the haskell build system, adding a post-unpack phase to
> > substitute the entire .cabal file?
> 
> Packages are build in isolated environments without network.
> Therefore it's not possible to access Hackage with a phase of the
> build system.  Even if it were possible it would be undesirable
> because it would make things non reproducible.
> 
> Files not included in the tar can be added as origin inputs to a
> package definition (see the testsuite for GHC in the ghc package).
> However, they are verified with hashes. Any change to those files will
> break the package.
> 
> Fede

Forgive me if my understanding of build systems in Guix is flawed, but
let me explain my idea with more detail:

First, make a data-only package called "ghc-all-cabal-files" containing
the checkout of a specific commit of the all-cabal-files repository from
github.  We can periodically update this package, but there is no
traditional "release"---we just keep pulling the HEAD of the hackage
branch.  This package would then act as a helper package for the haskell
build system---every haskell package should implicitly use this package
as input.  Then we can write a post-unpack phase for the
haskell-build-system which updates the unpacked .cabal file iff it finds
a newer .cabal file in ghc-all-cabal-files (we know how to determine if
the cabal file is newer: it will have a higher "x-revision" value, or
that key will merely exist).

One problem I have not fully solved is the technical debt associated
with keeping the proposed ghc-all-cabal-files package up-to-date.  I
believe updating it would require all haskell packages to be rebuilt.
We could create a build system argument called use-newest-cabal-file to
toggle the feature, in which case we would only switch it to #t if we
already know the .cabal file to be stale.  Then only a small subset of
packages would need to be rebuilt, and there is less technical debt than
the current solution which involves monkey-patching every cabal file
that needs it.

Troy

signature.asc
Description: signature