Re: Store channel specification in profile

[Ludovic Courtès]

Howdy!

Turns out I just watched your talk (since I couldn’t be there at the
time!).  I was surprised you didn’t explicitly mention ‘guix search’ and
the shortcomings you’d like to address (well, not directly), but I liked
the perspectives!

  https://video.fosdem.org/2020/AW1.125/gnuguixpackagemanager.webm

Pierre Neidhardt <address@hidden> skribis:

>> One thing to keep in mind, though, is that if the ‘specifications.scm’
>> is part of the profile, it must be future-proof.  That is, the APIs it
>> uses must essentially be guaranteed to remain forever.  That’s a very
>> strong constraint.
>
> I think that's OK.  In the example format I suggested, we use keys, which
> make it easy to extend the format later on, if need be.

I feel there a Common Lisp vs. Scheme cultural clash here.  ;-)

The way I see it, a public procedure should take clearly defined,
structured objects, and of only a single type (that’s a pattern mostly
followed throughout Guix).

So, if I were to implement the proposed ‘specifications->manifest*’, it
would take a record to describe the package/version/channel.  But then
that record wouldn’t be different from a <manifest-entry> referencing an
<inferior-package>.  IOW: the API already exists, mostly.

I would not use sexps (like the one you propose) as substitutes for
records in an API.  I know that’s the traditional Lisp way to do it, but
it’s IMO too clumsy, error-prone (unless you use a pattern matcher, but
even then it’s not enough), inefficient, etc.

I would, however, use sexps as a serialization format.  Compared to an
API, an object serialized to an sexp has the advantage that we can write
code to handle changes in the serialization format, so it’s future-proof
if we get it right.  But then: we’re back to ‘manifest’.  :-)

I hope this is a bit clearer, but I realize it’s tricky to discuss such
things!

>> In contrast, versioned data formats like the famous ‘manifest’ file
>> don’t have this problem at all, but they’re less directly usable from
>> the CLI.
>
> We can version the specifications.scm as well.  Can you explain what the
> problem is, I think I'm missing the point.

‘manifest’ looks like this:

  (manifest
    (version 3)
    …)

We have an explicit ‘read-manifest’ procedure that can handle version 3,
but also prior versions, and this is all transparent.

You cannot do that with code.  Code is just evaluated, and if it’s
incompatible, if fails in some unspecified way.

>> We also need to distinguish between APIs, which should use first-class
>> objects (<channel>, etc.), and data formats, which are plain sexps.
>> (The above example is a mixture of both and in fact looks very similar
>> to what’s already in the ‘manifest’ file.)  But then again, that means
>> relying on a larger chunk of the API.
>
> I'm not sure I understood what you meant here.  Which APIs?

‘specification->package’ is part of the API.

>> So, all in all, I think I’d rather see it implemented as ‘guix package
>> --export’ or similar.
>
> I think I didn't understand why you'd prefer to have a command instead
> of systematically generating the file inside the profile.  Or maybe we
> could generate the file somewhere else if that's the problem?
>
> In my opinion, the `--export` approach would be cumbersome because it
> means that for users who want to save the specifications file, they'd
> need to systematically call it on every profile-modifying command, e.g.
>
>   guix package -m manifest.scm && guix package --export

I agree that ‘--export’ is less convenient.  Note that ‘guix system
reconfigure’ does exactly what you have in mind: it stores a
‘channels.scm’ and a ‘config.scm’ file in the system (in addition to
serialized & versioned metadata in the ‘provenance’ file) because that’s
so convenient:

  guix time-machine -C /run/current-system/channels.scm --
    system build --save-provenance -C /run/current-system/configuration.scm

But in this case it’s OK: ‘channels.scm’ uses a tiny teeny subset of the
API, and ‘configuration.scm’ is evaluated in the right context where the
APIs it expects are available.

Does that make sense?

We cannot do the same thing with profiles because of the possibly
multiple provenances.

>> As far as faithfulness is concerned, we should also keep in mind that we
>> don’t always have all the information needed to reconstruct what’s in a
>> profile.  For example, we lack information about the package
>> transformation options that were used (if any),
>
> Like `--with-input'?
> I didn't think of this.  Can't we extend the format then to include all
> transformation options, with future-proof provisions?

We could store package transformations as manifest entry properties.

However, that’ll be an approximation: the exact implementation of
‘--with-input’, for instance, can vary over time.

>> and we lack information about arbitrary user code that might have been
>> used to generate manifest entries.
>
> Is this a problem?

All I’m saying is that we can only approximate all these things.
Because of that, it may make more sense to not over-engineer the thing
and focus on making a rough approximation.

After all, the goal of the functionality we’re discussing is to allow
users to move towards the declarative ‘manifest.scm’ style, right?

Thanks,
Ludo’.