Re: Guix on clusters and in HPC

[Ludovic Courtès]

Hi!

Eric Bavier <address@hidden> skribis:

>>   - non-root usage
>
> The Singularity project advertises that it does not use a root-owned
> daemon http://singularity.lbl.gov/about#no-root-owned-daemon-processes
> but it does not in the same section explain that it uses a setuid
> helper instead: http://singularity.lbl.gov/docs-security which also 
> summarizes some of the current limitations and trade-offs of user namespaces.

Interesting, thanks for the pointers, especially the second one which
moderates the claim of the first one.

Do you how widely Singularity is being deployed?

The build daemon used to have a small setuid helper that people could
use instead of running the whole daemon as root; it was removed in Nix
and in commit d43eb499a6c112af609118803c6cd33fbcedfa43 on our side.

The reason for the removal was that nobody was using it, and that it was
presumably unhelpful in overcoming the “non-root” use case.

I feel like it may be easier to get user namespaces enabled than to get
a setuid helper installed.  WDYT?

>>   - central daemon usage (like at MDC, but improved)
>
> For many-user systems, I think we'd need to put in place some controls
> to keep users from stepping on each others feet when it comes to interacting
> with the deamon.  E.g. One user spends a bunch of time building her
> application; before she gets a chance to use it, another user comes along
> and runs 'guix gc'.

That’s not a problem: packages in a profile are protected from GC, and
profiles generated by ‘guix environment’ are also protected for the
duration of the session.

With ‘guix build’, one has to use ‘-r’ to make sure the package won’t be
GC’d as soon as ‘guix build’ completes.

> Can a user run 'nice 10 guix build ...' and have it DTRT?

No it won’t DTRT.

> On existing systems, the root partition may not be as large as Guix might
> like and there may not be opporunities to mount a separate partition for the
> store.  While it's nice that Guix would give users the change to share
> package build results, often disk partitions are weighted in favor of /home
> (possibly because of the current widespread practice of users building their
> own packages in $HOME).  Until that changes, sysadmins might like some more
> powerful tools for auditing store disk usage to answer questions such as
> "which user profiles are exclusively occupying the most store space?" or even
> some way to specify expiration dates for certain profiles.

I see what you mean, though it’s again a “cultural” thing.  I can see
that the shared store would effectively allow sysadmins to save space,
but yeah.

>>     + admin/social issues
>>       * daemon runs as root
>
> So, Singularity uses a setuid helper, and Shifter needs to run the Docker
> daemon.  It may be easier to convince sysadmins to run Guix's daemon
> given those other examples.  Of course, if we can do what we need to
> with even fewer privileges, that'd be great.

Good to know!

>>       * daemon needs Internet access
>
> There are many HPC sites that are air-gapped for security reasons.  Of those
> sites that I know, the ones that allow any outside software to be put on the
> machine after the initial system installation require CD/DVD installation 
> media.
> IMO for such sites, and for other users wishing to take Guix off the grid, it
> would be nice to be able to prepopulate the installation media, whether USB or
> CD, with more package outputs and/or source (e.g. like Trisquel's "Sources 
> DVD").
> Or similarly a way to "mount" some media that contains a bunch of package
> definitions for GUIX_PACKAGE_PATH as well as the corresponding source or 
> output
> for a specific Guix release.

Probably ‘guix build --sources=transitive’ and similar tools can help
here?  Then we can populate a store on a DVD or something and import it
on the machine.

>>   - package variants, experimentation
>>     + for experiments, as in Section 4.2 of
>>     [[https://hal.inria.fr/hal-01161771/en][the RepPar paper]]
>>       * in the meantime we added
>>       
>> [[https://www.gnu.org/software/guix/manual/html_node/Package-Transformation-Options.html][--with-input
>>       et al.]]; need more?
>>     + for
>>     
>> [[https://lists.gnu.org/archive/html/guix-devel/2016-10/msg00005.html][CPU-specific
>>     optimizations]]
>>     + somehow support -mtune=native (and even profile-guided
>>       optimizations?)
>>     + simplify the API to switch compilers, libcs, etc.
>
> +1 for all these
>
> Even though we intend to not specifically support proprietary compilers, some
> users may still want to explore building their packages with other compilers,
> like e.g. Clang and Rose

Yup.

>>   - workflow, reproducible science
>>     + implement
>>     [[http://debbugs.gnu.org/cgi/bugreport.cgi?bug=22629][channels]]
>
> Perhaps what I discussed above re installation media could fold into this.

I think it’s orthogonal.

Thanks a lot for your feedback!

Ludo’.