Re: [PATCH] profiles: Generate database file for manpages

[Maxim Cournoyer]

Hi Ludovic,

address@hidden (Ludovic Courtès) writes:

> Hi Maxim,
>
> Maxim Cournoyer <address@hidden> skribis:
>
>> I've managed to finalize a profile hooks which takes care of generating
>> the manpages database file. This file is used by apropos or man -k to
>> provide results.
>
> Awesome!  I’ve been willing to have this for some time.
>

Thanks for providing feedback on this so quickly!

>> I've tested and found it to be working for the regular user profile. To
>> try it, apply the patch and use a guix command that causes a new profile
>> generation.
>>
>> For example: guix package -r git -i git
>> Then: "apropos git" should return plenty of results.
>
> For some reason I have to use “man -K” (capital):
>
> ~$ apropos guile
> guile: nothing appropriate.
> ~$ man -k guile
> guile: nothing appropriate.
> ~$ man -K guile
> --Man-- next: skribilo(1) [ view (return) | skip (Ctrl-D) | quit (Ctrl-C) ]
> --Man-- next: guile(1) [ view (return) | skip (Ctrl-D) | quit (Ctrl-C) ]
> --Man-- next: guix-pull(1) [ view (return) | skip (Ctrl-D) | quit (Ctrl-C) ]
> --Man-- next: guix-package(1) [ view (return) | skip (Ctrl-D) | quit (Ctrl-C) 
> ]
> --Man-- next: gv(3guile) [ view (return) | skip (Ctrl-D) | quit (Ctrl-C) ]
> ~$ env | grep MANPATH
> MANPATH=/home/ludo/.guix-profile/share/man:/run/current-system/profile/share/man:/home/ludo/.guix-profile/share/man:/run/current-system/profile/share/man:/home/ludo/.opam/system/man
>
>
> Any idea why?
>

Unfortunately some of the simplifications you made appear to break it
:). I've tried your reworked patch and got the same result as you (no
result found except when using man -K). When I use the original patch it
works as I originally described.

>> It requires extra processing time to do the work since the complete
>> database (on my system, there are thousands or manpages being indexed)
>> is recreated everytime a new profile is generated.
>
> On my laptop (with an SSD), I found it to be reasonable:
>
> $ time guix build 
> /gnu/store/rkri628apz2a2i2jvav11ylv2736fvv3-manual-database.drv --check
> @ build-started 
> /gnu/store/rkri628apz2a2i2jvav11ylv2736fvv3-manual-database.drv - 
> x86_64-linux 
> /var/log/guix/drvs/rk//ri628apz2a2i2jvav11ylv2736fvv3-manual-database.drv.bz2
> /gnu/store/g7qxdk1cn7fwkamkaisi4428k6ijg0w2-manual-database
>
> real  0m2.647s
> user  0m0.116s
> sys   0m0.004s
> $ guix package -I |wc -l
> 229
>
>
>> For now it doesn't work for guix environments (yet), but this should be
>> easy to fix (it seems the $GUIX_ENVIRONMENT/share/man should be merged
>> with $HOME/.guix-profile/share/man, but I need to look at it more
>> carefully).
>
> It seems to work but only with capital -K:
>
> $ ./pre-inst-env guix environment --ad-hoc guile man-db --pure -- man -K guile
> man: can't execute cat: No such file or directory
> man: command exited with status 255: sed -e '/^[[:space:]]*$/{ N; 
> /^[[:space:]]*\n[[:space:]]*$/D; }' | (cd <fd 3> && LESS=-ix8RmPm Manual page 
> guile(1) ?ltline %lt?L/%L.:byte %bB?s/%s..?e (END):?pB %pB\%.. (press h for 
> help or q to quit)$PM Manual page guile(1) ?ltline %lt?L/%L.:byte 
> %bB?s/%s..?e (END):?pB %pB\%.. (press h for help or q to quit)$ 
> MAN_PN=guile(1) cat)
>
> (And we should hard-code the file name of ‘cat’ in ‘man’…)
>
>>
>> The mandb database file (index.db) is used by the "apropos" (whatis) or
>> "man -k" commands. This change introduces a profile hook to generate
>> such database file.
>>
>> Co-authored by Ludovic Courtès
>>
>> * guix/profiles.scm (manual-database): New procedure.
>> (%default-profile-hooks): Add it.
>
> I made some simplifications to the patch.  WDYT?
>

Looks good, except for the 2 problematic changes in the reworked patch
identified below.

> Thanks,
> Ludo’.
>
> diff --git a/guix/profiles.scm b/guix/profiles.scm
> index 795c9447f..f565f358b 100644
> --- a/guix/profiles.scm
> +++ b/guix/profiles.scm
> @@ -7,6 +7,7 @@
>  ;;; Copyright © 2016 Ricardo Wurmus <address@hidden>
>  ;;; Copyright © 2016 Chris Marusich <address@hidden>
>  ;;; Copyright © 2017 Huang Ying <address@hidden>
> +;;; Copyright © 2017 Maxim Cournoyer <address@hidden>
>  ;;;
>  ;;; This file is part of GNU Guix.
>  ;;;
> @@ -946,10 +947,73 @@ files for the fonts of the @var{manifest} entries."
>                      #:local-build? #t
>                      #:substitutable? #f))
>  
> +(define (manual-database manifest)
> +  (define man-db                                  ;lazy reference
> +    (module-ref (resolve-interface '(gnu packages man)) 'man-db))
> +
> +  (define build
> +    #~(begin
> +        (use-modules (guix build utils)
> +                     (srfi srfi-1))
> +
> +        (define entries
> +          (filter-map (lambda (directory)
> +                        (let ((man (string-append directory "/share/man")))
> +                          (and (directory-exists? man)
> +                               man)))
> +                      '#$(manifest-inputs manifest)))
> +
> +        (define manpages-collection-dir
> +          (string-append (getenv "PWD") "/manpages-collection"))
> +
> +        (define man-directory
> +          (string-append #$output "/share/man"))
> +
> +        (define (populate-manpages-collection-dir entries)
> +          (let ((manpages (append-map find-files entries)))
> +            (for-each (lambda (manpage)
> +                        (let* ((dest-file (string-append
> +                                           manpages-collection-dir "/"
> +                                           (strip-store-file-name manpage))))

Unfortunately we cannot simply use strip-store-file-name, since what
mandb expects to find in our manpages-collection-dir is a collection of
subdirectories such as man1, man3, man7, etc. such as would be found
under "/usr/share/man" on a traditional distro. This change instead
makes it look like:

bazaar-2.7.0/          gdbm-1.12/              libxau-1.0.8/  ...

I guess this is what broke it.

> +                          (mkdir-p (dirname dest-file))
> +                          (catch 'system-error
> +                            (lambda ()
> +                              (symlink manpage dest-file))
> +                            (lambda args
> +                              ;; Different packages may contain the same
> +                              ;; manpage.  Simply ignore the symlink error.
> +                              #t))))
> +                      manpages)))
> +
> +        (mkdir-p manpages-collection-dir)
> +        (populate-manpages-collection-dir entries)
> +
> +        ;; Create a mandb config file which contains a custom made
> +        ;; manpath. The associated catpath is the location where the database
> +        ;; gets generated.
> +        (copy-file #+(file-append man-db "/etc/man_db.conf")
> +                   "man_db.conf")
> +
> +        (substitute* "man_db.conf"
> +          (("MANDB_MAP\\>.*$")
> +           (string-append "MANDB_MAP " manpages-collection-dir " "
> +                          man-directory "\n")))

This change here will replace every MANDB_MAP entries with duplicated
information:

MANDB_MAP /tmp/guix-build-manual-database.drv-0/manpages-collection 
/gnu/store/6w3gq10pgi8qmyq9i989glxs8md4miyk-manual-database/share/man
MANDB_MAP /tmp/guix-build-manual-database.drv-0/manpages-collection 
/gnu/store/6w3gq10pgi8qmyq9i989glxs8md4miyk-manual-database/share/man
MANDB_MAP /tmp/guix-build-manual-database.drv-0/manpages-collection 
/gnu/store/6w3gq10pgi8qmyq9i989glxs8md4miyk-manual-database/share/man
MANDB_MAP /tmp/guix-build-manual-database.drv-0/manpages-collection 
/gnu/store/6w3gq10pgi8qmyq9i989glxs8md4miyk-manual-database/share/man
MANDB_MAP /tmp/guix-build-manual-database.drv-0/manpages-collection 
/gnu/store/6w3gq10pgi8qmyq9i989glxs8md4miyk-manual-database/share/man
MANDB_MAP /tmp/guix-build-manual-database.drv-0/manpages-collection
/gnu/store/6w3gq10pgi8qmyq9i989glxs8md4miyk-manual-database/share/man

I don't think this is desirable. At first I was removing
all the MANDB_MAP entries and then adding only our, but there's nothing
to gain from doing this since the original man_db.conf file (bundled
with mandb) is still somehow read and contains those entries anyway
(which are harmless).

> +
> +        (mkdir-p man-directory)
> +        (setenv "MANPATH" (string-join entries ":"))
> +        (zero? (system* #+(file-append man-db "/bin/mandb")
> +                        "--quiet" "--create"
> +                        "-C" "man_db.conf"))))
> +
> +  (gexp->derivation "manual-database" build
> +                    #:modules '((guix build utils))
> +                    #:local-build? #t))
> +
>  (define %default-profile-hooks
>    ;; This is the list of derivation-returning procedures that are called by
>    ;; default when making a non-empty profile.
>    (list info-dir-file
> +        manual-database
>          fonts-dir-file
>          ghc-package-cache-file
>          ca-certificate-bundle


I have reverted these two problematic changes and attached a new patch. Please
try it out and let me know how it works :)

Thanks,

Maxim

0001-profiles-Generate-database-file-for-manpages.patch
Description: Text Data

signature.asc
Description: PGP signature