guix-devel
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Preparing for the libc/locale upgrade


From: Ludovic Courtès
Subject: Preparing for the libc/locale upgrade
Date: Thu, 24 Sep 2015 21:25:30 +0200
User-agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux)

Problem: Programs linked against libc < 2.22 abort when trying to load
libc 2.22-or-later locale data:

  https://lists.gnu.org/archive/html/guix-devel/2015-08/msg00737.html

Programs linked against libc 2.22 do not abort when trying to load 2.21
locale data, but they fail to load data for the LC_COLLATE category (in
which case ‘setlocale’ returns ENOENT and the program prints a warning)
due to the addition of the _NL_COLLATE_ENCODING_TYPE element.

In all situations, as a last resort, one can always run:

  export LC_ALL=C

to sidestep the incompatibility issues.

Since ‘core-updates’ will soon be merged, bringing libc 2.22, here’s a
preview of the problems you may encounter and possible solutions.

Consequences on GuixSD:

  • If programs in user profile use libc 2.22 but
    /run/current-system/locale provides libc 2.21 locales: no hard
    problem, but the LC_COLLATE problem.

  • If LOCPATH points to 2.22 locales but /run/current-system/profile
    contains 2.21 programs (Coreutils, sed, grep, etc.), these will
    abort.  Solution: unset LOCPATH so that programs use the 2.21 locale
    data from /run/current-system/locale.

  • If /run/current-system is upgraded to 2.22 but user profiles contain
    2.21 programs, these will abort.  Solution: upgrade user profiles,
    or install glibc-utf8-locale-2.21 in user profile (preferably before
    upgrading GuixSD to libc 2.22.)

Consequences for Guix on foreign distros:

  • If the host distro provides binaries that use libc < 2.22 and you
    use a mixture of Guix-provided and distro-provided programs, this is
    pretty bad.

    Solution: unset LOCPATH and say goodbye to locales for Guix-provided
    packages (setting LOCPATH=$HOME/.guix-profile/lib/locale would break
    all the distro-provided programs), or use exclusively Guix-provided
    programs, or use the “C” locale.

Lesson learned:

  • On GuixSD, we should use a versioned directory,
    /run/current-system/locale/X.Y, as the default locale directory,
    which would avoid problems.

  • The attached patches allow GuixSD users to choose a different libc
    than the current one to build the locales that go to
    /run/current-system/locale.  However this isn’t really satisfying.

Comments welcome!

Ludo’.

>From beb9975ed28e12b78f7043d76f2895c97cd7e4a8 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <address@hidden>
Date: Thu, 24 Sep 2015 21:03:45 +0200
Subject: [PATCH 2/2] system: locale: Add a 'glibc-for-locales' knob.

* gnu/system.scm (<operating-system>)[glibc-for-locales]: New field.
  (operating-system-locale-directory): Honor it.
* gnu/system/locale.scm (sanity-check): New procedure.
  (locale-directory): Use it.
* doc/guix.texi (Locales)[Locale Data Compatibility Considerations]: New
  section.
---
 doc/guix.texi         | 42 +++++++++++++++++++++++
 gnu/system.scm        |  5 ++-
 gnu/system/locale.scm | 95 +++++++++++++++++++++++++++++++++------------------
 3 files changed, 108 insertions(+), 34 deletions(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index 07c5add..6c0adda 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -5158,6 +5158,10 @@ Library Reference Manual}).  @xref{Locales}, for more 
information.
 The list of locale definitions to be compiled and that may be used at
 run time.  @xref{Locales}.
 
address@hidden @code{glibc-for-locales} (default: @var{glibc-2.21})
+The address@hidden package whose locale data and tools are used to build
+the locale definitions.  @xref{Locales}.
+
 @item @code{name-service-switch} (default: @var{%default-nss})
 Configuration of libc's name service switch (NSS)---a
 @code{<name-service-switch>} object.  @xref{Name Service Switch}, for
@@ -5615,6 +5619,44 @@ instance it has @code{uk_UA.utf8} but @emph{not}, say,
 @code{uk_UA.UTF-8}.
 @end defvr
 
address@hidden Locale Data Compatibility Considerations
+
address@hidden incompatibility, of locale data
address@hidden declarations provide a @code{glibc-for-locales}
+field to specify the address@hidden package that is used to compile
+locale declarations (@pxref{operating-system Reference}).  ``Why would I
+care?'', you may ask.  Well, it turns out that the binary format of
+locale data may be incompatible from one libc version to another.
+
address@hidden See <https://sourceware.org/ml/libc-alpha/2015-09/msg00575.html>
address@hidden and 
<https://lists.gnu.org/archive/html/guix-devel/2015-08/msg00737.html>.
+For instance, a program linked against libc version 2.21 is unable to
+read locale data produced with libc 2.22; worse, that program
address@hidden instead of simply ignoring the incompatible locale
address@hidden 2.23 and later of address@hidden will simply skip
+the incompatible locale data, which is already an improvement.}.  On the
+other hand, a program linked against libc 2.22 can read locale data from
+libc 2.21, except for @code{LC_COLLATE} data; thus calls to
address@hidden may fail, but programs will not abort.
+
+The ``problem'' in GuixSD is that users have a lot of freedom: They can
+choose whether and when to upgrade software in their profiles, and might
+be using a libc version different from the one the system administrator
+used to build the system-wide locale data.
+
+Fortunately, users can also install their own locale data and define
address@hidden accordingly (@pxref{locales-and-locpath, @code{LOCPATH}
+and locale packages}).
+
+Still, it is best if the system-wide locale data at
address@hidden/run/current-system/locale} is built using an older libc.  That
+way, both programs linked against the new libc and programs linked
+against the older one can load locale data.  For this reason, the
address@hidden field defaults to a libc version slightly older
+than the current one, with the caveat that some locale categories may be
+incompatible.
+
+
 @node Services
 @subsection Services
 
diff --git a/gnu/system.scm b/gnu/system.scm
index cee5f37..8b5f981 100644
--- a/gnu/system.scm
+++ b/gnu/system.scm
@@ -146,6 +146,8 @@
             (default "en_US.utf8"))
   (locale-definitions operating-system-locale-definitions ; list of 
<locale-definition>
                       (default %default-locale-definitions))
+  (glibc-for-locales operating-system-glibc-for-locales ; <package>
+                     (default glibc-2.21))
   (name-service-switch operating-system-name-service-switch ; 
<name-service-switch>
                        (default %default-nss))
 
@@ -866,7 +868,8 @@ listed in OS.  The C library expects to find it under
     (raise (condition
             (&message (message "system locale lacks a definition")))))
 
-  (locale-directory (operating-system-locale-definitions os)))
+  (locale-directory (operating-system-locale-definitions os)
+                    #:libc (operating-system-glibc-for-locales os)))
 
 (define (kernel->grub-label kernel)
   "Return a label for the GRUB menu entry that boots KERNEL."
diff --git a/gnu/system/locale.scm b/gnu/system/locale.scm
index 393dd42..f7e17f8 100644
--- a/gnu/system/locale.scm
+++ b/gnu/system/locale.scm
@@ -20,8 +20,11 @@
   #:use-module (guix gexp)
   #:use-module (guix records)
   #:use-module (gnu packages base)
+  #:use-module (gnu packages guile)
+  #:use-module (gnu packages linux)
   #:use-module (gnu packages compression)
   #:use-module (srfi srfi-26)
+  #:use-module (ice-9 match)
   #:export (locale-definition
             locale-definition?
             locale-definition-name
@@ -59,6 +62,25 @@
                       (string-append #$output "/"
                                      #$(locale-definition-name locale))))))
 
+(define (sanity-check locale-data locale)
+  "Return an expression that returns true if a program linked against the
+current glibc package can read locale data for LOCALE from LOCALE-DATA."
+  #~(begin
+      (format #t "checking whether current libc can \
+load this locale data from '~a'...~%"
+              #$locale-data)
+      (setenv "LOCPATH" #$locale-data)
+      (setenv "LC_ALL" #$locale)
+
+      ;; If this fails with an assertion failure, then this is bad.  See
+      ;; <https://lists.gnu.org/archive/html/guix-devel/2015-08/msg00737.html>
+      ;; and <https://sourceware.org/ml/libc-alpha/2015-09/msg00575.html>.
+      ;; We can't check whether (setlocale LC_ALL "") works because it is
+      ;; likely to fail, at least because of one locale category whose data
+      ;; format has changed incompatibly.
+      (zero? (system* (string-append #$(canonical-package glibc)
+                                     "/bin/locale")))))
+
 (define* (locale-directory locales
                            #:key (libc (canonical-package glibc)))
   "Return a directory containing all of LOCALES compiled."
@@ -70,8 +92,15 @@
         (setenv "PATH" (string-append #$gzip "/bin"))
 
         (exit
-         (and #$@(map (cut localedef-command <> #:libc libc)
-                      locales)))))
+         (and (system* (string-append #$libc "/bin/localedef") "--version")
+              #$@(map (cut localedef-command <> #:libc libc)
+                      locales)
+              #$(match locales
+                  ((first . _)
+                   (sanity-check #~#$output
+                                 (locale-definition-name first)))
+                  (()
+                   #t))))))
 
   (gexp->derivation "locale" build
                     #:local-build? #t))
@@ -97,37 +126,37 @@
            (source "en_US")
            (charset "UTF-8"))
           (utf8-locales "ca_ES"
-                        "cs_CZ"
-                        "da_DK"
-                        "de_DE"
-                        "el_GR"
-                        "en_AU"
-                        "en_CA"
-                        "en_GB"
-                        "en_US"
-                        "es_AR"
-                        "es_CL"
-                        "es_ES"
-                        "es_MX"
-                        "fi_FI"
-                        "fr_BE"
-                        "fr_CA"
-                        "fr_CH"
-                        "fr_FR"
-                        "ga_IE"
-                        "it_IT"
-                        "ja_JP"
-                        "ko_KR"
-                        "nb_NO"
-                        "nl_NL"
-                        "pl_PL"
-                        "pt_PT"
-                        "ro_RO"
-                        "ru_RU"
-                        "sv_SE"
-                        "tr_TR"
-                        "uk_UA"
-                        "vi_VN"
+                        ;; "cs_CZ"
+                        ;; "da_DK"
+                        ;; "de_DE"
+                        ;; "el_GR"
+                        ;; "en_AU"
+                        ;; "en_CA"
+                        ;; "en_GB"
+                        ;; "en_US"
+                        ;; "es_AR"
+                        ;; "es_CL"
+                        ;; "es_ES"
+                        ;; "es_MX"
+                        ;; "fi_FI"
+                        ;; "fr_BE"
+                        ;; "fr_CA"
+                        ;; "fr_CH"
+                        ;; "fr_FR"
+                        ;; "ga_IE"
+                        ;; "it_IT"
+                        ;; "ja_JP"
+                        ;; "ko_KR"
+                        ;; "nb_NO"
+                        ;; "nl_NL"
+                        ;; "pl_PL"
+                        ;; "pt_PT"
+                        ;; "ro_RO"
+                        ;; "ru_RU"
+                        ;; "sv_SE"
+                        ;; "tr_TR"
+                        ;; "uk_UA"
+                        ;; "vi_VN"
                         "zh_CN"))))
 
 ;;; locale.scm ends here
-- 
2.5.0

>From 5024853a917706ba9ef0353444ce54d3b846b5be Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <address@hidden>
Date: Thu, 24 Sep 2015 16:21:45 +0200
Subject: [PATCH 1/2] gnu: glibc: Add version 2.21.

* gnu/packages/base.scm (glibc-2.21): New variable.
---
 gnu/packages/base.scm | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/gnu/packages/base.scm b/gnu/packages/base.scm
index 69db178..d263093 100644
--- a/gnu/packages/base.scm
+++ b/gnu/packages/base.scm
@@ -618,6 +618,21 @@ with the Linux kernel.")
    (license lgpl2.0+)
    (home-page "http://www.gnu.org/software/libc/";)))
 
+(define-public glibc-2.21
+  ;; The old libc, which we use mostly to build locale data in the old format
+  ;; (which the new libc can cope with.)
+  (package
+    (inherit glibc)
+    (version "2.21")
+    (source (origin
+              (inherit (package-source glibc))
+              (uri (string-append "mirror://gnu/glibc/glibc-"
+                                  version ".tar.xz"))
+              (sha256
+               (base32
+                "1f135546j34s9bfkydmx2nhh9vwxlx60jldi80zmsnln6wj3dsxf"))
+              (patches (list (search-patch "glibc-ldd-x86_64.patch")))))))
+
 (define-public glibc-locales
   (package
     (inherit glibc)
-- 
2.5.0


reply via email to

[Prev in Thread] Current Thread [Next in Thread]