[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Preparing for the libc/locale upgrade
From: |
Ludovic Courtès |
Subject: |
Preparing for the libc/locale upgrade |
Date: |
Thu, 24 Sep 2015 21:25:30 +0200 |
User-agent: |
Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux) |
Problem: Programs linked against libc < 2.22 abort when trying to load
libc 2.22-or-later locale data:
https://lists.gnu.org/archive/html/guix-devel/2015-08/msg00737.html
Programs linked against libc 2.22 do not abort when trying to load 2.21
locale data, but they fail to load data for the LC_COLLATE category (in
which case ‘setlocale’ returns ENOENT and the program prints a warning)
due to the addition of the _NL_COLLATE_ENCODING_TYPE element.
In all situations, as a last resort, one can always run:
export LC_ALL=C
to sidestep the incompatibility issues.
Since ‘core-updates’ will soon be merged, bringing libc 2.22, here’s a
preview of the problems you may encounter and possible solutions.
Consequences on GuixSD:
• If programs in user profile use libc 2.22 but
/run/current-system/locale provides libc 2.21 locales: no hard
problem, but the LC_COLLATE problem.
• If LOCPATH points to 2.22 locales but /run/current-system/profile
contains 2.21 programs (Coreutils, sed, grep, etc.), these will
abort. Solution: unset LOCPATH so that programs use the 2.21 locale
data from /run/current-system/locale.
• If /run/current-system is upgraded to 2.22 but user profiles contain
2.21 programs, these will abort. Solution: upgrade user profiles,
or install glibc-utf8-locale-2.21 in user profile (preferably before
upgrading GuixSD to libc 2.22.)
Consequences for Guix on foreign distros:
• If the host distro provides binaries that use libc < 2.22 and you
use a mixture of Guix-provided and distro-provided programs, this is
pretty bad.
Solution: unset LOCPATH and say goodbye to locales for Guix-provided
packages (setting LOCPATH=$HOME/.guix-profile/lib/locale would break
all the distro-provided programs), or use exclusively Guix-provided
programs, or use the “C” locale.
Lesson learned:
• On GuixSD, we should use a versioned directory,
/run/current-system/locale/X.Y, as the default locale directory,
which would avoid problems.
• The attached patches allow GuixSD users to choose a different libc
than the current one to build the locales that go to
/run/current-system/locale. However this isn’t really satisfying.
Comments welcome!
Ludo’.
>From beb9975ed28e12b78f7043d76f2895c97cd7e4a8 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <address@hidden>
Date: Thu, 24 Sep 2015 21:03:45 +0200
Subject: [PATCH 2/2] system: locale: Add a 'glibc-for-locales' knob.
* gnu/system.scm (<operating-system>)[glibc-for-locales]: New field.
(operating-system-locale-directory): Honor it.
* gnu/system/locale.scm (sanity-check): New procedure.
(locale-directory): Use it.
* doc/guix.texi (Locales)[Locale Data Compatibility Considerations]: New
section.
---
doc/guix.texi | 42 +++++++++++++++++++++++
gnu/system.scm | 5 ++-
gnu/system/locale.scm | 95 +++++++++++++++++++++++++++++++++------------------
3 files changed, 108 insertions(+), 34 deletions(-)
diff --git a/doc/guix.texi b/doc/guix.texi
index 07c5add..6c0adda 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -5158,6 +5158,10 @@ Library Reference Manual}). @xref{Locales}, for more
information.
The list of locale definitions to be compiled and that may be used at
run time. @xref{Locales}.
address@hidden @code{glibc-for-locales} (default: @var{glibc-2.21})
+The address@hidden package whose locale data and tools are used to build
+the locale definitions. @xref{Locales}.
+
@item @code{name-service-switch} (default: @var{%default-nss})
Configuration of libc's name service switch (NSS)---a
@code{<name-service-switch>} object. @xref{Name Service Switch}, for
@@ -5615,6 +5619,44 @@ instance it has @code{uk_UA.utf8} but @emph{not}, say,
@code{uk_UA.UTF-8}.
@end defvr
address@hidden Locale Data Compatibility Considerations
+
address@hidden incompatibility, of locale data
address@hidden declarations provide a @code{glibc-for-locales}
+field to specify the address@hidden package that is used to compile
+locale declarations (@pxref{operating-system Reference}). ``Why would I
+care?'', you may ask. Well, it turns out that the binary format of
+locale data may be incompatible from one libc version to another.
+
address@hidden See <https://sourceware.org/ml/libc-alpha/2015-09/msg00575.html>
address@hidden and
<https://lists.gnu.org/archive/html/guix-devel/2015-08/msg00737.html>.
+For instance, a program linked against libc version 2.21 is unable to
+read locale data produced with libc 2.22; worse, that program
address@hidden instead of simply ignoring the incompatible locale
address@hidden 2.23 and later of address@hidden will simply skip
+the incompatible locale data, which is already an improvement.}. On the
+other hand, a program linked against libc 2.22 can read locale data from
+libc 2.21, except for @code{LC_COLLATE} data; thus calls to
address@hidden may fail, but programs will not abort.
+
+The ``problem'' in GuixSD is that users have a lot of freedom: They can
+choose whether and when to upgrade software in their profiles, and might
+be using a libc version different from the one the system administrator
+used to build the system-wide locale data.
+
+Fortunately, users can also install their own locale data and define
address@hidden accordingly (@pxref{locales-and-locpath, @code{LOCPATH}
+and locale packages}).
+
+Still, it is best if the system-wide locale data at
address@hidden/run/current-system/locale} is built using an older libc. That
+way, both programs linked against the new libc and programs linked
+against the older one can load locale data. For this reason, the
address@hidden field defaults to a libc version slightly older
+than the current one, with the caveat that some locale categories may be
+incompatible.
+
+
@node Services
@subsection Services
diff --git a/gnu/system.scm b/gnu/system.scm
index cee5f37..8b5f981 100644
--- a/gnu/system.scm
+++ b/gnu/system.scm
@@ -146,6 +146,8 @@
(default "en_US.utf8"))
(locale-definitions operating-system-locale-definitions ; list of
<locale-definition>
(default %default-locale-definitions))
+ (glibc-for-locales operating-system-glibc-for-locales ; <package>
+ (default glibc-2.21))
(name-service-switch operating-system-name-service-switch ;
<name-service-switch>
(default %default-nss))
@@ -866,7 +868,8 @@ listed in OS. The C library expects to find it under
(raise (condition
(&message (message "system locale lacks a definition")))))
- (locale-directory (operating-system-locale-definitions os)))
+ (locale-directory (operating-system-locale-definitions os)
+ #:libc (operating-system-glibc-for-locales os)))
(define (kernel->grub-label kernel)
"Return a label for the GRUB menu entry that boots KERNEL."
diff --git a/gnu/system/locale.scm b/gnu/system/locale.scm
index 393dd42..f7e17f8 100644
--- a/gnu/system/locale.scm
+++ b/gnu/system/locale.scm
@@ -20,8 +20,11 @@
#:use-module (guix gexp)
#:use-module (guix records)
#:use-module (gnu packages base)
+ #:use-module (gnu packages guile)
+ #:use-module (gnu packages linux)
#:use-module (gnu packages compression)
#:use-module (srfi srfi-26)
+ #:use-module (ice-9 match)
#:export (locale-definition
locale-definition?
locale-definition-name
@@ -59,6 +62,25 @@
(string-append #$output "/"
#$(locale-definition-name locale))))))
+(define (sanity-check locale-data locale)
+ "Return an expression that returns true if a program linked against the
+current glibc package can read locale data for LOCALE from LOCALE-DATA."
+ #~(begin
+ (format #t "checking whether current libc can \
+load this locale data from '~a'...~%"
+ #$locale-data)
+ (setenv "LOCPATH" #$locale-data)
+ (setenv "LC_ALL" #$locale)
+
+ ;; If this fails with an assertion failure, then this is bad. See
+ ;; <https://lists.gnu.org/archive/html/guix-devel/2015-08/msg00737.html>
+ ;; and <https://sourceware.org/ml/libc-alpha/2015-09/msg00575.html>.
+ ;; We can't check whether (setlocale LC_ALL "") works because it is
+ ;; likely to fail, at least because of one locale category whose data
+ ;; format has changed incompatibly.
+ (zero? (system* (string-append #$(canonical-package glibc)
+ "/bin/locale")))))
+
(define* (locale-directory locales
#:key (libc (canonical-package glibc)))
"Return a directory containing all of LOCALES compiled."
@@ -70,8 +92,15 @@
(setenv "PATH" (string-append #$gzip "/bin"))
(exit
- (and #$@(map (cut localedef-command <> #:libc libc)
- locales)))))
+ (and (system* (string-append #$libc "/bin/localedef") "--version")
+ #$@(map (cut localedef-command <> #:libc libc)
+ locales)
+ #$(match locales
+ ((first . _)
+ (sanity-check #~#$output
+ (locale-definition-name first)))
+ (()
+ #t))))))
(gexp->derivation "locale" build
#:local-build? #t))
@@ -97,37 +126,37 @@
(source "en_US")
(charset "UTF-8"))
(utf8-locales "ca_ES"
- "cs_CZ"
- "da_DK"
- "de_DE"
- "el_GR"
- "en_AU"
- "en_CA"
- "en_GB"
- "en_US"
- "es_AR"
- "es_CL"
- "es_ES"
- "es_MX"
- "fi_FI"
- "fr_BE"
- "fr_CA"
- "fr_CH"
- "fr_FR"
- "ga_IE"
- "it_IT"
- "ja_JP"
- "ko_KR"
- "nb_NO"
- "nl_NL"
- "pl_PL"
- "pt_PT"
- "ro_RO"
- "ru_RU"
- "sv_SE"
- "tr_TR"
- "uk_UA"
- "vi_VN"
+ ;; "cs_CZ"
+ ;; "da_DK"
+ ;; "de_DE"
+ ;; "el_GR"
+ ;; "en_AU"
+ ;; "en_CA"
+ ;; "en_GB"
+ ;; "en_US"
+ ;; "es_AR"
+ ;; "es_CL"
+ ;; "es_ES"
+ ;; "es_MX"
+ ;; "fi_FI"
+ ;; "fr_BE"
+ ;; "fr_CA"
+ ;; "fr_CH"
+ ;; "fr_FR"
+ ;; "ga_IE"
+ ;; "it_IT"
+ ;; "ja_JP"
+ ;; "ko_KR"
+ ;; "nb_NO"
+ ;; "nl_NL"
+ ;; "pl_PL"
+ ;; "pt_PT"
+ ;; "ro_RO"
+ ;; "ru_RU"
+ ;; "sv_SE"
+ ;; "tr_TR"
+ ;; "uk_UA"
+ ;; "vi_VN"
"zh_CN"))))
;;; locale.scm ends here
--
2.5.0
>From 5024853a917706ba9ef0353444ce54d3b846b5be Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <address@hidden>
Date: Thu, 24 Sep 2015 16:21:45 +0200
Subject: [PATCH 1/2] gnu: glibc: Add version 2.21.
* gnu/packages/base.scm (glibc-2.21): New variable.
---
gnu/packages/base.scm | 15 +++++++++++++++
1 file changed, 15 insertions(+)
diff --git a/gnu/packages/base.scm b/gnu/packages/base.scm
index 69db178..d263093 100644
--- a/gnu/packages/base.scm
+++ b/gnu/packages/base.scm
@@ -618,6 +618,21 @@ with the Linux kernel.")
(license lgpl2.0+)
(home-page "http://www.gnu.org/software/libc/")))
+(define-public glibc-2.21
+ ;; The old libc, which we use mostly to build locale data in the old format
+ ;; (which the new libc can cope with.)
+ (package
+ (inherit glibc)
+ (version "2.21")
+ (source (origin
+ (inherit (package-source glibc))
+ (uri (string-append "mirror://gnu/glibc/glibc-"
+ version ".tar.xz"))
+ (sha256
+ (base32
+ "1f135546j34s9bfkydmx2nhh9vwxlx60jldi80zmsnln6wj3dsxf"))
+ (patches (list (search-patch "glibc-ldd-x86_64.patch")))))))
+
(define-public glibc-locales
(package
(inherit glibc)
--
2.5.0
- Preparing for the libc/locale upgrade,
Ludovic Courtès <=