bug#23205: 25.0.92; sort-tests is flaky

[Philipp Stephani]

In the emacs-25 branch, in the directory test/automated, run:

declare -i failures=0
for ((i=0; i<1000; i++)); do
if ! output="$(make sort-tests 2>&1)"; then
((failures++))
echo "Iteration $i: $output"
fi
done
echo "Failures: $failures"

There will be typically a small number of failures (< 10), caused by
e.g.

Test sort-tests--lines condition:
(ert-test-failed
((should
(string=
(substring ... 0 -1)
sorted-words))
:form
(string= "Zi
zI" "zI
Zi")
:value nil))

This is presumably due to non-stable sorting. Either the sorting should
be made stable, or the test shouldn't assume stable sorting.



In GNU Emacs 25.0.92.1 (x86_64-unknown-linux-gnu, GTK+ Version 3.10.8)
of 2016-03-24 built on localhost
Repository revision: 077b78cffc0711717ae8976ef302b788e88d27a8
Windowing system distributor 'The X.Org Foundation', version 11.0.11701000
System Description: Ubuntu 14.04 LTS

Configured features:
XPM JPEG TIFF GIF PNG SOUND GSETTINGS NOTIFY FREETYPE XFT ZLIB
TOOLKIT_SCROLL_BARS GTK3 X11

Important settings:
value of $LANG: en_US.UTF-8
locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
tooltip-mode: t
global-eldoc-mode: t
electric-indent-mode: t
mouse-wheel-mode: t
tool-bar-mode: t
menu-bar-mode: t
file-name-shadow-mode: t
global-font-lock-mode: t
font-lock-mode: t
blink-cursor-mode: t
auto-composition-mode: t
auto-encryption-mode: t
auto-compression-mode: t
line-number-mode: t
transient-mark-mode: t

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message dired format-spec rfc822 mml
mml-sec password-cache epg epg-config gnus-util mm-decode mm-bodies
mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail
rfc2047 rfc2045 ietf-drums mm-util help-fns help-mode easymenu
cl-loaddefs pcase cl-lib mail-prsvr mail-utils time-date mule-util
tooltip eldoc electric uniquify ediff-hook vc-hooks lisp-float-type
mwheel x-win term/common-win x-dnd tool-bar dnd fontset image regexp-opt
fringe tabulated-list newcomment elisp-mode lisp-mode prog-mode register
page menu-bar rfn-eshadow timer select scroll-bar mouse jit-lock
font-lock syntax facemenu font-core frame cl-generic cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european
ethiopic indian cyrillic chinese charscript case-table epa-hook
jka-cmpr-hook help simple abbrev minibuffer cl-preloaded nadvice
loaddefs button faces cus-face macroexp files text-properties overlay
sha1 md5 base64 format env code-pages mule custom widget
hashtable-print-readable backquote inotify dynamic-setting
system-font-setting font-render-setting move-toolbar gtk x-toolkit x
multi-tty make-network-process emacs)

Memory information:
((conses 16 87235 4859)
(symbols 48 19673 0)
(miscs 40 278 121)
(strings 32 14530 4256)
(string-bytes 1 423141)
(vectors 16 11958)
(vector-slots 8 433605 5930)
(floats 8 162 10)
(intervals 56 184 0)
(buffers 976 11)
(heap 1024 50063 1046))

--

Google Germany GmbH
Erika-Mann-Straße 33
80636 München

Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg
Geschäftsführer: Matthew Scott Sucherman, Paul Terence Manicle

Diese E-Mail ist vertraulich. Wenn Sie nicht der richtige Adressat sind, leiten Sie diese bitte nicht weiter, informieren Sie den Absender und löschen Sie die E-Mail und alle Anhänge. Vielen Dank.

This e-mail is confidential. If you are not the right addressee please do not forward it, please inform the sender, and please erase this e-mail including any attachments. Thanks.