bug#73473: 31.0.50; Minibuffer completions include nonsense prefix candidates

[Jordan Ellis Coppard]

Hello,


I've noticed that my completions have slowed down A LOT on recent Emacs 
builds, it turns out nonsense completion candidates (which I am calling 
"prefix candidates") are being included as... completion candidates. By 
my rough estimate this results in about... 10,000 extra completion 
candidates when invoking `C-h o` (with my personal Emacs configuration).

To reproduce, using `emacs -Q` of course:

1. Open Emacs.
2. C-h o foo TAB

Observe that completion candidate `footnote-` is listed and that there 
are 80 completion candidates.

If I do the same on the currently released Emacs or another (older) 
build of Emacs (30.0.50) there are 79 (not 80) completion candidates and 
`footnote-` is not listed as one.

This seems to be occurring for almost every unique prefix possible, 
compounding as time goes on. So later on `f-` is listed as a completion 
candidate, then also `go-` and so on. These are not valid completion 
candidates. `footnote-` is not a symbol. Over time the huge increase in 
completion candidate volume (and perhaps what is causing this behind the 
scenes) results in such a slowdown that I can visibly see Emacs crawl to 
1 FPS when I type in minibuffer completion, when the exact same init.el 
on said older 30.0.50 is faster.

This is reproduceable on `emacs -Q` as above, (and the useful bug report 
probably ends right here) but as an aside: my configuration is minimal, 
my Emacs starts up in about 0.3s and I keep things minimal because I do 
not want Emacs to ever take longer than instantaneously to respond to 
keystrokes (if it has to do some long-running job, or something which 
takes time that's different since feedback that such a thing has kicked 
off would/should still be instant).

Tangentially to this, it would be nice if completions could be decoupled 
from the UI (i.e. they could stream in, with an API to let the user know 
when the list is complete) because even on 30.0.50 I can often times 
type faster than completion can handle which drives me insane (UI 
hitching)... but that's an aside.


--- diagnostic info ---


In GNU Emacs 31.0.50 (build 1, aarch64-apple-darwin23.5.0, NS
 appkit-2487.60 Version 14.5 (Build 23F79)) of 2024-09-16 built on
 yote.local
Windowing system distributor 'Apple', version 10.3.2487
System Description:  macOS 14.5

Configured using:
 'configure --prefix=/opt/local --disable-silent-rules --without-dbus
 --without-gconf --without-libotf --without-m17n-flt --with-libgmp
 --with-gnutls --with-xml2 --with-modules --with-sqlite3 --with-webp
 --infodir /opt/local/share/info/emacs --with-native-compilation=no
 --with-ns --with-lcms2 --without-harfbuzz --without-imagemagick
 --without-xaw3d --with-rsvg --with-xwidgets
 --with-native-compilation=aot --with-tree-sitter 'CFLAGS=-pipe -Os
 -Wno-attributes
 -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk -arch
 arm64' 'CPPFLAGS=-I/opt/local/include
 -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk'
 'LDFLAGS=-L/opt/local/lib -Wl,-headerpad_max_install_names -Wl,-no_pie
 -Wl,-rpath /opt/local/lib/gcc14 -Wl,-rpath /opt/local/lib
 -Wl,-syslibroot,/Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk
 -arch arm64''

Configured features:
ACL GIF GLIB GMP GNUTLS JPEG LCMS2 LIBXML2 MODULES NATIVE_COMP NOTIFY
KQUEUE NS PDUMPER PNG RSVG SQLITE3 THREADS TIFF TOOLKIT_SCROLL_BARS
TREE_SITTER WEBP XIM XWIDGETS ZLIB

Important settings:
  value of $LC_ALL: en_US.UTF-8
  value of $LC_CTYPE: en_US.UTF-8
  value of $LANG: en_AU.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  minibuffer-regexp-mode: t
  line-number-mode: t
  indent-tabs-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message mailcap yank-media puny dired
dired-loaddefs rfc822 mml mml-sec password-cache epa derived epg rfc6068
epg-config gnus-util text-property-search time-date subr-x mm-decode
mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader
cl-loaddefs cl-lib sendmail rfc2047 rfc2045 ietf-drums mm-util
mail-prsvr mail-utils rmc iso-transl tooltip cconv eldoc paren electric
uniquify ediff-hook vc-hooks lisp-float-type elisp-mode mwheel
term/ns-win ns-win ucs-normalize mule-util term/common-win tool-bar dnd
fontset image regexp-opt fringe tabulated-list replace newcomment
text-mode lisp-mode prog-mode register page tab-bar menu-bar rfn-eshadow
isearch easymenu timer select scroll-bar mouse jit-lock font-lock syntax
font-core term/tty-colors frame minibuffer nadvice seq simple cl-generic
indonesian philippine cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
composite emoji-zwj charscript charprop case-table epa-hook
jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button loaddefs
theme-loaddefs faces cus-face macroexp files window text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget keymap
hashtable-print-readable backquote threads xwidget-internal kqueue cocoa
ns lcms2 multi-tty make-network-process native-compile emacs)

Memory information:
((conses 16 48976 9047) (symbols 48 5284 0) (strings 32 13815 1840)
 (string-bytes 1 469136) (vectors 16 9182)
 (vector-slots 8 130405 8531) (floats 8 22 13) (intervals 56 279 5)
 (buffers 992 10))