bug#20703: 24.4; Stack overflow in regexp matcher

[lee]

Dmitry Gutov <dgutov@yandex.ru> writes:

> Please keep the bug address in Cc.
>
> On 06/01/2015 09:03 PM, lee wrote:
>
>>> What if you try `M-x find-tag'?
>>
>> That works.
>
> What if you type `M-x find-tag TAB' (to ask Emacs for all available tags)?

Processing goes to 42% before the debugger comes up:


Debugger entered--Lisp error: (error "Stack overflow in regexp matcher")
  
re-search-forward("^\\(\\([^]+[^-a-zA-Z0-9_+*$:]+\\)?\\([-a-zA-Z0-9_+*$?:]+\\)[^-a-zA-Z0-9_+*$?:]*\\)\\(\\([^\n.]+\\).\\)?\\([0-9]+\\)?,\\([0-9]+\\)?\n"
 nil t)
  etags-tags-completion-table()
  #[0 "\303\211C\304\305\"\210\212\306.\242\205.\307!\2038.    
\262.\211\242\2031.\310\311\312\313\314\315.!\316\"\317\320%.\"\210\202       
.\211.\240\210\202      .)\304\321\"\210\211\242\211.\207" [buffer-file-name 
tags-completion-table-function tags-completion-table nil message "Making tags 
completion table for %s..." visit-tags-table-buffer t mapatoms make-byte-code 
257 "\301\302.!\300\242\"\207" vconcat vector [intern symbol-name] 4 "\n\n(fn 
SYM)" "Making tags completion table for %s...done"] 9 "\n\n(fn)"]()
  funcall(#[0 "\303\211C\304\305\"\210\212\306.\242\205.\307!\2038.    
\262.\211\242\2031.\310\311\312\313\314\315.!\316\"\317\320%.\"\210\202       
.\211.\240\210\202      .)\304\321\"\210\211\242\211.\207" [buffer-file-name 
tags-completion-table-function tags-completion-table nil message "Making tags 
completion table for %s..." visit-tags-table-buffer t mapatoms make-byte-code 
257 "\301\302.!\300\242\"\207" vconcat vector [intern symbol-name] 4 "\n\n(fn 
SYM)" "Making tags completion table for %s...done"] 9 "\n\n(fn)"])
  tags-completion-table()
  #[771 "r\300q\210\212\302.\303 \210)\304.\305 .$*\207" [#<buffer 
llagent.cpp> enable-recursive-minibuffers t visit-tags-table-buffer 
complete-with-action tags-completion-table] 8 "\n\n(fn STRING PRED ACTION)"]("" 
nil metadata)
  completion-metadata("" #[771 "r\300q\210\212\302.\303 \210)\304.\305 
.$*\207" [#<buffer llagent.cpp> enable-recursive-minibuffers t 
visit-tags-table-buffer complete-with-action tags-completion-table] 8 "\n\n(fn 
STRING PRED ACTION)"] nil)
  completion--field-metadata(28)
  completion--do-completion(28 28)
  completion--in-region-1(28 28)
  #[1028 
".
.\n\203!.\304.!\203..\202.\305.!\305.\306\".F.\307\310!\210\311.\"*\207"
 [minibuffer-completion-predicate minibuffer-completion-table 
completion-in-region-mode-predicate completion-in-region--data markerp 
copy-marker t completion-in-region-mode 1 completion--in-region-1] 8 "\n\n(fn 
START END COLLECTION PREDICATE)"](28 28 #[771 "r\300q\210\212\302.\303 
\210)\304.\305 .$*\207" [#<buffer llagent.cpp> enable-recursive-minibuffers t 
visit-tags-table-buffer complete-with-action tags-completion-table] 8 "\n\n(fn 
STRING PRED ACTION)"] nil)
  apply(#[1028 
".
.\n\203!.\304.!\203..\202.\305.!\305.\306\".F.\307\310!\210\311.\"*\207"
 [minibuffer-completion-predicate minibuffer-completion-table 
completion-in-region-mode-predicate completion-in-region--data markerp 
copy-marker t completion-in-region-mode 1 completion--in-region-1] 8 "\n\n(fn 
START END COLLECTION PREDICATE)"] (28 28 #[771 "r\300q\210\212\302.\303 
\210)\304.\305 .$*\207" [#<buffer llagent.cpp> enable-recursive-minibuffers t 
visit-tags-table-buffer complete-with-action tags-completion-table] 8 "\n\n(fn 
STRING PRED ACTION)"] nil))
  #[771 
".:\2030.@\301=\203.\300\242\302.A\"\303.#\207\304.@\305\306\307\310\311\312\300!\313\"\314\315%.A.#.#\207\304\316.\"\207"
 [(#0) t append nil apply apply-partially make-byte-code 642 "\300\242..#\207" 
vconcat vector [] 7 "\n\n(fn FUNS GLOBAL &rest ARGS)" #[1028 
".
.\n\203!.\304.!\203..\202.\305.!\305.\306\".F.\307\310!\210\311.\"*\207"
 [minibuffer-completion-predicate minibuffer-completion-table 
completion-in-region-mode-predicate completion-in-region--data markerp 
copy-marker t completion-in-region-mode 1 completion--in-region-1] 8 "\n\n(fn 
START END COLLECTION PREDICATE)"]] 12 "\n\n(fn FUNS GLOBAL ARGS)"](nil nil (28 
28 #[771 "r\300q\210\212\302.\303 \210)\304.\305 .$*\207" [#<buffer 
llagent.cpp> enable-recursive-minibuffers t visit-tags-table-buffer 
complete-with-action tags-completion-table] 8 "\n\n(fn STRING PRED ACTION)"] 
nil))
  completion--in-region(28 28 #[771 "r\300q\210\212\302.\303 \210)\304.\305 
.$*\207" [#<buffer llagent.cpp> enable-recursive-minibuffers t 
visit-tags-table-buffer complete-with-action tags-completion-table] 8 "\n\n(fn 
STRING PRED ACTION)"] nil)
  completion-in-region(28 28 #[771 "r\300q\210\212\302.\303 \210)\304.\305 
.$*\207" [#<buffer llagent.cpp> enable-recursive-minibuffers t 
visit-tags-table-buffer complete-with-action tags-completion-table] 8 "\n\n(fn 
STRING PRED ACTION)"] nil)
  minibuffer-complete()
  call-interactively(minibuffer-complete nil nil)
  command-execute(minibuffer-complete)
  read-from-minibuffer("Find tag (default gAgent): " nil (keymap (menu-bar 
keymap (minibuf "Minibuf" keymap (tab menu-item "Complete" minibuffer-complete 
:help "Complete as far as possible") (space menu-item "Complete Word" 
minibuffer-complete-word :help "Complete at most one word") (63 menu-item "List 
Completions" minibuffer-completion-help :help "Display all possible 
completions") "Minibuf")) (27 keymap (118 . switch-to-completions)) (prior . 
switch-to-completions) (63 . minibuffer-completion-help) (32 . 
minibuffer-complete-word) (9 . minibuffer-complete) keymap (menu-bar keymap 
(minibuf "Minibuf" keymap (previous menu-item "Previous History Item" 
previous-history-element :help "Put previous minibuffer history element in the 
minibuffer") (next menu-item "Next History Item" next-history-element :help 
"Put next minibuffer history element in the minibuffer") (isearch-backward 
menu-item "Isearch History Backward" isearch-backward :help "Incrementally 
search minibuffer history backward") (isearch-forward menu-item "Isearch 
History Forward" isearch-forward :help "Incrementally search minibuffer history 
forward") (return menu-item "Enter" exit-minibuffer :key-sequence "." :help 
"Terminate input and exit minibuffer") (quit menu-item "Quit" 
abort-recursive-edit :help "Abort input and exit minibuffer") "Minibuf")) (10 . 
exit-minibuffer) (13 . exit-minibuffer) (7 . abort-recursive-edit) (C-tab . 
file-cache-minibuffer-complete) (9 . self-insert-command) (XF86Back . 
previous-history-element) (up . previous-history-element) (prior . 
previous-history-element) (XF86Forward . next-history-element) (down . 
next-history-element) (next . next-history-element) (27 keymap (114 . 
previous-matching-history-element) (115 . next-matching-history-element) (112 . 
previous-history-element) (110 . next-history-element))) nil nil "gAgent" nil)
  completing-read-default("Find tag (default gAgent): " #[771 
"r\300q\210\212\302.\303 \210)\304.\305 .$*\207" [#<buffer llagent.cpp> 
enable-recursive-minibuffers t visit-tags-table-buffer complete-with-action 
tags-completion-table] 8 "\n\n(fn STRING PRED ACTION)"] nil nil nil nil 
"gAgent" nil)
  completing-read("Find tag (default gAgent): " #[771 "r\300q\210\212\302.\303 
\210)\304.\305 .$*\207" [#<buffer llagent.cpp> enable-recursive-minibuffers t 
visit-tags-table-buffer complete-with-action tags-completion-table] 8 "\n\n(fn 
STRING PRED ACTION)"] nil nil nil nil "gAgent")
  find-tag-tag("Find tag: ")
  find-tag-interactive("Find tag: ")
  call-interactively(find-tag record nil)
  command-execute(find-tag record)
  execute-extended-command(nil "find-tag")
  call-interactively(execute-extended-command nil nil)
  command-execute(execute-extended-command)


When I use C-p j and interrupt making the completion table with C-g
before the debugger comes up, I can enter what tag I'm searching for and
it can be found.

>> I cut off roughly the bottom half of the TAGS file and tried again. With
>> that, I'm getting the error when progress is at 85% instead of
>> 42%. Cutting off the bottom half again, leaving about 1/4 of the
>> original file, does not yield an error and says no matching tags were
>> found.
>
> The idea was to split the file in half, and do a sort of binary
> search. E.g., try cutting off the top half in the first step now.
>
>> So I guess the problem might have to do with the size of the TAGS file
>> ...
>
> Not necessarily. The TAGS file is parsed sequentially, without
> recursion in the Lisp code.
>
> In all likelihood, there is a problematic line around 42% of the
> original TAGS, and the error goes away when that line is not in the
> file anymore.
>
> We need to know that line to fix the bug.

I tried to find the line and only got to the point where so much of the
file was cut out that I didn't manage to go back to a step at which I'm
getting the error.  If I have some time this weekend, I can try again.

Isn't there a way to get a better hint than the pretty vague "42%"?


-- 
Again we must be afraid of speaking of daemons for fear that daemons
might swallow us.  Finally, this fear has become reasonable.