The sample function invert_case_line() causes segfault with negative arguments

[林宏雄]

> the version number and release status of Readline (e.g., 4.2-release)

readline-8.3-testing branch

(The issue has been included since the old versions.)

> the machine and OS that it is running on

I've confirmed on x86 Ubuntu and macOS, but the issue should be independent with OS.

> a list of the compilation flags or the contents of `config.h', if appropriate

Run configure without additional options.

> a description of the bug

The sample function invert_case_line() causes segmentation fault with negative arguments.

> a recipe for recreating the bug reliably

1. apply patch1.diff attached.

This adds main() in example/rlbasic.c on example/manexample.c,
bind invert_case_line() to M-u.
It also add manexamp on Makefile.in.

2. run the following commands

./configure
make -C examples
examples/manexamp

3. type Meta - Meta u

This causes a segmentation fault.

> a fix for the bug if you have one!

A simple fix is to remove the if-block that swaps start and end variables.

@@ -111,13 +111,6 @@ invert_case_line (int count, int key)
   else if (end < 0)
     end = -1;
 
-  if (start > end)
-    {
-      int temp = start;
-      start = end;
-      end = temp;
-    }
-
   if (start == end)
     return 0

patch2.diff also includes a fix to make cursor behavior similar to other readline commands.

Note that invert_case_line() in doc/rltech.texi also have to be fixed.

--
Hiroo Hayashi

patch2.diff
Description: Binary data

patch1.diff
Description: Binary data