Re: Filename completion broken on single quote

[Chet Ramey]

On 10/15/11 10:34 AM, lolilolicon wrote:
> On Sat, Oct 15, 2011 at 1:30 AM, Chet Ramey <chet.ramey@case.edu> wrote:
>> On 10/13/11 10:37 AM, lolilolicon wrote:
>>> Inside an empty directory:
>>>
>>>   touch 1 1\'1
>>>   complete -r ls
>>>   ls 1\'<TAB>       # ls 1\'1
>>>   complete -f ls
>>>   ls 1\'<TAB>       # ls 1
>>>   ls 1\\\'<TAB>     # ls 1\'1
>>>
>>> (Note: the comment on the right is the result after pressing <TAB>)
>>>
>>> Is this broken behavior?
>>> Why on earth does the user have to double-escape at all?
>>> Is there any benefit of this?
>>
>> Thanks for the report.  This looks like a problem, or an omission, with
>> the effects of the `-f' option to complete, since it works as expected
>> without using programmable completion or when using `-o default'.  I
>> will take a look.
>>
> 
> OK, some more strange test results.
> 
> In the interactive bash shell, I did this (in an empty directory):
> 
>   $ mkdir 1\'1          $ mkdir 2@2
>   $ touch 1\'1/one      $ touch 2@2/two
>   $ compgen -f "1'1"    $ compgen -f "2@2"
>   1'1                   2@2
>   $ compgen -f "1\'1"   $ compgen -f "2\@2"
>   $ compgen -f "1'1/"   $ compgen -f "2@2/"
>                         2@2/two
>   $ compgen -f "1\'1/"  $ compgen -f "2\@2/"
>   1\'1/one              2\@2/two
> 
> (Note: Put side by side for comparision.)
> 
> A bit inconsistency here. Then I wrote a bash script to do the same thing,
> 
>   #!/bin/bash
> 
>   shopt -s progcomp
> 
>   mkdir -p 1\'1 2@2
>   touch 1\'1/one 2@2/two
> 
>   for i in \
>     "1'1" "1\'1" "1'1/" "1\'1/" \
>     "2@2" "2\@2" "2@2/" "2\@2/"; do
>     printf -- '%5s => %s\n' "$i" "$(compgen -f "$i")"
>   done
>   # (Same results using `compgen -o default')
> 
> yet with different results:
> 
>     1'1 => 1'1
>    1\'1 =>
>    1'1/ => 1'1/one
>   1\'1/ =>
>     2@2 => 2@2
>    2\@2 =>
>    2@2/ => 2@2/two
>   2\@2/ =>
> 
> What's going on with all this inconsistency?

Maybe that a single quote is a quote character, but an at sign has no
special meaning.  compgen expects its arguments to come in quoted as
they would be when entered from the command line, when using readline
to try and complete them.  "1'1" gives compgen an unquoted single quote,
since the double quotes are removed before compgen sees it.

The real problem is that readline runs the filename dequoting function
based on whether or not it found a quote character, and compgen (and the
underlying programmable completion functions) has to either check, guess,
or dequote unconditionally when run from the command line.

Three-plus years ago, when this came up, I wrote:

==========
For historical reasons, complete/compgen dequote the filename they're
passed, removing backslash escapes and interpreting embedded quoted
substrings.  (One of the things bash should do when it does that is to
be better about obeying the shell rules about backslash-escaped characters
and double quotes, but that doesn't matter for this example -- though that
function has several problems.)

When it's called as the result of readline dispatching on a particular
character, this is appropriate -- the shell hasn't done any expansion
or quote removal, so the filenames still have embedded quoting.  When
called from the command line, as in these examples, it's not appropriate,
since the shell has already expanded the argument and stripped the
quotes.
==========

Some of the details have changed since then, but the essentials are
still the same.

There are three cases to consider: "straight" programmable completion
(complete -f), `compgen' run from a programmable completion function
(foo=( $( compgen -f "$word") ) ), and `compgen -f' run from the command
line.  They're not all the same, but the programmable completion code's
filename completion function has to somehow accommodate them (and,
frankly, the running-from-the-command-line case is the least important).

I will work out some improved heuristics and implement them for the next
version, and things will get better for both straight `complete -f' and
when running compgen from the command line.

> Also, dumb question, but is `compgen' supposed to accept quoted arguments or
> dequoted ones? e.g., "1\'1/" or "1'1/"?

It should accept arguments quoted as they would be during completion,
but you have to take into account the word expansions performed on its
arguments.

Chet
-- 
``The lyf so short, the craft so long to lerne.'' - Chaucer
                 ``Ars longa, vita brevis'' - Hippocrates
Chet Ramey, ITS, CWRU    chet@case.edu    http://cnswww.cns.cwru.edu/~chet/