Re: Problems with ANSI colors in shell mode in Emacs 21.2.95.

[Francesco Potorti`]

>> Can you determine whether the shell is outputting these ^A and ^B
>> characters?

Following are hints, not proofs.  I am still investigating, but now I'm
going home, so I'll let you know what I found.  By the way, a shorter
way to reproduce the problem is by doing, at the shell pronpt in a shell
buffer:

PS1="\[\033[1;34m\][\h|\[\033[1;31m\]\W\[\033[1;34m\]]>\[\033[0m\] "

Also, the set of change logs from 21.2.90 to 21.2.91 is at
<ftp://alpha.gnu.org/gnu/emacs/pretest/emacs-21.2.91.announce>.  The
problem should be there, as Lute says that 21.2.90 works, while 21.2.91
does not.

>I don't know of a way to determine what the shell is outputting.  Can
>somebody give me a suggestion on how to do this?

Apparently the shell does output those characters.

I did not verify this directly, but rather by reading the bash source.
It seems that bash puts placeholders around non printable strings, and
strips them before outputting the string.  For some reason, these are
not stripped in our case.

I found this in parse.y:
================================================================
#if defined (READLINE)
            case '[':
            case ']':
              temp = (char *)xmalloc (3);
              temp[0] = '\001';
              temp[1] = (c == '[') ? RL_PROMPT_START_IGNORE : 
RL_PROMPT_END_IGNORE;
              temp[2] = '\0';
              goto add_string;
#endif /* READLINE */
================================================================

this in lib/readline/readline.h:
================================================================
   /* Definitions available for use by readline clients. */
#define RL_PROMPT_START_IGNORE  '\001'
#define RL_PROMPT_END_IGNORE    '\002'
================================================================

and this in lib/readline/display.c:
================================================================
/* Expand the prompt string S and return the number of visible
   characters in *LP, if LP is not null.  This is currently more-or-less
   a placeholder for expansion.  LIP, if non-null is a place to store the
   index of the last invisible character in the returned string. NIFLP,
   if non-zero, is a place to store the number of invisible characters in
   the first prompt line. */

/* Current implementation:
        \001 (^A) start non-visible characters
        \002 (^B) end non-visible characters
   all characters except \001 and \002 (following a \001) are copied to
   the returned string; all characters except those between \001 and
   \002 are assumed to be `visible'. */ 

static char *
expand_prompt (pmt, lp, lip, niflp)
     char *pmt;
     int *lp, *lip, *niflp;
{
  char *r, *ret, *p;
  int l, rl, last, ignoring, ninvis, invfl;

  /* Short-circuit if we can. */
  if (strchr (pmt, RL_PROMPT_START_IGNORE) == 0)
    {
      r = savestring (pmt);
      if (lp)
        *lp = strlen (r);
      return r;
    }

  l = strlen (pmt);
  r = ret = (char *)xmalloc (l + 1);

  invfl = 0;    /* invisible chars in first line of prompt */

  for (rl = ignoring = last = ninvis = 0, p = pmt; p && *p; p++)
    {
      /* This code strips the invisible character string markers
         RL_PROMPT_START_IGNORE and RL_PROMPT_END_IGNORE */
      if (*p == RL_PROMPT_START_IGNORE)
        {
          ignoring++;
          continue;
        }
      else if (ignoring && *p == RL_PROMPT_END_IGNORE)
        {
          ignoring = 0;
          last = r - ret - 1;
          continue;
        }
      else
        {
          *r++ = *p;
          if (!ignoring)
            rl++;
          else
            ninvis++;
          if (rl == _rl_screenwidth)
            invfl = ninvis;
        }
    }

  if (rl < _rl_screenwidth)
    invfl = ninvis;

  *r = '\0';
  if (lp)
    *lp = rl;
  if (lip)
    *lip = last;
  if (niflp)
    *niflp = invfl;
  return ret;
}

/* Just strip out RL_PROMPT_START_IGNORE and RL_PROMPT_END_IGNORE from
   PMT and return the rest of PMT. */
char *
_rl_strip_prompt (pmt)
     char *pmt;
{
  char *ret;

  ret = expand_prompt (pmt, (int *)NULL, (int *)NULL, (int *)NULL);
  return ret;
}