RE: bugs found on bison/m4 on windows

[Kees Dekker]

Hi,

Thanks for very fast response. Unfortunately, I do not know a bug reporting
mailaddress for the 'windows guys', like Stepan suggested to me (in another
mail).. Probably you can help me. Because I was off yesterday for a part of
the day, my response is a little bit delayed. I hope you will apply
suggested solutions below.

About the first problem:
The problem is in bison (due to forgetting to strip some quotes), see below,
cases a. and c AND in m4, cases b. and d.
Possible cases:
a. set BISON_PKGDATADIR="dir with spaces but with quotes"
b. set BISON_PKGDATADIR=dir with spaces without quotes
c. set BISON_PKGDATADIR="dir_withhout_spaces_with_quotes"
d. set BISON_PKGDATADIR=dir_withhout_spaces_without_quotes

It is of course possible to remove leading and trailing spaces in bison (for
both M4 as well as BISON_PKGDATADIR), but actually I think it is a user
fault (I changed my mind). You may like to add a (currently not existing)
unquote function to unquote pkgdatadir and/or m4 in output.c:548
(output_skeleton()). So cases a. and c. should be corrected by user.
However, m4 do not like case b. Therefore, you may think about skipping a
fix for this. If you like, I can write a unquote function if needed. But for
now, I can live with the current behavior of bison regarding
BISON_PKGDATADIR.

Unfortunatelly, my build of M4(sources) (with CygWin) do not have the
problems for cases b. and d. (see above). Only the binary version of m4.exe,
downloaded from http://gnuwin32.sourceforge.net/packages/m4.htm has problems
when BISON_PKGDATADIR contains spaces (and no surrounding quotes), but I
cannot find a email address for who will pick up bug reports... The email
address in the README file, provided with the sources is
'address@hidden', but that email address is not mentioned on
the mentioned home page for m4 (http://www.seindal.dk/rene/gnu).

About the second problem (in M4) for binary/text mode read:
Problem can be easily reproduced if yacc.c is in MS-DOS text mode (at
default, when installing bison, yacc.c is in Unix style text mode). So if M4
is openening files in text mode (rather than bin mode) it is ok (tested by
me). Note that if no 't' or 'b' modifier is specified in fopen(), and if
msvcrt.dll is used, and if _fmode (a global variable) is not set, that fopen
opens default in text mode (i.e. similar to what is wanted). To prevent
incompatibility with Unix systems, I suggest to remove the current 'b'
modifier in the fopen() calls in patch_search() (path.c:107 and 127). For
more info about fopen on Windows, see:
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/vclib/html/
_crt_fopen.2c_._wfopen.asp

However, when building whith cygwin, fopen seems not to understand textmode,
so in that case, file is still untranslated, and \r\n is sent to bison. Just
for reliability, please also consider next change (scan-skel.l:47-), i.e.
change from strcmp to strncmp to prevent string compare problems if \r\n is
at the end of the string.

old code:

"@output ".*\n {
  char const *filename = yytext + sizeof "@output " - 1;
  yytext[yyleng - 1] = '\0';

  if (*filename == '@')
    {
      if (strcmp (filename, "@output_header_name@") == 0)
        filename = spec_defines_file;
      else if (strcmp (filename, "@output_parser_name@") == 0)
        filename = parser_file_name;
      else
        abort ();
    }

new code:

"@output ".*\n {
  char const *filename = yytext + sizeof "@output " - 1;
  int cmplen;

  /* skip \r if \r\n pattern (DOS-textstyle) found at the end of the string
*/
  if ((yytext[yyleng - 2] = '\r') && (yytext[yyleng - 1] = '\n'))
    cmplen = yyleng - 1;
  else
    cmplen = yyleng;

  yytext[yyleng - 1] = '\0';

  if (*filename == '@')
  {
    if (strncmp (filename, "@output_header_name@", cmplen) == 0)
        filename = spec_defines_file;
    else if (strncmp (filename, "@output_parser_name@", cmplen) == 0)
        filename = parser_file_name;
    else
        abort ();
  }

Unfortunately, I could not test the change above, since my build environment
(cyfwin) is not properly working.

BTW. Most Unix systems already have (f)lex/yacc, so usually we are using
bison on the platforms which doesn't have a good lexer/yacc (i.e. Windows
:-)).

Regards,
Kees Dekker

-----Original Message-----
From: Paul Eggert [mailto:address@hidden
Sent: Thursday, July 01, 2004 00:28
To: Kees Dekker
Cc: 'address@hidden'; 'address@hidden'
Subject: Re: bugs found on bison/m4 on windows


Kees Dekker <address@hidden> writes:

> 1. m4 is not able to open a file from a directory with spaces if
> BISON_PKGDATADIR dir contains spaces. I guess this could be easily fixed
by
> escaping the arguments when m4 is called from bison in output.c, function
> output().

This sounds like a bug in the Microsoft Windows port, not in Bison
itself.  Bison simply passes the strings to execvp, and if they
contain spaces execvp should do the correct thing with them.  My guess
is that your execve is busted, or your modified version of Bison
didn't use execve, or something like that.  Please contact the people
who did your Microsoft Windows port and notify them of the problem.

> 1. open files in m4 in text mode instead of binary mode

Yes, that sounds like the correct fix to the second problem.  I
suspect the problem is that Bison is creating these streams with the
"pipe" system call, and that on Microsoft Windows "pipe" initially
puts file descriptors in binary mode, but when Bison uses fdopen (fd,
"r") and reads the files in (stdio) text mode, the resulting behavior
is undefined.  Does this diagnosis appear correct to you?  If so, I'll
modify Bison to change the pipe file-descriptors to use text mode,
before invoking fdopen.

Thanks.