Re: error: invalid character `' (ASCII 0) near line 164, column 2 (problem avoided)

[Ben Abbott]

On Thursday, December 31, 2009, at 01:04PM, "Michael D Godfrey" 
<address@hidden> wrote:
>
> Ben,
> 
> I just looked on my Linux Fedora 12 system. It uses
> flex 2.3.5.  So, it appears that the Mac version must be broken
> in some way that is not just the wrong version.  So, you
> might try compiling the 2.3.5 version from sourceforge.
> They also have earlier versions available in case 2.3.5
> fails.
> 
> Michael
> 

Michael, do you mean "2.5.35"?

Using 2.5.35 from sourceforge, Octave no longer produces the (ASCII 0) errors!

I took a look at Apple's patches to flex. It looks to me like the one below is 
responsible.

http://www.opensource.apple.com/source/flex/flex-24.1/patches/scanEOF.diff

---------------
--- flex.skl+orig       2006-12-20 12:29:20.000000000 -0800
+++ flex.skl    2006-12-20 12:29:32.000000000 -0800
@@ -1818,7 +1818,7 @@ m4_ifdef( [[M4_YY_USE_LINENO]],
                                case EOB_ACT_END_OF_FILE:
                                        {
                                        if ( yywrap( M4_YY_CALL_ONLY_ARG ) )
-                                               return EOF;
+                                               return 0;
 
                                        if ( ! 
YY_G(yy_did_buffer_switch_on_eof) )
                                                YY_NEW_FILE;
---------------

Looking at my lex.cc produced using the sourceforge version of flex ...

2834 case 89:
2835 YY_RULE_SETUP
2836 #line 1032 "lex.ll"
2837 {
2838     LEXER_DEBUG (".");
2839 
2840     xunput (octave_text[0], octave_text);
2841 
2842     int c = text_yyinput ();
2843 
2844     if (c != EOF)
2845       {
2846         current_input_column++;
2847 
2848         error ("invalid character `%s' (ASCII %d) near line %d, column %d",
2849                undo_string_escape (static_cast<char> (c)), c,
2850                input_line_number, current_input_column);
2851 
2852         return LEXICAL_ERROR;
2853       }
2854     else
2855       TOK_RETURN (END_OF_INPUT);
2856   }
2857         YY_BREAK

When using Apple's flex, I was able to avoid the (ASCII 0) error by commenting 
out lines 2844-2854. Alternatively, we could modify lex.ll so that line 2844 
reads something like …

2844     if (! (c == EOF || (c == 0 && ismac ())))

The lines from lex.ll are below. Line 1039 is what needs to be changed.

1032 . {
1033     LEXER_DEBUG (".");
1034 
1035     xunput (yytext[0], yytext);
1036 
1037     int c = text_yyinput ();
1038 
1039     if (c != EOF)
1040       {
1041         current_input_column++;
1042 
1043         error ("invalid character `%s' (ASCII %d) near line %d, column %d",
1044                undo_string_escape (static_cast<char> (c)), c,
1045                input_line_number, current_input_column);
1046 
1047         return LEXICAL_ERROR;
1048       }
1049     else
1050       TOK_RETURN (END_OF_INPUT);
1051   }

I doubt "ismac()" is available. Forgive my lame attempt at cpp/c++, but would 
replacing line 1039 of lex.ll with the lines below work?

#if defined (__APPLE__) && defined (__MACH__)
     if (! (c == EOF || (c == 0 && ismac ())))
#else /* not MacOS X */
     if (c != EOF)
#endif

Ben