Re: error: invalid character `' (ASCII 0) near line 164, column 2 [changeset]

[Ben Abbott]

On Dec 31, 2009, at 4:40 PM, Ben Abbott wrote:

> On Thursday, December 31, 2009, at 03:38PM, "Ben Abbott" <address@hidden> 
> wrote:
>> 
>> On Thursday, December 31, 2009, at 03:17PM, "Ben Abbott" <address@hidden> 
>> wrote:
>>> On Thursday, December 31, 2009, at 01:04PM, "Michael D Godfrey" 
>>> <address@hidden> wrote:
>>>> 
>>>> Ben,
>>>> 
>>>> I just looked on my Linux Fedora 12 system. It uses
>>>> flex 2.3.5.  So, it appears that the Mac version must be broken
>>>> in some way that is not just the wrong version.  So, you
>>>> might try compiling the 2.3.5 version from sourceforge.
>>>> They also have earlier versions available in case 2.3.5
>>>> fails.
>>>> 
>>>> Michael
>>>> 
>>> 
>>> Michael, do you mean "2.5.35"?
>>> 
>>> Using 2.5.35 from sourceforge, Octave no longer produces the (ASCII 0) 
>>> errors!
>>> 
>>> I took a look at Apple's patches to flex. It looks to me like the one below 
>>> is responsible.
>>> 
>>> http://www.opensource.apple.com/source/flex/flex-24.1/patches/scanEOF.diff
>>> 
>>> ---------------
>>> --- flex.skl+orig   2006-12-20 12:29:20.000000000 -0800
>>> +++ flex.skl        2006-12-20 12:29:32.000000000 -0800
>>> @@ -1818,7 +1818,7 @@ m4_ifdef( [[M4_YY_USE_LINENO]],
>>>                             case EOB_ACT_END_OF_FILE:
>>>                                     {
>>>                                     if ( yywrap( M4_YY_CALL_ONLY_ARG ) )
>>> -                                           return EOF;
>>> +                                           return 0;
>>> 
>>>                                     if ( ! 
>>> YY_G(yy_did_buffer_switch_on_eof) )
>>>                                             YY_NEW_FILE;
>>> ---------------
>>> 
>>> Looking at my lex.cc produced using the sourceforge version of flex ...
>>> 
>>> 2834 case 89:
>>> 2835 YY_RULE_SETUP
>>> 2836 #line 1032 "lex.ll"
>>> 2837 {
>>> 2838     LEXER_DEBUG (".");
>>> 2839 
>>> 2840     xunput (octave_text[0], octave_text);
>>> 2841 
>>> 2842     int c = text_yyinput ();
>>> 2843 
>>> 2844     if (c != EOF)
>>> 2845       {
>>> 2846         current_input_column++;
>>> 2847 
>>> 2848         error ("invalid character `%s' (ASCII %d) near line %d, column 
>>> %d",
>>> 2849                undo_string_escape (static_cast<char> (c)), c,
>>> 2850                input_line_number, current_input_column);
>>> 2851 
>>> 2852         return LEXICAL_ERROR;
>>> 2853       }
>>> 2854     else
>>> 2855       TOK_RETURN (END_OF_INPUT);
>>> 2856   }
>>> 2857         YY_BREAK
>>> 
>>> When using Apple's flex, I was able to avoid the (ASCII 0) error by 
>>> commenting out lines 2844-2854. Alternatively, we could modify lex.ll so 
>>> that line 2844 reads something like …
>>> 
>>> 2844     if (! (c == EOF || (c == 0 && ismac ())))
>>> 
>>> The lines from lex.ll are below. Line 1039 is what needs to be changed.
>>> 
>>> 1032 . {
>>> 1033     LEXER_DEBUG (".");
>>> 1034 
>>> 1035     xunput (yytext[0], yytext);
>>> 1036 
>>> 1037     int c = text_yyinput ();
>>> 1038 
>>> 1039     if (c != EOF)
>>> 1040       {
>>> 1041         current_input_column++;
>>> 1042 
>>> 1043         error ("invalid character `%s' (ASCII %d) near line %d, column 
>>> %d",
>>> 1044                undo_string_escape (static_cast<char> (c)), c,
>>> 1045                input_line_number, current_input_column);
>>> 1046 
>>> 1047         return LEXICAL_ERROR;
>>> 1048       }
>>> 1049     else
>>> 1050       TOK_RETURN (END_OF_INPUT);
>>> 1051   }
>>> 
>>> I doubt "ismac()" is available. Forgive my lame attempt at cpp/c++, but 
>>> would replacing line 1039 of lex.ll with the lines below work?
>>> 
>>> #if defined (__APPLE__) && defined (__MACH__)
>>>    if (! (c == EOF || (c == 0 && ismac ())))
>>> #else /* not MacOS X */
>>>    if (c != EOF)
>>> #endif
>>> 
>>> Ben
>> 
>> Opps, that should have been ...
>> 
>> #if defined (__APPLE__) && defined (__MACH__)
>>    if (! (c == EOF || c == 0))
>> #else /* not MacOS X */
>>    if (c != EOF)
>> #endif
>> 
>> I tried this change, and it worked for me with the sourceforge flex, I'm 
>> building with Apple's now. If it works, I'll post a changeset to the list. 
>> As I don't trust myself with c++, I'd appreciate some feedback.
>> 
>> Ben
>> 
> 
> Using the attached changeset, I'm now able to build successfully using both 
> the flex from sourceforge and the one bundled with MacOS X.
> 
> If someone can give it a sanity check, I'll push it.
> 
> Ben
> <changeset.patch>

I've just noticed that there are many other instances of "if (c == EOF)" and 
such. I haven't studied the code to have any in depth understanding, but I'm 
doubtful this changeset is a proper solution.

Perhaps a more reliable solution is to use the flex from sourceforge?

Ben