Re: BOM mark from Windows notepad

[Bertalan Fodor (LilyPondTool)]

Hehe, we've got this:

<INITIAL,chords,lyrics,figures,notes>{BOM_UTF8}/.* {
 if (this->lexloc->line_number () != 1 || this->lexloc->column_number 
() != 0)
   {
     LexerError (_ ("stray UTF-8 BOM encountered").c_str ());
     exit (1);

That means, we correctly parse the BOM, but exit if it is not the first 
char.

Hans Aberg wrote:
> On 13 Nov 2009, at 10:08, Bertalan Fodor (LilyPondTool) wrote:
>
> > I think changing the LilyPond parser to support BOM in the middle (ie 
> > not at the beginning) of the file is very hard. Actually if it is not 
> > at the beginning, then it should be treated as a regular character, 
> > which might not be present just anywhere in the file.
>
> Why would that be? Did you not have a Flex generated .l file? If the 
> input .l file is in UTF-8 and Flex in 8-bit mode, add a rule
>   "<BOM>" {}
> where <BOM> is the UTF-8 representation of the BOM. It will than add 
> act as space, breaking tokens, but otherwise ignored. So it acts a 
> zero-width space.
>
>   Hans