Re: Parse inline scheme using per-expression port (issue 557330043 by address@hidden)

[dak]

On 2020/02/09 17:18:19, hanwenn wrote:
> David, please take another look.

First, you are aware that the rationale for

    This commit fixes a problem with GUILE 2.2.6, where LilyPond
    calculates offsets in the source file as bytes, while GUILE
interprets
    the source file as UTF-8 encoded Unicode. As a result, files with
    Unicode before embedded scheme break completely.

is fixed in current master?  So we are, due to the late merges into
master of our Guilev2 related work, in the somewhat unhappy situation
that both you and I have worked on the problem and produced different
solutions, you mainly by changing approach and writing a different
function that essentially has the same problem but does not get thrown
permanently out of synch by just restarting every time.  My own solution
basically was obtained by several iterations of pestering the Guile
developers to get something fixed.  It currently is in a form that has
gotten their active blessing.

Now my original solution was aimed at staying in binary even during
Scheme times while Antonio Ospite does go all-in on UTF-8 in Scheme.  So
the result of combining my and his work is a chimæra which I am less
confident about than my own code on its own.  Being in binary mode for
Flex means that we can deal robustly with code that is not properly in
UTF-8.  For example, we had input files that were ASCII-only except for
some comments containing latters in the Latin-1 plane.  In binary mode,
our lexer deals gracefully with such abominations while Guile balks at
such files when doing UTF-8 processing.

I'll take a look at some test cases with either version and see whether
I can trigger obvious problems.  That would not really make an
evaluation of code quality, just give some better idea whether one of
the approaches would warrant further work before getting selected.

https://codereview.appspot.com/557330043/