Re: [Chicken-users] Basic abnf usage?

[Matt Gushee]

Hi, Moritz--

On Sat, Mar 28, 2015 at 5:33 AM, Moritz Heidkamp <address@hidden> wrote:

ah, that's what you are referring to, I see! It's like that because I
didn't want to force a utf8 dependency on the user.

Maybe in that case it would be good if the API doc said something like: "comparse is compatible with UTF-8, but many of the built-in combinators do not work with UTF-8 characters, so you may need to construct your own. For example: ..."

 

This doesn't work because `sake' is a string and Comparse operates on
the byte level by default (in accordance with CHICKEN core string
procedures).

Well, yes. And, though I understand that if you *know* your programs only need to process single-byte characters, it is convenient (and better performance-wise, though I wonder how much in the year 2015) to equate characters with bytes. I'm of the opinion (shared by many I18n experts, if I'm not mistaken) that a high-level language in the 21st century should have in its core a rock-solid character abstraction that is never, ever conflated with a byte. There are a lot of things I love about Chicken, but the (IMHO obsolete) string implementation is not one of them.

 

We could create a comparse-utf8 egg to facilitate this. It's not
currently on my agenda but I will put it in my Comparse notes for future
reference. If you feel inclined to create one, I'm happy to provide you
with code review and feedback!

I was thinking about that. I looked through the source code to see what string handling functions are used that are not provided by the utf8 egg, and thus would need to be reimplemented. So far I've found:

    substring/shared

    string-kmp-partial-search

    make-kmp-restart-vector

Substring/shared is not too big a deal, but that KMP stuff is a bit daunting. Maybe I'll look into it if I have time. I do like the comparse API, and would like to be able to use it.

--

Matt