RE: returning two tokens

[Vincent Zweije]

Buday Gergely <mailto:address@hidden> wrote:

> It's not clear yet what will come on the top of my lexer - I haven't
> figured out if yacc/bison is the right tool, or something
> hand-written.
>
> Another solution is to write a wrapper that transforms this special
> token into two tokens.

> It's a text-to-phoneme transformer, and here x should yield a 'k' and
> an 's'. So I have to stick to it.

A difficult job.

Putting it straight into flex seems to go against what flex
is built for: tokenising, i.e. grouping input characters
into tokens.

Go with the wrapper solution.  Put an extra layer on top of the
scanner that expands letters into constituent sounds.  Shouldn't
be too hard.

On the other hand, at some point you'll have to compose multiple
letters into single sounds.  Do you intend to do that using flex?
If so, you'll probably have to expand the 'x' *before* it goes
into the scanner, instead of after!

Good luck.                                              Vincent.