How about this, then... (Not quite non-greedy wildcard)

[Magnus Lie Hetland]

Sorry for being so pig-headed about using Bison for this problem
<wink> but here is what seems like the most promising option at
present is, perhaps, the following:

  1. Write code (using Bison functionality) to detect all
     possible/legal lookahead tokens.

  2. Give Flex access to this functionality (either through a tie-in
     bitvector or by letting it call the function directly).

  3. Have Flex skip all illegal tokens (and output them).

This is basically more or less the same as you suggested, except (if I
understood your suggestion) for point 1. The set of legal
lookahead-tokens will, of course, vary, as opposed to a global set of
posisble tokens.

Now, I've found some code for this, although I don't quite understand
it yet (I've just started looking at it):

http://mail.gnu.org/archive/html/help-bison/2002-10/msg00057.html

It seems to me that this should solve my problem (or, at least, the
simpler version, where a single token is all that is needed to end a
plain text segment).

Does it seem feasible to let Flex call the relevant function as part
of its lexing (rather than using a global variable to represent the
result and calling the function in actions all over the place)?

(And: This functionality had better be in yylex -- how do I
modify/wrap yylex in Flex without using copy-paste "inheritance"?
Maybe this is the wrong list to ask about that...)

In addition, I'd like to do error recovery to convert all failed
markup segments to plain text (for robustness), but I guess I'll deal
with that later :)

- M


-- 
Magnus Lie Hetland              "Wake up!"  - Rage Against The Machine
http://hetland.org              "Shut up!"  - Linkin Park