Re: forward-sexp when on a floating point number

[Stefan Monnier]

> FWIW, I did something like this for (La)TeX here:
> https://github.com/mbork/tex-plus.el (note: this is still WiP, and
> contains more than just tokenizing TeX).  Either TeX token syntax is
> weird and/or difficult, either I really suck at writing lexers/parsers,
> or this is indeed a (potentially) difficult problem.  (Probably all
> three.)

Don't know about the middle one, but the other two indeed apply.  In the
general case, the esiest way to solve this problem is probably to go
back to a safe earlier state and then lex forward from there.

A "safe earlier state" could be "right after a character which can only
appear at the end of a token" [ of course, there's no guarantee that such
a character exists ].

Also "going back and then lex forward" implies a potential serious
performance problem, so it would require some form of caching.


        Stefan