Embedded modifiers in the regex engine

[Dima Kogan]

Hi.

I've been thinking of ways to make some fancier aspects of isearch and
hi-lock work better, specifically, the way we handle the different
modes: case-fold, char-fold, lax-whitespace, etc.

The relevant bugs I filed recently:

  http://debbugs.gnu.org/22541
  http://debbugs.gnu.org/22520
  http://debbugs.gnu.org/22479

In short, different parts of emacs (isearch, isearch history, hi-lock,
etc) treat these modes inconsistently, which results in unexpected
behavior.

The best solution I can think of to clean this up is also the most
intrusive: adding support for pcre-style embedded modifiers to
activate/deactivate the modes.

So for instance "\\(?i\\)asdf" would be interpreted as a case-folding
regex regardless of the value of case-fold-search. I think this would be
a great thing to have in general, but for the specific issues in the
bugs above, it'd make things simpler and more correct.

As an example, currently hi-lock generates a complicated-looking regex
to emulate char-folding and case-folding. If we supported the modifiers,
this change would simply be a prepend of "\\(?i\\)" or whatever other
modes we want. This is simple and expected to be bug-free on the hi-lock
level. Bugs such as hi-lock not supporting char-fold and case-fold at
the same time would not happen.

Clearly this is a big change to a core component, so I want to talk
about it first. I looked at our regex implementation, and it looks
possible to add this. But I've seen talk of merging our regex
implementation with the glibc one, so the merge should clearly happen
first.

Also I don't want to touch this without a test suite for our regex
engine. So that would need to happen beforehand as well.

Again, I think this feature would be useful even beyond the context of
these bugs. Thanks for the input.

dima