bug#15212: 24.3.50; c++-mode doesn't support raw string literals

[Alan Mackenzie]

Hello again, Ivan.

On Tue, May 31, 2016 at 08:22:07AM -0600, Ivan Andrus wrote:
> On May 29, 2016, at 3:36 PM, Alan Mackenzie <acm@muc.de> wrote:

>> I've now got a patch, which I'd be grateful if you could try out, both
>> to see if there are any bugs, and also to get your general impression.
>> I think there are one or two bugs left in the code, and it needs tidying
>> up quite a lot.  So this won't be the final version.


> Awesome.  I’ll keep looking and let you know of any bugs I find.

> I did find one.  According to
> http://en.cppreference.com/w/cpp/language/string_literal the delimiter can
> contain any characters except parentheses, backslash and spaces.

Yes, I've read that and got angry with it.  It's vague - it's not clear
what is meant by "any source character" - the C++11 page in Wikipedia
says that control characters are excluded.  In practice, I suspect it
won't matter all that much - most of the time the delimiter will just be
"\"(" - anybody trying to do anything fancy in the delimiter deserves
everything she gets.  ;-)

> Using square brackets confuses c++-mode though:

> char brackets [] = R"0x22[(foobar)0x22[";

> Now, I’ve never actually seen such a construct in the wild, but it would be
> good to fix it regardless.  The *Messages* buffer shows

> File mode specification error: (invalid-regexp Unmatched [ or [^)
> Error during redisplay: (jit-lock-function 1013) signaled (invalid-regexp
> "Unmatched [ or [^")

> which seems to point to a missing regexp-quote, and indeed it thinks

>   char bar [] = R"YYY*(bar)YYY";

> is a valid string literal.

Yes indeed!  I was doing a regexp search when an ordinary search was
needed (twice).  And a third occasion did indeed need a regexp-quote.
Here's a (supplementary) patch to fix these glitches:



--- cc-engine.el~       2016-05-29 22:21:06.000000000 +0000
+++ cc-engine.el        2016-05-31 20:49:48.000000000 +0000
@@ -5836,7 +5836,7 @@
              id (match-string-no-properties 1))
        (goto-char (1+ open-paren-pos))
        (when (and (not (c-get-char-property open-paren-pos 'syntax-table))
-                  (search-forward-regexp (concat ")" id "\"") nil t))
+                  (search-forward (concat ")" id "\"") nil t))
          (setq close-paren-pos (match-beginning 0)
                close-quote-pos (1- (point))))))
     (and open-quote-pos
@@ -5908,7 +5908,7 @@
              (open-quote (1+ (match-beginning 0)))
              (open-paren (match-end 1))
              )
-         (if (search-forward-regexp (concat ")" id "\"") nil t)
+         (if (search-forward (concat ")" id "\"") nil t)
              (let ((end-string (match-beginning 0))
                    (after-quote (match-end 0))
                    )
--- cc-fonts.el~        2016-05-29 17:49:34.000000000 +0000
+++ cc-fonts.el 2016-05-31 21:01:14.000000000 +0000
@@ -1598,7 +1598,8 @@
        (c-put-font-lock-face (match-beginning 1) (match-end 2)
                              'default)
        (when (search-forward-regexp
-              (concat ")\\(" (match-string-no-properties 2) "\\)\"")
+              (concat ")\\(" (regexp-quote (match-string-no-properties 2))
+                      "\\)\"")
               limit t)
          (c-put-font-lock-face (match-beginning 1) (point)
                                'default)))))



> Moreover, I was somehow able to get it into a bad state where changing the
> delimiters wouldn’t update fontification.  I’ll see if I can come up with a
> recipe for how to reproduce it reliably.

That's happened to me, too.  Maybe it's connected with the above error.
But then again, maybe not.  I'll keep trying to reproduce it, too.


>> The way I am fontifying these is thus:
>> (i) For a correctly terminated raw string, everything between the ( and )
>> inclusive gets string face, everything else just the default face:


>>            R"foo(bar)foo"
>>                 ^^^^^
>>          font-lock-string-face.


> I was wondering how this would work.  It’s a little weird that regular
> string delimiters are fontified with font-lock-string-face, but these
> aren’t.  But I think I like this way better since it’s much easier to
> confuse these delimiters with the contents of the string than normal string
> delimiters.

Glad you like it!

[ .... ]

>> Thanks a bunch for your work on this.

Thank you!

> -Ivan

-- 
Alan Mackenzie (Nuremberg, Germany).