Re: [Qemu-devel] [PATCH 07/11] qapi: qapi.py: allow the "'" character be escaped

[Peter Maydell]

On 25 July 2012 20:18, Luiz Capitulino <address@hidden> wrote:
> Peter Maydell <address@hidden> wrote:
>> On 25 July 2012 17:54, Luiz Capitulino <address@hidden> wrote:
>> > --- a/scripts/qapi.py
>> > +++ b/scripts/qapi.py
>> > @@ -21,7 +21,9 @@ def tokenize(data):
>> >          elif data[0] == "'":
>> >              data = data[1:]
>> >              string = ''
>> > -            while data[0] != "'":
>> > +            while True:
>> > +                if data[0] == "'" and string[len(string)-1] != "\\":
>> > +                    break
>> >                  string += data[0]
>> >                  data = data[1:]
>> >              data = data[1:]
>>
>> Won't this cause us to look at string[-1] if
>> the input data has two ' characters in a row?
>
> Non escaped? If you meant '' that's a zero length string and should work, but
> if you meant 'foo '' bar' that's illegal, because ' characters should be 
> escaped.

I meant the zero length string case. yes. We come in with data = "''",
strip the first ' and set string to empty. Then in the first time
in the while loop data[0] is "'" but len(string) is 0 and so we'll
do string[-1] which I think will throw an exception.

...and yep, quick test of a nobbbled qapi-schema.json confirms:
$ python /home/pm215/src/qemu/qemu/scripts/qapi-types.py -h -o "." <
/home/pm215/src/qemu/qemu/qapi-schema.json
Traceback (most recent call last):
  File "/home/pm215/src/qemu/qemu/scripts/qapi-types.py", line 260, in <module>
    exprs = parse_schema(sys.stdin)
  File "/home/pm215/src/qemu/qemu/scripts/qapi.py", line 78, in parse_schema
    expr_eval = evaluate(expr)
  File "/home/pm215/src/qemu/qemu/scripts/qapi.py", line 64, in evaluate
    return parse(map(lambda x: x, tokenize(string)))[0]
  File "/home/pm215/src/qemu/qemu/scripts/qapi.py", line 25, in tokenize
    if data[0] == "'" and string[len(string)-1] != "\\":
IndexError: string index out of range

Try this (very lightly tested but seems to work):
(feel free to do something nicer than raising an exception on
the syntax error, and sorry I'm feeling too lazy to make this
an actual patch email)

Signed-off-by: Peter Maydell <address@hidden>

--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -21,10 +21,16 @@ def tokenize(data):
         elif data[0] == "'":
             data = data[1:]
             string = ''
-            while data[0] != "'":
-                string += data[0]
-                data = data[1:]
-            data = data[1:]
+            while True:
+                pos = data.find("'")
+                if pos == -1:
+                    raise Exception("Mismatched quotes")
+                string += data[0:pos]
+                data = data[pos+1:]
+                if len(string) == 0 or string[-1] != "\\":
+                    # found a ' and it wasn't escaped
+                    break
+                string = string[0:-1] + "'"
             yield string

 def parse(tokens):

(if anybody wants to be able to use '\\' to escape escapes then
this approach is a bit stuffed, of course.)

> PS: Peter, I get claustrophobic when reading emails from you :)

I can add more blank lines if that helps? :-)

-- PMM