Re: [Qemu-devel] RFC: libyajl for JSON

[Markus Armbruster]

Eric Blake <address@hidden> writes:

> On 11/02/2015 12:10 PM, Markus Armbruster wrote:
>
>>>> * Single-quoted strings
>
>> 
>>> So that is an absolute must for whatever parser we choose.  My first
>>> glance at libyajl is that it does NOT currently allow single quotes (not
>>> even with any of its existing options), so we'd have to pre-process
>>> input before feeding it to yajl.
>
>>>
>>> I'll ask on the yajl mailing lists, to get a feel for community
>>> receptiveness, before attempting to actually write patches on that front.
>> 
>> Makes sense.
>
> On IRC, I got pointed to a couple of forks that have already done this,
> such as:
>
> https://github.com/ludocode/yajl/commits/master
>
> although the upstream author Lloyd was not on the channel at the time.
> Meanwhile, there's 47 open issues on the upstream repo:
>
> https://github.com/lloyd/yajl/issues
>
> which implies slow response by the author, but at least he WAS asking if
> anyone would like to help him with maintenance:
>
> https://github.com/lloyd/yajl/issues/161
>
> |  lloyd commented on Sep 24
> | ok, give me a minute to hand you a patch on a branch to review.
>
> |  lloyd commented on Sep 24
> | ok, I'll merge down and roll a new release within a day.
>
> [still hasn't happened yet...]
>
> |  lamont-granquist commented on Sep 25
> | hey @lloyd would you consider adding other maintainers?
>
> |  lloyd commented on Oct 1
> | I would absolutely consider additional maintainers. Who's interested?
>
> so I don't know what the future holds for extending things upstream.
>
> To date, I don't have a github account, by conscious personal choice; so
> I can't really take him up on that offer directly.  So far, I've been
> trying the mailing list to see if he answers that in addition to the
> github PR stream, but the archives show it to be pretty full of spam:
> http://librelist.com/browser/yajl/

I don't do github, either.

>>> Yes; the yajl parser has 4 modes of parse operation based on which of
>>> three callbacks you implement: double-only (yajl_double), long long-only
>>> (yajl_integer), double-and-int (both yajl_double and yajl_integer, not
>>> sure which one has precedence if input satisfies both formats), or
>>> custom number (yajl_number, which is given a string, and you turn it
>>> into the format of your choice).  Likewise, the yajl formatter has 2
>>> functions: yajl_gen_double() (formats doubles) and yajl_gen_number()
>>> (you provide literal strings formatted how you like).
>> 
>> Good.
>
> And one of the open bugs on the tracker was the same one we've
> encountered ourselves: yajl is locale-sensitive when using strtod() for
> parsing and when using printf() for printing doubles:
>
> https://github.com/lloyd/yajl/issues/79
>
> [I would love for C/POSIX to add strtod_l and printf_l, but that's a
> thread for another day]
>
>> 
>>>> More extensions or pitfalls might be lurking in our parser.  Therefore,
>>>> replacing our parser by a suitable library is not without risk.  I guess
>>>> we could do it over a full development cycle.  No way for 2.5.
>>>>
>>>
>>> Absolutely not for 2.5; we've missed softfreeze, and this would count as
>>> a new feature.
>
> Another interesting bug report against yajl: one reporter made the point
> [1] that yajl is already a superset of the canonical RFC 4627 definition
> of JSON [2], because while the RFC states that a JSON document has this
> terminal state:
>       JSON-text = object / array
> yajl will accept _any_ JSON value (not just objects/arrays) as the
> overall document.
>
> [1] https://github.com/lloyd/yajl/issues/154
> [2] https://www.ietf.org/rfc/rfc4627.txt

I believe our parser does the same.

> So at this point, I want to see if lloyd makes any progress towards an
> actual yajl release and/or adding a co-maintainer, before even trying to
> get formal upstream support for single quoting.  We could always create
> a git submodule with our own choice of fork (since there are already
> forks that do single-quote parsing) - but the mantra of 'upstream first'
> has a lot of merit (I'm reluctant to fork without good reason).

The value proposition of replacing our flawed JSON parser isn't in
saving big on maintenance, it's in not having to find and fix its flaws.

If the replacement needs a lot of work to fit our needs, the value
proposition becomes negative.

A JSON parser shouldn't require much maintenance, as JSON is simple,
doesn't change, and parsing has few system dependencies.

That said, maintaining fork of an unresponsive upstream is always
awkward.

Wait and see how yajl makes progress sounds sensible to me.  However, I
already hate the idea of releasing 2.5 with the colossal memory waste
unfixed.  I'm willing to sit on my hands for 2.5 only because it's not a
regression.  I'm unwilling to leave it unfixed much longer.

Point is, the longer we wait, the more we'll get invested in our own
parser.