Re: [PATCH 00/29] Convert QAPI doc comments to generate rST instead of texinfo

[Markus Armbruster]

Peter Maydell <address@hidden> writes:

> On Fri, 7 Feb 2020 at 17:00, Markus Armbruster <address@hidden> wrote:
>>
>> Peter Maydell <address@hidden> writes:
>>
>> > This series switches all our QAPI doc comments over from
>> > texinfo format to rST.
>> >
>> > The basic approach is somewhat similar to how we deal with kerneldoc
>> > and hxtool: we have a custom Sphinx extension which is passed a
>> > filename which is the json file it should run the QAPI parser over and
>> > generate documentation for. Unlike 'kerneldoc' but somewhat like
>> > hxtool, I have chosed to generate documentation by generating a tree
>> > of docutils nodes, rather than by generating rST source that is then
>> > fed to the rST parser to generate docutils nodes.  Individual lumps of
>> > doc comment go to the rST parser, but the structured parts we render
>> > directly. This makes it easier to get the structure and heading level
>> > nesting correct.
>> >
>> > Rather than trying to exactly handle all the existing comments I have
>> > opted (as Markus suggested) to tweak them where this seemed more
>> > sensible than contorting the rST generator to deal with
>> > weirdnesses. The principal changes are:
>> >  * whitespace is now significant, and multiline definitions must have
>> >    their second and subsequent lines indented to match the first line
>> >  * general rST format markup is permitted, not just the small set of
>> >    markup the old texinfo generator handled. For most things (notably
>> >    bulleted and itemized lists) the old format is the same as rST was.
>> >  * Specific things that might trip people up:
>> >    - instead of *bold* and _italic_ rST has **bold** and *italic*
>>
>> Actually, qapi-code-gen.txt documents and doc.py implements *strong* and
>> _emphasis_.  Texinfo commonly renders them as bold and italic when the
>> output format supports that.  rST has **strong** and *emphasis*.
>>
>> Your series adjusts emphasis markup for rST [PATCH 18].  Since it
>> doesn't touch strong markup, strong silently becomes emphasis.  I guess
>> that's okay, perhaps even an improvement, but double-checking the actual
>> uses of this markup wouldn't hurt.
>
> Yeah, that would be a good plan.
>  git grep '\*[^*]*\*' qapi/*.json
> (and eyeball-filtering out the false hits) shows just one use:
>
> qapi/introspect.json:# Note: the QAPI schema is also used to help
> define *internal*
>
> I can put a patch on the end which converts that to **internal**
> once the rST generator is in use.

I wrote that one, and I think changing it from strong to emphasis is an
improvement.  Let's point to it in the commit message, and call it a
day.

>> >    - lists need a preceding and following blank line
>> >    - a lone literal '*' will need to be backslash-escaped to
>> >      avoid a rST syntax error
>> >  * the old leading '|' for example (literal text) blocks is replaced
>> >    by the standard rST '::' literal block.
>> >  * headings and subheadings must now be in a freeform documentation
>> >    comment of their own
>>
>> Can we simply use rST instead?  See my review of PATCH 18.
>
> No, we can't (see my reply to that review). In theory you could have
> the heading syntax be a rST heading, but that is not feasible to
> recognise in the Python script[*] and it gives the impression that
> it is just an inline rST heading, not something that's more complicated
> and structured.
>
> [*] You'd need to manually re-implement the weird thing rST does
> where practically any kind of underlining is valid and it figures
> out which one means which depth by looking at the usage through
> the whole document. You'd have to do bizarre stuff like running
> through the entire set of doc comments once doing no output but
> just looking at heading underline characters to guess which one
> is which depth, and then once you'd figured that out you could
> do it all over again actually generating the doc.

I understand the difficulty of parsing rST (Paolo called it "the Perl of
markup languages" for a reason).  What I don't yet understand is (1) why
we need to recognize the document structure of doc comments, and (2) why
we can do that by recognizing '=' markup, but ignore the native rST
document structure markup.