Re: qmp-shell for GSoC/Outreachy?

[John Snow]

On 2/6/20 4:40 AM, Markus Armbruster wrote:
> "Dr. David Alan Gilbert" <address@hidden> writes:
> 
>> * John Snow (address@hidden) wrote:
>>> I'm forking the subject as I believe Markus wanted to focus on the
>>> machine interface aspect.
>>>
>>> I feel that a new human interface is *related* to that goal: the
>>> splitting of, and commitment to, separate human and machine interfaces
>>> powered by a single root schema.
> 
> A bit of history.
> 
> QMP initially shared HMP's "schema": qemu-monitor.hx.  The command
> handler was either a traditional HMP one, or a QMP-enabling pair (QMP
> handler, HMP formatter).  The idea was to convert traditional HMP
> handlers one by one, then ditch support for them.
> 
> With a QMP handler, HMP became a wrapper around QMP.  The input wrapping
> was data-driven: @args_type specifies how to map HMP arguments to QMP.
> The output wrapper was code, namely the HMP formatter.
> 
> This design turned out to tie QMP to HMP too tightly.  It assumes QMP
> and HMP commands are identical apart from argument syntax and output
> formatting.  They often should not be.  QMP wants building blocks:
> simple commands with simple replies, in particular simple failure modes.
> HMP wants convenience.  QMP wants rigor.  HMP has uses where that's a
> painful and unnecessary.
> 
> So we split qemu-monitor.hx into hmp-commands.hx and qmp-commands.hx.
> The former reverted back to the pre-QMP state, and the latter lost
> support for HMP wrappers.  QMP was liberated from having to reimplement
> HMP.  HMP was liberated from always having to do QMP first.
> 
> qmp-commands.hx was eventually replaced by the QAPI schema.
> 
> The lesson here is that to make "powered by a single root schema" work
> well, we'll likely have to put in more smarts than we did back then.
> 

Yes, understood: this is rigorous work that we've failed at before. It's
a big task; but I feel like we are increasingly hurt by our focus on QMP
and leaving HMP in the dust.

Unifying them might be the only reasonable forward-thinking thing, even
if it's hard.

> More on that below in reply to David's reply.
> 
>>> I am a big believer in making QEMU usable directly to human users as I
>>> feel the pipeline of "tinker to deployment" is important for a
>>> successful project, for many reasons:
>>>
>>> - QEMU should be easy to pick up and learn.
>>>
>>> - Supporting QEMU's use directly as an "end-user" program increases
>>> proficiency in the user population at large, which (can) lead to better
>>> answers and engagement on e.g. Reddit, StackOverflow, IRC
>>>
>>> - Evolving deployments from QEMU-only to libvirt+ or above (RHV, oVirt,
>>> kubevirt) should be a smooth and gradual process as additional
>>> complexity is desired.
>>>
>>> - Focusing on QEMU's usability allows our project to be consumed easier
>>> by new cloud-focused projects. If they are already familiar with (and
>>> happy with) our project, it is more likely to be used instead of seeking
>>> out alternatives. This is about reducing friction.
>>>
>>> So, for those reasons ... even though I feel that a machine-focused API
>>> is our #1 priority as it caters to our existing users, we should also
>>> focus on what it will take to grow mindshare for QEMU's value in the
>>> ecosystem.
>>>
>>> Slick interfaces and documentation go a long, long way to doing that.
>>>
>>> So: I feel that any new machine-only paradigm or overhaul needs to be
>>> accompanied with some new sugar to help the medicine go down, so-to-speak.
> 
> Points taken.
> 
>>> On 2/5/20 8:09 AM, Kevin Wolf wrote:
>>>> Am 28.01.2020 um 11:59 hat Kevin Wolf geschrieben:
>>>>>>> The other part that it needs to solve is how to be available by default
>>>>>>> without specifying anything on the command line. Basically, if I press
>>>>>>> Ctrl-Alt-2, I want to get to a monitor shell. If that shell is
>>>>>>> implemented internally or by an external Python process, I don't mind.
>>>>>>
>>>>>> That is a harder part. (I rarely use Ctrl-Alt-2 actually; I mostly
>>>>>> use HMP on stdin).
>>>>>
>>>>> I don't think it would be that hard, actually.
>>>>>
>>>>> If you have a -qmp-shell option that takes a chardev and defaults to vc,
>>>>> you've solved the part with both stdio and Ctrl-Alt-2. Now all you need
>>>>> to do is launch the Python child process, pass it a pair of pipes for
>>>>> communication and forward everything between the pipes and the chardev.
>>>>>
>>>>> (That's the theory anyway.)
>>>>
>>>> If someone is interested, I did a quick proof-of-concept hack:
>>>>
>>>>     https://repo.or.cz/qemu/kevin.git/shortlog/refs/heads/qmp-shell
>>>>
>>>> It doesn't clean up anything properly (including the qmp-shell processes
>>>> it starts), but it spawns a usable qmp-shell on a user-specified
>>>> character device. stdio seems to work, though without readline
>>>> functionality (I suppose I still have line-buffering somewhere), vc
>>>> doesn't really work at all yet.
>>>>
>>>> Try it out like this:
>>>>
>>>>     $ ./qemu-storage-daemon --chardev stdio,id=m --monitor m,mode=qmp-shell
>>>>     monitor_qmp_event: 1
>>>>     Welcome to the QMP low-level shell!
>>>>     Connected to QEMU 4.2.50
>>>>
>>>>     (QEMU) query-version
>>>>     {"return": {"qemu": {"micro": 50, "major": 4, "minor": 2}, "package": 
>>>> "v4.2.0-1188-gd95a3885a9"}}
>>>>     (QEMU) quit
>>>>
>>>> (Or use x86_64-softmmu/qemu-system-x86_64, but it's based on the
>>>> refactorings in the storage daemon branch, so why not try both at once?)
>>>>
>>>> Polishing this to make it mergable would still require substantial work,
>>>> so at the moment I'm not planning to do this. But if someone wants to
>>>> pick it up, feel free (just let us know).
>>>>
>>>> Hm, in fact... A qmp-shell GSoC project?
>>>>
>>>
>>> That would be great. I worry that we should have a clear vision for the
>>> syntax before we give this project to an intern, though. With a clear
>>> vision and an outline for deliverables, it's an incredibly appropriate
>>> project.
>>>
>>> Some things I think we want to define before we start:
>>>
>>> 1. What are we trying to achieve with a standalone shell?
> 
> Projects without a clear goal rarely succeed.  Success within three
> months is even rarer.
> 
>>> 2. What syntax should it use?
> 
> Leaving that to a GSoC student amounts to setting up for failure.
> 
>>> I think those are the hardest parts.
>>>
>>> Below, some musings:
>>>
>>> - An integrated QMP shell would be a great usability boost to users of
>>> bare QEMU.
>>>
>>> - It is undesirable in general to support two interfaces. Feature
>>> disparity is a problem, as is needing to document and test two separate
>>> interfaces. The quality disparity between the two is also an issue.
>>>
>>> - Offering HMP via the GTK interface but not QMP is a discoverability
>>> problem. Unfamiliar users might assume that HMP is our flagship
>>> interface. It is not.
>>>
>>> - We are unlikely to re-expand HMP to cover everything QMP does; writing
>>> a QMP shell that makes QMP easy to interface with is a better solution
>>> for removing redundancy and complexity.
>>>
>>> - I suspect that the target audience for users of naked QEMU are:
>>>   - QEMU developers
>>>   - Upper-layer developers (RHV, oVirt, KubeVirt, libvirt, kata, et al)
>>> researching, testing, and debugging integration.
>>>   - Devops professionals testing, implementing and debugging
>>>     configuration & infrastructure
>>>   - Security/infosec researchers
>>>   - Embedded platform developers
>>>   - Academic researchers
>>>
>>>
>>>
>>> So please correct me if I am off the mark;
>>>
>>> Design Goals:
>>>   - The removal of HMP
>>>   - An easy-to-use interface that remains reasonably "close" to the
>>> machine API such that it provides a smooth transition to scripting QEMU.
>>>   - Integration with our GTK interface for discoverability and convenience
>>>
>>> Syntax:
>>>   - TBD? Do we agree that the current syntax in qmp-shell is "bad" and
>>> should be replaced? If yes, what should it look like?
>>
>> I believe it should be a python shell with added commands.
>>
>> Simple things should be simple.
>>   e.g. adding a disk from a local file should be trivial.
>>
>> Complex things can be complex - but it would be better if they were
>> simple.
>>
>>   It's OK if the worst case of a blockdev is a bit hairy, but
>>   watch out for cases where the hairyness creeps in unnecessarily.
> 
> Designing interfaces to complex machinery is hard.  Experience tells
> that we do okay when we focus on the building blocks first.  That's
> -blockdev.  When we start with trying to make simple things simple, we
> end in swamps.  That's -drive.
> 
> Focus on building blocks is of course no excuse for unnecessary
> hairiness.
> 
> It's also no reason not to build more convenient things on top of the
> building blocks.  I doubt they should go into QMP, though.
> 
>> If the user screwsup, it should give an error that prompts the user
>> to the parameter they got wrong.
>>
>> Output from commands should normally be pretty formatted (with an option
>> to display raw json for those needing it).
>>   e.g. that 'query-version' should give either just the package
>>   version (as info version currently does) or:
>>       4.2.50  Package: v4.2.0-1188-gd95a3885a9
>>
>> We shouldn't lose any HMP commands that some people find useful
>>   Ditching HMP isn't an option until we've got almost all of it
>>   covered.
> 
> In particular, we currently use HMP for debugging and monitoring
> purposes, where we don't need or want QMP's rigor, neither its rigorous
> interface stability, nor its structured I/O.  We want the "whipuptitude"
> we get from monitor_printf().  This is actually a point David has made
> several times.
> 

Yes, the "whipupitude" is something I am keen on preserving -- I just
want that whipupitude to be based on a schema.

Can we accomplish this with qmp-shell plugins that use special QMP
interfaces? I think so.

I'm imagining a special feature flag for "experimental" interfaces in
QMP that aren't thought through enough to become real interfaces that
these sugar commands -- implemented primarily in the shell - use.

They might do some bad things: they're synchronous, they might return a
bit too much data, they might return too much data. But it's only for
debug use, and we declare the API unstable/fluid, so we don't care.

Maybe this is a fool's errand, though, and we SHOULD just keep HMP.
Maybe we just need to QAPI-fy our HMP shell so that we can document it
better, and expand QAPI power to describe the format of HMP commands.

The core issue I take with HMP is that it's underdocumented, tends to
rot, and rarely keeps up with configuration idiom changes in QEMU -- it
*feels* like a legacy interface, not an up-to-date debugging interface.

However we fix this is actually not personally important to me.


> To have a qmp-shell replace HMP, I think it needs to be able to
> 
> * Go beyond 1:1
> 
>   We tried a 1:1 mapping between HMP and QMP commands, and it didn't
>   work out.  HMP's replacement should let us build convenient commands
>   from QMP building blocks.
> 
>   We tried a 1:1 mapping between HMP and QMP command arguments, guided
>   by @args_type.  Worked out for simple cases, but was too constricting.
> 
> * Preserve "whipuptitude" [David]
> 
>   I figure that means allowing some in QMP.  Without compromising its
>   core mission, of course.
> 
> * As discoverable as HMP is now [Kevin]
> 
> * Help, completion and such at least on par with what HMP provides now
> 
> Highly desirable:
> 
> * Support transitioning to the machine interface [John]
> 
>   Let humans start playing with the human interface, and when they feel
>   the need to automate, help them transition to QMP.
> 
> Back to John's question on qmp-shell syntax, which hasn't been answered
> so far.
> 
> JSON is a data-interchange format.  It doesn't try to be a configuration
> format or programming language syntax for human use.  It gets pressed
> into these roles with entirely predictable poor results.
> 

More or less; but it's still pretty decent as a format if you're editing
it in an IDE that can assist you in doing so.

No editor I am aware of at present is capable of editing JSON or YAML
against a schema, though: the prevailing paradigm appears to be:

- Write file in editor
- Run some validator command
- Deploy and pray

>From what I can tell, this is how it works in the cloud world, too.

> Pain points of JSON include having to count parenthesises and having to
> quote so bloody much.  Additional QMP pain points include long names and
> excessive nesting.
> 
> For the configuration format role, more usable alternatives exist.  YAML
> is a popular one.
> 

Yes, but doesn't solve our parsing issue for interactive shells.

> qmp-shell is a REPL.  It needs a REPL-friendly syntax.  I doubt YAML is
> or even tries to be REPL-friendly.  I'd love to be wrong; the first rule
> of language design is "don't".
> 

Yes.

> Other language suggestions?
> 
> On making JSON suck less in this role:
> 
> LISP REPLs demonstrate that computers can assist effectively with
> counting parenthesises, and with completing long names.
> 
> We could make quoting optional for sufficiently nice object member
> names.  QAPI naming rules ensure niceness, actually.
> 
> We could make quoting optional for certain string literals.  Simple
> enough for literals that can only be a string, like abc.  For literals
> that could be something else like 123 or true, omitting quotes creates
> ambiguity.  When the schema accepts only one of the possible types, the
> ambiguity goes away.  Complexity stays, however.
> 
> Excessive nesting should ideally be attacked in QMP itself, but backward
> compatibility makes that hard.
> 

I wish I had a good idea on how to make a QMP wrapper that works in a
REPL that didn't involve copy/pasting JSON. I guess it's why so many
people *do* just cat data into the socket.