Re: [Qemu-block] Addressability of Block-jobs after bdrv_swap removal

[John Snow]

On 12/10/2015 05:44 AM, Kevin Wolf wrote:
> Am 09.12.2015 um 21:59 hat John Snow geschrieben:
>> I have a question about how the device name of block jobs gets reported
>> now after the bdrv_swap removal patch.
>>
>> Previously, it appears that the job stayed attached to the root-node (if
>> somewhat hackishly, apparently) so that we could at all stages report
>> our name without having to cache the device we started under.
> 
> Yes, I think that was effectively what happened.
> 
> I don't remember the exact details, but what I remember from the
> bdrv_swap() removal work is that it was confusing. After removing
> bdrv_swap() doing it either way turned out wrong because the job had to
> stay at the same BDS and at the same time at the root, too, so that the
> device name would continue to exist. Doing both is impossible without
> swapping the C objects.
> 
>> However, since QMP commands refer to block-jobs solely through their
>> device name, do we have any cases post-removal where a job becomes
>> "unreachable" through its advertised name?
>>
>> e.g.
>>
>> the block-job-ready event uses the device name to advertise which job
>> has just converged. The user/client would then be responsible for
>> sending qmp-block-job-complete device=XXX to finish the job when desired.
>>
>> I don't see one at a quick glance, but we don't have any cases where we
>> perform any graph manipulation before we expect the user to interface
>> with the job again, right?
>>
>> (It's always done right at completion time, at least for drive-mirror.
>> Do any other jobs adjust the graph? If it's ever anything except right
>> before completion time, we may lose the ability to pause/resume,
>> set-speed, etc.)
>>
>> Does this sound about right, or have I fatally misunderstood the situation?
> 
> Other jobs change the graph as well, but none do so before completion,
> so I don't think theere will be further user interaction. The only
> interesting part so far is the device name that is sent in the
> completion event.
> 

Yes, it's a little awkward -- but at least it could be treated as simply
an identifier to know which job that was submitted has now completed.

An additional, actually unique non-changing ID would make it bulletproof.

>> (post-script: I was thinking of adding a unique per-job ID that could be
>> reported alongside any events or errors where the job's device name was
>> reported, so that users could provide this ID to find the job. Each BB
>> would have a per-tree list of jobs with globally unique IDs, and
>> regardless of what node the job was currently attached to, we could
>> retrieve that job unambiguously. This would be useful if the above
>> question reveals an API problem, or more generally for multiple block
>> jobs where we'll need IDs to reference jobs anyway.)
> 
> I actually introduced a job->id field, but we're not using it
> consistently yet. I needed it so I could still access the right name
> for the completion message, as I said above.
> 

Yes. Currently it's only being used to report back the device name, so
it could even be considered just a device name cache.

> What could turn out a bit nasty is that we have to maintain API
> compatibility. The best option that I could think of so far is that we
> change the current device name in all QMP commands into a block job ID
> while still calling it 'device' for compatibility. If an ID isn't
> specified in the command that starts a block job, the device_name is
> used like today. If that default ID is already taken, we fail the
> command; this doesn't impact compatibility because today you can't set
> any ID other than the device_name and you can only have one job per
> device.
> 

That's one option, with the obvious caveat being "device" is now
aggressively misnamed. I was also thinking we could /add/ an actual id=
parameter to block job creation and querying such that it could be
provided in lieu of the device name, and all events/errors/etc could
report back both the somewhat-inaccurate "device" name and the new ID.

This has the effect of allowing new API users to just stick to ID
exclusively while ignoring device, but older clients could still
somewhat ignorantly continue using the device field.

Clients aware of e.g. multiple block jobs can happily use ID=, and
clients unaware of said feature can use the older subset.

Downside of course is it adds another "one or the other but not both"
QMP command which I know everybody loves.

> If we don't like the misnomer 'device' for the block job ID, we would
> have to store whether the id == device_name default was applied and then
> add a 'device' key in all return values and events where this was the
> case.
> 

Close to what I was thinking.

>> Each BB would have a per-tree list of jobs with globally unique IDs
> 
> Why that? Block jobs don't belong to a single BB. They are artificially
> attached to a single BDS today, but design-wise that doesn't make a lot
> of sense for any job other than streaming, because all other currently
> existing jobs have a source and a target rather than a single node.
> 

A naive first stab at the problem -- I was thinking only of the problem
when a job attached to the root node modifies the graph such that its
node is no longer the root. I wasn't thinking about cross-graph jobs.

It could simply be a global structure. A list is probably sufficient for
now.

> Or maybe we should now introduce a completely new API for background jobs
> (not just block jobs) and implement the old API on top of that. In the
> long run we could move e.g. migration to the same API.
> 
> Kevin
> 

A new API would be a nice clean break to accomplish quite a few things
at once, like a revamped permissions system replacing op blockers,
introducing multiple block jobs, and other goodness.

If we named these new features together, we probably could just
introduce a new set of QMP commands to manage them and leave the
existing BlockJob APIs as legacy interfaces to be phased out in the
future sometime.

Thoughts on that vs. trying to extend the existing Block Jobs API?

--js