Re: [Qemu-devel] Should we auto-generate IDs?

[Programmingkid]

On Aug 27, 2015, at 12:22 PM, Daniel P. Berrange wrote:

> On Thu, Aug 27, 2015 at 12:08:25PM -0400, Programmingkid wrote:
>> 
>> On Aug 27, 2015, at 12:02 PM, Daniel P. Berrange wrote:
>> 
>>> On Thu, Aug 27, 2015 at 11:58:22AM -0400, Programmingkid wrote:
>>>> 
>>>> On Aug 27, 2015, at 11:40 AM, Jeff Cody wrote:
>>>> 
>>>>> On Thu, Aug 27, 2015 at 11:20:20AM -0400, Programmingkid wrote:
>>>>>> 
>>>>>> On Aug 27, 2015, at 10:42 AM, Daniel P. Berrange wrote:
>>>>>> 
>>>>>>> On Thu, Aug 27, 2015 at 10:34:05AM -0400, Programmingkid wrote:
>>>>>>>> 
>>>>>>>> On Aug 27, 2015, at 10:02 AM, Eric Blake wrote:
>>>>>>>> 
>>>>>>>>> On 08/27/2015 07:56 AM, Programmingkid wrote:
>>>>>>>>> 
>>>>>>>>>>> If we did have auto-generated names, we would need to come up with a
>>>>>>>>>>> scheme that is not going to clash with any existing naming that 
>>>>>>>>>>> users
>>>>>>>>>>> of QEMU may already be doing, otherwise we risk causing a 
>>>>>>>>>>> regression.
>>>>>>>>>>> Something as simple as what you suggest has non-trivial chance of
>>>>>>>>>>> clashing.
>>>>>>>>>> 
>>>>>>>>>> Actually there is a way to prevent clashing. When QEMU 
>>>>>>>>>> auto-generates a
>>>>>>>>>> name, it could scan all the ID's to see if there is a clash. If the 
>>>>>>>>>> ID is already
>>>>>>>>>> taken, just increment the ID until it is detected to be unique. The 
>>>>>>>>>> previous
>>>>>>>>>> threads on this subject has patches that did just that. This means 
>>>>>>>>>> that a
>>>>>>>>>> ID scheme that is just a single number would work without clashes. 
>>>>>>>>> 
>>>>>>>>> No, because you cannot predict what FUTURE names the user will 
>>>>>>>>> request.
>>>>>>>>> The name generated by qemu must be IMPOSSIBLE to request manually, and
>>>>>>>>> not just one that happens not to clash at the current moment.
>>>>>>>> 
>>>>>>>> If I add a device with an ID that is taken, QEMU can just say sorry 
>>>>>>>> that ID is already
>>>>>>>> taken. I could just increment the ID myself until I find one that is 
>>>>>>>> unique. It is a
>>>>>>>> simple algorithm. Maybe you are talking about some program that has 
>>>>>>>> hard coded
>>>>>>>> ID's it depends on. If that is the case, perhaps this management 
>>>>>>>> program could be
>>>>>>>> made to be a little flexible. Or use a 160-bit SHA-1 generated ID that 
>>>>>>>> is virtually
>>>>>>>> guaranteed to always be unique.
>>>>>>> 
>>>>>>> If breaking existing apps was OK, we would just mandate that users 
>>>>>>> always
>>>>>>> provide an ID which trivially avoids the problem of not having an ID to
>>>>>>> use when deleting the object. We want to /avoid/ breaking existing apps
>>>>>>> though, so saying an app should change the way it works in order to cope
>>>>>>> with QEMU's ID generation is not appropriate. Hence any use of 
>>>>>>> auto-generated
>>>>>>> IDs, must use a separate namespace to avoid such clashes.
>>>>>> 
>>>>>> This is making the assumption that an auto-generated ID system would 
>>>>>> break any existing
>>>>>> application. We don't know this. In fact, I predict a future patch would 
>>>>>> allow existing
>>>>>> applications to continue running without modification. The patch would 
>>>>>> be a win-win
>>>>>> for both users and any management application.
>>>>>> 
>>>>> 
>>>>> Daniel's assumption is spot on.  The idea of "QEMU can just say sorry
>>>>> that ID is already taken" will break existing applications.
>>>>> 
>>>>> But we are all striving to make your prediction true, with this very
>>>>> conversation.
>>>> 
>>>> Ok, it sounds like some are concerned that Libvirt would not be able to 
>>>> work with this
>>>> feature. Let me ask you this: does Libvirt interact with QEMU before the 
>>>> user has a
>>>> chance to do so? If the answer is yes, then that means Libvirt will have 
>>>> finished 
>>>> creating all its devices with their ID's long before the user has a chance 
>>>> to enter
>>>> his own devices.
>>> 
>>> Just to be clear - libvirt will *never* use an auto-generated device
>>> IDs feature. It is way more complicated to let QEMU assign device IDs
>>> and then auto-detect them from some 'info device-list' output, than
>>> to just specify IDs upfront at device/object creation time which
>>> it already does[1]. There is simply no benefit to auto-generating device
>>> IDs for a mgmt app like libvirt, and plenty of downside.  Auto-generated
>>> IDs will only be of interest to humans talking to the monitor directly
>>> without a mgmt app involved.
>> 
>> I've haven't used Libvirt but I do believe in the saying "never say never". 
>> The
>> rest of what you said does sound accurate. 
> 
> I say that based on history of the development of libvirt. Many, many
> years ago now, with QEMU 0.8.x, -device / device_add did not exist,
> so you had to configure devices using args like -drive, or before
> that, -hda, -hdb, etc. With that old syntax, QEMU would in fact
> auto-generate a unique ID for each device. Libvirt then had to
> figure out what that unique ID would be which was non-trivial to
> get right, and risked changing with new QEMU releases. It also
> did not cope well when hotplug was combined with migraiton, as
> two guest machines with identical guest hardware could have
> different assigned IDs depending on the sequence of hotplug/unplug
> operations performed. All in all it was a world of hurt  and
> we were very happy when -device came along and allowed libvirt
> to specify the deivce IDs upfront itself. So yes, I am confident
> we will not go back to letting QEMU auto-generate IDs in libvirt.

Your patch does make device_del work, but still doesn't solve the 
auto-generated ID's issue. 

I have the feeling you think an auto-generated ID system will be in
Libvirt's way. It won't. Libvirt can specify its own ID's as much as it
wants. The auto-generated ID system would only work when no ID is
specified. So you have nothing to worry about. When Libvirt starts up
QEMU, it can specify all the ID's it wants without problem. This feature
only works if the user forgot to give an ID. Libvirt and the user can be
happy with this feature.