Re: [Qemu-devel] Should we auto-generate IDs?

[Programmingkid]

On Aug 27, 2015, at 12:22 PM, Daniel P. Berrange wrote:

> On Thu, Aug 27, 2015 at 12:08:25PM -0400, Programmingkid wrote:
>> 
>> On Aug 27, 2015, at 12:02 PM, Daniel P. Berrange wrote:
>> 
>>> On Thu, Aug 27, 2015 at 11:58:22AM -0400, Programmingkid wrote:
>>>> 
>>>> On Aug 27, 2015, at 11:40 AM, Jeff Cody wrote:
>>>> 
>>>>> On Thu, Aug 27, 2015 at 11:20:20AM -0400, Programmingkid wrote:
>>>>>> 
>>>>>> On Aug 27, 2015, at 10:42 AM, Daniel P. Berrange wrote:
>>>>>> 
>>>>>>> On Thu, Aug 27, 2015 at 10:34:05AM -0400, Programmingkid wrote:
>>>>>>>> 
>>>>>>>> On Aug 27, 2015, at 10:02 AM, Eric Blake wrote:
>>>>>>>> 
>>>>>>>>> On 08/27/2015 07:56 AM, Programmingkid wrote:
>>>>>>>>> 
>>>>>>>>>>> If we did have auto-generated names, we would need to come up with a
>>>>>>>>>>> scheme that is not going to clash with any existing naming that 
>>>>>>>>>>> users
>>>>>>>>>>> of QEMU may already be doing, otherwise we risk causing a 
>>>>>>>>>>> regression.
>>>>>>>>>>> Something as simple as what you suggest has non-trivial chance of
>>>>>>>>>>> clashing.
>>>>>>>>>> 
>>>>>>>>>> Actually there is a way to prevent clashing. When QEMU 
>>>>>>>>>> auto-generates a
>>>>>>>>>> name, it could scan all the ID's to see if there is a clash. If the 
>>>>>>>>>> ID is already
>>>>>>>>>> taken, just increment the ID until it is detected to be unique. The 
>>>>>>>>>> previous
>>>>>>>>>> threads on this subject has patches that did just that. This means 
>>>>>>>>>> that a
>>>>>>>>>> ID scheme that is just a single number would work without clashes. 
>>>>>>>>> 
>>>>>>>>> No, because you cannot predict what FUTURE names the user will 
>>>>>>>>> request.
>>>>>>>>> The name generated by qemu must be IMPOSSIBLE to request manually, and
>>>>>>>>> not just one that happens not to clash at the current moment.
>>>>>>>> 
>>>>>>>> If I add a device with an ID that is taken, QEMU can just say sorry 
>>>>>>>> that ID is already
>>>>>>>> taken. I could just increment the ID myself until I find one that is 
>>>>>>>> unique. It is a
>>>>>>>> simple algorithm. Maybe you are talking about some program that has 
>>>>>>>> hard coded
>>>>>>>> ID's it depends on. If that is the case, perhaps this management 
>>>>>>>> program could be
>>>>>>>> made to be a little flexible. Or use a 160-bit SHA-1 generated ID that 
>>>>>>>> is virtually
>>>>>>>> guaranteed to always be unique.
>>>>>>> 
>>>>>>> If breaking existing apps was OK, we would just mandate that users 
>>>>>>> always
>>>>>>> provide an ID which trivially avoids the problem of not having an ID to
>>>>>>> use when deleting the object. We want to /avoid/ breaking existing apps
>>>>>>> though, so saying an app should change the way it works in order to cope
>>>>>>> with QEMU's ID generation is not appropriate. Hence any use of 
>>>>>>> auto-generated
>>>>>>> IDs, must use a separate namespace to avoid such clashes.
>>>>>> 
>>>>>> This is making the assumption that an auto-generated ID system would 
>>>>>> break any existing
>>>>>> application. We don't know this. In fact, I predict a future patch would 
>>>>>> allow existing
>>>>>> applications to continue running without modification. The patch would 
>>>>>> be a win-win
>>>>>> for both users and any management application.
>>>>>> 
>>>>> 
>>>>> Daniel's assumption is spot on.  The idea of "QEMU can just say sorry
>>>>> that ID is already taken" will break existing applications.
>>>>> 
>>>>> But we are all striving to make your prediction true, with this very
>>>>> conversation.
>>>> 
>>>> Ok, it sounds like some are concerned that Libvirt would not be able to 
>>>> work with this
>>>> feature. Let me ask you this: does Libvirt interact with QEMU before the 
>>>> user has a
>>>> chance to do so? If the answer is yes, then that means Libvirt will have 
>>>> finished 
>>>> creating all its devices with their ID's long before the user has a chance 
>>>> to enter
>>>> his own devices.
>>> 
>>> Just to be clear - libvirt will *never* use an auto-generated device
>>> IDs feature. It is way more complicated to let QEMU assign device IDs
>>> and then auto-detect them from some 'info device-list' output, than
>>> to just specify IDs upfront at device/object creation time which
>>> it already does[1]. There is simply no benefit to auto-generating device
>>> IDs for a mgmt app like libvirt, and plenty of downside.  Auto-generated
>>> IDs will only be of interest to humans talking to the monitor directly
>>> without a mgmt app involved.
>> 
>> I've haven't used Libvirt but I do believe in the saying "never say never". 
>> The
>> rest of what you said does sound accurate. 
> 
> I say that based on history of the development of libvirt. Many, many
> years ago now, with QEMU 0.8.x, -device / device_add did not exist,
> so you had to configure devices using args like -drive, or before
> that, -hda, -hdb, etc. With that old syntax, QEMU would in fact
> auto-generate a unique ID for each device. Libvirt then had to
> figure out what that unique ID would be which was non-trivial to
> get right, and risked changing with new QEMU releases. It also
> did not cope well when hotplug was combined with migraiton, as
> two guest machines with identical guest hardware could have
> different assigned IDs depending on the sequence of hotplug/unplug
> operations performed. All in all it was a world of hurt  and
> we were very happy when -device came along and allowed libvirt
> to specify the deivce IDs upfront itself. So yes, I am confident
> we will not go back to letting QEMU auto-generate IDs in libvirt.

Thank you very much for this history. I didn't know about this.