Re: [Qemu-devel] Should we auto-generate IDs?

[Daniel P. Berrange]

On Thu, Aug 27, 2015 at 12:08:25PM -0400, Programmingkid wrote:
> 
> On Aug 27, 2015, at 12:02 PM, Daniel P. Berrange wrote:
> 
> > On Thu, Aug 27, 2015 at 11:58:22AM -0400, Programmingkid wrote:
> >> 
> >> On Aug 27, 2015, at 11:40 AM, Jeff Cody wrote:
> >> 
> >>> On Thu, Aug 27, 2015 at 11:20:20AM -0400, Programmingkid wrote:
> >>>> 
> >>>> On Aug 27, 2015, at 10:42 AM, Daniel P. Berrange wrote:
> >>>> 
> >>>>> On Thu, Aug 27, 2015 at 10:34:05AM -0400, Programmingkid wrote:
> >>>>>> 
> >>>>>> On Aug 27, 2015, at 10:02 AM, Eric Blake wrote:
> >>>>>> 
> >>>>>>> On 08/27/2015 07:56 AM, Programmingkid wrote:
> >>>>>>> 
> >>>>>>>>> If we did have auto-generated names, we would need to come up with a
> >>>>>>>>> scheme that is not going to clash with any existing naming that 
> >>>>>>>>> users
> >>>>>>>>> of QEMU may already be doing, otherwise we risk causing a 
> >>>>>>>>> regression.
> >>>>>>>>> Something as simple as what you suggest has non-trivial chance of
> >>>>>>>>> clashing.
> >>>>>>>> 
> >>>>>>>> Actually there is a way to prevent clashing. When QEMU 
> >>>>>>>> auto-generates a
> >>>>>>>> name, it could scan all the ID's to see if there is a clash. If the 
> >>>>>>>> ID is already
> >>>>>>>> taken, just increment the ID until it is detected to be unique. The 
> >>>>>>>> previous
> >>>>>>>> threads on this subject has patches that did just that. This means 
> >>>>>>>> that a
> >>>>>>>> ID scheme that is just a single number would work without clashes. 
> >>>>>>> 
> >>>>>>> No, because you cannot predict what FUTURE names the user will 
> >>>>>>> request.
> >>>>>>> The name generated by qemu must be IMPOSSIBLE to request manually, and
> >>>>>>> not just one that happens not to clash at the current moment.
> >>>>>> 
> >>>>>> If I add a device with an ID that is taken, QEMU can just say sorry 
> >>>>>> that ID is already
> >>>>>> taken. I could just increment the ID myself until I find one that is 
> >>>>>> unique. It is a
> >>>>>> simple algorithm. Maybe you are talking about some program that has 
> >>>>>> hard coded
> >>>>>> ID's it depends on. If that is the case, perhaps this management 
> >>>>>> program could be
> >>>>>> made to be a little flexible. Or use a 160-bit SHA-1 generated ID that 
> >>>>>> is virtually
> >>>>>> guaranteed to always be unique.
> >>>>> 
> >>>>> If breaking existing apps was OK, we would just mandate that users 
> >>>>> always
> >>>>> provide an ID which trivially avoids the problem of not having an ID to
> >>>>> use when deleting the object. We want to /avoid/ breaking existing apps
> >>>>> though, so saying an app should change the way it works in order to cope
> >>>>> with QEMU's ID generation is not appropriate. Hence any use of 
> >>>>> auto-generated
> >>>>> IDs, must use a separate namespace to avoid such clashes.
> >>>> 
> >>>> This is making the assumption that an auto-generated ID system would 
> >>>> break any existing
> >>>> application. We don't know this. In fact, I predict a future patch would 
> >>>> allow existing
> >>>> applications to continue running without modification. The patch would 
> >>>> be a win-win
> >>>> for both users and any management application.
> >>>> 
> >>> 
> >>> Daniel's assumption is spot on.  The idea of "QEMU can just say sorry
> >>> that ID is already taken" will break existing applications.
> >>> 
> >>> But we are all striving to make your prediction true, with this very
> >>> conversation.
> >> 
> >> Ok, it sounds like some are concerned that Libvirt would not be able to 
> >> work with this
> >> feature. Let me ask you this: does Libvirt interact with QEMU before the 
> >> user has a
> >> chance to do so? If the answer is yes, then that means Libvirt will have 
> >> finished 
> >> creating all its devices with their ID's long before the user has a chance 
> >> to enter
> >> his own devices.
> > 
> > Just to be clear - libvirt will *never* use an auto-generated device
> > IDs feature. It is way more complicated to let QEMU assign device IDs
> > and then auto-detect them from some 'info device-list' output, than
> > to just specify IDs upfront at device/object creation time which
> > it already does[1]. There is simply no benefit to auto-generating device
> > IDs for a mgmt app like libvirt, and plenty of downside.  Auto-generated
> > IDs will only be of interest to humans talking to the monitor directly
> > without a mgmt app involved.
> 
> I've haven't used Libvirt but I do believe in the saying "never say never". 
> The
> rest of what you said does sound accurate. 

I say that based on history of the development of libvirt. Many, many
years ago now, with QEMU 0.8.x, -device / device_add did not exist,
so you had to configure devices using args like -drive, or before
that, -hda, -hdb, etc. With that old syntax, QEMU would in fact
auto-generate a unique ID for each device. Libvirt then had to
figure out what that unique ID would be which was non-trivial to
get right, and risked changing with new QEMU releases. It also
did not cope well when hotplug was combined with migraiton, as
two guest machines with identical guest hardware could have
different assigned IDs depending on the sequence of hotplug/unplug
operations performed. All in all it was a world of hurt  and
we were very happy when -device came along and allowed libvirt
to specify the deivce IDs upfront itself. So yes, I am confident
we will not go back to letting QEMU auto-generate IDs in libvirt.

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|