Re: [PATCH v2 4/4] virtio-net: Add support for USO features

[Michael S. Tsirkin]

On Tue, Jul 30, 2024 at 03:11:03PM -0400, Peter Xu wrote:
> On Tue, Jul 30, 2024 at 07:46:12PM +0100, Daniel P. Berrangé wrote:
> > On Tue, Jul 30, 2024 at 02:13:51PM -0400, Peter Xu wrote:
> > > On Mon, Jul 29, 2024 at 06:26:41PM +0100, Daniel P. Berrangé wrote:
> > > > On Mon, Jul 29, 2024 at 01:00:30PM -0400, Peter Xu wrote:
> > > > > On Mon, Jul 29, 2024 at 04:58:03PM +0100, Daniel P. Berrangé wrote:
> > > > > > 
> > > > > > We've got two mutually conflicting goals with the machine type
> > > > > > definitions.
> > > > > > 
> > > > > > Primarily we use them to ensure stable ABI, but an important
> > > > > > secondary goal is to enable new tunables to have new defaults
> > > > > > set, without having to update every mgmt app.  The latter
> > > > > > works very well when the defaults have no dependancy on the
> > > > > > platform kernel/OS, but breaks migration when they do have a
> > > > > > platform dependancy.
> > > > > > 
> > > > > > >   - Firstly, never quietly flipping any bit that affects the 
> > > > > > > ABI...
> > > > > > > 
> > > > > > >   - Have a default value of off, then QEMU will always allow the 
> > > > > > > VM to boot
> > > > > > >     by default, while advanced users can opt-in on new features.  
> > > > > > > We can't
> > > > > > >     make this ON by default otherwise some VMs can already fail 
> > > > > > > to boot,
> > > > > > > 
> > > > > > >   - If the host doesn't support the feature while the cmdline 
> > > > > > > enabled it,
> > > > > > >     it needs to fail QEMU boot rather than flipping, so that it 
> > > > > > > says "hey,
> > > > > > >     this host does not support running such VM specified, due to 
> > > > > > > XXX
> > > > > > >     feature missing".
> > > > > > > 
> > > > > > > That's the only way an user could understand what happened, and 
> > > > > > > IMHO that's
> > > > > > > a clean way that we stick with QEMU cmdline on defining the guest 
> > > > > > > ABI,
> > > > > > > while in which the machine type is the fundation of such 
> > > > > > > definition, as the
> > > > > > > machine type can decides many of the rest compat properties.  And 
> > > > > > > that's
> > > > > > > the whole point of the compat properties too (to make sure the 
> > > > > > > guest ABI is
> > > > > > > stable).
> > > > > > > 
> > > > > > > If kernel breaks it easily, all compat property things that we 
> > > > > > > maintain can
> > > > > > > already stop making sense in general, because it didn't define 
> > > > > > > the whole
> > > > > > > guest ABI..
> > > > > > > 
> > > > > > > So AFAIU that's really what we used for years, I hope I didn't 
> > > > > > > overlook
> > > > > > > somehting.  And maybe we don't yet need the "-platform" layer if 
> > > > > > > we can
> > > > > > > keep up with this rule?
> > > > > > 
> > > > > > We've failed at this for years wrt enabling use of new defaults 
> > > > > > that have
> > > > > > a platform depedancy, so historical practice isn't a good reference.
> > > > > > 
> > > > > > There are 100's (possibly 1000's) of tunables set implicitly as 
> > > > > > part of
> > > > > > the machine type, and of those, libvirt likely only exposes a few 
> > > > > > 10's
> > > > > > of tunables. The vast majority are low level details that no mgmt 
> > > > > > app
> > > > > > wants to know about, they just want to accept QEMU's new defaults,
> > > > > > while preserving machine ABI. This is a good thing. No one wants the
> > > > > > burden of wiring up every single tunable into libvirt and mgmt apps.
> > > > > > 
> > > > > > This is what the "-platform" concept would be intended to preserve. 
> > > > > > It
> > > > > > would allow a way to enable groups of settings that have a platform 
> > > > > > level
> > > > > > dependancy, without ever having to teach either libvirt or the mgmt 
> > > > > > apps
> > > > > > about the individual tunables.
> > > > > 
> > > > > Do you think we can achieve similar goal by simply turning the 
> > > > > feature to
> > > > > ON only after a few QEMU releases?  I also mentioned that idea below.
> > > > > 
> > > > > https://lore.kernel.org/r/ZqQNKZ9_OPhDq2AK@x1n
> > > > > 
> > > > > So far it really sounds like the right thing to do to me to fix all 
> > > > > similar
> > > > > issues, even without introducing anything new we need to maintain.
> > > > 
> > > > Turning a feature with a platform dependency to "on" implies that
> > > > the machine type will cease to work out of the box for platforms
> > > > which lack the feature. IMHO that's not acceptable behaviour for
> > > > any of our supported platforms.
> > > 
> > > Right, that's why I was thinking whether we should just always be on the
> > > safe side, even if I just replied in the other email to Akihiko, that we 
> > > do
> > > have the option to make this more aggresive by turning those to ON after
> > > even 1-2 years or even less.. and we have control of how aggressive this
> > > can be.
> > > 
> > > > 
> > > > IOW, "after a few QEMU releases" implies a delay of as much as
> > > > 5 years, while we wait for platforms which don't support the
> > > > feature to drop out of our supported targets list.  I don't
> > > > think that'll satisfy the desire to get the new feature
> > > > available to users as soon as practical for their particular
> > > > platform.
> > > 
> > > The feature is always available since the 1st day, right?  We just need 
> > > the
> > > user to opt-in, by specifying ON in the cmdline.
> > > 
> > > That'll be my take on this that QEMU's default VM setup should be always
> > > bootable, migratable, and so on.  Then user opt-in on stuff like this one,
> > > where there's implication on the ABIs.  The "user" can also include
> > > Libvirt.  I mean when something is really important, Libvirt should, IMHO,
> > > opt-in by treating that similarly like many cpu properties, and by probing
> > > the host first.
> > > 
> > > IIUC there aren't a lot of things like that (part of guest ABI & host
> > > kernel / HW dependent), am I right?  Otherwise I would expect more 
> > > failures
> > > like this one, but it isn't as much as that yet.  IIUC it means the 
> > > efforts
> > > to make Libvirt get involved should be hopefully under control too.  The
> > > worst case is Libvirt doesn't auto-on it, but again the user should always
> > > have the option to turn it on when it's necessary.
> > 
> > If it is left to libvirt, then it would very likely end up being a user
> > opt-in, not auto-enabled.
> 
> Not sure whether there's other opinions, but that's definitely fine by me.
> 
> I think it even makes more sense, as even if Libvirt probed the host and
> auto-on the feature, it also means Libvirt made a decision for the user,
> saying "having a better performance" is more important than "being able to
> migrate this VM everywhere".
> 
> I don't see a way that can make such fair decision besides requesting the
> user to opt-in always for those, then the user is fully aware what is
> enabled, with the hope that when a migration fails later with "target host
> doesn't support feature XXX" the user is crystal clear on what happened.
> 
> Thanks,
> 
> -- 
> Peter Xu


This is not what we did historically. Why should we start now?

-- 
MST