Re: [Qemu-devel] [RFC PATCH 1/1] s390x/css: unresrict cssids

[Cornelia Huck]

On Wed, 29 Nov 2017 19:51:23 +0100
Christian Borntraeger <address@hidden> wrote:

> On 11/28/2017 03:45 PM, Cornelia Huck wrote:
> > On Tue, 28 Nov 2017 15:17:49 +0100
> > Christian Borntraeger <address@hidden> wrote:
> >   
> >> On 11/28/2017 03:01 PM, Cornelia Huck wrote:  
> >>> On Tue, 28 Nov 2017 14:25:08 +0100
> >>> Christian Borntraeger <address@hidden> wrote:  
> >   
> >>>> What I want now is to enable vfio-ccw for libvirt and Linux guests and 
> >>>> for that
> >>>> we need to lift the restriction of having vfio not in FE.    
> >>>
> >>> This, however, worries me. I don't really consider vfio-ccw ready for
> >>> prime time yet. We still need to figure out path handling at the very
> >>> least. And I'm still not sure that our non-handling of halt/clear won't
> >>> cause major issues with e.g. a storage server error recovery.
> >>>
> >>> Can we flag vfio-ccw as experimental or such in libvirt?    
> >>
> >> We should have flagged vfio-ccw experimental in QEMU then.
> >> e.g. by using x-vfio-ccw or whatever.I dont think we can do that
> >> after the facts.  
> > 
> > Yes, we should have done that... can't fix that now, unfortunately.
> >   
> >>
> >> I am not that deep into vfio-ccw, but I was under the impression that
> >> the missing features should not affect the vfio interface that is
> >> created by libvirt. After all this should just be a "-device 
> >> vfio-ccw,....."
> >> statement and some setup steps beforehand (unbind + setup of vfio-ccw)  
> > 
> > The halt/clear stuff is highly unlikely to influence the configuration
> > interface. I'm still not too clear about path handling, though. Will we
> > need different modes (hypervisor managed vs. full passthrough? probably
> > only passthrough)? What about pgid handling - do we need some kind of
> > pgid manager? (Keep in mind that you get to set the pgid once. This has
> > implications for e.g. reserve/release.)
> > 
> > Also, what about devices other than ECKD DASD? Has there been any
> > testing? Tapes should be interesting; the other interesting ccw devices
> > are QDIO-based and I'm not sure we want to spend anything on supporting
> > those.  
> 
> DASD is probably the most important thing today, QDIO might never happen
> (or very late).

One thing I'd find interesting about tapes is very long running channel
programs (like rewind) and how they interact with halt/clear. But yes,
I would think that DASD is the most important one in practice.

> See my proposal below.
> 
> > 
> > The interface is probably fine, but I'd like to get an idea about the
> > path stuff (is the attachment spec that contains the pgid stuff
> > publicly available, btw.?)
> >   
> >>
> >> If your concern is the serviceability I think it would be valid for a RHEL
> >> KVM to disable vfio-ccw in the kernel. Maybe we could even provide a config
> >> for QEMU?  
> > 
> > It's fine just to turn off vfio-ccw in the kernel.
> >   
> >> PS: I learned from the PCI and CCW experience that I do not want
> >> to upstream kernel/qemu code unless I have a working libvirt code that
> >> shows that the thing will work.  
> > 
> > Yes, I understand the wish to verify the interface, and I think it's a
> > good idea. What I'm worried about is that this might be the precursor
> > to a push to support it, and I don't think vfio-ccw is ready for that
> > yet.
> >   
> FWIW, libvirt should not care if a device works in all cases or not because 
> libvirt versions should work with all kind of QEMU/kernel combinations. 
> Fencing in libvirt that the kernel part is not up to the task is making
> me feel the same way as you when you see  the css-unrestricted property
> at a device ;-)

I'm more worried about the political angle than the technical. If we
don't get pressure to support this too soon, I don't care that much
about experimental or not.

> Having said that, I think that having vfio-ccw support has a value (and it
> actually works for an unnamed test infrastructure). In addition to that I 
> am much more likely to actually test this continuously if I have libvirt 
> support.

Good to hear that it works for a non-Linux guest. Any plans to test
something like storage server failover?

[I'd really love to do something about the path handling stuff, but the
combination of scant documentation and scantier time works against me.]

> 
> 
> So what about the following: 
> 
> 1. We will implement the libvirt support with either a or b:
> a: if we find a solution to our "where to put the property" dispute use that 
> to decide
> if we can add vfio-ccw
> b if not: just provide a patch that lifts the restriction without any 
> property.
> in libvirt blindy assume  that free assignment will work. old QEMUs will 
> complain at
> startup and  libvirt will print the QEMU error. This is similar to other 
> situations 
> where libvirt cannot fully bprobe  if the QEMU will work or not. (not having 
> vfio-ccw
> support in the kernel will certainly allow libvirt to reject this upfront)

I hope we end up with a solution that works for everyone...

> 
> 2. at the same time we mark vfio-ccw experimental in the kernel to make clear
>  that this is still work in progress

I'm not sure we can do that after the fact, but let's do it if we can.

> 
> 3. (optional) we could even whitelist devices that work in a reasonable 
> fashion (e.g.
> do not whitelist qdio but whitelist DASD)

I think it's not quite that easy with vfio-ccw working at the
subchannel level.