Re: [Qemu-devel] MSI interrupt support with vioscsi.c miniport driver

[Vadim Rozenfeld]

On Wed, 2014-02-19 at 15:25 -0800, Nicholas A. Bellinger wrote:
> On Wed, 2014-02-19 at 19:03 +1100, Vadim Rozenfeld wrote:
> > On Tue, 2014-02-18 at 13:00 -0800, Nicholas A. Bellinger wrote:
> > > On Mon, 2014-02-10 at 11:05 -0800, Nicholas A. Bellinger wrote:
> > > 
> > > <SNIP>
> > > 
> > > > > > > Hi Yan,
> > > > > > > 
> > > > > > > So recently I've been doing some KVM guest performance comparisons
> > > > > > > between the scsi-mq prototype using virtio-scsi + vhost-scsi, and
> > > > > > > Windows Server 2012 with vioscsi.sys (virtio-win-0.1-74.iso) +
> > > > > > > vhost-scsi using PCIe flash backend devices.
> > > > > > > 
> > > > > > > I've noticed that small block random performance for the MSFT 
> > > > > > > guest is
> > > > > > > at around ~80K IOPs with multiple vioscsi LUNs + adapters, which 
> > > > > > > ends up
> > > > > > > being well below what the Linux guest with scsi-mq + virtio-scsi 
> > > > > > > is
> > > > > > > capable of (~500K).
> > > > > > > 
> > > > > > > After searching through the various vioscsi registry settings, it
> > > > > > > appears that MSIEnabled is being explicitly disabled 
> > > > > > > (0x00000000), that
> > > > > > > is different from what vioscsi.inx is currently defining:
> > > > > > > 
> > > > > > > [pnpsafe_pci_addreg_msix]
> > > > > > > HKR, "Interrupt Management",, 0x00000010
> > > > > > > HKR, "Interrupt Management\MessageSignaledInterruptProperties",, 
> > > > > > > 0x00000010
> > > > > > > HKR, "Interrupt Management\MessageSignaledInterruptProperties", 
> > > > > > > MSISupported, 0x00010001, 0
> > > > > > > HKR, "Interrupt Management\MessageSignaledInterruptProperties", 
> > > > > > > MessageNumberLimit, 0x00010001, 4
> > > > > > > 
> > > > > > > Looking deeper at vioscsi.c code, I've noticed that 
> > > > > > > MSI_SUPPORTED=0 is
> > > > > > > explicitly disabled at build time in SOURCES + vioscsi.vcxproj, 
> > > > > > > as well
> > > > > > > as VioScsiFindAdapter() code always ends setting msix_enabled = 
> > > > > > > FALSE
> > > > > > > here, regardless of MSI_SUPPORTED:
> > > > > > > 
> > > > > > >  
> > > > > > > https://github.com/YanVugenfirer/kvm-guest-drivers-windows/blob/master/vioscsi/vioscsi.c#L340
> > > > > > > 
> > > > > > > Also looking at virtio_stor.c for the raw block driver, 
> > > > > > > MSI_SUPPORTED=1
> > > > > > > appears to be the default setting for the driver included in the 
> > > > > > > offical
> > > > > > > virtio-win iso builds, right..?
> > > > > > > 
> > > > > > > Sooo, I'd like to try enabling MSI_SUPPORTED=1 in a test 
> > > > > > > vioscsi.sys
> > > > > > > build of my own, but before going down the WDK development rabbit 
> > > > > > > whole,
> > > > > > > I'd like to better understand why you've explicitly disabled this 
> > > > > > > logic
> > > > > > > within vioscsi.c code to start..?
> > > > > > > 
> > > > > > > Is there anything that needs to be addressed / carried over from
> > > > > > > virtio_stor.c in order to get MSI_SUPPORTED=1 to work with 
> > > > > > > vioscsi.c
> > > > > > > miniport code..?
> > > > > 
> > > > > Hi Nicholas,
> > > > > 
> > > > > I was thinking about enabling MSI in RHEL 6.6 (build 74) but for some
> > > > > reasons decided to keep it disabled until adding mq support.
> > > > > 
> > > > > 
> > > > > You definitely should be able to turn on MSI_SUPPORTED, rebuild the
> > > > > driver, and switch MSISupported to 1 to make vioscsi driver working in
> > > > > MSI mode.
> > > > >    
> > > > 
> > > > Thanks for the quick response.  We'll give MSI_SUPPORTED=1 a shot over
> > > > the next days with a test build on Server 2012 / Server 2008 R2 and see
> > > > how things go..
> > > > 
> > > 
> > > Just a quick update on progress.
> > > 
> > > I've been able to successfully build + load a unsigned vioscsi.sys
> > > driver on Server 2012 with WDK 8.0.
> > > 
> > > Running with MSI_SUPPORTED=1 against vhost-scsi results in a significant
> > > performance and efficiency gain, on the order of 100K to 225K IOPs for
> > > 4K block random I/O workload, depending on read/write mix.
> > > 
> > > Below is a simple patch to enable MSI operation by default.  Any chance
> > > to apply this separate from future mq efforts..?
> > 
> > Yes, we differently can enable MSI and rebuild vioscsi.
> > But then we need to re-spin WHQL testing for this particular
> > driver. This process requires a lot of resources, and I doubt that
> > it will be initiated soon, unless we have some significant amount of
> > bug-fixes.
> > 
> 
> Any idea on a rough time frame to expect an official WHQL build with MSI
> enabled..?
In June for sure :)
> Or, would it be possible to generate some -BETA builds that are at least
> signed and don't require extra hoops to jump through for testing..?

It's doable. 
I hope we can make a new build next week.
Best regards,
Vadim.

> 
> Thanks again,
> 
> --nab
> 
>