Re: [Qemu-devel] [PATCH 7/7 v5] VMXNET3 paravirtualized device implementation Interface type "vmxnet3" added.

[Anthony Liguori]

On 04/16/2012 02:49 PM, Paolo Bonzini wrote:
> > > Regarding the testing - we ran WHQL networking tests on the device.
> > > If we provide the logs will it be sufficient? I believe the test
> > > coverage is much more comprehensive than anything that we will do with
> > > qtest.
> >
> > I'm not sure I'd agree about comprehensive
>
> Let's just say that passing WHQL can easily be months of work.

I'm very well aware of that.  But WHQL is designed to test the drivers as much 
as it's a hardware certification.

The bits I'm more interested about is edge case testing (things that could pose 
a security concern).  Since WHQL interfaces at the expected paths for the 
driver, it's unlikely that it can test any of this.

> > As you've seen from this thread, there's no a tremendous amount of
> > interest in supporting this device.
>
> That's your opinion.  Personally I would be very glad to help getting
> the vmw_pvscsi device in QEMU via the SCSI tree, and I don't see
> why VMXNET3 should be different.
>
> > But VMXNET3 isn't really special here.  From this point forward, I
> > would expect all new devices to come with a qtest-based test case.
>
> I find this to be hard to justify.
>
> With a grand total of 1 device tested, and with a coverage of almost
> zero even for that device, I think it's only sane to consider qtest
> a proof of concept.

How else are we going to get there other than asking people to use it?

Look, it's pretty darn simple to add a basic test for vmxnet3 to qtest that 
initializes the device.  I don't see what the big deal is asking for that.

> We can talk again when QEMU has:
>
> * libos bindings for at least PCI and the APICs, perhaps the 8259 too,
> and examples of how to use those;

Stefan's posted patches for the PCI bits.

> * mocks for network devices (block device mocks are needed too, but
> not for this device of course).
>
> I helped moving qtest forward hoping that other people (like Stefan
> is doing, and like Andreas did for QOM) would contribute other pieces
> of the infrastructure.  I certainly would have spent my time otherwise,
> had I known that the immediate outcome was making QEMU development slow
> and unwelcoming due to unreasonable prerequisites for contributing new
> devices.
>
> Certainly it is not reasonable to expect infrequent contributors to do
> our homework.

Perhaps the issue here is about what is expected from a test case?  All I expect 
is something that does basic device initialization and begins interacting with 
the device.

It doesn't need to start as an exhaustive test but I think there's tremendous 
value in at least having something to start with.  Otherwise, we'll continue to 
exist in the same chicken and the egg state.

Regards,

Anthony Liguroi

> Paolo