Re: [Qemu-devel] [PATCH RFC] tests: Run qtest cases in parallel

[Gonglei (Arei)]

> -----Original Message-----
> From: Daniel P. Berrange [mailto:address@hidden
> Sent: Tuesday, September 27, 2016 6:15 PM
> To: Fam Zheng
> Cc: Gonglei (Arei); address@hidden; John Snow; address@hidden
> Subject: Re: [Qemu-devel] [PATCH RFC] tests: Run qtest cases in parallel
> 
> On Fri, Sep 23, 2016 at 05:59:05PM +0800, Fam Zheng wrote:
> > On Fri, 09/23 09:39, Gonglei (Arei) wrote:
> > >
> > > Hi Fam,
> > >
> > >
> > > > -----Original Message-----
> > > > From: Qemu-devel
> > > > [mailto:address@hidden On
> > > > Behalf Of Fam Zheng
> > > > Sent: Friday, September 23, 2016 3:58 PM
> > > > To: John Snow
> > > > Cc: address@hidden; address@hidden
> > > > Subject: Re: [Qemu-devel] [PATCH RFC] tests: Run qtest cases in parallel
> > > >
> > > > On Wed, 09/21 14:24, John Snow wrote:
> > > > >
> > > > >
> > > > > On 08/12/2016 05:19 AM, Fam Zheng wrote:
> > > > > > Previously all test cases in a category, such as check-qtest-y, are
> > > > > > executed in a single long gtester command. This patch separates each
> > > > > > test into its own make target to allow better parallism.
> > > > > >
> > >
> > > That's will be great if we can specify a test to run, especially for the
> scenario
> > > which add one use qtest case.
> > >
> > > For example:
> > >
> > >  # make check test-crypto-cipher
> > >
> > > then only run the tests/ test-crypto-cipher.
> > >
> > > Do you think it makes sense?
> >
> > Or more likely:
> >
> >     # make check TESTS="test-crypto-cipher test-crypto-hash ..."
> >
> > Usually I just extract the gtester command line with V=1 and run it from my
> > shell prompt.  Feel free to send a patch, though.
> 
> Shouldn't even need todo that in most cases - I tend to just do
> 
>   make tests/test-crypto-cipher && ./tess/test-crypto-cipher
> 
> If there are tests which rely on some environment set by the Makefile,
> then really they should be fixed to have sensible defaults so that they
> can be directly executed.
> 
Thanks for your reminding! It works fine.

# ./tests/test-crypto-cipher 
/crypto/cipher/aes-ecb-128: OK
/crypto/cipher/aes-ecb-192: OK
/crypto/cipher/aes-ecb-256: OK
/crypto/cipher/aes-cbc-128: OK
/crypto/cipher/aes-cbc-192: OK
/crypto/cipher/aes-cbc-256: OK
/crypto/cipher/des-rfb-ecb-56: OK
/crypto/cipher/cast5-128: OK
/crypto/cipher/serpent-128: OK
/crypto/cipher/serpent-192: OK
/crypto/cipher/serpent-256a: OK
/crypto/cipher/serpent-256b: OK
/crypto/cipher/twofish-128: OK
/crypto/cipher/twofish-256: OK
/crypto/cipher/aes-xts-128-1: OK
/crypto/cipher/aes-xts-128-2: OK
/crypto/cipher/aes-xts-128-3: OK
/crypto/cipher/aes-xts-128-4: OK
/crypto/cipher/cast5-xts-128: OK
/crypto/cipher/aes-ctr-128: OK
/crypto/cipher/aes-ctr-192: OK
/crypto/cipher/aes-ctr-256: OK
/crypto/cipher/null-iv: OK
/crypto/cipher/short-plaintext: OK

# ./tests/virtio-net-test
**
ERROR:tests/libqtest.c:561:qtest_get_arch: assertion failed: (qemu != NULL)
Aborted (core dumped)

# ./tests/virtio-blk-test
**
ERROR:tests/libqtest.c:561:qtest_get_arch: assertion failed: (qemu != NULL)
Aborted (core dumped)

But they work after I set the environment variable to specify architecture:
 
# QTEST_QEMU_BINARY=x86_64-softmmu/qemu-system-x86_64 ./tests/virtio-net-test
/x86_64/virtio/net/pci/basic: OK
/x86_64/virtio/net/pci/rx_stop_cont: OK
/x86_64/virtio/net/pci/hotplug: OK

# QTEST_QEMU_BINARY=x86_64-softmmu/qemu-system-x86_64 ./tests/virtio-blk-test
/x86_64/virtio/blk/pci/basic: OK
/x86_64/virtio/blk/pci/indirect: OK
/x86_64/virtio/blk/pci/config: OK
/x86_64/virtio/blk/pci/msix: OK
/x86_64/virtio/blk/pci/idx: OK
/x86_64/virtio/blk/pci/hotplug: OK

So, Maybe we should add check if the environment relied on is set
before executing specific operations in this kind of tests. Right?

Regards,
-Gonglei