Re: [Qemu-devel] [PATCH] qemu-iotests: Fail test if explict test case number is unknown

[Fam Zheng]

On Wed, 09/24 10:05, Stefan Hajnoczi wrote:
> On Tue, Sep 23, 2014 at 10:26:26AM +0800, Fam Zheng wrote:
> > When we expand a number range, we just print "$id - unknown test,
> > ignored", this is convenient if we want to run a range of tests.
> > 
> > When we designate a test case number explicitly, we shouldn't just
> > ignore it if the case script doesn't exist.
> > 
> > Print an error and fail the test.
> 
> I guess you want this because new test case numbers are often sparse?
> For example, 096, 096, 098 are missing.  Presumably those patches are
> waiting to be merged but we already merged 100.
> 
> It's helps to explain "why" this patch is useful.

OK.

Because when explicitly running one test just beyond the biggest case number
(happens when you write a new test but didn't update group file), with the exit
status 0 and "PASS" in the last line of output, it is hard to realize that
group file is missed without paying a fairly close attention.

# ./check -qcow2 999
999 - unknown test
QEMU          -- 
/home/fam/build/last/tests/qemu-iotests/../../x86_64-softmmu/qemu-system-x86_64
QEMU_IMG      -- /home/fam/build/last/tests/qemu-iotests/../../qemu-img
QEMU_IO       -- /home/fam/build/last/tests/qemu-iotests/../../qemu-io
QEMU_NBD      -- /home/fam/build/last/tests/qemu-iotests/../../qemu-nbd
IMGFMT        -- qcow2 (compat=1.1)
IMGPROTO      -- file
PLATFORM      -- Linux/x86_64 fam-t430 3.16.2-201.fc20.x86_64
SOCKET_SCM_HELPER -- /home/fam/build/last/tests/qemu-iotests/socket_scm_helper

Passed all 0 tests
PASS

> 
> > diff --git a/tests/qemu-iotests/common b/tests/qemu-iotests/common
> > index 70df659..2403a20 100644
> > --- a/tests/qemu-iotests/common
> > +++ b/tests/qemu-iotests/common
> > @@ -382,10 +382,16 @@ BEGIN        { for (t='$start'; t<='$end'; t++) 
> > printf "%03d\n",t }' \
> >                      echo $id >>$tmp.list
> >                  else
> >                      # oops
> > -                    echo "$id - unknown test, ignored"
> > +                    if [ "$start" == "$end" -a "$id" == "$end" ]
> > +                    then
> > +                        echo "$id - unknown test"
> > +                        exit 1
> > +                    else
> > +                        echo "$id - unknown test, ignored"
> > +                    fi
> >                  fi
> >              fi
> > -        done
> > +        done || exit 1
> 
> What is the purpose of this line?

The exit inside the loop is in a subshell so won't cause the whole script to
exit.

Fam