Re: autotest and make distcheck

[Ralf Wildenhues]

* Andrei Kholodnyi wrote on Sun, Sep 12, 2010 at 07:09:58PM CEST:
> On Sun, Sep 12, 2010 at 6:19 PM, Ralf Wildenhues wrote:
> >
> > Well, distcheck (which comes from Automake) really aims to be a "come
> > on, let me ensure this package is good in all kinds of ways" target,
> > so it also tries out installcheck.  This makes sense, too, because it
> > already builds and installs the package somewhere; for packages where
> > that is an expensive operation, you might not want to do it more often
> > than necessary.
> 
> I see your point here, of cause it makes sense.
> However how it will work e.g. in a case of cross-compile?

Not much different from native compile, and not necessarily much
different from 'make check': either there is a simulator to execute your
programs, or you skip execution tests.  Now, there is no direct support
in Autotest for this, but you can write a shell function or m4 macro for
this.

For example, the Libtool testsuite does something like this in the
LT_AT_EXEC_CHECK macro in
<http://git.savannah.gnu.org/cgit/libtool.git/tree/tests/testsuite.at>.
You would probably have to use something slightly different in your
package, since that macro is pretty tailored to Libtool; if you need
help we can figure something out.

More generally though, many packages do not require that 'make
distcheck' works in a cross-compile situation.

> I can do build/install/uninstall it on my host but testsuite will
> always fail since it is for the different HW.

Sure.

> > If your 'installcheck' is not generally usable, then I suggest just not
> > hooking the autotest testsuite execution to it (i.e., omit the
> > installcheck-local rule).  If it is generally usable, then I wonder why
> > it shouldn't work in the distcheck setting for you.
> 
> Well, if some tests fail then installcheck fails as well.
> But tests could fail e.g. due to known bugs etc,
> and it should not prevent me from checking whether install/deinstall
> of my package works properly.

For known bugs you can use AT_XFAIL_IF to mark them as expected
failures, or skip tests (using an exit status of 77) that you don't run
in some situation.  Please be aware that AT_XFAIL_IF is evaluated right
at AT_SETUP time.

Hope that helps.

Cheers,
Ralf