Re[2]: [lmi] lmi tests under cygwin

[Vadim Zeitlin]

On Tue, 01 Nov 2005 17:40:51 +0000 Greg Chicares <address@hidden> wrote:

GC> On 2005-11-1 16:22 UTC, Vadim Zeitlin wrote:
GC> > 
GC> >  I've tried to run tests under cygwin, as promised, and many of them fail
GC> > with this exception:
GC> > 
GC> >   Not all alert function pointers have been set.
GC> > 
GC> > I didn't have time to look into this in details yet but maybe you already
GC> > know what is it due and where should I concentrate my efforts?
GC> 
GC> I don't know what would cause it. Which tests fail this way? The 'alert'
GC> facility has its own unit test, 'alert_test.cpp'; does that one fail?

 Yes, it does fail and, looking in the code, it seems that it does it just
because set_alert_functions() is never called. Which, in turn, surely
happens because ensure_setup in alert_cli.cpp is not initialized because
the relevant object file must be discarded by the linker as nothing uses
it. I didn't know GNU ld did it but I do know that MS link.exe does it and
we even have special macros in wx/link.h which are used to work around this
"feature". So while I can try to fix it at Makefile.am level somehow, I
think it would be better to do it in the source.

GC> >  Other than that:
GC> > 
GC> > 0. the output is all but unreadable because of the license message 
repeated
GC> >    40 times; would it be possible to add code checking if environment
GC> >    variable LMI_LICENSE_OK is set [to 1] and set license_accepted to false
GC> >    in cpp_main.cpp if it is? I can't [easily] pass command line arguments
GC> >    to all tests from Makefile.am but I can set an environment variable.
GC> 
GC> That sounds like a strange limitation

 Yes, but after spending more time looking into this I think this is just
the way it is. You can have a global TESTS_ENVIRONMENT which is defined
before the tests run but you can't configure environment for individual
tests nor pass them command line arguments.

 This is indeed not very convenient. And I can't even say that this is the
price to pay for the ease of use as you could keep it (you don't have to
write any special rules for tests, this is taken care of completely
automatically) but still add the possibility to define test_foo_CMDLINE
easily. Unfortunately even if I submit a patch to automake and even
supposing it is accepted, it still wouldn't help us right now as automake
releases are not very frequent (last one was in 2003 AFAIK).

GC> Does this do what you need?
GC>   sed -e'/^This program is free software/,/MA 02111-1307, USA\.$/d'
GC> I wouldn't mind adding that to 'fancy.make' if you like.

 Err, no, this wouldn't change anything for me as the new makefiles don't
use fancy.make at all...


GC> > 1. test_comma_punct fails: **** test failed:   '-999' == '-,999'
GC> >    but it gives a message just before about it being ok with gcc < 4.0
GC> > 
GC> >    I wonder if it wouldn't be better to detect compiler version and not
GC> >    fail the test with gcc < 4.0?
GC> 
GC> What compiler version are you using?

 Latest from cygwin:

        gcc version 3.4.4 (cygming special) (gdc 0.12, using dmd 0.125)

GC> If you've tried this with 4.0.1, do you observe the problem?

 Test succeedes with

        gcc version 4.0.2 (Debian 4.0.2-2)
        
GC> However, you're asking for something different: to suppress the test
GC> for compilers known to fail it anyway. That's one school of thought:
GC> all unit tests must always pass.

 Yes, indeed, this is mine.

GC> I subscribe to a different school,
GC> which holds that errors should not be hidden. One could argue it
GC> either way; this is simply the choice I've made.

 Ok, I'll keep this in mind.


GC> > 2. test_value_cast, test_path_utility, test_math_functors
GC> >    fail with multiple errors
GC> > 
GC> >    test_numeric_io fails with test failed:   '15' == '16'
GC> > 
GC> >    is this expected?
GC> 
GC> Does it say which line? Is it line 121? The source for lines 120-121 is:
GC>     // TODO ?? Fails for como with mingw, but succeeds with 0.45036 .
GC>     BOOST_TEST_EQUAL(15, floating_point_decimals(0.4503599627370497));

 This is exactly this line.

GC> I've never looked into that. Your report is valuable: it suggests that
GC> this is not some como quirk, and that it should be looked into. I don't
GC> have the time to do that right now, but feel free to consider it if you
GC> feel inclined.

 To be honest I'm a bit lost in what the code does right now, but I'll try
to look.

GC> > 4. test_expression_template_0 succeeds but takes ~400 seconds to do it
GC> >    (on a 3GHz Xeon CPU), isn't it a bit too long?
GC> 
GC> Which test takes that long?

 The naive STL one, but not only it:

GC> Here's what I observe:
GC> 
GC>     Speed test: C
GC>   [1.658e-007] 10000000 iterations took 1657 milliseconds
GC>     Speed test: STL naive
GC>   [1.009e-006] 100000 iterations took 100 milliseconds
GC>     Speed test: STL smart
GC>   [4.184e-007] 1000000 iterations took 418 milliseconds
GC>     Speed test: valarray
GC>   [2.331e-007] 1000000 iterations took 233 milliseconds
GC> 
GC> The timer is designed to limit the number of iterations so that
GC> no single test takes more than ten seconds:
GC>     double const max_seconds = 10.0;
GC> but it won't perform less than one iteration. Would you please
GC> cut and paste your output, so that I can see the iteration count
GC> and timing for each of the four tests?

 Here it is:

  Speed test: C
[4.820e-07] 10000000 iterations took 4820 milliseconds
  Speed test: STL naive
[2.007e-05] 10000000 iterations took 200686 milliseconds
  Speed test: STL smart
[1.522e-05] 10000000 iterations took 152174 milliseconds
  Speed test: valarray
[3.779e-06] 10000000 iterations took 37793 milliseconds

 I don't understand why does it do 10^7 iterations here and just 10^5 in
your case but I didn't have time to look at the code yet.

 
 Thanks,
VZ