Re: [lmi] Manual test of automated GUI test

[Vadim Zeitlin]

On Sat, 18 Oct 2014 22:44:58 +0000 Greg Chicares <address@hidden> wrote:

GC> I've built lmi with all the latest GUI-testing patches, using a
GC> wx snapshot as of 2014-10-18T16:40Z. It seemed like a good idea
GC> to start with deliberate mischief, so I did this:
GC>   ./wx_test.exe --data_path=/opt/lmi/data
GC> without a command-line argument to suppress the automatic
GC> "Help | About" dialog--remembering that running the last version
GC> this way caused an unanticipated problem.

 Sorry, I've only tested with this special command line argument so far, I
thought I had mentioned this but apparently I didn't, sorry again about
this.

GC> [Others could easily do this by accident, so would it make sense
GC> to forestall it...e.g., by issuing an 'Esc' keypress before doing
GC> anything else in the GUI test, if that's possible?]

 OK, I'll look into doing this.

GC> Anyway, I ran the new version this way, and saw this messagebox:
GC> 
GC> Programming error
GC> A document created during unattended test hasn't been closed, please report 
this.

 This is an assert trying to catch the errors in the test code itself, as
the tests are supposed to close any windows they open. I'll try to see if I
can reproduce this when running it with the same command-line options as
you do.

GC> Here are the contents of the "Log Messages" window:
GC> 
GC> 22:37:38: Starting automatic tests:
GC> 22:37:39:     about_dialog_version: ERROR (Assertion failure: Expected 
wxDialog dialog was not shown. [file 
/opt/lmi/local/include/wx-3.1/wx/testing.h, line 315, in ReportFailure()].)

 I think the test expectations were completely skewed by the initially
shown dialog.

GC> 22:37:39: Error: can't open file 'wx_test.conf' (error 2: the system cannot 
find the file specified.)

 This is expected, this file needs to be created manually using the example
from my 2014-09-08T11:07:20Z email as the model. However this test failure
should not result in the failure of the subsequent tests, so I think they
are also failing because of that initial dialog.

 This does point to a big problem with the current tests: once things go
off the rails, there is no hope to get back onto them as there is no known
sane initial point we can always return to. We could try to blindly close
any open dialogs and left over documents, but I'm not sure this is such a
good idea.

GC> When I clicked "OK", it crashed;

 I hadn't seen this yet. I'll try to debug it yet a.s.a.p. but for now I
wanted to send this email to tell you about the need for the option
suppressing the initial dialog. I do hope it's going to work better with
it, but please let me know if it still doesn't.

 Thanks,
VZ