Re: [lmi] Patches to run the tests with, or without, any password

[Greg Chicares]

On 2014-11-11 22:26Z, Boutin, Wendy wrote:
> Vadim Zeitlin wrote: 
> [...]
>>  This change also takes care of the failure in validate_output_mec test.
>> However there is still one other test case which passes with the
>> unutterable password, but fails without it: input_validation one. The
>> reason for this is clear enough, in normal use trying to run
>> CoiMultiplier.cns results in the following message box
>> 
>>      Error
>>      Input validation problems for '':
>>      Lowest COI multiplier entered is 0.85, but 1 is the lowest multiplier 
>> allowed.
>>      [file .\input_realization.cpp, line 260]
>> 
>> instead of the expected, as described in the item 15 of the testing
>> specification, one:
>> 
>>      Input validation problems for '':
>>      COI multiplier entered is '0', but it must contain at least one number 
>> other than zero.
>>      [file .\input_realization.cpp, line 260]
>> 
>>  The only solution I see here is to check whether the password was given
>> explicitly in the test. But even then I am not sure if the test should be
>> skipped if it wasn't given, or just check for a different error message?
> 
> I think it's better to check for the second error message then to test whether
> a password was used or not. It's a good validation check. I'm guessing Kim 
> [...]

To make sure, I asked Kim. First of all, here's the test she
runs manually, both (1) with no password, (2) with '--mellon'
(but not with the unspeakable password, which has the same
effect as (2) here). Create a new '.ill' document, and choose
a group executive carveout product (one that allows a COI
multiplier less than one). Test only its
  <object class="InputSequenceEntry" name="CurrentCoiMultiplier">
field, in numerous ways (to which I've added one or two),
hitting "OK" after each:
  (a) erase the field
  (b) enter a negative number
  (c) enter zero
  (d) enter a positive number less than the minimum
  (e) enter the minimum
  (f) enter the number one
  (g) enter a number greater than one
and make sure any diagnostics observed are exactly as
prescribed by Input::RealizeCurrentCoiMultiplier().

Years ago, with a vendor illustration system (not lmi), an end
user did this:
  Tab to the COI-multiplier input field
  hit spacebar by accident
  ignore that accident, and enter other parameters
  press OK
and a disaster ensued. The blank was interpreted as zero, which
was silently accepted. An illustration was produced. With all
mortality charges multiplied by zero, it showed values much more
favorable than reality; that's a compliance problem. To verify
that lmi continues to guard against that, we take pains to edit
the input and force the above values into the text control. It's
not the same thing as running a census that was previously saved
with each of those values in a separate cell.

Here's how we'd like to change that test so that it encompasses
everything that is at present done manually. First, open an
'.ill' file that we'll provide to replace "CoiMultiplier.cns".
Its parameters will all be defaults, except that we'll choose a
group executive carveout product. Then we'll force values (a-j)
above into the
  <object class="InputSequenceEntry" name="CurrentCoiMultiplier">
field, simulating "OK" after each one, and test the outcome
against the diagnostics that Input::RealizeCurrentCoiMultiplier()
prescribes.

The specific diagnostics depend in certain cases on whether the
test suite is run (1) with no password, or (2) with either of
the available passwords (both of which behave the same here).
I think these will run with no diagnostic:
  (1) e,f,g
  (2) d,e,f,g
while the others will elicit various diagnostics.

The "minimum" is 0.9 for (1), and effectively zero for (2).
I say "effectively" because the datatype is unrestricted as to
sign at present, but there's a code comment:
    std::vector<tnr_unrestricted_double> CurrentCoiMultiplierRealized_      ; 
// tnr_nonnegative_double
that suggests someday we might restrict it to be nonnegative; yet
negatives, as you will see, elicit a downstream diagnostic.

Vadim--because you don't have proprietary product files, you can
just use the 'sample' product, whose "minimum" is one, and then
tests (1)(d,e) will fail for you; or, if you don't want them to
fail, change its DB_MinInputCoiMult parameter to 0.9 .

Because this test uses a proprietary produce, we'll want to run
it only with the '--proprietary' flag discussed elsewhere, which
is enough to prevent failure on Vadim's machine. (The original
discussion said:
|  --extra [that name could use improvement] Also run tests that
|    depend on proprietary files or products.
and I now think '--proprietary' is the best name for it.)