Re: [lmi] Re: mdgrid and enabling/disabling axis

[Greg Chicares]

On 2006-1-11 21:52 UTC, Vadim Zeitlin wrote:
> 
>  I'm continuing this private discussion we had with Eugene on the mailing
> list as I think it may be interesting to the others -- and I also hope you
> may help to find the best answers to the questions below.

Pleased to meet you, Eugene--or perhaps I should say очень приятно познакомиться
с вами, Евгений, as I see you're using CHARSET=KOI8-R.

> ET> (1). Allow a user to add/remove axis (dimension) to/from a product domain
> ET> Data: We should probably add a flag for every axis used in mdgrid that
> ET> indicates whether the axis participates in a product domain (product value
> ET> being dependant or not on the axis value). It implies that the underlying
> ET> MultiDimGridTable have to support enableing/disableing of a particular 
> axis
> ET> in a product domain. Maybe those flags (mentioned in the sentence above)
> ET> should be put inside MultiDimGridTableAny and not into the MultiDimGrid).

I'm not sure how you're using the term "product". The way we use it in our
industry seems odd to outsiders: the word connotes something concrete, but
financial "products" are intangible. Anyway, we'd say 'sample' is a product
because a file 'sample.pol' exists, and we'd say that the files referenced
in that file (probably 'sample.*') embody the product. I think we've shared
all the 'sample.*' files with you, but we have dozens of other products.

The legacy 'ihs' application edits products by editing these files separately.
Ultimately we'd like to change the formats of all these files to xml, then
combine them into one file, say, "sample.product". I see that xml conversion
as an almost orthogonal task. It's a large enough task that it makes sense to
handle it separately. For now, it might be easier for the new product editor
to edit the separate files, though I'm not sure we've really decided that.
Maybe it would even be a good idea first to complete a prototype that edits
only '.db4' files, because they're the most complex file type. I'm not trying
to change the requirements here--I'm just pointing out that we still have
freedom to change the order in which tasks are done.

With today's separate files, MDGrid would be used only for '.db4' files. You
have chosen a good term, so let's say that an entity in this file "depends"
on an axis iff the user must be able to enable or disable it. We can say the
entity "varies" across an axis if that axis is not "degenerately" unused
(and therefore disabled). For example, we have an entity 'CurrCOITable' that,
in 'sample.db4', varies by Gender and Smoker. For another product, it might
vary by Class as well. Users often copy a file like 'sample.db4' and edit it
to create another product, so they need to be able to enable or disable the
Class axis for this entity. The axes for '.db4' entities are
  {Gender, Class, Smoking, Issue Age, Underwriting, State, Duration}
and in this case we'd say that 'CurrCOITable' "varies" by Gender and Smoker,
and "depends" on all seven axes (in today's legacy application at least, but
see below). Therefore, the other five axes are "degenerate" in this case.

Every entity in every '.db4' file today depends on every one of those seven
axes, and that's the complete set--there are no others, today. That's the way
the legacy application works, but it's really not the right design. It's not
right even today, because, in the semantics of the problem domain, some
entities actually must not vary across any axis. For example, 'CurrCOITable'
really must never depend on the Duration axis. You can make it vary by that
axis in the legacy application, but that produces an error message downstream
when you actually use the product file in lmi. That's a problem we'd like to
fix in the new product editor, so that 'CurrCOITable' will depend on
  {Gender, Class, Smoking, Issue Age, Underwriting, State} (not Duration)
and the user won't get today's "Oops, you weren't supposed to do that" error.

The legacy editor's "all axes are dependent" approach is even less right for
the ultimate solution, where entities that reside today in another file, e.g.
'.pol', might depend on
  {State, Duration}
yet all entities would be stored in the same xml file.

Thus, each entity's semantics determine which axes it depends on. But these
dependencies are today embodied, haphazardly, in lmi's modules that use the
product files. They aren't expressed in the product files yet, but will need
to be added there when we convert the files to xml.

IOW, I think we're both saying the same thing, as Vadim says more tersely:

>  I agree. Indeed, the information about whether the values depend on/vary
> along the axis must be passed to the table -- it may be important for it.
> 
> ET> UI: At that point there are two approaches discussed.
> ET> First one with checkboxes near every axis that control directly those 
> flags
> ET> from above.
> ET> Second one with the additional 'Invariant' (invariant is inappropriate 
> here,
> ET> because it means that something (product for example) does not depend on 
> the
> ET> axis, and its not the axis that is invariant -- better 'unapplicable')
> ET> option in drop-down axis value lists.

Good, let's make sure we choose the terminology carefully. The particular terms
we choose don't matter as long as they're clearly defined, so feel free to
suggest better terms than these to relate an entity to an axis:

  depend
    the entity could potentially use the axis nondegenerately; equivalently,
    the entity could potentially have distinct values across the axis

  vary
    the entity actually uses the axis nondegenerately; equivalently,
    the entity actually has distinct values across the axis

  varying implies dependent; and (not dependent) implies (not varying)

But that's not enough terms--we need a third, like "displayed", meaning:
  the entity depends on the axis
  the entity varies across the axis
  the axis has been chosen as either the X axis or the Y axis
  a slice involving this axis is presented in the grid control

Now...

  legacy implementation (uses checkbox method)
    in mode 0, where entity cannot be redimensioned:
      checkbox is checked iff axis is displayed
      checkbox is enabled if entity varies across axis
        (except that no more than two can be checked: so as soon as
        two have been checked, the others are disabled)
    in mode 0, where entity can be redimensioned:
      checkbox is enabled if entity depends on axis
      grid not shown

  new product editor with checkbox method
    would this be different from the legacy implementation?

  new product editor with combobox method; dimensions locked
    if displayed
      combobox could be either disabled or hidden
    if varying but not displayed
      combobox forces a choice among actual axis values
    if dependent but not varying
      combobox would show a "special value"

Thus, I have to agree with you: I was mistaken to suggest "Invariant" in an
earlier email. In idiomatic US usage, the "special value" should preferably
be "Not applicable", or "N/A" if the former doesn't fit. "Invariant" is also
wrong because it's not in idiomatic use except among mathematicians, who
would in any case use it more precisely.

Then we have one last case, and I'm not sure how to handle it, but I'd guess:

  new product editor with combobox method; dimensions not locked
    if displayed
      combobox could be either disabled or hidden
    if dependent but not displayed
      combobox offers a "special value" as well as all actual axis values

Here's where I think the combobox method becomes unwieldy. Take, for example:

  Strike:   "Not applicable" | {"European", "American"}

First, suppose Strike is displayed as the X axis, and the user wants to
redimension the entity so that it no longer varies across that axis. How
would that be done? It seems to require two steps:
 - change the X axis selection, so that Strike is no longer displayed
   its combobox is now shown, with, say, "European" selected
   (whether to choose "European" or "American" is a problem raised below)
 - change the selection to "Not applicable"

Second, suppose Strike is not displayed. The grid shows a slice with one
Strike value, say, "European". The user intends to change that to "American"
but accidentally chooses "Not applicable". Now the entity has already been
reduced in dimension, so picking "American" won't restore it. Data was lost
without an explicit confirmation, and that can be frustrating.

We haven't eliminated the "redimension or don't redimension" modality of the
legacy interface. In effect, we've pushed the modality into the comboboxes,
with some surprising consequences. For "State", we'd have 53 items that don't
change dimensions, plus one "magic" value that does. This reminds me of old
text interfaces that said "Enter your age, or 99 to quit". Maybe the second
case above could be addressed by popping up an "Are you sure?" messagebox
(another expression of modality), but I don't see how the first one can.
I'm beginning to think that users might perceive the combobox method as less
friendly than the checkbox method.

> ET> Question: how should we address the merging problem? When user marks axis 
> as
> ET> unapplicable (disables it from product domain) and there are different
> ET> values of product for different values of that dimension, which value 
> should
> ET> editor choose? A good example is a product that depends only on Gender 
> that
> ET> has, say value of 5 for 'Male' and 10 for 'Female', imagine that user
> ET> disables that Gender dimension from the product -- what value should it 
> have
> ET> after -- 5 or 10 (or even NULL)?
> 
>  Excellent question... I don't think there is a single best solution for
> all situations so I believe we should provide some flexibility here. For
> example, my first idea was to check if the values for all genders were
> really the same -- but while this would probably work well in this case,
> it might be not the best thing to do if there are a lot of values. So,
> again, IMHO this should be configurable. I see 2 ways to do it:
> 
> - supposing we have some MDGridTable::ExcludeAxis(axis) (as per above), we
>   should delegate the decision to it, e.g. it could return true/false to
>   allow/forbid disabling this axis

Using the same terminology as above:

  if not dependent
    then the axis can't be introduced
    i.e., (not dependent) implies (not varying)
    or an axis can't be actually used if it can't be potentially used

and I think that's what the "dependent" relationship is needed for.

But to forbid disabling an axis would mean "dependent implies varying", which
is not true.

Any entity can be redimensioned to a scalar.

> - ask the user what to do
> 
> Does anyone see anything better?

We're replacing an (N+1)-dimensional array with an N-dimensional slice.
Arbitrarily pick the first axis-value, and use its data-values, deleting the
data-values that formerly existed along other axes. This matrix

  00  01
  10  11

with axes A and B would become

  00 01 if axis A is removed, or

  00 10 if axis B is removed.

> check if the values for all genders were really the same
If they are, then this method does the right thing.

> - ask the user what to do
That would seem to call for a dialog, which complicates the design.
But here's another idea: quoting from above,

  if varying but not displayed
    combobox forces a choice among actual axis values

and we could use the chosen axis-value (instead of the first one) to
perform destructive slicing. That keeps the number of UI elements
small, but maybe it's too "cute".

> ET> (2). Allow a programmer to control the set of axis available (for adding
> ET> into a product domain) to a user for a given product
> ET> Data: nothing to be done probably as we have already stated the set of
> ET> dimensions to vary the product value upon in the MultiDimGrid constructor.
> ET> UI: it should be addressed along with 1_UI.
> ET> Question: could we imagine the below UI:
> ET> /-----------------\
> ET> |  X axis: Strike |
> ET> |  Y axis: Gender |
> ET> |                 |
> ET> |x Strike: -----  |
> ET> |x Gender: -----  |
> ET> |x Option: value? |
> ET> |-----------------|
> ET> |  Maturity  +add |
> ET> |  Smoking   +add |
> ET> \-----------------/
> 
>  Sorry, what are the "+add"? I thought we'd just have just unchecked
> checkboxes near the unused axis?

I take "add" as meaning "inject a new dimension". This is interesting.
It expresses the underlying modality through grouping in the layout.
Let me make a copy and add commentary on the right to see whether I
understand this correctly:

> ET> /-----------------\
> ET> |  X axis: Strike |  GWC>   displayed
> ET> |  Y axis: Gender |  GWC>   displayed
> ET> |                 |
> ET> |x Strike: -----  |  GWC>   varying but not displayed
> ET> |x Gender: -----  |  GWC>   varying but not displayed
> ET> |x Option: value? |  GWC>   varying but not displayed
> ET> |-----------------|  GWC> [crossing this line means redimensioning]
> ET> |  Maturity  +add |  GWC>   dependent but not varying
> ET> |  Smoking   +add |  GWC>   dependent but not varying
> ET> \-----------------/

                            "not dependent" isn't shown: it may as
                            well not exist

Now "+add" might be a small pushbutton with a plus sign, which causes
a dependent axis to become varying: e.g., pressing Maturity + gives:

...
> ET> |x Gender: -----  |  GWC>   varying but not displayed
> ET> |x Option: value? |  GWC>   varying but not displayed
> ET> |  Maturity  ---  |  GWC>   moved up here: now varying but not displayed
> ET> |-----------------|  GWC> [crossing this line means redimensioning]
> ET> |  Smoking   +add |  GWC>   dependent but not varying
> ET> \-----------------/

Have I understood this correctly?

Now, of course, we need a way to move something from above the line to
below it. That could be done with a minus-sign pushbutton for every axis
above the line:

> ET> |  Maturity  --- GWC> [-]
> ET> |-------------------------|
> ET> |  Smoking       GWC> [+] |
> ET> \-------------------------/

Compared to the "checkbox" design (which is really equivalent):

  advantage: grouped order of axes on screen distinguishes
    displayed
    varying but not displayed
    dependent but not varying

  disadvantage: order of axes changes when entity is redimensioned
    e.g. if six axes are below the line and you click '+' next to
    the sixth one, then it moves above the other five; if you were
    staring at it, then it almost seems to vanish

But redimensioning is an infrequent operation, and I think users might
really prefer this to all the other approaches we've discussed. It's
hard to say for sure without a prototype to share with a few end users.

> ET> Or another one:
> ET> /-----------------------------\
> ET> |      X axis: Strike         |
> ET> |      Y axis: Gender         |
> ET> |                             |
> ET> |      Strike: -----          |
> ET> |      Gender: -----          |
> ET> |      Option: value?         |
> ET> |-------------------^^close^^-|
> ET> | /--------\     /----------\ |
> ET> | | Strike | =>x | Maturity | |
> ET> | | Gender |     | Smoking  | |
> ET> | | Option | +<= |          | |
> ET> | \--------/     \----------/ |
> ET> \-----------------------------/
> ET> The ^^close^^ could be a button that shows/hides the bottom part (plays 
> the
> ET> role of that global checkbox).I hope its pretty self-explanatory.
> 
>  I think this one would take too much screen estate. But it's still an
> idea to consider...

Yes, the preceding idea accomplishes the same thing in less space,
and seems easier to use.

> ET> (3). Prevent user from accidentally adding or removing an axis(dimension)
> ET> from a product domain
> ET> Data: nothing to be done, because its a UI-only feature.
> ET> UI: Two approaches were discussed:
> ET> First one with a sort of a modal dialog (from the legacy editor approach)
> ET> that allows to select the applicable axis for the product value domain.
> ET> Second one with a global checkbox (or button) freezing enabling/disabling 
> of
> ET> axis (or removing that 'Unapplicable' option from drop-down value lists 
> for
> ET> applicable dimensions and disable unapplicable axis drop-down lists)
> 
>  To be honest, more I think about it, less sure I become how useful this
> feature is. Even if you accidentally check the "use this axis" checkbox or
> select a value different from "N/A", it's not really a problem to uncheck
> it back or to select "N/A", is it? Maybe we could start without it and ask
> the users how do they feel with this UI and add this extra protection only
> if it is deemed necessary by them?

Many users would never redimension anything. The rest would seldom do it.
What concerns me most is that changing the selection in a combobox, perhaps
by accident or through ignorance, could irreversibly change the data,
without any confirmation. Users would need to use the combobox routinely
to select which slice of data to display. Mingling one item that changes
changes the data in with other items that don't seems likely to cause
frustration.

The other designs use separate controls for redimensioning. Users who don't
need to perform that operation can just ignore those controls. They don't
have to beware of unsafe options in the controls they need to use. For
those designs, I don't think we'd need any global lock-dimensions switch.
When editing read-only files, we could simply disable the controls that
change dimensionality (as well as disabling changes to grid data).

> ET> (4). Read-only mode in the mdgrid widget
> ET> Data: Probably a flag or a boolean function in MultiDimGridTable that
> ET> controls whether we can alter product value domain. And another parameter 
> to
> ET> MultiDimGrid indicate whether we are in read only mode.
> 
>  If we have MultiDimGridTable::IsReadOnly(), we probably don't need
> anything else in the grid, do we?
> 
> ET> UI: if read-only mode then don't show UI features concerning 1
> 
>  Notice that for this we'd need to be able to detect whether the values
> depend or not on the given axis... So maybe we finally need a function to
> do this in MultiDimGridTable -- and it could be also used to check if the
> user can disable this axis (see second half of point (1) above).
> 
> ET> and disable editing in the wxGrid table showing read/only product
> ET> values.
> 
>  Yes.

I think we can omit all of this as long as we choose a design that
makes it difficult to redimension the data by accident--by using
separate controls instead of overloading combobox items to combine
dimension-selection with dimension-changing.