RE: [Heartlogic-dev] OHL v2 alpha test

[William L. Jarrold]

On Wed, 23 Mar 2005, Joshua N Pritikin wrote:

> On Wed, 2005-03-23 at 01:00 -0600, William L. Jarrold wrote:
> > Darn, I think I am stuck in power outage time again.  I guess 12pm IST is
> > 12am CST perhaps?
> >
> > Well, I think it is most efficient if I reply to your msgs when I have had
> > a chance to e.g. try the levels once the power it back on.  My day is
> > chock full tomorrow so alas it will have to wait until Thurs.  Ug.
>
> Gosh, feedback within 2 or 3 days?  I'm acclimated to waiting 1-2 months
> to hear back from you.  ;-)

Yeah, sorry.  I suck.

> > Also, when I messed around with it yesterday I could not make any sense of
> > what the stats meant.  E.g. N did not seem to correspond to the number of
> > replies I had done and M did not seem to be a mean and S (or I forget what
> > the letter or abbrev was) did not seem to correspond to an sdev.  Perhaps
> > this is just one of those buggy things you have not gotten around to
> > fixing yet.
>
> No, you guessed correctly.  I am using the abbreviations according the
> APA Publication Manual.  What perhaps isn't obvious is that the samples
> include everyone who has rated the particular construal, not just you.

I thought it might be doing that.  But even so, the numbers did not seem 
to make sense...Well, it doesn't matter.  The numbers now seem to make 
sense.

Anal point: In my dissertation items were rated on a scale of 1 to 5 
rather than -2 to +2 as I belive that you do.  I am not sure if 
statistical analysis requires 1-5 or not.  Shouldn't be too hard to 
convert the data.

> > Anyway, I spoke to Peter and he is not too concerned with total and
> > complete alignment with CLib.
> >
> > I think the main task facing me is to come up with a set of test cases
> > that go beyond the dissertation.  E.g. Tracy wants a banana.  Tracy gets a
> > banana/apple/strawberry/piece of coal.  Then modify the model to handle
> > them all.
>
> OK
>
> > Another task might be to simply get you all the surveys from the
> > dissertation.  Ug.  This has been on the list forever.
>
> Why?  I believe you already sent me one survey.

Because we would like to be able to replicate the dissertation.  If
the findins of the dissertation hold up on the WWW then we can be
that much surer that the methodology of sampling random WWW users
works.

> I diligently copied
> everything for OHL v1.  For OHL v2, there are a few changes:
>
> 1. There are a few minor changes in wording.
>
> 2. The reason and the emotion are rated separately, not together.

Hrm, I'm confused, I didn't seem an emotion in many of the items.  I tried 
about 10 items and I did not see an emotion work in most of them.  I did 
see one in which "tracy feels happy."  That's an emotion.  I saw one in 
which the closest thing to an emotion word was the word "anticipation." 
But anticipation is not really inherently positive or negative and 
therefore does not fit e.g. Ortony's definition of emotion.  It might be 
considered an affective state (i.e. a level of arousal).  I found several
cases of "indifferent."  I suppose that might be considered an emotion. 
It was in my dissertation....I really think we need to keep the emotions 
small.  E.g. just focus on happy vs sad vs indifferent.  Each item should
have an emotion in it.

There is the heading "Understood As."  This seems to be a survey that is 
more about how a computer understands a story and less about how one makes
inferences about the emotions of a characters in a story.

By the way what do all those levels (i.e. g t s 3 m M) mean?  Having 
different levels makes it difficult to obtain experimental validity.  Ugh, 
I hate to rain down on all the work you have done for this.

> Why?
> Because I believe the emotion is dependent on the reason.  Hence, if the
> reason is unbelievable then there is no point in rating the emotion.

Well, it might make sense to have several ratings per item.  E.g. a rating 
of the believability of the emotion alone in response to the scenario cue, 
of the reason alone in response to the scenario cue and of the reason and 
the emotion together...  This adds to complexity of the data analysis.  I 
still think that we should use the old basic framework of emotion and 
reason for that emotion.  Later we can work on believability of less 
structured model output (e.g. were the way the emotion word figures into 
the appraisal might vary...and also using compound appraisals).

Well, I'm still very excited about how this interface has progressed.  But 
we seem to have some pretty big differences about methodology.  Lets keep
working on our differences.  I think we both really want OHL to make a 
mark.

Bill

>