RE: [Heartlogic-dev] new idea (fwd)

[Josh White]

> > I don't think you'll get much participation unless you tell 
> the user the
> > computer's best guess AFTER they click the answer.   The 
> point is the user
> > can see how their answer improved the computer.
> 
> this feedback there is a slight risk that a person would 
> start to think like Cyc rather than a naive and natural 
> human

True...  But I think that's the price we'll have to pay for getting free
participants.  It would be cleaner to pay participants a token amount of
money to participate - I think that will greatly improve the quality because
people feel like they're working, not like they're entertaining themselves.

www.hotornot.com has the same problem. Try it out with a friend standing
over your shoulder - you'll see what I mean.  You can feel yourself start to
sway your own ratings, or even lie outright.  I've seen three different
people using it, and they all did not tell the comlete truth.   Still,
overall, the scores are pretty accurate.  You do find extremely ugly people
getting a higher score than you'd expect - that's about the only bias that
shows up, that I've noticed.

> distracting...The real problem will be keeping people's 
> motivation and excitment up.)..

Well said.  For that reason, I suggest: 

 - add a personality to the AI - name, sex, etc.  I think HAL is perfect,
but whatever. 
  
 -  keep the answer and the next question on the same page, same as hotornot

 - put the scientific explanations one link deep, from that page

Example page:
---------------------------
You said HAL was wrong to say "Vienna is wet."

HAL 1000 thinks Vienna is wet because
 (1) "Rivers are a kind of water."
 (2) "If water touches x then x is wet."
 (3) "The Danube is a river."
 (4) "The Danube runs through Vienna."
 (5) "If a river runs through a region it touches that region."
                        (Click [here] for the science behind this project.)

5 other users agreed with you, and1 other user did not.  So far, you humans
are convincing HAL he is wrong.

Hal also believes Paris is in France. Is that right?

[true]
[false]

----------------------------

-Josh

> But OTHER items will be intentially reversed.  We would 
> predict that humans would rate these as less believable than 
> unreversed items.
> 
> This is what I did in my dissertation.  In study 2 half of 
> the items were unreversed and the other half were reversed.  
> In study 3, a third of the items were unreversed, another 
> third were "slight" reversed, and another third were 
> "strongly" reversed.
> 
> There are two reasons we want to do this reversal stuff.  One reason 
> is to catch liars or vandals.
> 
> The OTHER reason is to allow us to compare the mean 
> believability of different groups of items.  E.g....
> 
> unversed items vs reversed items 
> human generated items vs machine generated items
> deductions vs ground facts
> 
> ...this is all part of the computational ablation paradigm 
> and it figured big time in my dissertation.  It is an example 
> of what I mean by good and rigorous methodology.
> 
> [Now, it occurs to me that there is a THIRD more minor, 
> user-interfacey sorta reason to do this...That is that we want quick
> *coarse* judgements about whethher a commonsense assertion is 
> a good one or not.  One way to obtain such coarseness is to 
> throw in a fair number of ridiculous assertions.]
> 
> Well, I should enlist some other AI gurus opinions on this 
> before I spout off too loudly about good and rigorous 
> methodology.  Speaking of AI gurus, Peter, are you on this list yet?
> 
> So Josh and Joshua does this make sense conceptually, designwise?
> 
> Joshua, can you implement this.  Note: Just getting the 
> believability ratings up there is step one.  Implementning 
> the Feedback to User is step two.
> 
> Bill
> 
> >
> > -Josh
> >
>