Re: [Gzz] ``async_storm--benja``: Supporting aynchronicity in Storm

[Tuomas Lukka]

> >
> >"expect it to be available": too strong. 
> >We don't really want to guarantee this: e.g. 
> >     get
> >     ...
> >     finished...
> >     ...
> >     (long time passes, system is busy, disk almost full,
> >      block removed in other thread to make room in cache)
> >     ...
> >     try to get block..
> >
> >We shouldn't mark all those blocks nonremovable.
> >
> 
> Hmm. Ok. First I was a bit worried about cases where we want blocking 
> access (e.g. during startup, when we're waiting for the first pointer to 
> be resolved and really cannot do anything else)-- but in these cases, we 
> can write:
> 
> listener = BlockingJobListener()
> while(true) {
> block = pool.get(id, listener);
> if(block != null) break;
> block.wait();
> }

Ouch... that's not good. getSYNC()?

> >>Issues
> >>======
> >>
> >>- Should we keep variations of the Storm methods that do not
> >>take a ``JobListener``? If so, what should these do
> >>if data is not available locally: Implement blocking IO,
> >>spawn a lookup job that does not inform any listener
> >>when it finishes, or simply return ``null``
> >>and do nothing?
> >>
> >> RESOLVED: I'm leaning towards not providing these methods
> >> for now, since it's not clear what they'd do.)
> >>   
> >>
> >
> >I'd say they should do exactly what get() should do if called
> >with a null Listener, and that behaviour *needs* to be defined.
> >
> 
> NullPointerException? ;-)

No. Too much burden to require it always.

> >I'd give the system some leeway here: 
> >     1) if locally stored, return block
> >     2) if not locally stored, return null, and three options:
> >             - do nothing
> >             - start looking for it actively
> >             - mark it as "get this later if happen by" - somewhere
> >               between the two above options.
> >
> 
> Ok, sounds good.

Ok

> >>Changes
> >>=======
> >>
> >>I propose a similar interface for Storm, ``JobListener``::
> >>
> >>  /** Called when new data has arrived. */
> >>  void newDataAvailable();
> >>
> >>  /** Called when the lookup finishes.
> >>   *  @param timeout Whether the lookup was finished
> >>   *                 because of a time-out. If this
> >>   *                 is false, the lookup was finished
> >>   *                 because the pool implementation
> >>   *                 believes that all relevant data
> >>   *                 has been retrieved now.
> >>   */
> >>  void finished(boolean timeout);
> >>
> >>The two separate callbacks make it easy to use the same
> >>interface for lookups like for xu links, where we update
> >>the view each time new information arrives, and
> >>lookups like for pointer blocks, where we can only
> >>continue when all relevant data is known.
> >>   
> >>
> >
> >Do we *want* to do that? Why should the one interface be used for both?
> >
> >I'd prefer
> >
> >     abstract class BlockListener {
> >             public void finished(BlockId id) {
> >             }
> >             public void timeout(BlockId id) {
> >             }
> >     }
> >
> >     interface DataListener {
> >             void newDataAvailable();
> >     }
> >
> 
> a) finished() is not specific to blocks: For most lookups, we want to be 
> able to know when they're finished (e.g. pool.getIds()).

Shouldn't pool.getIds() block?

> b) Listeners should not be abstract classes, since some people like to 
> implement them as mix-ins.

Java has local classes so this is not necessarily a good thing.
Mix-ins are usually used to get some *code*, which is impossible in Java.

> c) I think timeout() and finished() will usually be implemented the same 
> way, so why have two different methods?

Really? For timeout() I'd usually do nothing, for finished, a redraw.

> We could have a JobListener and a JobCompletionListener, with the former 
> inheriting the latter, if you prefer (because for blocks, 
> newDataAvailable() won't be called).

That might be.

        Tuomas