Re: [Gzz] ``async_storm--benja``: Supporting aynchronicity in Storm

[Tuomas Lukka]

On Sun, Nov 24, 2002 at 03:29:26PM +0100, Benja Fallenstein wrote:
> =========================================================
> ``async_storm--benja``: Supporting aynchronicity in Storm
> =========================================================

> This PEG defines a pattern to support asynchronicity.
> A ``JobListener`` is introduced that can be used
> to inform clients when new data arrives. All Storm lookups
> take an optional ``JobListener`` parameter.
> If the requested data is available locally, they
> simply return it. Otherwise, they return ``null`` and
> start a new thread to retrieve the data. Once that job is done,
> the ``JobListener`` is informed; it can at that time
> re-request the data and expect it to be available.

"expect it to be available": too strong. 
We don't really want to guarantee this: e.g. 
        get
        ...
        finished...
        ...
        (long time passes, system is busy, disk almost full,
         block removed in other thread to make room in cache)
        ...
        try to get block..

We shouldn't mark all those blocks nonremovable.



> 
> Issues
> ======
> 
> - Should we keep variations of the Storm methods that do not
>  take a ``JobListener``? If so, what should these do
>  if data is not available locally: Implement blocking IO,
>  spawn a lookup job that does not inform any listener
>  when it finishes, or simply return ``null``
>  and do nothing?
> 
>   RESOLVED: I'm leaning towards not providing these methods
>   for now, since it's not clear what they'd do.)

I'd say they should do exactly what get() should do if called
with a null Listener, and that behaviour *needs* to be defined.

I'd give the system some leeway here: 
        1) if locally stored, return block
        2) if not locally stored, return null, and three options:
                - do nothing
                - start looking for it actively
                - mark it as "get this later if happen by" - somewhere
                  between the two above options.


> Changes
> =======
> 
> I propose a similar interface for Storm, ``JobListener``::
> 
>    /** Called when new data has arrived. */
>    void newDataAvailable();
> 
>    /** Called when the lookup finishes.
>     *  @param timeout Whether the lookup was finished
>     *                 because of a time-out. If this
>     *                 is false, the lookup was finished
>     *                 because the pool implementation
>     *                 believes that all relevant data
>     *                 has been retrieved now.
>     */
>    void finished(boolean timeout);
> 
> The two separate callbacks make it easy to use the same
> interface for lookups like for xu links, where we update
> the view each time new information arrives, and
> lookups like for pointer blocks, where we can only
> continue when all relevant data is known.

Do we *want* to do that? Why should the one interface be used for both?

I'd prefer

        abstract class BlockListener {
                public void finished(BlockId id) {
                }
                public void timeout(BlockId id) {
                }
        }

        interface DataListener {
                void newDataAvailable();
        }


> The lookup methods in ``StormPool`` would simply get an additional
> ``JobListener`` parameter::
> 
>    Block get(BlockId id, JobListener listener);

Should spec more exactly: if returns null, listener will be called later.


        Tuomas