Re: [Pan-devel] Ya know Pan2 is really single threaded, right?

[Charles Kerr]

Calin A. Culianu wrote:

> Why does pan2 ever create a threadpool?  It's actually single-threaded!

Hi Calin,

I've talked about your nice patch in bugzilla, so here I'll just answer
in general about the rationale behind Pan's use of threads.

At one point Pan was fully threaded.  This made things more responsive
than 0.14.x would've been otherwise, but at the cost of some stability
and a _lot_ of code complexity.

To reduce that complexity in the rewrite, Pan's eliminated threads
wherever possible and used them only to prevent freezing during
self-contained work (socket creation/connection).  The only other
candidate for threads I can think of is UUDecode.

> I discovered this because UUDecoding attachements led to pan2 stalling 
> completely (the UI was frozen too -- the whole app was stuck inside 
> UUDecodeFile()).  This is because pan2 does all its task processing from 
> the main thread's idle loop!

This used to be bugzilla ticket #353317, slated for Pan 1.1, but
since your patch is coming along so well we should use it sooner. =)

> The other 4 threads that are created are only briefly used to create a 
> socket connection to the nntp server, then the actual handshaking and 
> logging-in is handled from the main thread's idle loop!

Yes. Everything touching NNTP's state machine is in the main thread.

> Furthermore, any and all nntp stuff happens in the main thread.  Sure, 
> there are 4 task contexts that keep getting shuffled around, but 
> basically only 1 task at a time ever runs, and it runs in the main thread.
>
> Meanwhile, you have this perfectly lovely thread pool sitting there 
> twiddling its thumbs.
>
> My gripe with this for the most part is that UUDecode() of large 
> binaries (sometimes a single attachment can be 50mb) stalls the whole 
> app and leads to poor network utilization overall.

I strongly agree that UUDecode belongs in a worker thread, but
wouldn't go beyond that -- the next bottleneck is networking,
and threads don't help there.  The single-threaded version of
Pan downloads just as fast as the multithreaded one, and is
significantly smaller.

Even our threadpool poster child, UUDecode, has problems:
UUDecode is single-threaded, so the best we can do is a worker
thread and a FIFO queue.  Anything else will devolve into that
and is overkill.

cheers,
Charles