Re: [Pan-users] pan 0.139 crash with large groups

[Duncan]

Bob posted on Wed, 05 Sep 2012 17:57:10 +0000 as excerpted:

> I am seeing the same thing with 0.139 on Mint 13 (mate), but the lockup
> seems to be heat related.  Binary group has 108,000+ entries, upon click
> to enter the group processor core temp rises 16 to 18 degrees F.  If
> downloads are in progress, tasks are stopped and the screen info goes
> away. Closing the application (pan), waiting a few seconds for the temp
> to drop and restarting pan continues the downloads, as if nothing
> happened.
> 
> This is happening with an AMD Phenom II x 4 965 @3.4 Ghz with 8 gig of
> memory.

Ouch!

For large binaries in particular, it's worth noting that there's a couple 
ways of splitting them; binaries over ~ 100 MB are often split both 
ways.  There's the traditional news client built-in attachment splitting, 
done during the post itself, and "pre-splitting" the file into parts 
before posting, for reassembly after download (tho some of the automated 
posters can handle this automatically as well).

The first kind, attachment splitting, divides single files as posted into 
a series of individual posts (as appearing on the server) of 
approximately the same size.  The reason this is significant is that PAN 
NORMALLY RECOMBINES THESE TRANSPARENTLY, AND SHOWS ONLY ONE ARTICLE THAT 
MAY IN FACT BE 100+ individual parts!

So your 108k of entries could easily be 10 million+ individual parts (as 
appearing on the server)!  If indeed that's the case, no WONDER pan heats 
up the CPU and uses a LOT of memory, potentially hitting the 32-bit 4-gig-
per-app barrier!

Presumably with 8 gigs RAM you're running 64-bit (altho you could do it 
with the 64-gig mem option on 32-bit, but that adds another level of 
indirection so is less efficient), so it shouldn't be 32-bit issues.  Tho 
of course you could be running a 64-bit kernel with 32-bit userland 
(either the traditional way or the new x32), and there pan could still 
run into the 32-bit 4-gig-per-app barrier even if the machine has 8 gig 
of RAM.  But with full 64-bit, the 4-gig per-app barrier disappears 
(unless of course you have a per-app limit set locally, via ulimit or 
/etc/security/limits.* or the like), then it's whatever you set).  
However, pan *IS* known for pressing the limits on 10-million-plus post 
groups, even with 8-gig-plus RAM.  There has long been talk of converting 
pan to use a post database so it doesn't need to keep all that info in 
actual memory, but it hasn't happened yet.


That's the memory side.  CPU-temp side, if your system's going unstable 
when it hits 100% CPU for too long, it's time to examine your cooling 
solution, and/or get a CPU upgrade and/or turn down your overclocking 
(note that if the system was overclocked previously, it may have gotten 
to the point where it can't even maintain rated clocking with any 
stability, and you may have to underclock).

FWIW, I run gentoo here, so /regularly/ run flat-out while building 
updates like the kde 4.9.1 that came out yesterday, or the job-uncapped 
kernel builds that fill my 16 gigs RAM and run up 500+ one-minute load 
averages. =:^)  (AMD "bulldozer" fx6100 6-core rated 3.3 GHz but 
marginally overclocked to 3.6 GHz w/ stock cooling.  At full load, stays 
right at the rated 95W rating.)

If you're having overheat/stability problems due to CPU usage, like I 
said, you better investigate!

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman