Re: [Pan-users] pan 0.139 crash with large groups

[Bob]

On Thu, 06 Sep 2012 18:54:56 +0000, Bob wrote:

> On Thu, 06 Sep 2012 01:46:24 +0000, Duncan wrote:
> 
>> Bob posted on Wed, 05 Sep 2012 17:57:10 +0000 as excerpted:
>> 
>>> I am seeing the same thing with 0.139 on Mint 13 (mate), but the
>>> lockup seems to be heat related.  Binary group has 108,000+ entries,
>>> upon click to enter the group processor core temp rises 16 to 18
>>> degrees F. If downloads are in progress, tasks are stopped and the
>>> screen info goes away. Closing the application (pan), waiting a few
>>> seconds for the temp to drop and restarting pan continues the
>>> downloads, as if nothing happened.
>>> 
>>> This is happening with an AMD Phenom II x 4 965 @3.4 Ghz with 8 gig of
>>> memory.
>> 
>> Ouch!
>> 
>> For large binaries in particular, it's worth noting that there's a
>> couple ways of splitting them; binaries over ~ 100 MB are often split
>> both ways.  There's the traditional news client built-in attachment
>> splitting, done during the post itself, and "pre-splitting" the file
>> into parts before posting, for reassembly after download (tho some of
>> the automated posters can handle this automatically as well).
>> 
>> The first kind, attachment splitting, divides single files as posted
>> into a series of individual posts (as appearing on the server) of
>> approximately the same size.  The reason this is significant is that
>> PAN NORMALLY RECOMBINES THESE TRANSPARENTLY, AND SHOWS ONLY ONE ARTICLE
>> THAT MAY IN FACT BE 100+ individual parts!
>> 
>> So your 108k of entries could easily be 10 million+ individual parts
>> (as appearing on the server)!  If indeed that's the case, no WONDER pan
>> heats up the CPU and uses a LOT of memory, potentially hitting the
>> 32-bit 4-gig-
>> per-app barrier!
>> 
>> Presumably with 8 gigs RAM you're running 64-bit (altho you could do it
>> with the 64-gig mem option on 32-bit, but that adds another level of
>> indirection so is less efficient), so it shouldn't be 32-bit issues.
>> Tho of course you could be running a 64-bit kernel with 32-bit userland
>> (either the traditional way or the new x32), and there pan could still
>> run into the 32-bit 4-gig-per-app barrier even if the machine has 8 gig
>> of RAM.  But with full 64-bit, the 4-gig per-app barrier disappears
>> (unless of course you have a per-app limit set locally, via ulimit or
>> /etc/security/limits.* or the like), then it's whatever you set).
>> However, pan *IS* known for pressing the limits on 10-million-plus post
>> groups, even with 8-gig-plus RAM.  There has long been talk of
>> converting pan to use a post database so it doesn't need to keep all
>> that info in actual memory, but it hasn't happened yet.
>> 
>> 
>> That's the memory side.  CPU-temp side, if your system's going unstable
>> when it hits 100% CPU for too long, it's time to examine your cooling
>> solution, and/or get a CPU upgrade and/or turn down your overclocking
>> (note that if the system was overclocked previously, it may have gotten
>> to the point where it can't even maintain rated clocking with any
>> stability, and you may have to underclock).
>> 

> I think I will pull the processor/fan/heatsink and check it out.  I just
> may have put too much thermal paste on the interface between the
> processor and heatsink.
> If nothing else, I may re-install the phenom 2 x 4 925 which I
> originally used to build the system. It is rated at 2.8Ghz but will oc
> to 3ghz with stock cooling.
> 
> I do hope that the database project comes about, it would probably be a
> real help.
> 
> Bob

The system is running a 64bit version of Mint Maya with Mate, instead of
Gnome 3, Cinnamon or pUnity.  Desktop looks just like the old Gnome 2
desktop with top and bottom panels and my dark colors.

Cold front hit last night, and my AC is working better than it has since last 
spring.  Outside temp is 79 deg and got down to 66 last night.

Pulled the processor/fan/heatsink yesterday and cleaned off the processor
heatsink interface.  Probably did have too much paste on the sink.  Noticed
that the fan/heatsink was a coolermaster set.  Very similar to the old Volcano
heatsink fan I used to have on my old 32bit Athlon 3000+ back in the Windows
3.1, W-98, W2K, ME and early XP-pro days.

The fan runs normally about 5500 rpm, at least when I am looking in pc 
health in bios.  Still have some heat buildup when running pan, but nothing 
like I did.  Changed the settings on the groups to only keep the last 6 months, 
and that in itself dropped the group numbers to around 60,000.  

With the front moving in, and better cooling (AC and heatsink/fan) system
temp dropped to around 90F from 101F to 105F.  Checked the stats on the
965 and found that 165F is when the temp warnings come into effect. 

So, all in all, I feel that the problem has somewhat been alleviated.

Bob