Re: [Pan-users] Pan/0.139 - threading issue

[Duncan]

Dave posted on Tue, 20 Aug 2013 00:21:15 +0100 as excerpted:

> Is anyone seeing newsgroup theading issue with Pan 0.139?
> It's natively compiled on FreeBSD 9.1

I use pan built from live-git, here (as can be seen from my headers, since
I'm posting with pan via the gmane list2news service), so I'm a but past
0.139 release, but I don't believe there have been any threading changes
in quite some time... several versions, tho not being a C++ coder (tho I
can do sysadmin's level coding, that is, the occasional patch) I go more
by the git commit comments than the code.

I suppose it's possible that it's a library issue, however.  What version
of gmime?  (FWIW, 2.6.16, here on gentoo/~amd64.)

> On the group in question:   
> newsgroup:  free.virginmedia.test
> subject:   Testing custom User Agent header
> date:  19th Aug, 11pm local (GMT/UTC+1)
> 
> There are replies I've made to myself but they don't thread properly,
> showing up as replies to the original post rather than the one it's
> actually in reply  to.  Others, using other newsreaders including OE
> and Agent have commented on the threading problems.
> 
> I've tested replies with full automatic message quoting, highlighting
> text then reply to only quote specific test, switched off/on body pane
> word wrap, and wondered if the splitting of headers in replies might be
> a cause, eg References: header splits in the middle of references
> instead of at the end, ie at a comma.  Likewise,the User Agent: header
> is huge and gets split across line.

It'd have to be the references header.  None of the rest of the things you
mention should have anything at all to do with threading.  It's worth 
noting that (AFAIK) pan threads exclusively via message-id as found in 
the references header, while certain other news and/or mail clients, 
including OE at one point (altho I'm not sure it still applies, I 
switched to freedomware instead of crossing the eXPrivacy line, tho back 
in the day I ran IE/OE betas so I know more than average about historical 
versions), fall back to subject line threading as well -- they'll thread 
identical (but for Re: etc) subject lines together even in the absence of 
references headers.

The RFCs (Internet Request For Comments documents, which ultimately 
become STDs, Standards, but by that point everybody is used to referring 
to them by the RFC number, so RFCs is how they're normally referenced) 
are the definitive standard here, specifying the contents and format of 
the references header as up-thread message-IDs, as well as the proper 
method for "header folding."

> On the other hand, looking at the "Testing custom User Agent header"
> thread using knode,  everything looks just fine with all messages
> threaded and indented correctly.

... Which means the information must be there in the references header 
for knode to use.  It's apparently a bit less strict in its header 
parsing, however, or unfolding folded headers differently (maybe without 
an added space/comma at the split?), tho my memory of the internet 
messaging RFCs is fuzzy enough I couldn't tell you which would be 
"correct" without looking it up.

> If you can't access that newsgroup but have some ideas/help/advice, I
> can post headers here if required.

That would be helpful.  Probably just the references headers, perhaps 
with one header on either side just to be sure we keep the context, if it 
happens to be important.  Be sure to retain verbatim header folding, as 
that's almost certainly the issue.

(I very likely technically have access to the group as I have an 
unexpiring block account, but I don't regularly use it (so the unexpiring 
bit is good! =:^), and if the information I need is all there I tend to 
reply immediately, where as if I have to look, I'll often skip it (not 
often) or mark it unread again, to reply to later (the usual case)... 
which can be MONTHS (!!) later, at which point I may well decide it's not 
likely to be relevant any longer and ultimately skip it then.  Posting 
the headers is thus likely to be in practice the fastest way to a proper 
reply, at least from me.)

Heinrich (who would likely be creating the fix if needed) or others may 
well reply in the mean time.


Meanwhile, based on the evidence so far (your mention of "folded" 
references header, header folding being the term used in the RFCs, and 
knode apparently getting it right) I strongly suspect that it's down to 
that, tho whether it's on the posting end, folding the references header 
incorrectly according to the RFCs, or the receiving end, unfolding them 
incorrectly, I can't say.

What I CAN say, however, is that the RFCs have a very well known general 
policy of being strict in observance with what you send, but rather more 
tolerant when parsing what you receive.  So it's quite likely that the 
information is all there (as can be seen from knode getting it right), 
but either pan isn't being sufficiently RFC-strict in how it folds the 
references header when it sends the message, and knode is simply being 
tolerant in reconstruction as it should be (while other clients including 
pan aren't quite that tolerant), or pan is using an RFC allowed but 
relatively uncommon and thus not often tested mid-ID split, and due to 
the relative rarity of the case, few clients are properly coping with an 
RFC-folding-compliant references header.

I may just go lookup the RFC and see what it says as that's an 
interesting question regardless, but I'll send this off first, in case I 
don't get to that.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman