Re: [Nmh-workers] 128 byte field name limit in NAMESZ breaks scan(1)

[David Levine]

Mark wrote:

> In the message dated: Wed, 22 Oct 2008 09:00:12 BST,
> The pithy ruminations from Peter Maydell on
> <Re: [Nmh-workers] 128 byte field name limit in NAMESZ breaks scan(1)> were:
> => David Levine wrote:
> => >Mark wrote:
> => >> Peter Maydell wrote:
> => >> => I think this has been reported on the bug tracking system before.
> => >> => (https://savannah.nongnu.org/bugs/?14975)
> => >>
> => >> Yes, that does sound like the issue I'm seeing. Note that the bug
> => >> was opened in 2005. I'm really not a "C" programmer, but it took me
> => >> almost as long to read the bug report as it did to increase the
> => >> value of NAMESZ. As far as I'm concerned, this is a trivial fix to a
> => >> signficant interface issue (ie., not being able to see the fields in
> => >> the scan listing).
> =>
> => Yes, but I'm not sure it's the right fix. Most of that bug is me trying
>
> Philosophically, it may not be the "right" fix. Practically, it fixes the
> problem with scan.
>
> Perhaps the "right" fix is to simply ignore lines in a mail file that preceed
> the body text but are not valid headers. As per RFC 1122:
>
>       "Be liberal in what you accept, and conservative in what you send."
>
> This seems to be the current behavior, as long as the invalid line is less
> than 128 characters.
>
> When scan encounters a "From " line (note the missing colon), scan
> is happy to accept it, unless it's over 127 characters...in which
> case the complaint is about the length, implying that there's
> nothing wrong with a "From " line in the header block.
>
> => to get information from the guy who reported it. (It didn't help that the
> => file he attached as a test case had got mangled in transit somehow so that
> => it started not "From ..." but ">From ...".)
> =>
> => >> => in a mail spool file and so get stripped out as the messages are
> => >> => split). So perhaps this bug should be fixed by making whatever
> => >> => path you're using to put messages in the folder strip out the
> => >> => envelope From as inc does.
> => >
> => >Ah, I had procmail configured to insert the 'From ' line,
> => >with its -f option.
> =>
> => Well, er, don't do that then.

I don't any more :-]

> => The mh-mail(5) manpage is pretty clear about what the format of files
> => in an nmh mail directory ought to be. These 'From ' lines simply aren't
>
> Actually, the man page states:
>
>       It should be noted that although neither Bell nor Berkeley mailers
>       produce message files in the format that  nmh  prefers,  nmh  can
>       read message files in that antiquated format.
>
> That "antiquated format", as far as I can tell, is the mbox
> specification, where messages begin with a "From " line. Thus, my
> reading of the mh-mail manpage is that nmh doesn't necessarily use
> the "From " line, doesn't write a "From " line, but that it should
> accept a "From " line. In fact, it _does_ accept a "From " line
> (thus, there's no need to try to track down every program in the
> universe that creates or doesn't remove an existing "From " line and
> provide instructions for how to make them compatible with nmh). The
> problem seems to be that nmh just doesn't handle "From " lines (and,
> I suspect, other headers) longer than 128 characters.
>
> => valid within it.
> =>
> => If the code writing 'From ' lines is part of nmh we should fix it;
> => if it's third-party then we should probably identify and document
> => how to configure that code to write valid nmh files.

How about adding this to docs/MAIL.FILTERING after the current
paragraphs that discuss the "From " line:

  Alternatively, you might be able to suppress generation of the "From "
  line.  If your procmail invocation includes the -f or -r option, try
  removing it.  Those options add "From " lines to the beginning of
  incoming messages that do not have them.

> => >> Perhaps, as the bug report from 2005 stated, that would break mail
> => >> filtering that actually looks at the "From " line. Why strip it
> => >> out...it's data about the mail delivery process, and may have some
> => >> value to some other process (spam filtering? mail statistics? random
> => >> number generation?).
> =>
> => If you want the information you should write it in as an X-Envelope-From:
> => header or similar.
>
> Maybe. Some programs (anything that deals with the mbox format, for example)
> expect "From ", not "X-Envelope-From:".
>
> =>
> => >> Personally, I think that the philosophical argument about whether
> => >> tools strip out the "From " line or not is really separate from the
> => >> fact that the the scan listing breaks because of an arbitrarily
> => >> small value in NAMESZ. The first issue is difficult to resolve (as
> => >> evidenced by the 3 year-old bug), while the second issue is trivial
> => >> to fix (though maybe not as "pure" as some would want)....but what
> => >> do I know? :)
> => >
> => >I agree.  Any objection to raising NAMESZ to 512?
> =>
> => It's still an arbitrary limit. I'd be more interested in raising
> => it if there were actually problems because of messages with header
> => components with names longer than 128. (The 'From ' lines don't
> => count because they're not valid syntax.)
>
> Again, the mh-mail manpage states that nmh will read messages that contain
> those lines, even if they aren't used by nmh, so they are not defacto invalid
> syntax to nmh.
>
> =>
> => (If you do raise it then my reading of RFC2822 says 998 would
> => be more defensible than 512.)
>
> Perhaps the mh-mail man page needs to be updated. My mh-mail
> manpage, from the 1.3 source, cites a limit of 63 characters, as per
> RFC822.

That's different, that's for header names.  That should say 126 now
(excluding the :, which is included in the 127 bytes in the buffer).

The 998 is the limit on the length, excluding the trailing CR/LF, of any
line.

> In any case, raising the limit does seem to be the answer...

Agreed.  I don't think that dynamic buffer allocation is worth
the trouble here.  We're just trying to avoid a harmless warning.

David


> Mark "feeling pedantic this morning" Bergman
>
> =>
> => -- PMM
> =>

______________________________________________________________________
This email has been scanned by the MessageLabs Email Security System.
For more information please visit http://www.messagelabs.com/email 
______________________________________________________________________