bug#13328: 24.2; Rmail does not properly decode MIME messages containing "From " lines or save such attachments correctly

[Mark Lillibridge]

[|'s added to quote From lines]

    Run Rmail (emacs -q) on the following mbox file (remove |'s at line
start first):

|From mdl  Wed Oct 26 10:18:58 2011
|X-RMAIL-ATTRIBUTES: --EF----
|X-RMAIL-KEYWORDS: *
|X-Spam-Status: No, score=-4.7 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_MED,
|       RP_MATCHES_RCVD autolearn=unavailable version=3.3.1
|From: Mark Lillibridge <mark.lillibridge@hp.com>
|To: <bug-gnu-emacs@gnu.org>
|Subject: 23.3; rmail-edit-current-message adds extra '>'s to Froms...
|Date: Sat, 22 Oct 2011 12:05:57 -0700
|Reply-To: <mark.lillibridge@hp.com>
|
|
|Start with a Rmail message (this one will do) not containing MIME that
|contains lines like the following:
|
|>From level 1
|>>From level 2
|>>>From level 3
|
|From mdl  Tue Jan  1 12:27:17 2013
|Delivered-To: lillibridge@gmail.com
|Received: from gmail-pop.l.google.com [74.125.25.108]
|       by foil.strangled.net with POP3 (fetchmail-6.3.18)
|       for <mdl@localhost> (single-drop); Tue, 01 Jan 2013 12:27:17 -0800 (PST)
|Received: by 10.49.49.9 with SMTP id q9csp680843qen;
|        Tue, 1 Jan 2013 12:27:08 -0800 (PST)
|X-Received: by 10.52.69.201 with SMTP id g9mr59443696vdu.98.1357072028236;
|        Tue, 01 Jan 2013 12:27:08 -0800 (PST)
|Return-Path: <mark.lillibridge@hp.com>
|Received: from alum-mailsec-relay-12.mit.edu (ALUM-MAILSEC-RELAY-12.MIT.EDU. 
[18.7.68.32])
|        by mx.google.com with ESMTPS id o8si46143160vdv.77.2013.01.01.12.27.08
|        (version=TLSv1/SSLv3 cipher=OTHER);
|        Tue, 01 Jan 2013 12:27:08 -0800 (PST)
|Received: from alum-mailsec-scanner-7.mit.edu (ALUM-MAILSEC-SCANNER-7.MIT.EDU 
[18.7.68.19])
|       by alum-mailsec-relay-12.mit.edu (8.13.8/8.12.8) with ESMTP id 
r01KR7kv027035
|       for <mdl@alum.mit.edu>; Tue, 1 Jan 2013 15:27:07 -0500
|Received: from madara.hpl.hp.com (madara.hpl.hp.com [192.6.19.124])
|       by alum-mailsec-scanner-7.mit.edu (Symantec Messaging Gateway) with 
SMTP id A5.53.02235.B9643E05; Tue,  1 Jan 2013 15:27:07 -0500 (EST)
|Received: from postal.hpl.hp.com (postal.hpl.hp.com [15.25.115.25])
|       by madara.hpl.hp.com (8.14.4/8.14.4/HPL-PA Relay) with ESMTP id 
r01KR5Ae021394
|       (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT)
|       for <mdl@alum.mit.edu>; Tue, 1 Jan 2013 12:27:06 -0800
|Received: from build-debian-1.u.hpl.hp.com (build-debian-1.u.hpl.hp.com 
[15.8.240.3])
|       by postal.hpl.hp.com (8.14.3/8.14.3/HPL-PA Hub) with ESMTP id 
r01KR4Df022085;
|       Tue, 1 Jan 2013 12:27:04 -0800
|From: Mark Lillibridge <mark.lillibridge@hp.com>
|To: mdl@alum.mit.edu
|Subject: test message containing From's in an attachment
|Reply-to: mark.lillibridge@hp.com
|Date: Tue, 01 Jan 2013 12:27:04 -0800
|Message-ID: <qmhip7gmyxz.fsf@build-debian-1.u.hpl.hp.com>
|MIME-Version: 1.0
|Content-Type: text/plain
|Content-Disposition: attachment; filename=froms.txt
|X-RMAIL-ATTRIBUTES: ------U-
|
|>From level 1
|>>From level 2
|>>>From level 3
|
|From mdl  Tue Jan  1 13:04:30 2013
|Delivered-To: lillibridge@gmail.com
|Received: from gmail-pop.l.google.com [74.125.25.108]
|       by foil.strangled.net with POP3 (fetchmail-6.3.18)
|       for <mdl@localhost> (single-drop); Tue, 01 Jan 2013 13:04:30 -0800 (PST)
|Received: by 10.49.49.9 with SMTP id q9csp681694qen;
|        Tue, 1 Jan 2013 13:04:13 -0800 (PST)
|X-Received: by 10.52.75.100 with SMTP id b4mr59369212vdw.52.1357074253041;
|        Tue, 01 Jan 2013 13:04:13 -0800 (PST)
|Return-Path: <mark.lillibridge@hp.com>
|Received: from alum-mailsec-relay-12.mit.edu (ALUM-MAILSEC-RELAY-12.MIT.EDU. 
[18.7.68.32])
|        by mx.google.com with ESMTPS id u1si24918746vdh.137.2013.01.01.13.04.12
|        (version=TLSv1/SSLv3 cipher=OTHER);
|        Tue, 01 Jan 2013 13:04:13 -0800 (PST)
|Received: from alum-mailsec-scanner-3.mit.edu (ALUM-MAILSEC-SCANNER-3.MIT.EDU 
[18.7.68.14])
|       by alum-mailsec-relay-12.mit.edu (8.13.8/8.12.8) with ESMTP id 
r01L47Pu030367
|       for <mdl@alum.mit.edu>; Tue, 1 Jan 2013 16:04:12 -0500
|Authentication-Results: symauth.service.identifier
|Received: from madara.hpl.hp.com (madara.hpl.hp.com [192.6.19.124])
|       by alum-mailsec-scanner-3.mit.edu (Symantec Messaging Gateway) with 
SMTP id E2.ED.02222.B4F43E05; Tue,  1 Jan 2013 16:04:12 -0500 (EST)
|Received: from masterns.hpl.hp.com (masterns.hpl.hp.com [15.0.48.4])
|       by madara.hpl.hp.com (8.14.4/8.14.4/HPL-PA Relay) with ESMTP id 
r01L4Auq022776
|       (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT)
|       for <mdl@alum.mit.edu>; Tue, 1 Jan 2013 13:04:10 -0800
|Received: from build-debian-1.u.hpl.hp.com (build-debian-1.u.hpl.hp.com 
[15.8.240.3])
|       by masterns.hpl.hp.com (8.14.3/8.14.3/HPL-PA Hub) with ESMTP id 
r01L47j3018514;
|       Tue, 1 Jan 2013 13:04:08 -0800
|From: Mark Lillibridge <mark.lillibridge@hp.com>
|To: mdl@alum.mit.edu
|Subject: test message containing From's in an attachment
|Reply-to: mark.lillibridge@hp.com
|Date: Tue, 01 Jan 2013 13:04:07 -0800
|Message-ID: <qmhhan0mx88.fsf@build-debian-1.u.hpl.hp.com>
|MIME-Version: 1.0
|Content-Type: multipart/mixed; boundary="=-=-="
|X-RMAIL-ATTRIBUTES: --------
|
|--=-=-=
|Content-Type: text/plain
|
|
|body part
|
|
|--=-=-=
|Content-Type: text/plain
|Content-Disposition: attachment; filename=froms.txt
|
|>From level 1
|>>From level 2
|>>>From level 3
|
|--=-=-=--
|

    The result should be 3 messages, the first message (non-MIME) of
which looks like:

|X-Spam-Status: No, score=-4.7 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_MED,
|       RP_MATCHES_RCVD autolearn=unavailable version=3.3.1
|From: Mark Lillibridge <mark.lillibridge@hp.com>
|To: <bug-gnu-emacs@gnu.org>
|Subject: 23.3; rmail-edit-current-message adds extra '>'s to Froms...
|Date: Sat, 22 Oct 2011 12:05:57 -0700
|Reply-To: <mark.lillibridge@hp.com>
|
|
|Start with a Rmail message (this one will do) not containing MIME that
|contains lines like the following:
|
|From level 1
|>From level 2
|>>From level 3
|

This is properly decoded assuming mboxrd except for the extra blank line
at the end (bug #10080).  The second message, however, is incorrectly
decoded (toggle the show to see the froms.txt contents):

|From: Mark Lillibridge <mark.lillibridge@hp.com>
|To: mdl@alum.mit.edu
|Subject: test message containing From's in an attachment
|Reply-to: mark.lillibridge@hp.com
|Date: Tue, 01 Jan 2013 12:27:04 -0800
|Content-Type: text/plain
|Content-Disposition: attachment; filename=froms.txt
|
|
|[text/plain Hide Save:froms.txt (46B)]
|
|>From level 1
|>>From level 2
|>>>From level 3
|

The From lines should look identical to the first message.  The third
message demonstrates this problem with a non-in-line attachment:

|Authentication-Results: symauth.service.identifier
|From: Mark Lillibridge <mark.lillibridge@hp.com>
|To: mdl@alum.mit.edu
|Subject: test message containing From's in an attachment
|Reply-to: mark.lillibridge@hp.com
|Date: Tue, 01 Jan 2013 13:04:07 -0800
|Content-Type: multipart/mixed; boundary="=-=-="
|
|
|[1:text/plain Hide]
|
|
|body part
|
|
|[2:text/plain Hide Save:froms.txt (45B)]
|
|>From level 1
|>>From level 2
|>>>From level 3


    If I understand the resolution of bug 6574 correctly, it added a
variable for switching between mboxrd and mboxo (a broken but common
variant).  For reference, with mboxo all three sets of From lines should
read:

|From level 1
|>>From level 2
|>>>From level 3


    Saving one of the attachments gives incorrect results as well so
that functionality doesn't work correctly either:

froms.txt as saved:
|>From level 1
|>>From level 2
|>>>From level 3

- Mark