Re: SMPT no longer works in Starttls upon upgrading from emacs-22.3-bin-i386 to emacs-23.1-bin-i386 on Windows XP Professional, Service Pack 3.

[Benjamin L. Russell]

DekuDekuplex@Yahoo.com (Benjamin L. Russell) writes:

> X-Draft-From: ("gnu.emacs.gnus" 38318)
> References: <80y6owpd14.fsf@Yahoo.com> <804orkb9jc.fsf@Yahoo.com>
>       <80fxb489tw.fsf@Yahoo.com> <87ws4cuzdb.fsf@rapitore.luna>
>       <80fxay2bn3.fsf@yahoo.com> <871vmhemls.fsf@rapitore.luna>
> Gcc: nnml:archive
> Newsgroups: gnu.emacs.gnus
> Subject: Re: SMPT no longer works in Starttls upon upgrading from 
> emacs-22.3-bin-i386 to emacs-23.1-bin-i386 on Windows XP Professional, 
> Service Pack 3.
> Bcc: 'Benjamin L. Russell' <DekuDekuplex@Yahoo.com>, 'Benjamin Leon Russell 
> (Gmail)' <DekuDekuplex@Gmail.com>, 'Benjamin Leon Russell (Yahoo.co.jp)' 
> <Dekuplex@Yahoo.co.jp>
> From: DekuDekuplex@Yahoo.com (Benjamin L. Russell)
> Date: Wed, 09 Sep 2009 15:13:55 +0900
> --text follows this line--
> Marco Maggi <mrc.mgg@gmail.com> writes:
>
>> "Benjamin L. Russell" wrote:
>>> Apparently,  it   is  possible  to   send  e-mail  through
>>> gnutils-cli  successfully  using  an  immediate  encrypted
>>> bridge
>>> [...]
>>> However, when I try to  send an EOF to the process through
>>> the  Emacs  eshell function  "eshell-send-eof-to-process",
>>> for some reason, the TLS handshake fails; [...]
>>
>> Here is  a copy and paste  of an eshell session  on my Linux
>> system (I am doing this  from Italy, whatever this means for
>> IP addresses):
>>
>> | $ gnutls-cli --crlf --starttls --port 587 plus.smtp.mail.yahoo.com
>> | Resolving 'plus.smtp.mail.yahoo.com'...
>> | Connecting to '76.13.13.48:587'...
>> | 
>> | - Simple Client Mode:
>> | 
>> | 220 smtp110.prem.mail.ac4.yahoo.com ESMTP
>> | ehlo localhost
>> | 250-smtp110.prem.mail.ac4.yahoo.com
>> | 250-AUTH LOGIN PLAIN XYMCOOKIE
>> | 250-PIPELINING
>> | 250 8BITMIME
>>
>> among the 250 lines there is no STARTTLS capability, so:
>>
>> | starttls
>> | 502 Command not implemented
>>
>> which means  that the  server does not  support establishing
>> the TLS  layer with STARTTLS  through the 587  interface; so
>> the only option is to  use port 465 and the immediate bridge
>> (SMTP-over-SSL mode).
>>
>>   If  this is  the  case, it  is  not a  problem with  Gnus,
>> Message,  or Emacs  in itself.   Rather it  is a  problem of
>> "smtpmail.el" because, AFAICT,  it only supports delayed TLS
>> layer creation (STARTTLS mode), as I already reported to the
>> Emacs mailing list[1].
>>
>>   After a  quick inspection of  "smtpmail.el.gz" distributed
>> with Emacs 23,  I say that the feature  request has not been
>> satisfied; at least, around line 678, I do not see code that
>> can  be  used to  build  the  TLS  bridge (send  SIGALRM  or
>> whatever) before sending the EHLO greetings.
>>
>>   So  either you must  use an  external Mail  Delivery Agent
>> (which, by the way, is what  I do with the MBFL script).  Is
>> my understanding  that you want to use  MSMTP[2] correct?  I
>> do not use it, but it seems it can act as MDA for you.
>>
>>   Again,   first  you   should  make   sure  that   you  can
>> successfully send email using a raw command line with MSMTP.
>
> Done.  As you mentioned, it works with port 465, but not with port 587,
> as follows:
>
>> $ ./msmtp.exe --host=plus.smtp.mail.yahoo.com 
>> --domain=plus.smtp.mail.yahoo.com --port=587 --auth=on --user=dekudekuplex 
>> --tls=on --tls-starttls=off --tls-certcheck=off 
>> --from=dekudekuplex@yahoo.com dekudekuplex@yahoo.com
>> msmtp: TLS handshake failed: error:140770FC:SSL 
>> routines:SSL23_GET_SERVER_HELLO:
>> unknown protocol
>> msmtp: could not send mail
>> 
>> Benjamin@200706-05 /usr/sbin
>> $ ./msmtp.exe --host=plus.smtp.mail.yahoo.com 
>> --domain=plus.smtp.mail.yahoo.com --port=465 --auth=on --user=dekudekuplex 
>> --tls=on --tls-starttls=off --tls-certcheck=off 
>> --from=dekudekuplex@yahoo.com dekudekuplex@yahoo.com
>> password for dekudekuplex at plus.smtp.mail.yahoo.com:
>> This is a test of msmtp.exe.
>> .
>> 
>
> Receipt of the sent e-mail message was verified at DekuDekuplex@Yahoo.com.
>
>> If that works, IMHO:
>>
>> (setq send-mail-function 'message-send-mail-with-sendmail
>>       message-send-mail-function 'message-send-mail-with-sendmail)
>>
>> should work,  with the right configuration,  because you can
>> select all the relevant options with the MSMTP configuration
>> file.
>>
>>> Additionally,  here  are  the  relevant portions  from  my
>>> msmptrc.txt file
>>>
>>> defaults
>>> tls on
>>> tls_certcheck off
>>> tls_starttls on
>>
>> IMHO, reading [4], it should be "tls_starttls off", meaning:
>> use TLS with SMTP-over-SSL mode not with STARTTLS mode.
>
> Unfortunately, it still results in the following error message:
>
>> Sending via mail...
>> message-send-mail-with-sendmail: Sending...failed with exit value 78
>
> I copied my msmtp configuration file from ~/msmtprc.txt to ~/.msmtprc,
> since according to the output of "/usr/sbin/msmtp.exe --version", the
> Cygwin version, as opposed to the native Windows version, uses the
> latter file as a user configuration file; _viz._:
>
>> $ ./msmtp.exe --version
>> msmtp version 1.4.13
>> TLS/SSL library: OpenSSL
>> Authentication library: GNU SASL
>> Supported authentication methods:
>> plain cram-md5 digest-md5 external login
>> IDN support: enabled
>> NLS: disabled
>> System configuration file name: /etc/msmtprc
>> User configuration file name: /cygdrive/c/home/.msmtprc
>
>> Copyright (C) 2007 Martin Lambers and others.
>> This is free software.  You may redistribute copies of it under the terms of
>> the GNU General Public License <http://www.gnu.org/licenses/gpl.html>.
>> There is NO WARRANTY, to the extent permitted by law.
>
> Here are the revised contents of my ~/.msmtprc file (note the changes
> from "tls_starttls on" to "tls_starttls off" in both the "defaults" and
> "account dekudekuplex" sections, and the change from port 587 to port 465):
>
>> ########################
>> defaults
>> tls on
>> tls_certcheck off
>> tls_starttls off
>
>> account dekudekuplex
>> host plus.smtp.mail.yahoo.com
>> domain plus.smtp.mail.yahoo.com
>> tls on
>> tls_certcheck off
>> tls_starttls off
>> auth on
>> user dekudekuplex@yahoo.com
>> from dekudekuplex@yahoo.com
>> password 
>> port 465
>
>> logfile c:\home\.msmtplog.txt
>> ########################
>
> Additionally, here are the revised relevant portions of my ~/.gnus.el
> file:
>
>> (setq send-mail-function 'message-send-mail-with-sendmail)
>> 
>> (setq message-send-mail-function 'message-send-mail-with-sendmail)
>> 
>> ; The following settings were adapted from the article "Re: Debugging msmtp 
>> in gnus," by Sivaram Neelakantan <nsivaram.net@gmail.com>, dated "Wed, 02 
>> Sep 2009 22:36:12 +0530," on gnu.emacs.gnus
>> (setq sendmail-program "c:/cygwin/usr/sbin/msmtp.exe")
>> ; (setq sendmail-program "c:/bin/msmtprc/1.4.17/msmtp.exe")
>> (setq message-sendmail-extra-arguments '("-a" "dekudekuplex"))
>> (setq mail-host-address "yahoo.com")
>> (setq smtpmail-debug-info t) ; only to debug problems set to t if needed
>> (setq smtpmail-debug-verb t)
>> 
>> (setq starttls-use-gnutls t)
>> 
>> (setq starttls-gnutls-program "c:/usr/bin/gnutls-cli.exe")
>> 
>> ;; Configure outbound mail (SMTP)
>> (setq smtpmail-starttls-credentials '(("plus.smtp.mail.yahoo.com" 465 nil 
>> nil))
>>       smtpmail-smtp-server "plus.smtp.mail.yahoo.com"
>>       smtpmail-default-smtp-server "plus.smtp.mail.yahoo.com"
>>       send-mail-function 'message-send-mail-with-sendmail
>>       message-send-mail-function 'message-send-mail-with-sendmail      
>>       smtpmail-smtp-service 465
>>       smtpmail-auth-credentials '(("plus.smtp.mail.yahoo.com"
>>                                 465
>>                                 "dekudekuplex@yahoo.com"
>>                                 nil)))
>
> For some reason, "exit value 78" results in Gnus even if msmtp.exe
> succeeds in sending an e-mail message at the command line with identical
> settings.
>
>>   As a  last resort, if you  know how to use  MSMTP from the
>> command line  to send email using  port 465, you  may try to
>> adapt  my  library[3]  for   sending  email  with  a  custom
>> "message-send-mail-function" function.

Incidentally, when using msmtp.exe at the command line, for some reason,
I was unable to get msmtp.exe to work correctly when specifying a user
configuration file; msmtp.exe kept complaining about the content of the
file.

For example, at first, I specified the following contents for
~/.msmtprc, as stated before:

--8<---------------cut here---------------start------------->8---
########################
defaults
tls on
tls_certcheck off
tls_starttls off

account dekudekuplex
host plus.smtp.mail.yahoo.com
domain plus.smtp.mail.yahoo.com
tls on
tls_certcheck off
tls_starttls off
auth on
user dekudekuplex@yahoo.com
from dekudekuplex@yahoo.com
password 
port 465

logfile c:\home\.msmtplog.txt
########################
--8<---------------cut here---------------end--------------->8---

This resulted in the following error message upon invocation:

--8<---------------cut here---------------start------------->8---
Benjamin@200706-05 /usr/sbin
$ ./msmtp.exe --file="/cygdrive/c/home/.msmtprc" --account=dekudekuplex 
--domain=plus.smtp.mail.yahoo.com --port=465 --auth=on --user=dekudekuplex 
--tls=on --tls-starttls=off --tls-certcheck=off --from=dekudekuplex@yahoo.com 
dekudekuplex@yahoo.com
msmtp: /cygdrive/c/home/.msmtprc: line 1: unknown command ・ｿ###################
#####

--8<---------------cut here---------------end--------------->8---

Removing the comments resulted in the following interaction:

--8<---------------cut here---------------start------------->8---
Benjamin@200706-05 /usr/sbin
$ ./msmtp.exe --file="/cygdrive/c/home/.msmtprc" --account=dekudekuplex --domai
n=plus.smtp.mail.yahoo.com --port=465 --auth=on --user=dekudekuplex --tls=on --
tls-starttls=off --tls-certcheck=off --from=dekudekuplex@yahoo.com dekudekuplex
@yahoo.com
msmtp: /cygdrive/c/home/.msmtprc: line 1: unknown command ・ｿdefaults

--8<---------------cut here---------------end--------------->8---

Then I trimmed the file down to a bare-bones version, leaving only the
information for the account "dekudekuplex", as follows:

--8<---------------cut here---------------start------------->8---
host plus.smtp.mail.yahoo.com
domain plus.smtp.mail.yahoo.com
tls on
tls_certcheck off
tls_starttls off
auth on
user dekudekuplex@yahoo.com
from dekudekuplex@yahoo.com
password 
port 465

logfile c:\home\.msmtplog.txt
--8<---------------cut here---------------end--------------->8---

However, this still resulted in the following error message upon
invocation:

--8<---------------cut here---------------start------------->8---
Benjamin@200706-05 /usr/sbin
$ ./msmtp.exe --file="/cygdrive/c/home/.msmtprc" --account=dekudekuplex --domai
n=plus.smtp.mail.yahoo.com --port=465 --auth=on --user=dekudekuplex --tls=on --
tls-starttls=off --tls-certcheck=off --from=dekudekuplex@yahoo.com dekudekuplex
@yahoo.com
msmtp: /cygdrive/c/home/.msmtprc: line 1: unknown command ・ｿhost

--8<---------------cut here---------------end--------------->8---

Finally, I recast all the information into a sequence of
command-line-style flags, with one flag per line, _viz._:

--8<---------------cut here---------------start------------->8---
--host=plus.smtp.mail.yahoo.com
--domain=plus.smtp.mail.yahoo.com
--tls=on
--tls_certcheck=off
--tls_starttls=off
--auth=on
--user=dekudekuplex@yahoo.com
--from=dekudekuplex@yahoo.com

--port=465

--logfile="c:\home\.msmtplog.txt"

--8<---------------cut here---------------end--------------->8---

Unfortunately, this still resulted in the following error message upon
invocation:

--8<---------------cut here---------------start------------->8---
Benjamin@200706-05 /usr/sbin
$ ./msmtp.exe --file="/cygdrive/c/home/.msmtprc" --account=dekudekuplex --domai
n=plus.smtp.mail.yahoo.com --port=465 --auth=on --user=dekudekuplex --tls=on --
tls-starttls=off --tls-certcheck=off --from=dekudekuplex@yahoo.com dekudekuplex
@yahoo.com
msmtp: /cygdrive/c/home/.msmtprc: line 1: unknown command ・ｿ--host=plus.smtp.ma
il.yahoo.com

--8<---------------cut here---------------end--------------->8---

This is probably an irrelevant issue, but msmtp.exe should be able to
accept configuration options with specification of a user configuration
file using the "--file=" option.

Would you happen to know how to get this option to work?

-- Benjamin L. Russell
-- 
Benjamin L. Russell  /   DekuDekuplex at Yahoo dot com
http://dekudekuplex.wordpress.com/
Translator/Interpreter / Mobile:  +011 81 80-3603-6725
"Furuike ya, kawazu tobikomu mizu no oto." -- Matsuo Basho^