bug#22493: 25.1.50; open-gnutls-stream doesn't respect :nowait, so the connections are synchronous

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: 22493@debbugs.gnu.org
> Date: Thu, 04 Feb 2016 13:47:05 +1100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> With async DNS, this will fail, because the process-send-string happens
> >> before the connection had completed.  (And this isn't a TLS socket.)  So
> >> I think that (like I said on the emacs-devel threads) we may have to
> >> change the :nowait stuff to allow a more fine-grained control.
> >
> > If you make process-send-string wait until DNS completes, the problem
> > will disappear.
> 
> Yes.  That was basically what the
> 
> #ifdef HAVE_GNUTLS
>   /* The TLS connection hasn't been set up yet, so we can't write
>      anything on the socket. */
>   if (!NILP (p->gnutls_boot_parameters))
>     return;
> #endif
> 
> in send_process achieves (in the TLS case).

No, I meant to actually wait: loop checking the async DNS status until
that indicates the DNS resolution is complete, or fails.

>   if (!NILP (p->gnutls_boot_parameters) || p->outfd == 0)
>     return;
> 
> then that will have the same effect on non-TLS sockets.  But as noted in
> the previous email, set-process-coding-system would also have to be
> changed to remove these outfd guards...

Every API that expects a fully-capable process object should have such
a wait for DNS added to it.  And if we make GnuTLS negotiation run in
the background, they should also wait for that to complete.  An
application that doesn't want to wait will have to "do other things"
without calling these functions, and from time to time check whether
the background processing is complete (which might require an
additional API).

> OK: eww has displayed a web page, and now wants to fetch an image over
> https.  Our goal is to have Emacs stop as little as possible while this
> is happening in the background, so that the user can scroll around or do
> whatever the user wants to do while eww is working on getting the image.
> 
> So eww issues a url-retrieve command in the background, which ends up
> calling make-network-process :nowait t.  This returns immediately, with
> no user-noticeable hangs.  getaddrinfo_a has been issued, and after a
> while a response is received and is noticed by the idle loop.
> 
> Emacs then calls connect_network_process.  This basically is a wrapper
> around the connect(2), and it's pretty async on a non-blocking socket,
> so this should not be noticeable to the user.
> 
> Everything up until now has happened identically for TLS and non-TLS
> sockets.
> 
> Now, for a TLS socket, we have to do the negotiating.  This is currently
> a blocking process, but it doesn't have to be.  The GnuTLS library in
> itself is non-blocking, but our interaction with it currently is.
> 
> So we call gnutls_boot after the connection has happened.  I originally
> did this with a sentinel on a process, but that doesn't really work,
> because the callers of make_network_process want their own sentinels on
> the process.  So I call gnutls_boot from the C layer instead of from a
> sentinel.
> 
> No matter how we call gnutls_boot, it will currently hang Emacs while
> it's transferring all those certificates back and forth.

That last sentence is exactly the point I was trying to make all
along: we have to wait for this, therefore any time savings from
running gnutls_boot in the background are minor or even non-existent.
So I question the need for complicating the heck out of the underlying
code, for no practical gain.

> > Either way, if we decide that making gnutls-boot run in the background
> > is a Good Thing, then I feel that doing so requires code restructuring
> > that must involve application-level considerations.
> 
> Actually, almost all of the major network usages in Emacs today require
> no changes.  The only one I've found is erc.  url.el, Gnus, smtpmail,
> imap.el, nntp.el all work without any changes with the current
> implementation.

Why shouldn't we assume that the problem you saw in erc is the tip of
an iceberg, and the other places are happy exceptions?  Who knows how
many other packages are out there that are like erc?

And again, I don't see the gains in doing gnutls-boot asynchronously.
DNS resolution, yes, but gnutls-boot is another matter.