Re: [libmicrohttpd] shutdown of listen socket does not work on solaris 10

[Christian Grothoff]

I see.  But given that on Linux it works without a pipe and that a pipe 
also "consumes" two extra FDs, I think the answer should be slightly 
different: revert 14584 *conditionally* for non-Linux, non-FreeRTOS 
OSes.  So we'd have pipes on Solaris/*BSD/OS X, and no-pipes on 
Linux/FreeRTOS.  That should make everyone happy (at the expense of a 
few additional #ifdef's in the code, a price we can probably pay, right?).

Now, I won't have time to do this today, but I do take patches along 
these lines ;-).

Happy hacking,

Christian

On 09/18/2011 01:13 AM, Will Bryant wrote:
> Coincidentally I discovered the same problem on Mac OS X last night, while 
> trying to debug issue #1760.  There we had noticed that the shutdown call was 
> failing and I had also noticed that my app does not shut down ever since I 
> upgraded past 0.9.7, and the library's tests were not running properly.
>
> It turns out that libmicrohttpd has been failing tests (well, they hang rather than 
> failing) on OS X ever since 0.9.8.  I did a bisect to find the offending commit and found 
> it was r14584, "also eliminating use of pipe, thereby addressing #1662".
>
> So basically because the original requestor on FreeRTOS couldn't use pipe(), a 
> solution has been adopted that I believe doesn't work on many unixes other than 
> Linux - it's definitely broken on OS X and Solaris, and my understanding from 
> #1674 is that it was also broken on Cygwin - though the #1674 workaround which 
> has been committed presumably works on cygwin but doesn't seem to work on OS X 
> when I try it.
>
> Can anyone confirm if shutdown still works on the BSDs?  I would have guessed 
> not since OS X tends to be closer to the BSD behavior.
>
> I agree with Christian that using close() on the socket is not an acceptable 
> solution due to the race condition.
>
> I personally think that the best thing to do would be to revert the r14584 
> commit to start with, because the pipe solution was a good one for most 
> supported platforms, and if the FreeRTOS support is still required, let the 
> requestor work on another solution for platforms missing most of the posix spec 
> (it could use a localhost socket pair, for example) or patch the library to use 
> the shutdown technique on that platform only.
>
> Adding complicated workarounds for platforms that don't have many basic OS 
> features seems like an unfair burden on libmicrohttpd.
>
>
> On 16/09/2011, at 11:53 PM, Frank Meier wrote:
>
> > hi
> >
> > I tried to use libmicrohttpd on solaris 10, and my simple programm only 
> > starting and stopping the daemon (see below) did hang in.
> > after some investigation (see debug code) I found, that the shutdown call in daemon.c 
> > fails, since the daemon socket is not a connected fullduplex but a listening socket. this 
> > means the daemon thread keeps hanging in select() and main hangs in pthread_join() (see 
> > pstack output). problem is since the socket doesn't get closed, select does not 
> > interrupt. After adding CLOSE (fd) after the point of SHUTDOWN () the programm did work. 
> > It looks like solaris and linux behave differently when shutdown is called on a listening 
> > socket. But on the other hand I'm not sure if it is correct to call shutdown in the first 
> > place, since the man page states "The  shutdown() call causes all or part of a 
> > full-duplex connection on the socket associated with sockfd to be shut down" and the 
> > listening socket does not have a connection.
> >
> > I think the socket should be closed with CLOSE before calling pthread_join() 
> > and use SHUTDOWN only on connection sockets.
> >
> > regards, Frank
> >
> >
> > /*test programm*/
> > struct MHD_Daemon* mhdDaemon = 0;
> > mhdDaemon = MHD_start_daemon (MHD_USE_THREAD_PER_CONNECTION, mhdPort, NULL, 
> > NULL,&answer_to_connection, NULL, MHD_OPTION_END);
> > if (mhdDaemon)
> > {
> >     MHD_stop_daemon (mhdDaemon);  //<-here it hangs
> >     mhdDaemon = 0;
> > }
> >
> > /*debug code in daemon.c*/
> > --- daemon/daemon.c     (revision 16873)
> > +++ daemon/daemon.c     (working copy)
> > @@ -2477,7 +2477,11 @@
> >        daemon->worker_pool[i].shutdown = MHD_YES;
> >        daemon->worker_pool[i].socket_fd = -1;
> >      }
> > -  SHUTDOWN (fd, SHUT_RDWR);
> > +       int res = SHUTDOWN (fd, SHUT_RDWR);
> > +       int tmpErrno = errno;
> > +       printf("shutdown res:%d  %s\n", res, strerror(tmpErrno));
> > +
> > +       CLOSE (fd);
> >
> > ->output with debug code:
> > shutdown res:-1  Transport endpoint is not connected
> >
> > /*pstack output ->  damon thread is in select, main hangs in pthread join*/
> > $ pstack 16802
> > 16802:  ./test.bin
> > -----------------  lwp# 1 / thread# 1  --------------------
> > feee9365 lwp_wait (2, 80478d8)
> > feee2c1f _thrp_join (2, 0, 8047918, 1) + 5a
> > feee2d9e pthread_join (2, 8047918, feeec7e0, fef9665d) + 2b
> > fef96777 MHD_stop_daemon (8060d40, 0, 0, fee63301) + 12b
> > 0805080a main     (1, 8047988, 8047990) + 6f
> > 080506ac _start   (1, 8047a88, 0, 8047a93, 8047abe, 8047b78) + 60
> > -----------------  lwp# 2 / thread# 2  --------------------
> > feee8da5 pollsys  (fed5ed50, 1, 0, 0)
> > fee93afa pselect  (4, fed5ef30, fed5eeb0, fed5ee30, 0, 0) + 18e
> > fee93df0 select   (4, fed5ef30, fed5eeb0, fed5ee30, 0, 0) + 82
> > fef95ba4 MHD_select (fec50200, fec50200, fed5efec, feee59b9, 8060d40, 0) + 130
> > fef96248 MHD_select_thread (8060d40) + 18
> > feee59b9 _thr_setup (fec50200) + 4e
> > feee5ca0 _lwp_start (fec50200, 0, 0, fed5eff8, feee5ca0, fec50200)