Re: portable openpty, forkpty

[Simon Josefsson]

Bruno Haible <address@hidden> writes:

>> This is from a package created with './gnulib-tool --dir m
>> --create-testdir --with-tests forkpty'
>
> The only version of MacOS X I have access to is 10.5, and on this

Mine is a powerpc system, maybe it could explain the difference?

> system it works fine, both with the system-provided 'openpty'
>    ./configure CPPFLAGS=-Wall
> as well as with the gnulib replacements
>    ac_cv_func_openpty=no \
>    ac_cv_func_forkpty=no \
>    ac_cv_func_login_tty=no \
>    ac_cv_have_decl_openpty=no \
>    ac_cv_have_decl_forkpty=no \
>    ac_cv_have_decl_login_tty=no \
>    ./configure CPPFLAGS=-Wall

I tried again now with latest gnulib, and both these fails in the same
way here:

forkpty returned -1
FAIL: test-forkpty
openpty returned -1
FAIL: test-login_tty
openpty returned -1
FAIL: test-openpty

Gnulib decides to replace openpty by default, so the tests are probably
identical.  The reason openpty gets replaced appears to be:

checking whether openpty is declared without a macro... yes
checking whether openpty is declared... (cached) yes
checking for const-safe openpty signature... no

HAVE_OPENPTY is not set in config.h though -- isn't that a bug?  The
system has openpty.

I get a warning during compile though:

test-forkpty.c:24: warning: initialization from incompatible pointer type

Which is:

SIGNATURE_CHECK (forkpty, int, (int *, char *, struct termios *,
                                struct winsize *));

According to the Mac OS X manpage the signature is:

     pid_t
     forkpty(int *amaster, char *name, struct termios *termp,
         struct winsize *winp);

Libc has this signature:

extern int forkpty (int *__amaster, char *__name,
                    const struct termios *__termp,
                    const struct winsize *__winp) __THROW;

Is pid_t and int guaranteed to be the same size?

> Can you check why openpty and login_tty fail? The tests are written
> in a way that can easily be single-stepped. (To build for debugging and
> single-stepping on MacOS X, use CFLAGS=-ggdb, then you can debug with
> gdb.)
>
> The implementation of openpty on a particular platform is quite easy
> to understand, once you stepped through it. It's only the #ifs which
> are complicated.

Single-stepping the test case with system function this happens:

  {
    int master;
    int slave;

    /* Open a pseudo-terminal, as a master-slave pair.  */
    {
      int res = openpty (&master, &slave, NULL, NULL, NULL);
      if (res != 0)
        {
          fprintf (stderr, "openpty returned %d\n", res);
=>        return 1;

The 'res' variable is -1 at this point.

Single stepping rpl_openpty:

  master = open ("/dev/ptmx", O_RDWR | O_NOCTTY);
  if (master < 0)
=>  return -1;

And indeed my Mac OS X system doesn't have /dev/ptmx:

espresso:~ jas$ ls -la /dev/ptmx
ls: /dev/ptmx: No such file or directory
espresso:~ jas$ 

Defining HAVE_OPENPTY to 1 in config.h and re-compiling and running the
self-check works better -- but test-openpty hangs on exit.  Ctrl-c in
gdb doesn't resume control to gdb, so I'm not sure how to debug that.

/Simon