Re: [PATCH] lib/isatty.c - Remove IsConsoleHandle(h)

[Michael Goffioul]

On Mon, Jun 9, 2014 at 11:03 AM, Eli Zaretskii <address@hidden> wrote:

> Date: Mon, 09 Jun 2014 09:33:18 +0100
> From: Pádraig Brady <address@hidden>
> Cc: address@hidden

>
> It's an optimized test rather than an optimization for isatty() itself.
> Without this call the isatty() replacement is moot because the
> reason it exists on mingw is to double check the handle as there
> true is returned for the NUL device.

Indeed, that's the main reason why just isatty is not enough.

> > More importantly, it breaks on Windows 8, where all handles are multiples of 4.  The result is a false negative, and an unclean output from a freshly compiled glib.
>
> Ugh fair enough. So it seems these lower 2 bits are still significant,
> just not used for tagging consoles any more?
> So is isatty(nul_handle) still returning true there?
> If not, then we could use a direct test of isatty(nul_handle)
> to enable the replacement.
> Note also the replacement is useful on "MSVC 9" to avoid an
> exception for isatty(invalid_handle), which we'd have to consider.

You can call one of the console functions to test if a handle is
connected to a console.  E.g., GetConsoleMode for the input handle and
GetConsoleScreenBufferInfo for output.  These functions fail when the
handle is not a console handle.

See the following threads, with problem reporting and solution testing:

http://lists.gnu.org/archive/html/bug-gnulib/2012-12/msg00092.html

http://lists.gnu.org/archive/html/bug-gnulib/2013-01/msg00007.html

Michael.