[Bug-gnulib] Re: strtok_r

[Simon Josefsson]

Bruno Haible <address@hidden> writes:

> About the .h file:
>
>> +#if defined HAVE_DECL_STRTOK_R && !HAVE_DECL_STRTOK_R
>
> What does the first half of this test mean? HAVE_DECL_STRTOK_R is meant
> to be defined to either 0 or 1 in config.h, so what's the purpose of the
> test?

I use it mostly as an idiom, for the reasons discussed in the autoconf
manual.  A run of

grep 'DECL' *.h|grep defined|grep \! 

indicate more uses of this in gnulib, although it seems I might be
responsible for a few of them.

> Furthermore, it's lacking good documentation. Since I'm too lazy to write
> one, maybe you can add the reference to the POSIX description at
>   http://www.opengroup.org/onlinepubs/009695399/functions/strtok.html
>
> Also, it would be good to add the caveats, partially found in the Linux
> man page:
>
> /* This is a variant of strtok() that is multithread-safe.
>
>    Caveat: It modifies the original string.
>    Caveat: It doesn't work with multibyte strings unless all of the delimiter
>            characters are ASCII characters < 0x30.

Huh? 0x30?  Shouldn't that be 0x80?  My man page also mention:

              These functions cannot be used on constant strings.

              The identity of the delimiting character is lost.

>
>    See also strsep().  */
>
> I'm adding a "See also" in the opposite direction as well.

So how about the comment below?

It seems strtok_r in glibc CVS now should be usable, except for the
doc fix below.

I'll install this in gnulib now.

/* Parse S into tokens separated by characters in DELIM.
   If S is NULL, the saved pointer in SAVE_PTR is used as
   the next starting point.  For example:
        char s[] = "-abc-=-def";
        char *sp;
        x = strtok_r(s, "-", &sp);      // x = "abc", sp = "=-def"
        x = strtok_r(NULL, "-=", &sp);  // x = "def", sp = NULL
        x = strtok_r(NULL, "=", &sp);   // x = NULL
                // s = "abc\0-def\0"

   For the POSIX documentation for this function, see:
   http://www.opengroup.org/onlinepubs/009695399/functions/strtok.html

   Caveat: It modifies the original string.
   Caveat: These functions cannot be used on constant strings.
   Caveat: The identity of the delimiting character is lost.
   Caveat: It doesn't work with multibyte strings unless all of the delimiter
           characters are ASCII characters < 0x80.

   See also strsep().
*/

Thanks.