Re: thank you for libidn 1.34 however ...

[Dennis Clarke]

On 31/03/18 03:22 PM, Tim Rühsen wrote:
> Hi Dennis,
>
>
> thanks for your detailed report, really appreciated !


are you sure ?   ;-)

n0$ dbx ./fuzz/libidn_stringprep_fuzzer 
./time_1522524034-pid_10364-uid_16411-gid_20002-fid_libidn_stringpre.core
Reading libidn_stringprep_fuzzer
dbx: warning: core object name "libidn_stringpr" matches
object name "libidn_stringprep_fuzzer" within the limit of 14. assuming 
they match
core file header read successfully
Reading ld.so.1
Reading libidn.so.11.6.16
Reading libintl.so.8.1.5
Reading libc.so.1
Reading libiconv.so.2.6.0
Reading libc_psr.so.1
program terminated by signal SEGV (no mapping at the fault address)
Current function is _g_utf8_normalize_wc
  798     while ((max_len < 0 || p < str + max_len) && *p)
(dbx) where 

=>[1] _g_utf8_normalize_wc(str = (nil), max_len = -1, mode = 
G_NORMALIZE_ALL_COMPOSE), line 798 in "nfkc.c"
  [2] stringprep_ucs4_nfkc_normalize(str = 0x100127b90, len = 5), line 
1120 in "nfkc.c"
  [3] stringprep_4i(ucs4 = 0x100127b90, len = 0xffffffff7fffec60, 
maxucs4len = 17U, flags = <unknown enum member 0>, profile = 
0xffffffff7ef3d950), line 170 in "stringprep.c"
  [4] stringprep_4zi_1(ucs4 = 0x100127b90, ucs4len = 5U, maxucs4len = 
17U, flags = <unknown enum member 0>, profile = 0xffffffff7ef3d950), 
line 290 in "stringprep.c"
  [5] stringprep_4zi(ucs4 = 0x100127b90, maxucs4len = 17U, flags = 
<unknown enum member 0>, profile = 0xffffffff7ef3d950), line 336 in 
"stringprep.c"
  [6] LLVMFuzzerTestOneInput(data = 0x100127a70 "\n", size = 68U), line 
94 in "libidn_stringprep_fuzzer.c"
  [7] test_all_from(dirname = 0xffffffff7ffff170 
"/usr/local/build/libidn-1.34_SunOS5.10_sparcv9.001/fuzz/libidn_stringprep_fuzzer.in"), 
line 71 in "main.c"
  [8] main(argc = 1, argv = 0xffffffff7ffff328), line 105 in "main.c"
(dbx)

yikes ...


> Regarding the alloca(): Solaris surely has one (maybe a define), you
> have to #include <alloca.h> in fuzz/main.c.

Not likely.  It isn't anywhere in the SUSv3 or POSIX.1-2001 world. Not
that I can see. Easy enough to work around I think by just not using it
and sticking with calloc ( personally I am a fan of calloc over malloc )
like this :

int main(int argc, char **argv){
    int ret_val = 0;
    /* if VALGRIND testing is enabled, we have to call ourselves
       with valgrind checking */
    if (argc == 1) {
        const char *valgrind = getenv("TESTS_VALGRIND");
        if (valgrind && *valgrind) {
            size_t cmdsize = strlen(valgrind) + strlen(argv[0]) + 32;
            char *cmd = malloc(cmdsize);
            snprintf(cmd, cmdsize,
                       "TESTS_VALGRIND="" %s %s", valgrind, argv[0]);
            ret_val = ( system(cmd) != 0 );
            free(cmd);
            return ret_val;
        }
    }


Mind if I tighten that up and lose tabs over spaces and such ?

> Will fix this for the next release - likely you can remove the whole block:
>
> if (valgrind && *valgrind) {
>
> ...
>
> }

oh .. well yeah .. that too.

> > However my next issue was a sigsegv in ./fuzz/libidn_stringprep_fuzzer
> thus...
>
> Was this also on Solaris ? I am not aware of any system dependent code
> where the crash happens...
> will test this next week on Solaris.

yep .. on Solaris is where I see the sigsegv and thankfully I have the
 coreadm in place to do a full memory dump for me.

> > On yet another system I get some messy CFLAGS that create other problems
>
> Looks like a hard-coded compiler flag... such is always bad (has to be
> removed).
> In this case it works with gcc and clang, the two compilers we build with.

Right.  I have gcc 7.2.x pretty much everywhere however the really
strictly compliant compiler is Oracle Studio 12.6 as well as an older
rev Oracle Studio 12.4 which doesn't do strange things like function
call substitutions and it works ... really really well.  Certainly it
is the C99 acid tester compiler becasue if it compile with Oracle Studio
c99 with strict cflags then it works everywhere. Pretty much a promise.

> > ...I may need to step up to the plate here
> > and provide some online datacenter access for folks in open source
> > projects to do across multiple platforms and multiple os testing ...
>
> Always a good idea ! But there are some points to think of
> 1. libidn has been obsoleted by libidn2 though there is no replacement
> for stringprep yet.

yep ... I have both.

> 2. because of 1. we don't much work into libidn any more, so the goal is
> use libidn2 everywhere instead of libidn

yep .. makes total sense.  However may as well ensure libidn works too.

> 3. we have very limited time/capacity (any help appreciated, up to new
> maintainers :-)). that is the main reason
> we didn't test on all kinds of different systems (though we have access
> to Solaris and the gcc build platform).

I will do what I can here.

> 4. the libidn code was available for testing since >1.5 years for
> everybody. If there is no feedback for a while I

well damn ... smack me .. where was it ?  A github repo or something
 like that?  damn.  Sorry.

> expect everybody tested on their beloved platform. Of course I know not
> many people do so, but hey, this is
> not only free open source, it is also open development for everyone.

yep .. I just happen to have a slew of systems across all manner of 
weird and wonderful architectures and that includes a very very old 1998 
pentium system which runs Debian buster/sid wonderfully well.

> So if you are willing to put some work into libidn, think about libidn2
> first - the API is pretty much compatible.
> And of course: let me know if I can help you to step in.

I will stay in touch .. I really want to step along through that sigsegv 
issue and see what went wrong .. whereever it did.

Dennis