Re: lynx-dev stopping when viewing a site

[Leonid Pauzner]

18-Aug-99 07:09 Klaus Weide wrote:

> It seems I found the relevant difference that was introduced by your
> 1999-03-04 reorganization.  The value uck returned by UCTransUniChar() is
> -3.  That means UCTransUniCharStr() is not attempted, since that happens only
> if -4 is returned (a normal "this character is not mapped" indication).  The
> -3 is some other error condition.  It must come from this code in a call to
> conv_uni_to_pc(code, 0):

Thanks, that describe a lot.
I had a feeling that my kludge of NULL
(to save a space from variable area) instead of real table
may have some problems - here they are.


>         if (!hashtable_contents_valid)
>             return -3;

> The variable hashtable_contents_valid gets normally set to 1 (in
> con_clear_unimap) when the first non-default output character set is
> selected with UC_con_set_unimap(), and after that it never gets reset.
> Don't ask me what *exactly* it's good for... but is is basically some
> internal consistency check, to prevent looking up something in a table
> before it has been initialized.

> In our case, con_clear_unimap(0) has never been called.  The reason is
> that your UC_CHARSET_SETUP_foo macros initialize UCInfo[..].unitable
> to NULL.  UC_current_unitable is also initially NULL, so that comparisons
> like the following (this one is from UCTransUniChar)

>         ut = UCInfo[UChndl_out].unitable;
>         if (ut != UC_current_unitable) {

More UCTransUniChar* code cited below.
If we change (!isdefault) with (!isdefault && UCInfo[...].unitable != NULL)
we will probably bypass all calls to dedicated charset's tables
`conv_uni_to_pc(unicode, 0)' so initialization may not be necessary
since we only use default table - calls to `conv_uni_to_pc(unicode, 1)'

Will look more closely tomorrow, should went out just now.

    if (!isdefault) {
        ut = UCInfo[UChndl_out].unitable;
        if (ut != UC_current_unitable) {
            rc = UC_con_set_unimap(UChndl_out, 1);
            if (rc < 0) {
                return rc;
            }
        }
        rc = conv_uni_to_pc(unicode, 0);
        if (rc >= 0)
            return rc;
    }
    if (isdefault || trydefault) {
        rc = conv_uni_to_pc(unicode, 1);
        if (rc >= 0)
            return rc;
    }
    if (!isdefault && (rc == -4)) {
        rc = conv_uni_to_pc(0xfffd, 0);
    }
    if ((isdefault || trydefault) && (rc == -4)) {
        rc = conv_uni_to_pc(0xfffd, 1);
    }
    return rc;


> yield a false result - NULL is compared to NULL.  So con_clear_unimap(0)
> never gets called, and hashtable_contents_valid never gets initialized.

> But this (and the hang that indirectly resulted from it) only happens
> if no other non-default, "normal" display character set has been used
> (for actual lookup of a character) since the beginning of the session.
> By first viewing some text (with non-ASCII characters) in some other
> display character set (non-ASCII, non-CJK, non-transparent), and *then*
> changing to a CJK d.c.s. and loading the problematic text, the hanging
> may not happen!  (Somebody should confirm this.)  Not sure whether the
> translation will be correct for all characters in that case.

> With Henry's change this will not happen, since his 'unitable' pointers
> are not NULL.  (OTOH with his change there are multiple character sets
> that claim to be the default (with 'isdefault'); that doesn't look right.)
> Before your changes around 1999-03-04, CJK character sets were not
> present in UCInfo[] at all, and LYCharSet_UC[i].UChndl always had -1
> for them, so that UCTransUniChar() would use the def7 default directly.

>> So the problem assumed somethere else. Klaus was reporting a fix for
>> LYUCFullyTranslate_1() - seems this is a source of the real problem in 
>> subject.

> That was the other half of the problem.  With that change alone, lynx
> should not hang any more, but the remaining problem is also "real".
> Some Latin 1 characters will appear untranslatable (and some fallback
> will be used), but it depends on previous history.

> PS. Please snip your quotes, don't append quoted patches all the time!

>    Klaus