Re: NSString lowercaseString

[Sebastian Reitenbach]

On Wednesday, August 8, 2012 12:10 CEST, Richard Frith-Macdonald 
<richard@tiptree.demon.co.uk> wrote:

>
> On 3 Aug 2012, at 18:21, Sebastian Reitenbach wrote:
>
> >
> > On Thursday, August 2, 2012 11:48 CEST, Richard Frith-Macdonald 
> > <richard@tiptree.demon.co.uk> wrote:
> >
> >>
> >> On 1 Aug 2012, at 19:04, Sebastian Reitenbach wrote:
> >>
> >>> Hi,
> >>>
> >>> On Wednesday, August 1, 2012 18:00 CEST, "Sebastian Reitenbach" 
> >>> <sebastia@l00-bugdead-prods.de> wrote:
> >>>
> >>> On the Apple documentation I found -[NSString lowercaseStringWithLocale:] 
> >>> bug GNUstep doesn't seem to have that.
> >>> When it would be available, would that maybe help me?
> >>
> >> I've kept out of this because I don't have an easy answer ... I'm not even 
> >> sure what the problem is  (or more likely, problems are).
> >>
> >> However, it doesn't appear that anyone else has spotted anything simple 
> >> either, and while we don't understand the problem, it seems premature to 
> >> be looking for a new method to provide the solution.
> >>
> >> So, can I please suggest looking at this slowly and systematically.
> >>
> >> It seems possible to me that there are are a variety of areas where a 
> >> problem may be, probably four main ones:
> >>
> >> 1. what is the compiler putting into the binary?
> >> 2. how is base interpreting what it finds?
> >> 3. how is the case conversion being done?
> >> 4. is there a problem logging it?
> >>
> >> The way to find all this out is by stepping through the executable in gdb 
> >> and examining what you find there.
> >>
> >> If you debug the test program setting a break point immediately before the 
> >> literal string is used, you can step into the methods.
> >>
> >> eg. for    NSString *l = [@"TöÖst" lowercaseString];
> >> you can step into the lowercaseString method and look at the receiver and 
> >> see what class it is, and look  at the underlying character data in the 
> >> literal string and see whether the compiler has actually generated UTF-8
> >> you can then step through and see how characters are being converted to  
> >> lowercase etc.
> >> Is the lowercaseString implementation from the base NSString class the one 
> >> being used?
> >> Is the uni_tolower() function working?
> >> and so on.
> >>
> >>
> >
> > So far I only tried to debug the version on OpenBSD, because there I have 
> > the OGo problem. On the Linux box, there I also have an older version 
> > installed maybe,
> > and therefore and cannot easily upgrade, to make it better comparable. The 
> > thing I just recognized, is on Linux I get:
> > 2012-08-01 08:49:57.974 lowercase[16574] autorelease called without pool 
> > for object (0x72dce8) of class GSCInlineString in thread <NSThread: 
> > 0x6b0cc8>
> > and on OpenBSD:
> > 2012-08-01 10:38:52.851 lowercase[5483] autorelease called without pool for 
> > object (0x209c1c5c8) of class GSUnicodeInlineString in thread <NSThread: 
> > 0x20750be08>
> > the difference of the GSCInlineString vs. GSUnicodeInlineString
>
> This most probably reflects the version of gnustep you have installed:
>
> On OpenBSD you get a unicode string because base supports utf-8 in string 
> literals and has converted the utf-8 literal to a unicode string in order to 
> do the case conversion.
>
> On GNU/Linux you get a C string because you are probably using an older 
> version of base where only ascii characters were legal in string literals.
>
> So, in the second case (assuming this really is an older version of base 
> beign used) the application code is buggy because it contains an illegal 
> string literal: the documentation for OpenStep and GNUstep used to say 
> behavior was 'undefined' if you put non-ascii data in a string literal ... so 
> *any* behavior by the base library is correct, and the program needs fixing.
>
> It's the first case we need to look at... using an up to date version of 
> GNUstep where support for utf8 in string literals has been added.  I've just 
> modified svn trunk to check in more places that the content of a literal 
> really looks like valid utf8, so with luck if you update base, you might get 
> an exception raised if your toolchain has compiled non-utf8 data.  Please 
> give it a try.
>
>

Thanks Richard.

I added some NSLog before it calls lowercaseString:
2012-08-08 16:56:46.667 ogo-webui[9109] LSBaseSearch.m: _formatForStringValue: 
%%ö%% lowercase: %%ö%%, string type: GSCInlineString
2012-08-08 16:56:46.667 ogo-webui[9109] LSBaseSearch.m: _formatForStringValue: 
%%ö%% lowercase: %%ö%%, string type: GSCInlineString
2012-08-08 16:56:46.667 ogo-webui[9109] LSBaseSearch.m: _formatForStringValue: 
%%ö%% lowercase: %%ö%%, string type: GSCInlineString
2012-08-08 16:56:46.668 ogo-webui[9109] LSBaseSearch.m: _formatForStringValue: 
%%ö%% lowercase: %%ö%%, string type: GSCInlineString
so the character doesn't get stripped anymore, but it still doesn't find it in 
the database. I need to investigate more on that.

The only thing I chanced was updating gnustep-base to SVN from release 1.24.0, 
now I get a lot of warnings like this:

ERROR: Removing exception handler that is not on top of the stack. (You 
probably called return in an NS_DURING block.)
ERROR: Removing exception handler that is not on top of the stack. (You 
probably called return in an NS_DURING block.)
ERROR: Removing exception handler that is not on top of the stack. (You 
probably called return in an NS_DURING block.)

With OGo and also with SOGo. I'm not sure whether this is because of your last 
change or a different change in the meantime.
I'll investigate.



Sebastian