Re: [Openexr-devel] RE: UNICODE support in openexr file I/O

[Chris Cox]

Where are you viewing the filename?
Windows and MacOS X should know about UTF-8 filenames. (Photoshop's
EULA/Legal folder is a good source of localized filenames ;-)

Have you tried putting a BOM at the beginning of the string (just in case)?


"random junk" = mojibake

Ok, that's more appropriate when it's random Japanese characters, but I
still like the name: "changed characters" or "ghost characters"
http://en.wikipedia.org/wiki/Mojibake

Chris



On 1/26/06 9:34 AM, "Nick Porcino" <address@hidden> wrote:

> 
> Hello list, I am stumped.
> 
> I am trying to create a file named Kyoto-to in Kanji. name below is the UTF-8
> encoding of Kyoto-to. I've tried a number of experiments, but all yield a
> goobledy-gook filename. I have appended my tests below.
> 
> If someone can provide the correct answer that would be swell!
> 
> 
> 
> const char name[] =
> {0xe4, 0xba, 0xac, 0xe9, 0x83, 0xbd, 0xe5, 0xb8, 0x82, 0x23, 0};
> 
> 
> 
> 
> 
> TEST ONE - fopen
> 
> 
> fopen(name, "wb") yields a file called:
> 
> 京都市#
> 
> (random junk)
> 
> 
> 
> 
> 
> 
> TEST TWO - CreateFile (windows)
> 
>     HANDLE hFile = CreateFile(TEXT(name),    // file to open
>         GENERIC_WRITE,          // open for writing
>         FILE_SHARE_WRITE,       // share for writing
>         NULL,                  // default security
>         CREATE_ALWAYS,         // create file only
>         FILE_ATTRIBUTE_NORMAL, // normal file
>         NULL);   
> 
> äºéƒ½å¸‚#
> 
> (random junk)
> 
> 
> 
> TEST THREE - _wfopen
> 
> 
>     _wfopen((const wchar_t*)name, (const wchar_t*)"wb");
> 
> äºéƒ½å¸‚#
> 
> (random junk)
> 
> 
> 
> 
> 
> 
> TEST FOUR - _wfopen + UTF-8 -> 16 conversion
> 
> Finally:
> 
> 
> using std::wstring;
> 
> wstring 
> toWideString( const char* pStr , int len )
> {
>     // figure out how many wide characters we are going to get
>     int nChars = MultiByteToWideChar( CP_ACP , 0 , pStr , len , NULL , 0 ) ;
>     if ( len == -1 )
>         -- nChars ;
>     if ( nChars == 0 )
>         return L"" ;
> 
>     wstring buf ;
>     buf.resize( nChars ) ;
>     MultiByteToWideChar( CP_ACP , 0 , pStr , len ,
>         const_cast<wchar_t*>(buf.c_str()) , nChars ) ;
> 
>     return buf ;
> }
> 
> int _tmain(int argc, _TCHAR* argv[])
> {
>     wstring wname = toWideString(name, strlen(name));
>     FILE* x = _wfopen(wname.c_str(), (const wchar_t*)"wb");
> return 0;
> }
> 
> äºéƒ½å¸‚#
> 
> (random junk)
> 
> 
> 
> -----Original Message-----
> From: Florian Kainz [mailto:address@hidden
> Sent: Wednesday, January 25, 2006 3:17 PM
> To: Nick Porcino
> Cc: Drew Hess
> Subject: Re: [Openexr-devel] RE: UNICODE support in openexr file I/O
> 
> According to a hex dump of a UTF-8-encoded e-mail message that I sent
> to myself, the byte sequence for 京都市 is:
> 
>      const char name[] =
>          {0xe4, 0xba, 0xac, 0xe9, 0x83, 0xbd, 0xe5, 0xb8, 0x82, 0x23};
> 
> 
> Nick Porcino wrote:
>> I'm kind of pressed for time... I can try something tho'
>> 
>> Florian, could you give me something like this -
>> 
>> char name[] = { 0xa, 0x21, 0x23, ...., 0 };
>> 
>> that UTF-8 encodes one of those strings like the name of Bangkok or
>> whatever and I'll write a little test app for you
>> 
>> -----Original Message-----
>> From: Drew Hess [mailto:address@hidden
>> Sent: Wednesday, January 25, 2006 10:25 AM
>> To: Nick Porcino
>> Cc: Florian Kainz
>> Subject: Re: [Openexr-devel] RE: UNICODE support in openexr file I/O
>> 
>> 
>> "Nick Porcino" <address@hidden> writes:
>> 
>>> The code I posted earlier converts between UTF-8 and 16. So if you do
>>> the conversion of the strings 8<>16 outside of OpenEXR you are covered
>>> for OpenEXR's internal strings.
>>> 
>>> Santiago, are you saying that if you convert UTF-16 to UTF-8, then use
>>> the converted string as a filename, that the OS itself displays the
>>> filename as garbage?
>> 
> _______________________________________________
> Openexr-devel mailing list
> address@hidden
> http://lists.nongnu.org/mailman/listinfo/openexr-devel