Re: [Monotone-devel] mtn 0.43dev: "failed to convert string from UTF-8 to ANSI_X3.4-1968"

[Zack Weinberg]

2009/3/14 Derek Scherger <address@hidden>:
> 2009/3/7 Jack Lloyd <address@hidden>
>> mtn: fatal: error: failed to convert string from UTF-8 to ANSI_X3.4-1968:
>> '-----------------------------------------------------------------
>> mtn: error: Revision: 424e1cf5155ae4473250978ae6b7e44e12775741
...
>> mtn: error: Added to <E2><80><98>contrib/<E2><80><99> an useful script by
>> Richard Levitte.
...
>
> I've tracked this problem down to the following change
>
> Revision: 9c0bb9b509e63897e6a52c907988cef2d4a8fe0d
> Ancestor: 1c069d8736d169b25eaa747d9a5ceabcb59e79b6
> Author: address@hidden
> Date: 2009-01-23T22:43:07
> Branch: net.venge.monotone.stripped
> ChangeLog:
>
> Configuration cleanup part 2.

Thanks for finding the problem change.  I've pushed a fix in rev
63361f1c30b0d8e84bbe32b758414e36fdb786e6.

What's going on here is, revision 424e1cf... has curly quote marks in
its change log (U+2018, U+2019) which are not representable in the "C"
locale.  We used to have special hacks to our local copy of libidn
that would tell iconv() to substitute fallback characters (plain old
ASCII U+0027 APOSTROPHE, in this case) if possible, and fall back to
question marks otherwise.  Obviously we can't do that anymore.  The
.stripped branch substituted a somewhat flimsier hack in our code
(charset_convert()) that was dependent on an autoconf test.  I thought
that autoconf test was unused, so I took it out in "configuration
cleanup part 2".  Boom.

What I've done now is revise the flimsy hack in charset_convert() so
it doesn't need an autoconf test.  However, with iconv()
implementations that do not support the //TRANSLIT modifier on the
destination charset (this is all of them except glibc & gnu iconv, I
believe) "mtn log" will blindly dump out the unconverted string if it
contains unrepresentable characters.  I don't have any good ideas for
how to avoid that, right now.

zw