Re: [bug-gnu-libiconv] iconv(3), E2BIG, and outbuf, outbytesleft

[Jeffrey Walton]

On Fri, Sep 23, 2011 at 1:36 PM, Bruno Haible <address@hidden> wrote:
> Hi,
>
> Jeffrey Walton wrote:
>> I'm working in C++, and I would like to convert string -> wstring.
>
> On Unix platforms, it makes no sense to use 'wstring'.
>  - The wchar_t[] representation is platform dependent AND locale dependent.
>  - The functions for wide character input (fgetwc() etc.) in ISO C cannot
>    accommodate input that is not in the expected encoding, and offer no way
>    to handle such error in a reliable way.
>
> And on native Win32 and AIX platforms, where wchar_t[] is in UTF-16 encoding,
> <wctype.h> is unusable.
>
> What's left is the use of wchar_t[] for interfacing with Win32 APIs; that's
> probably the only reasonable use of wchar_t[] and wstring.
Yep, the library is basically duplicating Java and Win32 code.

>> Is there a way to determine the required output buffer size in advance?
>
> No. You can repeatedly convert into a temporary output buffer of limited
> size, and keep track how many bytes were generated this way.
That's too bad.

>> If no, I believe I can loop on inbyteleft. But I can't tell if outbuf
>> and outbytesleft are updated on when { -1, E2BIG } is returned. Is
>> outbuf and outbytesleft updated when the input exceeds the output?
>
> They may be updated, yes, together with inbuf and inbytesleft. When you
> encounter E2BIG the usual way to proceed is to find room, that is,
> adjust outbuf and increase outbytesleft in such a way that the conversion
> is likely to continue successfully.
I was trying to work from a single page on the stack, and feed the
converted text [on the stack] into the output string one page at a
time. The output string is reserved but not sized.

For what its worth, little strings are easy; but I did not want to do
something like 'size *= 2;' with a large string.

> If you are writing GPL'ed code and are willing to use gnulib, you can
> use the mem_cd_iconv() function from the 'striconv' module of gnulib.
> It handles all the complications: you pass it the input string in memory,
> and get the output string in memory.
Open source, but closer to BSD than GPL.

The Ghazala Khan murder is a shame. Its unfortunate that archaic crap
goes on. I wonder how much the men would like it if the women got to
respond in kind for offenses such as staying out late with the
boys....

Thanks Bruno,
Jeff