Re: Huge file adventure (+patch)

[Stefan Monnier]

> And, of course, there is a patch to address an issues described above.
> Comments are very welcome because I'm not hooked too much in coding
> machinery (yet).

Sounds good, in general.

>        else
>       {
> -       coding->dst_object
> -         = make_unibyte_string ((char *) coding->destination,
> -                                coding->produced);
> -       xfree (coding->destination);
> +       /* This is used to avoid creating huge Lisp string.
> +          NOTE: caller who set `raw_destination' is also
> +          responsible to free `destination' buffer.  */
> +       if (coding->raw_destination)
> +         coding->dst_object = Qnil;
> +       else
> +         {
> +           coding->dst_object
> +             = make_unibyte_string ((char *) coding->destination,
> +                                    coding->produced);
> +           xfree (coding->destination);
> +         }
>       }
>      }
 
You can remove the { between "else" and "if" (with the side-benefit that
the unchanged code will stay at the same indentation level).

>         if (CODING_REQUIRE_ENCODING (coding))
>           {
> -           encode_coding_object (coding, string,
> -                                 start, string_char_to_byte (string, start),
> -                                 end, string_char_to_byte (string, end), Qt);
> +           ptrdiff_t nchars = min (end - start, E_WRITE_MAX);
> +
> +           /* Avoid creating huge Lisp string in encode_coding_object.  */
> +           if (nchars == E_WRITE_MAX)
> +             coding->raw_destination = 1;
> +
> +           encode_coding_object
> +             (coding, string, start, string_char_to_byte (string, start),
> +              start + nchars, string_char_to_byte (string, start + nchars),
> +              Qt);
>           }

Where/how do you make sure we loop back here for the rest
(when end-start > E_WRITE_MAX) ?


        Stefan