bug#8528: 24.0.50; 32-bit Emacs with apparent 128M buffer size limit

[Eli Zaretskii]

> Date: Wed, 20 Apr 2011 23:18:55 -0700
> From: Paul Eggert <eggert@cs.ucla.edu>
> CC: Evans Winner <ego111@gmail.com>, 8528@debbugs.gnu.org
> 
> On 04/20/11 22:52, Eli Zaretskii wrote:
> > Paul, could you please tell where do you see twice doubling of the
> > file size in insert-file-contents?
> 
> I assumed that it was because the internal buffers contain an
> Emacs-encoded version of the file, which could be as long as four
> times the actual file size, because a single byte in the file
> might expand to 4 bytes inside Emacs in some cases.

Actually, it could potentially expand even 5-fold (because Emacs
extends UTF-8 to codepoints as large as 0x3FFFFF).  But we test the
buffer size and avoid overflowing it in many other places, both
further down in insert-file-contents and in insdel.c.  If those are
not enough, we could add more such tests, particularly after decoding
the file's contents, where we know the full buffer size in bytes.

So I think artificially limiting the maximum size of a file that can
be visited in that particular place in insert-file-contents is too
harsh.

> That would explain the behavior that you saw: if your file's
> internal encoding was the same as the external, you wouldn't observe any
> problem.  The problem would be exhibited only with files containing
> many characters that bloat when read into memory.

Right, but wouldn't you agree that such a limitation is too stringent?
E.g., I should be able to use find-file-literally to visit a 512MB
file, but currently I cannot.