Re: Making --with-wide-int the default

[Eli Zaretskii]

> From: David Kastrup <address@hidden>
> Cc: address@hidden,  address@hidden,  address@hidden
> Date: Fri, 16 Oct 2015 12:27:04 +0200
> 
> >> > How can GMP help extend the maximum size of buffers and strings beyond
> >> > what a 32-bit EMACS_INT allows?
> >> 
> >> By choosing an appropriate data type for representing buffer/string
> >> sizes in C and converting back-and-forth from the Lisp type as needed.
> >> Pretty much the same way we do it now.
> >
> > Sorry, I don't foollow: what "appropriate data type"?  Would that be
> > 'long long'?  If so, that's exactly what we do now, which you say is
> > "going to 64-bit unilaterally".  What am I missing?
> 
> A change in the size of the Lisp data type.

We are talking about the _implementation_ of Lisp data types,
i.e. about the underlying C data types.  What am I missing on that
level?

> >> I think it would be a reasonable restriction to keep to 2GB size of
> >> strings and buffers when working with a 32bit executable.  That's
> >> what people expect on a 32-bit architecture.
> >
> > That's what we have now in 32-bit builds --with-wide-int.  So I'm not
> > sure why you mention that as some kind of change related to this
> > discussion.
> 
> Isn't that also changing the size of a Lisp cell?  And of integer
> arithmetic?

Part of it, yes.  But since a Lisp cell can hold buffer or string
position, what else can you expect?

> >> When you are editing gigabyte files, at some point of time, the Lisp
> >> representation of the respective offsets in the high part of the
> >> buffer will become the responsibility of GMP, yes.  I'm not worried
> >> about that.
> >
> > I don't understand.  C doesn't have dynamic types.
> 
> But Lisp does.

I was talking about C implementation of Lisp types.

> > If the variable that holds buffer positions needs to support 61-bit
> > offsets, it will have to be a 64-bit integral data type from the
> > get-go.
> 
> I repeat: we are talking about a 32-bit binary where restricting buffer
> and string size to 32bit offsets would be reasonable and expected.

You cannot have full 32-bit offsets without enlarging the width of a
Lisp integer.  We could use 34 or 35-bit numbers, but that would be
slower than using a 64-bit 'long long'.

> Choose a C type of your choosing for dealing with buffer offsets, create
> aliases for its conversions, and you are good to go.

We _have_ chosen: it's a 64-bit 'long long'.  What do you suggest to
choose instead?  Any suggested type should be representable as an
Emacs integer.

> > And having 61-bit integers for integer arithmetics is also a valuable
> > feature.
> 
> 61-bit is some arbitrary junk number.  Transparently degrading from
> 29-bits to gmp numbers means that there are no arbitrary limits (or at
> least you are unlikely to hit them) and the performance for the vast
> majority of cases will be 29-bit performance.

I don't understand how you intend to "transparently degrade" in the
implementation of '+', for example.  Are you going to test each
argument for whether it fits into 32-bit limits, and if so, invoke
32-bit arithmetics, else 64-bit arithmetics?  If so, I'm quite sure
the test will steal cycles that will make this slower than just always
using 64-bit arithmetics.

If you mean something else, can you show some C code that would
demonstrate your ideas?

> > So the EMACS_INT type will have to be able to support that.
> 
> I don't see why when one can use GMP (which effectively uses the same
> kind arithmetic for 61-bit numbers as C does but does not stop there).

Because calling a GMP function will most probably yield slower code
than if we let the compiler emit a few instructions required for
64-bit arithmetics.