Re: Emacs Lisp's future

[Mark H Weaver]

Richard Stallman <address@hidden> writes:
>     So if we can't make use of Guile's strings because they're not good
>     enough, then we won't use them.
>
> Unfortunately, that would put a major crimp in interoperability between
> Emacs Lisp programs and Scheme programs.
>
> Can the Guile developers work on making Guile strings flexible
> enough that Emacs can use them?  So that they can do the jobs
> Emacs Lisp strings do?

I would like to change Guile's internal string representation to use a
generalization of UTF-8, as used by Emacs.  There are two sticking
points that require more thought, however:

* I'm concerned that there are security implications to supporting the
  "raw byte" code points.  I can expand on this more if you'd like.

  However, I think this will not be a problem, because the
  string<->bytevector conversion procedures could support two modes of
  operation: one mode that supports these raw bytes, for use by emacs
  and maybe some other purposes (e.g. dealing with POSIX file names),
  and another mode that refuses to accept or produce invalid UTF-8,
  which would be used by programs where security is a concern.  I'm
  inclined to make the standard Scheme procedures use the strict mode
  by default.

* I'm not sure that Guile strings should include property lists.  One
  can reasonably assume that competent Elisp programmers will keep in
  mind that Elisp strings are more than just characters, but we cannot
  expect that of Scheme programmers, and they've never had any tools to
  deal with it in any case.  Emacs lisp includes procedures such as
  'substring-no-properties', but Scheme has never had anything like
  that.

Supporting property lists in Scheme raises difficult questions
such as:

 * What should the Scheme procedures 'string=?' and 'equal?' do when
   comparing two strings with the equal character sequences but
   unequal property lists?

 * Should Scheme procedures such as 'substring', 'string-append',
   'string-upcase', etc, propagate the associated property list
   data?

 * Are there security implications to carrying around and possibly
   propagating (via Scheme's "substring") extra information that is
   effectively invisible to all procedures that have ever been
   available in Scheme?

 * What should Scheme's 'write' do when applied to a string that
   includes a property list?  ('write' is analogous to 'prin1').

Security concerns are more important for Guile than for Emacs, because
Guile is already being used to implement network programs such as web
servers, and generally aims to support a much wider range of
applications.

At the very least, we can plan to eventually make Emacs strings
representable as Guile strings plus property lists.  Going further
will require more consideration, I think.

* * *

While we're on the subject of interoperability between Emacs Lisp and
Scheme programs, I'm concerned about nil.  Modern Scheme requires that
() and #f are distinct objects, and that () is treated as true by 'if'.
This has been the case for long enough now that it's not uncommon for
modern Scheme code to depend on these facts.  Of course, Elisp code
depends on the end-of-list and false values being the same object.

The way we're coping with this is by having three distinct objects in
Guile: (), #f, and #nil, where both () and #nil are considered
end-of-list markers by 'null?', both #f and #nil are considered false by
Scheme 'if', and all three are seen as null by Elisp.

First of all, clearly this is not quite correct.  Scheme code might
assume that if a value is false, it cannot be the empty list, or vice
versa, but the hope is that it will mostly work in practice.

However, I see a problem that will become more common if Scheme and
Elisp code become more intertwined.  The problem occurs when Elisp code
sees that a value 'x' is null and then copies it somewhere, where the
original 'x' was conceptually an end-of-list, but the copy is
conceptually a boolean false (or vice versa).

(I'm aware that for some Elisp values, it may not even be possible to
say it's "conceptually an end-of-list" or "conceptually a boolean
false", but please bear with me).

The problem comes when 'x' originated in Scheme code as (), is later
copied by Elisp code into something that's conceptually a boolean, and
then that copy is inspected by Scheme code.  The intent was that the
copied boolean would be false, but the Scheme code will see () and treat
it as true.

What do you think?  Do I worry too much? :)

      Mark