Re: What's missing in ELisp that makes people want to use cl-lib?

[João Távora]

On Wed, Nov 15, 2023 at 2:05 PM Dmitry Gutov <dmitry@gutov.dev> wrote:
>
> On 15/11/2023 14:14, João Távora wrote:
> >>> But say I did that seq-do, then what is the seq-contains-p generic good
> >>> for then?  Why some many redundant generics that the user learns the
> >>> hard way have to be consistent with each other?
> >> So any new type gets a lot of functions implemented very easily, and
> >> could add specialized implementations for performance.
> > Fair enough.  Except, as you noted, those optimizations can be
> > completely nullified overnight, when trying to optimize something
> > else.
>
> I don't see where they would be "completely nullified".

You wrote "tank" :-)  I was talking about those kinds of optimisations,
not the ones you're making.  What I meant is in order to optimize the
list/sequence/#equal/#eql/#eq case you tank the user's optimizations
to custom sequences.

> The gains seem
> consistent, even if they don't extend as much to certain scenarios, like
> custom test-fn.

In that particular corner, both your (the library dev's) optimisations
and the user optimisations are tanked, yes.

> Even that one should be improved, though.

The only way you are eventually going to get to equal performance
with cl-lib's fastest variants (but only the non-destructive, of course)
is if you start every seq.el entry point with sth like

   (if (sequence-p seq) (call-equivalent-cl-lib-version seq) ...)

But then I don't see how that fits in with the cl-defgeneric,
which does dispatching before that if.  The only way I see this
is to invent a new kind of specializer in cl-generic.el like
'definitely-not-sequence'.  There could be some other solution,
maybe Michael as a clue.

Anyway we do it, it's much easier than trying to wrangle entry-point
by entry-point by adding more generics and finding pitfalls like this
one about seq-contains-p.  And this I think, is what Michael is
suggesting: fair enough.

But of course we will be outlawing every extension like the :m6-sparse
example I gave.

And also, IMHO, we end up with a much poorer protocol in terms
of versatility (no find-by-key :key, no :from-end, no :start/:end).
But this part is not about performance, rather interface, and that's
not the topic of this subthread.

> If that is the only drawback we can find, it is an unavoidable one.

There is still the other example I gave you.  I think your only
way out is to outlaw my :m6-sparse extension.  You basically have
to outlaw everything (:around, :before, specializers) that
cl-defgeneric gives the user when it comes to the sequence types
for which you want to have acceptable performance.

And even outlaw more stuff.  For example, these generics even have
specializers on all mandatory arguments?

For example, why does seq-do have FUNCTION as a required argument???
It should be &optional or a &key, with a default value.  Which
cl-degeneric obviously supports. That way specializations on FUNCTION
become impossible,  or at least become much harder and there's less
risk of tanking user code.  Design mistake IMO.

> But that is the price one has to pay for correcting a design mistake.
> We're not going to do this every Tuesday.

Sure, but that price increases manyfold if we start suggesting
seq.el as a replacement for all your sequence processing needs.
We just shouldn't.

Why working on the :m6-sparse extension, I noticed Emacs becomes
noticeably slower, and I suspect that's because while I was editing,
all the seq  functions I was writing where being probed for
applicability, while core things like completion, buffer-switching
etc are calling seq-foo generics.

I find this performance aspect very bad.  Maybe it can be obviated,
but only if you drop the '(if (sequence-p seq)' bomb into seq.el
somehow.  I don't see how we can avoid that one.

João