Re: What's missing in ELisp that makes people want to use cl-lib?

[Dmitry Gutov]

On 15/11/2023 17:02, João Távora wrote:
> On Wed, Nov 15, 2023 at 2:05 PM Dmitry Gutov <dmitry@gutov.dev> wrote:
> > On 15/11/2023 14:14, João Távora wrote:
> > > > > But say I did that seq-do, then what is the seq-contains-p generic good
> > > > > for then?  Why some many redundant generics that the user learns the
> > > > > hard way have to be consistent with each other?
> > > > So any new type gets a lot of functions implemented very easily, and
> > > > could add specialized implementations for performance.
> > > Fair enough.  Except, as you noted, those optimizations can be
> > > completely nullified overnight, when trying to optimize something
> > > else.
> >
> > I don't see where they would be "completely nullified".
>
> You wrote "tank" :-) I was talking about those kinds of optimisations,
> not the ones you're making.  What I meant is in order to optimize the
> list/sequence/#equal/#eql/#eq case you tank the user's optimizations
> to custom sequences.

Only until the author of the third-party data type updates their code to 
include the method seq-contains-pred.

And if that code is not inside Emacs, its time-to-market measures in 
days, not months or years.

> > Even that one should be improved, though.
>
> The only way you are eventually going to get to equal performance
> with cl-lib's fastest variants (but only the non-destructive, of course)
> is if you start every seq.el entry point with sth like
>
>     (if (sequence-p seq) (call-equivalent-cl-lib-version seq) ...)
>
> But then I don't see how that fits in with the cl-defgeneric,
> which does dispatching before that if.  The only way I see this
> is to invent a new kind of specializer in cl-generic.el like
> 'definitely-not-sequence'.  There could be some other solution,
> maybe Michael as a clue.
>
> Anyway we do it, it's much easier than trying to wrangle entry-point
> by entry-point by adding more generics and finding pitfalls like this
> one about seq-contains-p.  And this I think, is what Michael is
> suggesting: fair enough.

I don't immediately see that new generics are needed. But if someone 
finds a workable approach (with hard numbers), good.

> But of course we will be outlawing every extension like the :m6-sparse
> example I gave.
>
> And also, IMHO, we end up with a much poorer protocol in terms
> of versatility (no find-by-key :key, no :from-end, no :start/:end).
> But this part is not about performance, rather interface, and that's
> not the topic of this subthread.

cl-lib is more flexible in one aspect (its additional keywords 
vocabulary which basically multiplies the provided interface by 100x), 
but it's more rigid in the data it works with.

> > If that is the only drawback we can find, it is an unavoidable one.
>
> There is still the other example I gave you.  I think your only
> way out is to outlaw my :m6-sparse extension.  You basically have
> to outlaw everything (:around, :before, specializers) that
> cl-defgeneric gives the user when it comes to the sequence types
> for which you want to have acceptable performance.

Could you try to explain what I should find in the second example? What 
do you mean by "outlaw"? Does causing worse performance for a short time 
constitute "outlawing" something?

> And even outlaw more stuff.  For example, these generics even have
> specializers on all mandatory arguments?
>
> For example, why does seq-do have FUNCTION as a required argument???

Because FUNCTION is applied to SEQUENCE?

> It should be &optional or a &key, with a default value.  Which
> cl-degeneric obviously supports. That way specializations on FUNCTION
> become impossible,  or at least become much harder and there's less
> risk of tanking user code.  Design mistake IMO.

I'm reasonably sure nobody expects function to be anything but a 
straight function (symbol or a lambda), because that's how 'seq-do' is 
used throughout the code.

> > But that is the price one has to pay for correcting a design mistake.
> > We're not going to do this every Tuesday.
>
> Sure, but that price increases manyfold if we start suggesting
> seq.el as a replacement for all your sequence processing needs.

We can first fix the mistake and then go on to continue "suggesting it 
as a replacement". Or not.

I don't exactly see it that way, though. And you give an impression of 
arguing for the opposite: toward never using it at all.

> Why working on the :m6-sparse extension, I noticed Emacs becomes
> noticeably slower, and I suspect that's because while I was editing,
> all the seq  functions I was writing where being probed for
> applicability, while core things like completion, buffer-switching
> etc are calling seq-foo generics.

It could be helpful to do some profiling and see where the slowdown came 
from. Could it come exactly from the set operations?

> I find this performance aspect very bad.  Maybe it can be obviated,
> but only if you drop the '(if (sequence-p seq)' bomb into seq.el
> somehow.  I don't see how we can avoid that one.

I don't quite see the need. And it's unlikely to be of reliable help: my 
observations say that method dispatch simply becomes slower as soon as a 
generic function gets a second implementation. And that implementation 
might arrive from any third-party code.