Re: [NonGNU ELPA] Add package gptel

[Karthik Chikmagalur]

> This sounds useful, but I think that using it involves an injustice.
> I can't be absolutely certain, but it looks that way.
>
> Plesae correct me if I am mistaken, but it seems that the function of
> this package is to help the user use services which are examples of
> SaaSS (Service as a Software Substitute).  For more explanation, see
> https://gnu.org/philosophy/who-does-that-server-really-serve.html.
>
>
> If I understand right, the package sends a commend to the server to
> request some computing job, and the server sends back the output from
> that job.  That implies that the service in question is SaaSS.

This is correct, this is what all LLM clients enable.  This includes llm
and Ellama, which are in GNU ELPA.

> Making a program available by hiding it in a server is one of the
> unjust alternatives to releasing free software.  (The other unjust
> alternative is releasing it as nonfree software.)  

This argument doesn't apply in full force here.  Most of the programs in
question require supercomputers to run, so there is no way for the user
to run them locally.  Even the smaller, open-source models that the user
can run locally require a powerful workstation with a high-end graphics
card, so they are beyond my reach and the reach of many users.  The
output of these local models is also markedly inferior to the commercial
alternatives.

In other words, there is (at present) no software the user can run as a
substitute for the service.  The quality of the local models is
improving slowly, but so are the ones available as a service and the
large gap between them is holding steady.

> However, most users are unware of this issue and think of hidden
> program as entirely legitimate.  They use those without the slightest
> idea that there is a reason to object to them.

To use these services, the user has to sign up for an account with the
companies in question, supply them with a credit card and generate an
API key, then configure the LLM client to use the API key.  People going
through all these steps are aware of the ersatz local alternatives,
especially since gptel's README lists all of them and links to
instructions to run the local LLMs as well. They have chosen not to (or
like me, cannot afford to) run them.

> It's up to us to educate the users about this, and gptel seems
> like te place to do it.
>
> How can that be done?  One way would be to make gptel
> display a brief explanation (a few lines) of why SaaSS
> is a bad thing, plus a link to more explanaion about the issue.
> It could do this the first time a user uses it,
> and maybe every 30 days after that.
>
> Have you got another idea for how to do this job?

As mentioned above, there is no alternative to using the service, so I'm
not sure how displaying a message about SaaSS will help.  gptel already
supports all the available options to run the (comparatively inferior)
open-source models locally, and makes it easy to use them.

The SaaSS options were the first ones available last year, when people
started using them.  In presenting a uniform interface to all LLMs from
Emacs, I would say gptel makes it easy for Emacs users to switch from
the proprietary SaaSS options to the newer self-hosted, open-source
models.  This is what I've heard from a few gptel users who made the
switch to Ollama (a local LLM hosting software) in past months.

> Whichever way we implement this, it should be added before
> putting the package into ELPA.

To be clear, I am proposing to add gptel to NonGNU ELPA, not GNU ELPA.

Karthik