Re: LLM-generated code in minuet-ai.el (was: minuet-ai.el)

[Milan Glacier]

Eli Zaretskii <eliz@gnu.org> writes:
>> This aspect of the package could actually be a complication.  The GNU
>> Project doesn’t yet have a coherent policy towards accepting code that
>> was produced by LLM.  Some say we should flee such code like the fire,
>> due to potential lawsuits for infringing the copyright of those whose
>> code served as the learning set of the model.  There’s also a relevant
>> issue of whether and to what extent you can claim copyright on code
>> which was partially written by a program.  There was a long discussion
>> about these issues on one of the GNU lists, but no firm conclusions
>> yet.
>> 
>> Can you please describe (in a separate thread) how you typically
>> prompt LLM to generate code for this package, and show a few examples
>> of code in the package that was originally generated by LLM?  This
>> could be important for the purpose of deciding whether we can accept
>> the package and whether you are its sole author.

> how you typically prompt LLM to generate code for this package

When using a model that supports FIM (fill-in-the-middle), the prompt
is the code itself.

Below is a brief explanation of how FIM would work: code with a
`PREFIX“” section (the beginning), optionally a `SUFFIX` (the end),
and a `MIDDLE` section, which is a placeholder.  The model’s task is
to generate code to fill the `MIDDLE` section, completing the code
based on the context provided by prefix and suffix. The prompt itself
is simply these sections concatenated, often with special tokens (for
example `<|fim_prefix|>` `<|fim_suffix|>` `<|fim_middle|>`) indicating
where each part begins and ends.

> show a few examples of code in the package that was originally
> generated by LLM?

1. Generate the docstring for variable description based on the
   variable name and its default value.

2. Generate the default value and docstring for similar variables
   after I have given the variable name, and once it is done I may
   apply some small tweak.

   An example is the defvar form of
   `minuet-auto-suggestion-throttle-delay` (line 71 in minuet.el) is
   generated because `minuet-auto-suggestion-debounce-delay` is
   included in the context.

   Another example is `minuet-gemini-options` (line 309 in minuet.el)
   are generated because `minuet-openai-options` is included in the
   context.

3. Generate the small utility function after I have written the
   docstring of the function. Note that I only generate small utilify
   function where the code is easily predictable on how would it look
   like.

   An example function is `minuet–eval-value` (in line 345 of the
   minuet.el file).

Note that none of the core functionality part are generated, one
reason is that elisp is not a major language and the LLM simply does
not have enough training data especially for package API. For example,
this package uses `plz` package to send web requests, and uses `plz`’s
API for `:filter` to process stream output and uses `:callback` API to
show the completion items in overlay when the request is completed.