[nongnu] elpa/gptel 0a2c07a2a4 1/4: gptel: Support model APIs without system messages

[ELPA Syncer]

branch: elpa/gptel
commit 0a2c07a2a41e5d96e8886fd7e761eefe55caa647
Author: Karthik Chikmagalur <karthikchikmagalur@gmail.com>
Commit: Karthik Chikmagalur <karthikchikmagalur@gmail.com>

    gptel: Support model APIs without system messages
    
    Add support for models that don't use system messages.  By default
    gptel assumes that chat models use system messages (also called
    system prompts or system instructions).  To indicate that a model
    does not, we add the capability `nosystem' to the model's
    capabilities list.  This specification is "inverted" since most
    models support system messages, and we would like to avoid having
    to specify this every time.  This method of configuration is still
    experimental and subject to change.
    
    Add support for unsetting a system message, i.e. setting it to
    nil.  This makes it possible to use models with their default
    system prompts that are baked into their API.  Ensure that gptel's
    transient menu correctly displays and distinguishes between (i) an
    unset system message and (ii) lack of system message support for
    the active model.
    
    In preparation for more model-specific options in gptel's
    transient menus, we refresh the transient menu automatically after
    changing the model.  This is a full refresh since Transient does
    not support more granular reactivity.
    
    * gptel.el (gptel--create-prompt, gptel-request): Handle models
    that don't support system messages correctly, in `gptel-request'
    and with regards to adding context chunks.  If `gptel-use-context'
    is set to `system' but the model does not support system messages,
    add the context to the user prompt instead.
    
    * gptel-transient.el (gptel-system-prompt--setup,
    gptel--format-system-message, gptel--get-directive,
    transient-infix-set, gptel-system-prompt--setup,
    gptel-system-prompt, gptel--suffix-send,
    gptel--suffix-system-message): Handle the display of "no system
    message" and "no support for system message" in the transient
    menus.  New helper function `gptel--format-system-message' to
    format the system message (or equivalent) for display.  Adjust the
    handling of the "additional directive" option so that it works
    correctly when the system message is unset.  The "additional
    directive" becomes the totality of the system message in this
    case.  For models that don't support system messages, the
    "additional directive" option no longer appears.
    
    * gptel-openai.el (gptel--process-models, gptel--parse-buffer):
    When creating model symbol-plists from model specifications,
    overwrite the existing symbol-plist instead of merging.  Without
    this, no keys can be removed from the plist, making it difficult
    to reset model capability descriptions etc.  Fix system message
    support per above description.
    
    * gptel-ollama.el (gptel--parse-buffer): Fix system message
    support per above description.
    
    * gptel-gemini.el (gptel--request-data, gptel--parse-buffer): Fix
    system message support per above description.  Even though the
    model `gemini-pro' does not support system messages, we continue
    to "fake" this by prepending the user prompt with the system
    message to minimize the disruption to gemini Users.  This behavior
    might be changed in the future.
    
    * gptel-anthropic.el (gptel--request-data): Fix system message
    support per above description.
---
 gptel-anthropic.el |  4 +++-
 gptel-gemini.el    |  7 +++++--
 gptel-ollama.el    |  9 ++++++---
 gptel-openai.el    | 17 ++++++++++++-----
 gptel-transient.el | 55 ++++++++++++++++++++++++++++++++++++++----------------
 gptel.el           | 11 ++++++++---
 6 files changed, 73 insertions(+), 30 deletions(-)

diff --git a/gptel-anthropic.el b/gptel-anthropic.el
index a7a447afd9..5f564cd1b2 100644
--- a/gptel-anthropic.el
+++ b/gptel-anthropic.el
@@ -65,12 +65,14 @@
   "JSON encode PROMPTS for sending to ChatGPT."
   (let ((prompts-plist
          `(:model ,(gptel--model-name gptel-model)
-           :system ,gptel--system-message
            :stream ,(or (and gptel-stream gptel-use-curl
                          (gptel-backend-stream gptel-backend))
                      :json-false)
            :max_tokens ,(or gptel-max-tokens 1024)
            :messages [,@prompts])))
+    (when (and gptel--system-message
+               (not (gptel--model-capable-p 'nosystem)))
+      (plist-put prompts-plist :system gptel--system-message))
     (when gptel-temperature
       (plist-put prompts-plist :temperature gptel-temperature))
     ;; Merge request params with model and backend params.
diff --git a/gptel-gemini.el b/gptel-gemini.el
index 80b6a4332c..6c78dca129 100644
--- a/gptel-gemini.el
+++ b/gptel-gemini.el
@@ -74,7 +74,9 @@
         params)
     ;; HACK only gemini-pro doesn't support system messages.  Need a less hacky
     ;; way to do this.
-    (unless (equal gptel-model 'gemini-pro)
+    (if (and gptel--system-message
+             (not (gptel--model-capable-p 'nosystem))
+             (not (equal gptel-model 'gemini-pro)))
       (plist-put prompts-plist :system_instruction
                  `(:parts (:text ,gptel--system-message))))
     (when gptel-temperature
@@ -131,7 +133,8 @@
             prompts))
     ;; HACK Prepend the system message to the first user prompt, but only for
     ;; this model.
-    (when (equal gptel-model 'gemini-pro)
+    (when (and (equal gptel-model 'gemini-pro)
+               gptel--system-message)
       (cl-callf
           (lambda (msg)
             (vconcat `((:text ,(concat gptel--system-message "\n\n"))) msg))
diff --git a/gptel-ollama.el b/gptel-ollama.el
index e55eeed662..69e403e5b5 100644
--- a/gptel-ollama.el
+++ b/gptel-ollama.el
@@ -129,9 +129,12 @@ Intended for internal use only.")
                   :content
                   (string-trim (buffer-substring-no-properties (point-min) 
(point-max))))
             prompts))
-    (cons (list :role "system"
-                :content gptel--system-message)
-          prompts)))
+    (if (and (not (gptel--model-capable-p 'nosystem))
+             gptel--system-message)
+        (cons (list :role "system"
+                    :content gptel--system-message)
+              prompts)
+      prompts)))
 
 (defun gptel--ollama-parse-multipart (parts)
   "Convert a multipart prompt PARTS to the Ollama API format.
diff --git a/gptel-openai.el b/gptel-openai.el
index c633fcd64d..293f196768 100644
--- a/gptel-openai.el
+++ b/gptel-openai.el
@@ -86,8 +86,12 @@
         (cons
          (cl-destructuring-bind (name . props) model
            (setf (symbol-plist name)
-                 (map-merge 'plist (symbol-plist name)
-                            props))
+                 ;; MAYBE: Merging existing symbol plists is safer, but makes 
it
+                 ;; difficult to reset a symbol plist, since removing keys from
+                 ;; it (as opposed to setting them to nil) is more work.
+                 ;;
+                 ;; (map-merge 'plist (symbol-plist name) props)
+                 props)
            (push name models-processed)))))
     (nreverse models-processed)))
 
@@ -186,9 +190,12 @@ with differing settings.")
                   :content
                   (gptel--trim-prefixes (buffer-substring-no-properties 
(point-min) (point-max))))
             prompts))
-    (cons (list :role "system"
-                :content gptel--system-message)
-          prompts)))
+    (if (and (not (gptel--model-capable-p 'nosystem))
+             gptel--system-message)
+        (cons (list :role "system"
+                    :content gptel--system-message)
+              prompts)
+      prompts)))
 
 ;; TODO This could be a generic function
 (defun gptel--openai-parse-multipart (parts)
diff --git a/gptel-transient.el b/gptel-transient.el
index 05209c25a0..cec8d2afdc 100644
--- a/gptel-transient.el
+++ b/gptel-transient.el
@@ -58,7 +58,7 @@ buffer-local value and set its default global value."
 
 Meant to be called when `gptel-menu' is active."
   (cl-some (lambda (s) (and (stringp s) (string-prefix-p ":" s)
-                       (concat "\n\n" (substring s 1))))
+                       (substring s 1)))
                   args))
 
 (defun gptel--instructions-make-overlay (text &optional ov)
@@ -104,6 +104,26 @@ Or is it the other way around?"
   (if (derived-mode-p 'prog-mode)
       "Refactor" "Rewrite"))
 
+(defun gptel--format-system-message (&optional message)
+  "Format the system MESSAGE for display in gptel's transient menus."
+  (setq message (or message gptel--system-message))
+  (if (gptel--model-capable-p 'nosystem)
+      (concat (propertize "[No system message support for model "
+                          'face 'transient-heading)
+              (propertize (gptel--model-name gptel-model)
+                          'face 'warning)
+              (propertize "]" 'face 'transient-heading))
+    (if message
+        (cl-etypecase message
+          (string (string-replace
+                   "\n" "⮐ "
+                   (truncate-string-to-width
+                    message
+                    (max (- (window-width) 12) 14) nil nil t)))
+          (function (gptel--format-system-message (funcall message)))
+          (list (gptel--format-system-message (car message))))
+      "[No system message set]")))
+
 (defvar gptel--crowdsourced-prompts-url
   
"https://raw.githubusercontent.com/f/awesome-chatgpt-prompts/main/prompts.csv";
   "URL for crowdsourced LLM system prompts.")
@@ -238,7 +258,8 @@ This is used only for setting this variable via 
`gptel-menu'.")
     (funcall (oref obj set-value)
              (oref obj model)
              (oset obj model-value model-value)
-             gptel--set-buffer-locally)))
+             gptel--set-buffer-locally))
+  (transient-setup))
 
 (defclass gptel-option-overlaid (transient-option)
   ((display-nil :initarg :display-nil)
@@ -282,13 +303,9 @@ Also format its value in the Transient menu."
 (transient-define-prefix gptel-menu ()
   "Change parameters of prompt to send to the LLM."
   ;; :incompatible '(("-m" "-n" "-k" "-e"))
-  [:description
-   (lambda ()
-     (string-replace
-      "\n" "⮐ "
-      (truncate-string-to-width
-       gptel--system-message (max (- (window-width) 12) 14) nil nil t)))
+  [:description gptel--format-system-message
    [""
+    :if (lambda () (not (gptel--model-capable-p 'nosystem)))
     "Instructions"
     ("s" "Set system message" gptel-system-prompt :transient t)
     (gptel--infix-add-directive)]
@@ -425,7 +442,13 @@ Also format its value in the Transient menu."
        finally return
        (nconc
         prompt-suffixes
-        (list (list "SPC" "Pick crowdsourced prompt"
+        (list (list "DEL" "None"
+                    (lambda () (interactive)
+                      (message "Directive unset")
+                      (gptel--set-with-scope 'gptel--system-message nil
+                                             gptel--set-buffer-locally))
+                    :transient 'transient--do-return)
+              (list "SPC" "Pick crowdsourced prompt"
                     'gptel--read-crowdsourced-prompt
                    ;; NOTE: Quitting the completing read when picking a
                    ;; crowdsourced prompt will cause the transient to exit
@@ -447,11 +470,7 @@ You are a poet. Reply only in verse.
 More extensive system messages can be useful for specific tasks.
 
 Customize `gptel-directives' for task-specific prompts."
-  [:description
-   (lambda () (string-replace
-          "\n" "⮐ "
-          (truncate-string-to-width
-           gptel--system-message (max (- (window-width) 12) 14) nil nil t)))
+  [:description gptel--format-system-message
    [(gptel--suffix-system-message)]
    [(gptel--infix-variable-scope)]]
    [:class transient-column
@@ -820,7 +839,11 @@ Or in an extended conversation:
              :position position
              :in-place (and in-place (not output-to-other-buffer-p))
              :stream stream
-             :system (concat gptel--system-message system-extra)
+             :system (if system-extra
+                         (concat (if gptel--system-message
+                                     (concat gptel--system-message "\n\n"))
+                                 system-extra)
+                       gptel--system-message)
              :callback callback
              :dry-run dry-run)
 
@@ -946,7 +969,7 @@ When LOCAL is non-nil, set the system message only in the 
current buffer."
         ;; (insert (propertize (make-separator-line) 'rear-nonsticky t))
         (set-marker msg-start (point))
         (save-excursion
-          (insert (buffer-local-value 'gptel--system-message orig-buf))
+          (insert (or (buffer-local-value 'gptel--system-message orig-buf) ""))
           (push-mark nil 'nomsg))
         (activate-mark))
       (display-buffer (current-buffer)
diff --git a/gptel.el b/gptel.el
index 37fc8e8948..c8a00456e4 100644
--- a/gptel.el
+++ b/gptel.el
@@ -1261,7 +1261,8 @@ Model parameters can be let-bound around calls to this 
function."
   (let* ((gptel--system-message
           ;Add context chunks to system message if required
           (if (and gptel-context--alist
-                   (eq gptel-use-context 'system))
+                   (eq gptel-use-context 'system)
+                   (not (gptel--model-capable-p 'nosystem)))
               (gptel-context--wrap system)
             system))
          (gptel-stream stream)
@@ -1436,8 +1437,12 @@ there."
                   (gptel--parse-buffer gptel-backend max-entries)))))
         ;; NOTE: prompts is modified in place here
         (when gptel-context--alist
-          ;; Inject context chunks into the last user prompt if required
-          (when (and (eq gptel-use-context 'user)
+          ;; Inject context chunks into the last user prompt if required.
+          ;; This is also the fallback for when `gptel-use-context' is set to
+          ;; 'system but the model does not support system messages.
+          (when (and gptel-use-context
+                     (or (eq gptel-use-context 'user)
+                         (gptel--model-capable-p 'nosystem))
                      (> (length prompts) 0)) ;FIXME context should be injected
                                              ;even when there are no prompts
             (gptel--wrap-user-prompt gptel-backend prompts))