[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[nongnu] elpa/gptel bcbbe67e08: gptel: Add arbitrary params to request
|
From: |
ELPA Syncer |
|
Subject: |
[nongnu] elpa/gptel bcbbe67e08: gptel: Add arbitrary params to request |
|
Date: |
Mon, 11 Nov 2024 13:00:05 -0500 (EST) |
branch: elpa/gptel
commit bcbbe67e08d423884c2554c95b91d5f0f9b833e3
Author: Karthik Chikmagalur <karthikchikmagalur@gmail.com>
Commit: Karthik Chikmagalur <karthikchikmagalur@gmail.com>
gptel: Add arbitrary params to request
Provide a way to add arbitrary parameters to the `gptel-request'
payload. This is required for customizing parameters that gptel
does not provide user options for.
Examples in #330, #415, #471.
This is done by adding a `:request-params' field to the backend
and the model. This should be a plist whose key/value pairs are
the additional request parameters and their values. In case of
conflict, the model-specific request-parameters override the
backend-specific ones, which override gptel's dynamically set user
options. Valid parameters depend on the specific API being
queried, and can be found in the corresponding API documentation.
* gptel.el (gptel--merge-plists, gptel--model-request-params): Add
utility functions.
* gptel-openai.el (gptel-backend, gptel-make-openai,
gptel-make-azure, gptel-make-gpt4all, gptel--request-data): Add
request-params slot to `gptel-backend' and keyword to
`gptel-make-openai' and its derivatives. Merge model and
backend-specific request parameters when sending a request.
* gptel-privategpt.el (gptel-make-privategpt): Add request-params
keyword.
* gptel-ollama.el (gptel-make-ollama, gptel--request-data): Add
request-params keywod to `gptel-make-ollama'. Merge model and
backend-specific request parameters when sending a request.
Remove ad hoc value of :num_ctx used according to #330 -- this is
no longer needed as :num_ctx can be set explicitly when defining
the Ollama model or backend.
* gptel-kagi.el (gptel--request-data): Add TODO to implement model
and backend-specific request parameters.
* gptel-gemini.el (gptel-make-gemini, gptel--request-data): Add
request-params keywod to `gptel-make-gemini'. Merge model and
backend-specific request parameters when sending a request.
* gptel-curl.el (gptel-curl-get-response): Check model and
backend-specific stream settings in their request-params slot when
sending the request. These take priority over the global
`gptel-stream' and the backend's `:stream' value.
* gptel-anthropic.el (gptel-make-anthropic, gptel--request-data):
Add request-params keywod to `gptel-make-ollama'. Merge model and
backend-specific request parameters when sending a request.
---
gptel-anthropic.el | 19 ++++++++++++++++---
gptel-curl.el | 11 ++++++++++-
gptel-gemini.el | 20 +++++++++++++++++---
gptel-kagi.el | 1 +
gptel-ollama.el | 22 ++++++++++++++--------
gptel-openai.el | 43 +++++++++++++++++++++++++++++++++----------
gptel-privategpt.el | 16 +++++++++++++---
gptel.el | 20 ++++++++++++++++++++
8 files changed, 124 insertions(+), 28 deletions(-)
diff --git a/gptel-anthropic.el b/gptel-anthropic.el
index 544854325e..a7a447afd9 100644
--- a/gptel-anthropic.el
+++ b/gptel-anthropic.el
@@ -73,7 +73,11 @@
:messages [,@prompts])))
(when gptel-temperature
(plist-put prompts-plist :temperature gptel-temperature))
- prompts-plist))
+ ;; Merge request params with model and backend params.
+ (gptel--merge-plists
+ prompts-plist
+ (gptel-backend-request-params gptel-backend)
+ (gptel--model-request-params gptel-model))))
(cl-defmethod gptel--parse-buffer ((_backend gptel-anthropic) &optional
max-entries)
(let ((prompts) (prop)
@@ -262,6 +266,9 @@ Keys:
- `:cutoff-date': the knowledge cutoff date.
+- `:request-params': a plist of additional request parameters to
+ include when using this model.
+
Information about the Anthropic models was obtained from the following
sources:
@@ -271,7 +278,7 @@ sources:
;;;###autoload
(cl-defun gptel-make-anthropic
- (name &key curl-args stream key
+ (name &key curl-args stream key request-params
(header
(lambda () (when-let (key (gptel--get-api-key))
`(("x-api-key" . ,key)
@@ -324,7 +331,12 @@ alist, like:
((\"Content-Type\" . \"application/json\"))
KEY is a variable whose value is the API key, or function that
-returns the key."
+returns the key.
+
+REQUEST-PARAMS (optional) is a plist of additional HTTP request
+parameters (as plist keys) and values supported by the API. Use
+these to set parameters that gptel does not provide user options
+for."
(declare (indent 1))
(let ((backend (gptel--make-anthropic
:curl-args curl-args
@@ -336,6 +348,7 @@ returns the key."
:protocol protocol
:endpoint endpoint
:stream stream
+ :request-params request-params
:url (if protocol
(concat protocol "://" host endpoint)
(concat host endpoint)))))
diff --git a/gptel-curl.el b/gptel-curl.el
index d428ee5296..6107b83c37 100644
--- a/gptel-curl.el
+++ b/gptel-curl.el
@@ -109,7 +109,16 @@ the response is inserted into the current buffer after
point."
(random) (emacs-pid) (user-full-name)
(recent-keys))))
(args (gptel-curl--get-args (plist-get info :data) token))
- (stream (and gptel-stream (gptel-backend-stream gptel-backend)))
+ (stream (and ;; Check model-specific request-params for streaming
preference
+ (let* ((model-params (gptel--model-request-params
gptel-model))
+ (stream-spec (plist-get model-params :stream)))
+ ;; If not present, there is no model-specific preference
+ (or (not (memq :stream model-params))
+ ;; If present, it must not be :json-false or nil
+ (and stream-spec (not (eq stream-spec :json-false)))))
+ ;; Check global and backend-specific streaming settings
+ gptel-stream
+ (gptel-backend-stream gptel-backend)))
(backend (buffer-local-value 'gptel-backend (plist-get info :buffer)))
(process (apply #'start-process "gptel-curl"
(generate-new-buffer "*gptel-curl*") "curl" args)))
diff --git a/gptel-gemini.el b/gptel-gemini.el
index 1d6630019e..80b6a4332c 100644
--- a/gptel-gemini.el
+++ b/gptel-gemini.el
@@ -88,7 +88,11 @@
(when params
(plist-put prompts-plist
:generationConfig params))
- prompts-plist))
+ ;; Merge request params with model and backend params.
+ (gptel--merge-plists
+ prompts-plist
+ (gptel-backend-request-params gptel-backend)
+ (gptel--model-request-params gptel-model))))
(cl-defmethod gptel--parse-buffer ((_backend gptel-gemini) &optional
max-entries)
(let ((prompts) (prop)
@@ -230,6 +234,9 @@ Keys:
- `:cutoff-date': the knowledge cutoff date.
+- `:request-params': a plist of additional request parameters to
+ include when using this model.
+
Information about the Gemini models was obtained from the following
source:
@@ -238,7 +245,8 @@ source:
;;;###autoload
(cl-defun gptel-make-gemini
- (name &key curl-args header key (stream nil)
+ (name &key curl-args header key request-params
+ (stream nil)
(host "generativelanguage.googleapis.com")
(protocol "https")
(models gptel--gemini-models)
@@ -289,7 +297,12 @@ alist, like:
((\"Content-Type\" . \"application/json\"))
KEY (optional) is a variable whose value is the API key, or
-function that returns the key."
+function that returns the key.
+
+REQUEST-PARAMS (optional) is a plist of additional HTTP request
+parameters (as plist keys) and values supported by the API. Use
+these to set parameters that gptel does not provide user options
+for."
(declare (indent 1))
(let ((backend (gptel--make-gemini
:curl-args curl-args
@@ -300,6 +313,7 @@ function that returns the key."
:protocol protocol
:endpoint endpoint
:stream stream
+ :request-params request-params
:key key
:url (lambda ()
(let ((method (if (and stream
diff --git a/gptel-kagi.el b/gptel-kagi.el
index 7e0304f7c0..3eecdc9864 100644
--- a/gptel-kagi.el
+++ b/gptel-kagi.el
@@ -69,6 +69,7 @@
(concat "\n\n" (mapconcat #'identity ref-strings "\n")))))
(concat output references)))
+;; TODO: Add model and backend-specific request-params support
(cl-defmethod gptel--request-data ((_backend gptel-kagi) prompts)
"JSON encode PROMPTS for Kagi."
(pcase-exhaustive (gptel--model-name gptel-model)
diff --git a/gptel-ollama.el b/gptel-ollama.el
index 0ec6c27a2d..e55eeed662 100644
--- a/gptel-ollama.el
+++ b/gptel-ollama.el
@@ -80,9 +80,7 @@ Intended for internal use only.")
:messages [,@prompts]
:stream ,(or (and gptel-stream gptel-use-curl
(gptel-backend-stream gptel-backend))
- :json-false)))
- ;; TODO num_ctx chosen according to #330, make customizable
- (options-plist (list :num_ctx 8192)))
+ :json-false))))
(when gptel-temperature
(setq options-plist
(plist-put options-plist :temperature
@@ -91,9 +89,11 @@ Intended for internal use only.")
(setq options-plist
(plist-put options-plist :num_predict
gptel-max-tokens)))
- (when options-plist
- (plist-put prompts-plist :options options-plist))
- prompts-plist))
+ ;; Merge request params with model and backend params.
+ (gptel--merge-plists
+ prompts-plist
+ (gptel-backend-request-params gptel-backend)
+ (gptel--model-request-params gptel-model))))
(cl-defmethod gptel--parse-buffer ((_backend gptel-ollama) &optional
max-entries)
(let ((prompts) (prop)
@@ -177,7 +177,7 @@ If INJECT-MEDIA is non-nil wrap it with base64-encoded
media files in the contex
;;;###autoload
(cl-defun gptel-make-ollama
- (name &key curl-args header key models stream
+ (name &key curl-args header key models stream request-params
(host "localhost:11434")
(protocol "http")
(endpoint "/api/chat"))
@@ -227,13 +227,18 @@ KEY (optional) is a variable whose value is the API key,
or
function that returns the key. This is typically not required
for local models like Ollama.
+REQUEST-PARAMS (optional) is a plist of additional HTTP request
+parameters (as plist keys) and values supported by the API. Use
+these to set parameters that gptel does not provide user options
+for.
+
Example:
-------
(gptel-make-ollama
\"Ollama\"
:host \"localhost:11434\"
- :models \\='(\"mistral:latest\")
+ :models \\='(mistral:latest)
:stream t)"
(declare (indent 1))
(let ((backend (gptel--make-ollama
@@ -246,6 +251,7 @@ Example:
:protocol protocol
:endpoint endpoint
:stream stream
+ :request-params request-params
:url (if protocol
(concat protocol "://" host endpoint)
(concat host endpoint)))))
diff --git a/gptel-openai.el b/gptel-openai.el
index 744ca04d74..c633fcd64d 100644
--- a/gptel-openai.el
+++ b/gptel-openai.el
@@ -50,6 +50,7 @@
(declare-function json-read "json")
(declare-function gptel-prompt-prefix-string "gptel")
(declare-function gptel-response-prefix-string "gptel")
+(declare-function gptel--merge-plists "gptel")
(declare-function gptel-context--wrap "gptel-context")
(defmacro gptel--json-read ()
@@ -104,7 +105,8 @@ with differing settings.")
(gptel-backend (:constructor gptel--make-backend)
(:copier gptel--copy-backend))
name host header protocol stream
- endpoint key models url curl-args)
+ endpoint key models url request-params
+ curl-args)
;;; OpenAI (ChatGPT)
(cl-defstruct (gptel-openai (:constructor gptel--make-openai)
@@ -140,8 +142,12 @@ with differing settings.")
(when gptel-temperature
(plist-put prompts-plist :temperature gptel-temperature))
(when gptel-max-tokens
- (plist-put prompts-plist :max_tokens gptel-max-tokens))
- prompts-plist))
+ (plist-put prompts-plist :max_completion_tokens gptel-max-tokens))
+ ;; Merge request params with model and backend params.
+ (gptel--merge-plists
+ prompts-plist
+ (gptel-backend-request-params gptel-backend)
+ (gptel--model-request-params gptel-model))))
(cl-defmethod gptel--parse-buffer ((_backend gptel-openai) &optional
max-entries)
(let ((prompts) (prop)
@@ -248,7 +254,7 @@ files in the context."
;;;###autoload
(cl-defun gptel-make-openai
- (name &key curl-args models stream key
+ (name &key curl-args models stream key request-params
(header
(lambda () (when-let (key (gptel--get-api-key))
`(("Authorization" . ,(concat "Bearer " key))))))
@@ -298,7 +304,12 @@ alist, like:
((\"Content-Type\" . \"application/json\"))
KEY (optional) is a variable whose value is the API key, or
-function that returns the key."
+function that returns the key.
+
+REQUEST-PARAMS (optional) is a plist of additional HTTP request
+parameters (as plist keys) and values supported by the API. Use
+these to set parameters that gptel does not provide user options
+for."
(declare (indent 1))
(let ((backend (gptel--make-openai
:curl-args curl-args
@@ -310,6 +321,7 @@ function that returns the key."
:protocol protocol
:endpoint endpoint
:stream stream
+ :request-params request-params
:url (if protocol
(concat protocol "://" host endpoint)
(concat host endpoint)))))
@@ -325,7 +337,7 @@ function that returns the key."
(protocol "https")
(header (lambda () `(("api-key" . ,(gptel--get-api-key)))))
(key 'gptel-api-key)
- models stream endpoint)
+ models stream endpoint request-params)
"Register an Azure backend for gptel with NAME.
Keyword arguments:
@@ -334,7 +346,7 @@ CURL-ARGS (optional) is a list of additional Curl arguments.
HOST is the API host.
-MODELS is a list of available model names.
+MODELS is a list of available model names, as symbols.
STREAM is a boolean to toggle streaming responses, defaults to
false.
@@ -351,6 +363,11 @@ alist, like:
KEY (optional) is a variable whose value is the API key, or
function that returns the key.
+REQUEST-PARAMS (optional) is a plist of additional HTTP request
+parameters (as plist keys) and values supported by the API. Use
+these to set parameters that gptel does not provide user options
+for.
+
Example:
-------
@@ -361,7 +378,7 @@ Example:
:endpoint
\"/openai/deployments/DEPLOYMENT_NAME/completions?api-version=2023-05-15\"
:stream t
- :models \\='(\"gpt-3.5-turbo\" \"gpt-4\"))"
+ :models \\='(gpt-3.5-turbo gpt-4))"
(declare (indent 1))
(let ((backend (gptel--make-openai
:curl-args curl-args
@@ -373,6 +390,7 @@ Example:
:protocol protocol
:endpoint endpoint
:stream stream
+ :request-params request-params
:url (if protocol
(concat protocol "://" host endpoint)
(concat host endpoint)))))
@@ -392,7 +410,7 @@ CURL-ARGS (optional) is a list of additional Curl arguments.
HOST is where GPT4All runs (with port), typically localhost:4891
-MODELS is a list of available model names.
+MODELS is a list of available model names, as symbols.
STREAM is a boolean to toggle streaming responses, defaults to
false.
@@ -411,6 +429,11 @@ KEY (optional) is a variable whose value is the API key, or
function that returns the key. This is typically not required for
local models like GPT4All.
+REQUEST-PARAMS (optional) is a plist of additional HTTP request
+parameters (as plist keys) and values supported by the API. Use
+these to set parameters that gptel does not provide user options
+for.
+
Example:
-------
@@ -418,7 +441,7 @@ Example:
\"GPT4All\"
:protocol \"http\"
:host \"localhost:4891\"
- :models \\='(\"mistral-7b-openorca.Q4_0.gguf\"))")
+ :models \\='(mistral-7b-openorca.Q4_0.gguf))")
(provide 'gptel-openai)
;;; gptel-openai.el ends here
diff --git a/gptel-privategpt.el b/gptel-privategpt.el
index a10695f786..0eca517af2 100644
--- a/gptel-privategpt.el
+++ b/gptel-privategpt.el
@@ -96,12 +96,16 @@
(plist-put prompts-plist :temperature gptel-temperature))
(when gptel-max-tokens
(plist-put prompts-plist :max_tokens gptel-max-tokens))
- prompts-plist))
+ ;; Merge request params with model and backend params.
+ (gptel--merge-plists
+ prompts-plist
+ (gptel-backend-request-params gptel-backend)
+ (gptel--model-request-params gptel-model))))
;;;###autoload
(cl-defun gptel-make-privategpt
- (name &key curl-args stream key
+ (name &key curl-args stream key request-params
(header
(lambda () (when-let (key (gptel--get-api-key))
`(("Authorization" . ,(concat "Bearer " key))))))
@@ -137,7 +141,12 @@ KEY is a variable whose value is the API key, or function
that
returns the key.
CONTEXT and SOURCES: if true (the default), use available context
-and provide sources used by the model to generate the response."
+and provide sources used by the model to generate the response.
+
+REQUEST-PARAMS (optional) is a plist of additional HTTP request
+parameters (as plist keys) and values supported by the API. Use
+these to set parameters that gptel does not provide user options
+for."
(declare (indent 1))
(let ((backend (gptel--make-privategpt
:curl-args curl-args
@@ -149,6 +158,7 @@ and provide sources used by the model to generate the
response."
:protocol protocol
:endpoint endpoint
:stream stream
+ :request-params request-params
:url (if protocol
(concat protocol "://" host endpoint)
(concat host endpoint))
diff --git a/gptel.el b/gptel.el
index 2e9e3809c5..37fc8e8948 100644
--- a/gptel.el
+++ b/gptel.el
@@ -560,6 +560,9 @@ Keys:
- `:cutoff-date': the knowledge cutoff date.
+- `:request-params': a plist of additional request parameters to
+ include when using this model.
+
Information about the OpenAI models was obtained from the following
sources:
@@ -760,6 +763,19 @@ and \"apikey\" as USER."
(symbol s)
(string (intern s))))
+(defun gptel--merge-plists (&rest plists)
+ "Merge PLISTS, altering the first one.
+
+Later plists in the sequence take precedence over earlier ones."
+ (let (;; (rtn (copy-sequence (pop plists)))
+ (rtn (pop plists))
+ p v ls)
+ (while plists
+ (setq ls (pop plists))
+ (while ls
+ (setq p (pop ls) v (pop ls))
+ (setq rtn (plist-put rtn p v))))
+ rtn))
(defun gptel-auto-scroll ()
"Scroll window if LLM response continues below viewport.
@@ -870,6 +886,10 @@ in any way.")
(car-safe (member mime (gptel--model-mimes
(or model gptel-model)))))
+(defsubst gptel--model-request-params (model)
+ "Get model-specific request parameters for MODEL."
+ (get model :request-params))
+
;;;; File handling
(defun gptel--base64-encode (file)
"Encode FILE as a base64 string.
| [Prev in Thread] |
Current Thread |
[Next in Thread] |
- [nongnu] elpa/gptel bcbbe67e08: gptel: Add arbitrary params to request,
ELPA Syncer <=