[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[elpa] externals/elisa a4f34972bb 1/7: Improve embeddings calculation in
From: |
ELPA Syncer |
Subject: |
[elpa] externals/elisa a4f34972bb 1/7: Improve embeddings calculation in Elisa |
Date: |
Sat, 23 Nov 2024 12:57:56 -0500 (EST) |
branch: externals/elisa
commit a4f34972bb0af16b0fb5e62fa8f00f11b0b3f0b1
Author: Sergey Kostyaev <kostyaev.sergey2@wb.ru>
Commit: Sergey Kostyaev <kostyaev.sergey2@wb.ru>
Improve embeddings calculation in Elisa
Added `elisa-embeddings` function for calculating embeddings for
chunks. If the LLM provider supports batch embeddings, use them;
otherwise, calculate embeddings individually using `llm-embedding`.
---
elisa.el | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/elisa.el b/elisa.el
index 5a1c4ab725..65ded645bf 100644
--- a/elisa.el
+++ b/elisa.el
@@ -455,6 +455,14 @@ FOREIGN KEY(collection_id) REFERENCES collections(rowid)
"Calculate breakpoint threshold for DISTANCES based on K standard
deviations."
(+ (elisa-avg distances) (* k (elisa-std-dev distances))))
+(defun elisa-embeddings (chunks)
+ "Calculate embeddings for CHUNKS.
+Return list of vectors."
+ (let ((provider elisa-embeddings-provider))
+ (if (member 'embeddings-batch (llm-capabilities provider))
+ (llm-batch-embeddings provider chunks)
+ (mapcar (lambda (chunk) (llm-embedding provider chunk)) chunks))))
+
(defun elisa-parse-info-manual (name collection-name)
"Parse info manual with NAME and save index to COLLECTION-NAME."
(with-temp-buffer
- [elpa] externals/elisa updated (4a4d1db359 -> 1c1e0f1715), ELPA Syncer, 2024/11/23
- [elpa] externals/elisa 5d1c1c0f0b 3/7: Fix elisp-check errors, ELPA Syncer, 2024/11/23
- [elpa] externals/elisa 1c1e0f1715 7/7: Bump version, ELPA Syncer, 2024/11/23
- [elpa] externals/elisa 9273c84961 4/7: Update required llm dependency, ELPA Syncer, 2024/11/23
- [elpa] externals/elisa 24b0ba8f2e 6/7: Merge pull request #28 from s-kostyaev/add-batch-embeddings-calculation, ELPA Syncer, 2024/11/23
- [elpa] externals/elisa a4f34972bb 1/7: Improve embeddings calculation in Elisa,
ELPA Syncer <=
- [elpa] externals/elisa a22b96e601 2/7: Speed up semantic splitting using batch embeddings, ELPA Syncer, 2024/11/23
- [elpa] externals/elisa 860936af49 5/7: Disable batch embeddings by default, ELPA Syncer, 2024/11/23