[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[elpa] externals/elisa 9ad7827337 70/98: Fix semantic split with single
From: |
ELPA Syncer |
Subject: |
[elpa] externals/elisa 9ad7827337 70/98: Fix semantic split with single chunk |
Date: |
Wed, 17 Jul 2024 18:58:05 -0400 (EDT) |
branch: externals/elisa
commit 9ad7827337cb9e3dc326c47e4882f45ea5231da7
Author: Sergey Kostyaev <kostyaev.sergey2@wb.ru>
Commit: Sergey Kostyaev <kostyaev.sergey2@wb.ru>
Fix semantic split with single chunk
---
elisa.el | 61 +++++++++++++++++++++++++++++++------------------------------
1 file changed, 31 insertions(+), 30 deletions(-)
diff --git a/elisa.el b/elisa.el
index 00f93a465c..72fe45f604 100644
--- a/elisa.el
+++ b/elisa.el
@@ -589,36 +589,37 @@ ARGS contains keys for fine control.
:threshold-amount K -- K is a breakpoint threshold amount.
than T, it will be packed into single semantic chunk."
- (when-let* ((func (or (plist-get args :function)
elisa-semantic-split-function))
- (k (or (plist-get args :threshold-amount)
elisa-breakpoint-threshold-amount))
- (chunks (funcall func))
- (embeddings (cl-remove-if
- #'not
- (mapcar (lambda (s)
- (when (length> (string-trim s) 0)
- (llm-embedding elisa-embeddings-provider
s)))
- chunks)))
- (distances (elisa--distances embeddings))
- (threshold (elisa-calculate-threshold k distances))
- (current (car chunks))
- (tail (cdr chunks)))
- (let* ((result nil))
- (mapc
- (lambda (el)
- (if (<= el threshold)
- (setq current (concat current (car tail)))
- (push current result)
- (setq current (car tail)))
- (setq tail (cdr tail)))
- distances)
- (push current result)
- (cl-remove-if
- #'string-empty-p
- (mapcar (lambda (s)
- (if s
- (string-trim s)
- ""))
- (nreverse result))))))
+ (if-let* ((func (or (plist-get args :function)
elisa-semantic-split-function))
+ (k (or (plist-get args :threshold-amount)
elisa-breakpoint-threshold-amount))
+ (chunks (funcall func))
+ (embeddings (cl-remove-if
+ #'not
+ (mapcar (lambda (s)
+ (when (length> (string-trim s) 0)
+ (llm-embedding elisa-embeddings-provider
s)))
+ chunks)))
+ (distances (elisa--distances embeddings))
+ (threshold (elisa-calculate-threshold k distances))
+ (current (car chunks))
+ (tail (cdr chunks)))
+ (let* ((result nil))
+ (mapc
+ (lambda (el)
+ (if (<= el threshold)
+ (setq current (concat current (car tail)))
+ (push current result)
+ (setq current (car tail)))
+ (setq tail (cdr tail)))
+ distances)
+ (push current result)
+ (cl-remove-if
+ #'string-empty-p
+ (mapcar (lambda (s)
+ (if s
+ (string-trim s)
+ ""))
+ (nreverse result))))
+ (list (buffer-substring-no-properties (point-min) (point-max)))))
(defun elisa--gitignore-to-elisp-regexp (pattern)
"Convert a .gitignore PATTERN to an Emacs Lisp regexp."
- [elpa] externals/elisa 273a1d492d 50/98: Add reranker to RAG pipeline, (continued)
- [elpa] externals/elisa 273a1d492d 50/98: Add reranker to RAG pipeline, ELPA Syncer, 2024/07/17
- [elpa] externals/elisa ade7ac0af9 52/98: Update info manual parsing, ELPA Syncer, 2024/07/17
- [elpa] externals/elisa 8a2c92dc34 54/98: Fix parsing info manuals, ELPA Syncer, 2024/07/17
- [elpa] externals/elisa cecc5cb13f 55/98: Make sync parsing interactive, ELPA Syncer, 2024/07/17
- [elpa] externals/elisa 4cad3085fd 57/98: Use more async calls to prevent emacs from blocking, ELPA Syncer, 2024/07/17
- [elpa] externals/elisa ad130b564f 60/98: Add parse file function, ELPA Syncer, 2024/07/17
- [elpa] externals/elisa b419fb2cf2 61/98: Add code for parsing directory as an elisa collection, ELPA Syncer, 2024/07/17
- [elpa] externals/elisa ef06534f46 62/98: Implement incremental parsing, ELPA Syncer, 2024/07/17
- [elpa] externals/elisa 0e32d7bb5c 63/98: Add async directory parsing, ELPA Syncer, 2024/07/17
- [elpa] externals/elisa f744ce305a 67/98: Add reparse current collection command, ELPA Syncer, 2024/07/17
- [elpa] externals/elisa 9ad7827337 70/98: Fix semantic split with single chunk,
ELPA Syncer <=
- [elpa] externals/elisa 439ed1d4f8 76/98: Make executable customization simpler, ELPA Syncer, 2024/07/17
- [elpa] externals/elisa e5691f59c5 80/98: Make syncronous functions non-interactive, ELPA Syncer, 2024/07/17
- [elpa] externals/elisa e92628390b 82/98: Update example configuration, ELPA Syncer, 2024/07/17
- [elpa] externals/elisa 39915439a4 84/98: Update installation instructions, ELPA Syncer, 2024/07/17
- [elpa] externals/elisa fbfe3b4ae1 86/98: Merge pull request #12 from s-kostyaev/semantic-split, ELPA Syncer, 2024/07/17
- [elpa] externals/elisa 3882b9b322 87/98: Bump version, ELPA Syncer, 2024/07/17
- [elpa] externals/elisa 04d580f072 92/98: add vector- and vss-path to injected variables on async, ELPA Syncer, 2024/07/17
- [elpa] externals/elisa 1acc89545d 31/98: Merge pull request #13 from s-kostyaev/fix-builtin-manuals-parsing, ELPA Syncer, 2024/07/17
- [elpa] externals/elisa 3eff22d4b6 53/98: Use new railways for info manuals, ELPA Syncer, 2024/07/17
- [elpa] externals/elisa 503083c0fb 58/98: Truncate long lines in done message, ELPA Syncer, 2024/07/17