Re: [PATCH] Bug: fragile org refile cache

[Maxim Nikulin]

On 29/04/2021 23:08, Ihor Radchenko wrote:
> Maxim Nikulin writes:
>
> > Curiously my experience is that avoiding this lazy cache with
> > backtracking and maintaining custom structure during sequential scan of
> > the buffer works several times faster.
>
> My experience is exactly opposite. Or maybe I miss something. Can you
> elaborate?

My benchmarks may be incorrect due to for development version I did not 
compile files. I did not purged outline path cache as well.
https://orgmode.org/list/s1qola$158l$1@ciao.gmane.io/

> Outline path without cache:
>
> (benchmark-run 1
>    (goto-char (point-min))
>    (while (re-search-forward "^\\*+" nil t)
>      (org-get-outline-path t nil))) => (6.051079914 1 0.2864724879999869)
>
> Outline path with cache:
>
> (benchmark-run 1
>    (goto-char (point-min))
>    (while (re-search-forward "^\\*+" nil t)
>      (org-get-outline-path t nil))) => (1.658461165 0 0.0)
                               ^ t
I suppose.

I agree with such test.

Notice however the following patch (warning: :level and :max-level a 
cached with the same key)
https://orgmode.org/list/s209r8$16en$1@ciao.gmane.io/

Avoiding call to org-get-outline-path and using custom structure during 
single pass scan through the buffer allowed to significantly improve 
performance.

> Just cleanup heading text:
>
> (benchmark-run 1
>    (goto-char (point-min))
>    (while (re-search-forward "^\\*+" nil t)
>      (let ((case-fold-search nil))
>        (looking-at org-complex-heading-regexp)
>        (if (not (match-end 4)) ""
>         ;; Remove statistics cookies.
>         (org-trim
>          (org-link-display-format
>           (replace-regexp-in-string
>            "\\[[0-9]+%\\]\\|\\[[0-9]+/[0-9]+\\]" ""
>            (match-string-no-properties 4)))))))) => (0.013364877 0 0.0)

I may be wrong with the following statement. Attempt to profile 
org-refile-get-targets could give quite different results. I have seen a 
note that Emacs use internally a cache for only 5 compiled regular 
expressions. Just one extra regexp and every matching function require 
compiling of its regexp just wiped from the cache. It is a time 
consuming procedure. I am unsure whether you added all regexps used 
(directly or through function calls) by inner loop of 
org-refile-get-targets.