bug-gnu-emacs
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

bug#26710: Fwd: 25.2; project-find-regexp makes emacs use 100% cpu


From: Dmitry Gutov
Subject: bug#26710: Fwd: 25.2; project-find-regexp makes emacs use 100% cpu
Date: Mon, 1 May 2017 05:42:21 +0300
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:53.0) Gecko/20100101 Thunderbird/53.0

On 30.04.2017 21:47, Eli Zaretskii wrote:

I'll try to look at this.  According to my profiling, the lion's share
of time is taken by xref--collect-matches, so that's the place to try
concurrency.

I think that's too late. By the time xref--collect-matches is called (and it's called for each hit), we've already spent time synchronously waiting for the find-grep invocation to finish.

When there are a lot of matches, xref--collect-matches can take some significant time, with opening the buffers and everything. That can be optimized, however, as a separate issue, and I don't think there's anything to parallelize there, since it all happens in Elisp.

What we _can_ manage to run in parallel, in the find-grep process in the background, and the post-processing of the results in Elisp. Depending on how matches there are in total, compared to the project size (which affects how long find-grep will run), the second part will still affect the responsiveness of the UI to a smaller or larger degree, but ultimately there's no way around this AFAIK, as long as Elisp threads do not provide parallelism.

- Being able to hook up in an asynchronous fashion to that sequence in a
(second?) background thread, to render the search results in the xref
buffer as soon as they appear.

For the other thread to be background, it will need to yield from time
to time, otherwise the user experience will be identical to what we
have today, because an un-yielding thread will hold the execution unit
until it does its job completely, and no other thread gets to run
until it does.

Here's how I imagine it:

- Main thread creates a thread P which invoked the find-grep, somehow creates a "generator" object, yields to the main thread.

- The main thread creates the "other thread" which creates a buffer for displaying the xref items and marks it as still loading (e.g. with a spinner in the mode-line). And then repeatedly calls the generator for the next element. There are three choices:

1. An element is returned. Render it into the buffer.
2. No next element available. That should automatically yield from the current thread until it becomes available. That kind of logic should reside somewhere inside the generator, along with storing the reference to the current thread, to resume it later. 3. No more items, the process in P is finished. Mark the xref buffer as completed.

The things I'm not clear on are:

- How to best "convert" the process buffer into a generator object.

- Which generator type to use. Not sure if any of the ones we already have (generator.el, or iterator.el and stream.el in ELPA) will help.

- If the thread P is needed at all, or we could just one the main one instead of it.

- Whether we should forget all that concurrency nonsense (at least for this problem), and instead of a generator go with callbacks, similar to the API of the dir-status-files VC command. This way, each callback will render a new line, and the last one will mark the buffer as completed.





reply via email to

[Prev in Thread] Current Thread [Next in Thread]