bug#16800: 24.3; flyspell works slow on very short words at the end of big file

[Aleksey Cherepanov]

On Fri, Feb 21, 2014 at 05:12:54PM +0200, Eli Zaretskii wrote:
> > Date: Fri, 21 Feb 2014 15:38:55 +0100
> > From: Agustin Martin <agustin.martin@hispalinux.es>
> > Cc: 16800@debbugs.gnu.org
> > 
> > On Wed, Feb 19, 2014 at 12:56:45AM +0400, Aleksey Cherepanov wrote:
> > > flyspell-duplicate-distance variable on its own could mitigate the
> > > problem but it changes the behaviour so I do not want to use this  
> > > variable.
> 
> What behavior does it change?  Do you really care to have a
> mis-spelled word be highlighted in a different face just because
> there's an identical mis-spelling half a megabyte away?

Yes, as a user I really care about this and as a programmer I believe
the bug could be solved well.

Also GNU coding standards say to avoid arbitrary limits (parts 2.1
and 4.2).
http://www.gnu.org/prep/standards/standards.html

But I could accept that this bug has low severity because there is
flyspell-duplicate-distance variable that could be used as a
workaround.

> > For the records, I was playing with a customized value of 50000 for that
> > distance and even if there is still a minor delay it is reasonable. I am
> > in a fast box, do not know in other boxes.
> 
> I would suggest to change the default to something finite, like 20000
> perhaps.  Having it set to -1 by default is IMO unwieldy, since
> buffers can be very large.
> 
> > > I tried to patch flyspell-word-search-backward and
> > > flyspell-word-search-forward functions from flyspell.el replacing
> > > search-backward with word-search-backward and search-forward with
> > > word-search-forward (perl -pe 's/\(search-/(word-search-/' ). It
> > > solved the problem but I do not know what it broke.
> 
> And this doesn't change behavior?  See below.

No, it seems that my setup works the same. See below.

> > > I expect problems with this solution because I do not know if
> > > flyspell's meaning of word is the same as emacs' one. I think it is
> > > described in flyspell-get-word function that is called after search-*
> > > in the patched functions.
> > 
> > I have never played with Emacs syntax tables, but  I'd expect differences
> > only if there is a mismatch between chars in OTHERCHARS and non
> > alphabetic chars that Emacs considers as possible parts of a word. 
> 
> The effect depends on the language, I think.

If I'd believe that it is a right solution I'd send a patch.

The difference is in word bounds. We are in trouble if flyspell's word
on its ends does not have ends of emacs' word. If flyspell's word has
ends of emacs' word on its ends and even contain them inside then we
are ok (try to search "a b" over "aa bb a b aa bb"). So could ends of
flyspell's word do not match with ends of emacs' word?

So I think word search instead of regular search does not change
behaviour of my setup with EN and RU languages (both at the same time)
and excplicitly specified ispell-dictionary-alist (I used some popular
instructions as is to setup it so long time ago).
ispell-dictionary-alist contains character sets used by
flyspell-get-word.

As an alternative I think we could generate regexps on the fly with
flyspell's word boundaries around words and search them. It would be
like
(re-search-forward (concat "\\<" (regexp-quote word) "\\>") bound t)
instead of
(word-search-forward word bound t)
but with flyspell's word boundaries instead of "\\<" and "\\>".

Thanks!

-- 
Regards,
Aleksey Cherepanov