Re: [BUG] Fill-paragraph issue with hidden emphasis markers and composed characters

[Tim Cross]

I was able to reproduce this using Emacs 28.0.50 and emacs -Q.

It is important to ensure both of the below paragraphs are completely
'unwrapped' i.e. all on one line and then wrap them with M-q. 

The result seems counter-intuitive to me - the second paragraph with an
extra character is wrapped correctly. The first paragraph has the first
line broken too early.

This is likely going to be a very difficult issue to fix. I'm not sure
how fill column mode is aware of add on features like prettify symbols
and hidden emphasis markers. Obviously it is aware at some level because
it does wrap the second paragraph correctly. My wild guess would be that
the second one works because the additional character breaks up the
'special' handling and allows fill column to correctly calculate the
'virtual' line length.

Lucas Viana <lucasvianareis@gmail.com> writes:

> Dear list, 
>
> I'm not sure if this is a general issue with fill-paragraph or with the way 
> org hides emphasis markers, but I'm facing a problem with bare emacs +
> org.
>
> Steps to reproduce:
> 1. Start emacs -Q (no configuration)
> 2. Set some variables:
>
> (setq org-pretty-entities t
>       org-hide-emphasis-markers t
>       fill-column 80)
>
> 3. Create an empty org-mode buffer with the following paragraph:
> ""
> This is a /italic sentence for a buggy/ \alpha\beta\gamma\delta\nu\epsilon
> paragraph in terms of filling, see, it's not filled yet and still the line 
> above
> ends early.
> ""
>
> So far so good, and with the previous config the emphasis markers are now 
> hidden and the greek letters appear as unicode characters.
>
> 4. Hit M-q (either fill-paragraph or org-fill-paragraph, both have the same 
> issue)
> The first line wrongly wraps after "buggy" and leaves a big empty space after 
> that, as if the greek letters were not composed.
>
> 5. In the same buffer, compare the behaviour with this paragraph (very 
> similar, but note the ! character after "buggy"):
> ""
> This is a /italic sentence for a buggy/ ! \alpha\beta\gamma\delta\nu\epsilon
> paragraph in terms of filling, see, it's not filled yet and still the line 
> above
> ends early.
> ""
>
> When I fill this slightly different paragraph with M-q, it wraps the text in 
> the right way and the first line is completely filled.
>
> So the issue happens when a hidden emphasis marker is immediately followed 
> (maybe after a space) by a composed character like \alpha. I am also
> facing the same issue when the characters are composed by 
> prettify-symbols-mode instead of org-pretty-entities, and also with other 
> types of
> hidden emphasis markers. If org-hide-emphasis-markers is disabled then the 
> issue doesn't happen.
>
> Thanks in advance!
>
> Emacs  : GNU Emacs 28.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version 
> 3.24.29, cairo version 1.17.4)
>  of 2021-07-09
> Package: Org mode version 9.5 (9.5-b83ae59 @ 
> /home/lucas/.emacs.d/.local/straight/build-28.0.50/org/)