Re: [PATCH] Highlight ANSI sequences in the whole buffer (was [PATCH] ANSI color on example blocks and fixed width elements)

[Nathaniel Nicandro]

Ihor Radchenko <yantar92@posteo.net> writes:

Hello, attached is another updated patch with the following changes:

- Made it possible to add headlines or inline tasks
  to `org-ansi-highlightable-elements', these are added by default now.

- To tackle the issue discussed previously about highlights spanning
  multiple lines (or elements) being removed when a line is modified I
  went ahead and used the font-lock-multiline property (see
  font-lock-extend-region-multiline and
  font-lock-extend-region-functions) across those regions so that on
  any edit of one of the lines, the region including all of the ANSI
  sequences that affect that line will be re-fontified.  This was the
  easier solution, but the downside is that it can cause large regions
  to be re-fontified when really all we want to do is apply the
  highlighting face to a small line change, for example.  An
  alternative solution would, when no ANSI sequences are being edited
  in the region being fontified and assuming a previous fontification
  cycle has applied highlights due to ANSI sequences already, only
  apply the highlighting face to the edited region instead of
  expanding the region before fontification.  The expansion
  unnecessarily wastes the fontification cycle on a region larger than
  what it needs to be since the information needed for highlighting
  the region according to ANSI sequences has already been computed on
  a previous fontification cycle.  In practice I don't think this
  inefficiency will matter much since I would assume most of these
  ANSI sequences will be inserted due to the results of code block
  execution or inserted by users who want to highlight small regions
  of the document so I would consider this problem solved by using
  font-lock-multiline for the time being.  WDYT?

- To tackle the issue of editing around the invisible ANSI sequences I
  left it up to the font-lock process to catch the invisible edits.
  Whenever an edit deletes a character of the sequence that renders
  the sequence invalid, the font-lock process will reveal the partial
  sequence.  But I had to limit what was considered a valid ANSI
  sequence to get it working in a somewhat acceptable way.

  The problem that I found was that if the buffer contains something
  like
  
  ^[[43mfoo
  
  (where ^[ is the ESC character and can be inserted with "C-q ESC" and
  the whole sequence ^[[43m is the ANSI sequence) what was happening was
  that deleting into the hidden sequence would leave the region in the
  state
  
  ^[[43foo
  
  and because the end byte of the ANSI sequence can be any character
  in the ASCII range [@A-Z[\]^_`a–z{|}~], ^[[43f would still be a
  valid ANSI sequence and would be hidden during the fontification
  process after the edit.  Since `ansi-color-apply-on-region' only
  really handles the sequences that end in an m byte, just rendering
  all other ones invisible, I limited the ANSI sequences handled by
  this patch to be only those sequences that end in m.  This way,
  after deleting into the sequence like in the above example the
  fontification process would not recognize the region as containing
  any sequence.  The downside to this solution is that sequences that
  end in any other end byte won't get conveniently hidden and the
  problem still persists if you have text that starts with an m and
  you delete into a hidden sequence.
  
  An alternative solution that doesn't constrain the end byte could be
  to add in some extra invisible character like a zero width space and
  then use something like the `modification-hooks' text property on
  the character to signify that a deletion at the boundary between the
  sequence and the text should really delete part of the sequence
  instead of the zero width space.  I haven't really worked out the
  details of this, for example how would it be detected which
  direction a deletion is coming from, the front or behind, but I'm
  throwing it out there to see if there are any other solutions other
  people might be aware of for a similar problem.
  
- Finally, code has been added to delete the overlays on the hidden
  sequences in `org-unfontify-region' so that multiple overlays are not
  created on re-fontifying regions containing those sequences.

Other than that, the code is the same as the last patch.

> P.S. I am not yet commenting on the details in the code.

Please let me know what you think of this patch and where I should be
focusing my efforts moving forward to get this submitted to Org.

One thing I would like to start doing is writing some tests for this
feature.  It would be great if someone could point me to some tests
that I can peruse so that I can get an idea of how I can go about
writing some of my own.  Also, are there any procedures or things I
should be aware of when trying to write my own tests?

0001-Highlight-ANSI-escape-sequences.patch
Description: Patch

-- 
Nathaniel