face colors on 256 colors terminals

[Dan Nicolaescu]

On a 256 color terminal emacs faces should use very similar colors to
the ones used on X11. It turns out that the color used are a bit
lighter. 
Here's a screenshot of a modified M-x list-faces-display: 
http://www.ics.uci.edu/~dann/col.jpg
(the frame on the left is an xterm, the one on the left is X11)

Running the  function below  on a 256 color terminal shows that there
are some problems mapping the gray0->gray100 colors, they are not
mapped to linearly lighter shades of gray. Also the colors that have 2
of the RGB values 0 and the third 255 (red1, blue1, green1) are not
mapped correctly.

(defun my-list-colors-display (&optional list)
  "Modified `list-colors-display'."
  (interactive)
  (when (and (null list) (> (display-color-cells) 0))
    (setq list (mapcar 'car color-name-rgb-alist))
    (when (memq (display-visual-class) '(gray-scale pseudo-color direct-color))
      ;; Don't show more than what the display can handle.
      (let ((lc (nthcdr (1- (display-color-cells)) list)))
        (if lc
            (setcdr lc nil)))))
  (with-output-to-temp-buffer "*MYColors*"
    (save-excursion
      (set-buffer standard-output)
      (let (s)
        (while list
          (setq s (point))
          (insert (car list))
          (indent-to 20)
          (put-text-property s (point) 'face
                             (cons 'background-color (car list)))
          (setq s (point))
          (insert "  " (car list) 
                  (if  window-system "" 
                    (concat " ("(car  (tty-color-desc (car list))) ")") ) "\n")
          (put-text-property s (point) 'face
                             (cons 'foreground-color (car list)))
          (setq list (cdr list)))))))


I don't know if the code that maps the colors in
`color-name-rgb-alist' to terminal colors is correct, but I think
there are issues elsewhere in the code. 

First the values in term/xterm.el:xterm-standard-colors need updating,
this patch uses the values found in xterm-200 (the latest version):

*** xterm.el     26 Mar 2005 14:01:35 -0800             1.10
--- xterm.el     05 Apr 2005 22:56:14 -0700             
***************
*** 107,121 ****
      ("red"            1 (205   0   0))        ; red3
      ("green"          2 (  0 205   0))        ; green3
      ("yellow"         3 (205 205   0))        ; yellow3
!     ("blue"           4 (  0   0 205))        ; blue3
      ("magenta"        5 (205   0 205))        ; magenta3
      ("cyan"           6 (  0 205 205))        ; cyan3
      ("white"          7 (229 229 229))        ; gray90
!     ("brightblack"    8 ( 77  77  77))        ; gray30
      ("brightred"      9 (255   0   0))        ; red
      ("brightgreen"   10 (  0 255   0))        ; green
      ("brightyellow"  11 (255 255   0))        ; yellow
!     ("brightblue"    12 (  0   0 255))        ; blue
      ("brightmagenta" 13 (255   0 255))        ; magenta
      ("brightcyan"    14 (  0 255 255))        ; cyan
      ("brightwhite"   15 (255 255 255)))       ; white
--- 107,121 ----
      ("red"            1 (205   0   0))        ; red3
      ("green"          2 (  0 205   0))        ; green3
      ("yellow"         3 (205 205   0))        ; yellow3
!     ("blue"           4 (  0   0 238))        ; blue2
      ("magenta"        5 (205   0 205))        ; magenta3
      ("cyan"           6 (  0 205 205))        ; cyan3
      ("white"          7 (229 229 229))        ; gray90
!     ("brightblack"    8 (127 127 127))        ; gray50
      ("brightred"      9 (255   0   0))        ; red
      ("brightgreen"   10 (  0 255   0))        ; green
      ("brightyellow"  11 (255 255   0))        ; yellow
!     ("brightblue"    12 (92   92 255))        ; rgb:5c/5c/ff
      ("brightmagenta" 13 (255   0 255))        ; magenta
      ("brightcyan"    14 (  0 255 255))        ; cyan
      ("brightwhite"   15 (255 255 255)))       ; white


This comment in tty-colors.el:tty-color-standard-values 

         ;; Translate the string "#XXYYZZ" into a list
         ;; of numbers (XX YY ZZ).  If the primary colors
         ;; are specified with less than 4 hex digits,
         ;; the used digits represent the most significant
         ;; bits of the value (e.g. #XYZ = #X000Y000Z000).

does not seem to match the way the `color-name-rgb-alist' seem to have
been created from the values in rgb.txt. 
A random example:
>From color-name-rgb-alist:
    ("lavenderblush"    65535   61680   62965)
                        ^^^^    ^^^^    ^^^^
                        0xffff  0xf0f0  0xf5f5      

>From rgb.txt: lavender blush     255   240   245
                                 0xff  0xf0  0xf5
So the 8 to 16 bit conversion seems use the same byte value for both
the high and low byte value. 
To make the code in tty-colors.el consistent with this observation
something like this could be done: 

*** tty-colors.el       20 Aug 2004 14:28:41 -0700      1.14
--- tty-colors.el       06 Apr 2005 00:17:47 -0700      
***************
*** 931,945 ****
                  (i2 (+ i1 ndig))
                  (i3 (+ i2 ndig)))
             (list
!             (lsh
!              (string-to-number (substring color i1 i2) 16)
!              (* 4 (- 4 ndig)))
!             (lsh
               (string-to-number (substring color i2 i3) 16)
!              (* 4 (- 4 ndig)))
!             (lsh
               (string-to-number (substring color i3) 16)
!              (* 4 (- 4 ndig))))))
          ((and (>= len 9) ;; X-style RGB:xx/yy/zz color spec
                (string= (substring color 0 4) "rgb:"))
           ;; Translate the string "RGB:XX/YY/ZZ" into a list
--- 931,950 ----
                  (i2 (+ i1 ndig))
                  (i3 (+ i2 ndig)))
             (list
!             (logior (lsh
!                      (string-to-number (substring color i1 i2) 16)
!                      (* 4 (- 4 ndig)))
!                     (string-to-number (substring color i1 i2) 16))
!             (logior 
               (string-to-number (substring color i2 i3) 16)
!              (lsh
!              (string-to-number (substring color i2 i3) 16)
!              (* 4 (- 4 ndig))))
!             (logior 
!              (string-to-number (substring color i3) 16)
!              (lsh
               (string-to-number (substring color i3) 16)
!              (* 4 (- 4 ndig)))))))
          ((and (>= len 9) ;; X-style RGB:xx/yy/zz color spec
                (string= (substring color 0 4) "rgb:"))
           ;; Translate the string "RGB:XX/YY/ZZ" into a list

The formulas used to compute the 16bit RGB values for the different
terminal colors don't match anymore what is found in 256colres.pl file
from xterm-200.

Here is a possible fix (only for 256 colors terminals): 

*** xterm.el    26 Mar 2005 14:01:35 -0800      1.10
--- xterm.el    06 Apr 2005 00:11:06 -0700      
*** 123,129 ****
  
  (defun xterm-rgb-convert-to-16bit (prim)
    "Convert an 8-bit primary color value PRIM to a corresponding 16-bit value."
!   (min 65535 (round (* (/ prim 255.0) 65535.0))))
  
  (defun xterm-register-default-colors ()
    "Register the default set of colors for xterm or compatible emulator.
--- 123,129 ----
  
  (defun xterm-rgb-convert-to-16bit (prim)
    "Convert an 8-bit primary color value PRIM to a corresponding 16-bit value."
!   (logior prim (lsh prim 8)))
  
  (defun xterm-register-default-colors ()
    "Register the default set of colors for xterm or compatible emulator.
***************
*** 160,168 ****
            (tty-color-define (format "color-%d" (- 256 ncolors))
                              (- 256 ncolors)
                              (mapcar 'xterm-rgb-convert-to-16bit
!                                     (list (round (* r 42.5))
!                                           (round (* g 42.5))
!                                           (round (* b 42.5)))))
            (setq b (1+ b))
            (if (> b 5)
                (setq g (1+ g)
--- 160,169 ----
            (tty-color-define (format "color-%d" (- 256 ncolors))
                              (- 256 ncolors)
                              (mapcar 'xterm-rgb-convert-to-16bit
!                                     (list (if (zerop r) 0 (+ (* r 40) 55))
!                                           (if (zerop g) 0 (+ (* g 40) 55))
!                                           (if (zerop b) 0 (+ (* b 40) 55)))))
! 
            (setq b (1+ b))
            (if (> b 5)
                (setq g (1+ g)


With this fixes the gray colors are mapped linearly, and so are the
red1, blue1 and green1 colors (I checked the pixels in a screenshot).

Some standard face definitions use colors like "red" or "blue". They 
should be changed "red1" (or "blue1") because on X11 "red" is 255/0/0,
but in an xterm is 205/0/0. "red1" will be mapped correctly on both
X11 and an xterm. 

Here is a screenshot after these patches:
http://www.ics.uci.edu/~dann/col-after.jpg

It can be seen that the colors used for the standard faces seem a bit
less different between the 2 emacs frames (one X11 and one running in
and xterm).

If anyone has more insights about this, please share them.

Thanks!

                --dan