Re: UTF-8 in grout and a performance regression (was: synchronous and asynchronous grout)

[onf]

Hi Branden,

On Fri Dec 20, 2024 at 3:45 AM CET, G. Branden Robinson wrote:
> At 2024-12-19T22:48:41+0100, onf wrote:
> > > The number of people who read GNU troff output ("grout"), whether
> > > with their eyeballs or with a program they've written, is *tiny*.
> > 
> > If that's the case, I wonder why you're concerned about a tiny
> > fraction of a tiny fraction of people not being able to display those
> > characters..?
>
> Because I think a lot of the occurrences of someone staring at grout are
> going to come from people attempting to troubleshoot problems.  The very
> first time they see the output format may be when they are in a
> frustrating situation.  Under such circumstances, a representation
> format that will work even over a serial line to a bad DEC VT100
> emulator is a good thing to have.  Rendering a blue 🎈 and a brown 💩 is
> not a high priority.

Fair enough.

> [...]
> > > I think a lot of people, even when troubleshooting, fail to consider
> > > grout as an inspection site in the first place.  They try to reason
> > > from groff input to whatever is emitted by the output driver.  And
> > > that works, often enough.
> > 
> > That's true, but it can come in handy. I did examine the intermediate
> > output when trying to understand .ne's behavior, for example.
>
> Yes.  I think we could do more to encourage people to understand the
> availability of the grout format as a resource on more occasions, in a
> similar way that Dave Kemper finally got the value of "groff -a" output
> through my thick skull.

Does it have any other use nowadays besides regression testing?

> [...]
> > It wasn't my intention to make you appear argumentative, I simply
> > failed to understand part of what you were trying to say. I feel like
> > you are ascribing unreasonable amount of animosity to me.
>
> Would you settle for a _reasonable_ amount?  ;-)
>
> When someone is simultaneously articulate, seems knowledgeable, _and_
> doesn't seem to be making sense, I admit I tend to get suspicious.  But
> I'm also quick to revise my opinion given further evidence.  And also
> quick to forgive errors.  I'd better be--I make mistakes _all_ _the_
> _time_, and strive to own up to them, in emails, in commit
> messages, and in ChangeLogs.  Humans are fallible creatures.  That's why
> we should make our machines robust.

I think that "knowledgeable" part might not be based entirely in
reality... :)

> > I just felt like the argument that emitting \[u...] is preferrable due
> > to some tiny number of people who don't have Unicode-compliant
> > terminals was fairly weak, and it would be cool if the output was more
> > readable than this:
> >   tv
> >   Cu0065_030C
> >   h5444
> >   tt
> >   Cvs
> >   h4313
> >   C'i
> > 
> > (that's "větší")
>
> I agree.  For situations like that, UTF-8-empowered grout would be an
> advantage.
>
> But *roff is not Markdown.  If the main thing the formatter was doing
> was streaming an array of character codes from one place to another, we
> wouldn't need most of the features we have.
>
> My assessment, based on some experience troubleshooting and debugging
> groff documents, is that accurate glyph selection in formatted text just
> is not something we screw up very often.

The reason I would like to see it is not necessarily to aid in
troubleshooting glyph selection, but in navigating the output.
When I have several pages of text and there's a problem near one
of the words, being able to find that word easily would be handy.
In fact I took the example above from just such a document and
locating the word I saw in PDF output in grout took me way much
longer than it should have.

> [...]
> > By the way, when you said that some terminals' fonts might be missing
> > the relevant glyphs... I have the opposite problem, my groff fonts
> > support only a fraction of what my terminal can display.
>
> They're both problems.  Not everybody selects a terminal emulator font
> with generous coverage, and I think there are still severe limitations
> on the Linux console device.

True.

> Not having good coverage font-wise is, largely, the install-font.sh
> problem.
>
> https://savannah.gnu.org/bugs/?58831
>
> I've been jammed up on wanting to rewrite (or supplement) that tool in a
> form that will be practical for distributions to use, but since I never
> get around to it, I now think the better thing to do is ship it, albeit
> not in /usr/bin.  But in a place where it will get packaged.  [...]

I've been using afmtodit to install fonts; it seemed to work better than
install-font.sh the last time I tried. (By the way, can anyone explain to
me the rationale behind giving scripts filetype suffixes? I thought the
standard way is to give them a shebang and make them executable...)
I thought the reason for not having good Unicode coverage was simply that
either the font never had it in the first place, or PostScript fonts did
not support it well enough.

~ onf