Re: burden of maintainance

[joakim]

Eli Zaretskii <address@hidden> writes:

>> From: Tassilo Horn <address@hidden>
>> Cc: Przemysław Wojnowski <address@hidden>,
>>   address@hidden,  address@hidden
>> Date: Fri, 02 Oct 2015 19:18:08 +0200
>> 
>> Eli Zaretskii <address@hidden> writes:
>> 
>> >> Can you give an example? Of course not everything can be tested, but
>> >> many things can.
>> >
>> > We are mis-communicating again: I meant features which display
>> > something or involve interactive keyboard input.
>> 
>> Emacs is in a better position here than most other tools because it has
>> keyboard macros and can easily inspect the things it displays, e.g., the
>> text properties or overlays at point.
>
> We may be mis-communicating here.  Because I cannot see how examining
> text properties or overlays can test whether their display on screen
> is correct.  What am I missing?  Can you show an example of what you
> had in mind?

As a data point, there is an interesting GUI test toolkit called Sikuli,
It uses the OpenCV computer vision toolkit to analyze that the display
matches expectations. Sikuli is free software if I recall correctly.


>
> If we are talking about testing the display engine, it should be
> possible and relatively easy to provide Lisp access to the "glyph
> matrices", which are data structures the display engine builds that
> describe what _should_ be on the screen.  You can see one specialized
> example of that in the function bidi-resolved-levels which I wrote for
> testing the bidirectional display.
>
> But doing the above is only half the way.  The other half is the
> terminal-specific back-end, either X, w32, NS, or TTY, which actually
> converts the glyph matrices to draw commands and delivers the results
> to the glass.  To test that, we need some way of "sniffing" the
> results of this drawing, which requires terminal-specific
> infrastructure.
>
> We then need some "language" to describe the expected results to be
> displayed on the screen.  For TTY, this is almost trivial, but for
> sophisticated GUI display features it's more complicated (think of
> fonts, stretches of white space, composed characters, fringe bitmaps,
> tool-bar icons, tooltips, etc.).  We also need a way of describing the
> results of cursor movement.
>
> When all of these are implemented, then yes, writing display tests
> should be relatively easy.  But today all this infrastructure doesn't
> exist.  We don't even have its detailed design.
>
>> I can think of something like "ERT keyboard-macro tests" where you would
>> basically record a keyboard macro and during that, you have some way to
>> add some assertions before resuming recording.  The whole recorded thing
>> could then be stored as an ERT test.
>
> For testing keyboard input, we need more than that.  For example,
> consider the simple feature of input history -- you'd need a way to
> inject M-p, M-n, etc., and a way of recording the text that is
> displayed as result and returned to the caller.
>
> And please don't forget that keyboard input is only one kind of events
> supported by Emacs.  There are others, starting with mouse.  We'd need
> ways to simulate or trigger all of them.
>
>

-- 
Joakim Verona