bug#25618: 25.1.91; More precise limit for tagging expensive tests

[Michael Albinus]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Tino Calancha <tino.calancha@gmail.com>
>> Date: Sat, 04 Feb 2017 22:42:44 +0900
>> 
>> 
>> Prefer set a numeric limit of seconds than using the word 'few'.
>> 
>> For instance, in my box the test `cl-seq-test-bug24264' in
>> test/lisp/emacs-lisp/cl-seq-tests.el
>> takes 7 s.
>> With an explicit numeric threshold is easier to decide whether this test
>> must be tagged as expensive or not.
>> 
>> In following patch i define a few as 2-3 seconds.  Other people might
>> prefer 4-5 or even higher.
>> Please, suggest a proper value to consider a test as expensive.
>
> I think 2-3 sec is negligible, and 7 sec is not long enough to annoy.
> 10 sec or more is beginning to sound like it comes close to "too
> long", and 20 sec is definitely too long.

Unfortunately, there is no mean in ert to see how long a single test
runs. I often run expensive tests (well, I'm testing Tramp, filenotify,
autorevert :-) and it would help me to know the time for a single test
case run.

Maybe somebody could add it? I haven't the time just now; but if it
annoys me too much, I'll do it myself, later on, maybe.

This ticket might be a good place to remember this. Pls don't close it
after changing CONTRIBUTE.

Best regards, Michael.