Re: testing macros and fixtures

[Phillip Lord]

Artur Malabarba <address@hidden> writes:

> 2015-10-12 22:13 GMT+01:00 Phillip Lord
>
>> ERT is quite nice, but one of the things that I have found lacking is a
>> nice set of predicates, for use within should.
>>
>> So, when I wrote test for my "lentic" package I needed some functions
>> like, so that I could do things like:
>>
>> (should
>>   (test-eq-after-this
>>      "blah-before.txt"
>>      "blah-after.txt"
>>      (insert "hello")))
>>
>> which opens "blah-before.txt" runs (insert "hello") then compares the
>> result with "blah-after.txt".
>
> I think that's a pretty specific use-case. So it's best to let people write
> their own macros for this.
>
> (defmacro should-eq-after-body (file-before file-after &rest body)
>   `(let ((bef
>           (with-temp-buffer
>             (insert-file-contents file-before)
>             ,@body
>             (buffer-string))))
>      (should (string= bef (with-temp-buffer
>                             (insert-file-contents file-after)
>                             (buffer-string))))))

I'd imagine it's pretty common -- assuming that (insert "hello") is
anything at all of course.

>> My version of this also does a diff of the
>> results if the two are not equal.
>
> This certainly seems very useful. Maybe ert should do this on *all*
> multi-line string comparisons instead of doing its default
> “different-strings” report (which, most of the time, just says “strings are
> of different length”).

Yep. I use a function to replace "string=" that looks like this.

(defun lentic-test-equal-loudly (a b)
  "Actually, this just tests equality and shouts if not."
  ;; change this to t to disable noisy printout
  (if lentic-test-quiet
      (string= a b)
    (if (string= a b)
        t
      (message "Results:\n%s\n:Complete\nShouldbe:\n%s\nComplete:" 
cloned-results cloned-file)
      (let* ((a-buffer
              (generate-new-buffer "a"))
             (b-buffer
              (generate-new-buffer "b"))
             (a-file
              (make-temp-file
               (buffer-name a-buffer)))
             (b-file
              (make-temp-file
               (buffer-name b-buffer))))
        (with-current-buffer
            a-buffer
          (insert a)
          (write-file a-file))
        (with-current-buffer
            b-buffer
          (insert b)
          (write-file b-file))
        (message "diff:%senddiff:"
                 (with-temp-buffer
                   (call-process
                    "diff"
                    nil
                    (current-buffer)
                    nil
                    "-c"
                    a-file
                    b-file)
                   (buffer-string))))
      nil)))


An egregious hack, of course, but it was quick and does what I need.


> Yes, there are many packages that use custom-deisgned temp-buffers for
> testing. In fact, most non-trivial packages do. I'd really like to see ert
> offer a common interface for this. The difficulty for that is that each
> package has very different needs when it comes to testing in buffers, so I
> have no idea what this common interface could be.

I think I'll put this next into my todo list. Maintaining the current
(dreadful) infrastructure that I have for lentic is painful to say the
least, and I think a lot of this should be reusuable.

As you say, there are lots of examples.

Phil