[Orgmode] Testing --- again...

[Sebastian Rose]

Hi,


I thought about testing again recently.  This is something, that never
really got started.  For a reason:  there's no framework for testing.

I therefore wrote a very rough proposal,  found  on
http://github.com/SebastianRose/org-test

The idea is, to provide two simple commands:


  *  org-test-test-current-defun
     will search for tests for the defun point is in or behind
     (`beginning-of-defun') and execute them surrounded by

  (let ((select (or selector "^org"))
        (deactivate-mark nil))
    (save-excursion
      (save-match-data


  *  org-test-test-buffer-file
     will search for tests for the entire file and execute them the same
     way.

If you use one of these commands, all currently registered ERT tests are
deleted, and files are reloaded (since you're likely to work on the
tests, too).  To repeat the tests without reloading, you will use the
ERT commands like `ert-results-rerun-all-tests', bound to `r' in the ERT
results buffer.



I choose ERT (git clone http://github.com/ohler/ert.git) because that's
likely to go into Emacs core (or elpa.gnu.org).




The idea is to search the directory structure from the current source
file upwards for a directory named "tests/" if it exists.  Else ask the
user.  Similar to what `add-change-log-entry' does.

Below that directory, a tree like the source tree exists:

project
   +-- lisp/
   |     +-- a.el
   |     `-- b/
   |         +-- b.el
   |
   `-- tests/
         +-- a.el/
         |     +-- tests.el
         |     `-- a-defun.el
         `-- b/
             +-- b.el/
                   +-- tests.el
                   `-- b-defun.el

If this setup exists, when editing defun-x in lisp/a.el,
`M-x org-test-test-current-defun' will load tests/a.el/defun-x.el
(fallback: tests.el there) and execute all tests with selector
"^a-defun".
`M-x org-test-test-buffer-file' in that same source file will load all
*.el files in tests/a.el/ and execute all ERT tests for selector "^a".


Thus tests for
    org-mode/lisp/org-protocol.el
will be searched in the directory
    org-mode/tests/lisp/org-protocol.el/*.el


Once the basic route of testing is clear, I'd like to "translate" the
existing tests for org-html.el to work with ERT, which will involve
writing more tools (create output buffers, compare output with control
files using ediff etc.).  I know Lennart Borgman has wrote that stuff
for nXhtml already.  I hope we can use his stuff and help here.

The directory org-mode/lisp/tests/ would not need to be part of the
"official" Org mode package.  It could as well be checked out
separately, if "tests" is part of org-mode/lisp/.gitignore (e.g.).


Any thoughts?




  Sebastian