Re: [Orgmode] Re: Testing --- again...

[Eric Schulte]

Hi,

I've taken the liberty of merging our two testing branches into the new
"combined-testing" branch.  This now includes both ert and jump.el as
git submodules, which can be installed with

  git submodule init
  git submodule update

after checking out the branch.  In merging the two org-test.el files, I
was able to remove much of the existing code through using which-func,
jump.el.  The new navigation functionality will work regardless of the
directory structure, so tests for file

  lisp/org-foo.el

can be located in either of the following

  testing/lisp/test-org-foo.el
  testing/lisp/org-foo.el/test.el

Additionally the navigation functions defined in jump.el should be
easily extensible to accommodate new naming schemas, so ideally every
test author for a particular file can use whatever naming system they
prefer.  Note that the navigation function `org-test-jump' when called
with a prefix argument uses your code from
`org-test-edit-buffer-file-tests' to create the test file if it is not
already present.

I think I retained most of the functionality from your version of
org-test.el in the merge but please let me know if I broke something.
In my mind this merged version should be small clean and maintainable
and hopefully flexible enough to handle whatever needs we identify
during the course of writing the tests.  How would you feel about using
this as the new base of test development?

Best -- Eric

also I have some inline comments below

Sebastian Rose <address@hidden> writes:

> "Eric Schulte" <address@hidden> writes:
>> "Eric Schulte" <address@hidden> writes:
>> To illustrate my suggestions, I've thrown together a couple simple Babel
>> tests roughly following this outline, currently up in the
>> `schulte-testing' branch of the Org-mode repo.
>>
>> To try it out
>> 1) load the testing/org-test.el file
>> 2) run `org-load-tests' to load up the entire org-mode test suite
>> 3) run `ert' to run the test suite.
>> 4) or jump to the definition of `org-babel-get-src-block-info' and run
>>    `org-test-current-function' to just run the tests for that function
>>
>> Best -- Eric
>
> Yes.  I know how running tests looks like:  It looks good :)
>
> Hahaa, I like that code.  It looks so simple :)
>
>
> To load all files below testing/  is a good suggestion probably.  The
> unloading and loading as I implemented it is simply superfluous, if we
> suggest that testers no how to eval a buffer or defun (and if they can
> change it, they can eval it).
>
>
>
> * Question:
>
>   (defun org-test-current-function ()
>     "Test the current function."
>     (interactive)
>     (ert (car (which-function))))
>   
>   `org-test-test-current-defun' in the ert-testing branch does that, too.
>   (But with that superfluous loading/unloading stuff though :-/) But it
>   encloses each test inside save-excursion and Co.
>   

I believe ert takes care of the save-excursion stuff

>   
>   What does which-func.el that this function does not:
>   
>   <#part type="application/emacs-lisp" disposition=inline>
>   (defun org-test-which-func ()
>     "Return the name of the current defun."
>     (save-excursion
>       (save-match-data
>         (end-of-line)
>         (beginning-of-defun)
>         (if (looking-at "(defun[[:space:]]+\\([^([:space:]]*\\)[[:space:]]*(")
>           (match-string-no-properties 1)
>       (error "No defun found around point.")))))
>   <#/part>
>   
>   ??
>

I'm not sure that it does include anything new, but it's nice to re-use
existing packages.

>
>
>
> * Keymap
>
>   We should add keys to the org-mode-map.
>   
>    C-c t
>   
>   is still free here.  Or is it in you use it for babel somehow?
>   
>    C-c t f     org-test-test-current-function
>    C-c t b     org-test-test-current-buffer-file
>
>

Wouldn't we want the keys in the elisp-mode key map, since we'll be
doing the testing work from elisp?  Although I guess we may want to run
tests from within Org-mode files, but then we could do that with a Babel
block and dump the results to a table. :)

>
>
>
> * ERT Selectors
>
>   I see a little namespace problem coming up.  Imagine testing org.el.
>   Which ert selector would we use?
>   
>   "^org" ???  :-/
>
>   Should we create hashes of filenames as selectors (just kidding)?
>   Or use the entire filename "^org.el"?  The relative path
>   "^lisp/org.el"? 
>

oh, good point, maybe we'd need to use the "eql" or "tag" ert selectors
in this case.

>
>
>
> * About the directory structure:
>
>   It does not burden the user, as tests are loaded and executed
>   automatically (per function, per file or all, depending on the command
>   used).  But imagine the entire thing grows and someone would add tests
>   for all the stuff in emacs/lisp/.  Or add tests files for single
>   functions simply because there is so much to test that one file would
>   be hard to handle.
>   This could cause a lot of clutter.
>
>   Is this too hypothetical?  Hmmm - it might be...
>
>   If you checkout ert-testing, eval testing/org-test.el and do
>
>      M-x org-test-edit-buffer-file-tests
>

In the combined-testing branch you'd do this by calling org-test-jump
with a prefix argument.

>
>   you're in your test file in the correct directory (which is created
>   if it doesn't exist).
>
>      M-x org-test-edit-current-defuns-tests
>
>   creates a file named after the defun you're in.  You'll need to be in
>   an elisp file of course.  The testing/ directory will resemble the
>   directory structure of the project.
>
>   Every one who saw that directory structure simply asked "don't you
>   think it's overkill?" :D  I'll probably drop that.
>

heh, yea I'd lean towards getting into the writing of tests and then
leaving our implementation flexible enough so that we can adopt this
more fine-grained directory structure when/if it becomes necessary.

>
>   
>
>   We could go on on two rails for a while.  I'll need this week to dig
>   deeper into ERT and come to conclusions.
>
>   It would be a good thing if we could agree about the keys and the
>   selectors (again: "^org"), so that it's painless to switch branches ;)
>
>
>
>
>
>
>   Sebastian