Re: [Orgmode] How you can help

[Rick Moynihan]

Avdi Grimm wrote:
> A few points, from someone with a decent amount of testing background:
>
> * As someone who has contributed to OS projects, the lack of a
> pre-existing set of regression tests in org-mode is actually *the*
> most significant blocker to my getting involved in org-mode
> development.  I do all of my coding - both professional and personal -
> in the context of tests, and not having the existing framework means
> that before I can start working on new features I first need to spend
> time yak-shaving on testing infrastructure.
>
> *  I think there's a lot of over-thinking going on here.  Here's the
> test-first coding discipline in a nutshell:
>    1. Identify a problem/missing feature.
>    2. Write a test (possibly using a unit-testing framework to help)
> which will pass when the bug has been fixed or feature added. This can
> be as simple as calling a function and validating its return value.
>    3.  Run the test.  Verify it FAILS.
>    4. Write code to make the test PASS.
>    5. Refactor, if you introduced any code duplication in step 4.
>    6. Run all the tests, to make sure you didn't break anything else.
>    7. Commit.
>
> If someone would be so kind as to identify a small bug or feature, I
> would be happy to demonstrate this workflow in the form of code, time
> permitting.

I've just started reading and entirely agree that unit testing org-mode 
would be a good thing.  It'd also help people like myself familiarise 
themselves with elisp, org-mode's internals and the expectations of 
certain features.

Anyway, given that nobody seems to have found you a simple definitive 
bug, here's a suggestion:

Why not look through the git repo for a trivial bug carsten fixed in the 
past, branch the repo at the point the bug existed and write a 
regression test for it?  Then test the fix against the commit that fixes it.

This might also prove a nice way for other people to introduce tests 
into org-mode.  By targetting areas where we have had bugs previously, 
we can build up coverage, and develop regression tests for known 
(historical) issues.

Source control systems (and especially git) are wonderful things.  A 
really nice feature of git which might help moving forward is git bisect.

git bisect is typically used to find the commit that introduced a 
particular bug or regression.  The typical workflow is this:

1. You realise there is a bug on a branch head, but you don't know when 
it was been introduced.
2. With git bisect you mark the commit that you know has the bug as 'bad'.
3. With git bisect you mark the last known good commit.
4. You run git bisect, and it will recursively find you a commit in the 
middle of your good and bad commits, and ask you whether it is good or 
bad.  It proceeds in a binary-chop manner, until it finds the precise 
commit that introduced the bug.

The awesome thing is that you can completely automate this process by 
writing a program/test that can test for the presence of this bug!

If it were desirable to make use of this feature, then it might be 
better if the unit tests were written in a different git repo (perhaps 
also referenced as a git submodule so the tests can also be located 
inside the org-mode repo for convenience).

Doing this would ensure that the tests can exist outside of org-mode, 
allowing you to easily run bisect.  Avoiding the issue of bisecting to a 
point before the test existed.

Anyway, it's just a suggestion!  And all the best on introducing some 
tests! :-)

R.