Re: Extra paragraphs incorrectly spawning when ":end:" appears.

[Ihor Radchenko]

"Tom Alexander" <tom@fizz.buzz> writes:

> This test document should have 1 paragraph but org-mode is parsing it as 2:
> ```
> foo
> :end:
> baz
> ```
>
> which parses as:
> ```
> (section
>   (paragraph "foo\n")
>   (paragraph ":end:\nbaz\n")
> )
> ```
>
> The paragraph documentation[1] states that:
>> Empty lines and other elements end paragraphs.
>
> But the document contains no empty lines and we can see in the output that it 
> only contains paragraphs.

The documentation is not accurate here.

The parser uses anything that _potentially_ looks like the beginning of
another element to calculate paragraph boundaries
(`org-element-paragraph-separate'). ":end:" is potentially a drawer and
thus ends the preceding paragraph.

Later, ":end:" line is parsed as a new structural element using
`org-element-drawer-parser'. The drawer parser detects that there is no
closing :end: line and thus falls back to paragraph parsing:

(defun org-element-drawer-parser (limit affiliated)
...
        ;; Incomplete drawer: parse it as a paragraph.
        (org-element-paragraph-parser limit affiliated)

The same logic applies to a number of other incomplete elements.

The reason behind the current logic and not re-parsing the preceding
paragraph when we encounter incomplete drawer/block/etc is that Org
parser is written to do a single pass - we never re-parse already parsed
parts. Doing things otherwise, while could solve certain non-intuitive
behaviors, would be problematic performance-wise.

So, the actual paragraph separator that should be used is
`org-element-paragraph-separate' regexp.

We need to fix the WORG syntax description accordingly.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>