Re: [Orgmode] Re: [babel] Painless integration of source blocks with language

[Eric Schulte]

Seth Burleigh <address@hidden> writes:

> My bad. I believe i did look up the functions, but they didn't do what i
> needed (or so i thought).
> The code doesn't matter too much, let me explain the idea.
>
> A file may contain many blocks of code. Lets look at a arbitrary block A. In
> the end, block A will somehow become noweb embedded into a source block
> which is actually tangled to a file. Lets call this source block C.
> find-chunk finds this source block, given the position of block A, checking
> also that the source has a type of "clojure". So, basically it searches for
> the string <<A>>, if it finds it, checks if the source block (with an
> arbitrary name, lets say my-block) is tangled, if not, it searches for
> <<my-block>> until it actually finds a tangled source block.
>

This is functional which is currently missing in Babel (e.g. finding
where code block A is referenced), I can see the appeal for such a
function.

>
> tangle-chunk is used to tangle block A by finding block C (using find-chunk)
> and tangling that. It is basically copied from org-babel except that it uses
> mkdir to create any parent directories that do not exist (i think this
> should be included as an option in tangle file, btw, if it isn't)
>

Does tangle chunk tangle just block A, or just block C, or does it
re-tangle the entire file.  If just selected code blocks, how does it
know where to insert them into the file?

WRT: creating missing directories, maybe this would be better as an
option rather than the default, as I often mistype a tangle path and
it's useful for me when an error is thrown upon trying to tangle to a
non-existent directory.  Also, for packaging up repositories, there is
an `org-babel-pre-tangle-hook' which could be used to prepare a
directory structure in which to tangle source files.

>
> So, conceptually we have many blocks which are noweb embedded into various
> source files, however indirectly. We want to find the source files that
> these blocks are embedded in so that we can (in the future) navigate from
> the block to the file number.
>  '

Agreed, the lack of the ability to trace noweb references through to raw
source files (and more importantly to trace back from source files
through to the original Babel files) is becoming a pain-point in the
current code base.

It is currently possible to both jump back from source code to the
embedded code block using `org-babel-tangle-jump-to-org', it should be
fairly straightforward to write a function for jumping from a code block
to source using the same comment lines as anchors.

As for how to trace back through noweb links, the best option seem to be
using the existing jump function to navigate from raw source to the
embedded block, keeping track of the point's offset form the beginning
of the block, then using `org-babel-expand-src-block' to expand the body
of the embedded code block marking noweb references with text properties
as they are inserted into the expanding body, and then using the point
offset to place the point into the appropriate noweb reference.  This
process could then recurse on the embedded noweb code block until it
ends up in a non-noweb portion of an expanded code block body.

> 
> Thats all that does, from line 110 up. I copied it from my previous posting.
> From below that, i implement an emacs overlay, created by new-chunk.
> Basically all it does is associate a chunk of code in the buffer to a chunk
> of code in another chunk. We can 'add a link', i.e. append a chunk to its
> 'lp-link property.
>
> Next, we add the ability to replace the code of one chunk with another. This
> is what push-chunk does. Now, all that we have left is the code to create
> chunks from source blocks.
>

Meaning you can update a portion of the raw source file and push the
changes back to the org file?  This sounds similar to what is done by
`org-babel-detangle' only it uses marks and overlays to maintain the
source->org mapping rather than comments (which is a cleaner interactive
solution, but lacks persistence beyond a single Emacs session, or the
ability to say fold in changes to the raw source made by someone else on
a group project).

>
> Pushing code from one chunk to another code work with noweb embedding, but
> it was just a proof of concept anyways.
>

That's great, what was the logic used to push back through noweb
references?  Did this rely solely on the overlays you created during
tangling?  The analog of that approach would be to insert noweb
references wrapped in comments, which may be the simplest solution, as
long as there are not cases where the additional comments would be
unwanted in the raw source code file.  Perhaps there is a way for Babel
to use marks rather than comments and then save the mark information in
the org file, this would be great in that it wouldn't require any Babel
comments in the source file, although I'm not sure how robust marks are
to offline editing.

>
> What i was imagining when i did this is that i would have a source code file
> which has various 'chunks' highlighted in different colors (various
> background colors), each having a 'link' to the appropriate source code
> chunk in the babel file. Thus, we could push changes from the source code
> files to the org-mode file and navigate between. This would eliminate the
> code markers. I was imagining that the tangled source code files could be
> viewed simply as an indirect buffer created from the org file, so we
> wouldn't need code markers to save the chunk info for future sessions - we
> would just generate it all from the org-file.
>

Yes, this is similar to what I imagined after reading your first post,
and I think it is a great goal to shoot for.  Unfortunately I personally
will be very busy through nearly the end of January...

Thanks for the explanation, Cheers -- Eric

>
>  What is needed now is to be able to push from one chunk to another, except
> leaving any <<srcname>> blocks as they are.  With that, i believe the ideas
> mentioned in the first post could be fully implemented.