[Orgmode] Re: [babel] How to kill two birds with one stone?

[Sébastien Vauban]

Hi Dan,

Dan Davison wrote:
> Cool post. I hope someone has some good ideas in this thread. Some quick
> responses / questions below.
>
>> Note, in the latter code block, that I did not even tried to really chain
>> steps 2 and 3: I'm rewriting step 3, including step 2 inside it.
>>
>> *I certainly miss a smarter way* to achieve the above.
>
> I think a relevant point here is that Org doesn't yet have the ability to
> pass data on standard input to a code block. I.e. a :stdin header arg. I
> don't think it's that hard, someone would just need to rework
> `org-babel-eval' so that it puts the code into a temporary file, freeing up
> stdin to be used for data

I think you exactly spotted the problem.

> (and therefore we would no longer be able to use
> shell-command-on-region but some other command (call-process I think?).)

I just don't understand this last sentence, by lack of knowledge on Babel's
internals.

> Then you'd be able to do something like
>
> #+srcname: search-links-and-generate-dot-arrow
> #+header: :stdin search-files-pointing-to-this-file
> #+begin_src sh :results output :var f="charge_dim"
> while read f; do
>     echo "    $(basename $i) -> $f";
> done
> #+end_src
>
> I'll be interested to see the solution to all this.

I've tried to rewrite the example in a much cleaner way, showing what it
should like in the best of the worlds, with a working code at hand...

BTW, I think the following could be of use, with maybe slight modifications,
even for projects like Worg: identifying all relationships between files,
showing files that aren't referenced, etc.

#+TITLE:     Graph file dependencies
#+DATE:      2011-02-06
#+BABEL:    :dir ~/src/Worg

* Context

We want to demonstrate how to document a script in a very neat way (IMHO),
that is:

- By defining and explaining multiple small code blocks, using them later in a
  tangle file for constructing the "full code".

- By showing the effect of every small code block, that is what it returns
  when applied on test input data.

The latter is the problem, as the code has to be able to take a results set as
if it would come from =stdin=.

* Code

For the sake of clarity, a real-life example that graph dependencies between
files (based on their /basename/).

** List all files

Simple file command, ignoring =.svn= directories.

#+srcname: file-tree
#+begin_src sh :results output
find . -not \( -name .svn -prune \) -type f -print | head -n 5
#+end_src

#+results: file-tree
#+begin_example
./digraph.dot
./full-code.sh
./graph-circo.pdf
./graph-dot.pdf
./graph-fdp.pdf
#+end_example

** Search recursively for anything about a file

Grep-search through files, ignoring =.svn= directories.

#+srcname: search-files-pointing-to-this-file
#+begin_src sh :results output :var toname="charge_dim"
find . -not \( -name .svn -prune \) -type f -print0 |\
xargs -0 grep -i --files-with-matches "$toname"
#+end_src

#+results: search-files-pointing-to-this-file
: ./graph-file-dependencies.txt

** Convert to DOT

In real life, the following block of code must read its input from =stdin=.

For /in situ execution/, I should be able to say that =stdin= is equal to any
results set (here: =search-files-pointing-to-this-file=).

#+srcname: make-dot-arrow-for-files-pointing-to-this-file
#+begin_src sh :results output :var toname="charge_dim"
while read -r fromname
do
    echo "    \"${fromname##*/}\" -> \"$toname\""
done
#+end_src

#+results: make-dot-arrow-for-files-pointing-to-this-file

** Full code: generate the DOT file

#+begin_src sh :results output :file digraph.dot :noweb yes :tangle full-code.sh
echo 'digraph G {'
echo '    node [shape=diamond,style=filled,color=lightgrey]; ".cvsignore";'
echo '    node [shape=box,color=yellow];'
echo ''

<<file-tree>> |\
while read -r fname
do
    toname="${fname##*/}" # basename of fname
    echo "# Files pointing to \"$toname\"..."
    <<search-files-pointing-to-this-file>> |\
    <<make-dot-arrow-for-files-pointing-to-this-file>>
    echo ""
done

echo '}'
#+end_src

#+results:
[[file:digraph.dot]]

** Draw multiple graphs

#+srcname: draw-graphs
#+begin_src sh :dir ~/Projects :var infile="digraph.dot" :var 
outfileprefix="digraph"
for cmd in dot neato twopi circo fdp sfdp;
do
    echo $cmd...
    time $cmd -Gcharset=latin1 -Tpdf -o$outfileprefix-$cmd.pdf $infile
    echo ""
done
#+end_src

Best regards,
  Seb

-- 
Sébastien Vauban