Re: [Texmacs-dev] literate programming for TeXmacs

[Felix Breuer]

On Thu, 19 Jan 2006 21:00:30 +0100
David MENTRE <address@hidden> wrote:

> Hello,
> 
> Felix Breuer <address@hidden> writes:
> 
> > For use with literate programming, we would have two clones of every
> > code snippet. The one would reside in an "untangled" part of the document,
> > and the other would reside in the tangled part. Via some mechanism, we
> > could make sure that the content is actually stored in the tangled part
> > of the document, so that, given the new serialization format, we would
> > then get the tangled version of the code automatically. If the user
> > makes changes to the code via a text editor, there is no need to propagate
> > the changes to the untangled part, because the content is serialized only
> > once.
> 
> Your clone idea seems, at first read, similar to my reference
> suggestion (albeit more elaborated ;).
> 
> If I have understood correctly, the "serialized form" is compilable
> source code, that includes both original source code in a first part,
> with appropriate labels as comments, and a second texmacs part that
> references the code snippets in the first part, possibly
> re-ordered. 

Yes, though it is up to the user to decide how to organize/arrange
the document.

The "serialized form" is the entire texmacs document in a format that
closely resembles the .tm format: Everything is serialized in standard
texmacs markup only that each line is commented. Special blocks of verbatim
text are the exception: they are serialized using the "verbatim" method
and not commented. So what is now

  <TeXmacs|1.0.5.11>

  <style|generic>

  <\body>
    <section|My Program!>

    Here comes my function.

    <\code>
      bool less(int i, int j) {

      \ \ \ return (i \<less\> j);

      }
    </code>

    Blah blah.
  </body>

  <\references>
    <\collection>
      <associate|auto-1|<tuple|1|?>>
    </collection>
  </references>

would then be

  //<TeXmacs|1.0.5.11>
  //
  //<style|generic>
  //
  //<\body>
  //  <section|My Program!>
  //
  //  Here comes my function.
  //
  //  <\code>
  bool less(int i, int j) {
     return (i < j);
  }
  //  </code>
  //
  //  Blah blah.
  //</body>
  //
  //<\references>
  //  <\collection>
  //    <associate|auto-1|<tuple|1|?>>
  //  </collection>
  //</references>

I think this is pretty close to (if not exactly the same as?) what
you suggested initially. Extensions of the form .tm.c (or tm.scm ...)
for files in this format would be great.

If we implemented the above as an export/import format for TeXmacs
we would have a working literate programming mechanism, that is 
completely independent of the clones/references mechanism. Even
if clones were never implemented we would have a lp system that
works well (even though writing object oriented code would be
a bit of a hassle).



> When the user opens the LP file in texmacs, he only sees the
> re-ordered and literate version. Right?

Well, what the user sees depends on the style he uses and on how much
work we want to put into implementing the clones mechanism.

= Specification

The clone node has the following format

  <clone|id|role|content>

Here id is a freeform string that is used as an identifier. All clone
nodes that have the same id are said to be "instances of the same clone".

Role is freeform text string that the user can use to identify the role
of that instance of a clone in its context. If we, for example, clone
a piece of code one role might be "tangled" and the other might be "literate".
See below for a use of this.

Content is an optional parameter containing the data of that clone. Only
one instance of a given clone may have this parameter.

Note 1: Clones are not particular to literate programming. They may be useful
in many contexts.

Note 2: In a way, a clone is the opposite of a switch. While a switch is a node
that has two different contents, a clone is a set of nodes that share a single
content.

An example in the literate programming case (pseudo .tm format):

  <section|Class Foo>

  The class Foo has the following methods:

  <\code>
                
        
        
        
    class Foo {
      public:
        <clone|method bar|tangled>
        <clone|method baz|tangled>
    };  
  </code>

  <subsection|Bar>

  The one function.

  <\clone|method bar|literate>
        
    <\code>
       void bar() {
         printf("bar!");
       }
    </code>
  </clone>

  <subsection|Baz>

  The other function.

  <\clone|method bar|literate>
        
    <\code>
       int baz() {
         return 42;
       }
    </code>
  </clone>

How this tree is visually represented, i.e. what the users sees on screen,
is dependent on the style file in use and on whether or not we implement
some additional functionality in the TeXmacs core. I have two alternative
implementations in mind:, the first of which should be possible to do in 
a plugin (on the scheme level).


= Implementation

- Variant 1

We write a scheme script that takes an id and a role and moves the content
of the clone "id" to the instance that has the role "role".

The user might interact with that script in three ways:

1) On each clone there is a hotspot (like the dot on a switch). If the user
clicks on it the content of that clone is moved to this instance.

2) The user can select a menu item that, for each clone, moves the contents
to the instance with a given, user-specified role.

3) For use with literate programming, there might be a script that
  a. moves all clones to their tangled instance, then
  b. exports the document in .tm.c format, then
  c. moves all clones to their literate instance again.

This would already be very useful and should be relatively easy to do.
(Says me who has never written scheme before ;)


- Variant 2

We make the internal handling of clones transparent to the user: TeXmacs
*displays* the content of the clone at each and every instance of the
clone even though the tree looks like above. The user can edit any of the
instances in place and the change is made instantly everywhere, because
the data is only at one point in the tree.

If this is possible at all without reimplementing significant parts of TeXmacs
is beyond my knowledge. I guess it is not realistically possible, because we
would have to touch the display/rendering code, the cursor movement, etc.

But we still have Variant 1  :)  And for the new serialization format, we don't
need either.


Well, I have written enough for today.

Best,
Felix