Re: Literate Executables

[Tim Daly]

As I mentioned, I really like your idea of embedding code in various

support tools such as make, grep, etc. It would prevent code-rot as

it would be possible (assuming C++ would hold still for a week) to

be able to recreate the support tool. Axiom uses git. That makes it

possible to check out a particular instance at any given time in the past.

On a practical level generating the PDFs from inside the Axiom

executable is questionable. Axiom has about 1.2 million lines of

code. The PDFs that contain the code, the images, etc. would

be MUCH larger. It would be much easier to create an Axiom

command for a given function to link a URL to the PDF and pop

the PDF up in a browser. Axiom's PDFs can contain 6000 pages.

On a philosophical level, the question is one of emphasis.

Axiom is a large computer algebra system. Understanding it in

detail requires a PhD level of mathematical education. Indeed some

of the algorithms were implemented as part of a PhD thesis. It is

not reasonable to expect anyone to understand some of the

algorithms without access to explanation. The code is opaque.

As mentioned before, my approach to literate programming leans

toward 'textbooks'. The textbook has an expected structure which

includes an introduction, a table of contents, chapters, sections,

cross-references, an index, and a bibliography. People know what

to expect and how to navigate a book. This gives structure to the

explanation and makes it easier to find things.

Also of philosophical note, the fundamental point of a literate program,

in my approach, is explanation (not documentation). So the PDF book

is the primary element (as it would be in physics textbook). The code

is secondary  (as equations would be in physics textbooks). The

explanation communicates to the human in hopefully precise terms

whereas the code communicates to the machine and could have many

different implementations.

To your point, the project goal of Axiom emphasizes literate programming

because, unlike other kinds of software, the mathematical algorithms will

not soon go out of date. However, open source software tends to die when

the lead developers stop working on a project. The literate programming

approach is an effort to let Axiom live beyond the current authors (some

of whom are already dead).

The above considerations leads me to the conclusion that the PDF

is the thing that generates code rather than the code generating the PDF.

Tim Daly

http://axiom-developer.org/~daly

On Thu, Dec 1, 2022 at 10:00 PM Terence Kelly <tpkelly@eecs.umich.edu> wrote:

Hi Tim,

Thanks for your feedback, and for pointing me to Axiom.

One "literacy" aspect of literate executables is that the latter can "give
an account of themselves," in the sense of dumping their own source ---
and their own documentation, which of course can be a literate program!

Imagine if the Axiom executable(s) could dump the PDFs containing the
Axiom literate programs.  That would ensure that any user capable of
executing Axiom had the documentation *and* the source at her fingertips,
with no possibility of executables and documentation becoming separated or
mismatched.

I'd be happy to help if you'd like to try this.  As my example code ("grep
literatizer") illustrates, endowing a program with literacy (in my sense)
is remarkably easy.

Thanks again & let me know.

-- Terence


On Wed, 30 Nov 2022, Tim Daly wrote:

> I'm a huge believer in literate programming. In fact I'm trying to embed
> a complete computer algebra system (Axiom) in literate form [0]. The
> PDFs listed on that page contain the actual source code that creates
> Axiom.
>
> I believe you've achieved 'the letter of the literate concept' but
> missed the 'spirit', as expounded by Knuth.
>
> Your ACM article on Literate Executables
> https://queue.acm.org/detail.cfm?id=3570938 inverts the approach I've
> taken. I like your proposal.
>
> In your approach the executable is the primary carrier. For small,
> stand-alone executables this is an excellent idea. It has the potential
> to eliminate code-rot in supporting tools.
>
> For a large system like Axiom, one key struggle is that Axiom has
> various layers and subsystems. Working on one part of the system does
> not affect other parts.
>
> However, even more fundamental is that literate programming is about
> Explanation, not Documentation. Think of a physics textbook. The
> equations (code) are quite opaque without the surrounding paragraphs of
> explanation. The same is true for complex software.
>
> In Axiom's approach the PDF is the primary carrier (the textbook).
>
> The development cycle is to change the latex source code for the PDF.
> Then type 'make' which (a) recreates the PDF and (b) recompiles the
> code. Rinse and repeat. At all times the PDF and code are in sync. The
> purpose of the PDF is to explain the concepts and hold the related code.
>
> My point is that your use of 'literate' in 'literate executables' is
> (slightly) missing the fundamental point of literate software. It is
> true that it keeps the code in sync with the executable and keeps the
> code available for any given version. However, it misses the fundamental
> literate point of 'Explanation'.
>
> Tim Daly
> http://daly.axiom-developer.org/~daly
>
> [0] Axiom
> https://en.wikipedia.org/wiki/Axiom_(computer_algebra_system)