[Axiom-developer] Re: AxiomUI project funded

[Kai Kaminski]

Bob McElrath wrote:

> Kai Kaminski address@hidden wrote:
>  
>
> > Sure. The cl-typesetting library is currently developed by Marc Battyani 
> > (http://www.fractalconcept.com/) and is "intended to be an alternative 
> > to the TeX like typesetting systems." It is still under heavy 
> > development but is already used by some people. 
> >    
>
> I think there is simply too much momentum behind TeX and MathML to
> contemplate an alternative typesetting system.  Dealing with these two
> will be headache enough...
The cl-typesetting project is not so much about creating a new input 
syntax for documents but a typesetting engine. For example someone 
contributed a module supporting XML-FO and IIRC they would love to have 
a module supporting LaTeX-like input. But I'm not promoting doing 
expression rendering ourselves at this point, so maybe we can just leave 
it at that.

> > But Axiom doesn't only send mathematical expressions as output. What 
> > about plots, for example? We'd also have to integrate an SVG library in 
> > Axiom then, or generate the SVG in the browser using Javascript. Not 
> > only am I confused as to wether the latter is actually possible, it's 
> > also a nuisance to have to use Javascript. Another example are 
> > conditions. If some part of Axiom throws a condition it would be cool if 
> > Axiom could communicate that to the front-end together with a list of 
> > restarts. Then the front-end could ask the user what to do and offer him 
> > to send a bug report. Of course, Axiom could instead generate an 
> > HTML-page offering the user a list of restarts, but that would mean even 
> > more non-math code in Axiom.
> >    
>
> We should enumerate an API specifying what input the back-end expects,
> and what output it can give.
>  
I agree.

> > Well, for other reason I want a separate GUI back-end written in Lisp 
> > anyway. That back-end would read the sexp-based output and emit XML/HTML 
> > for the browser. Implementing an sexp parser in a web browser is not a 
> > good idea, I agree. One thing to consider is that Axiom has its 
> > knowledge in some Lisp data structures anyway, I suppose. Hence writing 
> > the output in a format that is native to Lisp (sexps) should be very 
> > straightforward.
> >    
>
> So you're proposing *three* applications?
> 1) axiom
> 2) sexp -> xml converter
> 3) web browser/javascript
>  
Not really. I'm proposing that the GUI be implemented as a separate 
process from Axiom. That's all. The first version of that GUI happens to 
use a web browser to display output, but that may change in the future. 
Suppose in a couple of month someone starts working on wxWidgets 
bindings. Instead of generating 3d plots as X3D documents we might want 
to use wxWidgets OpenGL widget to display them directly. There is 
another widget allowing the embedding of Mozilla, hence we could still 
use Mozilla for the HyperDoc pages.

Even in the beginning part 2) will be more than an sexp-to-xml 
converter. Take plots as an example. Axiom supplies a list of points 
that have to be connected with straight lines/splines/whatever. Then 
part 2) generates SVG output, which isn't too hard but not trivial 
either. Suppose the plot contains some text, like a title or a 
description. We can't just stuff the text into the SVG file, because in 
XML certain characters have to be escaped. I don't want to implement all 
this in Javascript. But I don't want to have an SVG generator in Axiom 
either.

> I think the middle application is probably unnecessary, but let's get
> into the nitty gritty of what we want to exchange.  
Well, maybe it is, but I'm not convinced yet. I agree that we should 
start by clarifying what kinds of output Axiom will have to send and 
define a useful format. We might consider using something compatible to 
xmls (http://common-lisp.net/project/xmls/). The other question is how 
we implement that in Axiom. I don't know Axiom's architecture, and Bill 
said it might be too difficult. Opinions?

> I suspect that any potential GUI will want the same things, and will
> want those things in either sexp or XML format.  Building both those
> into axiom will be fruitful.  But I think the middle layer would end up
> being nothing more than a format converter... no?
>  
That's exactly what I'm aiming at. Axiom's output should be 
GUI-independent, contain as much information as possible and be easy to 
parse from a variety of languages. Sexps and XML seem to fit the bill 
(XML can often be seen as a verbose way of writing sexps). The middle 
part is not just a format converter. The GUI will become more complex 
over time. Suppose we want to support LaTeX as an input syntax for 
HyperDoc pages or something similar (Martin Rubey wants this, as I 
understand). Do we want to put a LaTeX parser into Axiom? Do we want to 
implement one in Javascript? Sure, we can possibly use existing 
converters to HTML/XML, but they might not quite do what we need and 
even if they do, we might havee to do some additional processing, like 
injecting some Javascript.

GUIs are complex and Javascript is an awful language without a 
development environment (correct me if I'm wrong). If, some day, we add 
complex behaviour to the Axiom GUI I'd much rather implement that in 
Common Lisp and push the results to a web browser than implementing all 
that in Javascript.

> I think the API should consist of RPC calls, with arguments marshalled
> in either sexp or XML.  Unless there is a strong reason to believe we
> cannot simply do a 1:1 conversion between XML and sexp, I advocate a
> single API and a single XML <-> sexp converter.  This is already
> present, to a large degree, in the axiom language.
>  
As long as we don't do really fancy XML this should be no problem. In 
fact the XML parser that I mentioned a few posts back has a nice mapping 
between sexps and XML.

> > But it shouldn't hang. And it should also continue to deliver help pages 
> > while a computation is running. Whenever I'm waiting some long-running 
> > process to finish I start reading documentation. If we don't have either 
> > threading or separate processes that's pretty much impossible.
> >    
>
> Why should axiom "deliver" help pages at all?  Why can't the interface
> read them directly off the disk?  And when those help pages contain an
> axiom session, why can't the interface start a new session for
> evaluation (so that it does not interfere with the main computation).
>  
AJAX wouldn't work then, but using Mozilla-specific extensions there is 
obviously a way around this as you have shown with your example. But if 
someone creates a new HyperDoc page he will have to include the 
necessary Javascript as well and he will have to write it in HTML. I 
would much prefer to have a different format for HyperDoc pages than 
HTML, because HTML doesn't capture the semantics of Axiom or HyperDoc. 
Consider the command line as an example. If we store the pages as HTML 
so that they can be read by a browser we would probably include some 
Javascript similar to what I've used for my dynamic jsMath demo. Now 
imagine that maybe in two years the W3C includes a tag in HTML that does 
pretty much the same, just better or more beautiful. Then we'll have to 
go through all pages using a command line, rip out the old code put in 
the new one. If we have our own format for HyperDoc pages (XML with a 
HyperDoc DTD, sexp-based, LaTeX-based,...) we just change the code in 
the middle process that is responsible for rendering the HyperDoc pages 
to HTML and they are all new and shiny. Of course we could just do the 
conversion once, every time something like this happens, but that would 
require us to keep two versions of every page around and sooner or later 
people would try and modify the rendered HTML instead of the page source.

> This will still require threading in the server, but it's simply
> threading of axiom sessions, and not truly "interface code".
>  
I guess we could just start several instances of Axiom.

> The only thing missing from this idea is "search".
>  
Good example. Search is complex. Do we implement that in Javascript or 
will the search code be part of Axiom? If we have a process between 
Axiom an browser we can just put it in there.

> > > It should be easy to use these existing pieces to enable a 
> > >  )set output sexp on
> > > and
> > >  )set output mathml on
> > > by converting sexp's to mathml.
> > >
> > >      
> > That's my point. If we have sexp output it is trivial to convert that to 
> > any kind of XML we'd like, apart from semantics, but no format would 
> > solve that problem.
> >    
>
> What do you mean by "no format would solve that problem"?  What are you
> referring to with "format"?  And the problem is conversion of sexp <->
> xml?
>  
What I meant is that converting sexps to XML is trivial in the sense 
that "(tag :attribute value ...)" becomes "<tag attribute="value"> ... 
</tag>". Rendering things to XHTML on the other hand can be more 
difficult. Tim once remarked that he couldn't see how we can associate 
meaning to XML data. The answer is that it's difficult, but that is not 
a specific problem of XML.

> Now that hyperdoc is working...I just discovered axiom's graphics
> capabilities.  Graphics will require a lot more thought.  My google
> searches turned up basically nothing as far as scientific graphics in
> lisp.
>  
I'm not sure what you mean here, but I assume that Axiom already 
incorporates code to generate plot data. We can use that to create SVG 
files, which look very nice. PDF is also easy, because of cl-pdf.

Kai