Re: texi2any implementation language

[Patrice Dumas]

On Thu, Dec 12, 2024 at 09:12:26PM +0000, Gavin Smith wrote:
> > > On MS-Windows, this is even more problematic, because there are
> > > several incompatible ports of Perl to Windows.  That is the reason why
> > > I always remove the XS modules from the binary distributions I put on
> > > ezwinports site -- I cannot possibly distribute the binaries of Perl I
> > > use to compile XS, and cannot even hope people are likely to have the
> > > same Perl installed.  With texi2any automatically falling back to Perl
> > > when XS modules cannot be found this is just a performance issue, but
> > > if Texinfo requires XS, this would mean users will be unable to run
> > > makeinfo in such cases.
> > 
> > In that case it's likely we'll have to keep on maintaining C and Perl
> > code side-by-side for as long as the program is written in Perl, even if
> > this makes making changes in the program more difficult.

If we really want to remove the Perl code corresponding to the Parser
and structuring, we could force users to install and use the same Perl
that was used to compile the XS.  That would be problematic for users
on MS-Windows (and on all the platforms with risks of ABI
incompatibilies), as Eli said, but we should keep in mind that this is a
possibility.  To me, keeping a dual C/Perl implementation of the Parser
+ structuring is a luxury that we can stop if we do not have the
manpower to maintain the dual implementation in the future, by dropping
the Pel implementation.

Also, my feeling is that, as long as we use a high-level language, even
if we switch to another high-level language than Perl, we will have the
same issue on MS-Windows of ABI incompatibilities for the C parts.

> Any replacement for Perl should have the feature that we can avoid having
> dual implementations of some code (e.g., XS versus pure Perl).

I do not think that it is necessarily a good thing to avoid dual
implementations.  I think that it depends on the part of the program

1) for HTML, since there is a customization API in Perl, I think that
   we should keep a dual implementation of the HTML converter for a long
   time.  It could be possible to embed a Perl interpreter
   for this converter only, if needed (see below).

2) for the other converters, to me the situations is the same than for
   the Parser.  If there is a converter rewritten in C (or C++), we
   could consider dropping the Perl implementation.  Same as for the
   Parser, this would be problematic for platforms where ABI discrepancies
   are likely such as MS-Windows, but should be considered if we lack
   manpower.  First, we would need volunteers to translate to C/C++ the
   existing converters.

3) for the main program, translating most parts to C is probably
   relatively easy, and I will probably do it at least partly, little by
   little as we use teximakehtml more to debug, although it is low priority.
   However, because of HTML customization in Perl, going C (or C++) only
   would require an interface for Texinfo::Config and embedding a Perl
   interpreter, which is  more specialized code and I do not think that it
   will happen soon, especially if we keep a dual implementation for
   portability on platforms with ABI discrepancies.

> Of course, a compiled language (Go, Rust, C++) wouldn't have this
> problem at all - the entire compiled program could be distributed
> with no dependencies on the version of an interpreter.

I think that the HTML customization would be better done in high-level
languages in any case, and because of history, there need to be at
least the possibility to do it in Perl.  This rules out using a compiled
program for all of texi2any, even if we replace Perl by another
high-level language.

> I know that C++/C hybrid programs are possible.  I'm led to believe
> that Rust/C hybrids are also possible (although I don't know a great
> deal about Rust).  I gather that Go/C hybrids are problematic and
> not particularly recommended (just based on what information I've
> been able to surmise based on a cursory search of the web, although
> I could be missing important information).  I'm not aware of other
> candidate languages.

To me, the real issue is not about existing code, nor about the choice
of the language(s).  A mix of Perl and C is not a better nor worse
choice as any other.  The issue is more about attracting developers,
which needs to be balanced against the maintainability of mixing many
languages.  I do not think that it is likely that switching to another
language would attract developers to help maintain texi2any itself, as
long as there is nobody voicing or showing both an interest for
texi2any on the long term and a will to use other languages.  Ever since
the IXIN stuff, which is quite old, I have not seen much volunteers with
a specific project that would require another language.

And I think that, if some change in language is to happen, there is
another way that do not require changing texi2any upfront, which I would
prefer, namely making easier to have converters in other languages than
C and Perl.  I think that this should rather be the direction, with a
possible switch of pieces of texi2any to other languages once we are
sure that there is enough willingness to maintain the code in other
language(s) the long term.

In more details, to me it is already possible to do converters in C++,
the C tree structure and options structure can be accessed from C++, the
only thing needed would be XS to interface with
Texinfo::Convert::Converter, which should be simple, only four
interfaces would be needed (converter_defaults, generic_converter_init,
output, convert), and the existing code could be used as example.
It could also be possible to use the interface to C converter for that.

Doing a converter in Rust would not be directly possible, as there is no
XS interface for Rust in Perl.  I know nothing about Rust, so the only
possibility could be through a C interface interfaced internally to
Rust.

For high level languages, this is more complicated, but still doable if
somebody feels motivated.  What would be needed is (when there is no
need for customization in the high level language):
* an interface to the C Parser/Document data, similar to what is done for
  Perl mainly in main/build_perl_info.c
* a possibility to embed an interpreter in C
* C code to call the high level language
* XS interface for that C code (the same 4 functions as above).

A main program in another language than Perl would, once the above is
done, require in addition an interface to the C Parser code and
to the converters that are in C/C++.

Yet another possibility would be to allow for HTML customization to be
in another language than Perl.  I think that we should be open to that,
but it is an important work as it would require
* an interface to the C Parser/Document data, similar to what is done for
  Perl mainly in main/build_perl_info.c
* a possibility to embed an interpreter in C
* the implementation of the HTML converter
* the implementation of the Language/C interface to get customization and of
  the C/Language interface (like convert/call_html_perl_function.c).

I think that we should facilitate such additions, by helping with the
interfaces, but, at that point, I think that what is missing much is not
the possibility to use other languages, but volunteers willing to code
the interface with the C Parser and to code the converters.

-- 
Pat