Extending LilyPond through packages

[Urs Liska]

I'm starting a new thread because the original thread has diversified 
too much already.

Am 05.12.2013 18:48, schrieb David Kastrup:
> > For starters, we could take
> > >https://github.com/openlilylib/snippets/tree/master/templates/predefined-instruments
> > > and expand it, and add such predefined "instruments" to official
> > > LilyPond.  I think it would make "structural" work much easier (esp.
> > > for beginners).
> We need to figure out how we can provide "style sheets", similar to how
> LaTeX makes it possible to define "document classes" (layout definitions
> and tools) and "packages" (raw functionality packaged into coherent
> interfaces).
>
> Moving in the direction where this is possible also takes some pressure
> of stable/unstable development and features/fixes: something which comes
> in its own, optionally used file is not disruptive to the core
> stability.


OK, I'll give it a first try.
Please be aware that this isn't a specification put up for review but a 
first shot to nourish a discussion.
As a working hypothesis I'll try to take LaTeX and TeXLive as a model.

Adding the concept of "document classes" and "packages/stylesheets" to 
LilyPond seems a very good idea to me. This would allow a consistent way 
to add functionality to LilyPond without a) bloating or b) compromising it.
Responsibility for stability and usefulness would be deferred to the 
package author and users, code is only loaded into LilyPond when the 
user decides to need it in a concrete score. I don't think it's a 
problem that users may run into "command not found" problems. This 
equally present in LaTeX, and it isn't really a problem.
Consequently this would allow us to be much more generous when it comes 
to accept contributions. CTAN respectively TeXLive more or less accept 
any contribution if it
- is suitably licensed
- has documentation (not only in binary form)
- doesn't create naming conflicts.
There is practically no selection based on quality of code, concept or 
being in line with the preferences of the maintainers.
I think this could be copied.

For now I'm concentrating on packages because document classes seem more 
complex, and one aspect should be enough for a first post.

----

Provide a directory structure in the LilyPond directory. This is 
automatically included in the search path and will be searched 
recursively, so I can simply use "myNewPackage" although it's actually 
in the "fancy/engravers" subdirectory.

Maybe it would be good to consequently accept a parallel "private" 
directory structure (like ~/texmf) where the user can manage his own 
packages or 'install' packages received from others or downloaded somewhere.
This isn't necessary because it could easily be dealt with through 
traditional search paths, but I'd say it's a nice way to get things 
consistent.
If a package isn't found in LilyPond itself search
- a default location (e.g. a directory in the homedir)
- look for an environment variable
- an include path (or all) specified on the command line

---

One question is whether the library should be included in the default 
installation/download or if it should be made optional.
Consequently one would have to discuss if one needs a package manager 
like tlmgr.

---

Basically a "package" can provide arbitrary code to be included.
It can for example provide
-  overrides to modify the output appearance (classic "stylesheet")
- new commands / functionality
- alternative header definitions with arbitrary header fields
- engravers or anything one would otherwise include too.

Provide commands like \usepackage and \RequirePackage (naming could be 
discussed) that include the package file.
Differences to \include
- search path management (recursive search)
- allow options:
  e.g. (in LaTeX syntax)
  - \usepackage[font="Libre Baskerville"]{mySongbookHeader}
  - \usepackage[staffsize=17]{beamerLayout}
  - \usepackage[keep-only-original]{originalBreaks} % ;-[]
  - \usepackage[console,color,html]{annotate} % (print messages to 
console, color grobs, export html list)

Optional arguments could be passed as a Scheme alist.

Maybe it would be good to provide commands to consistently handle such 
options inside the package so one doesn't have to reinvent the wheel in 
each new package.

---

A package should have its own manual like in LaTeX.
Integration a potentially high number of manuals in LilyPond's 
documentations seems not maintainable.

---

I think one wouldn't need much more as an interface for packages.
Some of this is also applicable for document classes. But I won't think 
about this right now although I can imagine them to be extremely useful.

Best
Urs