Re: Introducing the LIlypond Snippet Repository

[Han-Wen Nienhuys]

address@hidden writes:
> >   * I suspect that you could import the input/{test,regression}/
> >     directory wholesale.
> 
> Well, I thought about that, but many regressions show that lily does the
> right thing when writing stuff normally, so they're not a good candidate
> for explaining how to do strange staff.

You're right, but then again, it may also be worthwhile to explain how
to do "normal" stuff, because newbies may view "normal" stuff also as
unknown and strange.

> Unless, however, we want to have
> a "regression" flag and we have a separate search engine for
> regressions, mainly for developer. I'd be glad to do that if you think
> it's useful.

No, given what you write below, it would be more clumsy for
developers.  Nevertheless, some examples from regression/ might be
useful for LSR too.

> >   * is there a way to condense the DB into an file/package, both
> >     for offline viewing?
> 
> The DB is trivial: "mysqldump lsr" works like a charm. Packaging the
> engine is slightly more complex because I would like to have a *single*
> jar file, and java -jar lsr.jar should just run the engine. It can be
> done, but some more tomcat proficiency is involved.

Ok. Maybe that's a long term goal. Or we could have some sort of
mechanism to concatenate all the snippets together, much like the
current tips n tricks document.

> >   * every time we branch a devel version, the last stable DB is copied
> >     over (eg. 2.7), and we let the snippets evolve along with the
> >     branch. Once we release 2.8, the number is changed from 2.7 to 2.8
> 
> That's a good idea. In that case, certainly we need a way (currently
> there's none) to run the image updater in such a way to catch
> compilation mistakes in a log, so to know which snippet need fixing.

Yes, and there should be a way to run convert-ly on all snippets.

> >     This probably also requires that you can create a test-run DB,
> >     base.
> 
> ...sorry, can you rephrase that?

Oops. I was halfway during a sentence, I guess.

because of the development nature, one version of lily might not work
at all with the snippet DB, but you can only find that out by
trying. So you need a test environment, where you can try running the
latest lily on the LSR, but not export the results to the web.

> >     Hmmm. Or we developers should include an offline version in the
> >     RPM/docs package, and run the entire collection as a release-test,
> >     just like we don't release until the entire doc site build without
> >     hickups.
> 
> No, I think that instead I can just inhibit from showing what does not
> compile.

OK, but the failed ones should be available separately, since they are
essentially bugs.

> >     --safe, preferably sandboxed/chrooted. It is possible to do
> > 
> >     #(system "rm -rf / ")
> > 
> >     and other nasty code from .ly if --safe isn't used.
> 
> This is maybe the main problem, and some of you can help. The problem is
> that *not all snippets compile in safe mode, even apparently innocuous
> ones.*  The two snippets about fermate, for instance 
> 
> 
> \relative c'' {
>   \override Score.RehearsalMark #'break-visibility = #begin-of-line-invisible
>   c1 \mark \markup { \musicglyph #"scripts-ufermata" }
> }
> 
> do NOT compile... so it's a big problem, because important snippets are
> out if I use -s. Presently is active, but this will further confuse
> users.

This should not be a big problem. In this case, I solved it by adding
begin-of-line-invisible to safe-lily.scm. However, we should probably
have some Scheme level macro support for tagging a definition as
"safe", ie.

 (define-safe-public (begin-of-line-invisible .. ))

Nicolas?

Also, you could have a checkbox: "does not run with --safe" for a
snippet. In that way, the number of snippets that you have to review
manually is much smaller.

> chroot'ing would be a kind of mess--lilypond uses zillions of files. But
> in case I can rebuild a separate tree. If you have any suggestion about
> sandboxing, they're welcome--you know lily, so you know what it
> could

LilyPond looks at the variable LILYPONDPREFIX, which overrides

  /usr/share/lilypond/<version>

I'm not sure if chroot'ing would work with Pango and GUILE, though.

Hmm. On second thought, I guess it wouldn't work. 

> stand. Maybe with a careful sandboxing we can avoid using safe mode
> (maybe killing the process after 10s 8^).

You need that anyway.   You can easily make lily hang, even in safe
mode.

> > OK, just a minor concern here; are the state-of-the-art techniques
> > freely available: will there be any trade-secret/patent/copyright
> > problems in the future, or is the code available now?
> 
> I've added some stuff in What's This. Everything is GPL'd or LGPL'd. The
> techniques we used a published in scientific journals. I can't speak for
> Clarke & Cormack (the guys who published interval semantics first), but
> they never claimed to have filed any patents, in any paper. 

OK.

> > Hey, if you want, we can endow you with the official LilyPond
> > Development Team title of SNIPPET MEISTER, and have all the official
> > benefits, such as groupies, free beer at the Linux Audio Conference,
> > etc. :-)
> 
> I like the idea; unfortunately I don't drink 8^).
> 
> In any case, my job is nothing compared to the fantastic software you're
> working at. It's just that I hate C++ and never understood functional
> programming, so I can't really contribute to Lilypond directly. This was
> the best I could think of...

I think you hit the bull's eye though. Lack of infrastructure in the
documentation/snippets has been the #1 complaint about lilypond
usability in the past.

-- 

 Han-Wen Nienhuys   |   address@hidden   |   http://www.xs4all.nl/~hanwen