Re: Solution for including a file only once

[Michael J. O'Donnell]

Hans and Carl,

At present, I am seeking an add-on solution, involving a definition that
I can put in a utility file to include. I think I am near finding that.
I am not ready to hack the lexical analyzer, and I think that should be
pondered carefully, particularly when a global reconsideration of syntax
is planned. But at some point, a better integrated solution is desirable.

Haskell and ML have put a lot of thought into modules, which I think are
much better conceived than objects in C++. When we get to the GLISS
rethinking of syntax, it will be good to look carefully at what such
languages have already done. One of the two main implementors of
Standard ML of New Jersey is down the hall from me, and he's a module
proponent, so I can help with that when the time comes. The association
of \include with the lexical level is like the old C preprocessor
attitude. That sort of lexical manipulation is often a good choice in an
early design, when one is not sure of the right structure, and first
attempts at good structure tend to be too restrictive. But by now we
know how to do better without being too restrictive.

I noticed that Guile also has a module system, and I have a generally
good impression of the thinking behind Guile. As long as LilyPond uses
Guile, the first thing to see is whether it can use the Guile module
system directly for this sort of organization. The Guile macro system is
also a possible hook for include-like things. Include is essentially a
parameterless macro invoker. Guile follows work in the Scheme community
to apply macros at the syntax level instead of the lexical level. Once
syntax is well defined (which it has been in LISP much longer than in
other languages), that is usually a better approach than lexical-level
macros. Modules take similar facilities to the semantic level.

For the very short term, I expect that "\includeIfAbsent", or whatever
we call it, will be most useful if it duplicates the current behavior of
"\include" as much as possible, except when it does nothing due to a
previous inclusion. So, I'm going to stack thoughts about specific uses
and connections to name spaces. I will try to insert them into the GLISS
discussion---I don't know Hans' programming ideology, but I think that
systems like LilyPond should follow best programming language practice
as much as possible. No matter how simple designers expect the uses of
the system to be, later practice almost inevitably goes into regions
where the more advanced structure is important. And I think that modules
are near if not at the state of the art for organization of programming
information.

In case you want to see that much detail, I'll stick in my current
prototype:

% Include a file unless it has already been included

includeIfAbsent =

#(define-music-function (parser location fileName) (string?)

(let ((guardName (string-append "Already Got " fileName)))

(if (not (defined? (string->symbol guardName)))
(begin

(primitive-eval (list 'define (string->symbol guardName) #t))

(ly:parser-parse-string parser (string-concatenate (list "\\include \""
fileName "\"\n")))

#{ #}

)

#{ #}

) ) )

I mimicked some code for includePageLayoutFile from
music-functions-init.ly, which feeds its own constructed \include
command to a parser, which presumably has the same old lexical analyzer
as its front end. I wasn't sure which parser gets passed to a music
function, but it looks like I've got the right one. My current problem
seems to have to do with the need to return an actual bit of music,
which I would like to be an empty bit. It may be that includeIfAbsent
doesn't have to be a music function, but so far I haven't found an
alternative. The code above actually seems to create the right output,
but then it hits a segmentation fault, perhaps in some final end code.
That probably reveals some bug that only affects somewhat weird
programs. I'll try to chase it down further.

Cheers,

Mike O'D.
> On 11 Feb 2010, at 05:32, Carl Sorensen wrote:
>
>>> I have been hunting, so far in vain, for the code implementing
>>> \include,
>>> in the hope that I can tweak that code to get the right behavior.
>>
>> It appears to me that \include is implemented in two places:
>>
>> 1) lily/lexer.ll, lines 304-336
>>
>> 2) lily/lily-lexer.cc, lines 255-271
>>
>> Now, please understand that I'm not at all an expert on the
>> parser/lexer of
>> LilyPond. But I think that lexer.ll calls new_input, and lily-lexer.cc
>> defines new-input.
>
> The .ll (or .l) file is what is passed to the lexer, and you probably
> use Flex. It compiles the code to a .cc (or .c) file, which is then
> passed to the compiler which produces .o object code, a format that
> typically has embedded machine code, which is actually what is executed.
>
>> lily-lexer.ll is where the file identifier is extracted (lexed?), and
>> would
>> be an ideal place to see if the include has already been processed,
>> if the
>> necessary variable structure is available.
>
> So this is the file that one would change. Then 'make' will sense that
> this file hjas been changed, invoking flex, producing a new .cc file.
>
> From what I can see on git.savannah, the \include definition in
> lily/lily-lexer.ll just inserts the file in the stream using a method
> that Flex provides.
>
> By contrast, if one would want to have an import command that checks
> that a module isn't loaded twice, one should have table for the module
> names. Then an opened module is read all through, the definitions
> entered into Guile, and then the module is closed.
>
> This may not be so difficult to implement if there is some top level
> command where lilypond starts reading the first file. Then the new
> \\import just calls that command.
>
> Hans
>
>
>