Fwd: pid leakage in LilyPond

[Patrick Karl]

I sent this response earlier to James Lowe privately for unknown  
reasons.  It illustrates more completely what I'm talking about.

A little more looking around on my Mac shows:

> sully:/etc pat$ sysctl -a | grep proc
> kern.maxproc = 532
> kern.maxfilesperproc = 10240
> kern.maxprocperuid = 100
> kern.aioprocmax = 16
> kern.proc_low_pri_io = 0

kern.maxproc sizes a kernel table which has an entry for each process  
in the system.  So on my system, there can never be more than 532  
processes at any one time.  Sure, the pid's can increase as process  
come and go (probably to an upper limit of 32767), but this process  
table doesn't grow.

It looks like a more restrictive limit is kern.maxprocperuid, which  
is 100 on my system.  That means that no "user" can ever have more  
than 100 processes at any one time.

Other systems can have different values for these parameters.

Note that running lilypond from the command line in a terminal does  
not incur this problem, as I suspect there is ordinarily only one  
Compile/Typeset activity per execution, so only one zombie process  
per execution.  And zombies are released when the parent exits.

Also note, therefore, that periodically exiting out of LilyPond is a  
workaround, as all zombie children of an exiting process are killed.

Begin forwarded message:

On Feb 1, 2010, at 2:35 PM, James Lowe wrote:


> Patrick,
>
> Excuse my ignorance but what is 'interactive mode' in this context?

I mean what you do, as opposed to running in "batch" mode by entering  
a "lilypond" command in a terminal window.

> I use Mac OS X 10.5.8 with Lilypond 2.13.10 and the inbuilt  
> Lilypond editor, and I cmd-s, cmd-r my way through a piece from  
> scratch so that can be hundreds of these commands run concurrently  
> in one session. I checked in Activity Monitor and I only ever get 2  
> PIDs as far as I can tell, one for the app itself and one when I  
> hit cmd-r and while it is compiling. I don't see any state PIDs in  
> Activity monitor nor in terminal running ps.

I use Mac OS X 10.4.11 with Lilypond 2.12.2.  I'm pretty sure that  
Activity Monitor doesn't show all processes, even when "All  
Processes" is selected.  So I'll demonstrate what I'm talking about  
with the output of the ps command executed in a terminal window.

* I click on the LilyPond icon in dock and rename the starter file as  
scale_in_lilypond using File/Save As.

* Now I want to find the pid of LilyPond:

> sully:/ pat$ ps -alx | head -1; ps -alx | grep LilyPond
>   UID   PID  PPID CPU PRI NI      VSZ    RSS WCHAN  STAT  TT        
> TIME COMMAND
>   501  9800    75   0  46  0   160364  12468 -      S     ??     
> 0:01.91 /Applications/LilyPond.app/Contents
>   501  9805  9292   0  31  0    27812    360 -      U+    p1     
> 0:00.00 grep LilyPond

So its pid is 9800.  Now I want so see its subordinate processes:


> sully:/ pat$ ps -alx | grep 9800
>   501  9800    75   0  46  0   160364  12468 -      S     ??     
> 0:01.91 /Applications/LilyPond.app/Contents
>   501  9807  9292   0  31  0    18052    244 -      S+    p1     
> 0:00.01 grep 9800

So it has no subordinate processes initially; note that the presence  
of the "grep" pid is a red herring here and should be ignored.

Now I go to the LilyPond window and do cmd-s followed by cmd-r.   
After the typesetting is complete (as shown by the busy indicator in  
the LilyPond log window quitting its activity), I repeat the above ps/ 
grep to see LilyPond's subordinate processes:


> sully:/ pat$ ps -alx | grep 9800
>   501  9800    75   0  46  0   186344  17164 -      S     ??     
> 0:03.87 /Applications/LilyPond.app/Contents
>   501  9813  9800   0   0  0        0      0 -      Z     ??     
> 0:00.00 (open)
>   501  9816  9292   0  31  0    18052    244 -      S+    p1     
> 0:00.01 grep 9800

Note the new process, pid 9813 owned by pid 9800.  Also note that the  
10th field of that line is a "Z", which indicates a zombie process  
(this info is available in the man page for ps).  This process is not  
shown in the Activity Monitor window, even though the "Show" value is  
"All Processes".

Now I go back to LilyPond and do cmd-r three more times:

sully:/ pat$ ps -alx | grep 9800
  501  9800    75   0  46  0   186332  17208 -      S     ??     
0:04.57 /Applications/LilyPond.app/Contents
  501  9813  9800   0   0  0        0      0 -      Z     ??     
0:00.00 (open)
  501  9828  9800   0   0  0        0      0 -      Z     ??     
0:00.00 (open)
  501  9832  9800   0   0  0        0      0 -      Z     ??     
0:00.00 (open)
  501  9836  9800   0   0  0        0      0 -      Z     ??     
0:00.00 (open)
  501  9838  9292   0  31  0    18052    244 -      S+    p1     
0:00.01 grep 9800

So there you have it, a zombie pid for each cmd-r.

Of course this problem might have been fixed in LilyPond 2.13.

Regards,
Pat








> James
>
>
> -----Original Message-----
> From: address@hidden on  
> behalf of Patrick Karl
> Sent: Mon 01/02/2010 20:27
> To: address@hidden
> Subject: pid leakage in LilyPond
>
> I have noticed a resource problem in LilyPond, namely invocation of
> the "Compile/Typeset file" command in interactive mode always
> consumes one more pid.  When I enter music, I incrementally compile
> it as I go.  I do my work on a Mac running OS 10.4.  There are a
> fixed number of pids available for all the programs running on the
> Mac.  This means that as I continue to incrementally compile in
> LilyPond, the number of available pids grows less, eventually running
> out.
>
> If LilyPond were a C program, this behavior could be caused by
> executing an "exec" call to generate a child process but not later
> waiting for the child to complete.  This means that when the child
> process does complete, it becomes what is called a zombie or defunct
> pid.  I suspect something similar is happening in LilyPond.
>
>
> _______________________________________________
> lilypond-user mailing list
> address@hidden
> http://lists.gnu.org/mailman/listinfo/lilypond-user