Re: [monit-dev] Assertion failure in 4.10.1

[Brian Candler]

Additional informaion:

- I have just built monit from cvs, and I get the same problem

- The problem occurs even if I remove the non-existent processes
  (testapp_mongrel_1, testapp_mongrel_2)

- I can give you a backtrace from gdb:

# gdb ~candlerb/cvs/monit/monit
GNU gdb Red Hat Linux (6.3.0.0-1.143.el4rh)
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i386-redhat-linux-gnu"...Using host libthread_db 
library "/lib/tls/libthread_db.so.1".

(gdb) run -Iv -c /etc/monit.conf
Starting program: /home/candlerb/cvs/monit/monit -Iv -c /etc/monit.conf
[Thread debugging using libthread_db enabled]
[New Thread -1208420672 (LWP 23493)]
monit: Debug: Adding host allow 'localhost'
monit: Debug: Skipping redundant host 'localhost'
monit: Debug: Skipping redundant host 'localhost'
monit: Debug: Adding net allow 'X.X.X.0/255.255.252.0'.
Runtime constants:
 Control file       = /etc/monit.conf
 Log file           = syslog
 Pid file           = /var/run/monit.pid
 Debug              = True
 Log                = True
 Use syslog         = True
 Is Daemon          = True
 Use process engine = True
 Poll time          = 30 seconds
 Mail server(s)     = localhost:25 with timeout 5 seconds
 Mail from          = address@hidden
 Mail subject       = monit alert --  $EVENT $SERVICE
 Mail message       = $EVENT Service $SERV..(truncated)
 Start monit httpd  = True
 httpd bind address = Any/All
 httpd portnumber   = 2812
 httpd signature    = True
 Use ssl encryption = False
 httpd auth. style  = Host/Net allow list
 Alert mail to      = address@hidden
   Alert on         = Nonexist Timeout

The service list contains the following entries:

Process Name          = testapp_pen
 Group                = testapp
 Pid file             = /var/run/pen/testapp.pid
 Monitoring mode      = active
 Start program        = '/usr/bin/pen -F /etc/pen.d/testapp.conf -u nobody -p 
/var/run/pen/testapp.pid -C 127.0.0.1:9999 127.0.0.1:10000' timeout 30 second(s)
 Stop program         = '/bin/bash -c kill -s SIGTERM `cat 
/var/run/pen/testapp.pid`' timeout 30 second(s)
 Depends on Service   = testapp_pen.conf
 Pid                  = if changed 1 times within 1 cycle(s) then alert
 Ppid                 = if changed 1 times within 1 cycle(s) then alert
 CPU usage limit      = if greater than 50.0% 2 times within 2 cycle(s) then 
restart else if passed 1 times within 1 cycle(s) then alert
 Memory amount limit (incl. children) = If greater than 10240 2 times within 2 
cycle(s) then restart else if passed 1 times within 1 cycle(s) then alert
 Timeout              = If 2 restart within 3 cycles then unmonitor else if 
passed then alert

File Name             = testapp_pen.conf
 Path                 = /etc/pen.d/testapp.conf
 Monitoring mode      = active
 Checksum             = if changed MD5 1 times within 1 cycle(s) then restart

Process Name          = memcached
 Pid file             = /var/run/memcached/memcached.pid
 Monitoring mode      = active
 Start program        = '/etc/init.d/memcached start' timeout 30 second(s)
 Stop program         = '/etc/init.d/memcached stop' timeout 30 second(s)
 Pid                  = if changed 1 times within 1 cycle(s) then alert
 Ppid                 = if changed 1 times within 1 cycle(s) then alert
 CPU usage limit      = if greater than 80.0% 4 times within 4 cycle(s) then 
restart else if passed 1 times within 1 cycle(s) then alert

Process Name          = apache
 Group                = server
 Pid file             = /var/run/httpd.pid
 Monitoring mode      = active
 Start program        = '/etc/init.d/httpd start' timeout 30 second(s)
 Stop program         = '/etc/init.d/httpd stop' timeout 30 second(s)
 Depends on Service   = httpd.conf.d
 Depends on Service   = httpd.conf
 Pid                  = if changed 1 times within 1 cycle(s) then alert
 Ppid                 = if changed 1 times within 1 cycle(s) then alert
 CPU usage limit      = if greater than 95.0% 3 times within 3 cycle(s) then 
restart else if passed 1 times within 1 cycle(s) then alert
 Children             = If greater than 255 5 times within 5 cycle(s) then stop 
else if passed 1 times within 1 cycle(s) then alert
 Memory amount limit (incl. children) = If greater than 102400 1 times within 1 
cycle(s) then alert else if passed 1 times within 1 cycle(s) then alert
 Timeout              = If 2 restart within 3 cycles then unmonitor else if 
passed then alert

File Name             = httpd.conf
 Path                 = /etc/httpd/conf/httpd.conf
 Monitoring mode      = active
 Checksum             = if changed MD5 1 times within 1 cycle(s) then exec 
'/etc/init.d/httpd graceful' timeout 0 cycle(s)

Directory Name        = httpd.conf.d
 Path                 = /etc/httpd/conf.d
 Monitoring mode      = active
 Timestamp            = if changed 1 times within 1 cycle(s) then exec 
'/etc/init.d/httpd graceful' timeout 0 cycle(s)

System Name           = localhost.localdomain
 Monitoring mode      = active

-------------------------------------------------------------------------------
Starting monit daemon with http interface at [*:2812]
Starting monit HTTP server at [*:2812]
[New Thread -1208423520 (LWP 23496)]
monit HTTP server started
'localhost.localdomain' Monit started
'testapp_pen.conf' file existence check passed
'testapp_pen.conf' is a regular file
'testapp_pen.conf' checksum has not changed
'memcached' zombie check passed [status_flag=0000]
'memcached' cpu usage check passed [current cpu usage=0.0%]
'httpd.conf' file existence check passed
'httpd.conf' is a regular file
'httpd.conf' checksum has not changed
'httpd.conf.d' directory existence check passed
'httpd.conf.d' is directory
'httpd.conf.d' timestamp was not changed for /etc/httpd/conf.d
'testapp_pen' zombie check passed [status_flag=0000]
'testapp_pen' cpu usage check passed [current cpu usage=0.0%]
'testapp_pen' total mem amount check passed [current total mem amount=700kB]
'apache' zombie check passed [status_flag=0000]
'apache' cpu usage check passed [current cpu usage=0.0%]
'apache' children check passed [current children=9]
'apache' total mem amount check passed [current total mem amount=22560kB]
'localhost.localdomain' Monit has not changed
'testapp_pen.conf' file existence check passed
'testapp_pen.conf' is a regular file
'testapp_pen.conf' checksum was changed for /etc/pen.d/testapp.conf
'testapp_pen.conf' trying to restart
Monitoring disabled -- service testapp_pen
'testapp_pen' stop: /bin/bash -c kill -s SIGTERM `cat /var/run/pen/testapp.pid`
Detaching after fork from child process 23499.
monit: pidfile '/var/run/pen/testapp.pid' does not exist
monit: pidfile '/var/run/pen/testapp.pid' does not exist
Monitoring disabled -- service testapp_pen.conf
monit: pidfile '/var/run/pen/testapp.pid' does not exist
'testapp_pen' start: /usr/bin/pen -F /etc/pen.d/testapp.conf -u nobody -p 
/var/run/pen/testapp.pid -C 127.0.0.1:9999 127.0.0.1:10000
monit: pidfile '/var/run/pen/testapp.pid' does not exist
Detaching after fork from child process 23503.
/var/www/html/pen/testapp.html
Monitoring enabled -- service testapp_pen
AssertException: s at xmalloc.c:110
aborting..

Program received signal SIGABRT, Aborted.
[Switching to Thread -1208420672 (LWP 23493)]
0x005c27a2 in _dl_sysinfo_int80 () from /lib/ld-linux.so.2
(gdb) bt
#0  0x005c27a2 in _dl_sysinfo_int80 () from /lib/ld-linux.so.2
#1  0x006037a5 in raise () from /lib/tls/libc.so.6
#2  0x00605209 in abort () from /lib/tls/libc.so.6
#3  0x0805db87 in xstrdup (s=Variable "s" is not available.
) at xmalloc.c:110
#4  0x0805d9b6 in check_file (s=0x9804448) at validate.c:992
#5  0x0805b2c7 in validate () at validate.c:179
#6  0x080519b1 in main (argc=4, argv=0xbfe7b7b4) at monitor.c:482
(gdb) info threads
  2 Thread -1208423520 (LWP 23496)  0x005c27a2 in _dl_sysinfo_int80 ()
   from /lib/ld-linux.so.2
* 1 Thread -1208420672 (LWP 23493)  0x005c27a2 in _dl_sysinfo_int80 ()
   from /lib/ld-linux.so.2
(gdb) thread 2
[Switching to thread 2 (Thread -1208423520 (LWP 23496))]#0  0x005c27a2 in 
_dl_sysinfo_int80 () from /lib/ld-linux.so.2
(gdb) bt
#0  0x005c27a2 in _dl_sysinfo_int80 () from /lib/ld-linux.so.2
#1  0x0069d681 in ___newselect_nocancel () from /lib/tls/libc.so.6
#2  0x08051fcd in can_read (socket=3, timeout=1) at net.c:474
#3  0x08062fdf in start_httpd (port=2812, backlog=10, bindAddr=0x0)
    at http/engine.c:624
#4  0x0804fb49 in thread_wrapper (arg=0x0) at http.c:164
#5  0x0083d3cc in start_thread () from /lib/tls/libpthread.so.0
#6  0x006a4c3e in clone () from /lib/tls/libc.so.6
(gdb)

Anything else you need?

Regards,

Brian.