monit-general
[Top][All Lists]
Advanced
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Monit did not restart mysql

From: frwa onto
Subject: Re: Monit did not restart mysql
Date: Tue, 4 Jun 2013 21:29:29 +0800
Dear Martin,
                 Thank you for the start+stop command. The issue now when I do this monit stop mysqld and monit start mysqld I get email for both events. Why when I monit stop mysqld it does not show up message as how I do like this service mysqld stop? Secondly once I monit stop mysqld it did not restart? I have just uncomment this line set logfile syslog facility log_daemon. Then I did was initctl stop and then initctl start mysql. Next when I did this monit -v I can see now Log file = syslog. Is this correctly done is better to specific my own log file say /usr/local/monit.log. Thank you.

Regards,
Frwa.



On Tue, Jun 4, 2013 at 4:50 PM, Martin Pala <address@hidden> wrote:
1.) these are OK:

Stop program = '/etc/init.d/mysqld stop'

Stop program = '/sbin/stop mysqld'


2.) this is NOT OK:

Stop program = '/usr/bin/monit stop mysqld'


Regards,
Martin



On Jun 4, 2013, at 10:46 AM, frwa onto <address@hidden> wrote:

Dear Martin,
                  Why I cant use service mysqld stop and start for the start+stop command? Am I using the correct one? Thank you.

Regards,
Frwa.


On Tue, Jun 4, 2013 at 4:38 PM, Martin Pala <address@hidden> wrote:
No. If you'll change the start+stop commands inside monit to "monit <action> <service>" then monit will enter endless loop (given by the configuration) and recursively call itself. The start/stop commands have to be able to start/stop the service.

Regards,
Martin


On Jun 4, 2013, at 10:35 AM, frwa onto <address@hidden> wrote:

Dear Martin,
                  Thank you for you input yes I was always stopping it via service mysqld stop which is wrong here then. How about these ? Should I also change them to monit start mysql and monit stop mysql?
c timeout 30 second(s)
 Stop program         = '/etc/init.d/mysqld stop' timeout 30 second(s)


Regards,
Frwa.


On Tue, Jun 4, 2013 at 2:01 AM, Martin Pala <address@hidden> wrote:
If the service is monitored by monit, the stop/start should be done via monit. If you'll stop mysql via upstart directly ("stop mysqld") wihtout letting monit know about the stop, then monit will try to restart the process (if the pidfile will be removed  or the given PID doesn't exist). The correct way to stop monitored process is either: "monit stop mysqld" or "monit unmonitor mysql" (which disabled the monitoring) - then you can do any manual action without triggering monit automatic recovery.

The "1 times within 1 cycle" is just verbose expansion of then testing rule and means "perform action on first error" - and as the "cycle" word hints, it's based on the monit cycle length (configured with "set daemon X"). If you need more "solid" error  before the action (such as 8 failures within 10 cycles), then you can use the extended options.

See monit manual for more details: http://www.mmonit.com/monit/documentation/monit.html

Regards,
Martin



On Jun 3, 2013, at 6:31 AM, frwa onto <address@hidden> wrote:

Dear Martin,
                 Sorry kind of new to using the features. I did monit -v so I found there is nt log file enabled. If you notice I have now changed it to 127.0.0.1. So what will be on the look for this file right /var/run/mysqld/mysqld.pid ? Say when I stop mysql will this file will be gone. So then what will monit do run the stop is it? Then run the start? Can I ask for run and stop to be service mysqld stop and service mysqld start? Kind of confuse with these statement "with timeout 5 seconds and retry 0 time(s) 1 times within 1 cycle(s) then restart else if succeeded 1 times within 1 cycle(s) then alert" What is the 1 times within 1 cycle is it within the 1 minute of polling? Thank you.

monit -v
monit: Debug: Adding host allow 'localhost'
monit: Debug: Skipping redundant host 'localhost'
monit: Debug: Skipping redundant host 'localhost'
monit: Debug: Skipping redundant host 'localhost'
monit: Debug: Skipping redundant host 'localhost'
monit: Debug: Skipping redundant host 'localhost'
monit: Debug: Adding credentials for user 'admin'.
monit: Adding PAM group 'monit'.
monit: Adding PAM group 'users'.
Runtime constants:
 Control file       = /etc/monit.conf
 Log file           = (not defined)
 Pid file           = /var/run/monit.pid
 Id file            = /var/monit/id
 Debug              = True
 Log                = False
 Use syslog         = False
 Is Daemon          = True
 Use process engine = True
 Poll time          = 60 seconds with start delay 0 seconds
 Expect buffer      = 256 bytes
 Mail server(s)     = smtp.gmail.com:587(ssl) with timeout 30 seconds
 Mail from          = (not defined)
 Mail subject       = (not defined)
 Mail message       = (not defined)
 Start monit httpd  = True
 httpd bind address = localhost
 httpd portnumber   = 2812
 httpd signature    = True
 Use ssl encryption = False
 httpd auth. style  = Basic Authentication and Host/Net allow list
 Alert mail to      = address@hidden
   Alert on         = All events
 Alert mail to      = address@hidden
   Alert on         = All events

The service list contains the following entries:

Process Name          = mysqld
 Pid file             = /var/run/mysqld/mysqld.pid
 Monitoring mode      = active
 Start program        = '/etc/init.d/mysqld start' timeout 30 second(s)
 Stop program         = '/etc/init.d/mysqld stop' timeout 30 second(s)
 Existence            = if does not exist 1 times within 1 cycle(s) then restart else if succeeded 1 times within 1 cycle(s) then alert
 Pid                  = if changed 1 times within 1 cycle(s) then alert
 Ppid                 = if changed 1 times within 1 cycle(s) then alert
 Port                 = if failed [127.0.0.1:3306 [MYSQL via TCP] with timeout 5 seconds and retry 0 time(s) 1 times within 1 cycle(s) then restart else if succeeded 1 times within 1 cycle(s) then alert
 Timeout              = If restarted 5 times within 5 cycle(s) then unmonitor

System Name           = localhost.localdomain
 Monitoring mode      = active

-------------------------------------------------------------------------------
monit daemon with PID 8592 awakened



On Sun, Jun 2, 2013 at 4:43 PM, Martin Pala <address@hidden> wrote:
Hi,

please check monit logs and "monit status" for mysqld. You can run monit in debug mode using "-v" option to see more details.

From the description i'm not sure exactly what happened - it seems that mysql was running and you tried to kill/stop it to see whether monit will recover it? As you have two tests, the mysql error should be detected when the process stops (i.e. the PID in the /var/run/mysqld/mysqld.pid file is not present in the process table) and/or if the connection fails.

As Sven correctly pointed out the 127.0.1.1 is wrong, so the connection test probably always failed. Because you have restart action on connection failure, monit will try to restart it - and since there is also the timeout statement, it'll give up after 5 attempts and will disable the service monitoring.

My guess is, that the misconfigured connection test (127.0.1.1 instead of 127.0.0.1) caused monit to restart mysql five times (as the test failed after the restart too), which led to service unmonitoring => monit most probably doesn't watch mysql anymore and if you killed it, it won't restart it.

The solution if this state is to fix the configuration (127.0.1.1 -> 127.0.0.1) and enable mysql monitoring again: "monit monitor mysqld"


Regards,
Martin



On Jun 2, 2013, at 10:19 AM, frwa onto <address@hidden> wrote:

Hi,
    I have tried both 127.0.1.1 and 127.0.0.1 but yet its not working what else could be done?


On Sun, Jun 2, 2013 at 12:58 AM, frwa onto <address@hidden> wrote:
Sorry I got mixed up what different will 127.0.1.1 make indeed? Thank you .


On Sun, Jun 2, 2013 at 12:55 AM, frwa onto <address@hidden> wrote:
What ROFL ?


On Sun, Jun 2, 2013 at 12:44 AM, sven falempin <address@hidden> wrote:

ROFL ROFL

WELCOME TO FAIL
ProTip stop using your brain use CTRL+F or /



On Sat, Jun 1, 2013 at 11:03 AM, frwa onto <address@hidden> wrote:
Hi Sven,
            I already have it as 127.0.0.1 see below
check process mysqld with pidfile /var/run/mysqld/mysqld.pid
   start program = "/etc/init.d/mysqld start"
   stop program = "/etc/init.d/mysqld stop"
   if failed host 127.0.1.1 port 3306 protocol mysql then restart
   if 5 restarts within 5 cycles then timeout


On Sat, Jun 1, 2013 at 8:25 PM, sven falempin <address@hidden> wrote:
127.0.1.1

xd


On Sat, Jun 1, 2013 at 2:58 AM, frwa onto <address@hidden> wrote:
Hi All,
        We have this in our monit.conf. Earlier it was working smoothly when mysql goes down. Recently I just tried to stop mysqld and the monit did not restart it. But in the top I can see monit is running and I am using upstart so I just kill and it respawn again. Below is part of the config file.


check process mysqld with pidfile /var/run/mysqld/mysqld.pid
   start program = "/etc/init.d/mysqld start"
   stop program = "/etc/init.d/mysqld stop"
   if failed host 127.0.1.1 port 3306 protocol mysql then restart
   if 5 restarts within 5 cycles then timeout


Regards,
Frwa.

--
To unsubscribe:
https://lists.nongnu.org/mailman/listinfo/monit-general



--
---------------------------------------------------------------------------------------------------------------------
() ascii ribbon campaign - against html e-mail 
/\ 

--
To unsubscribe:
https://lists.nongnu.org/mailman/listinfo/monit-general


--
To unsubscribe:
https://lists.nongnu.org/mailman/listinfo/monit-general



--
---------------------------------------------------------------------------------------------------------------------
() ascii ribbon campaign - against html e-mail 
/\ 

--
To unsubscribe:
https://lists.nongnu.org/mailman/listinfo/monit-general



--
To unsubscribe:
https://lists.nongnu.org/mailman/listinfo/monit-general


--
To unsubscribe:
https://lists.nongnu.org/mailman/listinfo/monit-general

--
To unsubscribe:
https://lists.nongnu.org/mailman/listinfo/monit-general


--
To unsubscribe:
https://lists.nongnu.org/mailman/listinfo/monit-general

--
To unsubscribe:
https://lists.nongnu.org/mailman/listinfo/monit-general




--
To unsubscribe:
https://lists.nongnu.org/mailman/listinfo/monit-general

reply via email to
[Prev in Thread] Current Thread [Next in Thread]