monit-general
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Avoid alert on log rotation


From: Martin Pala
Subject: Re: Avoid alert on log rotation
Date: Fri, 6 Nov 2015 12:05:43 +0100

Hello,

you can postpone the action this way:

--8<--
check process myprocess with pidfile /var/run/myprocesspid.pid
if does not exist  for 2 cycles then restart  # wait for two cycles before restarting the process
--8<--

It is definitely not against monit goal that if the process is to be intentionally stopped, monit must also be notified about it to not try to restart the process automatically as from monit's point of view the process died. It is the same as if the process will run under upstart, systemd or sun cluster control: if it will be killed, upstart/systemd/suncluster will also restart it when they detect that the process died as they don't "know" that the service was intentionally stopped by external action intentionally (by logrotate in this case).

The monitoring is suppressed just for the timeframe where the service is under by 3rd party maintenance and enabled again when it's safe to monitor again. It is also possible to stop/start the service via monit instead just unmonitor/monitor.


Regards,
Martin


On 05 Nov 2015, at 17:35, Guillaume François <address@hidden> wrote:

Hello,

So we should also concluded there is currently no way to delayed/avoid the first alerts and that the only possibility is to "unmonitor" which can cause issues and is against goal of Monit ?

Explanation: if we "unmonitor" the check (sometime process, sometime program) and an issue prevent the script in charge of "monitoring" again to successfully finished, the service will be down and we won't get alerted as it is "unmonitored".

In fact "Log Rotation" was only one simple possible restart cause, but we have lots of custom scripts also for our custom tools like "restart if some conditions are met" with help of monit process exit code.

Maybe a workaround would be to cut the action script in several part so that even if the restart is failing, the monitoring is still issued.

Another workaround would be to check the status of the "monit check" and if this status is "unmonitor" for several cycles, force "monitoring" again in order to be notified that something bad happens.

What is your point of view about these ?

Best Regards.


2015-11-05 12:31 GMT+01:00 Martin Pala <address@hidden>:
Hello,

monit tests if the process is running - it cannot differentiate that the process was stopped intentionally by 3rd party action ... if you shutdown the process intentionally during logrotate, you should disable its monitoring first (part of logratotation "prerotate" script):

        monit unmonitor myprocess

and when logratotation finished, enable it again (part of logratotation "postrotate" script):

        monit monitor myprocess



Regards,
Martin



> On 05 Nov 2015, at 11:42, Guillaume François <address@hidden> wrote:
>
> Hello,
>
> Everyday we got an "non-exist" then "exist" alert when the process is shutdown then started for log rotation purpose.
>
> We would like to NOT receive the email linked to this kind of behavior but still get the email when the process is really down for some time.
>
> We would like something like
>
> check process myprocess with pidfile /var/run/myprocesspid.pid
>         start program = "/usr/sbin/service myprocess start"
>         stop program  = "/usr/sbin/service myprocess stop"
>         if failed port 5555 protocol tcp for 3 cycles then restart
>         if 5 restarts within 5 cycles then timeout
> wanted ->        alert on {exist, non-exist} only if 5 event in 5 cycles.
>
> can be interpreted as -> only send alert for event 'exist' and event 'non-exist' if alert is triggerred 5 time in the last 5 cycles
>
> We saw we can change the 'alert/noalert' rule for some 'email/event' but we cannot get a "conditional" rules for theses.
>
> Best Regards,
>
>
> --
> -----BEGIN GEEK CODE BLOCK-----
> Version: 3.1
> GCS/IT d(+) s++:- a C++$ ULC(+)>+++$ !P--- L+>$ !E---? W+++$ !N* !o-- K--? w(+) !O---? !M- !V--? PS+? !PE Y+ PGP++>+++ !t-- !5 !X- R(+)>++* tv-? b(-) DI !D- G(+)>+ e+++ h--() r->$ y?*
> ------END GEEK CODE BLOCK------
> --
> To unsubscribe:
> https://lists.nongnu.org/mailman/listinfo/monit-general


--
To unsubscribe:
https://lists.nongnu.org/mailman/listinfo/monit-general



--
-----BEGIN GEEK CODE BLOCK-----
Version: 3.1
GCS/IT d(+) s++:- a C++$ ULC(+)>+++$ !P--- L+>$ !E---? W+++$ !N* !o-- K--? w(+) !O---? !M- !V--? PS+? !PE Y+ PGP++>+++ !t-- !5 !X- R(+)>++* tv-? b(-) DI !D- G(+)>+ e+++ h--() r->$ y?*
------END GEEK CODE BLOCK------
--
To unsubscribe:
https://lists.nongnu.org/mailman/listinfo/monit-general


reply via email to

[Prev in Thread] Current Thread [Next in Thread]