help-cfengine
[Top][All Lists]
Advanced
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Possible bug: parsing/scheduling of editfiles sections in multiple cfeng

From: Berthold Cogel
Subject: Possible bug: parsing/scheduling of editfiles sections in multiple cfengine scripts
Date: Fri, 02 Sep 2005 13:40:56 +0200
User-agent: Mozilla Thunderbird 1.0.2-1.4.1.SL3 (X11/20050324) 
Hello!
We have seen here some strange behaviour in editfile sections in our configuration. Somehow the internal task schedule gets mixed up. 


Our Setup:
----------
cfengine version:       2.1.15
OS version:             Red Hat Enterprise Linux 3

update.conf     General update mechanism (get cfengine config from
                server)
cfagent.conf    Controls local update mechanism, includes files:
                cf.class, cf.common, cf.def, cf.update, cf.main

cf.class        Defines global managment classes, includes per-service
                definitions (*.def)
cf.common       Defines paths, plattform specific
cf.def          Defines evaluated classes (cf.eval might be a better
                name) like HOST_IP, HOST_IF (primary interface),
                HOST_NET
                (primary network) for the use in edit sections.
cf.update       Get configuration files and cfengine scripts from cfengine
                server, separated per service to deactivate distribution for
                a single service if needed.
cf.main         Includes per-service configuration (*.cf)

*.def           Maps global management and machine classes to service
                specific classes. As a result the service scripts (*.cf)
                are virtualy independant from management and machine
                classes. Additional class <service>_DEBUG defaults to
                'none'. For debugging this class can be set for single
                machines, sets of management classes or all computers
                (any).

*cf             Script for each service. Includes debug framework:

                control:

                        AllowRedefinitionOf = ( debug_dir )
                        debug_dir           = ( "" )
                        actionsequence      = ( directories ... copy            
                                                .... )

                        <service>_DEBUG::
                        debug_dir           = ( "/service/cfdebug" )


                copy:

                        <service_class>::
                        ${sourcedir}/configfile
                                dest=${debugdir}${targetdir}/configfile

                directories:
                        <service>_DEBUG::
                        ${debug_dir}${config_dir}       owner=root
                                                        group=root
                                                        mode=0700
                                                        
                If debugging is 'on' for a service for a specific
                machine, configfile is copied to '/service/cfdebug/etc'
                if targetdir is '/etc'.
                Changes in configurations or new configurations can be
                tested on the real machines.

service         Can be anything: a real service like sshd or the simple
                distribution of a shellscript.
At the moment we are distributing only parts of our configuration with cfengine. But we integrate more piece by piece. 


Problem:
--------
Instead of distributing a global resolv.conf, I want to generate an optimized file depending on wether a machine is a nameserver or has a nameserver in it's local network. I added an editfiles section to the script that is responsible for the distribution of resolv.conf (net.cf). This editfiles section will be evaluated only on one machine in debug mode.
As long as the actions equence for the responsible cf script *does not* include the editfiles statement, everything works as expected. But as soon as I include editfiles in the actionsequence, almost every editfile sequence in our configuration goes berserk.
We normaly copy a masterfile to a temporary location, edit the file and copy it to it's final destination to avoid unnecessary changes of valid configuration files. 

actionsequence = ( ... copy.pre editfiles copy.post ... copy ... )
In this case (net.cf) I simply generate a file by editing a 'new' file (AutoCreate). Because it's a test, the final copy to the destination is still missing. But there are other files that are copied. 

actionsequence = ( ... editfiles ... copy ... )
On *all* machines I see failures for already defined edit sequences. If I remove only(!) the edifiles statement in the above action sequence, everything works as expected. The editfile block in the script remains. 



Debugging:
----------

Defined actionsequences in various cf-scripts:

cert.cf:        ( directories copy )
ganglia.cf:     ( directories copy copy.pre editfiles copy.post
                shellcommands processes )
krb.cf:         ( directories copy )
logrotate.cf:   ( directories copy )
mail.cf:        ( directories copy.pre editfiles copy.post copy
                shellcommands processes )
net.cf:         ( directories copy )
scripts.cf:     ( directories copy.pre editfiles copy.post copy
                shellcommands )
sophos.cf:      ( directories copy )
ssh.cf:         ( directories files copy.pre editfiles copy.post copy
                shellcommands processes )
tsm.cf:         ( directories links copy copy.pre editfiles copy.post

Generated over all action sequence:
(processes directories copy directories copy directories copy.pre editfiles copy.post copy shellcommands directories copy copy.pre editfiles copy.post shellcommands processes directories links copy copy.pre editfiles copy.post shellcommands processes directories copy.pre editfiles copy.post copy shellcommands processes directories copy directories files copy.pre editfiles copy.post copy shellcommands processes directories copy directories copy directories copy ) 

All edit sections are executed in the first editfiles slot above.



Now I change net.cf to ( directories editfiles copy ). I get this sequence:
(processes directories copy directories copy directories editfiles copy.pre editfiles copy.post copy shellcommands directories copy copy.pre editfiles copy.post shellcommands processes directories links copy copy.pre editfiles copy.post shellcommands processes directories copy.pre editfiles copy.post copy shellcommands processes directories copy directories files copy.pre editfiles copy.post copy shellcommands processes directories copy directories copy directories copy )
Still all edit sections are executed in the first editfiles slot. But now this is done previous to copy.pre.
What happens now is that edited files from a previous run get searched for patterns that already have been replaced. After this, those files are overwritten by the original templates, which are then copied to the destination *without* editing. If a service has to be restarted due to configuration changes, this leads to failures for those services. 



Debug output for the correct behavior:
--------------------------------------
Unnecessary information is left out. Only relevant parts and scheduling informations from 'cfagent -v -q -d3'. 

<cut>

------------------------------------------------------------

DEFINED FILE EDITS

EDITFILE  /var/cfengine/config/tmp/gmond.conf.edit (n)(r=0)
 Context scope: main
 IfElapsed=1, ExpireAfter=120
 Backup [off] if (GANGLIA_RRZK)
 LocateLineMatching [#%GANGLIAGROUP%#] if (GANGLIA_RRZK)
 ReplaceLineWith [name  "Management"] if (GANGLIA_RRZK)
 LocateLineMatching [#%GANGLIAPORT%#] if (GANGLIA_RRZK)
 ReplaceLineWith [mcast_port  8650] if (GANGLIA_RRZK)
 LocateLineMatching [#%GANGLIAIF%#] if (GANGLIA_RRZK)
 ReplaceLineWith [mcast_if  eth2] if (GANGLIA_RRZK)

------------------------------------------------------------
<cut>

 Main Tree Sched: processes pass 1 @ Fri Sep  2 09:44:53 2005
 Main Tree Sched: directories pass 1 @ Fri Sep  2 09:44:53 2005
 Main Tree Sched: copy pass 1 @ Fri Sep  2 09:44:53 2005
 Main Tree Sched: directories pass 1 @ Fri Sep  2 09:44:53 2005
 Main Tree Sched: copy pass 1 @ Fri Sep  2 09:44:53 2005
 Main Tree Sched: directories pass 1 @ Fri Sep  2 09:44:53 2005
 Main Tree Sched: copy.pre pass 1 @ Fri Sep  2 09:44:53 2005
*********************************************************************


                  New temporary class additions
                  --------( Pass 1 )-------
                             pre
Checking copy from localhost:/var/cfengine/config/ganglia/gmond.conf to /var/cfengine/config/tmp/gmond.conf.edit !: Update of image /var/cfengine/config/tmp/gmond.conf.edit from master /var/cfengine/config/ganglia/gmond.conf on localhost !: Moved /var/cfengine/config/tmp/gmond.conf.edit.cfsaved to repository location /var/cfengine/backup/_var_cfengine_config_tmp_gmond.conf.edit.cfsaved !: Object /var/cfengine/config/tmp/gmond.conf.edit had permission 600, changed it to 644 


*********************************************************************
 Main Tree Sched: editfiles pass 1 @ Fri Sep  2 09:44:53 2005
*********************************************************************

Begin editing /var/cfengine/config/tmp/gmond.conf.edit
Edit: Search ended at line 25
Edit: (Found #%GANGLIAGROUP%#)
Replacing line 25 with name  "Management"...
Edit: Search ended at line 58
Edit: (Found #%GANGLIAPORT%#)
Replacing line 58 with mcast_port  8650...
Edit: Search ended at line 64
Edit: (Found #%GANGLIAIF%#)
Replacing line 64 with mcast_if  eth2...
End editing /var/cfengine/config/tmp/gmond.conf.edit
.....................................................................
!: Saving edit changes to file /var/cfengine/config/tmp/gmond.conf.edit

*********************************************************************
 Main Tree Sched: copy.post pass 1 @ Fri Sep  2 09:44:54 2005
*********************************************************************


                  New temporary class additions
                  --------( Pass 1 )-------
                             post
Checking copy from localhost:/var/cfengine/config/tmp/gmond.conf.edit to /etc/gmond.conf 

*********************************************************************
 Main Tree Sched: copy pass 1 @ Fri Sep  2 09:44:54 2005
 Main Tree Sched: shellcommands pass 1 @ Fri Sep  2 09:44:54 2005
 Main Tree Sched: directories pass 1 @ Fri Sep  2 09:44:54 2005
 Main Tree Sched: copy pass 1 @ Fri Sep  2 09:44:54 2005
 Main Tree Sched: copy.pre pass 1 @ Fri Sep  2 09:44:54 2005
*********************************************************************


                  New temporary class additions
                  --------( Pass 1 )-------
                             pre

*********************************************************************
 Main Tree Sched: editfiles pass 1 @ Fri Sep  2 09:44:54 2005
 Main Tree Sched: copy.post pass 1 @ Fri Sep  2 09:44:54 2005
*********************************************************************


                  New temporary class additions
                  --------( Pass 1 )-------
                             post

*********************************************************************
 Main Tree Sched: shellcommands pass 1 @ Fri Sep  2 09:44:54 2005
 Main Tree Sched: processes pass 1 @ Fri Sep  2 09:44:54 2005
 Main Tree Sched: directories pass 1 @ Fri Sep  2 09:44:54 2005
 Main Tree Sched: links pass 1 @ Fri Sep  2 09:44:54 2005
 Main Tree Sched: copy pass 1 @ Fri Sep  2 09:44:54 2005
 Main Tree Sched: copy.pre pass 1 @ Fri Sep  2 09:44:54 2005
*********************************************************************


                  New temporary class additions
                  --------( Pass 1 )-------
                             pre

*********************************************************************
 Main Tree Sched: editfiles pass 1 @ Fri Sep  2 09:44:54 2005
*********************************************************************


*********************************************************************
 Main Tree Sched: copy.post pass 1 @ Fri Sep  2 09:44:54 2005
*********************************************************************


                  New temporary class additions
                  --------( Pass 1 )-------
                             post

*********************************************************************
 Main Tree Sched: shellcommands pass 1 @ Fri Sep  2 09:44:54 2005
 Main Tree Sched: processes pass 1 @ Fri Sep  2 09:44:54 2005
 Main Tree Sched: directories pass 1 @ Fri Sep  2 09:44:54 2005
 Main Tree Sched: copy.pre pass 1 @ Fri Sep  2 09:44:54 2005
*********************************************************************


                  New temporary class additions
                  --------( Pass 1 )-------
                             pre

*********************************************************************
 Main Tree Sched: editfiles pass 1 @ Fri Sep  2 09:44:54 2005
 Main Tree Sched: copy.post pass 1 @ Fri Sep  2 09:44:54 2005
*********************************************************************


                  New temporary class additions
                  --------( Pass 1 )-------
                             post

*********************************************************************
 Main Tree Sched: copy pass 1 @ Fri Sep  2 09:44:54 2005
 Main Tree Sched: shellcommands pass 1 @ Fri Sep  2 09:44:54 2005
 Main Tree Sched: processes pass 1 @ Fri Sep  2 09:44:54 2005
 Main Tree Sched: directories pass 1 @ Fri Sep  2 09:44:54 2005
 Main Tree Sched: copy pass 1 @ Fri Sep  2 09:44:54 2005
 Main Tree Sched: directories pass 1 @ Fri Sep  2 09:44:54 2005
 Main Tree Sched: files pass 1 @ Fri Sep  2 09:44:54 2005
 Main Tree Sched: copy.pre pass 1 @ Fri Sep  2 09:44:54 2005
*********************************************************************


                  New temporary class additions
                  --------( Pass 1 )-------
                             pre

*********************************************************************
 Main Tree Sched: editfiles pass 1 @ Fri Sep  2 09:44:54 2005
 Main Tree Sched: copy.post pass 1 @ Fri Sep  2 09:44:54 2005
*********************************************************************


                  New temporary class additions
                  --------( Pass 1 )-------
                             post

*********************************************************************
 Main Tree Sched: copy pass 1 @ Fri Sep  2 09:44:54 2005
 Main Tree Sched: shellcommands pass 1 @ Fri Sep  2 09:44:54 2005
 Main Tree Sched: processes pass 1 @ Fri Sep  2 09:44:54 2005
 Main Tree Sched: directories pass 1 @ Fri Sep  2 09:44:54 2005
 Main Tree Sched: copy pass 1 @ Fri Sep  2 09:44:54 2005
 Main Tree Sched: directories pass 1 @ Fri Sep  2 09:44:54 2005
 Main Tree Sched: copy pass 1 @ Fri Sep  2 09:44:54 2005
 Main Tree Sched: directories pass 1 @ Fri Sep  2 09:44:54 2005
 Main Tree Sched: copy pass 1 @ Fri Sep  2 09:44:54 2005
Alerts
Summary of objects involved

<cut>



Debug output with additional editfile section:
----------------------------------------------
Unnecessary informations are left out. Only relevant parts and scheduling informations from 'cfagent -v -q -d3'. 


<cut>

------------------------------------------------------------

DEFINED FILE EDITS

EDITFILE  /var/cfengine/config/tmp/gmond.conf.edit (n)(r=0)
 Context scope: main
 IfElapsed=1, ExpireAfter=120
 Backup [off] if (GANGLIA_RRZK)
 LocateLineMatching [#%GANGLIAGROUP%#] if (GANGLIA_RRZK)
 ReplaceLineWith [name  "Management"] if (GANGLIA_RRZK)
 LocateLineMatching [#%GANGLIAPORT%#] if (GANGLIA_RRZK)
 ReplaceLineWith [mcast_port  8650] if (GANGLIA_RRZK)
 LocateLineMatching [#%GANGLIAIF%#] if (GANGLIA_RRZK)
 ReplaceLineWith [mcast_if  eth2] if (GANGLIA_RRZK)

------------------------------------------------------------

<cut>

 Main Tree Sched: processes pass 1 @ Fri Sep  2 10:30:48 2005
 Main Tree Sched: directories pass 1 @ Fri Sep  2 10:30:48 2005
 Main Tree Sched: copy pass 1 @ Fri Sep  2 10:30:48 2005
 Main Tree Sched: directories pass 1 @ Fri Sep  2 10:30:49 2005
 Main Tree Sched: editfiles pass 1 @ Fri Sep  2 10:30:49 2005
*********************************************************************

Begin editing /var/cfengine/config/tmp/gmond.conf.edit
Edit: Search ended at line 25
Edit: (Found #%GANGLIAGROUP%#)
Replacing line 25 with name  "Management"...
Edit: Search ended at line 58
Edit: (Found #%GANGLIAPORT%#)
Replacing line 58 with mcast_port  8650...
Edit: Search ended at line 64
Edit: (Found #%GANGLIAIF%#)
Replacing line 64 with mcast_if  eth2...
End editing /var/cfengine/config/tmp/gmond.conf.edit
.....................................................................
!: Saving edit changes to file /var/cfengine/config/tmp/gmond.conf.edit

*********************************************************************
 Main Tree Sched: copy pass 1 @ Fri Sep  2 10:30:49 2005
 Main Tree Sched: directories pass 1 @ Fri Sep  2 10:30:49 2005
 Main Tree Sched: copy.pre pass 1 @ Fri Sep  2 10:30:49 2005
*********************************************************************


                  New temporary class additions
                  --------( Pass 1 )-------
                             pre
Checking copy from localhost:/var/cfengine/config/ganglia/gmond.conf to /var/cfengine/config/tmp/gmond.conf.edit !: Update of image /var/cfengine/config/tmp/gmond.conf.edit from master /var/cfengine/config/ganglia/gmond.conf on localhost !: Moved /var/cfengine/config/tmp/gmond.conf.edit.cfsaved to repository location /var/cfengine/backup/_var_cfengine_config_tmp_gmond.conf.edit.cfsaved !: Object /var/cfengine/config/tmp/gmond.conf.edit had permission 600, changed it to 644 

*********************************************************************
 Main Tree Sched: editfiles pass 1 @ Fri Sep  2 10:30:49 2005
 Main Tree Sched: copy.post pass 1 @ Fri Sep  2 10:30:49 2005
*********************************************************************


                  New temporary class additions
                  --------( Pass 1 )-------
                             post

*********************************************************************
 Main Tree Sched: copy pass 1 @ Fri Sep  2 10:30:49 2005
 Main Tree Sched: shellcommands pass 1 @ Fri Sep  2 10:30:49 2005
 Main Tree Sched: directories pass 1 @ Fri Sep  2 10:30:49 2005
 Main Tree Sched: copy pass 1 @ Fri Sep  2 10:30:49 2005
 Main Tree Sched: copy.pre pass 1 @ Fri Sep  2 10:30:49 2005
*********************************************************************


                  New temporary class additions
                  --------( Pass 1 )-------
                             pre

*********************************************************************
 Main Tree Sched: editfiles pass 1 @ Fri Sep  2 10:30:49 2005
 Main Tree Sched: copy.post pass 1 @ Fri Sep  2 10:30:49 2005
*********************************************************************


                  New temporary class additions
                  --------( Pass 1 )-------
                             post

*********************************************************************
 Main Tree Sched: shellcommands pass 1 @ Fri Sep  2 10:30:49 2005
 Main Tree Sched: processes pass 1 @ Fri Sep  2 10:30:49 2005
 Main Tree Sched: directories pass 1 @ Fri Sep  2 10:30:49 2005
 Main Tree Sched: links pass 1 @ Fri Sep  2 10:30:49 2005
 Main Tree Sched: copy pass 1 @ Fri Sep  2 10:30:49 2005
 Main Tree Sched: copy.pre pass 1 @ Fri Sep  2 10:30:49 2005
*********************************************************************


                  New temporary class additions
                  --------( Pass 1 )-------
                             pre

*********************************************************************
 Main Tree Sched: editfiles pass 1 @ Fri Sep  2 10:30:49 2005
 Main Tree Sched: copy.post pass 1 @ Fri Sep  2 10:30:49 2005
*********************************************************************


                  New temporary class additions
                  --------( Pass 1 )-------
                             post

*********************************************************************
 Main Tree Sched: shellcommands pass 1 @ Fri Sep  2 10:30:49 2005
 Main Tree Sched: processes pass 1 @ Fri Sep  2 10:30:49 2005
 Main Tree Sched: directories pass 1 @ Fri Sep  2 10:30:49 2005
 Main Tree Sched: copy.pre pass 1 @ Fri Sep  2 10:30:49 2005
*********************************************************************


                  New temporary class additions
                  --------( Pass 1 )-------
                             pre

*********************************************************************
 Main Tree Sched: editfiles pass 1 @ Fri Sep  2 10:30:49 2005
 Main Tree Sched: copy.post pass 1 @ Fri Sep  2 10:30:49 2005
*********************************************************************


                  New temporary class additions
                  --------( Pass 1 )-------
                             post

*********************************************************************
 Main Tree Sched: copy pass 1 @ Fri Sep  2 10:30:49 2005
 Main Tree Sched: shellcommands pass 1 @ Fri Sep  2 10:30:49 2005
 Main Tree Sched: processes pass 1 @ Fri Sep  2 10:30:49 2005
 Main Tree Sched: directories pass 1 @ Fri Sep  2 10:30:49 2005
 Main Tree Sched: copy pass 1 @ Fri Sep  2 10:30:49 2005
 Main Tree Sched: directories pass 1 @ Fri Sep  2 10:30:49 2005
 Main Tree Sched: files pass 1 @ Fri Sep  2 10:30:49 2005
 Main Tree Sched: copy.pre pass 1 @ Fri Sep  2 10:30:49 2005
*********************************************************************


                  New temporary class additions
                  --------( Pass 1 )-------
                             pre

*********************************************************************
 Main Tree Sched: editfiles pass 1 @ Fri Sep  2 10:30:49 2005
 Main Tree Sched: copy.post pass 1 @ Fri Sep  2 10:30:49 2005
*********************************************************************


                  New temporary class additions
                  --------( Pass 1 )-------
                             post

*********************************************************************
 Main Tree Sched: copy pass 1 @ Fri Sep  2 10:30:49 2005
 Main Tree Sched: shellcommands pass 1 @ Fri Sep  2 10:30:49 2005
 Main Tree Sched: processes pass 1 @ Fri Sep  2 10:30:49 2005
 Main Tree Sched: directories pass 1 @ Fri Sep  2 10:30:49 2005
 Main Tree Sched: copy pass 1 @ Fri Sep  2 10:30:49 2005
 Main Tree Sched: directories pass 1 @ Fri Sep  2 10:30:49 2005
 Main Tree Sched: copy pass 1 @ Fri Sep  2 10:30:49 2005
 Main Tree Sched: directories pass 1 @ Fri Sep  2 10:30:49 2005
 Main Tree Sched: copy pass 1 @ Fri Sep  2 10:30:49 2005
Alerts
Summary of objects involved

<cut>



Regards,

Berthold Cogel
reply via email to
[Prev in Thread] Current Thread [Next in Thread]