[Savannah-cvs] [201] Initial documentation on setting up mgt0, internal0, frontend0.

[bob]

Revision: 201
          
http://svn.sv.gnu.org/viewvc/?view=rev&root=administration&revision=201
Author:   rwp
Date:     2016-07-26 22:10:28 +0000 (Tue, 26 Jul 2016)
Log Message:
-----------
Initial documentation on setting up mgt0, internal0, frontend0.

Added Paths:
-----------
    trunk/sviki/SavannahHosts.mdwn

Added: trunk/sviki/SavannahHosts.mdwn
===================================================================
--- trunk/sviki/SavannahHosts.mdwn                              (rev 0)
+++ trunk/sviki/SavannahHosts.mdwn      2016-07-26 22:10:28 UTC (rev 201)
@@ -0,0 +1,330 @@
+Savannah VM Hosts
+=================
+
+The current hosts are:
+
+    download
+    frontend
+    internal
+    mgt
+    vcs
+
+These are being migrated to new hosts:
+
+    download0
+    frontend0
+    internal0
+    mgt0
+    vcs0
+
+There have been many years of incomplete cleanup and agressive
+installation over the top of the system.  This has left the system in
+a problematic state.  Many programs have a packaged version installed
+with a locally modified version installed on top of it.  On two of the
+systems the kernels no longer upgrade cleanly.  By migrating onto
+freshly installed systems we gain several benefits.
+
+* We will know what we have installed
+* Fresh installation free of legacy
+* Upgrades will then work reliably again
+
+Common Elements
+---------------
+
+After pristine installation sevaral tasks are needed on any new host.
+Some things are common to all hosts.  These common elements are
+documented here.
+
+File /etc/apt/apt.conf.d/00local prevents Recommends from being
+automatically installed.  These systems should be kept minimal and
+focused on the task at hand without extra peripheral packages.
+
+    Aptitude::Recommends-Important "false";
+    APT::Install-Recommends "0";
+    APT::Get::Show-Upgraded "true";
+    Acquire::PDiffs "false";
+
+File /etc/apt/sources.list will contain the installation cdrom and dvd
+sources.  This file should be cleaned and pointing to the close mirrors.
+
+Update the host to the current packages.
+
+    apt-get update && apt-get upgrade && apt-get dist-upgrade && apt-get 
autoremove --purge && apt-get clean
+
+Set up the `etckeeper` package.  This package will track all changes
+to /etc in version control.  It will also track what packages are
+installed and removed.
+
+the Debian etckeeper package defaults to git and installs without
+error.  Unfortunately the Trisquel etckeeper package defaults to bzr
+and fails to install without errors.  But it can be worked around.
+Install git.  Then install etckeeper.  The installation of etckeeper
+on Trisquel will error and fail.  After installation edit the
+etckeeper.conf file setting the version control system to git.  Then
+manually finish the initialization.
+
+    apt-get install git
+    /root/.gitconfig
+    apt-get install etckeeper
+      ... fix /etc/etckeeper/etckeeper.conf
+        #VCS="git"
+        VCS="bzr"
+      etckeeper init
+      etckeeper commit "Initial commit"
+      etckeeper update-ignore
+
+Install and configure needed locales.
+
+    dpkg-reconfigure locales
+
+The VMs were handed to us including a `/usr/sbin/policy-rc.d` file
+that prevents services from starting during package upgrades.  (It has
+no effect during boot time.)  That was likely a mistake.  That file is
+something that is most often used in chroots.  This file must be
+removed.
+
+    rm -f /usr/sbin/policy-rc.d
+
+Install generally useful utilities.
+
+    apt-get install less screen htop wget dnsutils heirloom-mailx
+
+This includes Emacs.  At the time of this writing version 24 is
+current and that number appears in the package names.  At future times
+this version number will be the current version at that future time.
+
+    apt-get purge jove
+    apt-get install emacs emacs24-nox emacs24-el emacs24-common-non-dfsg 
emacs-goodies-el
+    apt-get install debian-el devscripts-el dpkg-dev-el
+
+    copy in a useful /root/.emacs
+
+Set up /root/.profile appropriately.
+
+    mkdir /root/bin
+
+    /root/.profile # mgt has special copy with reminders
+      LANG=en_US.UTF-8
+      export LANG
+      LC_COLLATE=C
+      export LC_COLLATE
+      PATH=$HOME/bin:$PATH
+
+Set up ssh access:
+
+    /root/.ssh/authorized_keys (add mgt0, chmod u+w)
+
+Bob finds this a required finger memory utility.
+
+    /usr/local/bin/ll
+      #!/bin/sh
+      exec ls -l "$@"
+
+There are no hardwired terminals and therefore clearing the screen
+after exit provides no security but is harmful to use.
+
+    rm -f /etc/skel/.bash_logout
+
+Various miscellaneous fixes.
+
+    ...edit /etc/skel/.profile fix everything
+    ...edit /etc/skel/.bashrc comment out --color
+    ...comment out or remove all pre-defined aliases
+    ...other fixes
+    ...same for /root/.profile /root/.bashrc
+
+Set up /tmp as a tmpfs of moderate size.
+
+    /etc/fstab
+      tmpfs  /tmp  tmpfs  size=1g  0  0
+
+Fixup /etc/network/interfaces for both IPv4 and IPv6.  Also at the
+present time a simple iptables firewall is loaded with a pre-up rule.
+
+    auto lo eth0
+    iface lo inet loopback
+
+    iface eth0 inet static
+      address 208.118.235.XX/24
+      gateway 208.118.235.1
+      pre-up iptables-restore < /etc/default/iptables-rules || :
+
+The above is incomplete as it only has the IPv4 address template and
+does not include IPv6.  The IPv6 section will be updated in a future
+edit.
+
+Bob's comment: I find Postfix to be more manageable than Exim.
+Therefore it is my choice to install Postfix.  Others will choose
+Exim.  The total capability of either is the same.  In many ways it
+doesn't matter very much as long as the system has an MTA and that it
+is configured properly.
+
+    apt-get install --purge postfix
+
+Normally no additional fixup is needed.  But with the Trisquel 7
+package there were errors during the initial installation which
+required corrective action.  This appears to be Trisquel packaging
+bugs because they don't exist in the upstream Debian packages.  It
+definitely is not a Postfix problem but exists in the packaging.
+
+    newaliases: warning: valid_hostname: misplaced delimiter: 
internal0.savannah.gnu.org.
+    newaliases: fatal: file /etc/postfix/main.cf: parameter myhostname: bad 
parameter value: internal0.savannah.gnu.org.
+    2016 Jul 26 17:25:55 internal0 fatal: file /etc/postfix/main.cf: parameter 
myhostname: bad parameter value: internal0.savannah.gnu.org.
+    dpkg: error processing package postfix (--configure): subprocess installed 
post-installation script returned error exit status 75
+
+Fix this problem.
+
+    postconf -e myhostname=internal0.savannah.gnu.org
+    postconf -e 'mydestination=$myhostname, localhost.$mydomain, localhost'
+    postconf -X smtpd_relay_restrictions  # package cleanup
+    dpkg --configure -a
+
+Those problems needed to be fixed before any other configuration was done.
+
+    postconf -e inet_interfaces=loopback-only
+    service postfix restart
+
+The postfix inet_interfaces=loopback-only configuration is an extra
+level of security for leaf node systems that do not ever receive
+email.  It configures to listen only on the loopback device making it
+completely isolated from the hostile Internet.  It isn't required as
+the default configuration prevents relaying but is a good idea.
+
+    echo root: address@hidden >> /etc/aliases
+    newaliases
+
+Install the machine specific iptables firewall.  This is a very custom
+file that has many details of every IP address and subnet.
+
+    /etc/default/iptables-rules (much fixup needed)
+
+Future directions: Bob's comment: I do not much care for the raw
+iptables firewall rules.  It is like programming in machine code.  Of
+course if you can't do it in machine code then it can't be done.
+However everything is so tedious in machine code.  I prefer to use a
+higher level language.  I would like to switch this to Shorewall which
+is a much easier to use firewall compiler.
+
+Ideally every host will have two IP addresses.  One will be a stable
+RFC1918 private LAN address.  The other will be a floating public IP
+address used for public services.  This allows the IP addres to float
+to a new node if needed.  At the present time this does not exist but
+I am working on getting this available for our use.
+
+Host mgt0
+---------
+
+The IPv4 address for mgt0 is 208.118.235.77.  It does not currently
+have an IPv6 address.  This is okay since mgt0 is accessed only
+through fencepost.  It currently does not have a floating address.
+This is also okay since the management node only very few public
+services.
+
+    auto lo eth0
+    iface lo inet loopback
+
+    iface eth0 inet static
+      address 208.118.235.77/24
+      gateway 208.118.235.1
+      pre-up iptables-restore < /etc/default/iptables-rules || :
+
+Host internal0
+--------------
+
+The IPv4 address for internal0 is 208.118.235.78.  It does not
+currently have an IPv6 address.  This is okay since internal0 is
+accessed only through mgt0.  It currently does not have a floating
+address.  This is also okay since the management node only very few
+public services.
+
+    auto lo eth0
+    iface lo inet loopback
+
+    iface eth0 inet static
+      address 208.118.235.77/24
+      gateway 208.118.235.1
+      pre-up iptables-restore < /etc/default/iptables-rules || :
+
+Install MySQL server.  In the future use MariaDB when it is available.
+
+    apt-get install mysql-server mysql-client
+
+Change the port binding to be global rather than localhost so that the
+web frontend can connect to it remotely.
+
+    $EDITOR /etc/mysql/my.cnf
+      ...bind-address 0.0.0.0
+
+Set up the ~/.my.cnf file for access.
+
+    /root/.my.cnf
+      [client]
+      user            = root
+      password        = ******
+    chmod go-rw /root/.my.cnf
+
+Create the database.  Create the remote web frontend user and allow it
+to connect.  And to the local root at localhost user too.
+
+    CREATE DATABASE savane CHARACTER SET utf8;
+    GRANT ALL ON savane.* TO address@hidden IDENTIFIED BY '******';
+    GRANT ALL ON savane.* TO address@hidden;
+    FLUSH PRIVILEGES;
+
+Host frontend0
+--------------
+
+The IPv4 address for internal0 is 208.118.235.77.  It does not
+currently have an IPv6 address.  This is needed to provide universal
+access to Savannah and is yet to be allocated.  It also needs a
+private LAN address and a floating WAN IPv4 address also yet to be
+allocated.
+
+    auto lo eth0
+    iface lo inet loopback
+
+    iface eth0 inet static
+      address 208.118.235.77/24
+      gateway 208.118.235.1
+      pre-up iptables-restore < /etc/default/iptables-rules || :
+
+Install MySQL client.  In the future use MariaDB when it is available.
+
+    apt-get install mysql-client
+
+Set up the ~/.my.cnf file for access.
+
+    /root/.my.cnf
+      [client]
+      host            = internal0
+      user            = savannahscripts
+      password        = ******
+      database        = savane
+    chmod go-rw /root/.my.cnf
+
+Install Apache and PHP5.  In the future this should be Nginx and PHP-FPM.
+
+    apt-get install apache2 apache2-mpm-prefork
+    apt-get install php5 php5-cli php5-fpm php5-gd php5-mysql 
libapache2-mod-php5
+
+Since this is a migration I am going to take the easy documentation
+way out and say migrate the hosts from the old frontend to the new
+frontend0 without specifying them here.  FIXME: Document the web
+virtual hosts better in this section.
+
+    /etc/apache2/sites-available from frontend
+
+Set up the HTTP SSL certificates.  Currently those are traditional ssl
+certificates.  Let's Encrypt certificates required newer software than
+we had available on the old frontend.  But now it is easy to set up
+the new Let's Encrypt Certbot certificates.  I am in the process of
+writing up that documentation.  It is complicated by the fact that the
+official EFF client tries to do too much and requires root priviledges
+that I do not want to give it.  Therefore I lock it down into a
+non-privileged user.  This makes the use of it more involved but it is
+much safer and more secure that way.  FIXME: Add Certbot documentation.
+
+    /etc/ssl/private installed from frontend
+
+FIXME: Need to reference installing Savane here.  Perhaps referencing
+[[RunningSavaneLocally]]?