[Savannah-cvs] [38] document how our ikiwiki works, and how it was set up

[karl]

Revision: 38
          http://svn.sv.gnu.org/viewvc/?view=rev&root=administration&revision=38
Author:   karl
Date:     2013-09-04 23:28:03 +0000 (Wed, 04 Sep 2013)
Log Message:
-----------
document how our ikiwiki works, and how it was set up

Modified Paths:
--------------
    trunk/sviki/HowToAdminThisWiki.mdwn

Modified: trunk/sviki/HowToAdminThisWiki.mdwn
===================================================================
--- trunk/sviki/HowToAdminThisWiki.mdwn 2013-09-04 23:27:13 UTC (rev 37)
+++ trunk/sviki/HowToAdminThisWiki.mdwn 2013-09-04 23:28:03 UTC (rev 38)
@@ -1,23 +1,169 @@
-This wiki is based on ikiwiki.  Access is through Subversion.  So:
+This wiki is based on ikiwiki.  Access is through Subversion.  So, to
+update the pages:
 
 0. Check out the subversion repository in the administration project.
 See <https://savannah.gnu.org/svn/?group=administration>.
 
-0. Make edits as you please on those local files.
+0. Make edits as you please on those local files, which use MarkDown
+<http://ikiwiki.info/ikiwiki/formatting>.  (ikiwiki compiles the
+markdown into HTML, in `/var/www/sviki`, but we need/should never touch
+those files.)
 
 0. Run `svn commit -m'log message'`.
 
 The result should show up under <http://savannah.gnu.org/maintenance/>
-in a few seconds, if all goes well.  (Check [[RecentChanges]] if you
-like, as well as the page being updated.)
+in a few seconds, if all goes well.  Check [[RecentChanges]] if you
+like.
 
+
 Wiki configuration
 ------------------
-In the file frontend:~sviki/ikiwiki.setup.  At present, we are using the
-default (quite minimal) configuration, with no extra plugins.
+In the file `frontend:~sviki/ikiwiki.setup`.  At present, we are using
+the default (quite minimal) configuration, with no extra plugins.
 
 That file is in RCS, for simplest version control.  Anyone in the
-`staff` group can edit it, which should be anyone who has shell access
-to the machine.  (All Savannah administrators are welcome to have shell
+`staff` group can edit it, which should be anyone with shell access to
+the machine.  (All Savannah administrators are welcome to have shell
 logins, but Savannah users do not, for obvious security reasons.)
 
+The web server configuration bits are in
+`frontend:/etc/apache2/sites-available/sv.inc`.
+
+
+How it works
+------------
+Our ikiwiki setup on Savannah is unusual.  ikiwiki expects the source
+repository and wiki to live on the same machine, and thus permissions
+are simply a matter of having the files be writable to the desired Unix
+users.
+
+For Savannah, we really wanted the source repository to be accessible in
+the standard Savannah ways, that is, through the `administration`
+project (<http://savannah.gnu.org/projects/administration>, which holds
+all the live Savannah stuff that is checked into version control).  That
+way, access would be controlled just as to any other source repository.
+We used the Subversion repository in `administration` as the wiki source
+partly because it was otherwise unused and partly because Karl (who did
+all this) likes Subversion.
+
+Well, using a Savannah project means that source repository lives on one
+host (vcs) while the web-accessible wiki lives on another (frontend).
+And frontend is where the actual refresh is run, using
+`~sviki/sviki-refresh`.  This necessitated the following:
+
+0. A post-commit hook on vcs which ssh's back to frontend to do the
+refresh.  This is in
+`vcs:/srv/svn/administration/hooks/post-commit.d/sv_ikiwiki_refresh`.
+
+0. A pseudo-user `sviki' on vcs, frontend, and mgt, with its own ssh key
+to allow communication (in `/etc/ssh/authorized_keys/sviki`).
+
+0. An empty directory `vcs:/srv/svn/.ssh` because ... it's a long story.
+ssh unconditionally tries to create the user's `~/.ssh` directory, even
+when (as here) it will not be used for anything.  On vcs, the
+post-commit is running as the user who did the commit, which is to say,
+a normal Savannah user who does not have a real home directory but
+exists only in the mysql database.  The users' homes are arbitrarily set
+to `/srv/svn` instead, so that is where the `.ssh` needs to exist.
+Thankfully ssh does not initially require specific permissions or
+ownership on it; as long as it exists, ssh won't try to create it, and
+then it will never be accessed again (due to the options with which
+`sv_ikiwiki_refresh` invokes ssh).
+
+
+Initial setup
+-------------
+For posterity, here is how the initial setup went.
+
+The first problem was that the previous Zope-based zwiki was cracked in
+summer 2012, with many pages deleted or junked.  We eventually got a
+restore from the FSF, but it never ran right, and then Zope would not
+restart after a reboot.  We could find no way of extracting the actual
+text from the huge .fs files hanging around.  Sigh.
+
+So we manually saved the HTML pages in the Wayback Machine, at
+<http://archive.org>.  Fortunately there had been essentially no edits
+since before the crack, and the pre-crack versions were available.  Yay
+for Wayback.
+
+So we had these HTML files.  We used Pandoc
+<http://johnmacfarlane.net/pandoc> to do the basic conversion to
+Markdown, plus some custom Perl massaging to eliminate the Wayback
+boilerplate, etc.  That script is in `frontend:~sviki/wayback-trans`,
+along with the original HTML files.  The Zope wiki pages that contained
+spaces in the names are probably broken links now, by the way.
+
+We had previously decided to use ikiwiki, to avoid ever having to go
+through the agony of a cracked wiki again.  So next, on frontend,
+we just install the basic Debian package; we don't need the very latest.
+   apt-get install ikiwiki
+
+Following steps on <http://ikiwiki.info/setup/byhand> ...
+    SRCDIR=/usr/local/share/sviki/liveco
+    DESTDIR=/var/www/sviki
+
+In `/etc/apache2/sites-available/sv.inc`, comment out zope stuff (since
+deleted), add:
+   Alias /maintenance /var/www/sviki
+And run:
+   service apache2 reload
+
+As me, create directories, initial `index.mdwn`:
+    ikiwiki --verbose $SRCDIR $DESTDIR 
--url=http://savannah.gnu.org/maintenance/
+Try visiting <http://savannah.gnu.org/maintenance/>.
+It works.  Amazing.
+
+Dump setup for easier editing:
+    !! --dumpsetup /usr/local/share/sviki/ikiwiki.setup
+
+Edit setup file for wiki name, etc., then update:
+    ikiwiki --setup ikiwiki.setup
+Skip CGI for now (forever?).
+
+Create `administration`'s svn repository (can do this anyway):
+    svn co svn+ssh://address@hidden/administration
+    svn mkdir branches tags trunk
+    svn mkdir trunk/sviki
+    cd trunk/sviki
+    svn propset svn:ignore .ikiwiki .
+    echo hi there > index.mdwn
+    svn commit
+
+(Well, the `svn co` command I actually ran was different from the above
+standard one, because I have a shell account on vcs and the other
+machines [[SavannahArchitecture]].  But that is an irrelevant detail
+here.)
+
+Back on frontend, ikiwiki will need to do updates, so:
+    apt-get install subversion
+
+Switch `/usr/local/share/sviki/liveco/` to an svn checkout.
+Set `rcs => 'svn'` in ikiwiki.setup; cmdline seemingly has no effect.
+Update:
+     ikiwiki --verbose --setup ikiwiki.setup
+
+Looks like the changes are written in `liveco/.ikiwiki`; I see
+<https://savannah.gnu.org/maintenance/recentchanges/> now, made from the
+svn log.
+
+Try a commit from our checked-out repository, then on frontend:
+    ikiwiki --setup ikiwiki.setup --refresh
+
+Looks ok.  Create user `sviki`, uid/gid 902, with generated `id_rsa`,
+and set up post-commit stuff on vcs, as described above.
+
+Edit `frontend:/etc/default/iptables-rules` to allow incoming ssh from
+vcs.  (and from all other Savannah hosts while we're at it.)
+
+Don't bother trying to start zope anymore:
+    update-rc.d zope2.10 remove
+That got rid of these symlinks:
+    rc0.d/K02zope2.10 -> ../init.d/zope2.10
+    rc1.d/K02zope2.10 -> ../init.d/zope2.10
+    rc2.d/K02zope2.10 -> ../init.d/zope2.10
+    rc3.d/K02zope2.10 -> ../init.d/zope2.10
+    rc4.d/K02zope2.10 -> ../init.d/zope2.10
+    rc5.d/K02zope2.10 -> ../init.d/zope2.10
+    rc6.d/K02zope2.10 -> ../init.d/zope2.10
+
+Maybe we are back in business.