[Savannah-hackers-public] Re: [gnu.org #429351] Access statistics for Savannah

[Alex Fernandez]

On Tue, Jun 23, 2009 at 9:57 PM, Sylvain Beucler<address@hidden> wrote:
>> >From my part I don't have any problems, now or ever. Let us ask
>> Sylvain first who was quite busy with the disk loss.
>
> No problem Alex, please go ahead.

Fine then. The objective of our little project is to create access
statistics for project developers. Since project web pages are stored
in the www.nongnu.org web server developers cannot access them now.
(There is another leg which consists of getting file download
statistics so they can also see how many people have downloaded a
particular version, but that is a different matter.)

As I understand it, Savannah (sv.nongnu.org) triggers an update to the
web pages by using a curl command. The gnu.org web server then reads
the corresponding web directories from sv.nongnu.org. What we would
need is for the web server to copy back the corresponding logs into
sv.nongnu.org, strictly for the www.nongnu.org domain (so there are no
conflicts with other gnu.org pages). Later on we might tackle GNU
projects in Savannah -- I understand they are served from
http://www.gnu.org/software/ so the log can probably be isolated too.

Then the log files will be processed by awstats and put at the
disposition of project developers if they want to see them or even
publish them. This latest part can be achieved by restricting the
pages analyzed by awstats to a certain project. (I don't know yet how
to publish the stats so that only project developers can see them,
perhaps they can even be mailed to developers if desired.)

I have created the directory
  /var/log/apache2/www.nongnu.org
in the cvs vserver so that we can do a test run. At the moment the
information will not be accessible to developers.

To summarize in a few questions:
  - Does the www.nongnu.org web server have ssh access to
sv.nongnu.org? Can it copy files there, in the specified directory?
  - Are the log files for www.nongnu.org kept separate from the
www.gnu.org domain so they can be copied as is? If not, can they be
separated, or at least grepped before copying?
  - How often are the web server logs rotated?
  - Can the copying task be automated?

I am cc'ing Savannah-hackers since they were interested, I hope
everyone is fine with it.

Alex.