Re: [Savannah-users] About projects web pages

[Bob Proulx]

Hello Danilo,

Danilo G. Baio wrote:
> I've noticed that in FreeBSD, some ports have their web pages broken.
> 
> This happens with all projects pointing to http://www.nongnu.org/%%PROJECT%%
...
> I think at least 100 ports on FreeBSD are affected with this.

First let me say that I am not one of the FSF admins.  I volunteer
with Savannah.  So all of this that I am going to say is second hand
information.  I can't login to the FSF systems and look.  Read details
with caution because it might be wrong! :-) But hopefully useful
anyway.

My first thought is that I don't know why this is being brought up
here on savannah-users since it isn't related to Savannah.  What can
we do about it?  I might think that gnu-misc-discuss might be better.
(Shrug.)  I don't know.  But there isn't anything Savannah Hackers can
do about it since it is on other systems that are not Savannah
systems.

Here is what I know.  Sometime last week it was reported that there
was a certificate problem with the www.nongnu.org system (on
wildebeest.gnu.org the system that hosts it).  This was a systematic
problem and affected *ALL* of the projects, not just the ones you
mentioned.  You weren't getting picked on in particular.  It was just
a fault and everything using it was broken.

The FSF admins started working on the problem but were unable to
complete it last week.  And unfortunately the debugging of the https
certificate problem apparently broke the http side of things too.
That appears to have been an Oops.  On Saturday more reports came in
to Savannah about this new problem of www.nongnu.org returning 404 Not
Found for everything.  But since www.nongnu.org on wildebeest is not a
Savannah system there wasn't anything we could do other than redirect
things over to sysadmin.  Which I did and on Saturday Ian jumped in
and got the http side of things back online.  Thanks Ian!  This wasn't
a system he was familiar with, he's new, and the https certificate
problem predated the http breakage so things had to get left at that
point for others to fix later.

Today on Sunday I check and see that the redirect from http to https
has been removed.  I am sure this is temporary just to avoid the
broken https certificate.  It's about the only temporary workaround
that can get things back working without an error.  Even though it
does drop support for https for the moment.  I know this will only be
temporary and https will be when it is working again.  Things are
online in a workaround state until the certificate fix is completed.

> The addresses http://www.nongnu.org/%%PROJECT%% will work again?

When I woke up this morning and looked I see that the above are all
working again now.  The only caveat is that they are (temporarily) no
longer redirecting to https as they had been doing.

> Or should we just update them to http://%%PROJECT%%.nongnu.org?

When things were in the broken state the %%PROJECT%%.nongnu.org URLs
were also broken too.  (At least they were for me.)  So that wouldn't
have helped.  :-(

In the future if something like this happens the best official answer
is to please file a bug ticket with the FSF admins.  Do this by
sending an email to < sysadmin AT gnu DOT org > where it will create a
ticket in their RT queue.  If something is particularly grave and
urgent then after filing that ticket also contacting them
interactively such as over IRC or phone or other channel to alert them
may be appropriate based upon the judgment of the situation at the
time.  For something like this I would nudge them so they are aware of
how widespread the problem is and that this will affect a lot of people.

Another good thing to know is that the FSF admins are governed by
union and state employment rules and therefore response time outside
of weekday business hours may be substantially delayed.  If something
is broken over the weekend it might not have any resources to work on
it until Monday morning.

Of course the weekend is when many of the rest of us are volunteering
on projects and so this creates a perfect convergence of problems
existing and delays before it will be addressed.  This means that if
you find something broken on Saturday then patience is needed until
Monday.  Sometimes all that can be done is to file tickets and wait.
I don't see a way around this problem other than to recognize that it
exists and has been that way for as long as I can remember.

Please do file a ticket to FSF sysadmin when you are affected by
problems.  Because if something is broken and no one says anything
then there is no reason for it to get fixed.  They might not even know
about it.  If something affects someone and a ticket is filed then it
will eventually get addressed.  On the other hand if something big is
broken and it affects a lot of people and they get a lot of tickets
then getting a lot of tickets will tell them that this is an important
problem and it must be prioritized high to be fixed sooner.  An old
saying applies here.  "The squeaky wheel gets the oil."

Bob