Re: [Pan-users] Need help debugging pan + gnutls-3.x.x

[Duncan]

walt posted on Mon, 10 Dec 2012 05:49:12 -0800 as excerpted:

> On 12/08/2012 05:49 PM, walt wrote:
>>
>> Third bizarre behavior of pan+gnutls-3 is that the "broken" server is
>> not *always* broken, but works intermittently, sometimes for days at a
>> time, and then breaks again for reasons I can't understand.  I just
>> started pan again at 17:30 PST and it connected perfectly to the
>> 'broken' server and stored its 6-byte cert file right beside the
>> 'working' server's 6-byte cert file, like this:
> 
> Yesterday the 'broken' server started and stopped working at least five
> times, and did it again this morning.  I still don't understand why this
> happens but at least I do have another possible clue:
> 
> When I set pan to *not* trust the server's cert, two different things
> may happen.  First, when the server is broken, pan never presents me
> with the cert for my approval, i.e. it seems to me that gnutls-3 is not
> actually fetching/reading the cert and therefore can't ask me to approve
> it.
> 
> Second, when the server suddenly starts working again, pan actually does
> present me with the cert for approval, and in fact it presents it two
> and sometimes three times, so I have to click away the dialog box more
> than once.  After that, pan works perfectly again for some unpredictable
> period of time before the server 'breaks' again.
> 
> To add to my confusion, I can use gnutls-cli-debug -p 563 to examine the
> server's cert perfectly whether the server is 'broken' or not.  That
> seems to imply that something in pan is changing rather than something
> in the server, doesn't it?
> 
> I remain mystified :(

I've been going to look into this myself (I've been running gnutls 3.x 
for quite some time but switched back to plain text when pan's secure 
code was still churning, and I need to try it again anyway), but I've not 
had the time as I'm working full time again. =:^)  Also, I forgot after 
your first mail, so this one reminded me.

>From your previous post, the short cert files are probably just hashes, 
giving pan just enough info to know whether it has accepted the cert yet 
or not.  Reading the source may confirm that one way or the other.

The broken/working/broken bit MAY be the NSP's server, serving different 
certs depending on what front-end you connect to.  IIRC you mentioned 
that the cert didn't always match the given domain name.  Have you 
verified whether you're at least always getting the same cert?  Do your 
NSPs give out enough info to know how their frontends are setup?

One thing you can do there is do a dnslookup and see if there's multiple 
IPs for the given domain name.  Of course you can do the reverse lookup 
on all the IPs as well and see if they're mapped the same both ways.

Sometimes, however, their balancing isn't exposed via IP -- they reverse 
proxy from a single public IP.  In that case, you can try using the debug 
tools (I've only done this with clear-text so don't know how much more 
complex it might be) to telnet (stellnet? ssh?)Naturally, the way the 
balancing worked, those frontends always seemed to be less loaded than 
the others, so  in and see what the greeting is.  Often, the greeting 
will tell you which frontend you're on.

Back when my ISP was providing outsourced news, for some time, one or two 
out of something over a dozen front-ends were broken.  Fortunately, they 
used round-robin assignment, as load-based would have always shunted 
people to the "unloaded" broken machines, but that meant once you DID 
connect to a bad machine by luck of the robin, it was hard to get to a 
different machine, since all connections were routed there based on your 
IP.  Technically oriented people could randomize their MAC address and 
thus get a new DHCP assigned IP, thus getting a new connection, but that 
was a lot of trouble and not everybody's that technically oriented.  The 
other alternative was to wait I think it was 15 minutes with NO news 
connection attempts for the NSP's robin assignments to expire, and try 
again.

I remember the guy that finally figured out what was going on, by 
telneting in and tracking the greeting he received.  The broken machines 
were on a different Highwinds server version (tornado, hurricane, 
whatever it was that was the front-ends) than the others, and they listed 
that in their greeting... but no GUI news clients display that these 
days, and very few even offer the ability to log it, so he had to track 
it down via manual telnet method.  But that was plain-text connections.  
Secure connections throw more complexity into the mix, but it should 
still be possible given the right debugging tools and sufficient patience.

Another debugging alternative would be to use the old stunnel method that 
pan users used for years before pan got its own secure support.  You 
could of course then use either plain telnet or a pcap of the plain-text 
traffic between pan and the tunnel to grab the greeting, etc.

Meanwhile, the multiple cert-accept dialogs could well be due to pan's 
multiple connections code.  If you dial back your allowed connections to 
only one, do you consistently only get one accept dialog?  If so, the 
problem should be fixed, but it could well be difficult to do so, and 
since under normal conditions once you accept the cert it shouldn't 
happen again (until the cert changes), it may be that Heinrich either 
thought it was fixed or decided to leave that bug to work on some other 
time.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman