[Antbear-devel] performance architecture- threads (Warning- long!!!)

[Daniel Ng]

Hi guys,

This is an interesting topic. We have to consider it
if we want a server with good performance...

I don't suppose any of you know a server architecture
expert? :)

Anyway, the best high-level explanation of nio I found
is here:
http://www.onjava.com/pub/a/onjava/2002/10/02/javanio.html?page=4

-so basically nio means socket write() + read() +
connect() return *immediately*, regardless of whether
anything was written/read/connected. So, you can have
just 1 thread dealing with *all* connections, as well
as listen for new connection requests! (Which is what
Thorsten has been saying all along- sorry for my
thick-headedness :) ).

I now think:

- 1 thread for responding to connected Gameclients
Supporters, Master(s)** and Gameservers* including
listening for new connections

- 1 thread for responding to requests from the GUI.

- 1 thread to look after 'house-cleaning' such as
updating the database every x minutes.

*We have to clear up:
-Who initiates the connection between the Gameservers
and the Antbear Master?- the Gameserver or the Antbear
Master?
-Who initiates the connection between the Supporter
and the Gameservers? -the Gameserver or the Supporter?
--> I am guessing it should be the Antbear which
initiates the connection in the above cases, according
to the protocol description.

**If there is more than one Master to connect to, then
there will be 1 thread per Master (have to think more
about this later).

Sorry for not doing the Collaboration Diagram as
promised. I have been spending a lot of time on thread
research instead. Will aim to work on the
Collaboration Diagram tomorrow.

I think this thread discussion will only affect the
Collaboration diagram- the static Class Diagram won't
change.

This summarises my answers below:


--- Thorsten Schemm <address@hidden> wrote:
> Hiho,
> 
> > 
> > Multi-Thread stuff:
> > 
> > So, do we agree on the following?-
> > 
> > -1 thread to handle interactions with connected
> > Master/Supporters (or 1 thread for *each*
> connected
> > Master/Supporter??)
> 
> First option.

Agreed now. But this thread should be shared with all
socket activities, not just with Master/Supporters. In
general, it could be argued that there is no
performance advantage in having a separate thread for
different connections unless you have > 1 CPU. See
p143 of-
http://www.oreilly.com/catalog/javanio/chapter/ch04.pdf

Also, at this stage we want to keep it simple, so
providing different classes of service as described in
the above article might be an extension for later.

> 
> > 
> > -1 thread to handle interactions with connected
> > Gameservers (or 1 thread for *each* connected
> > Gameserver??)
> 
> We need to distinguish between querying gameservers
> (active)and
> answering gameserver requests (passive) like when a
> gameserver registers
> itself at the master or when the gameserver gives a
> heartbeat. 
> (note to me: I need to look into this protocol
> sometime).
> 
> Yes, the 'passive part' should be treated by one
> thread only I would
> say.  Just like the MASTER/SUPPORTER handling.

I don't think whether the Antbear is passive or active
matters so much. I think what you are saying is that
for its passive role, it is not as important for the
Antbear to respond because the Antbear is not the one
waiting for a response, so it is not blocking any of
its own activities. However, because nio means  read()
+ write() return immediately, then there is no delay
in responding to other requests anyway.


> 
> Details on the 'active part':
> 
> Let me explain the way I would do it the 'old
> fashion way' without using
> the nio classes.  When using the nio classes things
> might change a lot,
> but you are the expert on that subject 8) .

Heh, heh, I wouldn't call myself an expert- I only
spent a couple of days reading it!!! :)

> Not using nio I would use one thread each
> gameserver.  In searchtool I
> do query the gameservers like this to avoid too much
> overhead that would
> be caused by actually creating a new thead for a new
> gameserver:
> 
> 1. place all servers in a
> stack(vector,list,whatever)
> 2. create a certain amount of threads, depending on
> the internet
> connection (40 threads is the default in searchtool)
> 3. now let every thread grab a server from the stack
> and process it.
> Once that one has been processed, grab another one,
> if there still are
> servers left. A query does not do much work, but it
> is waiting most of
> the time for a response, so not using multiple
> threads would lead to a
> great loss of performance.

That sounds very cool. Do you know if that's how most
servers work? I guess it's not really a typical server
--of course, it's a Masterserver. :)

Hey, if I were to download the searchtool and run it,
would that mean I could see all the HalfLife servers
on the Internet? I should try it!

nio only requires 1 thread to maintain multiple
connections in the above scenario, because it returns
immediately from connect(). 

If an Antbear wanted to make new connections to
multiple Gameservers, this is what would happen:

1) List of Gameservers obtained from
GUI/(commandline?)

2) The GUI has its own Select Key registered with the
continuous select() loop in ConnAdmin. This key was
initialised to have an object attached to it. GUI
tells AntbearEngine it wants to register new
Gameservers. AnbearEngine configures the object to
'registerNewGameservers', including a list of all the
new Gameservers. AntbearEngine fires this key (by
writing to a Pipe which is linked to the key- **Is
this the best way to do inter-thread
communications???). This entire step is done in the
GUI thread. 

3) AdminConn.Select() detects the key has been fired.
It looks at the key's attached object, which tells it
to 'registerNewGameserver'. Hence, it attempts to make
connections to the new Gameservers. At this point,
connect() returns immediately, so that all the new
connection requests are started without delay- ie. no
blocking. This is all done in the 'connecdtions'
thread.

4) If a particular new Gameserver connection is
successful, select() will detect an 'isConnectable'
event for it. The select() thread then makes a new
'isReadable' key for the new connection. This key is
fired every time there is something new to read on the
connection. (A similar thing is done for isWritable).
This entire step 4 is done for every single new
successful Gameserver connection. Again it is all done
in the 'connections' thread.

5) This ConnAdmin.select() method then tells the
AntBearEngine about the new connection by calling
something like AntbearEngine.registerNewGameserver().
This is done in the 'connections' thread.

I hope it works. :) Will test it.

> 
> > 
> > -1 thread to handle listening to and dealing with
> > existing Gameclient connections, and listening for
> new
> > connections of ALL types (ie.
> > Gameclient/server/Antbear)
> 
>  I hope I got you right here:
> One thread listening for all types of connections
> seems good to me.  But
> there is another thread (the engine) that does the
> processing, whatever
> request came in. Correct?

I'm not sure what you mean here. The 'connections'
thread handles all activities originating from
connection requests, so the thread would also call the
AntbearEngine methods as necessary. So, there is no
need for a separate thread for the AntbearEngine,
right?


> 
> > 
> > -1 thread to handle the GUI interactions.
> 
> Yes.  We should collect some ideas some time about
> the GUI and what we
> should be able to control with it.  The range goes
> from displaying
> statistics to being able to manually kill
> connections.

Agreed. Another discussion! :)

> 
> > I guess we need to know what is the
> maximum/typical
> > size of the player lists downloaded from a
> Gameserver?
> 
> Well, there is no real maximum.  But we could set
> one because there will
> hardly be any servers with more than 62 players on
> it.  The average will
> on a full server would be 16 to 20 players on one
> server.  Of course
> there will be a lot of empty servers.
> 
> > What is the max/typical size of player lists
> > downloaded from a Supporter? From a Master to a
> > client?
> 
> I have no idea yet about the size of a list we
> transfer between MASTER
> and SUPPORTER.  We have free hand here.
> The max list from a MASTER to a client would be
> around 30000 servers
> nowadays.  Half a year ago that used to be around
> 25000, so the number
> might increase.  Thanks to the MASTER <--> client
> protocol we can send
> the list in pieces.
> I could not say if there is a typical number because
> the size of the
> list depends on the specified filter.
> 

I guess why I asked these questions is because I want
to know if we are worried about time taken to retrieve
data from the databases? Then again, if the server is
struggling to fulfill database requests from the
outside world, then we can't really make it go faster
no matter how many threads we use, right? So, at least
nio takes care of the problem of delays due to low
bandwidth or slow remote computers.

Whew! I hope this all makes sense. Let me know what
you think...

> > 
> > 
> > 
> > Mutex- 
> > 
> > (Just to confirm, we will still need
> synchronisation
> > mutex stuff on the databases because you could
> > manually delete a peer connection via the GUI
> while at
> > the same time the Server was getting Gameclient
> > information from that connection.)
> 
> I did not get this, but it is late here 8) . I will
> have a closer look
> on this tomorrow.
> 
> > 
> > 
> > These questions may affect the Class Diagram so
> better
> > that we resolve them before we move on to coding,
> > etc.-do you think so?
> Well, I can't tell since I could not yet have a look
> at the diagrams due
> to my professional skill in nuking operatingsystems.
> Good night,
> 
> - Thorsten
> 
> 
> 
> _______________________________________________
> Antbear-devel mailing list
> address@hidden
>
http://mail.nongnu.org/mailman/listinfo/antbear-devel




__________________________________________________
Do you Yahoo!?
Yahoo! Web Hosting - Let the expert host your site
http://webhosting.yahoo.com