Re: [Phpgroupware-developers] Performance tests

[Alex Borges]

El vie, 27-02-2004 a las 04:32, Markus Kaemmerer escribió: 
> Hi,
> 
> today I made some tests regarding performance of phpGW. 
> 
> For the synchronization we need address data. For a quick start I
> wrote a little script to fetch 200 times the same contact (click on
> view in address list). I used wget to simulate a real click on this
> button. I used the same session (php4 and db sessions did not make a
> difference in performance) on my 1.6 GHz Pentium M with Apache 1.3.29,
> PHP 4.3.4, mmcache and MySQL 4.0. Without mmcache the test was about
> two times slower, but the relative values where the same.

This is a great way to test. We do it with elsa for massive testing.
Let me get this straight,  you hit on the uiaddressbook.view  method
right?
Good news is that yeah, the mmcache does make a huge difference. A
question now is if its actually taking all of 15 minutes (more than 3-5
is already hard to accept  for a user though) to sync 300 contacts as in
your previous test.

> phpGW created about 1.4 MB of HTML data for the 200 views. On my
> notebook this took about 44 seconds, leading to 220ms for one view
> without rendering the HTML output. In other tests I found that
> fetching a vCard from the addressbook internally took about 45-50ms. 
> 
> I changed some internal behaviors but could not found any major
> difference. I disabled caching of address data in sessions variables,
> this improved the performance about 5 percent. But this is not a major
> difference. I suggest to remove the cache handling and let the
> database do this thing. This removes complexity from the code and
> leads to a very little speed up.

Ok, one thing.... did you try with php4 sessions or db sessions?
Ive done this same kind of testing and my experience is that session
caching in php4 renders a performance increase. I see no reason to use
db session cache, except for clustered environments. 

If your profiling was done with php4 sessions, then there is no
discussion, we need to lay off the session cache.

> To have a comparison value I fetched 200 times the month view from an
> empty calendar. This took about 63 seconds, about 315ms per view. In
> this test phpGW created about 10 times more HTML data [14 MB] than in
> first test. 

This is a good baseline and shows how much faster it could be. Still,
the complexity difference of the tables (thus the sql) is also in about
the same order (9 table, pretty complex design vs 4 table, quite
straightforward design). Still, yeah, i think its time to do some
performance code audit. We will get to that soon (first correct
addressbook, worry about fast addressbook later)

> I think most time is consumed during the context switches and for
> initialization of all internal variables etc. in the phpGW API. I
> think it should be possible to do a little work here to reduce the
> initialization overhead as far as possible. This would improve the
> subjective and objective performance on every single click to phpGW
> because the user has to wait at least 0,2 seconds before the selected
> application starts to create and send HTML code. Remember, that I used
> a fast notebook with local database, mmcache and only one user. On
> larger installations this overhead would result in longer response
> times.

Naturally. For example, 14 with 1800, multiuser, direct hits to the
email app can render a 2 minute latency time for users. I agree that
there is a lot of room for performance fixing. 
One of those, is to see the way the preferences and acl data is cached.
THere are times when all the preferences are in memory for each page
hit, and all the groups for a user and, sometimes, all records of an
account.

In general, this happens because there is a programming model used in
some of this classes that bring the whole thing into a class attribute
called data and sort of cache it for that pagehit.

This is fast for smaller instalations, but as the data grows, memory
increases as well....then the apache starts eating up your memory, and
eventually hits the swap area. It gets ugly from there.

In the speciffic of the addressbook. Well, its a very complex thing
there. It loads all the sql_entity classes, the sql_builder, the
contacts_sql and addressbook bo/so/ui classes. This ammounts to nearly
10000 lines of code. I think those big classes can be further hacked to
lazyly load parts of themselves as needed, this may help out a bit for
some queries. 

Part of the problem here is that syncing is a weird thing because you
potentially need to put data in all tables (in the notes table, in the
addresses table, in the comm_ table, in the person/org tables). This
means you will use all of the code in the addressbook, and thats a lot.

> Every speed up on this would also improve synchronization speed,
> because we have to make at least one (XML-RPC) call for every
> synchronized item.

Yes.. As ive said, xml-rpc is an interesting chalenge precisely because
it is an RPC protocol. This means that its a pagehit for every function
called (them being add vcard, remove vcard or whatever).

In this line... perhaps we can change it so that the xml-rpc sync sends
a list of vcards, and we can sync the list in bulk instead of going one
by one? Then make a protocol for conflict handling on this basis?

> Markus
> 
> P.S. To bring some good news, too: we managed to synchronize our test
> devices (PocketPC, Calmeno Outlook Plugin, P800 and a Nokia handy via
> GPRS) in both directions including add and delete. I'll publish more
> information and code in the next few days.

This is GREAT news! BTW. some of us are eager to look at the sync app,
ive the feeling you have an internall one that actually works (current
one does not work in postgres). Please commit it into head!

> --
> Markus Kämmerer         Team Software Solutions
> pro|business AG, EXPO Plaza 1, 30539 Hannover
> E-Mail: address@hidden,  Phone.: 0511/60066-0
> WWW: http://www.probusiness.de/,    Mobile: 0177/5990932
> 
> **********************************************
>            address@hidden 2004
> 
> *     Halle 1, Stand 3k4 (Magirus Deutschland GmbH)
> *     Halle 1, Stand 7f2 (EMC Deutschland GmbH)
> *     Halle 6, Stand D46 (Land Niedersachsen)
> **********************************************
> 
> 
> _______________________________________________
> Phpgroupware-developers mailing list
> address@hidden
> http://mail.gnu.org/mailman/listinfo/phpgroupware-developers
>