Re: [Demexp-dev] On web client, XML and ONC RPC and their integration with demexp

[Augustin]

Hi David,

Thank you for all the time you have put into finding a solution to our RPC 
problem.

I don't have much specific information to share. I can only make a few general 
comments.

To start with, it is obvious that the choice at the end is yours to make. 
Whatever is decided, it is more than likely that you will be the one to have 
to implement it.

I will do my best to help, but really, this is not my area of expertise. 
I can code in PHP (fairly well, but I still can improve: I am an amateur, not 
a professional), but I CANNOT code a PHP module in C.

If you did come up with a solution that requires me to make significant 
changes to my development setup, I can do it (with some help, if needed). For 
example, if you did come up with a new PHP module, I can learn how to compile 
PHP so that it includes  the new module.
But in such a scenario, I would be more worried about the long term 
consequences of such a choice. I would be happy to code the Drupal client on 
my own for a few months, but if the difficulty in  setting up a development 
platform becomes a barrier for other developers to join me in my coding 
effort, we loose a bit part of the interest in choosing a CMS in PHP (namely, 
the number of PHP developers available).



On Thursday 31 August 2006 07:48 pm, David MENTRE wrote:
> Hello,
>
> To put this message into context, Augustin has recently proposed[1] to
> make a web interface for demexp using Drupal. One of the main issue of
> this proposal and that there is no implementation of ONC RPC (used to
> communicate with demexp server) in PHP (and hence for Drupal). However,
> Drupal has a XML RPC module[2].

The XML RPC module is enabled by default in any PHP installation:
http://php.net/xmlrpc
but apparently Drupal doesn't make use of those functions.

The Drupal API is well documented:
http://api.drupal.org/apis/HEAD/xmlrp
The xmlrpc functions are defined in two different files:
http://api.drupal.org/api/HEAD/file/includes/xmlrpc.inc
http://api.drupal.org/api/HEAD/file/includes/xmlrpcs.inc
(click on view source to see actual php code).

Those scripts are maintained by Drupal but were originally based on the 
following library:
http://scripts.incutio.com/xmlrpc/

(Drupal is separated into 'Core' and 'Contrib'. The quality of the core is 
dependable, but contrib modules vary in quality. xmlrpc is part of Drupal 
core).



> One approach would be to compile the XDR description of demexp protocol
> into C using rpcgen(3) and then make a PHP binding to it. While
> apparently not too difficult, this approach is not very versatile: if
> the protocol is modified, one needs to modify manually the PHP/C
> binding.

Also how difficult would it be to set up a server for development?


> A second approach would be to implement an RPC generator and its
> supporting XDR encoding/decoding library for PHP. I haven't much
> investigated this approach but it seems to me not a very simple
> path. Moreover, beside demexp, I don't think a lot of people in the PHP
> community are eager to use ONC RPC. :-)

Over the last week, I did spend some time investigating about ONC/XML RPC.
ONC is often said to be for LAN while XML is for the internet. 
In any case, while ONC is much older, it seems that XML RPC has won the battle 
years ago.
Most of the posts I found discussing the relative merits of the two standards 
date back to the last 1990s. It seems that this century, everybody accepts 
XML RPC as the web standard. 

You are right about ONC being more efficient/ compact, but this is also 
thought as of no consequence since the amount of data transferred is never a 
lot.

So, yes, if we code a ONC RPC interface, we'll probably be the only one to use 
it... and support it.



> A third approach would be to implement the XML RPC interface on the
> demexp side. This seems doable with current infrastructure. Gerd's has
> confirmed me that we could have simultaneously XML and ONC RPC inside
> demexp[4] and, it should be possible to use CDuce to implement XML
> parsing and generation[5]. However, I'm not entirely satisfied with this
> approach: it would necessitate to have two interfaces at the demexp
> server, XML RPC and ONC RPC, thus two interfaces to debug, test and
> maintain: complexity is the mother of bad engineering.

Why keep ONC RPC at all?
I feel bad about all the time you've spend on developping the hard client. 
But there are only 40 registered members using the client, while I am 
confident that a proper Drupal client would offer more functionality than the 
current client AND hundreds if not thousands of people might join in very 
quickly. Probably, the 40 original members would mostly stop using their hard 
clients in favor of using the web client which would include more features.

So, why not use XML RPC only, and announce that once the official server is 
upgraded, it will stop replying to ONC RP Calls?


I guess you will not like this solution, for reasons you have already 
mentioned, but I thought it was an idea worth mentioning.


Remember that XML RPC is the de-facto standard nowadays. 
We want to make future development easier.


> This leads me to a fourth approach: implement a XML RPC <-> ONC RPC
> proxy as an interface between the XML RPC side (Drupal) and the ONC RPC
> side (demexp server). It would act as a server on the XML RPC side and
> as a client on the ONC RPC side. This approach seems to me nicer because
> there is no need for a re-implementation of all server network API: the
> same ONC RPC interface is used for both the web and native clients on
> the server side. Moreover, such a proxy would be an independant
> software, with well defined interfaces, so easier to maintain,
> understand and modify.
>
> It remains to see how we could implement this fourth approach. Two
> languages comme to mind, OCaml (with CDuce and ocamlrpc) and Python
> (with Thomas's RPC module and one of numerous XML RPC Python module). I
> don't know which one would be the easier. The ideal approach would be to
> have a program that, from the XDR description of the demexp protocol,
> generates the code of the proxy, but that might be a little to
> complicated at first.

If I understand well, all of this would happen on the server side.
All of this would be completely transparent to me, who will code the client, 
wouldn't it?

Your idea sounds very similar to this one I found:
http://www.thescripts.com/forum/thread86837.html
""I've an old ONC XDR RPC server. I want to write clients in XML RPC.
 Thus, it is my intention to write a XML RPC <-> XDR RPC converter
 (open source, to share it in some public repository).
 In other words, my program will be a XML RPC server and XDR RPC
 client. Queries received in XML RPC server side will be forwaded and
 sent as request in XDR RPC side.""

The problem is that this guy never got a reply (even though this very same 
post had been posted to different forums), and he never left a contact email 
or web site.
There just doesn't seem to be much interest for such a proxy either.


Again, you are the one who will do most/all of the coding for this, so you 
decide. Tell me if you think I can help in any way, and then tell me the 
exact protocol to contact the server from a php script.

But please, do think about the long term consequences of the technical choice 
you are about to make. 


Let me tell you one thing: suppose that there is no RPC problem, that today I 
could write a simple php script to connect to the server and ask for some 
simple data (the list of questions, and for each the list of proposed answer) 
and get that data back from the server. 
If I could write this script today, then I could write the whole drupal demexp 
module within a few weeks only. The implications are important. You have 
ruled out 2007 as an opportunity to publicize your (our) project, but I 
haven't. By the end of this year, we could have a debugged, fairly ironed out 
web client that masses of people could use to register and participate in the 
Democratic Experience. demexp.org could become one of the most popular web 
site during the 2007 electoral campaign (I believe this and it is because I 
had this intuition that I contacted you in the first place, and offered to 
code the client).

(incidentally, it is also the reason why if we had money, I'd rather to have 
it spent on web infrastructure - dedicated web server rental - rather than on 
developers: if we become popular in 2007, we WILL need a better server. As 
long as you can deal with the server, I can handle the client).


My point is this: knowing well the Drupal API, I believe I could code the 
client fairly quickly and in time to play a significant role during the 2007 
elections (but you know that I cannot make any promise: no one knows what 
OTHER difficulty we may encounter... ).
It would then be a shame to waste one, two or even three months or more trying 
to find a solution to the RPC problem...

If instead of trying to manage at the same time the hard client and the web 
client, worry at the same time about the ONC RPC server and the XML RPC 
proxy, if instead of that you could concentrate on one task (server <-> XML 
RPC <-> client), how much quicker do you think you could be?




Maybe what I am saying does not make sense. 
Probably am I being too naive :)

You understand the problem much better than I ever could, so feel free to 
disregard what I said. 




Unfortunately, I am feeling a bit helpless at the moment. Just tell me if I 
can help at all.

Until then, I am only waiting for you (and Thomas) to make you final 
technical/tactical decision.
(speaking about Thomas, is the python client still an option?)


Blessings,


Augustin.







-- 
http://www.wechange.org/
Because we and the world need to change.
 
http://www.reuniting.info/
Intimate Relationships, peace and harmony in the couple.

http://www.gnosis-usa.com/
Revolutionary Psychology, White Tantrism, Dream Yoga...

http://www.masquilier.org/
Condorcet, Approval alternative, better voting methods.