[GRLUG] XMPP Proxy/Middleware Service?

[Adam Tauno Williams]

On Wed, 2009-05-27 at 23:52 -0400, Ben DeMott wrote:
> Ahh - a stateful server daemon to maintain a stateful session on
> behalf of a non-stateful client - and herein lies the problem.
> Sounds like a job for one of the Jabber c libraries - I'm sure google
> accomplishes this through Python (which gmail is written in) or a
> compiled language.

As the previous posted commented there is a very good XMPP
(consumer/client) implementation for Java by OpenFire (who provides our
actual XMPP server).  I avoid C or unmanaged code whenever possible.  So
Python, Java, or .NET.  I'm leaning toward Java because (a) I have
confidence the API is robust [everything OpenFire does is robust] and
(b) it will couple easily with xmlBlaster
<http://www.xmlblaster.org/index.html> which I've used before and seems
an ideal candidate for doing the grunt work of this job.  You can
perform an AJAX "poll" of xmlBlaster in the same manner as JSJaC + BOSH
uses to implement the client directly.  Then all that is required is a
thin shim between the XMPP server and the MOM (xmlBlaster).  And no
doubt once xmlBlaster is deployed we will find 1,001 other uses for it.
I want to avoid building any more than I absolutely have too.

> Here's some code to get you started ... (borrowed from apache)
> for ( ; ; ) {
>   // your session code
> }

:)

> Although I don't agree PHP is a wasteland - PHP as an SAPI (which is
> what you are referring to) wasn't designed to have it's execution
> piped to waiting or stateless threads from user requests through DMA
> unfortunately - However the PHP thread is capable of leaving open
> 'persistant' connections
> pfsockopen();

Sort of, PHP as a module can't spin up persistent threads (as the worker
will get killed off at least as often as MaxClients is exceeded).  And
PHP as a module [still] has some serious GC issues;  PHP as a persistent
process is really bad.

> A completely different approach is to open a PHP thread for each
> jabber login request received - then ignore user abort, and have the
> thread query a database on a regular basis to determine if the
> connection should be closed based on timeout / user abort.  

Essentially this is what I'm suggesting.  Only 
(a) I'd never use PHP for that.  As you said PHP is a SAPI, and it
really belongs there.
(b) I wouldn't use a database,  that is much too heavy a solution.  A
MOM or just memcache would be a better solution.  We avoid filesystem
I/O at nearly all costs.  Key/Value storage should be sufficient for
such a solution.  We've had very good results using memcache for IPC,
or in other cases just using tmpfs (RAM disk) and files.

> The never ending thread would persist for each user, and it may be
> smart to set the maximum memory size to something pretty small so if
> there are hundreds of simultaneous users PHP doesn't reserve all the
> systems memory across all of the possible threads.

Exactly.  Or use an environment that has mature GC. :)

> You might even be able to consolidate ALL users to one never ending
> thread on the server and just conditionally open and close as many
> different sockets as needed based on queries to a database.

A connection needs to be maintained for each user to the XMPP server.
We aren't building a "chat" solution, we are consuming an XMPP service.
So a thread per user relaying messages between the MOM and the XMPP
seems the simplest.

> Yet another and probably more elegant solution is to perform
> interprocess communication to a persistent php thread using sockets
> and the socket_create_pair() function.
> So when PHP get's namespaces and lambda functions in 6.0 will it be
> promoted to an oasis? Oh I can't wait to be Y combinating in my
> wasteland.

True.  I do most of my hacking in .NET (C#) and the functionality of
lamdba and LINQ is awesome.  And extension methods finally give back
what we had in Objective-C years [decades?] ago (via Categories);  now
if the Ruby people would stop claiming to have invented such things....
but then the Ruby people think they invented MVC. :)

> The real issue is that almost all PHP opensource applications have the
> goal of reaching the largest audience - 

Er... that is one way of interpreting it.  I think most PHP apps are a
race to get-something-working and most pretty much stall at that point.
I'm not dissing PHP, we have allot of PHP code that works well, but
there is ALLOT of *terrible* PHP code around,  and the PHP community at
large has some pretty immature coding practices (not to mention
suffering from an almost ubiquitous myopia that sees every problem as a
CRUD/SQL problem - "if all you have is a hammer everything looks like a
nail").

> with that goal comes the requirement to run on shared hosts, and with
> that comes the lack of applications that interact with or are
> concerned with persistent services such as XMPP; thus the wasteland.

Or PHP is rarely used in an enterprise environment where such things are
more commonly required.  In part because .NET/Java provide a much richer
tool chain for such work.  So it self-reinforces.  

PHP is really good at building mash-ups where you have backends to do
the "real" work.  Our Intranet pretty much just federates instances of
OpenGroupware, Cyrus IMAPd, Informix IDS, and BIE.  Now we just need a
*reliable* way to add XMPP to the stack.

> On Wed, May 27, 2009 at 5:06 PM, Ben Rousch <brousch at gmail.com> wrote:
>         I have done some (very) small experiments on something similar
>         to what you're asking for. I did not really look for any sort
>         of robust middleware.
>         I used the Jive/OpenFire/Ignite Realtime (Whatever their name
>         is these days) Smack Java library to create a xmpp client/bot.
>         A user would chat with my xmpp bot, and the bot would process
>         their commands and store the results in a database. I then
>         used LAMP and AJAX to poll the database and post on a web
>         site.