Major refactoring of queue handlers to support running multiple sites in one daemon.

Key changes:
* Initialization code moved from common.php to StatusNet class;
  can now switch configurations during runtime.
* As a consequence, configuration files must now be idempotent...
  Be careful with constant, function or class definitions.
* Control structure for daemons/QueueManager/QueueHandler has been refactored;
  the run loop is now managed by IoMaster run via scripts/queuedaemon.php
  IoManager subclasses are woken to handle socket input or polling, and may
  cover multiple sites.
* Plugins can implement notice queue handlers more easily by registering a
  QueueHandler class; no more need to add a daemon.

The new QueueDaemon runs from scripts/queuedaemon.php:

* This replaces most of the old *handler.php scripts; they've been refactored
  to the bare handler classes.
* Spawns multiple child processes to spread load; defaults to CPU count on
  Linux and Mac OS X systems, or override with --threads=N
* When multithreaded, child processes are automatically respawned on failure.
* Threads gracefully shut down and restart when passing a soft memory limit
  (defaults to 90% of memory_limit), limiting damage from memory leaks.
* Support for UDP-based monitoring: http://www.gitorious.org/snqmon

Rough control flow diagram:
QueueDaemon -> IoMaster -> IoManager
                           QueueManager [listen or poll] -> QueueHandler
                           XmppManager [ping & keepalive]
                           XmppConfirmManager [poll updates]

Todo:

* Respawning features not currently available running single-threaded.
* When running single-site, configuration changes aren't picked up.
* New sites or config changes affecting queue subscriptions are not yet
  handled without a daemon restart.
* SNMP monitoring output to integrate with general tools (nagios, ganglia)
* Convert XMPP confirmation message sends to use stomp queue instead of polling
* Convert xmppdaemon.php to IoManager?
* Convert Twitter status, friends import polling daemons to IoManager
* Clean up some error reporting and failure modes
* May need to adjust queue priorities for best perf in backlog/flood cases

Detailed code history available in my daemon-work branch:
http://www.gitorious.org/~brion/statusnet/brion-fixes/commits/daemon-work

1 files changed, 14 insertions, 180 deletions

diff --git a/lib/common.php b/lib/common.php
index b280afec0..61decebb7 100644
--- a/lib/common.php
+++ b/lib/common.php
@@ -76,159 +76,14 @@ require_once(INSTALLDIR.'/lib/language.php');
 require_once(INSTALLDIR.'/lib/event.php');
 require_once(INSTALLDIR.'/lib/plugin.php');
 
-function _sn_to_path($sn)
-{
-    $past_root = substr($sn, 1);
-    $last_slash = strrpos($past_root, '/');
-    if ($last_slash > 0) {
-        $p = substr($past_root, 0, $last_slash);
-    } else {
-        $p = '';
-    }
-    return $p;
-}
-
-// Save our sanity when code gets loaded through subroutines such as PHPUnit tests
-global $default, $config, $_server, $_path;
-
-// try to figure out where we are. $server and $path
-// can be set by including module, else we guess based
-// on HTTP info.
-
-if (isset($server)) {
-    $_server = $server;
-} else {
-    $_server = array_key_exists('SERVER_NAME', $_SERVER) ?
-      strtolower($_SERVER['SERVER_NAME']) :
-    null;
-}
-
-if (isset($path)) {
-    $_path = $path;
-} else {
-    $_path = (array_key_exists('SERVER_NAME', $_SERVER) && array_key_exists('SCRIPT_NAME', $_SERVER)) ?
-      _sn_to_path($_SERVER['SCRIPT_NAME']) :
-    null;
-}
-
-require_once(INSTALLDIR.'/lib/default.php');
-
-// Set config values initially to default values
-
-$config = $default;
-
-// default configuration, overwritten in config.php
-
-$config['db'] = &PEAR::getStaticProperty('DB_DataObject','options');
-
-$config['db'] = $default['db'];
-
-// Backward compatibility
-
-$config['site']['design'] =& $config['design'];
-
-if (function_exists('date_default_timezone_set')) {
-    /* Work internally in UTC */
-    date_default_timezone_set('UTC');
-}
-
 function addPlugin($name, $attrs = null)
 {
-    $name = ucfirst($name);
-    $pluginclass = "{$name}Plugin";
-
-    if (!class_exists($pluginclass)) {
-
-        $files = array("local/plugins/{$pluginclass}.php",
-                       "local/plugins/{$name}/{$pluginclass}.php",
-                       "local/{$pluginclass}.php",
-                       "local/{$name}/{$pluginclass}.php",
-                       "plugins/{$pluginclass}.php",
-                       "plugins/{$name}/{$pluginclass}.php");
-
-        foreach ($files as $file) {
-            $fullpath = INSTALLDIR.'/'.$file;
-            if (@file_exists($fullpath)) {
-                include_once($fullpath);
-                break;
-            }
-        }
-    }
-
-    $inst = new $pluginclass();
-
-    if (!empty($attrs)) {
-        foreach ($attrs as $aname => $avalue) {
-            $inst->$aname = $avalue;
-        }
-    }
-    return $inst;
-}
-
-// From most general to most specific:
-// server-wide, then vhost-wide, then for a path,
-// finally for a dir (usually only need one of the last two).
-
-if (isset($conffile)) {
-    $_config_files = array($conffile);
-} else {
-    $_config_files = array('/etc/statusnet/statusnet.php',
-                           '/etc/statusnet/laconica.php',
-                           '/etc/laconica/laconica.php',
-                           '/etc/statusnet/'.$_server.'.php',
-                           '/etc/laconica/'.$_server.'.php');
-
-    if (strlen($_path) > 0) {
-        $_config_files[] = '/etc/statusnet/'.$_server.'_'.$_path.'.php';
-        $_config_files[] = '/etc/laconica/'.$_server.'_'.$_path.'.php';
-    }
-
-    $_config_files[] = INSTALLDIR.'/config.php';
-}
-
-global $_have_a_config;
-$_have_a_config = false;
-
-foreach ($_config_files as $_config_file) {
-    if (@file_exists($_config_file)) {
-        include_once($_config_file);
-        $_have_a_config = true;
-    }
+    return StatusNet::addPlugin($name, $attrs);
 }
 
 function _have_config()
 {
-    global $_have_a_config;
-    return $_have_a_config;
-}
-
-// XXX: Throw a conniption if database not installed
-// XXX: Find a way to use htmlwriter for this instead of handcoded markup
-if (!_have_config()) {
-  echo '<p>'. _('No configuration file found. ') .'</p>';
-  echo '<p>'. _('I looked for configuration files in the following places: ') .'<br /> '. implode($_config_files, '<br />');
-  echo '<p>'. _('You may wish to run the installer to fix this.') .'</p>';
-  echo '<a href="install.php">'. _('Go to the installer.') .'</a>';
-  exit;
-}
-// Fixup for statusnet.ini
-
-$_db_name = substr($config['db']['database'], strrpos($config['db']['database'], '/') + 1);
-
-if ($_db_name != 'statusnet' && !array_key_exists('ini_'.$_db_name, $config['db'])) {
-    $config['db']['ini_'.$_db_name] = INSTALLDIR.'/classes/statusnet.ini';
-}
-
-// Backwards compatibility
-
-if (array_key_exists('memcached', $config)) {
-    if ($config['memcached']['enabled']) {
-        addPlugin('Memcache', array('servers' => $config['memcached']['server']));
-    }
-
-    if (!empty($config['memcached']['base'])) {
-        $config['cache']['base'] = $config['memcached']['base'];
-    }
+    return StatusNet::haveConfig();
 }
 
 function __autoload($cls)
@@ -247,27 +102,6 @@ function __autoload($cls)
     }
 }
 
-// Load default plugins
-
-foreach ($config['plugins']['default'] as $name => $params) {
-    if (is_null($params)) {
-        addPlugin($name);
-    } else if (is_array($params)) {
-        if (count($params) == 0) {
-            addPlugin($name);
-        } else {
-            $keys = array_keys($params);
-            if (is_string($keys[0])) {
-                addPlugin($name, $params);
-            } else {
-                foreach ($params as $paramset) {
-                    addPlugin($name, $paramset);
-                }
-            }
-        }
-    }
-}
-
 // XXX: how many of these could be auto-loaded on use?
 // XXX: note that these files should not use config options
 // at compile time since DB config options are not yet loaded.
@@ -283,20 +117,20 @@ require_once INSTALLDIR.'/lib/subs.php';
 require_once INSTALLDIR.'/lib/clientexception.php';
 require_once INSTALLDIR.'/lib/serverexception.php';
 
-// Load settings from database; note we need autoload for this
-
-Config::loadSettings();
-
-// XXX: if plugins should check the schema at runtime, do that here.
-
-if ($config['db']['schemacheck'] == 'runtime') {
-    Event::handle('CheckSchema');
+try {
+    StatusNet::init(@$server, @$path, @$conffile);
+} catch (NoConfigException $e) {
+    // XXX: Throw a conniption if database not installed
+    // XXX: Find a way to use htmlwriter for this instead of handcoded markup
+    echo '<p>'. _('No configuration file found. ') .'</p>';
+    echo '<p>'. _('I looked for configuration files in the following places: ') .'<br/> ';
+    echo implode($e->configFiles, '<br/>');
+    echo '<p>'. _('You may wish to run the installer to fix this.') .'</p>';
+    echo '<a href="install.php">'. _('Go to the installer.') .'</a>';
+    exit;
 }
 
+
 // XXX: other formats here
 
 define('NICKNAME_FMT', VALIDATE_NUM.VALIDATE_ALPHA_LOWER);
-
-// Give plugins a chance to initialize in a fully-prepared environment
-
-Event::handle('InitializePlugin');