summaryrefslogtreecommitdiff
path: root/db
diff options
context:
space:
mode:
authorBrion Vibber <brion@pobox.com>2010-01-22 12:52:36 -0800
committerBrion Vibber <brion@pobox.com>2010-01-22 12:52:36 -0800
commitc7507e7e9dafa6d6e054978e720e4fce3abc9929 (patch)
treed756f2a2c7c9b487b6ef0f066a87f3cde40bc65d /db
parenta3e484a0e898bb94dd45cd7807bea1a931d7c6a9 (diff)
XMPP queued output & initial retooling of DB queue manager to support non-Notice objects.
Queue handlers for XMPP individual & firehose output now send their XML stanzas to another output queue instead of connecting directly to the chat server. This lets us have as many general processing threads as we need, while all actual XMPP input and output go through a single daemon with a single connection open. This avoids problems with multiple connected resources: * multiple windows shown in some chat clients (psi, gajim, kopete) * extra load on server * incoming message delivery forwarding issues Database changes: * queue_item drops 'notice_id' in favor of a 'frame' blob. This is based on Craig Andrews' work branch to generalize queues to take any object, but conservatively leaving out the serialization for now. Table updater (preserves any existing queued items) in db/rc3to09.sql Code changes to watch out for: * Queue handlers should now define a handle() method instead of handle_notice() * QueueDaemon and XmppDaemon now share common i/o (IoMaster) and respawning thread management (RespawningDaemon) infrastructure. * The polling XmppConfirmManager has been dropped, as the message is queued directly when saving IM settings. * Enable $config['queue']['debug_memory'] to output current memory usage at each run through the event loop to watch for memory leaks To do: * Adapt XMPP i/o to component connection mode for multi-site support. * XMPP input can also be broken out to a queue, which would allow the actual notice save etc to be handled by general queue threads. * Make sure there are no problems with simply pushing serialized Notice objects to queues. * Find a way to improve interactive performance of the database-backed queue handler; polling is pretty painful to XMPP. * Possibly redo the way QueueHandlers are injected into a QueueManager. The grouping used to split out the XMPP output queue is a bit awkward. Conflicts: scripts/xmppdaemon.php
Diffstat (limited to 'db')
-rw-r--r--db/08to09.sql16
-rw-r--r--db/rc3to09.sql16
-rw-r--r--db/statusnet.sql5
3 files changed, 34 insertions, 3 deletions
diff --git a/db/08to09.sql b/db/08to09.sql
index d9c25bc72..b10e47dbc 100644
--- a/db/08to09.sql
+++ b/db/08to09.sql
@@ -94,3 +94,19 @@ create table user_location_prefs (
constraint primary key (user_id)
) ENGINE=InnoDB CHARACTER SET utf8 COLLATE utf8_bin;
+create table queue_item_new (
+ id integer auto_increment primary key comment 'unique identifier',
+ frame blob not null comment 'data: object reference or opaque string',
+ transport varchar(8) not null comment 'queue for what? "email", "jabber", "sms", "irc", ...',
+ created datetime not null comment 'date this record was created',
+ claimed datetime comment 'date this item was claimed',
+
+ index queue_item_created_idx (created)
+
+) ENGINE=InnoDB CHARACTER SET utf8 COLLATE utf8_bin;
+
+insert into queue_item_new (frame,transport,created,claimed)
+ select notice_id,transport,created,claimed from queue_item;
+alter table queue_item rename to queue_item_old;
+alter table queue_item_new rename to queue_item;
+
diff --git a/db/rc3to09.sql b/db/rc3to09.sql
new file mode 100644
index 000000000..02dc7a6e2
--- /dev/null
+++ b/db/rc3to09.sql
@@ -0,0 +1,16 @@
+create table queue_item_new (
+ id integer auto_increment primary key comment 'unique identifier',
+ frame blob not null comment 'data: object reference or opaque string',
+ transport varchar(8) not null comment 'queue for what? "email", "jabber", "sms", "irc", ...',
+ created datetime not null comment 'date this record was created',
+ claimed datetime comment 'date this item was claimed',
+
+ index queue_item_created_idx (created)
+
+) ENGINE=InnoDB CHARACTER SET utf8 COLLATE utf8_bin;
+
+insert into queue_item_new (frame,transport,created,claimed)
+ select notice_id,transport,created,claimed from queue_item;
+alter table queue_item rename to queue_item_old;
+alter table queue_item_new rename to queue_item;
+
diff --git a/db/statusnet.sql b/db/statusnet.sql
index cb33ccf33..a9ed66cb4 100644
--- a/db/statusnet.sql
+++ b/db/statusnet.sql
@@ -244,13 +244,12 @@ create table remember_me (
) ENGINE=InnoDB CHARACTER SET utf8 COLLATE utf8_bin;
create table queue_item (
-
- notice_id integer not null comment 'notice queued' references notice (id),
+ id integer auto_increment primary key comment 'unique identifier',
+ frame blob not null comment 'data: object reference or opaque string',
transport varchar(8) not null comment 'queue for what? "email", "jabber", "sms", "irc", ...',
created datetime not null comment 'date this record was created',
claimed datetime comment 'date this item was claimed',
- constraint primary key (notice_id, transport),
index queue_item_created_idx (created)
) ENGINE=InnoDB CHARACTER SET utf8 COLLATE utf8_bin;