Merge branch 'documentation-update' into 'master'

Documentation update

See merge request poezio/slixmpp!116

8 files changed, 1202 insertions, 0 deletions

diff --git a/docs/howto/create_plugin.rst b/docs/howto/create_plugin.rst
new file mode 100644
index 00000000..437374c7
--- /dev/null
+++ b/docs/howto/create_plugin.rst
@@ -0,0 +1,682 @@
+.. _create-plugin:
+
+Creating a Slixmpp Plugin
+===========================
+
+One of the goals of Slixmpp is to provide support for every draft or final
+XMPP extension (`XEP <http://xmpp.org/extensions/>`_). To do this, Slixmpp has a
+plugin mechanism for adding the functionalities required by each XEP. But even
+though plugins were made to quickly implement and prototype the official XMPP
+extensions, there is no reason you can't create your own plugin to implement
+your own custom XMPP-based protocol.
+
+This guide will help walk you through the steps to
+implement a rudimentary version of `XEP-0077 In-band
+Registration <http://xmpp.org/extensions/xep-0077.html>`_. In-band registration
+was implemented in example 14-6 (page 223) of `XMPP: The Definitive
+Guide <http://oreilly.com/catalog/9780596521271>`_ because there was no Slixmpp
+plugin for XEP-0077 at the time of writing. We will partially fix that issue
+here by turning the example implementation from *XMPP: The Definitive Guide*
+into a plugin. Again, note that this will not a complete implementation, and a
+different, more robust, official plugin for XEP-0077 may be added to Slixmpp
+in the future.
+
+.. note::
+
+    The example plugin created in this guide is for the server side of the
+    registration process only. It will **NOT** be able to register new accounts
+    on an XMPP server.
+
+First Steps
+-----------
+Every plugin inherits from the class :mod:`BasePlugin <slixmpp.plugins.base.BasePlugin`,
+and must include a ``plugin_init`` method. While the
+plugins distributed with Slixmpp must be placed in the plugins directory
+``slixmpp/plugins`` to be loaded, custom plugins may be loaded from any
+module. To do so, use the following form when registering the plugin:
+
+.. code-block:: python
+
+    self.register_plugin('myplugin', module=mod_containing_my_plugin)
+
+The plugin name must be the same as the plugin's class name.
+
+Now, we can open our favorite text editors and create ``xep_0077.py`` in
+``Slixmpp/slixmpp/plugins``. We want to do some basic house-keeping and
+declare the name and description of the XEP we are implementing. If you
+are creating your own custom plugin, you don't need to include the ``xep``
+attribute.
+
+.. code-block:: python
+
+    """
+    Creating a Slixmpp Plugin
+
+    This is a minimal implementation of XEP-0077 to serve
+    as a tutorial for creating Slixmpp plugins.
+    """
+
+    from slixmpp.plugins.base import BasePlugin
+
+    class xep_0077(BasePlugin):
+        """
+        XEP-0077 In-Band Registration
+        """
+
+        def plugin_init(self):
+            self.description = "In-Band Registration"
+            self.xep = "0077"
+
+Now that we have a basic plugin, we need to edit
+``slixmpp/plugins/__init__.py`` to include our new plugin by adding
+``'xep_0077'`` to the ``__all__`` declaration.
+
+Interacting with Other Plugins
+------------------------------
+
+In-band registration is a feature that should be advertised through `Service
+Discovery <http://xmpp.org/extensions/xep-0030.html>`_. To do that, we tell the
+``xep_0030`` plugin to add the ``"jabber:iq:register"`` feature. We put this
+call in a method named ``post_init`` which will be called once the plugin has
+been loaded; by doing so we advertise that we can do registrations only after we
+finish activating the plugin.
+
+The ``post_init`` method needs to call ``BasePlugin.post_init(self)``
+which will mark that ``post_init`` has been called for the plugin. Once the
+Slixmpp object begins processing, ``post_init`` will be called on any plugins
+that have not already run ``post_init``. This allows you to register plugins and
+their dependencies without needing to worry about the order in which you do so.
+
+**Note:** by adding this call we have introduced a dependency on the XEP-0030
+plugin. Be sure to register ``'xep_0030'`` as well as ``'xep_0077'``. Slixmpp
+does not automatically load plugin dependencies for you.
+
+.. code-block:: python
+
+    def post_init(self):
+        BasePlugin.post_init(self)
+        self.xmpp['xep_0030'].add_feature("jabber:iq:register")
+
+Creating Custom Stanza Objects
+------------------------------
+
+Now, the IQ stanzas needed to implement our version of XEP-0077 are not very
+complex, and we could just interact with the XML objects directly just like
+in the *XMPP: The Definitive Guide* example. However, creating custom stanza
+objects is good practice.
+
+We will create a new ``Registration`` stanza. Following the *XMPP: The
+Definitive Guide* example, we will add support for a username and password
+field. We also need two flags: ``registered`` and ``remove``. The ``registered``
+flag is sent when an already registered user attempts to register, along with
+their registration data. The ``remove`` flag is a request to unregister a user's
+account.
+
+Adding additional `fields specified in
+XEP-0077 <http://xmpp.org/extensions/xep-0077.html#registrar-formtypes-register>`_
+will not be difficult and is left as an exercise for the reader.
+
+Our ``Registration`` class needs to start with a few descriptions of its
+behaviour:
+
+* ``namespace``
+    The namespace our stanza object lives in. In this case,
+    ``"jabber:iq:register"``.
+
+* ``name``
+    The name of the root XML element. In this case, the ``query`` element.
+
+* ``plugin_attrib``
+    The name to access this type of stanza. In particular, given a
+    registration stanza, the ``Registration`` object can be found using:
+    ``iq_object['register']``.
+
+* ``interfaces``
+    A list of dictionary-like keys that can be used with the stanza object.
+    When using ``"key"``, if there exists a method of the form ``getKey``,
+    ``setKey``, or``delKey`` (depending on context) then the result of calling
+    that method will be returned. Otherwise, the value of the attribute ``key``
+    of the main stanza element is returned if one exists.
+
+    **Note:** The accessor methods currently use title case, and not camel case.
+    Thus if you need to access an item named ``"methodName"`` you will need to
+    use ``getMethodname``. This naming convention might change to full camel
+    case in a future version of Slixmpp.
+
+* ``sub_interfaces``
+    A subset of ``interfaces``, but these keys map to the text of any
+    subelements that are direct children of the main stanza element. Thus,
+    referencing ``iq_object['register']['username']`` will either execute
+    ``getUsername`` or return the value in the ``username`` element of the
+    query.
+
+    If you need to access an element, say ``elem``, that is not a direct child
+    of the main stanza element, you will need to add ``getElem``, ``setElem``,
+    and ``delElem``. See the note above about naming conventions.
+
+.. code-block:: python
+
+    from slixmpp.xmlstream import ElementBase, ET, JID, register_stanza_plugin
+    from slixmpp import Iq
+
+    class Registration(ElementBase):
+        namespace = 'jabber:iq:register'
+        name = 'query'
+        plugin_attrib = 'register'
+        interfaces = {'username', 'password', 'registered', 'remove'}
+        sub_interfaces = interfaces
+
+        def getRegistered(self):
+            present = self.xml.find('{%s}registered' % self.namespace)
+            return present is not None
+
+        def getRemove(self):
+            present = self.xml.find('{%s}remove' % self.namespace)
+            return present is not None
+
+        def setRegistered(self, registered):
+            if registered:
+                self.addField('registered')
+            else:
+                del self['registered']
+
+        def setRemove(self, remove):
+            if remove:
+                self.addField('remove')
+            else:
+                del self['remove']
+
+        def addField(self, name):
+            itemXML = ET.Element('{%s}%s' % (self.namespace, name))
+            self.xml.append(itemXML)
+
+Setting a ``sub_interface`` attribute to ``""`` will remove that subelement.
+Since we want to include empty registration fields in our form, we need the
+``addField`` method to add the empty elements.
+
+Since the ``registered`` and ``remove`` elements are just flags, we need to add
+custom logic to enforce the binary behavior.
+
+Extracting Stanzas from the XML Stream
+--------------------------------------
+
+Now that we have a custom stanza object, we need to be able to detect when we
+receive one. To do this, we register a stream handler that will pattern match
+stanzas off of the XML stream against our stanza object's element name and
+namespace. To do so, we need to create a ``Callback`` object which contains
+an XML fragment that can identify our stanza type. We can add this handler
+registration to our ``plugin_init`` method.
+
+Also, we need to associate our ``Registration`` class with IQ stanzas;
+that requires the use of the ``register_stanza_plugin`` function (in
+``slixmpp.xmlstream.stanzabase``) which takes the class of a parent stanza
+type followed by the substanza type. In our case, the parent stanza is an IQ
+stanza, and the substanza is our registration query.
+
+The ``__handleRegistration`` method referenced in the callback will be our
+handler function to process registration requests.
+
+.. code-block:: python
+
+    def plugin_init(self):
+        self.description = "In-Band Registration"
+        self.xep = "0077"
+
+        self.xmpp.register_handler(
+          Callback('In-Band Registration',
+            MatchXPath('{%s}iq/{jabber:iq:register}query' % self.xmpp.default_ns),
+            self.__handleRegistration))
+        register_stanza_plugin(Iq, Registration)
+
+Handling Incoming Stanzas and Triggering Events
+-----------------------------------------------
+There are six situations that we need to handle to finish our implementation of
+XEP-0077.
+
+**Registration Form Request from a New User:**
+
+    .. code-block:: xml
+
+        <iq type="result">
+         <query xmlns="jabber:iq:register">
+          <username />
+          <password />
+         </query>
+        </iq>
+
+**Registration Form Request from an Existing User:**
+
+    .. code-block:: xml
+
+        <iq type="result">
+         <query xmlns="jabber:iq:register">
+          <registered />
+          <username>Foo</username>
+          <password>hunter2</password>
+         </query>
+        </iq>
+
+**Unregister Account:**
+
+    .. code-block:: xml
+
+        <iq type="result">
+         <query xmlns="jabber:iq:register" />
+        </iq>
+
+**Incomplete Registration:**
+
+    .. code-block:: xml
+
+        <iq type="error">
+          <query xmlns="jabber:iq:register">
+            <username>Foo</username>
+          </query>
+         <error code="406" type="modify">
+          <not-acceptable xmlns="urn:ietf:params:xml:ns:xmpp-stanzas" />
+         </error>
+        </iq>
+
+**Conflicting Registrations:**
+
+    .. code-block:: xml
+
+        <iq type="error">
+         <query xmlns="jabber:iq:register">
+          <username>Foo</username>
+          <password>hunter2</password>
+         </query>
+         <error code="409" type="cancel">
+          <conflict xmlns="urn:ietf:params:xml:ns:xmpp-stanzas" />
+         </error>
+        </iq>
+
+**Successful Registration:**
+
+    .. code-block:: xml
+
+        <iq type="result">
+         <query xmlns="jabber:iq:register" />
+        </iq>
+
+Cases 1 and 2: Registration Requests
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Responding to registration requests depends on if the requesting user already
+has an account. If there is an account, the response should include the
+``registered`` flag and the user's current registration information. Otherwise,
+we just send the fields for our registration form.
+
+We will handle both cases by creating a ``sendRegistrationForm`` method that
+will create either an empty of full form depending on if we provide it with
+user data. Since we need to know which form fields to include (especially if we
+add support for the other fields specified in XEP-0077), we will also create a
+method ``setForm`` which will take the names of the fields we wish to include.
+
+.. code-block:: python
+
+    def plugin_init(self):
+        self.description = "In-Band Registration"
+        self.xep = "0077"
+        self.form_fields = ('username', 'password')
+        ... remainder of plugin_init
+
+    ...
+
+    def __handleRegistration(self, iq):
+        if iq['type'] == 'get':
+            # Registration form requested
+            userData = self.backend[iq['from'].bare]
+            self.sendRegistrationForm(iq, userData)
+
+    def setForm(self, *fields):
+        self.form_fields = fields
+
+    def sendRegistrationForm(self, iq, userData=None):
+        reg = iq['register']
+        if userData is None:
+            userData = {}
+        else:
+            reg['registered'] = True
+
+        for field in self.form_fields:
+            data = userData.get(field, '')
+            if data:
+                # Add field with existing data
+                reg[field] = data
+            else:
+                # Add a blank field
+                reg.addField(field)
+
+        iq.reply().set_payload(reg.xml)
+        iq.send()
+
+Note how we are able to access our ``Registration`` stanza object with
+``iq['register']``.
+
+A User Backend
+++++++++++++++
+You might have noticed the reference to ``self.backend``, which is an object
+that abstracts away storing and retrieving user information. Since it is not
+much more than a dictionary, we will leave the implementation details to the
+final, full source code example.
+
+Case 3: Unregister an Account
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+The next simplest case to consider is responding to a request to remove
+an account. If we receive a ``remove`` flag, we instruct the backend to
+remove the user's account. Since your application may need to know about
+when users are registered or unregistered, we trigger an event using
+``self.xmpp.event('unregister_user', iq)``. See the component examples below for
+how to respond to that event.
+
+.. code-block:: python
+
+     def __handleRegistration(self, iq):
+        if iq['type'] == 'get':
+            # Registration form requested
+            userData = self.backend[iq['from'].bare]
+            self.sendRegistrationForm(iq, userData)
+        elif iq['type'] == 'set':
+            # Remove an account
+            if iq['register']['remove']:
+                self.backend.unregister(iq['from'].bare)
+                self.xmpp.event('unregistered_user', iq)
+                iq.reply().send()
+                return
+
+Case 4: Incomplete Registration
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+For the next case we need to check the user's registration to ensure it has all
+of the fields we wanted. The simple option that we will use is to loop over the
+field names and check each one; however, this means that all fields we send to
+the user are required. Adding optional fields is left to the reader.
+
+Since we have received an incomplete form, we need to send an error message back
+to the user. We have to send a few different types of errors, so we will also
+create a ``_sendError`` method that will add the appropriate ``error`` element
+to the IQ reply.
+
+.. code-block:: python
+
+    def __handleRegistration(self, iq):
+        if iq['type'] == 'get':
+            # Registration form requested
+            userData = self.backend[iq['from'].bare]
+            self.sendRegistrationForm(iq, userData)
+        elif iq['type'] == 'set':
+            if iq['register']['remove']:
+                # Remove an account
+                self.backend.unregister(iq['from'].bare)
+                self.xmpp.event('unregistered_user', iq)
+                iq.reply().send()
+                return
+
+            for field in self.form_fields:
+                if not iq['register'][field]:
+                    # Incomplete Registration
+                    self._sendError(iq, '406', 'modify', 'not-acceptable'
+                                    "Please fill in all fields.")
+                    return
+
+    ...
+
+    def _sendError(self, iq, code, error_type, name, text=''):
+        iq.reply().set_payload(iq['register'].xml)
+        iq.error()
+        iq['error']['code'] = code
+        iq['error']['type'] = error_type
+        iq['error']['condition'] = name
+        iq['error']['text'] = text
+        iq.send()
+
+Cases 5 and 6: Conflicting and Successful Registration
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+We are down to the final decision on if we have a successful registration. We
+send the user's data to the backend with the ``self.backend.register`` method.
+If it returns ``True``, then registration has been successful. Otherwise,
+there has been a conflict with usernames and registration has failed. Like
+with unregistering an account, we trigger an event indicating that a user has
+been registered by using ``self.xmpp.event('registered_user', iq)``. See the
+component examples below for how to respond to this event.
+
+.. code-block:: python
+
+    def __handleRegistration(self, iq):
+        if iq['type'] == 'get':
+            # Registration form requested
+            userData = self.backend[iq['from'].bare]
+            self.sendRegistrationForm(iq, userData)
+        elif iq['type'] == 'set':
+            if iq['register']['remove']:
+                # Remove an account
+                self.backend.unregister(iq['from'].bare)
+                self.xmpp.event('unregistered_user', iq)
+                iq.reply().send()
+                return
+
+            for field in self.form_fields:
+                if not iq['register'][field]:
+                    # Incomplete Registration
+                    self._sendError(iq, '406', 'modify', 'not-acceptable',
+                                    "Please fill in all fields.")
+                    return
+
+            if self.backend.register(iq['from'].bare, iq['register']):
+                # Successful registration
+                self.xmpp.event('registered_user', iq)
+                iq.reply().set_payload(iq['register'].xml)
+                iq.send()
+            else:
+                # Conflicting registration
+                self._sendError(iq, '409', 'cancel', 'conflict',
+                                "That username is already taken.")
+
+Example Component Using the XEP-0077 Plugin
+-------------------------------------------
+Alright, the moment we've been working towards - actually using our plugin to
+simplify our other applications. Here is a basic component that simply manages
+user registrations and sends the user a welcoming message when they register,
+and a farewell message when they delete their account.
+
+Note that we have to register the ``'xep_0030'`` plugin first,
+and that we specified the form fields we wish to use with
+``self.xmpp.plugin['xep_0077'].setForm('username', 'password')``.
+
+.. code-block:: python
+
+    import slixmpp.componentxmpp
+
+    class Example(slixmpp.componentxmpp.ComponentXMPP):
+
+        def __init__(self, jid, password):
+            slixmpp.componentxmpp.ComponentXMPP.__init__(self, jid, password, 'localhost', 8888)
+
+            self.register_plugin('xep_0030')
+            self.register_plugin('xep_0077')
+            self.plugin['xep_0077'].setForm('username', 'password')
+
+            self.add_event_handler("registered_user", self.reg)
+            self.add_event_handler("unregistered_user", self.unreg)
+
+        def reg(self, iq):
+            msg = "Welcome! %s" % iq['register']['username']
+            self.send_message(iq['from'], msg, mfrom=self.fulljid)
+
+        def unreg(self, iq):
+            msg = "Bye! %s" % iq['register']['username']
+            self.send_message(iq['from'], msg, mfrom=self.fulljid)
+
+**Congratulations!** We now have a basic, functioning implementation of
+XEP-0077.
+
+Complete Source Code for XEP-0077 Plugin
+----------------------------------------
+Here is a copy of a more complete implementation of the plugin we created, but
+with some additional registration fields implemented.
+
+.. code-block:: python
+
+    """
+    Creating a Slixmpp Plugin
+
+    This is a minimal implementation of XEP-0077 to serve
+    as a tutorial for creating Slixmpp plugins.
+    """
+
+    from slixmpp.plugins.base import BasePlugin
+    from slixmpp.xmlstream.handler.callback import Callback
+    from slixmpp.xmlstream.matcher.xpath import MatchXPath
+    from slixmpp.xmlstream import ElementBase, ET, JID, register_stanza_plugin
+    from slixmpp import Iq
+    import copy
+
+
+    class Registration(ElementBase):
+        namespace = 'jabber:iq:register'
+        name = 'query'
+        plugin_attrib = 'register'
+        interfaces = {'username', 'password', 'email', 'nick', 'name',
+                      'first', 'last', 'address', 'city', 'state', 'zip',
+                      'phone', 'url', 'date', 'misc', 'text', 'key',
+                      'registered', 'remove', 'instructions'}
+        sub_interfaces = interfaces
+
+        def getRegistered(self):
+            present = self.xml.find('{%s}registered' % self.namespace)
+            return present is not None
+
+        def getRemove(self):
+            present = self.xml.find('{%s}remove' % self.namespace)
+            return present is not None
+
+        def setRegistered(self, registered):
+            if registered:
+                self.addField('registered')
+            else:
+                del self['registered']
+
+        def setRemove(self, remove):
+            if remove:
+                self.addField('remove')
+            else:
+                del self['remove']
+
+        def addField(self, name):
+            itemXML = ET.Element('{%s}%s' % (self.namespace, name))
+            self.xml.append(itemXML)
+
+
+    class UserStore(object):
+        def __init__(self):
+            self.users = {}
+
+        def __getitem__(self, jid):
+            return self.users.get(jid, None)
+
+        def register(self, jid, registration):
+            username = registration['username']
+
+            def filter_usernames(user):
+                return user != jid and self.users[user]['username'] == username
+
+            conflicts = filter(filter_usernames, self.users.keys())
+            if conflicts:
+                return False
+
+            self.users[jid] = registration
+            return True
+
+        def unregister(self, jid):
+            del self.users[jid]
+
+    class xep_0077(BasePlugin):
+        """
+        XEP-0077 In-Band Registration
+        """
+
+        def plugin_init(self):
+            self.description = "In-Band Registration"
+            self.xep = "0077"
+            self.form_fields = ('username', 'password')
+            self.form_instructions = ""
+            self.backend = UserStore()
+
+            self.xmpp.register_handler(
+                Callback('In-Band Registration',
+                         MatchXPath('{%s}iq/{jabber:iq:register}query' % self.xmpp.default_ns),
+                         self.__handleRegistration))
+            register_stanza_plugin(Iq, Registration)
+
+        def post_init(self):
+            BasePlugin.post_init(self)
+            self.xmpp['xep_0030'].add_feature("jabber:iq:register")
+
+        def __handleRegistration(self, iq):
+            if iq['type'] == 'get':
+                # Registration form requested
+                userData = self.backend[iq['from'].bare]
+                self.sendRegistrationForm(iq, userData)
+            elif iq['type'] == 'set':
+                if iq['register']['remove']:
+                    # Remove an account
+                    self.backend.unregister(iq['from'].bare)
+                    self.xmpp.event('unregistered_user', iq)
+                    iq.reply().send()
+                    return
+
+                for field in self.form_fields:
+                    if not iq['register'][field]:
+                        # Incomplete Registration
+                        self._sendError(iq, '406', 'modify', 'not-acceptable',
+                                        "Please fill in all fields.")
+                        return
+
+                if self.backend.register(iq['from'].bare, iq['register']):
+                    # Successful registration
+                    self.xmpp.event('registered_user', iq)
+                    reply = iq.reply()
+                    reply.set_payload(iq['register'].xml)
+                    reply.send()
+                else:
+                    # Conflicting registration
+                    self._sendError(iq, '409', 'cancel', 'conflict',
+                                    "That username is already taken.")
+
+        def setForm(self, *fields):
+            self.form_fields = fields
+
+        def setInstructions(self, instructions):
+            self.form_instructions = instructions
+
+        def sendRegistrationForm(self, iq, userData=None):
+            reg = iq['register']
+            if userData is None:
+                userData = {}
+            else:
+                reg['registered'] = True
+
+            if self.form_instructions:
+                reg['instructions'] = self.form_instructions
+
+            for field in self.form_fields:
+                data = userData.get(field, '')
+                if data:
+                    # Add field with existing data
+                    reg[field] = data
+                else:
+                    # Add a blank field
+                    reg.addField(field)
+
+            reply = iq.reply()
+            reply.set_payload(reg.xml)
+            reply.send()
+
+        def _sendError(self, iq, code, error_type, name, text=''):
+            reply = iq.reply()
+            reply.set_payload(iq['register'].xml)
+            reply.error()
+            reply['error']['code'] = code
+            reply['error']['type'] = error_type
+            reply['error']['condition'] = name
+            reply['error']['text'] = text
+            reply.send()
diff --git a/docs/howto/features.rst b/docs/howto/features.rst
new file mode 100644
index 00000000..4d93d5c3
--- /dev/null
+++ b/docs/howto/features.rst
@@ -0,0 +1,2 @@
+How to Use Stream Features
+==========================
diff --git a/docs/howto/guide_xep_0030.rst b/docs/howto/guide_xep_0030.rst
new file mode 100644
index 00000000..a3af4857
--- /dev/null
+++ b/docs/howto/guide_xep_0030.rst
@@ -0,0 +1,200 @@
+XEP-0030: Working with Service Discovery
+========================================
+
+XMPP networks can be composed of many individual clients, components,
+and servers. Determining the JIDs for these entities and the various
+features they may support is the role of `XEP-0030, Service
+Discovery <http://xmpp.org/extensions/xep-0030.html>`_, or "disco" for short.
+
+Every XMPP entity may possess what are called nodes. A node is just a name for
+some aspect of an XMPP entity. For example, if an XMPP entity provides `Ad-Hoc
+Commands <http://xmpp.org/extensions/xep-0050.html>`_, then it will have a node
+named ``http://jabber.org/protocol/commands`` which will contain information
+about the commands provided. Other agents using these ad-hoc commands will
+interact with the information provided by this node. Note that the node name is
+just an identifier; there is no inherent meaning.
+
+Working with service discovery is about creating and querying these nodes.
+According to XEP-0030, a node may contain three types of information:
+identities, features, and items. (Further, extensible, information types are
+defined in `XEP-0128 <http://xmpp.org/extensions/xep-0128.html>`_, but they are
+not yet implemented by Slixmpp.) Slixmpp provides methods to configure each
+of these node attributes.
+
+Configuring Service Discovery
+-----------------------------
+The design focus for the XEP-0030 plug-in is handling info and items requests
+in a dynamic fashion, allowing for complex policy decisions of who may receive
+information and how much, or use alternate backend storage mechanisms for all
+of the disco data. To do this, each action that the XEP-0030 plug-in performs
+is handed off to what is called a "node handler," which is just a callback
+function. These handlers are arranged in a hierarchy that allows for a single
+handler to manage an entire domain of JIDs (say for a component), while allowing
+other handler functions to override that global behaviour for certain JIDs, or
+even further limited to only certain JID and node combinations.
+
+The Dynamic Handler Hierarchy
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+* ``global``: (JID is None, node is None)
+
+  Handlers assigned at this level for an action (such as ``add_feature``) provide a global default
+  behaviour when the action is performed.
+
+* ``jid``: (JID assigned, node is None)
+
+  At this level, handlers provide a default behaviour for actions affecting any node owned by the
+  JID in question. This level is most useful for component connections; there is effectively no
+  difference between this and the global level when using a client connection.
+
+* ``node``: (JID assigned, node assigned)
+
+  A handler for this level is responsible for carrying out an action for only one node, and is the
+  most specific handler type available. These types of handlers will be most useful for "special"
+  nodes that require special processing different than others provided by the JID, such as using
+  access control lists, or consolidating data from other nodes.
+
+Default Static Handlers
+~~~~~~~~~~~~~~~~~~~~~~~
+The XEP-0030 plug-in provides a default set of handlers that work using in-memory
+disco stanzas. Each handler simply performs the appropriate lookup or storage
+operation using these stanzas without doing any complex operations such as
+checking an ACL, etc.
+
+You may find it necessary at some point to revert a particular node or JID to
+using the default, static handlers. To do so, use the method ``restore_defaults()``.
+You may also elect to only convert a given set of actions instead.
+
+Creating a Node Handler
+~~~~~~~~~~~~~~~~~~~~~~~
+Every node handler receives three arguments: the JID, the node, and a data
+parameter that will contain the relevant information for carrying out the
+handler's action, typically a dictionary.
+
+The JID will always have a value, defaulting to ``xmpp.boundjid.full`` for
+components or ``xmpp.boundjid.bare`` for clients. The node value may be None or
+a string.
+
+Only handlers for the actions ``get_info`` and ``get_items`` need to have return
+values. For these actions, DiscoInfo or DiscoItems stanzas are exepected as
+output. It is also acceptable for handlers for these actions to generate an
+XMPPError exception when necessary.
+
+Example Node Handler:
++++++++++++++++++++++
+Here is one of the built-in default handlers as an example:
+
+.. code-block:: python
+
+    def add_identity(self, jid, node, data):
+        """
+        Add a new identity to the JID/node combination.
+
+        The data parameter may provide:
+            category -- The general category to which the agent belongs.
+            itype    -- A more specific designation with the category.
+            name     -- Optional human readable name for this identity.
+            lang     -- Optional standard xml:lang value.
+        """
+        self.add_node(jid, node)
+        self.nodes[(jid, node)]['info'].add_identity(
+                data.get('category', ''),
+                data.get('itype', ''),
+                data.get('name', None),
+                data.get('lang', None))
+
+Adding Identities, Features, and Items
+--------------------------------------
+In order to maintain some backwards compatibility, the methods ``add_identity``,
+``add_feature``, and ``add_item`` do not follow the method signature pattern of
+the other API methods (i.e. jid, node, then other options), but rather retain
+the parameter orders from previous plug-in versions.
+
+Adding an Identity
+~~~~~~~~~~~~~~~~~~
+Adding an identity may be done using either the older positional notation, or
+with keyword parameters. The example below uses the keyword arguments, but in
+the same order as expected using positional arguments.
+
+.. code-block:: python
+
+    xmpp['xep_0030'].add_identity(category='client',
+                                  itype='bot',
+                                  name='Slixmpp',
+                                  node='foo',
+                                  jid=xmpp.boundjid.full,
+                                  lang='no')
+
+The JID and node values determine which handler will be used to perform the
+``add_identity`` action.
+
+The ``lang`` parameter allows for adding localized versions of identities using
+the ``xml:lang`` attribute.
+
+Adding a Feature
+~~~~~~~~~~~~~~~~
+The position ordering for ``add_feature()`` is to include the feature, then
+specify the node and then the JID. The JID and node values determine which
+handler will be used to perform the ``add_feature`` action.
+
+.. code-block:: python
+
+    xmpp['xep_0030'].add_feature(feature='jabber:x:data',
+                                 node='foo',
+                                 jid=xmpp.boundjid.full)
+
+Adding an Item
+~~~~~~~~~~~~~~
+The parameters to ``add_item()`` are potentially confusing due to the fact that
+adding an item requires two JID and node combinations: the JID and node of the
+item itself, and the JID and node that will own the item.
+
+.. code-block:: python
+
+    xmpp['xep_0030'].add_item(jid='myitemjid@example.com',
+                              name='An Item!',
+                              node='owner_node',
+                              subnode='item_node',
+                              ijid=xmpp.boundjid.full)
+
+.. note::
+
+    In this case, the owning JID and node are provided with the
+    parameters ``ijid`` and ``node``.
+
+Performing Disco Queries
+------------------------
+The methods ``get_info()`` and ``get_items()`` are used to query remote JIDs
+and their nodes for disco information. Since these methods are wrappers for
+sending Iq stanzas, they also accept all of the parameters of the ``Iq.send()``
+method. The ``get_items()`` method may also accept the boolean parameter
+``iterator``, which when set to ``True`` will return an iterator object using
+the `XEP-0059 <http://xmpp.org/extensions/xep-0059.html>`_ plug-in.
+
+.. code-block:: python
+
+    info = yield from self['xep_0030'].get_info(jid='foo@example.com',
+                                                node='bar',
+                                                ifrom='baz@mycomponent.example.com',
+                                                timeout=30)
+
+    items = self['xep_0030'].get_info(jid='foo@example.com',
+                                      node='bar',
+                                      iterator=True)
+
+For more examples on how to use basic disco queries, check the ``disco_browser.py``
+example in the ``examples`` directory.
+
+Local Queries
+~~~~~~~~~~~~~
+In some cases, it may be necessary to query the contents of a node owned by the
+client itself, or one of a component's many JIDs. The same method is used as for
+normal queries, with two differences. First, the parameter ``local=True`` must
+be used. Second, the return value will be a DiscoInfo or DiscoItems stanza, not
+a full Iq stanza.
+
+.. code-block:: python
+
+    info = self['xep_0030'].get_info(node='foo', local=True)
+    items = self['xep_0030'].get_items(jid='somejid@mycomponent.example.com',
+                                       node='bar',
+                                       local=True)
diff --git a/docs/howto/handlersmatchers.rst b/docs/howto/handlersmatchers.rst
new file mode 100644
index 00000000..628c4142
--- /dev/null
+++ b/docs/howto/handlersmatchers.rst
@@ -0,0 +1,4 @@
+.. _using-handlers-matchers:
+
+Using Stream Handlers and Matchers
+==================================
diff --git a/docs/howto/index.rst b/docs/howto/index.rst
new file mode 100644
index 00000000..b05dc499
--- /dev/null
+++ b/docs/howto/index.rst
@@ -0,0 +1,13 @@
+Tutorials, FAQs, and How To Guides
+----------------------------------
+
+.. toctree::
+    :maxdepth: 2
+
+    stanzas
+    create_plugin
+    features
+    sasl
+    handlersmatchers
+    guide_xep_0030
+    xmpp_tdg
diff --git a/docs/howto/sasl.rst b/docs/howto/sasl.rst
new file mode 100644
index 00000000..46c45c2a
--- /dev/null
+++ b/docs/howto/sasl.rst
@@ -0,0 +1,2 @@
+How SASL Authentication Works
+=============================
diff --git a/docs/howto/xeps.rst b/docs/howto/xeps.rst
new file mode 100644
index 00000000..c5b4592d
--- /dev/null
+++ b/docs/howto/xeps.rst
@@ -0,0 +1,50 @@
+Supported XEPS
+==============
+
+======= ============================= ================
+XEP     Description                   Notes
+======= ============================= ================
+`0004`_ Data forms
+`0009`_ Jabber RPC
+`0012`_ Last Activity
+`0030`_ Service Discovery
+`0033`_ Extended Stanza Addressing
+`0045`_ Multi-User Chat (MUC)         Client-side only
+`0050`_ Ad-hoc Commands
+`0059`_ Result Set Management
+`0060`_ Publish/Subscribe (PubSub)    Client-side only
+`0066`_ Out-of-band Data
+`0078`_ Non-SASL Authentication
+`0082`_ XMPP Date and Time Profiles
+`0085`_ Chat-State Notifications
+`0086`_ Error Condition Mappings
+`0092`_ Software Version
+`0128`_ Service Discovery Extensions
+`0202`_ Entity Time
+`0203`_ Delayed Delivery
+`0224`_ Attention
+`0249`_ Direct MUC Invitations
+======= ============================= ================
+
+
+.. _0004: http://xmpp.org/extensions/xep-0004.html
+.. _0009: http://xmpp.org/extensions/xep-0009.html
+.. _0012: http://xmpp.org/extensions/xep-0012.html
+.. _0030: http://xmpp.org/extensions/xep-0030.html
+.. _0033: http://xmpp.org/extensions/xep-0033.html
+.. _0045: http://xmpp.org/extensions/xep-0045.html
+.. _0050: http://xmpp.org/extensions/xep-0050.html
+.. _0059: http://xmpp.org/extensions/xep-0059.html
+.. _0060: http://xmpp.org/extensions/xep-0060.html
+.. _0066: http://xmpp.org/extensions/xep-0066.html
+.. _0078: http://xmpp.org/extensions/xep-0078.html
+.. _0082: http://xmpp.org/extensions/xep-0082.html
+.. _0085: http://xmpp.org/extensions/xep-0085.html
+.. _0086: http://xmpp.org/extensions/xep-0086.html
+.. _0092: http://xmpp.org/extensions/xep-0092.html
+.. _0128: http://xmpp.org/extensions/xep-0128.html
+.. _0199: http://xmpp.org/extensions/xep-0199.html
+.. _0202: http://xmpp.org/extensions/xep-0202.html
+.. _0203: http://xmpp.org/extensions/xep-0203.html
+.. _0224: http://xmpp.org/extensions/xep-0224.html
+.. _0249: http://xmpp.org/extensions/xep-0249.html
diff --git a/docs/howto/xmpp_tdg.rst b/docs/howto/xmpp_tdg.rst
new file mode 100644
index 00000000..53194e13
--- /dev/null
+++ b/docs/howto/xmpp_tdg.rst
@@ -0,0 +1,249 @@
+Following *XMPP: The Definitive Guide*
+======================================
+
+Slixmpp was featured in the first edition of the O'Reilly book
+`XMPP: The Definitive Guide <http://oreilly.com/catalog/9780596521271/>`_
+by Peter Saint-Andre, Kevin Smith, and Remko Tronçon. The original source code
+for the book's examples can be found at http://github.com/remko/xmpp-tdg. An
+updated version of the source code, maintained to stay current with the latest
+Slixmpp release, is available at http://github.com/legastero/xmpp-tdg.
+
+However, since publication, Slixmpp has advanced from version 0.2.1 to version
+1.0 and there have been several major API changes. The most notable is the
+introduction of :term:`stanza objects <stanza object>` which have simplified and
+standardized interactions with the XMPP XML stream.
+
+What follows is a walk-through of *The Definitive Guide* highlighting the
+changes needed to make the code examples work with version 1.0 of Slixmpp.
+These changes have been kept to a minimum to preserve the correlation with
+the book's explanations, so be aware that some code may not use current best
+practices.
+
+Example 2-2. (Page 26)
+----------------------
+
+**Implementation of a basic bot that echoes all incoming messages back to its sender.**
+
+The echo bot example requires a change to the ``handleIncomingMessage`` method
+to reflect the use of the ``Message`` :term:`stanza object`. The
+``"jid"`` field of the message object should now be ``"from"`` to match the
+``from`` attribute of the actual XML message stanza. Likewise, ``"message"``
+changes to ``"body"`` to match the ``body`` element of the message stanza.
+
+Updated Code
+~~~~~~~~~~~~
+
+.. code-block:: python
+
+    def handleIncomingMessage(self, message):
+        self.xmpp.send_message(message["from"], message["body"])
+
+`View full source (1) <http://github.com/legastero/xmpp-tdg/blob/master/code/EchoBot/EchoBot.py>`_ |
+`View original code (1) <http://github.com/remko/xmpp-tdg/blob/master/code/EchoBot/EchoBot.py>`_
+
+Example 14-1. (Page 215)
+------------------------
+
+**CheshiR IM bot implementation.**
+
+The main event handling method in the Bot class is meant to process both message
+events and presence update events. With the new changes in Slixmpp 1.0,
+extracting a CheshiR status "message" from both types of stanzas
+requires accessing different attributes. In the case of a message stanza, the
+``"body"`` attribute would contain the CheshiR message. For a presence event,
+the information is stored in the ``"status"`` attribute. To handle both cases,
+we can test the type of the given event object and look up the proper attribute
+based on the type.
+
+Like in the EchoBot example, the expression ``event["jid"]`` needs to change
+to ``event["from"]`` in order to get a JID object for the stanza's sender.
+Because other functions in CheshiR assume that the JID is a string, the ``jid``
+attribute is used to access the string version of the JID. A check is also added
+in case ``user`` is ``None``, but the check could (and probably should) be
+placed in ``addMessageFromUser``.
+
+Another change is needed in ``handleMessageAddedToBackend`` where
+an HTML-IM response is created. The HTML content should be enclosed in a single
+element, such as a ``<p>`` tag.
+
+Updated Code
+~~~~~~~~~~~~
+
+.. code-block:: python
+
+  def handleIncomingXMPPEvent(self, event):
+    msgLocations = {slixmpp.stanza.presence.Presence: "status",
+                    slixmpp.stanza.message.Message: "body"}
+
+    message = event[msgLocations[type(event)]]
+    user = self.backend.getUserFromJID(event["from"].jid)
+    if user is not None:
+      self.backend.addMessageFromUser(message, user)
+
+  def handleMessageAddedToBackend(self, message) :
+    body = message.user + ": " + message.text
+    htmlBody = "<p><a href='%(uri)s'>%(user)s</a>: %(message)s</p>" % {
+      "uri": self.url + "/" + message.user,
+      "user" : message.user, "message" : message.text }
+    for subscriberJID in self.backend.getSubscriberJIDs(message.user) :
+      self.xmpp.send_message(subscriberJID, body, mhtml=htmlBody)
+
+`View full source (2) <http://github.com/legastero/xmpp-tdg/blob/master/code/CheshiR/Bot.py>`_ |
+`View original code (2) <http://github.com/remko/xmpp-tdg/blob/master/code/CheshiR/Bot.py>`_
+
+
+Example 14-3. (Page 217)
+------------------------
+**Configurable CheshiR IM bot implementation.**
+
+.. note::
+    Since the CheshiR examples build on each other, see previous sections for
+    corrections to code that is not marked as new in the book example.
+
+The main difference for the configurable IM bot is the handling for the
+data form in ``handleConfigurationCommand``. The test for equality
+with the string ``"1"`` is no longer required; Slixmpp converts
+boolean data form fields to the values ``True`` and ``False``
+automatically.
+
+For the method ``handleIncomingXMPPPresence``, the attribute
+``"jid"`` is again converted to ``"from"`` to get a JID
+object for the presence stanza's sender, and the ``jid`` attribute is
+used to access the string version of that JID object. A check is also added in
+case ``user`` is ``None``, but the check could (and probably
+should) be placed in ``getShouldMonitorPresenceFromUser``.
+
+Updated Code
+~~~~~~~~~~~~
+
+.. code-block:: python
+
+  def handleConfigurationCommand(self, form, sessionId):
+    values = form.getValues()
+    monitorPresence =values["monitorPresence"]
+    jid = self.xmpp.plugin["xep_0050"].sessions[sessionId]["jid"]
+    user = self.backend.getUserFromJID(jid)
+    self.backend.setShouldMonitorPresenceFromUser(user, monitorPresence)
+
+  def handleIncomingXMPPPresence(self, event):
+    user = self.backend.getUserFromJID(event["from"].jid)
+    if user is not None:
+      if self.backend.getShouldMonitorPresenceFromUser(user):
+        self.handleIncomingXMPPEvent(event)
+
+`View full source (3) <http://github.com/legastero/xmpp-tdg/blob/master/code/CheshiR/ConfigurableBot.py>`_ |
+`View original code (3) <http://github.com/remko/xmpp-tdg/blob/master/code/CheshiR/ConfigurableBot.py>`_
+
+
+Example 14-4. (Page 220)
+------------------------
+**CheshiR IM server component implementation.**
+
+.. note::
+    Since the CheshiR examples build on each other, see previous sections for
+    corrections to code that is not marked as new in the book example.
+
+Like several previous examples, a needed change is to replace
+``subscription["from"]`` with ``subscription["from"].jid`` because the
+``BaseXMPP`` method ``make_presence`` requires the JID to be a string.
+
+A correction needs to be made in ``handleXMPPPresenceProbe`` because a line was
+left out of the original implementation; the variable ``user`` is undefined. The
+JID of the user can be extracted from the presence stanza's ``from`` attribute.
+
+Since this implementation of CheshiR uses an XMPP component, it must
+include a ``from`` attribute in all messages that it sends. Adding the
+``from`` attribute is done by including ``mfrom=self.xmpp.jid`` in calls to
+``self.xmpp.send_message``.
+
+Updated Code
+~~~~~~~~~~~~
+
+.. code-block:: python
+
+  def handleXMPPPresenceProbe(self, event) :
+    self.xmpp.send_presence(pto = event["from"])
+
+  def handleXMPPPresenceSubscription(self, subscription) :
+    if subscription["type"] == "subscribe" :
+      userJID = subscription["from"].jid
+      self.xmpp.send_presence_subscription(pto=userJID, ptype="subscribed")
+      self.xmpp.send_presence(pto = userJID)
+      self.xmpp.send_presence_subscription(pto=userJID, ptype="subscribe")
+
+  def handleMessageAddedToBackend(self, message) :
+    body = message.user + ": " + message.text
+    for subscriberJID in self.backend.getSubscriberJIDs(message.user) :
+      self.xmpp.send_message(subscriberJID, body, mfrom=self.xmpp.jid)
+
+`View full source (4) <http://github.com/legastero/xmpp-tdg/blob/master/code/CheshiR/SimpleComponent.py>`_ |
+`View original code (4) <http://github.com/remko/xmpp-tdg/blob/master/code/CheshiR/SimpleComponent.py>`_
+
+
+Example 14-6. (Page 223)
+------------------------
+**CheshiR IM server component with in-band registration support.**
+
+.. note::
+    Since the CheshiR examples build on each other, see previous sections for
+    corrections to code that is not marked as new in the book example.
+
+After applying the changes from Example 14-4 above, the registrable component
+implementation should work correctly.
+
+.. tip::
+    To see how to implement in-band registration as a Slixmpp plugin,
+    see the tutorial :ref:`tutorial-create-plugin`.
+
+`View full source (5) <http://github.com/legastero/xmpp-tdg/blob/master/code/CheshiR/RegistrableComponent.py>`_ |
+`View original code (5) <http://github.com/remko/xmpp-tdg/blob/master/code/CheshiR/RegistrableComponent.py>`_
+
+Example 14-7. (Page 225)
+------------------------
+**Extended CheshiR IM server component implementation.**
+
+.. note::
+    Since the CheshiR examples build on each other, see previous
+    sections for corrections to code that is not marked as new in the book
+    example.
+
+While the final code example can look daunting with all of the changes
+made, it requires very few modifications to work with the latest version of
+Slixmpp. Most differences are the result of CheshiR's backend functions
+expecting JIDs to be strings so that they can be stripped to bare JIDs. To
+resolve these, use the ``jid`` attribute of the JID objects. Also,
+references to ``"message"`` and ``"jid"`` attributes need to
+be changed to either ``"body"`` or ``"status"``, and either
+``"from"`` or ``"to"`` depending on if the object is a message
+or presence stanza and which of the JIDs from the stanza is needed.
+
+Updated Code
+~~~~~~~~~~~~
+
+.. code-block:: python
+
+  def handleIncomingXMPPMessage(self, event) :
+    message = self.addRecipientToMessage(event["body"], event["to"].jid)
+    user = self.backend.getUserFromJID(event["from"].jid)
+    self.backend.addMessageFromUser(message, user)
+
+  def handleIncomingXMPPPresence(self, event) :
+    if event["to"].jid == self.componentDomain :
+      user = self.backend.getUserFromJID(event["from"].jid)
+      self.backend.addMessageFromUser(event["status"], user)
+
+  ...
+
+  def handleXMPPPresenceSubscription(self, subscription) :
+    if subscription["type"] == "subscribe" :
+      userJID = subscription["from"].jid
+      user = self.backend.getUserFromJID(userJID)
+      contactJID = subscription["to"]
+      self.xmpp.send_presence_subscription(
+          pfrom=contactJID, pto=userJID, ptype="subscribed", pnick=user)
+      self.sendPresenceOfContactToUser(contactJID=contactJID, userJID=userJID)
+      if contactJID == self.componentDomain :
+        self.sendAllContactSubscriptionRequestsToUser(userJID)
+
+`View full source (6) <http://github.com/legastero/xmpp-tdg/blob/master/code/CheshiR/Component.py>`_ |
+`View original code (6) <http://github.com/remko/xmpp-tdg/blob/master/code/CheshiR/Component.py>`_