Merge branch 'async-interal-api-break-everything' into 'master'

Make the internal "api" async

See merge request poezio/slixmpp!128

13 files changed, 545 insertions, 2 deletions

diff --git a/docs/api/api.rst b/docs/api/api.rst
new file mode 100644
index 00000000..55642ef6
--- /dev/null
+++ b/docs/api/api.rst
@@ -0,0 +1,88 @@
+.. _internal-api:
+
+Internal "API"
+==============
+
+Slixmpp has a generic API registry that can be used by its plugins to allow
+access control, redefinition of behaviour, without having to inherit from the
+plugin or do more dark magic.
+
+The idea is that each api call can be replaced, most of them use a form
+of in-memory storage that can be, for example, replaced with database
+or file-based storaged.
+
+
+Each plugin is assigned an API proxy bound to itself, but only a few make use
+of it.
+
+See also :ref:`api-simple-tuto`.
+
+Description of a generic API call
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. code-block:: python
+
+    def get_toto(jid, node, ifrom, args):
+        return 'toto'
+
+    self.xmpp.plugin['xep_XXXX'].api.register(handler, 'get_toto')
+
+Each API call will receive 4 parameters (which can be ``None`` if data
+is not relevant to the operation), which are ``jid`` (``Optional[JID]``),
+``node`` (``Optional[str]``), ``ifrom`` (``Optional[JID]``), and ``args``
+(``Any``).
+
+- ``jid``, if relevant, represents the JID targeted by that operation
+- ``node``, if relevant is an arbitrary string, but was thought for, e.g.,
+  a pubsub or disco node.
+- ``ifrom``, if relevant, is the JID the event is coming from.
+- ``args`` is the event-specific data passed on by the plugin, often a dict
+  of arguments (can be None as well).
+
+.. note::
+    Since 1.8.0, API calls can be coroutines.
+
+
+Handler hierarchy
+~~~~~~~~~~~~~~~~~
+
+The ``self.api.register()`` signature is as follows:
+
+.. code-block:: python
+
+    def register(handler, op, jid=None, node=None, dedfault=False):
+        pass
+
+As you can see, :meth:`~.APIRegistry.register` takes an additional ctype
+parameter, but the :class:`~.APIWrapper` takes care of that for us (in most
+cases, it is the name of the XEP plugin, such as ``'xep_0XXX'``).
+
+When you register a handler, you register it for an ``op``, for **operation**.
+For example, ``get_vcard``.
+
+``handler`` and ``op`` are the only two required parameters (and in many cases,
+all you will ever need). You can, however, go further and register handlers
+for specific values of the ``jid`` and ``node`` parameters of the calls.
+
+The priority of the execution of handlers is as follows:
+
+- Check if a handler for both values of ``node`` and ``jid`` has been defined
+- If not found, check if a handler for this value of ``jid`` has been defined
+- If not found, check if a handler for this value of ``node`` has been defined
+- If still not found, get the global handler (no parameter registered)
+
+
+Raw documentation
+~~~~~~~~~~~~~~~~~
+
+This documentation is provided for reference, but :meth:`~.APIRegistry.register`
+should be all you need.
+
+
+.. module:: slixmpp.api
+
+.. autoclass:: APIRegistry
+    :members:
+
+.. autoclass:: APIWrapper
+
diff --git a/docs/api/index.rst b/docs/api/index.rst
index aefa0f56..e9aa6264 100644
--- a/docs/api/index.rst
+++ b/docs/api/index.rst
@@ -14,3 +14,4 @@ API Reference
     xmlstream/matcher
     xmlstream/xmlstream
     xmlstream/tostring
+    api
diff --git a/docs/api/plugins/xep_0012.rst b/docs/api/plugins/xep_0012.rst
index 9a12eac3..527e28d2 100644
--- a/docs/api/plugins/xep_0012.rst
+++ b/docs/api/plugins/xep_0012.rst
@@ -9,6 +9,44 @@ XEP-0012: Last Activity
     :exclude-members: session_bind, plugin_init, plugin_end
 
 
+.. _api-0012:
+
+Internal API methods
+--------------------
+
+This plugin uses an in-memory storage by default to keep track of the
+received and sent last activities.
+
+.. glossary::
+
+    get_last_activity
+        - **jid**: :class:`~.JID` of whom to retrieve the last activity
+        - **node**: unused
+        - **ifrom**: who the request is from (None = local)
+        - **args**: ``None`` or an :class:`~.Iq` that is requesting the
+        - **returns**
+          information.
+
+        Get the last activity of a JID from the storage.
+
+    set_last_activity
+        - **jid**: :class:`~.JID` of whom to set the last activity
+        - **node**: unused
+        - **ifrom**: unused
+        - **args**: A dict containing ``'seconds'`` and ``'status'``
+          ``{'seconds': Optional[int], 'status': Optional[str]}``
+
+        Set the last activity of a JID in the storage.
+
+    del_last_activity
+        - **jid**: :class:`~.JID` to delete from the storage
+        - **node**: unused
+        - **ifrom**: unused
+        - **args**: unused
+
+        Remove the last activity of a JID from the storage.
+
+
 Stanza elements
 ---------------
 
diff --git a/docs/api/plugins/xep_0027.rst b/docs/api/plugins/xep_0027.rst
index 418baada..3a7fb561 100644
--- a/docs/api/plugins/xep_0027.rst
+++ b/docs/api/plugins/xep_0027.rst
@@ -9,6 +9,50 @@ XEP-0027: Current Jabber OpenPGP Usage
     :exclude-members: session_bind, plugin_init, plugin_end
 
 
+.. _api-0027:
+
+Internal API methods
+--------------------
+
+The default API here is managing a JID→Keyid dict in-memory.
+
+.. glossary::
+
+    get_keyid
+        - **jid**: :class:`~.JID` to get.
+        - **node**: unused
+        - **ifrom**: unused
+        - **args**: unused
+        - **returns**: ``Optional[str]``, the keyid or None
+
+        Get the KeyiD for a JID, None if it is not found.
+
+    set_keyid
+        - **jid**: :class:`~.JID` to set the id for.
+        - **node**: unused
+        - **ifrom**: unused
+        - **args**: ``str``, keyid to set
+
+        Set the KeyiD for a JID.
+
+    del_keyid
+        - **jid**: :class:`~.JID` to delete from the mapping.
+        - **node**: unused
+        - **ifrom**: unused
+        - **args**: unused
+
+        Delete the KeyiD for a JID.
+
+    get_keyids
+        - **jid**: unused
+        - **node**: unused
+        - **ifrom**: unused
+        - **args**: unused
+        - **returns**: ``Dict[JID, str]`` the full internal mapping
+
+        Get all currently stored KeyIDs.
+
+
 Stanza elements
 ---------------
 
diff --git a/docs/api/plugins/xep_0047.rst b/docs/api/plugins/xep_0047.rst
index c8aea741..0e81ae10 100644
--- a/docs/api/plugins/xep_0047.rst
+++ b/docs/api/plugins/xep_0047.rst
@@ -8,11 +8,78 @@ XEP-0047: In-band Bytestreams
     :members:
     :exclude-members: session_bind, plugin_init, plugin_end
 
-.. module:: slixmpp.plugins.xep_0047
-
 .. autoclass:: IBBytestream
     :members:
 
+.. _api-0047:
+
+Internal API methods
+--------------------
+
+The API here is used to manage streams and authorize. The default handlers
+work with the config parameters.
+
+.. glossary::
+
+    authorized_sid (0047 version)
+        - **jid**: :class:`~.JID` receiving the stream initiation.
+        - **node**:  stream id
+        - **ifrom**: who the stream is from.
+        - **args**: :class:`~.Iq` of the stream request.
+        - **returns**: ``True`` if the stream should be accepted,
+          ``False`` otherwise.
+
+        Check if the stream should be accepted. Uses
+        the information setup by :term:`preauthorize_sid (0047 version)`
+        by default.
+
+    authorized (0047 version)
+        - **jid**: :class:`~.JID` receiving the stream initiation.
+        - **node**:  stream id
+        - **ifrom**: who the stream is from.
+        - **args**: :class:`~.Iq` of the stream request.
+        - **returns**: ``True`` if the stream should be accepted,
+          ``False`` otherwise.
+
+        A fallback handler (run after :term:`authorized_sid (0047 version)`)
+        to check if a stream should be accepted. Uses the ``auto_accept``
+        parameter by default.
+
+    preauthorize_sid (0047 version)
+        - **jid**: :class:`~.JID` receiving the stream initiation.
+        - **node**:  stream id
+        - **ifrom**: who the stream will be from.
+        - **args**: Unused.
+
+        Register a stream id to be accepted automatically (called from
+        other plugins such as XEP-0095).
+
+    get_stream
+        - **jid**: :class:`~.JID` of local receiver.
+        - **node**:  stream id
+        - **ifrom**: who the stream is from.
+        - **args**: unused
+        - **returns**: :class:`~.IBBytestream`
+
+        Return a currently opened stream between two JIDs.
+
+    set_stream
+        - **jid**: :class:`~.JID` of local receiver.
+        - **node**:  stream id
+        - **ifrom**: who the stream is from.
+        - **args**: unused
+
+        Register an opened stream between two JIDs.
+
+    del_stream
+        - **jid**: :class:`~.JID` of local receiver.
+        - **node**:  stream id
+        - **ifrom**: who the stream is from.
+        - **args**: unused
+
+        Delete a stream between two JIDs.
+
+
 Stanza elements
 ---------------
 
diff --git a/docs/api/plugins/xep_0054.rst b/docs/api/plugins/xep_0054.rst
index db8e29ef..ace5f10a 100644
--- a/docs/api/plugins/xep_0054.rst
+++ b/docs/api/plugins/xep_0054.rst
@@ -8,6 +8,40 @@ XEP-0054: vcard-temp
     :members:
     :exclude-members: session_bind, plugin_init, plugin_end
 
+.. _api-0054:
+
+Internal API methods
+--------------------
+
+This plugin maintains by default an in-memory cache of the received
+VCards.
+
+.. glossary::
+
+    set_vcard
+        - **jid**: :class:`~.JID` of whom to set the vcard
+        - **node**: unused
+        - **ifrom**: unused
+        - **args**: :class:`~.VCardTemp` object to store for this JID.
+
+        Set a VCard for a JID.
+
+    get_vcard
+        - **jid**: :class:`~.JID` of whom to set the vcard
+        - **node**: unused
+        - **ifrom**: :class:`~.JID` the request is coming from
+        - **args**: unused
+        - **returns**: :class:`~.VCardTemp` object for this JID or None.
+
+        Get a stored VCard for a JID.
+
+    del_vcard
+        - **jid**: :class:`~.JID` of whom to set the vcard
+        - **node**: unused
+        - **ifrom**: unused
+        - **args**: unused
+
+        Delete a stored VCard for a JID.
 
 Stanza elements
 ---------------
diff --git a/docs/api/plugins/xep_0065.rst b/docs/api/plugins/xep_0065.rst
index d6aec058..954af2b6 100644
--- a/docs/api/plugins/xep_0065.rst
+++ b/docs/api/plugins/xep_0065.rst
@@ -8,6 +8,48 @@ XEP-0065: SOCKS5 Bytestreams
     :members:
     :exclude-members: session_bind, plugin_init, plugin_end
 
+.. _api-0065:
+
+Internal API methods
+--------------------
+
+The internal API is used here to authorize or pre-authorize streams.
+
+.. glossary::
+
+    authorized_sid (0065 version)
+        - **jid**: :class:`~.JID` receiving the stream initiation.
+        - **node**:  stream id
+        - **ifrom**: who the stream is from.
+        - **args**: :class:`~.Iq` of the stream request.
+        - **returns**: ``True`` if the stream should be accepted,
+          ``False`` otherwise.
+
+        Check if the stream should be accepted. Uses
+        the information setup by :term:`preauthorize_sid (0065 version)`
+        by default.
+
+    authorized (0065 version)
+        - **jid**: :class:`~.JID` receiving the stream initiation.
+        - **node**:  stream id
+        - **ifrom**: who the stream is from.
+        - **args**: :class:`~.Iq` of the stream request.
+        - **returns**: ``True`` if the stream should be accepted,
+          ``False`` otherwise.
+
+        A fallback handler (run after :term:`authorized_sid (0065 version)`)
+        to check if a stream should be accepted. Uses the ``auto_accept``
+        parameter by default.
+
+    preauthorize_sid (0065 version)
+        - **jid**: :class:`~.JID` receiving the stream initiation.
+        - **node**:  stream id
+        - **ifrom**: who the stream will be from.
+        - **args**: Unused.
+
+        Register a stream id to be accepted automatically (called from
+        other plugins such as XEP-0095).
+
 
 Stanza elements
 ---------------
diff --git a/docs/api/plugins/xep_0077.rst b/docs/api/plugins/xep_0077.rst
index 725c16b0..33c433e0 100644
--- a/docs/api/plugins/xep_0077.rst
+++ b/docs/api/plugins/xep_0077.rst
@@ -8,6 +8,53 @@ XEP-0077: In-Band Registration
     :members:
     :exclude-members: session_bind, plugin_init, plugin_end
 
+Internal APi methods
+--------------------
+
+The API here is made to allow components to manage registered users.
+The default handlers make use of the plugin options and store users
+in memory.
+
+.. glossary::
+
+    user_get
+        - **jid**: unused
+        - **node**: unused
+        - **ifrom**: who the request is coming from
+        - **args**: :class:`~.Iq` registration request.
+        - **returns**: ``dict`` containing user data or None.
+
+        Get user data for a user.
+
+    user_validate
+        - **jid**: unused
+        - **node**: unused
+        - **ifrom**: who the request is coming from
+        - **args**: :class:`~.Iq` registration request, 'register' payload.
+        - **raises**: ValueError if some fields are invalid
+
+        Validate form fields and save user data.
+
+    user_remove
+        - **jid**: unused
+        - **node**: unused
+        - **ifrom**: who the request is coming from
+        - **args**: :class:`~.Iq` registration removal request.
+        - **raises**: KeyError if the user is not found.
+
+        Remove a user from the store.
+
+    make_registration_form
+        - **jid**: unused
+        - **node**: unused
+        - **ifrom**: who the request is coming from
+        - **args**: :class:`~.Iq` registration request.
+        - **raises**: KeyError if the user is not found.
+
+        Return an :class:`~.Iq` reply for the request, with a form and
+        options set. By default, use ``form_fields`` and ``form_instructions``
+        plugin config options.
+
 
 Stanza elements
 ---------------
diff --git a/docs/api/plugins/xep_0115.rst b/docs/api/plugins/xep_0115.rst
index 73d0e77b..e3507fb8 100644
--- a/docs/api/plugins/xep_0115.rst
+++ b/docs/api/plugins/xep_0115.rst
@@ -8,6 +8,54 @@ XEP-0115: Entity Capabilities
     :members:
     :exclude-members: session_bind, plugin_init, plugin_end
 
+.. _api-0115:
+
+Internal API methods
+--------------------
+
+This internal API extends the Disco internal API, and also manages an
+in-memory cache of verstring→disco info, and fulljid→verstring.
+
+.. glossary::
+
+    cache_caps
+        - **jid**: unused
+        - **node**:  unused
+        - **ifrom**: unused
+        - **args**: a ``dict`` containing the verstring and
+          :class:`~.DiscoInfo` payload (
+          ``{'verstring': Optional[str], 'info': Optional[DiscoInfo]}``)
+
+        Cache a verification string with its payload.
+
+    get_caps
+        - **jid**: JID to retrieve the verstring for (unused with the default
+          handler)
+        - **node**:  unused
+        - **ifrom**: unused
+        - **args**: a ``dict`` containing the verstring
+          ``{'verstring': str}``
+        - **returns**: The :class:`~.DiscoInfo` payload for that verstring.
+
+        Get a disco payload from a verstring.
+
+    assign_verstring
+        - **jid**: :class:`~.JID` (full) to assign the verstring to
+        - **node**:  unused
+        - **ifrom**: unused
+        - **args**: a ``dict`` containing the verstring
+          ``{'verstring': str}``
+
+        Cache JID→verstring information.
+
+    get_verstring
+        - **jid**: :class:`~.JID` to use for fetching the verstring
+        - **node**:  unused
+        - **ifrom**: unused
+        - **args**: unused
+        - **returns**: ``str``, the verstring
+
+        Retrieve a verstring for a JID.
 
 Stanza elements
 ---------------
diff --git a/docs/api/plugins/xep_0128.rst b/docs/api/plugins/xep_0128.rst
index 1e080928..7ecb436c 100644
--- a/docs/api/plugins/xep_0128.rst
+++ b/docs/api/plugins/xep_0128.rst
@@ -7,3 +7,38 @@ XEP-0128: Service Discovery Extensions
 .. autoclass:: XEP_0128
     :members:
     :exclude-members: session_bind, plugin_init, plugin_end
+
+.. _api-0128:
+
+Internal API methods
+--------------------
+
+
+
+.. glossary::
+
+    add_extended_info
+        - **jid**: JID to set the extended info for
+        - **node**:  note to set the info at
+        - **ifrom**: unused
+        - **args**: A :class:`~.Form` or list of forms to add to the disco
+          extended info for this JID/node.
+
+        Add extended info for a JID/node.
+
+    set_extended_info
+        - **jid**: JID to set the extended info for
+        - **node**:  note to set the info at
+        - **ifrom**: unused
+        - **args**: A :class:`~.Form` or list of forms to set as the disco
+          extended info for this JID/node.
+
+        Set extended info for a JID/node.
+
+    del_extended_info
+        - **jid**: JID to delete the extended info from
+        - **node**:  note to delete the info from
+        - **ifrom**: unused
+        - **args**: unused
+
+        Delete extended info for a JID/node.
diff --git a/docs/api/plugins/xep_0153.rst b/docs/api/plugins/xep_0153.rst
index 00e22098..d4ce342f 100644
--- a/docs/api/plugins/xep_0153.rst
+++ b/docs/api/plugins/xep_0153.rst
@@ -8,6 +8,43 @@ XEP-0153: vCard-Based Avatars
     :members:
     :exclude-members: session_bind, plugin_init, plugin_end
 
+.. _api-0153:
+
+Internal API methods
+--------------------
+
+The internal API is used here to maintain an in-memory JID→avatar hash
+cache.
+
+.. glossary::
+
+    set_hash
+        - **jid**: :class:`~.JID` of whom to retrieve the last activity
+        - **node**: unused
+        - **ifrom**: unused
+        - **args**: ``str``, avatar hash
+
+        Set the avatar hash for a JID.
+
+    reset_hash
+        - **jid**: :class:`~.JID` of whom to retrieve the last activity
+        - **node**: unused
+        - **ifrom**: :class:`~.JID` of the entity requesting the reset.
+        - **args**: unused
+        - **returns**
+          information.
+
+        Reset the avatar hash for a JID. This downloads the vcard and computes
+        the hash.
+
+    get_hash
+        - **jid**: :class:`~.JID` of whom to retrieve the last activity
+        - **node**: unused
+        - **ifrom**: unused
+        - **args**: unused
+        - **returns**: ``Optional[str]``, the avatar hash
+
+        Get the avatar hash for a JID.
 
 Stanza elements
 ---------------
diff --git a/docs/api/plugins/xep_0231.rst b/docs/api/plugins/xep_0231.rst
index 29f403ca..c8a863cb 100644
--- a/docs/api/plugins/xep_0231.rst
+++ b/docs/api/plugins/xep_0231.rst
@@ -8,6 +8,41 @@ XEP-0231: Bits of Binary
     :members:
     :exclude-members: session_bind, plugin_init, plugin_end
 
+.. _api-0231:
+
+Internal API methods
+--------------------
+
+The default API handlers for this plugin manage an in-memory cache of
+bits of binary by content-id.
+
+.. glossary::
+
+    set_bob
+        - **jid**: :class:`~.JID` sending the bob
+        - **node**: unused
+        - **ifrom**: :class:`~JID` receiving the bob
+        - **args**: :class:`~.BitsOfBinary` element.
+
+        Set a BoB in the cache.
+
+    get_bob
+        - **jid**: :class:`~.JID` receiving the bob
+        - **node**: unused
+        - **ifrom**: :class:`~JID` sending the bob
+        - **args**: ``str`` content-id of the bob
+        - **returns**: :class:`~.BitsOfBinary` element.
+
+        Get a BoB from the cache.
+
+    del_bob
+        - **jid**: unused
+        - **node**: unused
+        - **ifrom**: :class:`~JID` sending the bob
+        - **args**: ``str`` content-id of the bob
+
+        Delete a BoB from the cache.
+
 
 Stanza elements
 ---------------
diff --git a/docs/api/plugins/xep_0319.rst b/docs/api/plugins/xep_0319.rst
index a3ab9d28..7be01cb2 100644
--- a/docs/api/plugins/xep_0319.rst
+++ b/docs/api/plugins/xep_0319.rst
@@ -9,6 +9,33 @@ XEP-0319: Last User Interaction in Presence
     :exclude-members: session_bind, plugin_init, plugin_end
 
 
+.. _api-0319:
+
+Internal API methods
+--------------------
+
+The default API manages an in-memory cache of idle periods.
+
+.. glossary::
+
+    set_idle
+        - **jid**: :class:`~.JID` who has been idling
+        - **node**: unused
+        - **ifrom**: unused
+        - **args**: :class:`datetime`, timestamp of the idle start
+
+        Set the idle start for a JID.
+
+    get_idle
+        - **jid**: :class:`~.JID` to get the idle time of
+        - **node**: unused
+        - **ifrom**: unused
+        - **args**: : unused
+        - **returns**: :class:`datetime`
+
+        Get the idle start timestamp for a JID.
+
+
 Stanza elements
 ---------------