From b4368aaa0de46f71fe3a13b87f52d4a576ce0782 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?louiz=E2=80=99?= Date: Mon, 11 Nov 2019 01:17:06 +0100 Subject: Rename the doc from developper to developer Shame. --- doc/developer.rst | 240 +++++++++++++++++++++++++++++++++++++++++++++++++++++ doc/developper.rst | 240 ----------------------------------------------------- doc/index.rst | 2 +- 3 files changed, 241 insertions(+), 241 deletions(-) create mode 100644 doc/developer.rst delete mode 100644 doc/developper.rst (limited to 'doc') diff --git a/doc/developer.rst b/doc/developer.rst new file mode 100644 index 0000000..49daa5c --- /dev/null +++ b/doc/developer.rst @@ -0,0 +1,240 @@ +######################## +Developper documentation +######################## + +End-to-end test suite +--------------------- + +A powerful test suite has been developped to test biboumi’s behaviour in +many scenarios. Its goal is to simulate a real-world usage of biboumi, +including its interactions with a real IRC server an a real XMPP client. + +An IRC server is started, with a specific version and configuration, then, +for every scenario that we want to test: + +- Biboumi is started, with a specific configuration +- An XMPP “client” starts, communicates with biboumi and checks that + biboumi responds in the expected way. + +The XMPP client is actually not a real client, it’s a python script that +uses the slixmpp library to imitate an XMPP server that would transmit the +stanzas of one client to its component (biboumi). In real life, the +communication to biboumi is done between the XMPP server and biboumi, but +since the server just forwards the messages that clients send unmodified, +we’ll call that “the clients”. + +A scenario is a list of functions that will be executed one by one, to +verify the behaviour of one specific feature. Ideally, they should be +short and test one specific aspect. + +Available functions +~~~~~~~~~~~~~~~~~~~ + +.. py:function:: send_stanza(str) + + sends one stanza to biboumi. The stanza is written entirely + as a string (with a few automatic replacements). The “from” and “to” + values have to be specified everytime, because each stanza can come from + different clients and be directed to any IRC server/channel + + .. code-block:: python + + send_stanza("coucou"), + +.. py:function:: expect_stanza(xpath[, …]) + + Waits for a stanza to be received by biboumi, and checks that this + stanza matches one or more xpath. If the stanza doesn’t match all the + given xpaths, then the scenario ends and we report that as an error. + + .. code-block:: python + + expect_stanza("/message[@from='#foo@{biboumi_host}/{nick_one}']/body[text()='coucou']", + "/message/delay:delay[@from='#foo@{biboumi_host}']"), + + This waits for exactly 1 stanza, that is compared against 2 xpaths. Here + we check that it is a message, that it has the proper `from` value, the + correct body, and a . + +.. py:function:: expect_unordered(list_of_xpaths[, list_of_xpaths, …]) + + we wait for more than one stanzas, that could be received in any order. + For example, in certain scenario, we wait for two presence stanzas, but + it’s perfectly valid to receive them in any order (one is for one + client, the other one for an other client). To do that, we pass multiple + lists of xpath. Each list can contain one or more xpath (just like + `expect_stanza`). When a stanza is received, it is compared with all the + xpaths of the first list. If it doesn’t match, it is compared with the + xpaths of the second list, and so on. If nothing matchs, it’s an error + and we stop this scenario. If the stanza matches with one of the xpath + lists, we remove that list, and we wait for the next stanza, until there + are no more xpaths. + + .. code-block:: python + + expect_unordered( + [ + "/presence[@from='#foo%{irc_server_one}/{nick_one}'][@to='{jid_two}/{resource_one}'][@type='unavailable']/muc_user:x/muc_user:item[@nick='Bernard']", + "/presence/muc_user:x/muc_user:status[@code='303']", + ], + [ + "/presence[@from='#foo%{irc_server_one}/{nick_three}'][@to='{jid_two}/{resource_one}']", + ], + [ + "/presence[@from='#foo%{irc_server_one}/{nick_one}'][@to='{jid_one}/{resource_one}'][@type='unavailable']/muc_user:x/muc_user:item[@nick='Bernard']", + "/presence/muc_user:x/muc_user:status[@code='303']", + "/presence/muc_user:x/muc_user:status[@code='110']", + ], + [ + "/presence[@from='#foo%{irc_server_one}/{nick_three}'][@to='{jid_one}/{resource_one}']", + "/presence/muc_user:x/muc_user:status[@code='110']", + ], + ), + + This will wait for 4 stanzas that could be received in any order. + +To avoid many repetitions between each tests, some helpful sequences are +available, `sequences.connection(…)` and `sequences.connection_tls(…)`. +They do all the steps that are needed (send and receive stanzas) to +connect to the component, or an IRC server. + +It’s also possible to reuse one simple scenario into an other scenario. +The most notable example is to start your own scenario with +`scenarios.simple_channel_join.scenario`, if you need your client to be in +a channel before you can start your actual scenario. For example if you +want to test the behaviour of a topic change, you need to first join a +channel. Since this is a very common patern, it’s simpler to just included +this very basic scenario at the start of your own scenarios, instead of +copy pasting the same thing over and over. + +Examples of a scenario +~~~~~~~~~~~~~~~~~~~~~~ + +First example +^^^^^^^^^^^^^ + +Here we’ll describe how to write your own scenario, from scratch. For this, we will take an existing scenario and explain how it was written, line by line. + +See for example the scenario tests/end_to_end/scenarios/self_ping_on_real_channel.py + +.. code-block:: python + + from scenarios import * + +All the tests should start with this import. It imports the file +tests/end_to_end/scenarios/__init__.py This make all the functions +available (send_stanza, expect_stanza…) available, as well as some very +common scenarios that you often need to re-use. + +.. code-block:: python + + scenario = ( + # … + ) + +This is the only required element of your scenario. This object is a tuple of function calls OR other scenarios. + +.. code-block:: python + + scenarios.simple_channel_join.scenario, + +The first line of our scenario is actually including an other existing +scenario. You can find it at +tests/end_to_end/scenarios/simple_channel_join.py As its name shows, it’s +very basic: one client {jid_one}/{resource_one} just joins one room +#foo%{irc_server_one} with the nick {nick_one}. + +Since we want to test the behaviour of a ping to ourself when we are in a +room, we just join this room without repeating everything. + +It is possible to directly insert a scenario inside our scenario without +having to extract all the steps: the test suite is smart enough to detect +that and extract the inner steps automatically. + +.. code-block:: python + + # Send a ping to ourself + send_stanza(""), + expect_stanza("/iq[@from='#foo%{irc_server_one}/{nick_one}'][@type='result'][@to='{jid_one}/{resource_one}'][@id='first_ping']"), + +Here we simple send an iq stanza, properly formatted, using the same JIDs +{jid_one}/{resource_one} and #foo%{irc_server_one}/{nick_one} to ping +ourself in the room. We them immediately expect one stanza to be received, +that is the response to our ping. It only contains one single xpath +because everything we need to check can be expressed in one line. + +Note that it is recommended to explain all the steps of your scenario with +comments. This helps understand what is being tested, and why, without +having to analyze all the stanza individually. + +.. code-block:: python + + # Now join the same room, from the same bare JID, behind the same nick + send_stanza(""), + expect_stanza("/presence[@to='{jid_one}/{resource_two}'][@from='#foo%{irc_server_one}/{nick_one}']/muc_user:x/muc_user:item[@affiliation='admin'][@role='moderator']", + "/presence/muc_user:x/muc_user:status[@code='110']"), + + expect_stanza("/message[@from='#foo%{irc_server_one}'][@type='groupchat'][@to='{jid_one}/{resource_two}']/subject[not(text())]"), + +Here we send a presence stanza to join the same channel with an other +resource (note the {resource_two}). As a result, we expect two stanzas: +The first stanza (our self-presence) is checked against two xpaths, and +the second stanza (the empty subject of the room) against only one. + +.. code-block:: python + + # And re-send a self ping + send_stanza(""), + expect_stanza("/iq[@from='#foo%{irc_server_one}/{nick_one}'][@type='result'][@to='{jid_one}/{resource_one}'][@id='second_ping']"), + ## And re-do exactly the same thing, just change the resource initiating the self ping + send_stanza(""), + expect_stanza("/iq[@from='#foo%{irc_server_one}/{nick_one}'][@type='result'][@to='{jid_one}/{resource_two}'][@id='third_ping']"), + +And finally, we test a second ping, and check that the behaviour is correct that we now have two resources in that channel. + +Second example +^^^^^^^^^^^^^^ + +Sometimes we want to do more with the received stanzas. For example we +need to extract some values from the received stanzas, to reuse them in +future stanzas we send. The most obvious example is iq IDs, that we need +to extract, to reuse them in our response. + +Let’s use for example the tests/end_to_end/scenarios/execute_incomplete_hello_adhoc_command.py scenario: + +.. code-block:: python + + from scenarios import * + + scenario = ( + send_stanza(""), + expect_stanza("/iq[@type='result']/commands:command[@node='hello'][@sessionid][@status='executing']", + "/iq/commands:command/commands:actions/commands:complete", + after = save_value("sessionid", extract_attribute("/iq[@type='result']/commands:command[@node='hello']", "sessionid")) + ), + +Here is where the magic happens: as an additional argument to the +expect_stanza function, we pass an other function (callback) with the +“after=” keyword argument. This “after” callback gets called once the +expected stanza has been received and validated. Here we use +`save_value(key, value)`. This function just saves a value in our global +values that can be used with “send_stanza”, associated with the given +“key”. For example if you do `save_value("something_important", "blah")` +then you can use `{something_important}` in any future stanza that you +send and it will be replaced with “blah”. + +But this is only useful if we can save some value that we extract from the +stanza. That’s where `extract_attribute(xpath, attribute_name)` comes into +play. As the first argument, you pass an xpath corresponding to one +specific node of the XML that is received, and the second argument is just +the name of the attribute whose value you want. + +Here, we extract the value of the “sessionid=” in the node ``, and +we save that value, globally, with the name “sessionid”. + +.. code-block:: python + + send_stanza(""), + +Here we send a second iq, to continue our ad-hoc command, and we use {sessionid} to indicate that we are continuing the session we started before. diff --git a/doc/developper.rst b/doc/developper.rst deleted file mode 100644 index 1df9826..0000000 --- a/doc/developper.rst +++ /dev/null @@ -1,240 +0,0 @@ -######################## -Developper documentation -######################## - -End-to-end test suite ---------------------- - -A powerful test suite has been developped to test biboumi’s behaviour in -many scenarios. Its goal is to simulate a real-world usage of biboumi, -including its interactions with a real IRC server an a real XMPP client. - -An IRC server is started, with a specific version and configuration, then, -for every scenario that we want to test: - -- Biboumi is started, with a specific configuration -- An XMPP “client” starts, communicates with biboumi and checks that - biboumi responds in the expected way. - -The XMPP client is actually not a real client, it’s a python script that -uses the slixmpp library to imitate an XMPP server that would transmit the -stanzas of one client to its component (biboumi). In real life, the -communication to biboumi is done between the XMPP server and biboumi, but -since the server just forwards the messages that clients send unmodified, -we’ll call that “the clients”. - -A scenario is a list of functions that will be executed one by one, to -verify the behaviour of one specific feature. Ideally, they should be -short and test one specific aspect. - -Available functions -~~~~~~~~~~~~~~~~~~~ - -.. py:function:: send_stanza(str) - - sends one stanza to biboumi. The stanza is written entirely - as a string (with a few automatic replacements). The “from” and “to” - values have to be specified everytime, because each stanza can come from - different clients and be directed to any IRC server/channel - - .. code-block:: python - - send_stanza("coucou"), - -.. py:function:: expect_stanza(xpath[, …]) - - Waits for a stanza to be received by biboumi, and checks that this - stanza matches one or more xpath. If the stanza doesn’t match all the - given xpaths, then the scenario ends and we report that as an error. - - .. code-block:: python - - expect_stanza("/message[@from='#foo@{biboumi_host}/{nick_one}']/body[text()='coucou']", - "/message/delay:delay[@from='#foo@{biboumi_host}']"), - - This waits for exactly 1 stanza, that is compared against 2 xpaths. Here - we check that it is a message, that it has the proper `from` value, the - correct body, and a . - -.. py:function:: expect_unordered(list_of_xpaths[, list_of_xpaths, …]) - - we wait for more than one stanzas, that could be received in any order. - For example, in certain scenario, we wait for two presence stanzas, but - it’s perfectly valid to receive them in any order (one is for one - client, the other one for an other client). To do that, we pass multiple - lists of xpath. Each list can contain one or more xpath (just like - `expect_stanza`). When a stanza is received, it is compared with all the - xpaths of the first list. If it doesn’t match, it is compared with the - xpaths of the second list, and so on. If nothing matchs, it’s an error - and we stop this scenario. If the stanza matches with one of the xpath - lists, we remove that list, and we wait for the next stanza, until there - are no more xpaths. - - .. code-block:: python - - expect_unordered( - [ - "/presence[@from='#foo%{irc_server_one}/{nick_one}'][@to='{jid_two}/{resource_one}'][@type='unavailable']/muc_user:x/muc_user:item[@nick='Bernard']", - "/presence/muc_user:x/muc_user:status[@code='303']", - ], - [ - "/presence[@from='#foo%{irc_server_one}/{nick_three}'][@to='{jid_two}/{resource_one}']", - ], - [ - "/presence[@from='#foo%{irc_server_one}/{nick_one}'][@to='{jid_one}/{resource_one}'][@type='unavailable']/muc_user:x/muc_user:item[@nick='Bernard']", - "/presence/muc_user:x/muc_user:status[@code='303']", - "/presence/muc_user:x/muc_user:status[@code='110']", - ], - [ - "/presence[@from='#foo%{irc_server_one}/{nick_three}'][@to='{jid_one}/{resource_one}']", - "/presence/muc_user:x/muc_user:status[@code='110']", - ], - ), - - This will wait for 4 stanzas that could be received in any order. - -To avoid many repetitions between each tests, some helpful sequences are -available, `sequences.connection(…)` and `sequences.connection_tls(…)`. -They do all the steps that are needed (send and receive stanzas) to -connect to the component, or an IRC server. - -It’s also possible to reuse one simple scenario into an other scenario. -The most notable example is to start your own scenario with -`scenarios.simple_channel_join.scenario`, if you need your client to be in -a channel before you can start your actual scenario. For example if you -want to test the behaviour of a topic change, you need to first join a -channel. Since this is a very common patern, it’s simpler to just included -this very basic scenario at the start of your own scenarios, instead of -copy pasting the same thing over and over. - -Examples of a scenario -~~~~~~~~~~~~~~~~~~~~~~ - -First example -^^^^^^^^^^^^^ - -Here we’ll describe how to write your own scenario, from scratch. For this, we will take an existing scenario and explain how it was written, line by line. - -See for example the scenario tests/end_to_end/scenarios/self_ping_on_real_channel.py - -.. code-block:: python - - from scenarios import * - -All the tests should start with this import. It imports the file -tests/end_to_end/scenarios/__init__.py This make all the functions -available (send_stanza, expect_stanza…) available, as well as some very -common scenarios that you often need to re-use. - -.. code-block:: python - - scenario = ( - # … - ) - -This is the only required element of your scenario. This object is a tuple of funcion calls OR other scenarios. - -.. code-block:: python - - scenarios.simple_channel_join.scenario, - -The first line of our scenario is actually including an other existing -scenario. You can find it at -tests/end_to_end/scenarios/simple_channel_join.py As its name shows, it’s -very basic: one client {jid_one}/{resource_one} just joins one room -#foo%{irc_server_one} with the nick {nick_one}. - -Since we want to test the behaviour of a ping to ourself when we are in a -room, we just join this room without repeating everything. - -It is possible to directly insert a scenario inside our scenario without -having to extract all the steps: the test suite is smart enough to detect -that and extract the inner steps automatically. - -.. code-block:: python - - # Send a ping to ourself - send_stanza(""), - expect_stanza("/iq[@from='#foo%{irc_server_one}/{nick_one}'][@type='result'][@to='{jid_one}/{resource_one}'][@id='first_ping']"), - -Here we simple send an iq stanza, properly formatted, using the same JIDs -{jid_one}/{resource_one} and #foo%{irc_server_one}/{nick_one} to ping -ourself in the room. We them immediately expect one stanza to be received, -that is the response to our ping. It only contains one single xpath -because everything we need to check can be expressed in one line. - -Note that it is recommended to explain all the steps of your scenario with -comments. This helps understand what is being tested, and why, without -having to analyze all the stanza individually. - -.. code-block:: python - - # Now join the same room, from the same bare JID, behind the same nick - send_stanza(""), - expect_stanza("/presence[@to='{jid_one}/{resource_two}'][@from='#foo%{irc_server_one}/{nick_one}']/muc_user:x/muc_user:item[@affiliation='admin'][@role='moderator']", - "/presence/muc_user:x/muc_user:status[@code='110']"), - - expect_stanza("/message[@from='#foo%{irc_server_one}'][@type='groupchat'][@to='{jid_one}/{resource_two}']/subject[not(text())]"), - -Here we send a presence stanza to join the same channel with an other -resource (note the {resource_two}). As a result, we expect two stanzas: -The first stanza (our self-presence) is checked against two xpaths, and -the second stanza (the empty subject of the room) against only one. - -.. code-block:: python - - # And re-send a self ping - send_stanza(""), - expect_stanza("/iq[@from='#foo%{irc_server_one}/{nick_one}'][@type='result'][@to='{jid_one}/{resource_one}'][@id='second_ping']"), - ## And re-do exactly the same thing, just change the resource initiating the self ping - send_stanza(""), - expect_stanza("/iq[@from='#foo%{irc_server_one}/{nick_one}'][@type='result'][@to='{jid_one}/{resource_two}'][@id='third_ping']"), - -And finally, we test a second ping, and check that the behaviour is correct that we now have two resources in that channel. - -Second example -^^^^^^^^^^^^^^ - -Sometimes we want to do more with the received stanzas. For example we -need to extract some values from the received stanzas, to reuse them in -future stanzas we send. The most obvious example is iq IDs, that we need -to extract, to reuse them in our response. - -Let’s use for example the tests/end_to_end/scenarios/execute_incomplete_hello_adhoc_command.py scenario: - -.. code-block:: python - - from scenarios import * - - scenario = ( - send_stanza(""), - expect_stanza("/iq[@type='result']/commands:command[@node='hello'][@sessionid][@status='executing']", - "/iq/commands:command/commands:actions/commands:complete", - after = save_value("sessionid", extract_attribute("/iq[@type='result']/commands:command[@node='hello']", "sessionid")) - ), - -Here is where the magic happens: as an additional argument to the -expect_stanza function, we pass an other function (callback) with the -“after=” keyword argument. This “after” callback gets called once the -expected stanza has been received and validated. Here we use -`save_value(key, value)`. This function just saves a value in our global -values that can be used with “send_stanza”, associated with the given -“key”. For example if you do `save_value("something_important", "blah")` -then you can use `{something_important}` in any future stanza that you -send and it will be replaced with “blah”. - -But this is only useful if we can save some value that we extract from the -stanza. That’s where `extract_attribute(xpath, attribute_name)` comes into -play. As the first argument, you pass an xpath corresponding to one -specific node of the XML that is received, and the second argument is just -the name of the attribute whose value you want. - -Here, we extract the value of the “sessionid=” in the node ``, and -we save that value, globally, with the name “sessionid”. - -.. code-block:: python - - send_stanza(""), - -Here we send a second iq, to continue our ad-hoc command, and we use {sessionid} to indicate that we are continuing the session we started before. diff --git a/doc/index.rst b/doc/index.rst index e77d898..819a3e5 100644 --- a/doc/index.rst +++ b/doc/index.rst @@ -21,4 +21,4 @@ XMPP client as if these channels were XMPP MUCs. admin user contributing - developper + developer -- cgit v1.2.3