Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

✨ Allowing pytest cli arguments for host and port #8

Merged
merged 2 commits into from
Mar 29, 2024

Conversation

zedfmario
Copy link
Contributor

TL;DR

Adding two optional pytest command line arguments describing a MQTT broker:

  • --mqtt_host (defaults to localhost)
  • --mqtt_port (defaults to 1883)

Hi there,

First of all, thank you for the project. It's been super helpful while developing MQTT related features.

However, while using the library I found it didn't quite match my current needs, so I tried to find a way to solve my issue. I hope you find this PR suitable for your package. Since it's my first time contributing to your project, I'm not 100% sure I'm covering all the requirements. Feel free to suggest any changes you may find to match your criteria, please.

I needed to test against a remote MQTT broker, not running on localhost. All connections to the MQTT broker were hardcoded to localhost:1883 though. I added the chance to make both values configurable, based on pytest cli arguments, using the request fixture.

Example of use

Given the following docker-compose.yml file

services:
  mqtt:
    image: eclipse-mosquitto:2
    container_name: mqtt
    hostname: mqtt
    restart: unless-stopped
    ports:
      - "11883:1883/tcp"
    volumes:
      - ./mosquitto.conf:/mosquitto/config/mosquitto.conf:rw

and starting the MQTT server on port 11883

> docker compose up mqtt
[+] Running 1/0
 ✔ Container mqtt  Created                                                                                                                                                                                                                                                                           0.0s
Attaching to mqtt
mqtt  | 1701088758: mosquitto version 2.0.18 starting
mqtt  | 1701088758: Config loaded from /mosquitto/config/mosquitto.conf.
mqtt  | 1701088758: Opening ipv4 listen socket on port 1883.
mqtt  | 1701088758: Opening ipv6 listen socket on port 1883.
mqtt  | 1701088758: mosquitto version 2.0.18 running

> docker ps -a
CONTAINER ID   IMAGE                 COMMAND                  CREATED          STATUS         PORTS                     NAMES
690e16652154   eclipse-mosquitto:2   "/docker-entrypoint.…"   22 seconds ago   Up 3 seconds   0.0.0.0:11883->1883/tcp   mqtt

running pytest against the new port works

> pytest --mqtt_port 11883 testing
========================================================================================================================================== test session starts ===========================================================================================================================================
platform darwin -- Python 3.11.5, pytest-7.4.3, pluggy-1.3.0 -- /Workspace/pytest-mqtt/.venv/bin/python3.11
cachedir: .pytest_cache
rootdir: /Workspace/pytest-mqtt
configfile: pyproject.toml
plugins: fixture-order-0.1.4, mqtt-0.3.2, ordering-0.6
collected 10 items

testing/test_mosquitto.py::test_mosquitto_running SKIPPED (Unable to run together with other test cases)                                                                                                                                                                                           [ 10%]
testing/test_app.py::test_app_version PASSED                                                                                                                                                                                                                                                       [ 20%]
testing/test_capmqtt.py::test_mqtt_client_adapter PASSED                                                                                                                                                                                                                                           [ 30%]
testing/test_integration.py::test_basic_submit_text_receive_binary PASSED                                                                                                                                                                                                                          [ 40%]
testing/test_integration.py::test_basic_submit_and_receive_binary PASSED                                                                                                                                                                                                                           [ 50%]
testing/test_integration.py::test_basic_submit_text_receive_text_marker PASSED                                                                                                                                                                                                                     [ 60%]
testing/test_integration.py::test_basic_submit_text_receive_text_config PASSED                                                                                                                                                                                                                     [ 70%]
testing/test_module_settings.py::test_basic_submit_text_receive_text PASSED                                                                                                                                                                                                                        [ 80%]
testing/test_util.py::test_probe_tcp_connect_available PASSED                                                                                                                                                                                                                                      [ 90%]
testing/test_util.py::test_probe_tcp_connect_unavailable PASSED

and the logs on the mqtt broker running on docker:

mqtt  | 1701089046: New connection from 192.168.144.1:44976 on port 1883.
mqtt  | 1701089046: New connection from 192.168.144.1:44968 on port 1883.
mqtt  | 1701089046: Client <unknown> closed its connection.
mqtt  | 1701089046: New client connected from 192.168.144.1:44968 as auto-39EFDC26-0F83-C3F9-AA61-DCA97AFBF938 (p2, c1, k60).
mqtt  | 1701089046: Client auto-39EFDC26-0F83-C3F9-AA61-DCA97AFBF938 disconnected.
mqtt  | 1701089046: New connection from 192.168.144.1:44982 on port 1883.
mqtt  | 1701089046: New client connected from 192.168.144.1:44982 as auto-FFEA2F95-4EB1-EA94-FE76-6E55A5C25586 (p2, c1, k60).
mqtt  | 1701089047: Client auto-FFEA2F95-4EB1-EA94-FE76-6E55A5C25586 disconnected.
mqtt  | 1701089047: New connection from 192.168.144.1:44988 on port 1883.
mqtt  | 1701089047: New client connected from 192.168.144.1:44988 as auto-ADC8D5D6-E426-6308-D1E6-C744790E751D (p2, c1, k60).
mqtt  | 1701089047: Client auto-ADC8D5D6-E426-6308-D1E6-C744790E751D disconnected.
mqtt  | 1701089047: New connection from 192.168.144.1:45002 on port 1883.
mqtt  | 1701089047: New client connected from 192.168.144.1:45002 as auto-EB7D08D0-F56B-F118-AC3C-33BACBEE90C4 (p2, c1, k60).
mqtt  | 1701089047: Client auto-EB7D08D0-F56B-F118-AC3C-33BACBEE90C4 disconnected.
mqtt  | 1701089047: New connection from 192.168.144.1:45008 on port 1883.
mqtt  | 1701089047: New client connected from 192.168.144.1:45008 as auto-BE6EE2C2-B903-6E3F-38DE-3830E0F5FF78 (p2, c1, k60).
mqtt  | 1701089047: Client auto-BE6EE2C2-B903-6E3F-38DE-3830E0F5FF78 disconnected.
mqtt  | 1701089047: New connection from 192.168.144.1:45016 on port 1883.
mqtt  | 1701089047: New client connected from 192.168.144.1:45016 as auto-5DA27123-440C-7DBF-0DC5-649FE943F244 (p2, c1, k60).
mqtt  | 1701089048: Client auto-5DA27123-440C-7DBF-0DC5-649FE943F244 disconnected.

Copy link

codecov bot commented Nov 27, 2023

Codecov Report

Attention: Patch coverage is 95.83333% with 1 lines in your changes are missing coverage. Please review.

Project coverage is 95.45%. Comparing base (0e4c401) to head (6d94524).
Report is 1 commits behind head on main.

Files Patch % Lines
pytest_mqtt/mosquitto.py 90.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main       #8      +/-   ##
==========================================
+ Coverage   95.07%   95.45%   +0.38%     
==========================================
  Files           5        5              
  Lines         142      154      +12     
==========================================
+ Hits          135      147      +12     
  Misses          7        7              
Flag Coverage Δ
unittests 95.45% <95.83%> (+0.38%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@amotl
Copy link
Member

amotl commented Mar 28, 2024

Dear Mario,

thank you so much for submitting this patch, and apologies for missing it up until now. Your contribution is well received, and will be part of the next release of pytest-mqtt.

With kind regards,
Andreas.

@amotl amotl self-assigned this Mar 28, 2024
pyproject.toml Outdated
Comment on lines 63 to 66
[project.entry-points.pytest11]
mqttcliargs = "pytest_mqtt.mqttcliargs"
capmqtt = "pytest_mqtt.capmqtt"
mosquitto = "pytest_mqtt.mosquitto"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't think that new mqttcliargs fixture needs to be exported for public consumption. Now that the new {host,port} information is also available as attributes on MqttClientAdapter, it can easily be accessed through the capmqtt fixture, right?

def test_foo(capmqtt):
    assert capmqtt.mqtt_client.host == "localhost"
    assert capmqtt.mqtt_client.port == 1883

For improved convenience, they could be mirrored / also accessed by additional properties on MqttCaptureFixture, like:

class MqttCaptureFixture:

    @property
    def host(self):
        return self.mqtt_client.host

    @property
    def port(self):
        return self.mqtt_client.port
def test_foo(capmqtt):
    assert capmqtt.host == "localhost"
    assert capmqtt.port == 1883

Comment on lines 2 to 3
parser.addoption("--mqtt_host", action="store", default="localhost", help="mqtt host to be connected through")
parser.addoption("--mqtt_port", action="store", default=1883, help="mqtt port to be connected through")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a suggestion about "naming things".

Suggested change
parser.addoption("--mqtt_host", action="store", default="localhost", help="mqtt host to be connected through")
parser.addoption("--mqtt_port", action="store", default=1883, help="mqtt port to be connected through")
parser.addoption("--mqtt-host", action="store", default="localhost", help="mqtt host to be connected through")
parser.addoption("--mqtt-port", action="store", default=1883, help="mqtt port to be connected through")

pytest_mqtt/mqttcliargs.py Outdated Show resolved Hide resolved
@@ -0,0 +1,3 @@
def pytest_addoption(parser) -> None:
parser.addoption("--mqtt_host", action="store", default="localhost", help="mqtt host to be connected through")
parser.addoption("--mqtt_port", action="store", default=1883, help="mqtt port to be connected through")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about conveying type=int?
-- https://docs.python.org/3/library/argparse.html#type

Suggested change
parser.addoption("--mqtt_port", action="store", default=1883, help="mqtt port to be connected through")
parser.addoption("--mqtt_port", action="store", type=int, default=1883, help="mqtt port to be connected through")

Comment on lines 6 to 10
@pytest.fixture(scope="session")
def mqttcliargs(request) -> t.Tuple[str, int]:
host = request.config.getoption("--mqtt_host", "localhost")
port = int(request.config.getoption("--mqtt_port", 1883))
yield host, port
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about conveying type=int?

We think the mqttcliargs shortcut fixture, for carrying around host/port information like this, should be dissolved completely.

Instead, we suggest to use pytestconfig, for acquiring configuration options. Because default values are already defined in pytest_addoption, there is no need to define them once again redundantly.

Comment on lines 122 to 133
def capmqtt(request):
def capmqtt(request, mqttcliargs):
"""Access and control MQTT messages."""

# Configure `capmqtt` fixture, obtaining the `capmqtt_decode_utf8` setting from
# either a global or module-wide setting, or from a test case marker.
# https://docs.pytest.org/en/7.1.x/how-to/fixtures.html#fixtures-can-introspect-the-requesting-test-context
# https://docs.pytest.org/en/7.1.x/how-to/fixtures.html#using-markers-to-pass-data-to-fixtures

host, port = mqttcliargs

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead, you may already be able to use the built-in pytestconfig fixture, in order to retrieve configuration options.

def capmqtt(request, pytestconfig):
    host = pytestconfig.getoption("--host")
    port = int(pytestconfig.getoption("--port"))

-- https://docs.pytest.org/en/8.0.x/reference/reference.html#std-fixture-pytestconfig

Comment on lines 134 to 138
capmqtt_decode_utf8 = (
getattr(request.config.option, "capmqtt_decode_utf8", False)
or getattr(request.module, "capmqtt_decode_utf8", False)
or request.node.get_closest_marker("capmqtt_decode_utf8") is not None
)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When acquiring configuration options like this, they can be provided through three different means, see usage of capmqtt_decode_utf8.

Acquiring capmqtt_host and capmqtt_port could be implemented in the very same way, so they are not too special?

... in order to connect to an MQTT broker on a different endpoint than
`localhost:1883`.
@amotl
Copy link
Member

amotl commented Mar 29, 2024

Dear Mario,

we push a few adjustments to your branch, along the lines of our suggestions, and renamed the CLI parameters to --mqtt-host resp. --mqtt-port. Thanks again for your contribution!

With kind regards,
Andreas.

@amotl amotl merged commit 0e80622 into mqtt-tools:main Mar 29, 2024
8 checks passed
@amotl
Copy link
Member

amotl commented Mar 31, 2024

Hi again,

pytest-mqtt 0.4.0 has been released, including corresponding improvements from your pen, slightly adjusted. Thank you very much!

With kind regards,
Andreas.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants