Python integration tests with pytest and docker compose

I really like pytest. It’s a great tool for writing unit tests in Python, as it provides a lot of features out of the box. Some of them are quite magical, but they can make your life much easier, so at the end you need much less code.

Recently, I experimented with using pytest fixture to create a simple integration test skeleton which integrates docker.

Fixtures

Fixtures in pytest are a way to request for something in the test code. There are two basic types of fixtures.[1]

return fixtures which provide a certain value or object
- This means we execute some code before requesting a fixture
yield fixtures which provide a certain state
- This means we execute some code before and after requesting a fixture

In this case the latter is be needed, as we want to perform the test in a state when docker container is running. This way we can startup a docker container and clean it up regardless of the test results.

I’m using docker compose, as it provides more flexibility and it’s cleaner than CLI flags for pure docker. I’d use it even when you have only a single container, as overhead is minimal.

Implementation

You can see the test file skeleton here.

import shutil
import subprocess
import time

import pytest


def tested_function():
    ...


@pytest.fixture
def docker_container():
    out = subprocess.run(
        ('docker', 'compose', '--file', 'testing/docker-compose.yml', 'up', '--detach'),
        check=True,
    )
    time.sleep(10)
    yield out
    subprocess.run(
        ('docker', 'compose', '--file', 'testing/docker-compose.yml', 'down'),
        check=True,
    )


def test_integration(docker_container):
    actual = tested_function()
    expected = '<expected output>'
    assert actual == expected

The flow is quite simple here:

test_integration function starts
It requests docker_container, so the fixture function starts
We reach yield in docker_container and go back to the test function
docker_container value in test_integration is a return value of subprocess.run from the fixture
- I’m not using it in this example
Test code is run up to the end
Either if any exception is thrown or if the test code finished, we go back to the fixture and execute rest of the code
- In this case it’s docker compose down

The biggest drawback of this approach is time.sleep, as out is returned when docker compose up starts running, and it doesn’t mean that the container is ready for the test. Depending on your use case you would need less or even more time here.

A smarter approach would be getting the container notify the code that it’s ready in an event-driven fashion. The presented solution is currently enough for my use case, even though it might be quite limiting if we want to scale it and run multiple test cases.

But for my use case this was good enough, so I stopped here, and maybe I’ll expand it in the future.

Integration

By default pytest picks up all files which are named test_*.py or *_test.py. As integration tests are quite longer than unit tests, we don’t want them to be run each time along with unit tests.

I chose to rename the file with integration tests to integration.py which I put in tests folder along with other test files. This works fine on a small-scale project. In a bigger project, I’d probably create a separate folder in tests or maybe create a new top level folder like integration_tests.

With the simple approach running these tests is a simple pytest tests/integration.py, which I put under Makefile target integration-tests. This way I can type make i and let tab-completion do the rest for me.

In CI, I’m running them in the same job as unit tests, directly after them, as I assume that it doesn’t make sense to run costly integration tests if any of my unit tests is failing (even if GitHub pays for my CI 😛).

You can see the actual implementation in my event-scrapper-srt project.

To understand how they work here’s a great video from Anthony Sottile about it ↩