Doc on conftest.py hook interactions with pytest_addoption() possibly misleading

[TTsangSC]

Summary

The doc page "Writing hook functions" seems to suggest that conftest.py hooks are available at pytest_addoption() time, which isn't the case; only hooks installed by other third-party plugins are (EDIT (19 Mar): sometimes; see edit below).

Details

On the aforementioned page, the section "Using hooks in pytest_addoption" contains a minimal example of a plugin myplugin which defines a hook pytest_config_file_default_value(), from which pytest_addoption() should be able to extract a user-defined default for the command-line option --config-file via pluginmanager.hook (excerpt of the code block):

# contents of myplugin.py

...

def pytest_addoption(parser, pluginmanager):
    default_value = pluginmanager.hook.pytest_config_file_default_value()
    parser.addoption(
        "--config-file",
        help="Config file to use, defaults to %(default)s",
        default=default_value,
    )

It then goes on to say

The conftest.py that is using myplugin would simply define the hook as follows:

def pytest_config_file_default_value():
    return "config.yaml"

Maybe I'm misinterpreting, but it seems that it is implied that conftest.py::pytest_config_file_default_value() should be available at the time when myplugin.py::pytest_addoption() is executed, and thus the help text for the --config-file option should read Config file to use, defaults to config.yaml.

However, as the minimal example in the section below demonstrates, conftest.py implementations of hooks aren't visible to pluginmanager.hook at the time pytest_addoption() is run, while implementations living in other installed plugins are.

Example

(EDIT (19 Mar): made collapsible; click to expand)

#!/usr/bin/env bash

:
: 'Bash setup'
:
set -e
trap cleanup EXIT

function cleanup() {
    cd "${START_DIR}" && \
        [ -n "${TEST_DIR}" ] && \
        [ -d "${TEST_DIR}" ] && \
        rm -r "${TEST_DIR}" && \
        echo removed test dir "${TEST_DIR}"
}

START_DIR="${PWD}"
TEST_DIR="$(readlink -f "$(mktemp -d "./pytest-addoption-bug-XXXX")")"
cd "${TEST_DIR}"

:
: 'Setup a venv'
:
python3.13 -m venv venv
source venv/bin/activate
set -x
pip install --quiet --quiet --quiet pytest
pip list

:
: 'Create a minimal plugin that defines a new option'
:
mkdir plugin-1
cat >plugin-1/pyproject.toml <<-'!'
[build-system]
requires = ['setuptools']
build-backend = 'setuptools.build_meta'

[project]
name = 'plugin-1'
version = '0.0'

[project.entry-points.pytest11]
plugin_1 = 'plugin_1'
!
cat >plugin-1/plugin_1.py <<-'!'
from types import SimpleNamespace as namespace

import pytest


@pytest.hookspec(firstresult=True)
def pytest_foo_default() -> str:
    ...


def pytest_addhooks(pluginmanager: pytest.PytestPluginManager) -> None:
    pluginmanager.add_hookspecs(
        namespace(pytest_foo_default=pytest_foo_default),
    )


def pytest_addoption(
    parser: pytest.Parser, pluginmanager: pytest.PytestPluginManager
) -> None:
    default = pluginmanager.hook.pytest_foo_default()
    parser.addoption('--foo', help='default: %(default)s', default=default)
!
pip install --quiet --quiet --quiet --editable ./plugin-1
python -c "import inspect; import plugin_1; print(inspect.getsource(plugin_1))"
: 'The `grep` should output sth like `--foo=FOO default: None`'
pytest --help | grep -e --foo


:
: 'Create another plugin, which supplies a default for the option'
:
mkdir plugin-2
cat >plugin-2/pyproject.toml <<-'!'
[build-system]
requires = ['setuptools']
build-backend = 'setuptools.build_meta'

[project]
name = 'plugin-2'
version = '0.0'

[project.entry-points.pytest11]
plugin_2 = 'plugin_2'
!
cat >plugin-2/plugin_2.py <<-'!'
def pytest_foo_default() -> str:
    return 'bar'
!
pip install --quiet --quiet --quiet --editable ./plugin-2
python -c "import inspect; import plugin_2; print(inspect.getsource(plugin_2))"
: 'The `grep` should output sth like `--foo=FOO default: bar`'
pytest --help | grep -e --foo
pip uninstall --yes --quiet --quiet --quiet plugin-2


:
: 'Now create a package which supplies a default for the option in an'
: '"initial" conftest.py...'
: "except it isn't picked up by the hook"
:
mkdir -p package-3/package_3/tests
touch package-3/package_3/{,tests/}__init__.py
cat >package-3/pyproject.toml <<-'!'
[build-system]
requires = ['setuptools']
build-backend = 'setuptools.build_meta'

[project]
name = 'package-3'
version = '0.0'
!
cat >package-3/package_3/tests/conftest.py <<-'!'
def pytest_foo_default() -> str:
    return 'spam'
!
cat >package-3/package_3/tests/test_dummy.py <<-'!'
def test_dummy() -> None: pass
!
pip install --quiet --quiet --quiet --editable ./package-3
PYTHON_SCRIPT="from inspect import getsource as gs; "
PYTHON_SCRIPT+="from os.path import relpath as rp; "
PYTHON_SCRIPT+="from package_3.tests import conftest as cf, test_dummy as td; "
PYTHON_SCRIPT+="nl = '\\n'; "
PYTHON_SCRIPT+="[print('==> {0} <=={2}{2}{1}{2}'"
PYTHON_SCRIPT+=".format(rp(m.__file__), gs(m), nl)) for m in (cf, td)]"
python -c "${PYTHON_SCRIPT}"
: 'The test does collect...'
pytest --co -qq package-3/package_3/tests
: '... so the `grep` should output sth like `--foo=FOO default: spam`...'
: "but it doesn't"
pytest --help package-3/package_3/tests | grep -e --foo

Unless I'm mistaken, since the conftest.py lives in the directory explicitly supplied as a testpath, it should qualify as an "initial" conftest.py file. Nonetheless, running the above example shows that only plugin-2/plugin_2.py::pytest_foo_default() were picked up by plugin-1/plugin_1.py::pytest_addoption(), but not package-3/tests/conftest.py::pytest_foo_default().

While it is understandable that conftest.py configurations are located and loaded later than when the command-line options are defined, this contradicts what the above aforementioned doc seem to imply – that command-line option defaults can be loaded therefrom. Maybe it could use some clarification?

Versions and platform info

(venv)  $ pip list
Package   Version
--------- -------
iniconfig 2.0.0
packaging 24.2
pip       25.0
pluggy    1.5.0
pytest    8.3.5
(venv)  $ python -c "import platform; import sys; print(platform.version(), sys.version, sep='\\n')"
Darwin Kernel Version 23.6.0: Mon Jul 29 21:14:30 PDT 2024; root:xnu-10063.141.2~1/RELEASE_ARM64_T6030
3.13.2 (main, Feb  4 2025, 14:51:09) [Clang 16.0.0 (clang-1600.0.26.6)]

EDIT (19 Mar 2025)

In one of the tests for some plugin (see e.g. plugin-1 in the Example), I once wrote something like the below to test the aforementioned availability of hook implementations at pytest_addoption() time:

Test code (click to expand)

from pathlib import path
from textwrap import dedent
from uuid import uuid4


def write_file(path: Path, content: str) -> None:
    path.write_text(dedent(content).strip('\n'))


@pytest.fixture
def plugin(pytester: pytest.Pytester) -> str:
    # Build a single-file plugin package
    plugin_name = 'my-plugin-' + uuid4()
    path = pytester.mkdir(plugin_name)
    python_name = plugin_name.replace('-', '_')
    write_file(
        path / 'pyproject.toml',
        f"""
    [build-system]
    requires = ['setuptools']
    build-backend = 'setuptools.build_meta'

    [project]
    name = {0!r}
    version = '0.0'

    [project.entry-points-pytest11]
    {1} = {1!r}
        """.format(plugin_name, python_name),
    )
    write_file(
        path / (python_name + '.py'),
        """
    def pytest_foo_default() -> str:
        return 'bar'
        """,
    )
    # Install
    if pytester.run(
        sys.executable, '-m', 'pip', 'install', '-qqq', path,
    ).ret:
        raise RuntimeError('Cannot install plugin')
    yield python_name
    # Uninstall
    if pytester.run(
        sys.executable, '-m', 'pip', 'uninstall', '-qqq',
        python_name,
    ).ret:
        raise RuntimeError('Cannot install plugin')


def one_line(lines: list[str]) -> str:
    return ' '.join('\n'.join(lines).split())


def test_load_default_from_plugin(
    pytester: pytest.Pytester, plugin: str,
):
    """
    Test that the `pytest_foo_default()` defined in
    the above plugin is available at `pytest_addoption()`
    time.
    """
    assert any(
        plugin in line
        for line in pytester.runpytest('--co', '--trace-config').outlines
    )
    lines = one_line(pytester.runpytest('--help').outlines)
    assert '--foo default: bar' in lines

However, the test only passed around 50% of the time, failing on the last assertion. Examining the output of the first runpytest() call revealed that the 'bar' default is only available if the dynamically-generated plugin is loaded before the one I'm testing, but it's a coin toss whether that or the opposite happens. And since we don't have a way to control the ordering between third-party plugins yet (#935), it seems that there is no way to make plugin-supplied default values available when the command-line options are generated.

One exception is when a plugin defines the hook spec, a plugin (could be the same or another) supplies the hook implementation, and the conftest.py adds the command-line flag by defining pytest_addoption(). But that seems to be diametrically opposite to what the doc example says.

Checklist

• a detailed description of the bug or problem you are having
• output of pip list from the virtual environment you are using
• pytest and operating system versions
• minimal example if possible

[RonnyPfannschmidt]

Addoption is avaliable for so called initial contests

We man need to more explicitly document which files are eligible

[TTsangSC]

Thanks for the swift reply. It seems that the case you are referring to is when the pytest_addoption() lives in the initial conftest.py file itself, like in this SO question...? So, while initial conftest.py files can indeed add their own options, they are probably loaded too late to influence options added by installed third-party plugins. And since we don't have option conflict resolution yet, it isn't like conftest.py can overload the preexisting option with the new default either – at least not with pytest_addoption().

In any case, clearer documentations are always much welcome, and thanks for maintaining the package.