Skip to content

Test suite is red on main and CI never runs it: pytest is in neither the dev group nor pr-validation.yaml #585

Description

@rajarshidattapy

Summary

tests/test_responses_namespace.py has been failing on main since 2026-07-25. Nothing
caught it, because pytest is not installed by the project's own dev tooling and no
workflow ever invokes it.

Steps to reproduce

uv sync --group dev
uv run --group dev pytest tests

Actual behavior

The command cannot run at all — uv sync --group dev installs only mypy, pylint, and
pyright, so pytest is not in the venv. Forcing it in reveals the failure:

$ uv run --with pytest pytest -q
E  assert <class 'openrouter.responses.Responses'> is <class 'openrouter.beta_responses.BetaResponses'>
tests/test_responses_namespace.py:9: AssertionError
1 failed, 3 passed

Cause 1: pytest is not a dev dependency

pyproject.toml configures pytest and ships a tests/ directory:

[tool.pytest.ini_options]
asyncio_default_fixture_loop_scope = "function"
pythonpath = ["src"]

[dependency-groups]
dev = [
    "mypy ==1.15.0",
    "pylint ==3.2.3",
    "pyright ==1.1.398",
]

There is no pytest, and no pytest-asyncio despite the
asyncio_default_fixture_loop_scope setting — which currently produces
PytestConfigWarning: Unknown config option on any run that manages to get pytest onto
the path.

Cause 2: no CI step runs the tests

.github/workflows/pr-validation.yaml runs four steps — uv build, mypy src,
pyright src, pylint src --rcfile pylintrc. None of them execute tests/. All four pass
on main today, so PR validation is green while the test suite is red.

Why the assertion fails

The assertion is stale rather than catching a real defect:

  • c75a93c "feat(responses): take Responses API GA" (2026-07-24) added the test. At that
    commit beta.py did self.responses = Responses(...) — literally the same class, so
    type(client.responses) is type(client.beta.responses) held.
  • 1477485 "fix(responses): make beta.responses alias durable and pin version to 1.1.0"
    (2026-07-25) changed it to self.responses = BetaResponses(...), a distinct generated
    class. The identity check broke the next day and has been red for a week.

The alias itself is fine — both classes expose exactly the same public surface
(send, send_async, do_request, do_request_async, parent_ref,
sdk_configuration). Only the identity assertion is wrong.

Suggested fix

  1. Assert the alias's behavior instead of its identity — that both namespaces expose the same
    operations, and that the deprecation notice survives.
    deprecated-beta-responses-alias.overlay.yaml's own comments warn that the overlay can
    silently match nothing after a monorepo sync and drop that notice, and nothing currently
    checks for it.
  2. Add pytest and pytest-asyncio to the dev group, and a pytest tests step to
    pr-validation.yaml.

One caveat for step 2: pyproject.toml is listed in .speakeasy/gen.lock, so it is
generator-owned. Adding the dependencies only there will work until the next regeneration
overwrites them. The durable location is python.additionalDependencies.dev in
.speakeasy/gen.yaml, which is currently {}.

Environment

  • openrouter 1.1.23 (main @ cb401a8)
  • Python 3.13.6, uv 0.9.21
  • Windows 11

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions