Skip to content

Harden tests for caa/dns/rdap/email_auth (mutation-testing findings) - #8

Merged
MSAdministrator merged 2 commits into
mainfrom
test/mutation-hardened-tests
Jul 6, 2026
Merged

Harden tests for caa/dns/rdap/email_auth (mutation-testing findings)#8
MSAdministrator merged 2 commits into
mainfrom
test/mutation-hardened-tests

Conversation

@MSAdministrator

Copy link
Copy Markdown
Owner

Summary

Tests-only PR. A mutation-testing pass (mutmut) against main surfaced behavior that was executed but never asserted — often because tests mocked the very method under test. This adds 67 tests to close those gaps. No production code changes.

What was untested → now covered

dns.py

  • is_ip_in_cidr had no test at all (IPv4/IPv6 in/out of range, invalid IP/CIDR, boundaries).
  • Real TXT/SPF parsing (_parse_txt_record quote/whitespace handling) and get_report argument wiring were only exercised through mocks — added tests that run the real code paths.

caa.py

  • _query_caa was fully mocked in every existing test, so its rdata decoding, flag/critical handling, and error paths never ran. Added tests driving the real method via fake dnspython rdata.
  • Tree-climb query order + stop-at-first-match, strict result-dict shape, and the andor (forbids_issuance) / continuebreak (climb-past-error) logic branches.

rdap.py

  • The raw ip/domain/autnum HTTP methods had no coverage — added tests asserting verb/URL/timeout.
  • Strict parse_domain field assertions, the not-a-domain error guard, and _vcard_field edge cases (missing/short vcard, list vs scalar value).

email_auth.py

  • The SPF-resolution helpers (_query, _resolve_addresses, _resolve_mx, _query_dkim_selector, _rdata_to_string) were entirely unexercised.
  • Full-dict parser assertions for SPF/DKIM/DMARC/BIMI (every mechanism/tag form + defaults), and resolution-tree tests (node shape, a/mx/ptr/exists handling, loop guard, depth limit).

Why it matters

Several gaps were the "tests validate a mock, not the code" anti-pattern — the assertions passed regardless of whether the real function worked. These tests exercise the actual logic and lock in real contracts (e.g. CAA must keep climbing past a transient error; DNS must collect IPs from a later TXT record, not stop at the first).

Testing

  • uv run pytest: 344 passed (was 277).
  • uv run mypy src/domain_profiler: clean.
  • The mutmut tooling/config was not committed — this is pytest-only.

🤖 Generated with Claude Code

Adds 67 tests surfaced by a mutation-testing pass (mutmut) that exposed
untested behavior. No production code changes — tests only.

Highlights of what was untested and is now covered:
- dns: is_ip_in_cidr had NO test at all; real TXT/SPF parsing and
  get_report argument wiring were only exercised through mocks.
- caa: _query_caa was fully mocked in every test, so its rdata decoding,
  flag handling, and error paths were never run; added tree-climb order,
  result-shape, and the and->or / continue->break logic-branch tests.
- rdap: the raw ip/domain/autnum HTTP methods had no coverage; added
  strict parse_domain field assertions, error-guard, and _vcard_field
  edge cases.
- email_auth: the SPF resolution helpers (_query, _resolve_addresses,
  _resolve_mx, _query_dkim_selector, _rdata_to_string) were entirely
  unexercised; added full-dict parser assertions (SPF/DKIM/DMARC/BIMI)
  and resolution-tree node/mechanism/depth-limit tests.

These also caught real behavior worth locking in (CAA forbids_issuance
and-vs-or, tree-climb continue-vs-break, DNS TXT multi-record handling).

All 344 tests pass; mypy clean. No mutmut tooling committed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR hardens the test suite for DNS/CAA/RDAP/email-auth functionality based on mutation-testing findings, adding assertions that exercise real code paths rather than validating mocks.

Changes:

  • Expanded RDAP tests to cover strict parsed-field shapes, HTTP method wiring, and error-guard branches.
  • Added extensive SPF/DKIM/DMARC/BIMI parsing + SPF resolution-tree tests, including previously uncovered helper methods.
  • Added DNS tests for is_ip_in_cidr, real TXT/SPF parsing behavior, and get_report argument wiring; expanded CAA tests to exercise real _query_caa parsing and climb behavior.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File Description
tests/test_rdap.py Adds strict RDAP parsing assertions, raw HTTP method contract tests, and vCard/error-guard edge cases.
tests/test_email_auth.py Adds full-dict parser assertions and deep coverage for SPF/DKIM/DMARC/BIMI resolution helpers and tree behavior.
tests/test_dns.py Adds coverage for CIDR checks, TXT/SPF parsing in get_dns_info, and get_report argument propagation.
tests/test_caa.py Adds coverage for real _query_caa decoding/error cases and additional analyze() shape/tree-climb assertions.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread tests/test_rdap.py Outdated
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
@MSAdministrator
MSAdministrator merged commit b5fed63 into main Jul 6, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants