Harden tests for caa/dns/rdap/email_auth (mutation-testing findings) - #8
Merged
Conversation
Adds 67 tests surfaced by a mutation-testing pass (mutmut) that exposed untested behavior. No production code changes — tests only. Highlights of what was untested and is now covered: - dns: is_ip_in_cidr had NO test at all; real TXT/SPF parsing and get_report argument wiring were only exercised through mocks. - caa: _query_caa was fully mocked in every test, so its rdata decoding, flag handling, and error paths were never run; added tree-climb order, result-shape, and the and->or / continue->break logic-branch tests. - rdap: the raw ip/domain/autnum HTTP methods had no coverage; added strict parse_domain field assertions, error-guard, and _vcard_field edge cases. - email_auth: the SPF resolution helpers (_query, _resolve_addresses, _resolve_mx, _query_dkim_selector, _rdata_to_string) were entirely unexercised; added full-dict parser assertions (SPF/DKIM/DMARC/BIMI) and resolution-tree node/mechanism/depth-limit tests. These also caught real behavior worth locking in (CAA forbids_issuance and-vs-or, tree-climb continue-vs-break, DNS TXT multi-record handling). All 344 tests pass; mypy clean. No mutmut tooling committed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This PR hardens the test suite for DNS/CAA/RDAP/email-auth functionality based on mutation-testing findings, adding assertions that exercise real code paths rather than validating mocks.
Changes:
- Expanded RDAP tests to cover strict parsed-field shapes, HTTP method wiring, and error-guard branches.
- Added extensive SPF/DKIM/DMARC/BIMI parsing + SPF resolution-tree tests, including previously uncovered helper methods.
- Added DNS tests for
is_ip_in_cidr, real TXT/SPF parsing behavior, andget_reportargument wiring; expanded CAA tests to exercise real_query_caaparsing and climb behavior.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| tests/test_rdap.py | Adds strict RDAP parsing assertions, raw HTTP method contract tests, and vCard/error-guard edge cases. |
| tests/test_email_auth.py | Adds full-dict parser assertions and deep coverage for SPF/DKIM/DMARC/BIMI resolution helpers and tree behavior. |
| tests/test_dns.py | Adds coverage for CIDR checks, TXT/SPF parsing in get_dns_info, and get_report argument propagation. |
| tests/test_caa.py | Adds coverage for real _query_caa decoding/error cases and additional analyze() shape/tree-climb assertions. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Tests-only PR. A mutation-testing pass (mutmut) against
mainsurfaced behavior that was executed but never asserted — often because tests mocked the very method under test. This adds 67 tests to close those gaps. No production code changes.What was untested → now covered
dns.pyis_ip_in_cidrhad no test at all (IPv4/IPv6 in/out of range, invalid IP/CIDR, boundaries)._parse_txt_recordquote/whitespace handling) andget_reportargument wiring were only exercised through mocks — added tests that run the real code paths.caa.py_query_caawas fully mocked in every existing test, so its rdata decoding, flag/criticalhandling, and error paths never ran. Added tests driving the real method via fake dnspython rdata.and→or(forbids_issuance) /continue→break(climb-past-error) logic branches.rdap.pyip/domain/autnumHTTP methods had no coverage — added tests asserting verb/URL/timeout.parse_domainfield assertions, the not-a-domain error guard, and_vcard_fieldedge cases (missing/short vcard, list vs scalar value).email_auth.py_query,_resolve_addresses,_resolve_mx,_query_dkim_selector,_rdata_to_string) were entirely unexercised.a/mx/ptr/existshandling, loop guard, depth limit).Why it matters
Several gaps were the "tests validate a mock, not the code" anti-pattern — the assertions passed regardless of whether the real function worked. These tests exercise the actual logic and lock in real contracts (e.g. CAA must keep climbing past a transient error; DNS must collect IPs from a later TXT record, not stop at the first).
Testing
uv run pytest: 344 passed (was 277).uv run mypy src/domain_profiler: clean.🤖 Generated with Claude Code