Skip to content

fix(meta_ads): count conversions via a canonical exact-match counter (#340)#343

Merged
hyoshi merged 1 commit into
mainfrom
fix/meta-conversion-overcount
Jun 26, 2026
Merged

fix(meta_ads): count conversions via a canonical exact-match counter (#340)#343
hyoshi merged 1 commit into
mainfrom
fix/meta-conversion-overcount

Conversation

@hyoshi

@hyoshi hyoshi commented Jun 26, 2026

Copy link
Copy Markdown
Collaborator

Summary

meta_row_conversions (the Meta conversion counter feeding CPA → anomaly
detection / diagnose_performance / budget-efficiency) counted via a
substring scan "lead" in action_type or "purchase" in action_type, which
over-counted for two reasons:

  1. Alias double-count. Meta's Insights actions array returns one
    conversion under several overlapping action_type rows. The generic lead
    is, by Meta's own definition, the aggregate ("all offsite + all
    On-Facebook leads") of its components offsite_conversion.fb_pixel_lead +
    onsite_conversion.lead_grouped; purchase/omni_purchase is the
    analogous roll-up. Because actions is fetched unfiltered, aggregate +
    component rows co-occur, so summing every *lead*/*purchase* row counted
    the same conversions two or three times.
  2. Substring false positives. Operator-named custom conversions
    (offsite_conversion.custom.<slug>) carry free-text slugs that may contain
    lead/purchase and were swept in.

This deflated the absolute CPA emitted by diagnose_performance and the
budget-efficiency scorer (the anomaly-detector CPA-spike path is ratio-based
and largely shielded). It also diverged from the MCP-analysis counter
_extract_cv, which already used an exact {lead, purchase, complete_registration} set — so multiple live counters disagreed on what
counts as a conversion.

Fix

A single canonical counter mureo/meta_ads/_conversion_count.py
(count_conversions_from_actions + CONVERSION_ACTION_TYPES) that counts
only the deduped generic conversion action_types — the aggregate already
includes its components, so components are not added on top, and custom slugs
no longer match. All four live counters now route through it so they cannot
drift again:

  • mureo/analytics/builtin/_common.py meta_row_conversions (lazy import to
    keep adapter registration free of the mureo.meta_ads client weight; BYOD
    branch unchanged)
  • mureo/meta_ads/_analysis.py _extract_cv (delegates)
  • mureo/analytics/builtin/_budget_efficiency.py
  • mureo/meta_ads/_insights.py breakdown

The campaign-type classifier (_classify_result_indicator) keeps substring
matching — it is a boolean "any conversion signal?" check, not a counter, so
substring is correct there.

Behaviour notes

Test plan

  • tests/test_meta_conversion_count.py — dedup (aggregate+component),
    custom-slug false positive, view/engagement exclusion, non-list/empty/junk
    handling, _extract_cv byte-for-byte equivalence.
  • Updated the two fixtures that encoded the old substring behaviour
    (test_live_clients.py, test_budget_efficiency.py) to realistic
    aggregate+component arrays that prove the dedup; added a budget-efficiency
    custom-slug regression test.
  • ruff + black clean on mureo/; mypy --strict clean on changed
    modules; 1125 passed in the meta/analytics sweep (only the known
    installed-plugin env-gating noise excluded).
  • Reviewed by python-reviewer (adversarial) — it caught two further
    substring counters (_budget_efficiency.py, _insights.py) that this PR now
    also migrates.

Closes #340

https://claude.ai/code/session_011rAu94b1o1xWYZhARk1VmL

…340)

`meta_row_conversions` counted Meta conversions with a substring scan
(`"lead" in action_type or "purchase" in action_type`) that over-counted
for two reasons:

1. Alias double-count. Meta's Insights `actions` array returns one
   conversion under several overlapping action_type rows — the generic
   `lead` action_type is, by Meta's own definition, the aggregate ("all
   offsite + all On-Facebook leads") of its components
   `offsite_conversion.fb_pixel_lead` + `onsite_conversion.lead_grouped`;
   `purchase`/`omni_purchase` is the analogous roll-up. Because actions are
   fetched unfiltered, aggregate + component rows co-occur, so summing every
   `*lead*`/`*purchase*` row counted the same conversions two or three times.
2. Substring false positives. Operator-named custom conversions
   (`offsite_conversion.custom.<slug>`) carry free-text slugs that may
   contain `lead`/`purchase` and were swept in.

This deflated the absolute CPA emitted by `diagnose_performance` and the
budget-efficiency scorer (the anomaly-detector CPA-spike path is ratio-based
and largely shielded). It also diverged from the MCP-analysis path's
`_extract_cv`, which already used an exact `{lead, purchase,
complete_registration}` set — so there were multiple live counters
disagreeing on what counts as a conversion.

Fix: introduce a single canonical counter
`mureo/meta_ads/_conversion_count.py` (`count_conversions_from_actions` +
`CONVERSION_ACTION_TYPES`) that counts ONLY the deduped generic conversion
action_types — the aggregate already includes its components, so the
components are not added on top, and custom slugs no longer match. Route ALL
four live counters through it so they cannot drift again:

- `mureo/analytics/builtin/_common.py` `meta_row_conversions` (lazy import to
  keep adapter registration free of the meta_ads client weight; BYOD branch
  unchanged)
- `mureo/meta_ads/_analysis.py` `_extract_cv` (now delegates)
- `mureo/analytics/builtin/_budget_efficiency.py`
- `mureo/meta_ads/_insights.py` breakdown

The campaign-type classifier (`_classify_result_indicator`) keeps its
substring match — it is a boolean "any conversion signal?" check, not a
counter, so substring is correct there.

Note: `complete_registration` is now counted on the analytics/CPA path
(previously only `_extract_cv` counted it) — an intentional consistency
change. A known component-only / custom-event under-count is tracked as a
separate operator-canonical-event enhancement (#342).

Tests: new tests/test_meta_conversion_count.py (dedup, custom-slug false
positive, view/engagement exclusion, junk handling, _extract_cv equivalence);
updated the two fixtures that encoded the old substring behaviour to realistic
aggregate+component arrays that prove the dedup; added a budget-efficiency
custom-slug regression test.

Closes #340

Claude-Session: https://claude.ai/code/session_011rAu94b1o1xWYZhARk1VmL
@hyoshi hyoshi merged commit 7ebd4de into main Jun 26, 2026
9 checks passed
@hyoshi hyoshi deleted the fix/meta-conversion-overcount branch June 26, 2026 11:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug(meta_ads): meta_row_conversions が action_type の部分一致でコンバージョンを過大計上(別名の重複+ViewContent混入)

1 participant