Skip to content

fix(semantic-layers): honor CACHE_QUERY_BY_USER for semantic views - #45900

Open
surawut-jirasaktavee wants to merge 1 commit into
apache:masterfrom
surawut-jirasaktavee:fix/semantic-view-cache-per-user
Open

surawut-jirasaktavee wants to merge 1 commit into
apache:masterfrom
surawut-jirasaktavee:fix/semantic-view-cache-per-user

Conversation

@surawut-jirasaktavee

Copy link
Copy Markdown

SUMMARY

With CACHE_QUERY_BY_USER on, chart-data results for semantic views were still cached under one key for all users. QueryObject.cache_key adds the per-user key through add_impersonation_cache_key_if_needed(self.datasource.database, ...). A semantic view has no database, so the AttributeError is swallowed and no key is added. SemanticView.get_extra_cache_keys added nothing per user either. A provider whose results depend on the user (for example, the upstream engine applies per-user row filtering) could therefore return one user's cached result to another. The result cache is read before the semantic containment cache, so the containment cache's EXECUTION_CONTEXT scope does not help.

This change makes SemanticView.get_extra_cache_keys append ("semantic-user", principal_id) when CACHE_QUERY_BY_USER is on. principal_id is the server-authenticated user ID from current_host_execution_context(), the same per-user material datasets get from their username. It only appears inside the hashed cache key.

Design decisions:

  • Opt-in, no invalidation. With the flag off, the key material is unchanged. The recorded golden chart key in execution_context_test.py still matches.
  • Principal ID, not identity_digest. semantic-layer-cache-clear.md says SECRET_KEY must not affect semantic cache identity, and test_metadata_aliases_share_result_cache checks that a worker-cached result is still found after SECRET_KEY changes. The digest is an HMAC keyed with SECRET_KEY. With it, that test fails with the flag on (HTTP 202 instead of 200), and during a rolling SECRET_KEY rotation web and worker processes would compute different keys, so async chart queries would keep missing worker results. The principal ID is stable across rotations and processes.
  • Guests are not partitioned. A guest principal does not tell guest tokens apart (see the semantic-layer "Execution context" docs). Semantic views refuse guest row rules, and connection mode runs every guest under the same upstream account, so guests keep sharing results. Their key still differs from every logged-in user's key and from anonymous users' key.
  • Only CACHE_QUERY_BY_USER, not CACHE_IMPERSONATION. For datasets, CACHE_IMPERSONATION applies only when the database has impersonate_user enabled. Semantic views have no equivalent setting, so treating that flag as unconditional would change what it means. CACHE_QUERY_BY_USER is the operator's general per-user caching switch.
  • Placement. The fix goes in SemanticView.get_extra_cache_keys, not in a semantic-view branch in QueryObject.cache_key. That keeps the semantic-layer identity logic in the semantic-layer model, next to the metadata generation and producer-version keys already there.
  • No SDK contract change. Providers, SemanticCredentials and execution methods are untouched.

Relation to #45895. #45895 targets the same issue with identity_digest and catches any exception from the context lookup. This PR differs in two ways: the key does not depend on SECRET_KEY (see above), and a failed lookup raises instead of silently dropping the user partition, so it fails closed rather than sharing results across users.

Alternative considered: identity_digest. The SDK's opaque partition token (HMAC over installation namespace, principal, effective role IDs and guest flag) would also separate users, and would additionally split a user's entries when their roles change. Its cost is the SECRET_KEY dependency above: every rotation resets all per-user semantic result caches, web and worker keys disagree during a rolling rotation, and the existing alias test would need changing. Happy to switch if maintainers prefer it.

Out of scope: column-values cache. _column_values_cache_key in superset/datasource/api.py (filter dropdown values, also used by MCP filter validation) does not honor CACHE_QUERY_BY_USER, CACHE_IMPERSONATION or per_user_caching for datasets or semantic views. That gap predates this PR and is tracked in #45898; I'll fix it in a separate PR covering both.

Docs: docs/developer_docs/semantic-layer-cache-clear.md gains a paragraph describing the per-user result key.

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

N/A (backend only).

Before: users A and B running the same semantic-view chart query get the same cache_key, and B gets A's rows with is_cached: true.
After (flag on): A and B get different cache_keys, and each gets their own result.

TESTING INSTRUCTIONS

Unit tests:

pytest tests/unit_tests/semantic_layers/execution_context_test.py tests/unit_tests/semantic_layers/models_test.py tests/unit_tests/queries/query_object_test.py

With the flag forced on for the whole run (SUPERSET_FEATURE_CACHE_QUERY_BY_USER=true), the only new failures are tests asserting exact semantic cache keys or key lists, which now include the user entry; the SECRET_KEY alias tests pass.

New tests in tests/unit_tests/semantic_layers/execution_context_test.py:

  • test_chart_keys_honor_cache_query_by_user: with the flag on, two users get different chart-data keys and the same user gets a stable key. With the flag off, both users get the recorded pre-change key.
  • test_user_chart_keys_survive_secret_rotation_and_background_tasks: with the flag on, a user's key is the same in a web request and a background task, and after SECRET_KEY changes.
  • test_chart_keys_do_not_partition_guests: the flag does not change a guest's key, and a logged-in user's and an anonymous user's keys differ from it.

Manual check:

  1. Enable SEMANTIC_LAYERS and CACHE_QUERY_BY_USER, and configure a cache (for example, Redis).
  2. Add a semantic layer whose results depend on the querying user.
  3. Run the same chart query as two different users. The cache_key values in the responses differ, and the second user's result is not served from the first user's cache.
  4. Turn CACHE_QUERY_BY_USER off: both users get the same cache_key as before the change.

ADDITIONAL INFORMATION

  • Has associated issue: Fixes Semantic views ignore CACHE_QUERY_BY_USER: chart-data results are shared across users #45877
  • Required feature flags: CACHE_QUERY_BY_USER (behavior only changes when it is on)
  • Changes UI
  • Includes DB Migration (follow approval process in SIP-59)
    • Migration is atomic, supports rollback & is backwards-compatible
    • Confirm DB migration upgrade and downgrade tested
    • Runtime estimates and downtime expectations provided
  • Introduces new feature or API
  • Removes existing feature or API

🤖 Generated with Claude Code

Chart-data result-cache keys for semantic views were identical for every
user even with CACHE_QUERY_BY_USER on: QueryObject.cache_key adds the
per-user key from the datasource's database, which semantic views lack,
and the resulting AttributeError is swallowed. The ordinary result cache
is read before the semantic containment cache, so one user's cached
result could be served to another when provider results depend on the
user.

When the flag is on, SemanticView.get_extra_cache_keys partitions keys by
the server-authenticated principal ID, as datasets do by username. The
execution-context identity digest is not used because it rotates with
SECRET_KEY, which must not change semantic result identity. Guests are
not partitioned, since a guest principal does not tell guest tokens apart
and semantic views refuse guest row rules. Keys are unchanged when the
flag is off.

Fixes apache#45877

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@netlify

netlify Bot commented Oct 30, 2026

Copy link
Copy Markdown

✅ Deploy Preview for superset-docs-preview ready!

Name Link
🔨 Latest commit 8f0dae8
🔍 Latest deploy log https://app.netlify.com/projects/superset-docs-preview/deploys/6ae4dc03743751437d67378e
😎 Deploy Preview https://deploy-preview-45900--superset-docs-preview.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.
🤖 Make changes Run an agent on this branch

To edit notification comments on pull requests, go to your Netlify project configuration.

@github-actions github-actions Bot added the doc Namespace | Anything related to documentation label Oct 30, 2026
@codecov

codecov Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 84.72%. Comparing base (e33b7d9) to head (8f0dae8).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master   #45900      +/-   ##
==========================================
- Coverage   84.73%   84.72%   -0.02%     
==========================================
  Files        3069     3069              
  Lines      205767   205776       +9     
  Branches    48388    48390       +2     
==========================================
- Hits       174350   174335      -15     
- Misses      28123    28144      +21     
- Partials     3294     3297       +3     
Flag Coverage Δ
hive 34.50% <40.00%> (-0.01%) ⬇️
mysql 52.60% <40.00%> (-0.01%) ⬇️
postgres 52.60% <40.00%> (-0.01%) ⬇️
presto 36.23% <40.00%> (-0.01%) ⬇️
python 87.74% <100.00%> (-0.02%) ⬇️
sqlite ?
unit 82.52% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

doc Namespace | Anything related to documentation size/L

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Semantic views ignore CACHE_QUERY_BY_USER: chart-data results are shared across users

1 participant