Repository navigation
fix(semantic-layers): honor CACHE_QUERY_BY_USER for semantic views - #45900
Open
surawut-jirasaktavee wants to merge 1 commit into
Open
surawut-jirasaktavee wants to merge 1 commit into
surawut-jirasaktavee wants to merge 1 commit into
Conversation
Chart-data result-cache keys for semantic views were identical for every user even with CACHE_QUERY_BY_USER on: QueryObject.cache_key adds the per-user key from the datasource's database, which semantic views lack, and the resulting AttributeError is swallowed. The ordinary result cache is read before the semantic containment cache, so one user's cached result could be served to another when provider results depend on the user. When the flag is on, SemanticView.get_extra_cache_keys partitions keys by the server-authenticated principal ID, as datasets do by username. The execution-context identity digest is not used because it rotates with SECRET_KEY, which must not change semantic result identity. Guests are not partitioned, since a guest principal does not tell guest tokens apart and semantic views refuse guest row rules. Keys are unchanged when the flag is off. Fixes apache#45877 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 of 3 tasks
✅ Deploy Preview for superset-docs-preview ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #45900 +/- ##
==========================================
- Coverage 84.73% 84.72% -0.02%
==========================================
Files 3069 3069
Lines 205767 205776 +9
Branches 48388 48390 +2
==========================================
- Hits 174350 174335 -15
- Misses 28123 28144 +21
- Partials 3294 3297 +3
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
2 of 6 tasks
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
SUMMARY
With
CACHE_QUERY_BY_USERon, chart-data results for semantic views were still cached under one key for all users.QueryObject.cache_keyadds the per-user key throughadd_impersonation_cache_key_if_needed(self.datasource.database, ...). A semantic view has nodatabase, so theAttributeErroris swallowed and no key is added.SemanticView.get_extra_cache_keysadded nothing per user either. A provider whose results depend on the user (for example, the upstream engine applies per-user row filtering) could therefore return one user's cached result to another. The result cache is read before the semantic containment cache, so the containment cache'sEXECUTION_CONTEXTscope does not help.This change makes
SemanticView.get_extra_cache_keysappend("semantic-user", principal_id)whenCACHE_QUERY_BY_USERis on.principal_idis the server-authenticated user ID fromcurrent_host_execution_context(), the same per-user material datasets get from their username. It only appears inside the hashed cache key.Design decisions:
execution_context_test.pystill matches.identity_digest.semantic-layer-cache-clear.mdsaysSECRET_KEYmust not affect semantic cache identity, andtest_metadata_aliases_share_result_cachechecks that a worker-cached result is still found afterSECRET_KEYchanges. The digest is an HMAC keyed withSECRET_KEY. With it, that test fails with the flag on (HTTP 202 instead of 200), and during a rollingSECRET_KEYrotation web and worker processes would compute different keys, so async chart queries would keep missing worker results. The principal ID is stable across rotations and processes.CACHE_QUERY_BY_USER, notCACHE_IMPERSONATION. For datasets,CACHE_IMPERSONATIONapplies only when the database hasimpersonate_userenabled. Semantic views have no equivalent setting, so treating that flag as unconditional would change what it means.CACHE_QUERY_BY_USERis the operator's general per-user caching switch.SemanticView.get_extra_cache_keys, not in a semantic-view branch inQueryObject.cache_key. That keeps the semantic-layer identity logic in the semantic-layer model, next to the metadata generation and producer-version keys already there.SemanticCredentialsand execution methods are untouched.Relation to #45895. #45895 targets the same issue with
identity_digestand catches any exception from the context lookup. This PR differs in two ways: the key does not depend onSECRET_KEY(see above), and a failed lookup raises instead of silently dropping the user partition, so it fails closed rather than sharing results across users.Alternative considered:
identity_digest. The SDK's opaque partition token (HMAC over installation namespace, principal, effective role IDs and guest flag) would also separate users, and would additionally split a user's entries when their roles change. Its cost is theSECRET_KEYdependency above: every rotation resets all per-user semantic result caches, web and worker keys disagree during a rolling rotation, and the existing alias test would need changing. Happy to switch if maintainers prefer it.Out of scope: column-values cache.
_column_values_cache_keyinsuperset/datasource/api.py(filter dropdown values, also used by MCP filter validation) does not honorCACHE_QUERY_BY_USER,CACHE_IMPERSONATIONorper_user_cachingfor datasets or semantic views. That gap predates this PR and is tracked in #45898; I'll fix it in a separate PR covering both.Docs:
docs/developer_docs/semantic-layer-cache-clear.mdgains a paragraph describing the per-user result key.BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
N/A (backend only).
Before: users A and B running the same semantic-view chart query get the same
cache_key, and B gets A's rows withis_cached: true.After (flag on): A and B get different
cache_keys, and each gets their own result.TESTING INSTRUCTIONS
Unit tests:
With the flag forced on for the whole run (
SUPERSET_FEATURE_CACHE_QUERY_BY_USER=true), the only new failures are tests asserting exact semantic cache keys or key lists, which now include the user entry; theSECRET_KEYalias tests pass.New tests in
tests/unit_tests/semantic_layers/execution_context_test.py:test_chart_keys_honor_cache_query_by_user: with the flag on, two users get different chart-data keys and the same user gets a stable key. With the flag off, both users get the recorded pre-change key.test_user_chart_keys_survive_secret_rotation_and_background_tasks: with the flag on, a user's key is the same in a web request and a background task, and afterSECRET_KEYchanges.test_chart_keys_do_not_partition_guests: the flag does not change a guest's key, and a logged-in user's and an anonymous user's keys differ from it.Manual check:
SEMANTIC_LAYERSandCACHE_QUERY_BY_USER, and configure a cache (for example, Redis).cache_keyvalues in the responses differ, and the second user's result is not served from the first user's cache.CACHE_QUERY_BY_USERoff: both users get the samecache_keyas before the change.ADDITIONAL INFORMATION
CACHE_QUERY_BY_USER(behavior only changes when it is on)🤖 Generated with Claude Code