Skip to content

fix(embedded): tell admins how to fix a connection that refuses embedded viewers - #45890

Open
hughhhh wants to merge 41 commits into
apache:masterfrom
hughhhh:oauth2/embedded-docs-messages
Open

hughhhh wants to merge 41 commits into
apache:masterfrom
hughhhh:oauth2/embedded-docs-messages

Conversation

@hughhhh

@hughhhh hughhhh commented Oct 30, 2026

Copy link
Copy Markdown
Member

SUMMARY

Stacked on #45068. The diff includes all of #45068; review only the top two commits (5107dbad69, 4c72545c17). Merge after #45068.

With EMBEDDED_CREDENTIAL_FALLBACK on, an embedded dashboard on a Snowflake connection that has "Impersonate logged in user" on but no OAuth2 client stops loading for outside viewers. Refusing those viewers is intended, but nobody could tell why. The viewer saw only "Data error: An error occurred while fetching the data.", and the server log said the connection's settings "may have been lost", which is rarely the cause. On Snowflake, impersonation without OAuth2 never changed the login, so these connections were usually just never set up for OAuth2. The docs also left out several limits an admin needs to know before turning the flag on.

Refusal message (5107dbad69)

  • Guest errors are redacted to the generic message unless Superset wrote the error type. Both guest refusals (no OAuth2 client, no embedded credential) now raise DATABASE_SECURITY_ACCESS_ERROR, which survives that redaction, with a fixed message that names nothing about the connection: "This chart can't be shown here because its database connection isn't set up for embedded dashboards. Please contact the dashboard's owner."
  • A server-log warning names the connection and says how to fix it. With no OAuth2 client: turn off "Impersonate logged in user", or configure an OAuth2 client and embedded guest credentials. With no credential: add embedded guest credentials.

Docs (4c72545c17): UPDATING.md, embedding.mdx and the SIP now cover:

  • which embedded dashboards stop loading when the flag is turned on, and the two ways to fix them;
  • with GLOBAL_ASYNC_QUERIES on, logged-in users' charts on a Snowflake OAuth2 connection run as the base account (this predates the flag);
  • with "Per user caching" off, logged-in users on an OAuth2 connection can be shown each other's cached results;
  • connections created in SQLAlchemy URI mode show neither the OAuth2 nor the embedded credential section, so set those values in Secure extra instead;
  • turning off "Impersonate logged in user" moves embedded viewers, and logged-in users, onto the base account.

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

Not captured. The embedded viewer's banner changes from "Data error: An error occurred while fetching the data." to the message above.

TESTING INSTRUCTIONS

  1. Enable EMBEDDED_SUPERSET and EMBEDDED_CREDENTIAL_FALLBACK.
  2. Create a Snowflake connection with a username and password, turn on "Impersonate logged in user", and configure no OAuth2 client.
  3. Embed a dashboard with a chart on it and open it with a guest token. The chart shows the new message, and the server log has a Refused an embedded viewer's query on database connection ... warning naming the connection and both fixes.
  4. Turn off "Impersonate logged in user" and reload. The chart loads under the base account.

Unit tests: pytest tests/unit_tests/models/core_test.py

ADDITIONAL INFORMATION

  • Has associated issue:
  • Required feature flags: EMBEDDED_SUPERSET, EMBEDDED_CREDENTIAL_FALLBACK
  • Changes UI
  • Includes DB Migration (follow approval process in SIP-59)
    • Migration is atomic, supports rollback & is backwards-compatible
    • Confirm DB migration upgrade and downgrade tested
    • Runtime estimates and downtime expectations provided
  • Introduces new feature or API
  • Removes existing feature or API

Created with Claude Opus 5.5 in Claude Code.

🤖 Generated with Claude Code

hughhhh and others added 23 commits October 7, 2026 22:11
An embedded viewer authenticates with a guest token rather than a Superset
account, so there is no per-user OAuth2 token to resolve for them and no route
by which they could authorize one. `start_oauth2_dance` nonetheless reads
`g.user.id` to build its state, and `GuestUser` defines no `id` -- so a guest
who hits an auth failure on any OAuth2-enabled connection (Snowflake,
Databricks, Trino, GSheets) got an `AttributeError` and an HTTP 500 instead of
the driver's own error.

`needs_oauth2` now refuses the dance for guests, which covers the nine call
sites that gate on it. `check_for_oauth2` needs the check repeated because its
`OAuth2TokenRefreshError` arm reaches the dance without consulting
`needs_oauth2`. `start_oauth2_dance` itself now raises `OAuth2Error` rather
than `AttributeError` if a future caller skips both.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Snowflake's OAuth2 support is complete on the backend -- `supports_oauth2`,
`is_oauth2_enabled`, `get_oauth2_config`, an OAuth2-aware `impersonate_user`,
and `$.oauth2_client_info.secret` already masked -- but none of it is reachable
from the UI. `SnowflakeParametersSchema` never declared `oauth2_client_info`,
and `parameters_json_schema` never attached `x-encrypted-extra`, which is the
annotation the connection dialog uses both to render the field and to move its
value into `masked_encrypted_extra`. The only way to configure Snowflake OAuth2
was to hand-edit the Secure extra JSON blob.

Adds the field and the annotation, mirroring gsheets and bigquery. The
endpoints are deliberately left without defaults: they live on the customer's
own Snowflake account, so no URI would be right for every deployment.

No frontend change is needed -- `oauth2_client_info` is already in
`FormFieldOrder` and already mapped to `OAuth2ClientField`. The typing gap that
existing tests cast around is closed, and the leak-prevention properties of
`build_sqlalchemy_uri` and `update_params_from_encrypted_extra` are pinned by
test rather than left to inspection.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Per-user database OAuth2 and embedded dashboards do not currently compose. An
embedded viewer signs in with a guest token rather than a Superset account, so
there is no per-user OAuth2 token to resolve for them -- and an external viewer
often has no identity on the analytical database at all. Embedded dashboards on
a per-user OAuth2 connection therefore cannot run queries.

This lets one connection carry a username and password used *only* when the
requesting principal is an embedded guest, while logged-in users continue to
authenticate individually via OAuth2 with no route to the stored credential.

Gated behind a new `EMBEDDED_CREDENTIAL_FALLBACK` feature flag, default False,
so nothing changes for any existing deployment. The credential lives
per-connection in the existing `encrypted_extra` column -- no migration -- and
is configured through the connection dialog; the password masks on read while
the username stays visible so an admin can see who embedded queries run as.

`Database.get_embedded_fallback_credentials` is the single gate, requiring all
of: the deployment flag, an engine opt-in (Snowflake only), a complete stored
credential, an embedded guest principal with EMBEDDED_SUPERSET on, and a
connection that actually uses OAuth2.

The fallback path deliberately skips impersonation rather than extending it.
That is load-bearing twice: an opted-in engine sets its OAuth2 authenticator
during impersonation whether or not a token exists, and `effective_username`
resolves to a guest token's own `username` claim, which whoever mints the token
controls. The credential is set on the URL rather than in `connect_args` so the
engine cache key, which has no user component, separates the two principals.

Accompanied by a SIP draft in docs/sip/ to seed the formal discussion; the
security model and rejected alternatives are set out there.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
feat(db): support Snowflake embedded fallback key pairs
…spec

Splits the embedded-only fallback credential along one line: `Database` owns
*when* a credential may be used, and the DB engine spec owns *whether* the
engine supports one, *what shape* it has, and *how* it reaches the driver.

Before this, `Database._get_sqla_engine` inlined a `URL.create` that assumed
every credential maps onto URL `username`/`password`, and the shape rule --
`auth_method in {password, keypair}`, `privatekey_body`, `privatekey_pass` --
lived as a static method on the model. Snowflake's PEM key could not be
expressed that way, so `SnowflakeEngineSpec.update_params_from_encrypted_extra`
reached back into `Database.get_embedded_fallback_credentials()` to fetch the
credential a second time. That back-reference meant the engine spec could not be
read, imported or unit-tested against a core without the feature, and an engine
whose driver wants the credential anywhere else had nowhere to say so.

Three new members on `BaseEngineSpec`:

- `apply_embedded_credentials(database, credentials, url, engine_kwargs)` --
  called instead of `impersonate_user`. Default replaces the identity on the URL
  with `URL.create` (not `URL.set`, which ignores `password=None` and would
  leave a key pair carrying the connection's own password). Snowflake overrides
  it to add `connect_args["private_key"]`, and its
  `update_params_from_encrypted_extra` is now only a guard that asks
  `embedded_fallback_applies()`.
- `validate_embedded_credentials(credentials)` -- the canonical shape rule,
  moved out of the model to `superset.databases.utils` so the marshmallow
  validators can reach it without an engine in hand, and so `databases/utils.py`
  no longer imports `Database`.
- `get_embedded_credential_cache_key(credentials)` -- what `cache_keys.py`
  hardcoded.

`parameters_json_schema` was a near-verbatim copy on four engine specs; the
branch had made Snowflake's copy the only place that registered
`encrypted_field_properties` and gated the credential field on
`EMBEDDED_CREDENTIAL_FALLBACK`, so opting a second engine in meant copying all
of it. It is now `build_parameters_json_schema()` in `base.py`, called by
`BasicParametersMixin` and by a four-line delegation on Snowflake, which cannot
mix the mixin in because its `build_sqlalchemy_uri` takes a narrower parameters
type. `encrypted_field_properties` moved to `superset/databases/types.py`, which
exists so it can be imported outside an app context, and is re-exported from
`schemas.py`; this also drops a heavy import edge from `snowflake.py`.
The `encrypted_extra` key is now `constants.EMBEDDED_CREDENTIALS_KEY` rather
than a literal repeated in six places.

No behaviour change: the five-condition gate, the refusal when a guest has no
applicable credential, and the engine-cache separation are untouched. New tests
pin the extension points -- a spec subclass changing how the credential is
applied, one changing what counts as a credential, one changing the cache key,
and a second engine opting in with a schema field and a class attribute alone --
plus a regression test that the Snowflake spec no longer fetches the credential
in `update_params_from_encrypted_extra`.

This commit also carries uncommitted working-tree work that predates it and was
not authored in this session: the save-time completeness validation
(`databases/utils.py`, `databases/schemas.py`, `commands/database/validate.py`),
the `embedded_fallback_applies()` split and refusal error in `models/core.py`,
the data-cache key separation in `utils/cache_keys.py`, the key-pair UI in
`EmbeddedCredentialsField.tsx`, and the accompanying docs, tests and catalog.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Brings the branch up to b411454, 170 commits on from the base it was cut at.
Five files conflicted; all five were mechanical.

- `tests/unit_tests/db_engine_specs/test_base.py` and `test_snowflake.py`: both
  sides append at EOF. Kept both, including master's
  `test_expand_nested_columns_default_returns_input` and
  `test_extended_aggregation_func_median_uses_native_snowflake_syntax`.
- `superset/db_engine_specs/snowflake.py`: master added `import sqlalchemy as sa`
  (used by its new `_extended_aggregations`) immediately above the `apispec`
  imports this branch stopped using when it dropped the file's
  `parameters_json_schema` override. Kept `sa`, dropped the apispec imports.
- `UPDATING.md`: both sides insert at the top of `## Next`. Kept both blocks.
- `superset/translations/messages.pot`: took master's `% calculation` format-flag
  correction and its four new `Alerts & Reports` entries, keeping this branch's
  nine new msgids. The resolved template differs from master's only by the
  creation stamp and those nine strings -- no reordering, no dropped entries.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…client

Dashboard and chart bundles do not carry encrypted_extra, so a connection
promoted by export/import keeps "Impersonate logged in user" but loses its
OAuth2 client and embedded viewer credential. The fallback gate then no
longer applies, and guests go through impersonation on whatever identity
the engine derives from the URL -- for Snowflake, the base account, with no
error or warning.

An embedded guest on an opted-in, impersonating connection with no usable
OAuth2 client (global or stored) is now refused with a message telling the
admin the OAuth2 and embedded viewer settings are missing, for example after
an import. The cache key gets the same `embedded_credentials_missing`
marker so a warm logged-in entry cannot bypass the refusal. Logged-in users,
service-account connections, and deployments without the flag or engine
opt-in are unaffected.

Also pins that the base update_params_from_encrypted_extra drops the
embedded credential from engine kwargs, and stubs the fallback lookups on
the query-context and serialization tests' mocked databases, which
otherwise serialize a MagicMock into the cache key.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Fixes found validating the feature against QA's secret scenarios
(sc-124914) and the epic's bugs:

- Masking: every engine masks the embedded credential's password, private
  key and passphrase, as it already does the OAuth2 client secret, so an
  engine that overrides encrypted_extra_sensitive_fields can't expose it.
- Import over an existing connection never stores the mask as the secret.
  The import schema asks for any masked value the existing connection has
  no stored value for (e.g. a copy promoted by a dashboard bundle, or a
  repointed connection), and import_database refuses to store one that is
  still masked (QA OEF-SECRET-04).
- Empty secrets are no longer masked, so a removed credential round-trips
  and later saves aren't rejected as half-filled (QA OEF-SECRET-07).
- A non-blank credential is refused unless the connection has an OAuth2
  client (stored or deployment-wide) and impersonation on; elsewhere it
  saved and silently did nothing (sc-126624).
- A create no longer fails when only the post-create permission sync needs
  the admin's OAuth2 token, which reported 422 after the row was committed
  (sc-126623).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…n import

Dashboard, chart and dataset bundles wrote each database YAML without
encrypted_extra, so promoting a dashboard dropped the connection's OAuth2
client and embedded viewer credential with no warning, and the import had
no way to put them back.

- Export: the dataset exporter (which writes the database YAML for chart,
  dashboard and dataset bundles) carries Secure Extra like the database
  export does -- masked when it holds secrets.
- Import API: the dashboard, chart, dataset and saved-query import endpoints
  accept encrypted_extra_secrets, as the database import does.
- Import schema: those commands reuse an existing database rather than
  overwrite it, so ImportV1DatabaseSchema(overwrites_existing=False) skips
  asking for secrets that would never be applied; a new database still asks
  for every masked secret, and an overwrite that needs one is refused by
  import_database.
- Frontend: the dashboard, chart, dataset and saved-query lists pass the
  import modal its encrypted-extra field state, so the prompts render, and
  the import help text no longer says Secure Extra is missing from export
  files.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…role

The connection form always writes a role into the URI, and the embedded
login kept it, so a least-privilege fallback user -- or a programmatic
access token restricted to its own role -- was refused by Snowflake
(390186) and every embedded chart failed. Drop the connection's role on
the embedded login so Snowflake uses the fallback user's default role.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A guest on a per-user OAuth2 connection with no embedded credential, or
on one that lost its OAuth2 client, was refused with a 500. Before the
fallback, the same viewer got a 400 from the failed OAuth2 query. This
is a configuration problem an administrator fixes, not a server fault,
so return a 400 and keep it out of server-error monitoring.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ogin

A connection URI naming authenticator=SNOWFLAKE_JWT kept it on the guest
login, so the driver expected a key pair from a password or token guest
and failed before contacting Snowflake. Drop it as role already is; the
driver picks key-pair sign-in itself when the embedded credential
supplies private_key.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nnect

- A database error raised while a chart query is being built -- a
  virtual dataset's default-schema lookup connects -- escaped as an HTML
  500. Report it as a failed query (a 400) like any other.
- Log when the database rejects a guest's fallback credential at connect
  time, naming the connection and the exception class only.
- Never pass oauth2_client_info or embedded_credentials to the driver.
  Engines that copy all of encrypted_extra into the engine kwargs failed
  every connect after a switch away from Snowflake, a 500 on that save.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
QA review of the embedded guest credential dialog:

- Keep the section visible but disabled, with the reason, unless the
  connection has an OAuth2 client and impersonation is on -- the only
  case where the backend accepts a credential.
- Put the help text under the section title, use the base password
  input (with show/hide) for the secrets, add placeholders, turn off
  autofill, and show a Configured tag when a credential is stored.
- Mirror edits into Secure Extra so its JSON view stays current, and add
  an explicit Remove that deletes the block instead of leaving it empty.
- The impersonation tooltip mentions the Snowflake requirement.
- Docs: OAuth2 client information is on the Basic tab, setup is numbered
  steps, and a requirements and limitations list covers flags, OAuth2
  and impersonation, the default role, warehouse access, supported
  credential types, async queries and removal.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The "OAuth2 connections and guests" note said such a viewer gets the
database driver's own authentication error, contradicting the
EMBEDDED_CREDENTIAL_FALLBACK section. With the flag on, Superset refuses
the query before connecting; with it off, the driver's error is redacted
for guests. Either way the viewer sees the generic data error and the
reason is in the server logs.

Fixes sc-127527

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…on path

Chart data on a physical table connects through get_raw_connection(). On
SQLAlchemy 2, a failed raw_connection() doesn't dispatch handle_error, so
the "Embedded fallback credential rejected" line never fired for the most
common chart; only the virtual-dataset path (engine.connect()) logged it.

get_raw_connection now logs the rejection for engines that carry the
fallback credential, through one helper shared with the engine event that
logs each failure once and names the driver's exception class either way.

Fixes sc-127710

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rry only two blocks

From review of the 6.0 backport:

- The guest login dropped only role and authenticator, so a user,
  password, key or token in the connection's URI query or connect_args
  still reached the Snowflake driver and won over the embedded account.
  Drop every sign-in argument from both.
- Dashboard, chart and dataset bundles wrote the connection's whole
  Secure Extra, raw whenever masking left it unchanged, exposing settings
  an engine doesn't register as sensitive (a Postgres client key) to
  anyone who can export a dashboard. Carry only the masked
  oauth2_client_info and embedded_credentials blocks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A guest refused on a connection with "Impersonate logged in user" on but no
OAuth2 client saw only "Data error: An error occurred while fetching the
data.", and the server log said the settings "may have been lost". That
connection was usually just never set up for OAuth2 (sc-127218).

- Both guest refusals raise a fixed message under
  DATABASE_SECURITY_ACCESS_ERROR, which survives guest error redaction and
  names nothing about the connection.
- A warning naming the connection says how to fix it: turn off
  "Impersonate logged in user", or configure an OAuth2 client and embedded
  guest credentials (missing client); add embedded guest credentials
  (missing credential).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
UPDATING and the embedding docs say, before an admin turns on
EMBEDDED_CREDENTIAL_FALLBACK (sc-127218, sc-130323):

- embedded dashboards on Snowflake connections that impersonate users with
  no OAuth2 client stop serving outside viewers, and the two ways to fix it;
- with GLOBAL_ASYNC_QUERIES on, logged-in users' charts run as the base
  account (sc-130267);
- without per-user caching, logged-in users can see each other's results;
- SQLAlchemy URI mode connections show neither section (use Secure extra);
- turning off impersonation moves embedded viewers onto the base account.

The SIP describes the new refusal case and what the viewer sees.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
hughhhh and others added 6 commits October 30, 2026 08:50
…uth2-fallback

Conflicts: UPDATING.md (kept both Next entries), messages.pot (master's
template plus this branch's msgids, re-normalized with msgcat).

Semantic conflict: master's apache#45579 moved the import routes to
parse_json_form_field and dropped their json import, so this branch's
json.loads(encrypted_extra_secrets) in the chart, dashboard, dataset and
saved-query import routes raised NameError. Those reads use
parse_json_form_field too, so a malformed value is a 400 like the other
form fields.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The import routes' encrypted_extra_secrets form field was missing from
openapi.json, and messages.pot had drifted from the branch's later string
edits; both fail their CI drift checks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
EncryptedExtraInputChange stores embedded_credentials as an object, but
the action shared a payload type whose value is a string, so tsc
rejected the reducer test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An engine that masks Secure Extra with `$.*` (Databricks, the base spec)
turns `oauth2_client_info` into a bare mask string. Carrying it made a
fresh import ask for `$.oauth2_client_info` and store the answer as a
string, so every query then failed with "Invalid input type". Carry a
block only when it is still a dict, which keeps master's behavior for
those engines.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Earlier releases stored the mask when an overwrite import had nothing to
reveal. `unrevealable_masked_fields` counted that stored mask as a value,
so the schema asked for nothing and `import_database`'s backstop failed
the import with a 500. A stored mask now counts as missing.

The same 500 happened when the import moves the connection:
`import_database` never reveals stored secrets onto a new destination,
but the schema still treated them as revealable. The schema now shares
the importer's identity check (moved to `commands/database/utils.py`)
and asks for every masked value in that case.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ecrets

With `overwrite` and `overwrite_all` set, the dashboard importer overwrites
related databases, but validated them with a schema that assumed reuse and
skipped the missing-secret check. A secret the stored connection could not
reveal then reached `import_database`'s backstop as a 500 instead of the
structured prompt. The database schema now follows the effective overwrite
setting.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comment thread superset/models/core.py Resolved
hughhhh and others added 2 commits October 30, 2026 11:55
…uth2-fallback

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts:
#	superset/translations/messages.pot
@github-actions github-actions Bot added the requires:rebase Requires rebasing on top of current master label Oct 30, 2026
@codecov

codecov Bot commented Oct 31, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 95.76613% with 21 lines in your changes missing coverage. Please review.
✅ Project coverage is 84.96%. Comparing base (15e4aca) to head (32d5a84).
⚠️ Report is 3 commits behind head on master.

Files with missing lines Patch % Lines
superset/commands/database/validate.py 80.95% 3 Missing and 1 partial ⚠️
superset/commands/dataset/export.py 73.33% 3 Missing and 1 partial ⚠️
superset/databases/utils.py 90.24% 2 Missing and 2 partials ⚠️
superset/db_engine_specs/snowflake.py 90.62% 2 Missing and 1 partial ⚠️
...rset-frontend/src/components/ImportModal/index.tsx 66.66% 2 Missing ⚠️
superset/db_engine_specs/base.py 95.45% 1 Missing and 1 partial ⚠️
superset/databases/types.py 85.71% 0 Missing and 1 partial ⚠️
superset/models/core.py 98.76% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master   #45890      +/-   ##
==========================================
+ Coverage   84.91%   84.96%   +0.04%     
==========================================
  Files        3075     3077       +2     
  Lines      207804   208214     +410     
  Branches    48986    49103     +117     
==========================================
+ Hits       176452   176901     +449     
+ Misses      27992    27947      -45     
- Partials     3360     3366       +6     
Flag Coverage Δ
hive 34.22% <25.58%> (-0.03%) ⬇️
javascript 81.27% <98.71%> (+0.03%) ⬆️
mysql 52.22% <38.52%> (-0.05%) ⬇️
postgres 52.23% <39.11%> (-0.05%) ⬇️
presto 35.93% <25.88%> (-0.03%) ⬇️
python 87.92% <94.41%> (+0.05%) ⬆️
sqlite 51.98% <38.52%> (-0.05%) ⬇️
unit 82.80% <94.41%> (+0.11%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions github-actions Bot removed the requires:rebase Requires rebasing on top of current master label Oct 31, 2026
hughhhh and others added 2 commits November 2, 2026 10:56
- Treat a whitespace-only private key as missing.
- Refuse a masked embedded secret the connection has nothing stored for, which
  would otherwise be saved as the credential.
- Check imported masks with the incoming URI's engine in the import schema, so
  an overwrite that changes engine is not judged by the old engine's paths.
- Count any stored value except None, an empty string or a mask as revealable.
- Say that async guests get the generic error, not the fixed refusal message.
- Mention key pairs in the flag text.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comment thread superset/databases/schemas.py Resolved
Comment thread superset/commands/dataset/export.py Resolved
Comment thread superset/models/core.py Resolved
Comment thread superset/databases/utils.py Resolved
- Test connection no longer refuses the masked embedded secret the dialog
  sends back for a saved connection. It saves nothing, and the command unmasks.
- Importing a dashboard, chart or dataset bundle keeps Secure Extra settings the
  bundle does not carry. A database export still replaces the stored value.
- A guest on a connection with an incomplete OAuth2 client gets the refusal
  rather than the ValidationError from get_oauth2_config().
- Saving refuses a credential sent as a JSON string, which query time rejects.
- Test the chart data database error path through execute() as well as run().

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comment thread docs/docs/using-superset/embedding.mdx Resolved
- Partial bundles now say they are partial. The importer used to infer it
  from the keys present, so a database export holding only oauth2_client_info
  kept a revoked embedded credential.
- A Secure Extra edit retires the typed embedded credential, which onSave
  would otherwise copy back over the edit.
- Import asks for private keys in a multiline input, so PEM newlines survive.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comment thread superset/commands/dataset/export.py Resolved
Comment thread docs/docs/using-superset/embedding.mdx Resolved
- Export a bundle and overwrite-import it, so dropping the marker fails a test.
- The dialog refuses a save with impersonation off while a credential is set,
  and a PUT sending only impersonate_user is accepted. The SIP still described
  an inline warning.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comment thread UPDATING.md Resolved
Comment thread superset/utils/cache_keys.py Resolved
hughhhh and others added 3 commits November 2, 2026 16:34
- Replace the partial bundle marker with an argument from the dashboard importer,
  the only import that overwrites from a partial bundle. The marker made an
  earlier release reject the file.
- Key column value caching on the embedded and per-user identity, so a guest and
  a logged-in user do not share native filter values.
- Clear the typed OAuth2 client on a Secure Extra edit, as well as the credential.
- Use a multiline input for private keys in database file imports too.
- Note in UPDATING what exports now carry, whatever the flag.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts:
#	superset/commands/database/validate.py
#	tests/unit_tests/commands/databases/validate_test.py
#	tests/unit_tests/databases/api_test.py
#	tests/unit_tests/databases/schema_tests.py
Other tests leave an id on the shared import fixtures, which the dashboard
schema rejects, so the test failed in some orderings of the full suite.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
self.schemas = {
**self.schemas,
"databases/": ImportV1DatabaseSchema(
overwrites_existing=self.overwrite and self.overwrite_all

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With both flags set, this makes the database schema ask for any OAuth2 or embedded secret the stored connection can't reveal. But import_database() only overwrites when the caller also has can_write on Database (superset/commands/database/importers/v1/utils.py); without it, it returns the existing connection untouched. A role that can import dashboards but not edit connections, importing with overwrite=true&overwrite_all=true onto an existing database whose Secure Extra lacks the exported blocks, now fails validation asking for secrets that would never be applied, where that import used to succeed. Should the schema flag also account for the Database write permission, so validation and execution share one overwrite decision?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, same as my note on #45068 (#45068 (comment)), and it still reproduces on this head. As Alpha, an "overwrite all" dashboard import asks for a secret that then gets dropped, and /api/v1/assets/import/ 500s with "unknown reason"; both work on master. Checking can_write on Database next to overwrites_existing in validate_masked_encrypted_extra (schemas.py:1280) should cover both paths.

# A separate field prevents collisions with a logged-in username that equals
# the engine-provided guest key.
if embedded_credentials := database.get_embedded_fallback_credentials():
cache_dict["embedded_credentials_key"] = (

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This partitions the chart's own connection, but a chart-backed annotation layer reads through another dataset's connection, and _annotation_cache_context in query_context_processor.py binds only that source's RLS and the user id. Guests share one user id, so after a guest warms chart A, an admin who swaps the annotation source's embedded credential for a less-privileged one (or clears it) still gets A's cached annotation_data replayed, without reaching the refusal. Should the annotation source connection's embedded discriminator go into the parent key too?


## Migration plan and compatibility

Nothing changes for any existing deployment. The flag defaults to `False`; with it off,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This says nothing changes for an existing deployment and that export/import round-trips unchanged with the flag off. But superset/commands/dataset/export.py now writes the carried Secure Extra blocks whatever the flag, and a destination on an earlier release replaces its whole Secure Extra on an overwrite import, which UPDATING.md already warns about. An operator who reads only this section could upgrade the export source first and lose the destination's unrelated Secure Extra. Could this section match UPDATING.md? Also, the interface list above names validated_embedded_credentials() as a new Database method, but it is a module-level function in superset/databases/utils.py.

required
visibilityToggle
visibilityToggle={!isPrivateKeyField(field.path)}
renderAsTextArea={isPrivateKeyField(field.path)}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing in DatabaseModal/index.test.tsx exercises this renderer; the new textarea assertions cover ImportModal. If renderAsTextArea were dropped here, every existing test would still pass while a pasted multi-line key is flattened by a single-line input and the stored key can't be loaded. Could you add a test that imports a database asking for $.embedded_credentials.privatekey_body and its passphrase, pastes a multi-line PEM, and asserts a textarea for the key, a password input for the passphrase, and unchanged newlines in the submitted encrypted_extra_secrets?


if credentials.get("auth_method") == "keypair":
engine_kwargs.setdefault("connect_args", {})["private_key"] = (
cls._load_private_key(credentials)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A key pair with a wrong passphrase or a malformed PEM raises a plain ValueError from _load_private_key here, while the engine URL is still being built. That is before create_engine's error mapping and before _log_rejected_embedded_credential attaches its listener, and save-time validation only checks that the fields are present. The embedded viewer gets an unmapped server error instead of the refusal message, and the admin gets a bare traceback instead of the rejected-credential warning this change adds. Should a load failure here be turned into the same refusal and warning as a rejected sign-in?

@geido geido left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Went through this together with #45870 and #45068, since they land as one. Nothing in this PR's own changes worries me; the open bits are all in the shared import code:

  • can_write: +1'd @sadpandajoe's thread here, it still reproduces on this head.
  • Stored null secrets: +1'd @sadpandajoe's json.py:395 note on #45068 (#45068 (comment)), same story.
  • One new note inline: "overwrite all" can pair one instance's OAuth2 client id with another's secret.

For landing, since this and #45870 both carry all of #45068 and we squash:

  1. Fix the UPDATING.md conflict (keep both entries) and push; that re-runs lint-frontend, which failed on the usual tsc build race.
  2. Land this one first under a feat title that names #45068, then close #45068.
  3. Rebuild #45870 on master. What's left is the import fixes (9 files, no core.py).
  4. Rebuild #45889 and #45869 on master too. QA's d74a648 build had their early commits, so what QA signed off on isn't complete without them. When rebuilding #45889, routing its refusal through _refuse_embedded_guest would give viewers the proper message.

Happy to take another look once the threads are settled.

# embedded credential -- would otherwise be stored as the mask itself,
# so ask for those like a new DB. Nothing is revealed onto a
# connection the import moves, so then ask for every masked value.
masked_fields = json.unrevealable_masked_fields(

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Something I don't think anyone has flagged yet: if staging and prod share a connection (same UUID and URI) but each has its own OAuth2 client, an "overwrite all" import of a staging dashboard leaves prod with staging's client id and prod's secret, with no prompt, so OAuth2 breaks for everyone on that connection. Embedded credentials mix the same way (staging's username, prod's password). Master's database import already does this, but dashboard bundles never carried these blocks before. Could the secret count as unrevealable when the bundle's oauth2_client_info.id differs from the stored one, and the embedded password when the username differs?

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api Related to the REST API doc Namespace | Anything related to documentation i18n Namespace | Anything related to localization size/XXL

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants