[DSIP-105][Feature][Parent] Sensitive Variable Support — Masking in API/UI & Encrypted Storage

Search before asking

I had searched in the DSIP and found no similar DSIP.

Subtasks

Each subtask is one issue and one PR (same pattern as #6407). Design stays on this parent.

API/UI masking — Property.sensitive, mask ****** on read, keep-original on write — [DSIP-105][Feature][API] Add Property.sensitive and mask values in API/UI #18586 / PR [DSIP-105][API][UI] Add Property.sensitive with API/UI masking #18585
Definition encryption — reuse PasswordUtils (no datasource CRUD change) — [DSIP-105][Feature][API] Encrypt sensitive definition params with PasswordUtils #18587
Worker log masking — dynamic stdout redaction + cleanup — [DSIP-105][Feature][Worker] Mask sensitive parameter values in task logs #18588

Motivation

Currently, DolphinScheduler variables do not distinguish between sensitive and non-sensitive information. When users store secrets such as passwords or API keys as workflow global parameters or task local parameters, these values may be exposed in plaintext via:

UI pages that display parameter definitions
API responses (e.g., viewVariables, getWorkflowDefinition, queryWorkflowDefinitionByCode, etc.)
Database definition JSON (global_params / task_params value fields)

This change adds a variable-level sensitive flag on the existing Property model: mask values in API/UI, and apply datasource-level at-rest protection for sensitive definition values before persist (reuse PasswordUtils).

Delivery is split into three subtasks (see Subtasks): #18586 API/UI masking, #18587 definition encryption, #18588 Worker log masking. Until #18588 lands, task logs may still print sensitive values.

Out of scope

Item	Notes
Project parameters (`ProjectParameter` / `t_ds_project_parameter`)	Different storage model than `Property`; not in this DSIP. UI/docs should state this is unsupported for now
Workflow definition Export / Import	Feature removed or unmaintained; not considered
KMS / envelope encryption / key rotation	Out of scope
Encrypting instance `global_params`	This DSIP uses plaintext materialization at runtime; encrypting instance params can be a follow-up if the community requires it

Task stdout log masking: not in #18586 / #18587; implemented in #18588 (still part of this DSIP delivery — see Subtasks).

Goals (acceptance)

#18586 — API/UI masking

Definition parameters marked sensitive=true never return real values in any external API/UI response.
Unmarked parameters behave exactly as today.
Saving with ****** keeps the stored value (keep-original). Empty string is a real empty value.

#18587 — Definition encryption

When datasource.encryption.enable=true, sensitive definition values are not stored as plaintext.
Saving must not double-encrypt unchanged sensitive values.
After same-cluster Copy of a definition, sensitive parameters still work at runtime.

#18588 — Worker log masking

Task stdout (and agreed task log paths) must not contain plaintext values of that task’s sensitive parameters.
Dynamic mask patterns must be cleared when the task ends (success / failure / kill), so later tasks are not polluted.

Explicitly not claimed

With encryption disabled (default), definition storage may still be plaintext.
Protection is at the same level as datasource passwords (salt + Base64 obfuscation), not compliance-grade encryption.
Plaintext remains visible in Master/Worker memory and the task process.
[DSIP-105][Feature][API] Add Property.sensitive and mask values in API/UI #18586 / [DSIP-105][Feature][API] Encrypt sensitive definition params with PasswordUtils #18587 alone do not claim log safety.

Delivery (three subtasks)

Subtask	Issue	Scope	Depends on
API/UI masking	#18586	`Property.sensitive`; API/UI masking; save merge (keep-original `******` only); start-time merge; UI checkbox; unit tests	None
Definition encryption	#18587	Reuse `PasswordUtils` (no datasource CRUD); no double-encrypt; Copy keeps ciphertext as-is	#18586
Worker log masking	#18588	Dynamically register log mask patterns from this task’s sensitive param values; mandatory cleanup on task end; unit tests + log assertions	#18586

Merge order: #18586 first, then #18587 and #18588 (both can follow #18586 independently).

Design Details

1. Add `sensitive` boolean to `Property`

Do not introduce a new data type (e.g. SENSITIVE_VARCHAR). Only add a field on existing Property:

@Builder.Default
private boolean sensitive = false;

Missing / null in JSON → treat as false (backward compatible).
Serialized into existing JSON columns; no schema change.
Applies to workflow global params and task localParams.

2. Placeholder semantics

Reuse existing constant Constants.XXXXXX ("******").

Scenario	Meaning
API / UI display	Sensitive values are always `******`
Update submit	Sensitive value `****` or empty string** → “unchanged; keep DB value”
Real secret	Document that users must not use `******` as a real password (same convention as datasource)

UI: use a checkbox for sensitive; do not rely on type=password alone. Always echo ******; on edit, replace the whole value; if unchanged, submit ******.

3. Storage layers and threat model

Layer	Stored content	Notes
Definition (workflow/task definition and definition log)	`sensitive=true` values are ciphertext when encryption is enabled	Protects static config
Runtime (workflow instance `global_params`, task execution context)	Plaintext after decrypt + merge at start (materialized)	For param curing and task dispatch
External API / UI	Always `******`	Mask both definition and instance queries

Why runtime plaintext: curing, complement, rerun, and context dispatch already assume readable values; encrypting instance params would scatter decrypt across the execution path. This DSIP covers the main goals with definition encryption + API masking (+ PR2 log masking).

4. Encrypt / decrypt (definition write path only, PR1)

Reuse PasswordUtils.encodePassword / decodePassword.
Same switch as datasource: datasource.encryption.enable (default false).
After salt change, old ciphertext cannot be decrypted; users must re-enter (same as datasource).

Encode only when all of the following hold:

sensitive == true
value is non-empty
value is not a keep-original marker (not ******, not empty)
value is new plaintext from the client (see §5)

5. Write path (forbid double encryption, PR1)

Client submit
  → for each field:
       if sensitive && value is keep-original (****** or empty):
           write back the DB value for the same prop **as-is**
           (if already ciphertext, keep ciphertext)
           **do not** encode again
       else if sensitive && value is new plaintext:
           encode (if encryption enabled) → write
       else:
           write as non-sensitive as-is
  → persist

Match key: param name prop.
Creating a new prop with only a keep-original marker is invalid (nothing to merge → reject; require plaintext).

sensitive flag transitions:

Change	Behavior
`false → true`, new plaintext submitted	encode then store
`false → true`, keep-original submitted on update	allow; merge existing plaintext, then encode with `sensitive=true`
`true → false`	decode to plaintext, then store with `sensitive=false`
`true → true`, keep-original	keep DB ciphertext as-is; do not encode

Suggested hooks: before workflow definition save for global params; before task definition save for local params. Exact class names are implementation details; behavior above is normative.

6. Read path (copies only; do not mutate shared entities, PR1)

Shared helpers (names illustrative):

maskSensitiveProperties(...) → deep copy then mask, for API responses
decryptSensitiveProperties(...) → deep copy then decrypt, for internal execution/merge
mergeKeepOriginal(submitted, existingFromDb) → merge keep-original only; does not encrypt

Rules:

After loading definition from DB, use decrypt copies only on internal paths (start merge / task dispatch).
Any path returning to UI / OpenAPI / token clients must return mask copies only.
Never in-place replace values with ****** on a shared WorkflowDefinition / TaskDefinition entity that is later used for execution.

Masking must cover every API that returns Property / globalParams / localParams / DagData (including list/paging/detail/variables). Do not assume genDagData alone is sufficient.

Order of operations:

Write: merge keep-original → encode only new plaintext → persist
Read (external): DB → decrypt to copy if needed → mask → response
Read (execution): DB → decrypt to copy → merge/cure/dispatch (plaintext)

7. Start & Command (PR1)

Load definition globalParams → decrypt copy.
Merge with command / startParams: same name with keep-original → use decrypted definition value; new plaintext → may override.
Write merged plaintext into workflow instance global_params (runtime materialization).
Later prepareParamsMap uses instance/context plaintext; do not apply definition-style encode on instance fields.

Start UI: show ****** for sensitive items; submit keep-original if unchanged; allow override with a new value.

8. OUT parameters (PR1 API masking; PR2 for logs)

sensitive=true is allowed on OUT params.
Values written at runtime are masked in instance/variable APIs (PR1).
Sensitive values appearing in stdout are handled by PR2.

9. Copy (Export / Import not considered)

Operation	This DSIP
Export / Import	Out of scope (removed / unmaintained)
Copy (same-cluster definition copy, e.g. `batchCopyWorkflowDefinition`)	Server copies JSON as-is (including ciphertext and `sensitive`); no UI round-trip; no double encryption (PR1)

10. External surfaces that must be masked (PR1 principles)

Any response containing global/local Property must be masked, including but not limited to:

viewVariables (definition / instance)
queryWorkflowDefinitionByCode / getWorkflowDefinition / list / paging / byName
queryWorkflowInstanceById and other APIs returning dag/params
getTaskDefinition / queryTaskDefinitionDetail

Save APIs: merge keep-original first, then encrypt per §5, then persist.

11. UI (PR1)

Add sensitive checkbox on global params and task local params forms.
Echo sensitive values as ******; submit ****** when unchanged.
Leave project parameters unchanged; tip that sensitive marking is only for workflow global and task local params for now.

12. Task stdout log masking (PR2)

Goal: task logs must not contain plaintext values of this task’s sensitive parameters.

Design points:

Input: plaintext values of sensitive=true entries in this task’s prepareParamsMap (or equivalent context).
Behavior: when those values appear in logs → replace with ****** (may extend existing SensitiveDataConverter with per-task dynamic pattern registration).
Lifecycle: register at task start; must clear on task end (success / failure / kill); no static global leakage into later unrelated tasks.
Scope: stdout / task logs on the physical task execution path; details in the PR2 description.
Non-goals: project parameters; does not replace definition encryption; does not claim coverage of every plugin custom log file.

13. Risks & mitigations

Risk	Mitigation
`******` used as a real password	Document the prohibition; same as datasource
Double-encoding keep-original corrupts the secret	§5: write DB value as-is; never encode again
In-place mask pollutes execution	§6: deep copies required; tests cover this
Legacy data lacks `sensitive`	Default `false`
Salt change	Same as datasource: re-enter secrets
Legacy unencrypted data	`decodePassword` returns original when undecodable
Encryption off by default	Document honestly; same expectation as datasource
PR2 mask not cleared → cross-task pollution	Unified cleanup on all end paths (including kill); concurrency tests
Users assume logs are safe after PR1 only	Motivation / Goals + release notes point to PR2

Compatibility, Deprecation, and Migration Plan

Backward compatible: sensitive defaults to false; existing workflows unchanged.
No DB migration: field lives in existing JSON.
When datasource.encryption.enable=false, do not encrypt; old plaintext remains readable.
Rollout: backend (PR1) → frontend (PR1) → logging (PR2).
Rollback: PR2 and PR1 frontend can be rolled back independently; old clients ignore unknown fields.

Test Plan

PR1

Unit: Property.sensitive serialization; merge keep-original without double encrypt; false↔true transitions; mask/decrypt deep copies do not mutate originals; encryption enable true/false.
API integration: create definition with sensitive global/local params → queries return ****** → update only non-sensitive fields then run task still sees real secret → with enable=true, definition DB value is not plaintext → Copy still runs correctly.
Frontend: lint / prettier; variable views show no plaintext.
Regression: unmarked params match current behavior.

PR2

Unit: dynamic register/clear; consecutive or concurrent tasks do not cross-contaminate masks.
Integration or repeatable task-log case: when a sensitive value is printed by the script, logs show ******; cleanup still works after failure/kill.

Separate / Follow-up (not in these three subtasks)

Project parameter sensitive flag
Export / Import strategy
Encrypted instance global_params
Optional encryption switch independent of datasource.encryption

Design decisions (summary)

Definition ciphertext + runtime plaintext materialization + API masking
Export/Import not considered; Copy copies ciphertext as-is
false→true: create rejects ******; update allow keep-original (merge existing plaintext, then encode)
Keep-original = ****** or empty string; never re-encode keep-original
Three subtasks: [DSIP-105][Feature][API] Add Property.sensitive and mask values in API/UI #18586 API/UI masking; [DSIP-105][Feature][API] Encrypt sensitive definition params with PasswordUtils #18587 definition encryption; [DSIP-105][Feature][Worker] Mask sensitive parameter values in task logs #18588 stdout dynamic masking

Code of Conduct

I agree to follow this project's Code of Conduct

@det101

Hi @det101, this looks like a valuable feature, especially from a security standpoint 👍
Before starting implementation, do maintainers have a preferred design for how sensitive variables should behave (e.g. UI masking, logging behavior, export/import handling)?

Happy to work on this once the expected approach is clarified.

@det101

Please create an DSIP for this. Feature like this belong to DSIP. And you should provide more specific design detail in it. @det101

@SbloodyS

Thanks for the guidance @SbloodyS
That makes sense — this definitely fits the DSIP process.

@det101, could you please create a DSIP for this feature with the proposed design details?
It would be great if the DSIP could cover things like:

expected behavior of sensitive variables (UI masking, logging, export/import),

reuse of existing encryption/decryption mechanisms,

backward compatibility and migration (if any).

Once the DSIP is up, we can continue the discussion and move toward implementation.

It's better to add sensitive switch at the currently variable, rather than add a new type, otherwise you need to add a lot of type here.

@ruanwenjun

It's better to add sensitive switch at the currently variable, rather than add a new type, otherwise you need to add a lot of type here.

@ruanwenjun You're absolutely right that adding a sensitive switch/flag to the existing Property class is a much better design than introducing a new variable type. Creating a dedicated "Sensitive Variable" type would indeed lead to type proliferation and maintenance challenges.

feat: add sensitive variable type support for secure secret handling

feat: integrate sensitive data encryption/decryption in Property class

feat: add sensitive variable type support for secure secret handling

为了方便大家讨论，附上中文说明

[DSIP-105][Api-server] 敏感变量支持 — API/UI 脱敏与加密存储

动机

当前 DolphinScheduler 的变量不区分敏感与非敏感信息。当用户将密码、API Key 等作为工作流全局参数或任务本地参数存储时，这些值可能以明文出现在：

展示定义变量的页面
API 响应（如 viewVariables、getWorkflowDefinition、queryWorkflowDefinitionByCode 等）
数据库定义态 JSON（global_params / task_params 中的 value）

本变更在现有 Property 上增加变量级敏感标记：API/UI 脱敏；定义态敏感 value 落库前做与数据源同级的 at-rest 保护（复用 PasswordUtils）。

交付拆为 两个 PR（见 Delivery）：PR1 覆盖定义存储与 API/UI；PR2 覆盖任务 stdout 动态脱敏。PR1 合入后、PR2 完成前，任务日志仍可能打印敏感值。

Out of scope（相对整个 DSIP 的长期非目标，或明确不在对应 PR）

项	说明
项目参数（`ProjectParameter` / `t_ds_project_parameter`）	存储模型与 `Property` 不同，暂不实现；UI/文档注明暂不支持
工作流定义 Export / Import	功能已下线或不在维护范围，不考虑
KMS / 信封加密 / 密钥轮转	不在范围
实例 `global_params` 密文存储	本 DSIP 采用运行态明文物化；若社区后续要求再开 follow-up

任务 stdout 日志脱敏：不在 PR1，在 PR2 实现（仍属本 DSIP 交付，见 Delivery）。

Goals（验收口径）

PR1

标记为 sensitive=true 的定义参数，在所有对外 API/UI 中不返回真实 value。
定义态在 datasource.encryption.enable=true 时，敏感 value 不以明文持久化。
未标记敏感的参数行为与现网完全一致。
保存时对「未修改」敏感值不二次加密。
同集群 Copy 定义后，敏感参数仍可正常运行。

PR2

任务 stdout（及约定的任务日志路径）中，不出现本任务敏感参数的明文 value。
任务结束（成功/失败/kill）后清理动态 mask，不污染后续任务日志。

明确不声称

加密开关默认关闭时，定义态仍可能明文。
本方案与数据源密码同级（salt + Base64 混淆），不是合规级加密。
Master/Worker 内存、任务进程内仍可见明文。
PR1 单独合入时，不声称日志安全。

Delivery（两个 PR）

PR	范围	依赖
PR1	`Property.sensitive`；定义态加解密；API/UI 脱敏；保存 merge（防二次加密）；启动合并；Copy 原样拷贝；单测 + api-test；UI checkbox	无
PR2	按本次任务敏感参数值动态注册日志 mask；任务结束必须清理；单测 + 日志断言	依赖 PR1

合入顺序： 先 PR1，再 PR2。

Design Details

1. 在 `Property` 中新增 `sensitive` 布尔字段

不引入新的敏感变量类型（如 SENSITIVE_VARCHAR），仅在现有 Property 增加字段：

@Builder.Default
private boolean sensitive = false;

缺省 / JSON 缺失 / null → 视为 false（向后兼容）。
随现有 JSON 列序列化，无 schema 变更。
适用于工作流全局参数与任务 localParams。

2. 占位符语义

复用已有常量 Constants.XXXXXX（值为 "******"）。

场景	含义
API / UI 展示	敏感 value 一律为 `******`
更新提交	敏感项 value 为 `****` 或空串** → 「未修改，保留库中原值」
真实密钥	文档约定：用户不得将 `******` 作为真实密码（与数据源惯例一致）

UI：敏感标记使用 checkbox；不以 type=password 作为唯一手段。回显固定 ******；用户修改时整段替换；未改则提交 ******。

3. 存储分层与威胁模型

层	存储内容	说明
定义态（workflow/task definition 及 definition log）	`sensitive=true` 的 value 在加密开启时为密文	保护静态配置
运行态（workflow instance `global_params`、任务执行上下文）	启动时解密并合并后的明文（物化）	供参数固化与任务下发
对外 API / UI	永远 `******`	定义与实例查询均脱敏

选择「运行态明文」的原因：固化参数、补数、重跑、上下文下发已假设可读 value；实例再密文会把 decrypt 散落到整条执行链。本 DSIP 用 定义态加密 + API 脱敏（+ PR2 日志脱敏）覆盖主要目标。

4. 加解密（仅定义态写路径，PR1）

复用 PasswordUtils.encodePassword / decodePassword。
开关与数据源相同：datasource.encryption.enable（默认 false）。
salt 变更后旧密文无法解密，需用户重填（与数据源一致）。

仅当同时满足时 encode：

sensitive == true
value 非空
value 不是保留标记（非 ******、非空串）
value 来自客户端提交的新明文（见 §5）

5. 写路径（禁止二次加密，PR1）

客户端提交
  → 对每个字段：
       if sensitive && value 是保留标记（****** 或空）:
           用 DB 中同 prop 的 value **原样**写回
           （已是密文则保持密文）
           **禁止**再 encode
       else if sensitive && value 为新明文:
           encode（若加密开启）→ 写入
       else:
           按非敏感原样写入
  → 持久化

匹配键：参数名 prop。
新建 prop 不允许只提交保留标记（无旧值可合并 → 校验失败，要求提供明文）。

sensitive 开关切换：

变更	行为
`false → true`，提交新明文	encode 后存储
`false → true`，提交保留标记	拒绝，要求重填明文
`true → false`	先 decode 得明文，再以 `sensitive=false` 明文落库
`true → true`，保留标记	原样保留库中密文，不 encode

入口建议：全局参数在工作流定义保存前；本地参数在任务定义保存前。具体类名以实现为准，行为以本表为准。

6. 读路径（副本；禁止污染实体，PR1）

统一工具（名称示意）：

maskSensitiveProperties(...) → 深拷贝后脱敏，供 API 响应
decryptSensitiveProperties(...) → 深拷贝后解密，供内部执行/合并
mergeKeepOriginal(submitted, existingFromDb) → 仅合并保留标记，不负责加密

规则：

从 DB 读取定义后，仅在「启动合并 / 下发任务」等内部路径使用 decrypt 副本。
凡返回前端 / OpenAPI / Token 客户端的路径，只返回 mask 副本。
禁止在共享的 WorkflowDefinition / TaskDefinition 实体上原地改成 ****** 后再用于执行。

脱敏应覆盖所有返回 Property / globalParams / localParams / DagData 的接口（含 list/paging/detail/variables 等），不能假设仅 genDagData 一处即可盖全。

加解密与脱敏顺序：

写：merge 保留标记 →（仅新明文）encrypt → 持久化
读（对外）：DB →（如需）decrypt 到副本 → mask → 响应
读（执行）：DB → decrypt 到副本 → 合并/固化/下发（明文）

7. 启动与 Command（PR1）

加载定义 globalParams → decrypt 副本。
与 command / startParams 合并：同名为保留标记 → 用定义解密值；提交新明文 → 允许覆盖。
合并结果（明文）写入 workflow instance global_params（运行态物化）。
后续 prepareParamsMap 等使用实例/上下文中的明文，不再对实例字段做定义态 encode。

启动 UI：敏感项展示 ******；未改则提交保留标记；允许用户输入新值覆盖。

8. OUT 参数（PR1 API 脱敏；PR2 覆盖日志）

允许 sensitive=true 的 OUT。
运行写出的值出现在实例/变量类 API 时做脱敏（PR1）。
stdout 中的敏感 OUT/参数值由 PR2 脱敏。

9. Copy（Import / Export 不考虑）

操作	本 DSIP
Export / Import	Out of scope（功能下线/不维护）
Copy（同集群复制工作流定义，如 `batchCopyWorkflowDefinition`）	服务端直接复制 JSON（含密文与 `sensitive`），不经前端，不二次加密（PR1）

10. 需要脱敏的对外面（PR1 原则）

凡响应中出现全局/本地 Property 的接口均需 mask，包括但不限于：

viewVariables（定义 / 实例）
queryWorkflowDefinitionByCode / getWorkflowDefinition / list / paging / byName
queryWorkflowInstanceById 等返回 dag/params 的接口
getTaskDefinition / queryTaskDefinitionDetail

保存类 API：先 merge 保留标记，再按 §5 加密后持久化。

11. UI（PR1）

全局参数、任务本地参数表单增加 sensitive checkbox。
敏感 value 回显 ******；未修改提交 ******。
项目参数保持现状，并提示：敏感标记暂仅支持工作流全局参数与任务局部参数。

12. 任务 stdout 日志脱敏（PR2）

目标： 任务日志中不出现本任务敏感参数的明文 value。

要点：

输入：本任务 prepareParamsMap（或等价上下文）中 sensitive=true 的明文 value。
行为：日志中出现这些 value → 替换为 ******（可基于现有 SensitiveDataConverter 扩展为按任务动态注册 pattern）。
生命周期：任务开始注册；任务结束（成功/失败/kill）必须清理；禁止静态全局状态泄漏到后续无关任务。
范围：Physical 任务执行路径上的 stdout/任务日志；细则在 PR2 描述中展开。
非目标：不做项目参数；不替代定义态加密；不宣称覆盖所有插件自定义日志文件。

13. Risks & Mitigations

风险	缓解
`******` 被当作真实密码	文档约定禁止；与数据源一致
对保留值二次加密导致密钥损坏	§5：保留标记原样写回，禁止再 encode
原地 mask 污染执行路径	§6：强制深拷贝；测试覆盖
旧数据无 `sensitive`	默认 `false`
salt 变更	与数据源相同：需重填
旧数据未加密	`decodePassword` 不匹配则原样返回
加密默认关闭	文档诚实说明；与数据源同级预期
PR2 mask 未清理导致串任务	结束路径（含异常/kill）统一 cleanup；并发用例
仅合入 PR1 时用户误以为日志也安全	Motivation / Goals 写明；Release note 提示等 PR2

Compatibility, Deprecation, and Migration Plan

向后兼容：sensitive 默认 false，既有工作流行为不变。
无需 DB migration：字段在现有 JSON 中。
datasource.encryption.enable=false 时不加密；旧明文仍可读。
发布顺序：后端（PR1）→ 前端（PR1）→ 日志（PR2）。
回滚：可分别回滚 PR2 / PR1 前端；旧客户端忽略未知字段。

Test Plan

PR1

单元：Property.sensitive 序列化；merge 保留且不二次加密；false↔true 切换；mask/decrypt 深拷贝不污染原对象；encryption 开关 true/false。
API 集成：创建含敏感全局/局部参数的定义 → 查询均为 ****** → 只改非敏感字段保存后任务仍能读到真密钥 → enable=true 时定义态 DB 非明文 → Copy 后任务仍可用。
前端：lint / prettier；变量查看 UI 不出现明文。
回归：未标记敏感的参数与现网一致。

PR2

单元：动态注册/清理；并发或先后两个任务 mask 不串扰。
集成或可重复的任务日志用例：敏感值出现在脚本输出中时日志为 ******；任务失败/kill 后清理仍生效。

Separate / Follow-up（非本 DSIP 当前两 PR）

项目参数敏感标记
Export / Import 策略
实例 global_params 密文存储
独立于 datasource.encryption 的变量加密开关（可选增强）

已定取舍摘要

定义态加密 + 运行态明文物化 + API 脱敏
Import/Export 不考虑；Copy 原样拷贝密文
false→true 必须重填明文
保留标记 = ****** 或空串；禁止对保留值再 encode
拆 两个 PR：PR1 变量存储/API/UI；PR2 stdout 动态脱敏

@ruanwenjun

@ruanwenjun @SbloodyS This issue has been fully detailed; please review it. Thanks.

[Feature-17937] Support sensitive workflow variables fix apache#17937

@ruanwenjun

@ruanwenjun @SbloodyS The DSIP design has been revised based on review feedback. Please take another look when you have time.

Key updates:

Write path: keep-original (****** / empty) must not be re-encoded (avoid double encryption)
Storage model: definition ciphertext + runtime plaintext materialization + API/UI masking
Export/Import: out of scope; Copy keeps ciphertext as-is
Delivery: two PRs — PR1 storage/API/UI; PR2 stdout dynamic log masking
false → true requires re-entering plaintext

English body and Chinese comment are both updated on this issue. Thanks.

@det101

@det101 The design looks good to me, but when implementing it, we should break it down into smaller PRs, such as:

API module: Changes to property-related interfaces
API module: Changes to data sources
Worker module: Changes to log content

@ruanwenjun

@ruanwenjun Thanks, I'll split into smaller PRs as you suggested:

API: Property interfaces + UI masking
API: reuse PasswordUtils (no datasource CRUD change)
Worker: log masking

Keep-original is only ****** (write back DB value, do not re-encode).
Empty/null is a real empty value and is persisted as "" — same as current PasswordUtils.encodePassword / decodePassword (isEmpty → EMPTY, even when encryption is on). ****** is write-path only and is never decoded.

@det101

It is best to split it into different subtasks, each subtask corresponds to an issue, and the issue corresponds to a PR, and then put the issue of the subtask in the description of the main issue. Like #6407 @det101

@SbloodyS

@SbloodyS Split into subtask issues (one issue / one PR), listed on this parent like #6407:

[DSIP-105][Feature][API] Add Property.sensitive and mask values in API/UI #18586 API/UI masking — PR [DSIP-105][API][UI] Add Property.sensitive with API/UI masking #18585
[DSIP-105][Feature][API] Encrypt sensitive definition params with PasswordUtils #18587 Definition encryption (PasswordUtils, no datasource CRUD)
[DSIP-105][Feature][Worker] Mask sensitive parameter values in task logs #18588 Worker log masking

#18585 closes #18586 only (Fixes #18586, Refs #17937). Parent stays open until all three subtasks are done.

[DSIP-105][Feature][Parent] Sensitive Variable Support — Masking in API/UI & Encrypted Storage #17937

Description

Search before asking

Subtasks

Motivation

Out of scope

Goals (acceptance)

#18586 — API/UI masking

#18587 — Definition encryption

#18588 — Worker log masking

Explicitly not claimed

Delivery (three subtasks)

Design Details

1. Add sensitive boolean to Property

2. Placeholder semantics

3. Storage layers and threat model

4. Encrypt / decrypt (definition write path only, PR1)

5. Write path (forbid double encryption, PR1)

6. Read path (copies only; do not mutate shared entities, PR1)

7. Start & Command (PR1)

8. OUT parameters (PR1 API masking; PR2 for logs)

9. Copy (Export / Import not considered)

10. External surfaces that must be masked (PR1 principles)

11. UI (PR1)

12. Task stdout log masking (PR2)

13. Risks & mitigations

Compatibility, Deprecation, and Migration Plan

Test Plan

PR1

PR2

Separate / Follow-up (not in these three subtasks)

Design decisions (summary)

Code of Conduct

Activity

alisha-1000 commented on Feb 3, 2026

SbloodyS commented on Feb 4, 2026

alisha-1000 commented on Feb 4, 2026

ruanwenjun commented on Feb 4, 2026

det101 commented on Feb 5, 2026

det101 commented on Mar 10, 2026

[DSIP-105][Api-server] 敏感变量支持 — API/UI 脱敏与加密存储

动机

Out of scope（相对整个 DSIP 的长期非目标，或明确不在对应 PR）

Goals（验收口径）

PR1

PR2

明确不声称

Delivery（两个 PR）

Design Details

1. 在 Property 中新增 sensitive 布尔字段

2. 占位符语义

3. 存储分层与威胁模型

4. 加解密（仅定义态写路径，PR1）

5. 写路径（禁止二次加密，PR1）

6. 读路径（副本；禁止污染实体，PR1）

7. 启动与 Command（PR1）

8. OUT 参数（PR1 API 脱敏；PR2 覆盖日志）

9. Copy（Import / Export 不考虑）

10. 需要脱敏的对外面（PR1 原则）

11. UI（PR1）

12. 任务 stdout 日志脱敏（PR2）

13. Risks & Mitigations

Compatibility, Deprecation, and Migration Plan

Test Plan

PR1

PR2

Separate / Follow-up（非本 DSIP 当前两 PR）

已定取舍摘要

det101 commented on Mar 10, 2026

det101 commented on Aug 19, 2026

ruanwenjun commented on Aug 20, 2026

det101 commented on Aug 20, 2026

SbloodyS commented on Aug 25, 2026

det101 commented on Aug 25, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

1. Add `sensitive` boolean to `Property`

1. 在 `Property` 中新增 `sensitive` 布尔字段