Skip to content

feat: add severity gates and inline IaC suppressions - #133

Merged
dannysteenman merged 10 commits into
mainfrom
feat/severity-fail-on-suppression
Jul 29, 2026
Merged

feat: add severity gates and inline IaC suppressions#133
dannysteenman merged 10 commits into
mainfrom
feat/severity-fail-on-suppression

Conversation

@axonstone

Copy link
Copy Markdown
Contributor

Summary

Adds severity-aware prioritization and policy controls across CloudBurn's rules, SDK, and CLI.

  • Assigns high, medium, or low severity to all 82 built-in rules and propagates it through findings, JSON/table output, and rules list --severity.
  • Adds inclusive --fail-on <severity> gates to scan and discover, with per-mode fail-on config and preserved --exit-code behavior.
  • Adds resource-local Terraform and CloudFormation YAML suppressions with optional reasons, retained suppressed output, and exclusion from CI gates.
  • Adds coordinated package changesets and updates the public contracts, architecture docs, and examples.

Diagram

flowchart LR
  Rule["Rule metadata\nseverity"] --> Finding["Active findings"]
  IaC["Terraform / CFN YAML\nresource comments"] --> Parser["Parser suppression metadata"]
  Parser --> Partition{"Static result partition"}
  Finding --> Partition
  Partition -->|active| Output["Table / JSON output"]
  Partition -->|suppressed| Audit["suppressed array / count"]
  Output --> Gate{"--fail-on / fail-on / --exit-code"}
  Gate -->|qualifying active finding| Exit1["exit 1"]
  Gate -->|no qualifying finding| Exit0["exit 0"]
  Audit -. excluded .-> Gate
Loading

Validation

  • pnpm verify
  • 420 rules tests, 362 SDK tests, 123 CLI tests, and 14 documentation tests
  • Final targeted review found no remaining actionable suppression paths

Related issues

Closes #101
Closes #102
Closes #103

Follow-ups reviewed but intentionally kept separate: #104, #105, #106, #107, and #122.

@axonstone axonstone added the enhancement New feature or request label Jul 29, 2026
@axonstone
axonstone requested a review from dannysteenman July 29, 2026 13:37
@axonstone

axonstone commented Jul 29, 2026

Copy link
Copy Markdown
Contributor Author

roborev: Combined Review (59fca4a)

Verdict: Fail — one high-severity CI policy bypass remains.

High

  • Inline suppressions bypass the CI trust boundary. Even when CI pins --fail-on, repository authors can add cloudburn-ignore-all; suppressed findings then do not affect the exit code. An untrusted change can therefore neutralize the policy gate. Define the approval boundary and add enforcement such as rejecting suppressions, failing when suppressions exist, or requiring centrally approved suppressions. (config-schema.md:48, README.md:133)

Reviewers: 3 done | Synthesis: codex, 14s | Total: 14m14s

@axonstone

Copy link
Copy Markdown
Contributor Author

Re: comment 5118585392

Addressed in 1df381f. CI still intentionally skips implicit repository config as a trust boundary; the scan/discover help and CLI docs now clearly require --config .cloudburn.yml when CI should use that file, while workflows can pin --fail-on directly. The SDK now exports evaluateScanPolicy(), attaches threshold, qualifyingFindingCount, and violated under ScanResult.policy whenever effective failOn is configured for static or discovery scans, and the CLI consumes the shared evaluator. Added public policy and CloudBurnClient coverage; pnpm verify passes (420 rules, 366 SDK, 123 CLI, and 14 docs tests).

@dannysteenman
dannysteenman marked this pull request as ready for review July 29, 2026 14:37
@axonstone

Copy link
Copy Markdown
Contributor Author

Re: comment 5118585392

Already addressed in 1df381f. CI configuration trust behavior is now explicit in command help and CLI documentation, and the SDK exposes shared policy evaluation plus ScanResult.policy for configured failOn thresholds. The full verification suite passes.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1df381fca1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +69 to +70
if (character === '"' || character === "'") {
state.quote = character;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Do not open YAML quotes inside plain scalars

When a valid CloudFormation plain scalar contains an unmatched quote character—for example, Description: It's legacy before Resources—this branch sets state.quote even though YAML treats the apostrophe as ordinary plain-scalar content. That state persists across lines, so subsequent real # cloudburn-ignore... comments are treated as quoted text and discarded; the affected findings remain active and can incorrectly fail CI gates. Quote tracking should recognize actual quoted-scalar boundaries or use comment information from the parsed YAML document.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 59fca4a. YAML quote state now opens only where a quoted scalar can begin, so apostrophes inside plain scalars no longer hide later suppression comments. Added a CloudFormation regression test; pnpm verify passes with 367 SDK tests.

@dannysteenman
dannysteenman merged commit f3b9ab3 into main Jul 29, 2026
7 checks passed
@dannysteenman
dannysteenman deleted the feat/severity-fail-on-suppression branch July 29, 2026 18:00
@github-actions github-actions Bot mentioned this pull request Jul 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support inline finding suppression in IaC scans Add --fail-on severity threshold for CI exit codes Add severity levels to rules and findings

2 participants