Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions .github/workflows/pr-checks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -70,3 +70,27 @@ jobs:

- name: Build browser extension
run: pnpm --filter @profullstack/threatcrush-extension build

libinjection-wasm:
# The checked-in libinjection.wasm must be exactly what the vendored C
# sources build (apps/cli/scripts/build-libinjection-wasm.mjs), so the
# binary shipped to npm is reviewable through its sources.
name: libinjection.wasm is reproducible
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v7

- name: Setup Node.js
uses: actions/setup-node@v7
with:
node-version: 22

- name: Setup Zig
uses: mlugg/setup-zig@v2
with:
version: 0.16.0
use-cache: false

- name: Rebuild and compare
run: node apps/cli/scripts/build-libinjection-wasm.mjs --check
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,8 @@ node_modules

# Project-specific
data/
# libinjection's upstream test vectors, run by src/core/crs/__tests__
!/apps/cli/vendor/libinjection/data/
supabase/supabase/.temp/
.qwen/
.claude/
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@

---

ThreatCrush is a security daemon that runs on your server, **reading your logs and watching inbound connections** for live attacks. It checks every nginx request against 94 OWASP CRS rules (paranoia level 1) + 1 ThreatCrush rule with CRS anomaly scoring, runs 15 detection rules over auth, web and network events, auto-bans attackers, scans your codebase, spot-checks your URLs, and alerts you in real-time.
ThreatCrush is a security daemon that runs on your server, **reading your logs and watching inbound connections** for live attacks. It checks every nginx request against 96 OWASP CRS rules (paranoia level 1) + 1 ThreatCrush rule with CRS anomaly scoring, runs 15 detection rules over auth, web and network events, auto-bans attackers, scans your codebase, spot-checks your URLs, and alerts you in real-time.

```
$ threatcrush monitor
Expand Down Expand Up @@ -138,7 +138,7 @@ threatcrush store publish https://github.com/you/my-module # Publish your own

| Component | What it covers |
|-----------|----------------|
| `log-watcher` | nginx access log + syslog — 94 OWASP CRS rules (PL1: SQLi, XSS, path traversal, RFI, RCE, PHP/Java injection, SSRF, scanners) + 1 ThreatCrush rule (OS files in the path) scored on every request |
| `log-watcher` | nginx access log + syslog — 96 OWASP CRS rules (PL1: SQLi, XSS, path traversal, RFI, RCE, PHP/Java injection, SSRF, scanners) + 1 ThreatCrush rule (OS files in the path) scored on every request |
| `ssh-guard` | auth.log / secure — failed logins, brute force, root logins, user enumeration |
| `user-journal` | journald — the systemd journal |
| `network-monitor` | Inbound connections to your listening ports — port scans, SYN floods |
Expand Down
4 changes: 2 additions & 2 deletions apps/cli/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@

---

ThreatCrush is a security daemon that runs on your server, **reading your logs and watching inbound connections** for live attacks. It checks every nginx request against 94 OWASP CRS rules (paranoia level 1) + 1 ThreatCrush rule with CRS anomaly scoring, runs 15 detection rules over auth, web and network events, auto-bans attackers, scans your codebase, spot-checks your URLs, and alerts you in real-time.
ThreatCrush is a security daemon that runs on your server, **reading your logs and watching inbound connections** for live attacks. It checks every nginx request against 96 OWASP CRS rules (paranoia level 1) + 1 ThreatCrush rule with CRS anomaly scoring, runs 15 detection rules over auth, web and network events, auto-bans attackers, scans your codebase, spot-checks your URLs, and alerts you in real-time.

```
$ threatcrush monitor
Expand Down Expand Up @@ -139,7 +139,7 @@ threatcrush store publish https://github.com/you/my-module # Publish your own

| Component | What it covers |
|-----------|----------------|
| `log-watcher` | nginx access log + syslog — 94 OWASP CRS rules (PL1: SQLi, XSS, path traversal, RFI, RCE, PHP/Java injection, SSRF, scanners) + 1 ThreatCrush rule (OS files in the path) scored on every request |
| `log-watcher` | nginx access log + syslog — 96 OWASP CRS rules (PL1: SQLi, XSS, path traversal, RFI, RCE, PHP/Java injection, SSRF, scanners) + 1 ThreatCrush rule (OS files in the path) scored on every request |
| `ssh-guard` | auth.log / secure — failed logins, brute force, root logins, user enumeration |
| `user-journal` | journald — the systemd journal |
| `network-monitor` | Inbound connections to your listening ports — port scans, SYN floods |
Expand Down
17 changes: 14 additions & 3 deletions apps/cli/docs/auto-defence.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,9 +94,20 @@ under `min_severity = "critical"` they alert without banning, and a single
matching CRS rule is still not enough for a ban.

What an access log cannot show, these rules cannot see: request bodies,
cookies and other headers. The two libinjection rules (942100 SQLi, 941100
XSS) are not ported, so a bare `1' OR 1=1` scores nothing; `UNION SELECT`,
`SLEEP(`, script tags, traversal and the rest do.
cookies and other headers.

CRS's two libinjection rules run on libinjection itself (v4.0.0, BSD-3-Clause,
the version ModSecurity v3 builds against), compiled to WebAssembly and shipped
in the package: 942100 (`@detectSQLi`, on arguments, argument names, the
User-Agent and the Referer) and 941100 (`@detectXSS`, on arguments, argument
names and the User-Agent). They catch what no regex rule does, such as a bare
`1' OR 1=1` or `admin'--`. libinjection judges a value by its SQL token shape,
so a few ordinary strings look like SQLi to it, as they do under ModSecurity: a
number followed by `--` (`2019 -- 2020`, `10--20`) and a word followed by a
`/* … */` comment. Apostrophes, quotes and SQL words in prose (`O'Brien`,
`rock 'n' roll`, `"exact phrase"`, `where is george`, JSON) are not. A site
whose visitors search for number ranges written with `--` should set
`exclude_rules = [942100]`.

One rule of ThreatCrush's own is scored alongside CRS, the same way. It is not
CRS and is not counted as CRS; its id is in the local range (1–99,999), clear of
Expand Down
5 changes: 4 additions & 1 deletion apps/cli/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,8 @@
"start": "node dist/index.js",
"test": "vitest run",
"test:watch": "vitest",
"crs:build": "node scripts/build-crs-rules.mjs"
"crs:build": "node scripts/build-crs-rules.mjs",
"libinjection:build": "node scripts/build-libinjection-wasm.mjs"
},
"keywords": [
"security",
Expand All @@ -32,6 +33,8 @@
"dist/systemd/*.service",
"dist/crs/LICENSE",
"dist/crs/NOTICE",
"dist/libinjection/libinjection.wasm",
"dist/libinjection/COPYING",
"README.md",
"LICENSE"
],
Expand Down
7 changes: 6 additions & 1 deletion apps/cli/scripts/build-crs-rules.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ const TRANSFORMS = new Set([
]);

/** Operators implemented in src/core/crs/engine.ts. */
const OPERATORS = new Set(['rx', 'pm', 'pmFromFile', 'contains', 'streq', 'beginsWith', 'endsWith', 'within']);
const OPERATORS = new Set(['rx', 'pm', 'pmFromFile', 'contains', 'streq', 'beginsWith', 'endsWith', 'within', 'detectSQLi', 'detectXSS']);

const SEVERITIES = new Set(['CRITICAL', 'ERROR', 'WARNING', 'NOTICE']);

Expand Down Expand Up @@ -234,6 +234,11 @@ function buildOperator(text) {
for (const f of op.arg.split(/\s+/).filter(Boolean)) phrases.push(...readPhraseFile(f));
return { type: 'pm', phrases, negated: op.negated };
}
case 'detectSQLi':
case 'detectXSS':
// libinjection, compiled to WebAssembly (scripts/build-libinjection-wasm.mjs).
if (op.arg) throw new Skip(`@${op.name} with an argument`);
return { type: op.name, negated: op.negated };
default:
return { type: op.name, arg: op.arg, negated: op.negated };
}
Expand Down
107 changes: 107 additions & 0 deletions apps/cli/scripts/build-libinjection-wasm.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
#!/usr/bin/env node
// Builds src/core/crs/libinjection/libinjection.wasm from the vendored
// libinjection C sources (vendor/libinjection, BSD-3-Clause) and
// scripts/libinjection-wasm.c, the interface ThreatCrush calls.
//
// node scripts/build-libinjection-wasm.mjs rebuild and write the module
// node scripts/build-libinjection-wasm.mjs --check fail if the checked-in module
// differs from a fresh build
//
// The module is checked in, so building and testing ThreatCrush needs no C
// toolchain; only this script needs zig, at exactly ZIG_VERSION (another
// version compiles different bytes). `zig` is taken from $ZIG or the PATH.
//
// libinjection v4.0.0, commit 211782219663f889f471650150df12b623c5766e of
// github.com/libinjection/libinjection — the commit ModSecurity v3 builds
// against. Its src/ files are vendored unmodified, with COPYING and a subset of
// its test vectors (tests/, data/) that src/core/crs/__tests__ runs.
//
// wasm32-wasi in the reactor model links zig's bundled wasi-libc statically
// and leaves no entry point: libinjection needs only memchr/strlen-style
// functions and malloc, none of which reach the host, so the module imports
// nothing. NDEBUG compiles out libinjection's assert()s, which would otherwise
// pull in WASI's fd_write and proc_exit to report and abort.

import { execFileSync } from 'node:child_process';
import { createHash } from 'node:crypto';
import { mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { dirname, join } from 'node:path';
import { fileURLToPath } from 'node:url';

const HERE = dirname(fileURLToPath(import.meta.url));
const CLI_ROOT = join(HERE, '..');
const OUT = join(CLI_ROOT, 'src', 'core', 'crs', 'libinjection', 'libinjection.wasm');

export const ZIG_VERSION = '0.16.0';
export const LIBINJECTION_VERSION = '4.0.0';

// Relative to CLI_ROOT, which is the compiler's working directory, so no
// absolute path can reach the output.
const SOURCES = [
'vendor/libinjection/src/libinjection_sqli.c',
'vendor/libinjection/src/libinjection_xss.c',
'vendor/libinjection/src/libinjection_html5.c',
'scripts/libinjection-wasm.c',
];
const EXPORTS = ['tc_input', 'tc_sqli', 'tc_fingerprint', 'tc_xss'];

const sha256 = (bytes) => createHash('sha256').update(bytes).digest('hex');

function zig(args, options = {}) {
return execFileSync(process.env.ZIG || 'zig', args, { cwd: CLI_ROOT, encoding: 'utf8', ...options });
}

function build() {
let version;
try {
version = zig(['version']).trim();
} catch {
throw new Error(`zig ${ZIG_VERSION} is needed to build libinjection.wasm (set $ZIG or put it on the PATH)`);
}
if (version !== ZIG_VERSION) throw new Error(`zig ${ZIG_VERSION} is needed to build libinjection.wasm, found ${version}`);

const tmp = mkdtempSync(join(tmpdir(), 'libinjection-wasm-'));
try {
const bin = join(tmp, 'libinjection.wasm');
zig([
'build-exe',
'-target', 'wasm32-wasi',
'-mexec-model=reactor',
'-fno-entry',
'-O', 'ReleaseFast',
'-fstrip',
'-fsingle-threaded',
'-cflags', '-std=c99', '-DNDEBUG', '-Ivendor/libinjection/src', '--',
...SOURCES,
'-lc',
...EXPORTS.map((e) => `--export=${e}`),
`-femit-bin=${bin}`,
'--cache-dir', join(tmp, 'cache'),
], { stdio: ['ignore', 'inherit', 'inherit'] });
const bytes = readFileSync(bin);
const imports = WebAssembly.Module.imports(new WebAssembly.Module(bytes));
if (imports.length) throw new Error(`libinjection.wasm must import nothing, but imports ${imports.map((i) => `${i.module}.${i.name}`).join(', ')}`);
return bytes;
} finally {
rmSync(tmp, { recursive: true, force: true });
}
}

function main() {
const bytes = build();
console.log(`libinjection ${LIBINJECTION_VERSION}, zig ${ZIG_VERSION}: ${bytes.length} bytes, sha256 ${sha256(bytes)}`);
if (process.argv.includes('--check')) {
let current;
try { current = readFileSync(OUT); } catch { /* missing */ }
if (!current || sha256(current) !== sha256(bytes)) {
console.error(`${OUT} (sha256 ${current ? sha256(current) : 'missing'}) is not what the sources build — run scripts/build-libinjection-wasm.mjs`);
process.exit(1);
}
return;
}
writeFileSync(OUT, bytes);
console.log(`wrote ${OUT}`);
}

if (process.argv[1] === fileURLToPath(import.meta.url)) main();
41 changes: 41 additions & 0 deletions apps/cli/scripts/libinjection-wasm.c
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
/*
* The interface between libinjection (vendor/libinjection, BSD-3-Clause) and
* ThreatCrush's JavaScript wrapper (src/core/crs/libinjection.ts). Compiled
* with libinjection into src/core/crs/libinjection/libinjection.wasm by
* scripts/build-libinjection-wasm.mjs.
*
* JavaScript asks for an input buffer of n bytes, copies the value into it and
* calls tc_sqli(n) or tc_xss(n), which return libinjection's
* injection_result_t: 1 TRUE, 0 FALSE, -1 ERROR.
*/
#include <stdlib.h>

#include "libinjection.h"
#include "libinjection_xss.h"

static char *input;
static size_t input_cap;

/* libinjection writes at most 5 tokens and a NUL; ModSecurity passes 8 bytes. */
static char fingerprint[8];

/* A buffer of at least n bytes for the next call's input, or NULL if out of memory. */
char *tc_input(size_t n) {
if (n > input_cap || input == NULL) {
free(input);
input_cap = n > 256 ? n : 256;
input = malloc(input_cap);
if (input == NULL) input_cap = 0;
}
return input;
}

int tc_sqli(size_t n) {
fingerprint[0] = '\0';
return (int)libinjection_sqli(input, n, fingerprint);
}

/* The NUL-terminated fingerprint of the last tc_sqli call. */
const char *tc_fingerprint(void) { return fingerprint; }

int tc_xss(size_t n) { return (int)libinjection_xss(input, n); }
47 changes: 47 additions & 0 deletions apps/cli/src/core/__tests__/log-parser.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -213,6 +213,53 @@ describe('web attack detection (OWASP CRS, PL1)', () => {
expect(attackSeverity(assessment)).toBeNull();
});

it('bans tautology and comment SQLi that only libinjection sees (CRS 942100)', () => {
// No regex rule at PL1 matches these; libinjection's tokenizer does.
for (const request of [
'GET /login?user=1%27%20OR%201=1 HTTP/1.1',
'GET /login?user=1\\x27+OR+1=1 HTTP/1.1',
'GET /login?user=admin%27--&pass=x HTTP/1.1',
'GET /item?id=1%20or%202%201.e%2F1 HTTP/1.1',
]) {
const assessment = assessNginxRequest(line(request));
expect(assessment.matches.map((m) => m.id), request).toContain(942100);
expect(assessment.attackType, request).toBe('sqli');
expect(attackSeverity(assessment), request).not.toBeNull();
}
// 942100 also reads the User-Agent and Referer, as CRS does.
expect(assessNginxRequest(line('GET / HTTP/1.1', { ua: "1' OR 1=1--" })).matches.map((m) => m.id)).toContain(942100);
expect(assessNginxRequest(line('GET / HTTP/1.1', { referer: 'x%27 OR 1=1--' })).matches.map((m) => m.id)).toContain(942100);
});

it('scores XSS libinjection finds (CRS 941100) in arguments and the User-Agent', () => {
for (const entry of [
line('GET /x?q=%22%3E%3Csvg%20onload=alert(1)%3E HTTP/1.1'),
line('GET /x?%3Cimg%20src=x%20onerror=alert(1)%3E=1 HTTP/1.1'),
line('GET / HTTP/1.1', { ua: '\\x22><svg onload=alert(1)>' }),
]) {
const assessment = assessNginxRequest(entry);
expect(assessment.matches.map((m) => m.id), entry.raw).toContain(941100);
expect(assessment.attackType, entry.raw).toBe('xss');
expect(attackSeverity(assessment), entry.raw).not.toBeNull();
}
});

it('leaves apostrophes, quotes and SQL words in ordinary input alone', () => {
for (const path of [
"/search?q=O'Brien",
'/search?q=O%27Brien+and+Smith',
'/search?q=rock+%27n%27+roll',
'/search?q=%22exact+phrase%22',
'/search?q=don%27t+stop+believin%27',
'/search?q=where+is+george',
'/search?q=1+or+2',
'/api/items?filter=%7B%22status%22%3A%22open%22%2C%22n%22%3A1%7D',
'/profile?email=o%27brien%40example.com',
]) {
expect(assessNginxRequest(line(`GET ${path} HTTP/1.1`, { referer: `https://example.com${path}` })).score, path).toBe(0);
}
});

it('keeps detectAttackPattern answering with the attack type', () => {
expect(detectAttackPattern('/../../etc/passwd')).toBe('path_traversal');
expect(detectAttackPattern('/topics/rochester/podcasts.rss')).toBeNull();
Expand Down
10 changes: 7 additions & 3 deletions apps/cli/src/core/crs/NOTICE
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,13 @@ Changes made: the paranoia-level-1 rules from REQUEST-913, 930, 931, 932,
933, 934, 941, 942 and 944 whose targets appear in a web server access log
were converted by apps/cli/scripts/build-crs-rules.mjs into a JSON rule table
(regular expressions translated from PCRE to JavaScript syntax, phrase files
inlined, targets an access log cannot record removed). Rules that could not be
translated faithfully were left out. The evaluation engine is ThreatCrush's
own; it is not ModSecurity.
inlined, targets an access log cannot record removed). Rules 942100 and
941100 (@detectSQLi, @detectXSS) call libinjection, which is not CRS: it ships
compiled to WebAssembly in this package's dist/libinjection/, with its own
licence (BSD-3-Clause) in COPYING there; its sources are vendored under
apps/cli/vendor/libinjection. Rules that could not be translated faithfully
were left out. The evaluation engine is ThreatCrush's own; it is not
ModSecurity.

Also derived from CRS: ThreatCrush's own rule 10001 (THREATCRUSH_RULES in
rules.generated.ts), which is not part of CRS. It applies the phrases of CRS's
Expand Down
Loading
Loading