Consolidate reviewed fixes: supersede #62/#63/#65/#66/#67/#70/#72/#74 - #77
Merged
Conversation
Four datapath-correctness fixes to protector.bpf.c, verifier-validated by loading the collection (all programs accepted): - check_rate_limit incremented a CPU-shared rate_limit value with a plain read-modify-write, losing increments under a same-source flood spread across RX queues so the count stayed under the limit. Use __sync_fetch_and_add and test the post-increment value (matching blocked_ips/policy_blocks). - bad_flags / bad_flags_v6 were plain HASH maps capped at 1000, so a spoofed scan flood filled them and further inserts failed with E2BIG, blinding the userspace signal pipeline to new scanners. Switch to LRU_HASH (like every sibling per-source map) and size via BADFLAGS_MAP_SIZE. - A frame with more stacked VLAN tags than the parser unwinds left h_proto as a VLAN ethertype and fell through to XDP_PASS uninspected. Parse up to 3 tags and fail closed (XDP_DROP, honoring monitor/dry-run) when still tagged. - meta.window was stored in network byte order while Go decodes EventMetadata little-endian, byte-swapping the TCP window used for JA4T fingerprinting. Apply bpf_ntohs like seq/ts_val.
The XDP and TC programs located the TCP header at (void *)(ip + 1), a hardcoded 20-byte IP header, ignoring the attacker-controlled IHL field. A packet carrying any IP options (IHL>5) then had its option bytes parsed as the TCP header: a SYN+FIN/Xmas/NULL scan with one IP option read benign bytes where the flags should be and evaded check_tcp_flags, and the TC path misparsed JA4T/timestamp data the same way. ipv4_tcp_header validates IHL (5..15), computes the L4 offset as ihl*4, and bounds-checks before returning the header (NULL if invalid/truncated). Applied at the XDP flag check and both TC monitor paths. Verifier-accepted; BPF_PROG_TEST_RUN confirms a SYN+FIN scan carrying an IP option is now dropped while a benign SYN with options passes.
The XDP program dispatched the IPv6 ICMPv6/UDP/fragment/TCP checks on ip6->nexthdr directly, so a single extension header (e.g. an 8-byte Hop-by-Hop Options header in front of the L4 header) left nexthdr=0 and matched none of the checks - the packet fell through to XDP_PASS. One extension header defeated every IPv6 L4 protection; a UDP/ICMPv6 flood or a SYN/Xmas scan behind a Hop-by-Hop header bypassed rate limiting and flag detection entirely. parse_ipv6_l4 walks the extension-header chain (bounded to MAX_IPV6_EXT_HEADERS, respecting each header's length, with per-step packet-bounds checks and fail-closed on a truncated/over-long chain) to find the true upper-layer protocol and header offset. All IPv6 L4 decisions now run off that result, and the TCP flag check reads the header at its real post-extension-header offset. Replaces the unused, length-ignoring count_ipv6_ext_headers helper. Verifier-accepted; BPF_PROG_TEST_RUN confirms a UDP flood behind a Hop-by-Hop header is now dropped while a single such packet still passes.
The XDP program called bpf_perf_event_output on every XDP_DROP (incidents) and on every SYN (JA4T events). Under a multi-Mpps flood that emits one perf event per packet, saturating the ring buffer - which drops events for everyone and burns CPU on the per-event copy and userspace wakeup - precisely when the host is most loaded, a self-inflicted amplification on the drop path. Add per-CPU emission budgets (incident_budget, event_budget) that cap emits to MAX_EMITS_PER_SEC_PER_CPU per 1s window. Enforcement is untouched - the packet still drops and handshake tracking still runs for every SYN - only the audit/telemetry stream is sampled. Per-CPU counters avoid cross-core contention. Verifier-accepted; BPF_PROG_TEST_RUN over 3000 dropping packets confirms every packet is still dropped while incident emits cap at the per-CPU budget (1000) instead of 3000.
…d prev-state Follow-up fixes on the eBPF datapath work: - parse_ipv6_l4 now dispatches the L4 protocol discovered at exactly MAX_IPV6_EXT_HEADERS instead of rejecting it, and fails closed (drop in enforcement, pass in monitor) on truncated/over-limit chains rather than failing open. Adds INCIDENT_MALFORMED telemetry for the drop. - check_rate_limit no longer consumes the __sync_fetch_and_add return value, which would require BPF ISA v3 / Linux 5.12; it re-reads the counter so the atomic increment stays Linux 5.4-compatible. - Bound the process-lifetime userspace prev-state maps (prevBadFlagsSeen and the rate shadows) with a kernel-clock staleness window plus a hard cap, so a high-cardinality spoofed-source flood cannot leak memory while the kernel LRU maps churn.
…dels featuresToTensorAdvanced wrote fixed offsets copy(tensor[95:105], ...) and copy(tensor[105:115], ...) unconditionally regardless of the allocated tensor length. LoadONNXModel's auto-detect probe includes sizes 100, 106, and 110 (onnx.go loadONNXModelInternal), all narrower than the fixed 115-index write, so a model auto-detected at one of those sizes panicked with a slice-bounds-out-of-range on the first prediction carrying event history, crashing the analyzer process (engine.Predict runs on the detection goroutine with no recover). Add copyTruncated, which clamps each write to the tensor's actual capacity, and route the fingerprint/behavioral feature copies through it so smaller auto-detected sizes get a truncated but valid tensor instead of a panic. Sizes >=116 are unaffected (the existing branches already guard capacity for those layouts).
…ure layout The auto-detect probe accepted any width and the extraction path silently truncated the 126-feature advanced layout to fit narrower models, feeding misaligned/corrupted feature vectors into an enforcement model. Replace the generic truncation with explicit, exact layouts for the only widths this build can construct from repository contracts (41, 116, 126) and refuse to load any other detected width (and refuse when detection fails outright) with a clear error. Add sentinel/index-semantic tests that assert the meaning of each detection-block index, not just tensor length, plus a defensive test that unsupported widths yield a zeroed tensor without panicking.
Stop() closed the eBPF collection and the mmapped GeoIP database before waiting on the WaitGroup, while pollMaps and the perf-event readers were still mid-iteration: a concurrent GeoIP.Lookup reads munmapped memory (SIGSEGV) and map iteration continues on closed FDs. Move both closes after the drain wait, keeping the perf-reader closes before it since their blocking Read() only returns on Close.
executeCommand appends to and reassigns c.allowedNets on the command-stream goroutine while checkAllowlist ranges it unsynchronized from the map-polling, perf-event, and SPOE goroutines — a torn slice header can read out of bounds. Guard the slice with a dedicated RWMutex; kernel-map updates stay outside the lock. Adds a -race regression test.
sendPendingHandshakes aggregated per-source RTT as int64(SynAckTime-BeginTime) over every pending_handshakes entry. Those entries are by definition incomplete handshakes; one still awaiting a SYN-ACK has SynAckTime==0, so the unsigned subtraction underflowed into ~1.8e19 and, cast to int64, produced a large negative RTT that poisoned the aggregated HandshakeRttNs for that source (both the IPv4 and IPv6 paths). Accumulate only handshakes with a valid (completed) RTT via handshakeRTTNanos, track their count separately, and average with avgRTTNanos, which returns 0 instead of dividing by zero when none completed.
Iterate icmp_rates_v6/udp_rates_v6 alongside the v4 maps so IPv6 floods reach the analyzer, sharing one emitFloodSignal gate for v4/v6 parity. emitFloodSignal copies the source address before retaining it: the rate-map iterators reuse a single key buffer across iterations and signals are marshaled asynchronously off signalQueue, so aliasing the key shipped every v6 signal with the last-iterated IP (and raced the poll goroutine). The signal id is now formatted post-gate from a constant prefix, keeping the per-entry allocation off the sub-threshold and allowlisted paths.
sendICMPRates and sendUDPRates shared one sentCount (capped at rateMaxBatchSize) across the IPv4 loop and the IPv6 loop. A concurrent IPv4 flood that filled the batch made the IPv6 loop break on its first iteration, so every IPv6 flooding source went unreported for that poll - the IPv6 flood detection this change adds was silently defeated whenever a v4 flood ran alongside it. Give the IPv6 loop its own sentCountV6 budget so neither family can starve the other; the per-poll signal cap now applies per address family. The debug log reports the combined count.
The emitSignal callback resolves to Collector.sendSignal, which enqueues via a non-blocking select with ring-buffer drop — it can never block the SPOE handler. Wrapping it in a goroutine per HTTP request only added unbounded transient goroutine spawn under load and reordered signal delivery. Call it inline.
…h test cleanup Follow-up fixes on the collector service work: - pruneStaleState now also bounds prevICMPRatesV6/prevUDPRatesV6 so the IPv6 rate shadows can't grow for the process lifetime while their kernel LRU maps churn, matching the IPv4 behavior. - sendSignal is now truly non-blocking under concurrent producers. The old ring-buffer path dropped the oldest entry and then did an unconditional blocking send, which deadlocked a caller when other producers refilled the freed slot first - stalling synchronous SPOE callbacks and, with them, HAProxy request handling and shutdown. It now retries the enqueue non-blocking and sheds the new signal instead of blocking. - scripts/xdp_veth_test.sh picks per-run-unique veth/netns names, refuses to adopt pre-existing resources, tracks what it created, deletes only owned resources on cleanup (never a pre-existing yeet0/yeetns), and creates the bin/ output directory before building. - Add regression tests for the non-blocking contract and IPv6 prune.
NewLimiter only substituted defaults when IPRate was zero, so a partial config (IPRate set, CleanupInterval unset) reached cleanupLoop with a zero interval and panicked in time.NewTicker on a background goroutine, taking down the process. Clamp every field to its default when unset or non-positive.
- AllowASN no longer buckets on the "Unknown"/empty ASN sentinel that pkg/geoip returns for unresolved lookups. Previously every client with an unresolved ASN shared one TokenBucket, so an attacker could drain it and collaterally block unrelated victims (W5). - ipLimiters/asnLimiters now enforce a hard entry cap (MaxIPEntries/ MaxASNEntries, default 200000/50000), evicting the least-recently- used bucket on overflow instead of growing unbounded between the 5-minute idle cleanup passes (W13). - TokenBucket gained a setRate method, and SetIPRate/SetASNRate now apply the new rate/capacity to every already-tracked bucket, not just buckets created after the call (W27).
Two hot-path costs, one per signal from every collector stream: - AllowIP/AllowASN took the exclusive limiter lock even when the token bucket already existed (the overwhelmingly common case), serializing all signal processing. Use an RLock fast path with double-checked insert under the write lock. - checkRateLimit called GetStats (another lock round-trip) plus two Prometheus gauge writes per signal just to publish activity gauges. Publish them from a 10s ticker next to the tracking-map cleanup instead; enforcement behavior is unchanged. benchstat (10 runs, i7-12700K), parallel Allow hit path: 361.8ns -> 107.1ns (-70.4%) Also adds the package's first unit tests (burst exhaustion deny paths, nil-input allows, concurrent access under -race).
…t-zero rates Follow-up fixes on the rate limiter work: - evictOldestLocked no longer scans the whole map under the exclusive lock on every insert-at-cap, which an adversarial distinct-source flood turned into an O(n) serialization point. It now samples a small constant number of entries (Redis-style approximate LRU) and evicts the oldest of the sample, O(evictionSampleSize) regardless of map size. For maps smaller than the sample size this remains exact LRU. - setRate now settles token accrual for the elapsed interval at the OLD rate and advances lastSeen before installing the new rate/capacity, so the window straddling a rate change is credited at the rate actually in effect, then clamps to the new capacity. - An explicitly configured zero IP/ASN rate is preserved instead of being silently defaulted. Optional IPRateExact/ASNRateExact pointer fields carry that intent (nil keeps the historical default-on-zero behavior), so existing callers are unaffected. - Add regression tests for bounded high-cardinality eviction, old-rate accrual on rate change, and explicit-zero-rate semantics.
trackBlocked writes package-level maps from the per-collector gRPC stream handler goroutines with no synchronization. Concurrent rate-limit hits from multiple collectors trigger Go's "concurrent map writes" fatal error and crash the analyzer. Guard both maps with a mutex and add a -race regression test.
GetPattern now returns a complete deep copy via a ConnectionPattern.Clone() method that copies every reference field, including the previously-aliased IP slice, so a caller mutating the snapshot cannot corrupt the live tracked pattern. RecordConnection also copies the caller's IP on first sighting, since the passed net.IP may be backed by a reused buffer. The per-signal ML feature path uses a new narrow PatternSummary accessor that reads only the four fields it needs under the read lock, instead of deep-copying the pattern's nine slices/maps on every signal.
- pkg/patterns/tracker.go: RecordConnection's inter-connection timing was dead code - PacketTimings was only appended inside a guard requiring itself to be non-empty (so it could never bootstrap), and even then it diffed against a LastSeen the same call had just overwritten to "now" (~0 delta). Compute the delta against the previous LastSeen before overwriting it, seeding on the first call instead of recording a bogus sample, so detectMechanicalTiming and the IsBursty ML feature finally get real data. - pkg/analyzer/reputation/reputation.go: decay() and pruneIfNeededLocked deleted aged-out/low-score/evicted entries from sh.entries but never the parallel sh.asnStats (Seen/Offenders IP sets), which was only reconciled inside SetASNScoreCap - a method production never calls. Every ASN ever observed therefore retained up to ~2*maxASNHosts IP strings indefinitely. Added deleteEntryLocked to keep an ASN's asnStats in lockstep with its Entry lifecycle wherever an entry is removed. - pkg/analyzer/baseline/asn_baseline.go: RecordObservation folded every sample - including attack traffic - into the same Welford stats used to score it, with no gating, so a sustained attacker within an ASN could drag the baseline's own mean toward the attack rate and normalize identical follow-up traffic as non-anomalous. Once a metric's baseline is past warmup, samples that already score beyond the 3-sigma anomaly threshold against it are now excluded from folding into that baseline; warmup behavior (before minObservations) is unchanged.
reputation.New initialized only the ASN score cap to +Inf; the per-IP and per-JA4 caps were left at the zero value, so penalizeLocked clamped every IP and JA4 penalty back to 0. All per-IP and per-JA4 reputation scoring was therefore a silent no-op in production (nothing calls SetIPScoreCap/SetJA4ScoreCap), leaving only ASN scoring functional. Default all three caps to +Inf so IP/JA4 penalties accumulate; operators can still set an explicit ceiling. This re-activates dormant per-IP/JA4 reputation accumulation, so it is documented as an enforcement-impacting change with dry-run staging guidance in CHANGELOG and docs/operations.md.
Once a shard reached its per-shard budget, pruneIfNeededLocked evicted exactly the excess, so the next new-key Penalize was over budget again and re-ran a full-shard collect+sort (~15.6k entries at the default budget) under the shard write lock — ~1.5ms and ~365KB per signal under a distinct-IP flood, reintroducing the contention the sharding work removed. Evict down to ~90% of budget per pass so the sort runs once per ~budget/10 inserts. benchstat (6 runs, i7-12700K), new-key Penalize at cap: 1508.7us -> 2.2us (-99.9%), 366KB/op -> 527B/op Eviction stays oldest-first by LastSeen; shards now idle between 90% and 100% of budget instead of pinned at 100%.
Follow-up fixes on the reputation/patterns/baseline work: - The ASN baseline self-poisoning guard rejected any >3-sigma sample forever, so a sustained *legitimate* shift (new egress hardware, region, migrated services) scored anomalous indefinitely while LastSeen kept the stale entry alive. Rejected samples now fold into a per-metric shadow candidate, and the candidate replaces the baseline only once it is both self-consistent and has persisted past strict sample-count (300) and wall-clock (30m) gates. A transient attack ends long before those gates, so it still can never converge the baseline to its own rate; a permanent shift eventually does. The candidate resets whenever traffic returns to the current baseline. - ML IsBursty was computed as len(PacketTimings) > 5, i.e. just 'has more than a few samples', not burstiness. It now derives from the coefficient of variation of the inter-connection gap distribution (CV > 1 = clustered/ irregular spacing), computed in nanoseconds so it stays meaningful for sub-millisecond flood gaps. Exposed via PatternSummary.Bursty. - Add regression tests for convergence (adopt/reject/flap) and burstiness.
W1/W14: a learned-legitimate pattern short-circuited handleDetection with a bare return placed before the DDoS and honeypot evaluation, and CheckPattern matches on client-controlled UA/JA4H/ASN alone. So a client reproducing a whitelisted UA string or ASN could skip all L7 detection, including flood and honeypot handling. The DDoS verdict is now computed up front and the legitimate short-circuit is gated on it plus a deterministic-high-severity check, so an allowlist match no longer suppresses a honeypot hit, a JA4H known-bot match, or a DDoS pattern. W6: honeypot and JA4H known-bot signals top out below the 0.80 rule-confidence floor blendMLConfidence protects, so a strong-legitimate ML verdict capped them to 0.40 and they were not enforced. Lift rule confidence to that floor when a deterministic high-severity signal is present (mirroring the known-scanner boost). W15: checkAdaptiveDetection failed open when its EWMA map filled (returned no detection for new keys); the key is client-controlled, so a churn of distinct identifiers disabled adaptive detection for everyone. Evict the oldest entries in a batch instead. W20: AccumulateLearningData grew UniquePaths without bound and re-ran the crawl scan on every event; cap the set and skip the rescan once the pattern is found.
…rity containsDeterministicHighSeverity flagged every SignalJA4HBotMatch as deterministic ground truth, which bypasses the learned-legitimate allowlist and floors confidence to strongRuleConfidenceFloor. But a JA4DB *wildcard* (coarse-prefix) match is not a reliable attribution, so a legitimate client whose fingerprint merely collides on the a+b prefix became unblockable-out (no allowlist suppression, confidence floored past the threshold). Gate deterministic status on Metadata["match_type"] == "exact". A wildcard match still contributes its weight to scoring, so the anti-evasion intent is preserved; it is just suppressible again. Absent/non-string match_type fails closed to "not deterministic" (fewer false-positive blocks). Honeypot stays unconditionally deterministic.
processSignal recomputed signal.IP.String() up to six times and read the clock four times per signal; baseline.shardFor allocated a hash.Hash32 plus a []byte copy per call (invoked twice per signal via RecordObservation and CalculateAnomaly), and CalculateAnomaly built a 9-element slice per call just to take a max. Hoist the IP string and timestamp once per signal, inline an allocation-free FNV-1a in shardFor, and use the max builtin. benchstat vs main (10 runs, i7-12700K), BenchmarkProcessSignalHotKey: sec/op 90.95m -> 87.51m (-3.8%, p=0.005) allocs/op 165.2k -> 145.2k (-12.1%, p=0.000) No behavior change: shard assignment uses the same FNV-1a function, route keys are byte-identical, and MaxZScore is unchanged (all inputs are absolute values).
Batch eviction (evict the overflow plus a headroom margin in one sorted pass) replaces the per-insert O(n) oldest-scan on the clockskew, entropy, and pattern profile maps, so a spoofed-source flood driving these caches to their cap pays the scan once per batch instead of on every packet behind the map lock. The batch-size formula and its headroom fraction, previously copy-pasted at each call site, now live once in mapcleaner.EnforceMaxSizeBatch behind a named batchEvictHeadroom constant. The crawler-verification cache (a per-signal insert path a crawler-UA flood drives to its cap) is routed through the same helper, replacing its one-entry-per-insert eviction.
…istic JA4H Follow-up fixes on the AI-detection engine work, addressing the reputation-vs-ML interaction (#66) and the deterministic-evidence semantics (#70): - CalculateConfidence ignored its reputationScore parameter entirely, so accumulated bad-actor history could never influence confidence and a strong-legitimate ML verdict fed sparse inputs permanently vetoed it. It now contributes a bounded, threshold-graded term, and handleDetection floors confidence (and skips learned-legitimate suppression) for a source that has crossed the reputation ban threshold - the same protection honeypot/JA4H deterministic evidence already had. Added a public reputation.BanThreshold(). - Deterministic high-severity now requires a CONFIRMED known-bot, EXACT, JA4H match. The producer tags the JA4DB signal with known_bot, and the consumer requires known_bot && fp_type==ja4h && match_type==exact. Exact JA4/JA4T matches, non-bot JA4H matches, and wildcard collisions still contribute their weight to scoring but no longer bypass learned-legitimate suppression or floor confidence. - Add end-to-end tests: a reputation-bad actor blocks while an identical clean source does not under strong-legit ML; a known-bot exact JA4H blocks while a non-bot JA4H and an exact JA4 remain vetoable. Updated the consumer unit test for the stricter contract.
The prune loops in updatePathEntropy and trackHTTPErrors break on "len(events) <= maxEvents", but the slice is only re-sliced after the loop, so len is constant per call. Once a window exceeds the cap the break condition can never hold and every event is pruned, zeroing the accumulated path-entropy counts and 404/403 tallies. A scanner sustaining more than maxEvents in-window periodically resets its own tracking. Compare the remaining count (len(events)-i) instead, and add boundary tests for both windows.
…ttern feed
- Key each collector stream by peer address plus a monotonic sequence
instead of the never-set ctx.Value("collector-id") constant, so
Broadcast reaches every connected collector and a disconnect no longer
evicts a still-live sibling's stream.
- Hoist the per-IP block-command dedup out of the per-collector send so a
broadcast is deduped once per block decision, not suppressed for every
collector after the first.
- Stop honoring penalty_key/penalty_type from wire metadata: the signal
plane is unauthenticated gRPC, so an arbitrary target let any peer
poison a third party's reputation. Penalty requests now only penalize
the signal's own source IP and are dropped (fail closed) when the
signal carries no source IP.
- Feed IsIncompleteHandshake to the pattern tracker from
SIGNAL_INCOMPLETE_HANDSHAKE and document (SIMPLIFIED) which pattern
detectors remain inert because TCPContext lacks the proto fields.
…fable penalty Follow-up fixes on the analyzer service work: - Broadcast no longer reserves the block dedup slot when there are zero connected collectors. Reserving marked the IP recently-blocked for the TTL even though nothing enforced it, so a collector connecting within the window had the re-issued block suppressed and never received it. It now snapshots recipients first and only reserves/sends when at least one exists. - The penalty_key/penalty_type/penalty_reason wire metadata path is disabled. The signal plane is unauthenticated and the signal's own source IP is itself attacker-controlled, so honoring a metadata-driven penalty (even bound to the source) let any peer poison an arbitrary victim's reputation with an unvalidated, possibly non-finite weight. No legitimate collector emits these fields; reputation is driven only by signals the analyzer actually scores. - Collector admission is bounded by a new -max-collectors limit (default 1024) so an unauthenticated peer cannot grow the collectors map, its per-stream goroutines, or the Broadcast fan-out without limit; over-cap streams are refused with ResourceExhausted. - Add regression tests and document the new flag.
deleteSessionFromDisk globbed sessions_*.jsonl, but recordings are persisted as recording-*.jsonl (the pattern loadSessionsFromDisk reads), so /api/sessions/delete always returned "session not found". Use the right glob, remove the file outright when its only session is deleted, and don't rewrite a file that wasn't fully read. Also close each recording file per loop iteration in loadSessionsFromDisk instead of deferring all closes to function return, and surface scanner errors.
The proxyLag block shadowed the request's already-computed org with a
second GeoIP lookup and set ProxyLagEWMAByASN twice per request: once
with the shadowed org carrying the raw (non-EWMA) lag, and once with
the outer org carrying the EWMA. The two label sets can differ
("unknown" vs "Unknown" fallback), producing duplicate label-distinct
series where one contradicts the metric's EWMA semantics. Drop the
shadowed lookup and first Set, and guard the EWMA Set with the same
known-ASN check the removed block had.
W8: registerInspectorHandlers exposed unauthenticated, state-mutating routes (allowlist removal, learning-window clear, session/label deletion, forced recording) with no CSRF or DNS-rebinding protection. Since json.Decode ignores Content-Type, a cross-origin text/plain POST was a CORS-simple request that fired these side effects from any page the operator had open. Add a sameOriginOnly middleware that rejects requests with a non-loopback Host (blocks DNS rebinding) or a cross-origin Origin/Referer (blocks CSRF), wrap it around all state-mutating inspector routes. Same-origin browser use and Origin/Referer-less tooling (curl, scripts) are unaffected. W11: the JA4 wildcard fallback (EntriesByJA4Prefix) indexed and matched on only the coarse 'a' segment of a JA4 fingerprint, shared by thousands of unrelated clients, returning an arbitrary same-prefix entry as a positive "wildcard_tls" match. Require the 'a'+'b' segments instead. Also gate service_http.go's browser-signal consumption (SignalBrowserDetected and the ja4_info metadata CategorizeBot trusts verbatim to skip bot detection) to exact matches only, consistent with the existing RewardBrowser exact-gate.
JA4DB downloads, AI-crawler IP lists, and Shodan InternetDB responses were read with io.ReadAll and no size cap, so a compromised or hostile upstream could return an arbitrarily large body and drive unbounded analyzer memory growth (timeouts bound time, not bytes). Add a shared limitread.ReadAll helper that errors past a cap and size each source: 128 MiB for JA4DB feeds, 16 MiB for crawler lists, 1 MiB for InternetDB.
…large-session reads Follow-up fixes on the HTTP/inspector/JA4DB work: - The inspector same-origin guard was loopback-only, which blocked legitimate reverse-proxy/trusted-management deployments, and it ran for every method, so it could gate a read-only GET. It now enforces only for mutating methods (POST/PUT/PATCH/DELETE) and accepts loopback plus operator-configured -inspect-trusted-hosts for the Host and Origin/Referer checks; read-only GETs pass through untouched. DNS-rebinding and cross-origin CSRF protection for mutating requests is preserved. - Session-recording reads used bufio.Scanner's default 64 KiB line limit, so a normal-but-large JSONL record errored the scan - which dropped the session from listings and made a delete of that recording silently skip the file. A 16 MiB explicit scanner buffer covers real recordings. - The JA4 wildcard-vs-exact browser-trust gating from this PR combines with the stricter deterministic JA4H consumer: a wildcard match is neither trusted as a browser nor treated as deterministic known-bot evidence. - Retained both the #76 (FindByHeadersPrefix) and this PR's JA4 wildcard tests. - Add regression tests and document the new flag.
Residual hardening from the management-socket work: - The socket is now created 0600 atomically by tightening umask across the bind, closing the window between net.Listen (which creates it 0777&^umask) and the follow-up chmod where another local user could connect and read the blocked-IP set. The chmod is kept as belt-and-suspenders. - Refuse to create the socket in a world-writable parent directory without the sticky bit (a symlink-swap TOCTOU vector), and warn on a group-writable one. - Tests assert the socket is mode 0600 and that a world-writable non-sticky parent directory is rejected while a sticky one is accepted.
Residual fixes from the campaign-detection work: - Campaign IDs incorporated only the stable key, so two unrelated attack episodes that reused the same key (after the first fully aged out) were conflated under one campaign_id. Each campaign instance now carries an episode token assigned once at creation and folded into the ID: stable within an episode, distinct across episodes, still immune to the firstSeen slide within a continuous campaign. - The guard that stops a detected campaign from training its own corroborating baseline keyed on the detection reason, so a campaign whose reason flapped (e.g. alternating destination_ip_breadth / source_ip_breadth) retrained the baseline every tick and normalized its own attack rate. Baseline observation is now gated purely on elapsed time (at most once per Window per campaign), decoupled from detection emission, so reason flapping can no longer self-poison it. - Add lifecycle tests: distinct IDs across episodes end-to-end, and no baseline retraining under sustained reason flapping.
…S deadline Residual fixes from the bot-verification DNS work: - The gRPC VerifyBot RPC recomputed impersonation as 'known bot UA and not verified', ignoring the transient-failure forgiveness the HTTP handler applies. A real crawler with a briefly-flaky PTR was reported over gRPC as an impersonator. Both paths now share a single VerificationResult. IsForgivenTransientFailure() rule: a transient DNS failure within the cap is not impersonation; a definitive failure or an over-cap transient one is. - verifyDNS gave each DNS lookup (reverse PTR + forward-confirm) its own dnsTimeout, so the total budget was N*dnsTimeout and a hostile PTR zone could multiply the stall by chaining lookups. It now creates one shared dnsTimeout-bounded context covering the whole verification; the lookup seams take that context. - Add tests: the shared-deadline total budget, and the forgiveness rule table.
make e2e-ebpf-test failed to load tc_ingress_syn_monitor: 'BPF program is too large. Processed 1000001 insn'. clang compiles fine; the verifier rejects the load. Root cause: parse_tcp_timestamp walks TCP options in an unrolled loop that is inlined into the TC ingress SYN paths for both IPv4 and IPv6. PR #63 changed the IPv4 caller to locate the TCP header via the variable IHL offset (ipv4_tcp_header) instead of a fixed 20-byte offset - correct for reading flags past IPv4 options, but it feeds a variable-offset base into that loop. The per-iteration branch states of a variable-length option walk grow steeply with the iteration count; at 10 iterations (inlined twice) the accumulated state exploration tipped the program past the verifier's 1M-instruction limit. On current main the same 10-iteration walk verified fine only because it ran from a constant offset. A noinline BPF-to-BPF subprogram is not an option: passing PTR_TO_PACKET to a subprogram is only reliably verifiable on Linux 5.10+, and this project targets Linux 5.4+. Fix: bound the option walk to the first TCP_TS_MAX_OPTIONS (8) options. Real TCP SYNs place the timestamp option within the first few options (Linux/Windows/ macOS all at or before ~6), so this still finds it in practice, while keeping all of PR #63's intent intact - the IHL-correct header is still used for SYN/flags detection, handshake tracking, AND timestamp parsing (including for IPv4 option-bearing packets), IPv6 handling, rate limits, byte order, incident sampling, and monitor/enforcement behavior are unchanged. parse_tcp_timestamp stays inlined (5.4-safe). Validated by loading the compiled object against a real kernel in a privileged container: tc_ingress_syn_monitor drops from >1,000,001 to 250,774 processed insns (xdp_filter 17,857; tc_egress 109), all well under the 1M limit. The verifier's processed-insn count for this object was identical on that kernel and the GitHub CI kernel (both hit exactly 1,000,001 before the fix), so the margin transfers. Kernel eBPF load itself still requires the CI e2e-ebpf job.
awlx
force-pushed
the
awlx-consolidate-reviewed-fixes
branch
from
August 8, 2026 13:30
ebc22d1 to
d61ad17
Compare
The CI vulnerability scan (govulncheck) flagged GO-2026-6061 - vulnerabilities in the gRPC HTTP/2 transport server implementation - reachable through the collector/analyzer signal streams. It is fixed in grpc v1.82.1, an API-compatible patch release. Pre-existing on main; bumped here so the build-test job is green. govulncheck reports no vulnerabilities after the bump; builds and portable tests pass.
This was referenced Aug 8, 2026
Closed
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This single branch supersedes the remaining open PRs #62, #63, #65, #66, #67, #70, #72, #74 and also folds in residual follow-ups from already-merged #64, #69, #73. It is based on current
main.Each original PR's authored commits are cherry-picked (preserving @Tumult1337's attribution) and the confirmed review issues are fixed in focused follow-up commits on top.
Validation
gofmtclean on all changed Go files;git diff --checkclean;go vetclean../pkg/analyzer/... ./pkg/ml/... ./pkg/integration_test ./pkg/collector ./cmd/yeetctl ./pkg/patterns/... ./pkg/ratelimit/... ./pkg/utils/....-racegreen on the concurrency-sensitive packages (collector,analyzer,aidetection,ratelimit).analyzer,yeetctl,collector(host) build;collectorcross-compilesGOOS=linux GOARCH=amd64.This was validated on macOS, so the Linux/eBPF verifier and kernel-runtime paths were not exercised.
make bpffails locally only because Linux headers (linux/bpf.h) are absent — expected. The following still need to run on a Linux host with the eBPF toolchain in CI or manually:make bpf(compileprotector.bpf.c; confirms the reworked IPv6 ext-header loop and the 5.4-compatible atomic pass the verifier).make collector && make test(Linux collector build + tests).sudo ./scripts/xdp_veth_test.sh(veth E2E; the script's resource-safety rewrite is exercised here).make e2e-ebpf-testwhere applicable.Superseded PRs → retained intent + fixes
#62 collector service — shutdown ordering, allowlist race, RTT guard, IPv6 floods, inline SPOE. Fixes: bound/expire
prevICMPRatesV6/prevUDPRatesV6(andprevBadFlagsSeen*) with a kernel-clock staleness window + hard cap so a high-cardinality flood can't leak Go memory while kernel LRU maps churn;sendSignalis now truly non-blocking under concurrent producers (the old ring-buffer path could block a synchronous SPOE callback when producers refilled the freed slot);scripts/xdp_veth_test.shnow uses per-run-unique veth/netns names, tracks only resources it created, never deletes a pre-existingyeet0/yeetns, and creates itsbin/output dir.#63 eBPF datapath — IHL offset, IPv6 ext-headers, byte order, atomic counts, incident sampling. Fixes:
parse_ipv6_l4now dispatches the L4 protocol discovered at exactlyMAX_IPV6_EXT_HEADERSand fails closed (drop in enforcement, pass in monitor) on truncated/over-limit chains instead of failing open, with a newINCIDENT_MALFORMEDtelemetry code;check_rate_limitno longer consumes the__sync_fetch_and_addreturn value (that requires BPF ISA v3 / Linux 5.12) so it stays Linux 5.4-compatible; deep-VLAN fail-closed policy retained and documented as honoring monitor mode.#65 rate limiter — independent config defaults, ASN-sharing fix, bounded maps, contention cut. Fixes: replaced the O(map) LRU scan under the exclusive lock with Redis-style sampled eviction (O(sample), amortized-bounded under adversarial cardinality; still exact LRU for small maps);
setRatenow settles accrual at the OLD rate and advances the timestamp before installing the new rate/capacity; explicit zero IP/ASN rates are preserved via optionalIPRateExact/ASNRateExact(nil keeps the historical default-on-zero, so existing callers are unaffected).#66 reputation/patterns/baseline — score-cap defaults, deep-copy snapshots, stateful fixes, batch prune. Fixes: the reputation threshold path is now reachable under the default ML integration —
CalculateConfidenceactually usesreputationScore(it was ignored), and a source past the reputation ban threshold floors confidence so a strong-legitimate ML verdict can't veto it; the ASN baseline permits controlled convergence to a sustained, coherent legitimate shift (shadow candidate gated by sample-count and wall-clock so a transient attack still can't self-poison it) instead of rejecting >3σ forever; burstiness is computed from the coefficient of variation of the inter-connection interval distribution, notlen(PacketTimings) > 5.#67 HTTP/inspector/JA4DB — CSRF+DNS-rebind guard, JA4 wildcard matching, bounded reads, session-delete fix. Rebased on current
main. Fixes: exact-match trust only — a wildcard JA4 result is never treated as a trusted browser nor as deterministic known-bot evidence (match type carried through); the inspector guard now enforces only on mutating methods (read-only GET is never gated) and accepts loopback plus configurable-inspect-trusted-hostsfor reverse-proxy/trusted-management deployments; large session recordings are read with an explicit 16 MiB scanner buffer so a normal-but-large record no longer breaks listing or deletion; retained both the merged-#76 (FindByHeadersPrefix) and this PR's JA4 wildcard tests.#70 AI detection — don't erase strong detections; only exact JA4H matches are deterministic. Fixes: deterministic high-severity now requires a confirmed known-bot, exact, JA4H match — the producer tags the signal with
known_bot, and the consumer requiresknown_bot && fp_type==ja4h && match_type==exact. Exact JA4/JA4T matches, non-bot JA4H matches, and wildcard collisions still contribute weight but no longer bypass learned-legitimate suppression or floor confidence. End-to-end tests included.#72 ONNX — fix panic for 100/106/110-feature models. Fixes: replaced generic truncation of the 126-feature layout with explicit, exact layouts for the only widths provable from repository sources (41, 116, 126); any other detected width (100/106/110/144/…) is rejected at load rather than fed corrupted features (and load is rejected when width detection fails outright). Added sentinel/index-semantic tests, not just length/non-panic.
#74 analyzer service — window-prune flush-at-cap, stream keying, penalty-metadata trust. Fixes:
Broadcastno longer reserves the block dedup slot when there are zero connected collectors (which would suppress the block for a reconnecting collector); the wire-controlledpenalty_keymetadata path is disabled — the signal plane is unauthenticated andSignal.ipis attacker-controlled, so it could poison an arbitrary victim's reputation with an unvalidated weight (no legitimate collector emits it); collector admission is bounded by a new-max-collectorslimit (default 1024) so an unauthenticated peer can't grow the collectors map, its goroutines, or the broadcast fan-out without limit.Residual fixes from merged PRs
#64 management socket — created 0600 atomically via a tightened umask (closing the connect window between
net.Listen's0777&^umaskand the follow-up chmod), and creation is refused in a world-writable non-sticky parent directory (symlink-swap TOCTOU), with a group-writable warning. Tests assert mode 0600 and parent-dir rejection.#69 campaign detection — campaign IDs are stable within an episode but distinct across episodes (an episode token assigned once at creation, immune to the
firstSeenslide); a detected campaign can no longer train its own corroborating baseline even when its detection reason flaps (baseline observation is gated purely on elapsed time, decoupled from emission). Lifecycle tests included.#73 bot verification — the gRPC
VerifyBot.IsImpersonationnow honors the same transient-DNS-failure forgiveness as the HTTP handler via a sharedVerificationResult.IsForgivenTransientFailure()rule (a real crawler with a briefly-flaky PTR is not reported as a spoofer);verifyDNSuses a single shared deadline across the reverse + forward lookups instead of a per-lookup timeout, so a hostile PTR zone can't multiply the stall. Tests cover the total budget and the forgiveness table.Notes
-max-collectors,-inspect-trusted-hosts), the supported ONNX widths, and the inspector guard behavior.