v1.209.1 BotScan Go matcher — restore broken prefilter + fix srv3 collector OOM (helper-first)

[itcmsgr]

v1.209.1 — BotScan prefilter correctness + bounded matcher (Track-A completion hotfix)

Completes v1.209 Track-A by unblocking srv3 and fixing a fleet-wide prefilter-correctness defect the srv3 canary uncovered. Helper-first; nftband daemon byte-identical; nft schema 1.84.0 unchanged; no VERSION bump (release-prep is separate).

Root cause

The BotScan pattern file uses | as its field delimiter, but 3 patterns contain | alternation in field 2 (EXP_CGIBIN /cgi-bin/.*\.(sh|pl|cgi), EXP_SQLBACKUP, SCAN_BACKUP_SQL). The shell IFS='|' read mis-splits them into invalid regex (/cgi-bin/.*\.(sh), and GNU grep 3.6 rejects the entire grep -E -f "$_pf" file (rc=2). The shell ran it as grep … 2>/dev/null || true, so the prefilter silently returned empty → process_entry ran zero times → pattern-based BotScan detection (webshell/exploit/scanner/badbot) has been dead fleet-wide. (The 404/endpoint-flood path is separate and still produced signals/backlog.) Confirmed live on srv2 (prefilter on by default, patterns installed, grep rc=2).

Corrected corpus facts

Real corpus = 155 patterns (earlier 775/137 were worktree-duplicate / wrong-column artifacts): 98 pure-literal + 57 regex-requiring (char-class/quantifier/anchor; 0 backref/lookaround → all RE2-compatible). 3 are the |-mis-split alternation patterns.

Implementation

• internal/botscanmatch/ — Aho-Corasick literal prefilter (one anchor/pattern) + RE2 confirm so output == grep's matching-line set. Internal impl, no external dependency. Bounded memory (streaming, bounded automaton) → fits srv3's MemoryMax=256M, no whole-file slurp.
• cmd/nftban-botscan-matcher — helper (--filter/--check/--selftest/--version). Skips the 3 broken patterns visibly (--check → "152 usable, 3 skipped"; stderr count); runs the 152 valid.
• Shell wiring: prefilter engine resolved once/cycle via --check; replaces grep -E -f. build.sh + RPM/DEB install + ci-go build.

Restored-detection impact (must read)

v1.209.1 restores intended pattern-based BotScan detection for the 152 valid patterns. This may INCREASE legitimate BotScan bans, because previously-suppressed detections (real webshell/exploit probes) become active. This is correct/intended, not new behavior — the prefilter is supposed to filter, not error-to-empty.

Safety / fail-safe

On any helper problem (missing binary, zero usable patterns, compile failure), the shell falls through to unfiltered pass-through — the authoritative bash match_url_g matcher still runs on all lines (detection preserved, slower). It never silently empties like the old grep path. Skip count is surfaced once per cycle.

Package-native validation (lab2 DEB + lab4 RPM — both PASS)

• helper installed+executable; --version 1.0.0; --selftest rc0; --check → "152 usable, 3 skipped".
• Restoration proof: grep -E -f rc=2 / kept=0 (broken) vs helper kept=4 (restored); webshell c99.php kept.
• Parity: helper == grep(valid $_pf) byte-identical.
• Shell wired; fail-safe pass-through present; validate rc0; failed units 0; schema 1.84.0; BotGuard disabled.
• Daemon byte-identical: go list -deps ./cmd/nftband does not include botscanmatch; no cmd/nftband change. (Installed nftband sha differs only as a non-reproducible-build artifact — not a re-baseline.)

Non-scope

No VERSION bump · no schema/validator-JSON · no daemon logic · no source_index/firewall/ban-path/threshold/action change · no BotGuard default flip · no MalwareGuard/decision-cache · no full BotScan-in-daemon migration · the 3 delimiter-broken patterns are NOT activated (pattern files unchanged).

Follow-up

OPEN_BOTSCAN_PATTERN_DELIMITER_FIX — fix the pattern-file serialization/loader so |-containing patterns aren't mis-split, then re-enable the 3 alternation patterns with parity fixtures. Tracked separately; not in this PR.

[github-actions]

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

Scanned Files

None