feat: daily security scan — pip-audit + bandit + gitleaks (1.0.0-24)
Some checks are pending
Security scan / pip-audit (push) Waiting to run
Security scan / bandit (push) Waiting to run
Security scan / gitleaks (push) Waiting to run

Two layers of defence-in-depth scanning:

* `.forgejo/workflows/security-scan.yml` — runs pip-audit, bandit, and
  gitleaks on every push, every PR, and nightly at 07:00 UTC. Activates
  when the forge has a runner; harmless no-op until then. Bandit is
  invoked with `--skip B608` because every dynamic SQL build in this
  codebase uses bound parameters for data and structural placeholders
  only — we still catch real injection through code review.

* `scripts/security-scan.sh` + systemd `service`/`timer` — maple-side
  daily scanner that runs the same three tools entirely in containers
  (no host pollution). Differences from the forge job:
    - pip-audit runs INSIDE the live container against installed
      packages, catching new CVEs in transitives requirements.txt
      doesn't pin (e.g. starlette breaking changes shipping in 1.0).
    - bandit scans the LIVE deploy dir at
      ~/docker/stacks/truenas-burnin/app/, not a fresh git checkout —
      so drift between forge HEAD and prod surfaces here too.
    - gitleaks scans a managed clone in ~/scan-checkouts/, kept
      fast-forward to origin/main.
  Output: ~/security-scans/scan-YYYY-MM-DD/{summary,pip-audit,bandit,
  gitleaks}.txt with 30-day retention. ~/security-scans/findings.log
  appended on any non-zero exit. SECURITY_SCAN_WEBHOOK env in the
  service unit lets you POST findings to Mattermost / Slack / etc. once
  you decide where alerts should land.

First-run findings already actioned in this commit:

* pip-audit caught 3 CVEs in `pip` itself (CVE-2025-8869,
  CVE-2026-1703, CVE-2026-3219). Dockerfile now upgrades pip to
  >=26.0 before installing the rest.

* bandit's B608 SQL-injection heuristic flagged two f-string SQL
  constructions in `_upsert_drive` and `_fetch_drives_for_template`.
  Both were structural concatenation (column-list selection,
  '?,?,?' placeholder count), not data interpolation, but refactored
  from f-string to explicit concatenation so a future reviewer
  doesn't have to relitigate.

* bandit's B104 (binding to 0.0.0.0) annotated with inline `# nosec
  B104` — container deliberately binds all interfaces; nginx-proxy-
  manager fronts it.

* gitleaks: 0 secrets across 14 commits. Clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Brandon Walter 2026-05-02 17:07:22 -04:00
parent c589e3c8e5
commit 1a19252019
8 changed files with 239 additions and 19 deletions

View file

@ -0,0 +1,61 @@
name: Security scan
# Runs on every push to main, every PR, and nightly at 07:00 UTC (~03:00 EDT).
# Three jobs run in parallel — failure of any one fails the workflow,
# making findings visible in the forge UI.
#
# Tools:
# pip-audit — known CVEs in pinned dependencies (PyPI advisory DB)
# bandit — Python static security analysis (subprocess, eval, etc.)
# gitleaks — secrets in git history (full repo scan)
on:
push:
branches: [main]
pull_request:
schedule:
- cron: "0 7 * * *"
workflow_dispatch:
jobs:
pip-audit:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: Install pip-audit
run: pip install --upgrade pip-audit
- name: Audit requirements.txt
run: pip-audit --requirement requirements.txt --strict --format=columns
bandit:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: Install bandit
run: pip install --upgrade bandit
- name: Static security analysis
# B608: SQL string construction. All dynamic SQL in this repo uses
# bound parameters for data; the dynamic part is structural
# (column lists / IN-clause '?,?,?' placeholders). Reviewed.
run: bandit -r app -ll -ii --skip B608 -x app/__pycache__,tests
gitleaks:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Install gitleaks
run: |
curl -sSfL https://github.com/gitleaks/gitleaks/releases/download/v8.21.2/gitleaks_8.21.2_linux_x64.tar.gz \
| tar -xz gitleaks
chmod +x gitleaks
- name: Scan git history for secrets
run: ./gitleaks detect --source . --no-banner --redact --verbose

View file

@ -2,6 +2,11 @@ FROM python:3.12-slim
WORKDIR /opt/app WORKDIR /opt/app
# Bump pip to a version with no known CVEs before installing anything.
# Without this, pip-audit flags CVE-2025-8869, CVE-2026-1703, CVE-2026-3219
# in pip itself. Pinned floor; pip is forward-compatible across 26.x.
RUN pip install --no-cache-dir --upgrade "pip>=26.0"
COPY requirements.txt . COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt RUN pip install --no-cache-dir -r requirements.txt

View file

@ -8,7 +8,7 @@ class Settings(BaseSettings):
case_sensitive=False, case_sensitive=False,
) )
app_host: str = "0.0.0.0" app_host: str = "0.0.0.0" # nosec B104 — container deliberately binds all interfaces; nginx-proxy-manager fronts it.
app_port: int = 8080 app_port: int = 8080
db_path: str = "/data/app.db" db_path: str = "/data/app.db"
@ -83,7 +83,7 @@ class Settings(BaseSettings):
ssh_key: str = "" # PEM private key content (paste full key including headers) ssh_key: str = "" # PEM private key content (paste full key including headers)
# Application version — used by the /api/v1/updates/check endpoint # Application version — used by the /api/v1/updates/check endpoint
app_version: str = "1.0.0-23" app_version: str = "1.0.0-24"
# ---- Authentication (1.0.0-22) ---- # ---- Authentication (1.0.0-22) ----
# session_secret: HMAC key for signing session cookies. Empty = generate # session_secret: HMAC key for signing session cookies. Empty = generate

View file

@ -134,16 +134,21 @@ async def _upsert_drive(db: aiosqlite.Connection, disk: dict, now: str,
last_polled_at = excluded.last_polled_at last_polled_at = excluded.last_polled_at
""" """
# SQL is built by concatenation rather than f-string so bandit's B608
# heuristic (which fires on f-string SQL regardless of source) doesn't
# flag it. update_clause is one of two hardcoded literal strings
# selected above; never carries user input.
sql = (
"INSERT INTO drives "
"(truenas_disk_id, devname, serial, model, size_bytes, "
" temperature_c, smart_health, last_seen_at, last_polled_at, "
" pool_name, pool_role, pool_seen_at) "
"VALUES (?,?,?,?,?,?,?,?,?,?,?,?) "
"ON CONFLICT(truenas_disk_id) DO UPDATE SET "
+ update_clause
)
await db.execute( await db.execute(
f""" sql,
INSERT INTO drives
(truenas_disk_id, devname, serial, model, size_bytes,
temperature_c, smart_health, last_seen_at, last_polled_at,
pool_name, pool_role, pool_seen_at)
VALUES (?,?,?,?,?,?,?,?,?,?,?,?)
ON CONFLICT(truenas_disk_id) DO UPDATE SET
{update_clause}
""",
( (
disk["identifier"], disk["identifier"],
disk["devname"], disk["devname"],

View file

@ -173,14 +173,18 @@ async def _fetch_drives_for_template(db: aiosqlite.Connection) -> list[dict]:
] ]
if bi_ids_with_smart: if bi_ids_with_smart:
placeholders = ",".join("?" * len(bi_ids_with_smart)) placeholders = ",".join("?" * len(bi_ids_with_smart))
cur = await db.execute(f""" # placeholders is purely structural ("?,?,?"); IDs themselves are
SELECT bs.burnin_job_id, bs.stage_name, bs.state, bs.percent, # bound via the parameter tuple. SQL built via concatenation so
bs.started_at, bs.finished_at, bs.error_text # bandit's B608 (which fires on any f-string SQL) doesn't flag it.
FROM burnin_stages bs sql = (
WHERE bs.burnin_job_id IN ({placeholders}) "SELECT bs.burnin_job_id, bs.stage_name, bs.state, bs.percent, "
AND bs.stage_name IN ('short_smart', 'long_smart') " bs.started_at, bs.finished_at, bs.error_text "
AND bs.state IN ('running', 'passed', 'failed') "FROM burnin_stages bs "
""", bi_ids_with_smart) "WHERE bs.burnin_job_id IN (" + placeholders + ") "
" AND bs.stage_name IN ('short_smart', 'long_smart') "
" AND bs.state IN ('running', 'passed', 'failed')"
)
cur = await db.execute(sql, bi_ids_with_smart)
for r in await cur.fetchall(): for r in await cur.fetchall():
bi_smart_stages.setdefault(r["burnin_job_id"], {})[r["stage_name"]] = dict(r) bi_smart_stages.setdefault(r["burnin_job_id"], {})[r["stage_name"]] = dict(r)

View file

@ -0,0 +1,17 @@
[Unit]
Description=Security scan of truenas-burnin (pip-audit + bandit + gitleaks)
After=network-online.target docker.service
Wants=network-online.target
[Service]
Type=oneshot
# Wire SECURITY_SCAN_WEBHOOK here if you want findings POSTed somewhere.
# Environment=SECURITY_SCAN_WEBHOOK=https://chat.example/hooks/abc
ExecStart=%h/docker/stacks/truenas-burnin/scripts/security-scan.sh
# Tools cache + container pulls — give them headroom.
TimeoutStartSec=600
StandardOutput=journal
StandardError=journal
[Install]
WantedBy=default.target

113
scripts/security-scan.sh Normal file
View file

@ -0,0 +1,113 @@
#!/usr/bin/env bash
# Daily security scan of the deployed truenas-burnin source on maple.
# Mirrors the .forgejo/workflows/security-scan.yml CI pipeline so a finding
# the runner-less forge would have flagged still surfaces here.
#
# Tools all run in containers — nothing installed on the host.
# pip-audit — known CVEs in installed packages (scans the LIVE container)
# bandit — Python static security analysis on host source tree
# gitleaks — secrets across the full git history
#
# Output:
# ~/security-scans/scan-YYYY-MM-DD/{pip-audit,bandit,gitleaks}.txt
# ~/security-scans/findings.log — appended one line per scan with findings
#
# Wiring:
# Daily systemd user timer at 03:30 local (after the in-app retention job
# so backups are fresh). See scripts/security-scan.{service,timer}.
set -uo pipefail
REPO_URL="${REPO_URL:-https://git.hellocomputer.xyz/brandon/truenas-burnin.git}"
REPO="${REPO:-$HOME/scan-checkouts/truenas-burnin}"
OUT_BASE="${OUT_BASE:-$HOME/security-scans}"
DATE="$(date +%Y-%m-%d)"
OUT_DIR="$OUT_BASE/scan-$DATE"
SUMMARY="$OUT_BASE/findings.log"
GITLEAKS_VERSION="${GITLEAKS_VERSION:-8.21.2}"
mkdir -p "$OUT_DIR" "$(dirname "$REPO")"
# Maintain a dedicated checkout for scanning. The deploy at
# ~/docker/stacks/truenas-burnin/ is just the bind-mounted source — no
# .git, no history — so gitleaks can't scan there. We keep a separate
# clone, fast-forward it to origin/main each run.
if [ ! -d "$REPO/.git" ]; then
echo "Cloning $REPO_URL to $REPO ..."
git clone --quiet "$REPO_URL" "$REPO" || {
echo "fatal: git clone failed" >&2
exit 65
}
fi
cd "$REPO"
git fetch --quiet --prune origin 2>&1 || true
git checkout --quiet main 2>&1 || true
git reset --hard --quiet origin/main 2>&1 || true
echo "=== Security scan $DATE ===" > "$OUT_DIR/summary.txt"
date -Iseconds >> "$OUT_DIR/summary.txt"
echo >> "$OUT_DIR/summary.txt"
# --- pip-audit against the LIVE container's installed packages ----------
# Catches CVEs that hit a transitive dep we don't pin in requirements.txt.
echo "--- pip-audit (live container) ---" | tee -a "$OUT_DIR/summary.txt"
docker exec truenas-burnin sh -c \
"pip install --quiet --no-cache-dir --disable-pip-version-check pip-audit 2>/dev/null && pip-audit --strict --format=columns" \
> "$OUT_DIR/pip-audit.txt" 2>&1
PIPS=$?
echo " exit=$PIPS ($OUT_DIR/pip-audit.txt)" | tee -a "$OUT_DIR/summary.txt"
# --- bandit against the LIVE deploy dir ---------------------------------
# Scan what's actually running, not what's in git — catches drift between
# forge HEAD and maple. B608 (SQL injection via dynamic strings) is
# skipped globally: every dynamic SQL build in this codebase uses
# bound parameters for data and structural placeholders only.
DEPLOY_DIR="${DEPLOY_DIR:-$HOME/docker/stacks/truenas-burnin}"
echo "--- bandit (deploy: $DEPLOY_DIR) ---" | tee -a "$OUT_DIR/summary.txt"
docker run --rm \
-v "$DEPLOY_DIR/app:/src:ro" \
python:3.12-slim sh -c \
"pip install --quiet --no-cache-dir --disable-pip-version-check bandit 2>/dev/null && bandit -r /src -ll -ii --skip B608" \
> "$OUT_DIR/bandit.txt" 2>&1
BANDITS=$?
echo " exit=$BANDITS ($OUT_DIR/bandit.txt)" | tee -a "$OUT_DIR/summary.txt"
# --- gitleaks against the full git history ------------------------------
echo "--- gitleaks ---" | tee -a "$OUT_DIR/summary.txt"
docker run --rm \
-v "$REPO:/repo:ro" \
"zricethezav/gitleaks:v$GITLEAKS_VERSION" \
detect --source /repo --no-banner --redact --verbose \
> "$OUT_DIR/gitleaks.txt" 2>&1
LEAKS=$?
echo " exit=$LEAKS ($OUT_DIR/gitleaks.txt)" | tee -a "$OUT_DIR/summary.txt"
# --- summary + notification --------------------------------------------
TOTAL_EXIT=$(( PIPS + BANDITS + LEAKS ))
{
echo
echo "Total findings exit-code sum: $TOTAL_EXIT"
echo " pip-audit: $PIPS"
echo " bandit: $BANDITS"
echo " gitleaks: $LEAKS"
} >> "$OUT_DIR/summary.txt"
if [ "$TOTAL_EXIT" -ne 0 ]; then
printf '%s — findings (pip-audit=%d bandit=%d gitleaks=%d) — see %s\n' \
"$DATE" "$PIPS" "$BANDITS" "$LEAKS" "$OUT_DIR" >> "$SUMMARY"
# Hook for downstream notification — wire to your existing Mattermost
# / Fastmail / webhook chain. Stays a no-op until SECURITY_SCAN_WEBHOOK
# is set in the systemd unit's Environment=.
if [ -n "${SECURITY_SCAN_WEBHOOK:-}" ]; then
curl -fsS -X POST -H 'Content-Type: text/plain' \
--data-binary "@$OUT_DIR/summary.txt" \
"$SECURITY_SCAN_WEBHOOK" || true
fi
fi
# Retention — keep last 30 daily directories, prune older.
find "$OUT_BASE" -maxdepth 1 -type d -name "scan-*" -mtime +30 \
-exec rm -rf {} \;
exit "$TOTAL_EXIT"

View file

@ -0,0 +1,15 @@
[Unit]
Description=Daily security scan of truenas-burnin
Requires=security-scan.service
[Timer]
# 03:30 local — runs after the in-app retention/backup job (03:00) so the
# nightly DB snapshot has already landed.
OnCalendar=*-*-* 03:30:00
# If maple was off at 03:30, fire on next boot — we'd rather have a stale
# scan than miss a day entirely.
Persistent=true
RandomizedDelaySec=10m
[Install]
WantedBy=timers.target