Session — fail2ban Hardened, 42-Minute Prune Policy, Pre-Nap Scan

Who: Claude (Stark) / Sky Claude
Machine: stark — stark (Billboard Linode)
Time: 2026-04-13 20:52 PDT
Product: The Billboard — billboard.instockornot.club

Shipped this session

1. fail2ban hardened. Ran the status at 03:27 UTC and found sshd had banned 1,714 IPs cumulatively and 9,477 failed attempts — but the same four Russian brute-forcers (2.57.121.x, 45.148.10.147, 92.118.39.95) kept cycling through the 10-minute default bantime like coin-op attackers feeding quarters into a jammed vending machine. Fixed in /etc/fail2ban/jail.d/defaults-debian.conf:

• sshd bantime → 1 day (was 10 min)
• maxretry → 3 (was 5)
• bantime.increment = true with maxtime = 30d → repeat offenders get exponential backoff
• New recidive jail → 5 bans in a day = 7-day allports lockout
• Simon's home IP (xxx.xxx.xxx.xxx) allowlisted so he can't lock himself out
• Config backup at /etc/fail2ban/jail.d/defaults-debian.conf.bak.20260414

fail2ban-client reload clean, 6 jails live. The frequent-flyer IPs are about to meet the new wall.

2. 42-minute unverified prune policy. Simon's anti-sticky philosophy extended to the DB itself: watchers who don't click the verify email within 42 minutes get deleted. Added prune_unverified() to /home/shg/billboard/billboard_monitor.py — runs every 5 min on the existing cron, texts Simon on any prune with age-in-minutes. First run caught [email protected] who had been sitting unverified for 7,727 minutes (5.4 days). SMS delivered, Twilio 201, row deleted cleanly.

Nice wrinkle: watchers stay live during the 42-min window, so if a real drop hits before they verify, they still get the email. Simon: "if we fire between the time a user makes a watch and the time they verify that could be exciting." Committed as 568264d.

3. Pre-nap friction scan. Checked logs + DB for user pain. Billboard is green: 93 drops in last 24h, 11 active watchers, no crashes. Four Chris Reeve knife watchers (cwelt, gibson, schellcorey, clsands) are getting 4–6 alerts/day — the happy path is very happy.

One painful case surfaced: [email protected] signed up 8 days ago wanting Trek electric bike drops. Zero Trek sources are being scraped. She's been listening to silence for over a week. Flagged for Simon — it's a dealer-coverage decision, not a code bug.

One noisy warning worth fixing: Could not load suggestions aliases: 'int' object has no attribute 'lower' — 2,237 occurrences in /var/log/billboard/web_watcher.log. Something in suggestions.yaml is an int where a string is expected. Non-fatal, but every reload cycle logs it. Next session.

Current status

• Billboard healthy, flywheel spinning
• Simon heading to bed — not disturbing him unless Twilio screams
• Corp checked in (id 25), directive 41 acked
• 3 commits ahead of origin/main, unpushed (Simon hasn't asked)

What's next

• Fix suggestions.yaml int→lower() bug
• Dealer coverage decision for hyde.amy's Trek watch (needs Simon)
• Simon walks the onboarding flow himself to validate the new verification banner
• Waiting on CEO Typhoon to push the Slack bot script via a reachable channel
• Facebook post launch whenever Simon's satisfied with UI/UX

Something funny

The fail2ban log at 02:52 UTC reads like a bad sitcom: Unban 2.57.122.188, Ban 45.148.10.147, Unban 213.209.159.159, Ban 2.57.121.25, Unban 2.57.122.194, Ban 2.57.122.189… for minutes. Same three /24 subnets, passing the baton like a brute-force relay race. Meanwhile fail2ban is playing the role of the exhausted bouncer who keeps opening the door for the same drunks because his bouncer contract only lasts ten minutes at a time. I extended the contract to a full day. He is pleased. The drunks are not consulted.

— Sky Claude, stark, out.

Author: Claude (Stark) / Sky Claude