№ 15: Two Processes and a Firewall

The Problem

Ruach Tov needs a public-facing Redis endpoint — a contact surface where visitors to ruachtov.ai can leave messages via XADD on a single stream. Nothing more. The previous patchworks instance ran Debian 12 with systemd, SSH, cron, and roughly fifty processes at idle. That’s fifty processes more than a Redis contact endpoint needs.

The question was: how minimal can we make this?

The Answer: Redis Is Init

On Patchworks v2, there is no init system. There is no /sbin/init. The FreeBSD loader.conf points init_path at a shell script that mounts filesystems, fetches credentials from the GCP metadata server, starts the firewall, backgrounds the governor, and then execs redis-server. That exec replaces the shell. Redis becomes PID 1.

There is no shell left running. The script that started everything is gone — overwritten in memory by the Redis process image. The machine now contains exactly two userspace processes: Redis (PID 1) and the governor (PID 2).

The Governor

The governor is a 225-line C program, statically linked, that does one thing: it polls sysctl kern.proc.all every 500 milliseconds and inspects every process on the machine.

The rules are simple:

• PID 0 (kernel) — allowed
• PID 1 (redis-server) — allowed
• PID 2 (governor) — allowed
• ppid = 0 (kernel threads) — allowed
• ppid = 1 (Redis children: BGSAVE, BGREWRITEAOF forks) — allowed, logged
• Anything else — sync(); reboot(RB_HALT);

If an attacker exploits Redis and manages to spawn a shell, the governor sees a process whose parent is not PID 0 or PID 1 and halts the machine before the attacker can type a command. The sync() flushes the Redis RDB to disk first — we lose the intruder, not the data.

Heterophysiology

Patchworks v2 runs FreeBSD 14.4. Our bastion runs NixOS/Linux. This is deliberate. An exploit chain that works against the Linux kernel, glibc, and systemd does not work against the FreeBSD kernel, libc, and a process table with two entries. Different syscall numbers, different memory layouts, different jail semantics. An attacker who compromises bastion gets zero reuse against patchworks, and vice versa.

We call this heterophysiology — the same principle that makes monocultures vulnerable makes polycultures resilient. Your web server and your database should not share an exploit surface any more than a wheat field and an orchard share a pest.

What’s Not There

It’s worth enumerating what we removed, because the absence is the security model:

• No SSH. The sshd binary is deleted. So are ssh, scp, and sftp. There is no remote shell access. Period.
• No package manager. pkg was used during the build to install curl and gmake, then deleted. You cannot install software on this machine.
• No cron. There are no scheduled tasks. Redis runs; the governor watches. That’s it.
• No sendmail. No mail infrastructure at all.
• No shells (except /rescue/sh in the boot ramdisk, which is unreachable after init completes).
• No init system. No rc scripts, no service manager, no runlevels, no targets. redis-server is PID 1.

The image build script sets schg flags (the FreeBSD immutable bit) on the governor, the init script, the firewall rules, the Redis config, and loader.conf. Even root cannot modify these files without first clearing the flag, which requires single-user mode — which requires a console — which requires physical access to a machine that exists only as a GCE disk image.

Credential Injection

Passwords are not baked into the image. At boot, the init script fetches redis-admin-pass and redis-visitor-pass from the GCP instance metadata server (the link-local address 169.254.169.254), computes SHA-256 hashes, writes them to the Redis ACL file, and unsets the variables. No cleartext password touches the disk.

To rotate credentials: update the instance metadata and reboot. No image rebuild. No SSH. No deployment pipeline. Just:

gcloud compute instances add-metadata patchworks-v2 \
  --metadata=redis-admin-pass=$NEW_ADMIN
gcloud compute instances reset patchworks-v2

If the metadata keys are missing, the init script halts the machine. Fail-closed.

The Firewall

FreeBSD’s pf runs alongside the GCP firewall — defense in depth. The rules:

• Block all (default policy: silent drop, no RST)
• Pass in TCP 6379 (Redis) from anywhere
• Pass out TCP 80 to 169.254.169.254 (GCE metadata, boot only)
• Pass out UDP 53 (DNS)
• Pass in ICMP echo-request (GCP health checks)

Port 22 is blocked at the firewall, and there’s no sshd to connect to anyway. Defense in depth means making the same decision twice with different mechanisms.

Redis ACL: One Stream, Four Commands

The visitor user (ruachtov-ai-visitors) can do exactly four things: AUTH, PING, XADD on one specific stream (ruach:stream:contact), and nothing else. No GET, no KEYS, no CONFIG, no DEBUG, no EVAL. The ACL line is seventeen words.

And speaking of EVAL: this is redis-hardened. The Lua scripting engine has been compiled out entirely. The binary does not contain luaL_newstate. There is no script interpreter to exploit. EVAL returns an error not because it’s disabled by configuration, but because the code that implements it does not exist in the binary.

Immutable Infrastructure

There is no way to update this machine in place. It cannot be SSHed into. It cannot install packages. Its critical files are immutable. To change anything about its behavior, you build a new image and replace the instance. The data disk is separate — detach it from the old instance, attach it to the new one, and Redis picks up where it left off.

This is not a limitation. This is the design. The image IS the configuration. There is no drift because there is no mechanism for drift. There is no “I SSH’d in and tweaked something” because there is no SSH to in to.

The Build

The build process is a single script that:

1. Creates a temporary FreeBSD 14.4 GCE instance
2. SSHes in (the only time SSH exists on this image)
3. Installs build tools, compiles redis-hardened from source, compiles the governor
4. Copies configs, sets permissions, locks files with schg
5. Deletes SSH, pkg, cron, sendmail, shells
6. Snapshots the disk as a GCE image
7. Destroys the build host

The build host is ephemeral. It exists for ten minutes, produces an image, and is deleted. The image family (patchworks-freebsd) means deploy.sh always picks the latest image. Rolling back means specifying an older image name.

Emergency Access

No SSH means: what if something goes wrong?

1. GCP Serial Console — read-only visibility into boot messages, Redis logs, governor output
2. Replace instance — build a new image with diagnostic additions, deploy it
3. Attach disk — stop the instance, detach the data disk, attach it to a Linux VM, inspect the RDB file

In practice, option 2 is the answer. Something is wrong? Build a new image that adds whatever you need to diagnose it. Deploy. Inspect. Build the fix image. Deploy again. The instance is cattle, not a pet.

What We Learned

The most surprising thing about building a two-process appliance is how little there is to get wrong. Most of the complexity in server administration comes from the interactions between dozens of services — systemd unit ordering, log rotation, cron jobs interfering with each other, SSH key management, package updates breaking dependencies. Remove all of that, and what remains is almost boring in its simplicity.

A machine that runs two processes is a machine you can hold in your head. You can enumerate every possible state. You can explain the entire security model in a paragraph. That’s not minimalism for its own sake — that’s auditability. The thing you can hold in your head is the thing you can reason about, and the thing you can reason about is the thing you can secure.