Our internal applications do not run on a cloud provider. They run on six MacBooks and a Mac mini in our office, on a Kubernetes cluster we built and operate ourselves.
This began as a question: how much of our own infrastructure could we run on hardware we already own, rather than paying a platform to hold it for us? The answer, so far, is most of it. The cluster has been in service for a few months now without an incident. That is not long enough to prove anything, but it is long enough that the team has stopped thinking about it, which was the point.
The short version: one Mac mini as the control plane, six MacBooks as workers, k3s on top, and Tailscale as the only way in. Postgres and MinIO run in the cluster, pinned to different laptops.
What follows is the full account: how the pieces fit together, what it costs you, and what we got wrong along the way.
Four of the six workers. The Mac mini running the control plane sits elsewhere in the office, on purpose.
Why run our own cluster?
Our reasons weren't fancy. We wanted to own the boring, important layers ourselves: the database, object storage, TLS, networking. Hardware we already own doesn't send a monthly invoice.
And a lab teaches you nothing until real apps depend on it. Running the things the team actually needs is what forces you to get the unglamorous parts right. A test setup never does.
The hardware
One Mac mini and six MacBooks. It started with four workers. We added two more recently, which took about ten minutes each and is the whole argument for doing it this way.
These are not anyone's working laptop. They are racked on an ordinary metal shelf, lids shut, wired to permanent power, on an office connection that stays up around the clock. Nothing walks out of the building at six o'clock. They were retired from daily use and given a second job, which is a far better fate than a drawer.
The Mac mini is the control plane. It runs the Kubernetes server bits and nothing else — it's tainted, so app workloads stay off it. It also sits somewhere else in the office, not on the shelf with the workers, for the same reason we keep Postgres and MinIO apart: one spilled drink should not take the brain and the muscle together.
The MacBooks are the worker nodes, all Apple Silicon (arm64). That's where the apps, the databases, and storage actually live. Need more capacity? Join another laptop and the scheduler starts placing pods on it.
Kubernetes does not run on macOS. This is easy to gloss over. Each MacBook runs an Ubuntu VM through Lima, a tool for running Linux virtual machines on macOS, and the k3s worker runs inside that. The machine is a MacBook, but the node Kubernetes sees is Linux. It is also why image architecture bites us later, and it deserves a post of its own.
The software
k3s is the distribution, a lightweight Kubernetes that ships as a single binary. Full Kubernetes API, far fewer resources than a standard install, which matters when your servers are really just laptops. Postgres runs on CloudNativePG, an operator that manages Postgres clusters as Kubernetes resources. Object storage is MinIO, an S3-compatible server you can self-host. Images come from an in-cluster registry, and pod-to-pod networking across nodes runs on Flannel, the default overlay network in k3s.
Tailscale is the piece we would call out first. Tailscale is a mesh VPN: every machine gets a stable private address on a network only your own devices can join. It is the only way into the cluster. Nothing is exposed to the public internet, so the API server, the registry, and the databases are reachable only from machines on our own tailnet. Take Tailscale away and none of the rest would be safe to run on hardware sitting in an office.
How the pieces fit
A node is where, a namespace is who. A node is the physical machine something runs on. A namespace is which group it belongs to, and it's also the permission boundary. Every pod has both. Our apps sit in a staging namespace where developers can deploy. Shared plumbing like the registry and storage lives in a platform namespace that's admin-only. Same cluster, very different blast radius.
Storage dictates everything else. We use node-local disks, which means a volume is tied to one specific machine. A pod with data on it can't just wander off to another node. So the two stateful services are pinned to different workers on purpose: Postgres on one, MinIO on another. The logic is a little morbid but simple. A backup and the thing it's backing up should never sit on the same disk. One dead laptop shouldn't take both.
Networking is Tailscale from top to bottom. Every node gets a stable tailnet address, independent of whatever Wi-Fi it happens to be on. The cluster does not care if the office router hands out different addresses tomorrow. The internal DNS names resolve to those Tailscale IPs, so they only work when you're on the tailnet. The certificate is public, but the services behind it aren't, and that's the whole security model: the network is the wall.
Moving the apps over
Getting an app onto the cluster is the same four steps every time.
1. Build the Docker image. The workers are Apple Silicon (arm64), so each image is built for arm64 and tagged with the commit SHA. If it is built for the wrong architecture, no node can run it and the pod never starts, so the platform flag is not optional.
sh
docker buildx build --platform linux/arm64 \
-t registry.lab.example.com/app:$(git rev-parse --short HEAD) --push .
2. Provision the database. We give CloudNativePG a short manifest naming the role and database we want, and it creates both inside the cluster's shared Postgres. The database password comes from a Sealed Secret, not from the manifest.
yaml
apiVersion: postgresql.cnpg.io/v1
kind: DatabaseRole
metadata: { name: app, namespace: staging }
spec:
cluster: { name: postgres }
name: app
ensure: present
login: true
passwordSecret: { name: app-db-auth } # the sealed secret from step 3
---
apiVersion: postgresql.cnpg.io/v1
kind: Database
metadata: { name: app, namespace: staging }
spec:
cluster: { name: postgres }
name: app
owner: app
sh
# apply the sealed password (from step 3) and the role/database together
kubectl apply -f app-db-sealed.yaml -f app-db.yaml
kubectl get database,databaserole app -n staging # both should read APPLIED=true
3. Seal the secrets. Sealed Secrets lets us keep secrets in git safely. You encrypt a value against a public certificate, offline, and only the controller inside the cluster can decrypt it, so the sealed file is safe to commit. We use it for the database password and for the app's other secrets, such as its authentication key and object-storage credentials.
sh
kubectl create secret generic app-db-auth -n staging \
--from-literal=username=app --from-literal=password="$PW" --dry-run=client -o yaml \
| kubeseal --cert sealed-secrets-pub.pem --format yaml > app-db-sealed.yaml
4. Deploy. The app runs as a Deployment, a Service, and an Ingress on its own lab.example.com host, with a second Deployment for any background worker. Schema migrations run first as a one-shot job, so they happen exactly once before the app starts. Connection strings are assembled from the sealed secrets inside the manifest, so no password appears in plaintext. Once the pod is Running, the app is reachable over the tailnet.
sh
kubectl apply -f app-migrate.yaml # one-shot migration job, runs once
kubectl apply -f app.yaml # Deployment + Service + Ingress (+ worker)
Getting in
Access comes in two flavours.
For internal apps, people join the tailnet and the app's name just resolves. Two client flags matter and are easy to forget: --accept-routes so the device takes the route to the cluster, and --accept-dns so it can resolve the internal names. Miss either and the page hangs with no error to tell you why. We lost a bit of time to exactly that.
Sometimes an outside service has to reach in, like a chat platform posting webhook events. For that there's Tailscale Funnel, which exposes a single endpoint to the public internet while everything else stays private. It has some sharp edges (it's unreachable from inside your own tailnet, and it's a relay rather than a CDN), but for the odd inbound webhook it's fine. That's a post of its own.
The upshot is that the whole cluster is run remotely over Tailscale. Nobody sits at the machines. kubectl, logs, exec, port-forward, all of it from wherever you happen to be.
What actually bit us
Roughly in order of how much time it cost:
- amd64 images on arm64 nodes. Pods stuck
Pending, and the real cause three steps removed. Always--platform linux/arm64. port-forwardgiving up on big transfers. Fine for a quick query, useless for a restore. Do the heavy data work inside the cluster.- Funnel's quirks, above. Right tool for a webhook, wrong tool for anything a user waits on.
None of these were k3s's fault, which is sort of the point. The distribution was the easy bit. All the friction was in the reality around it: architecture, networking, and moving data about.
What it costs you
Self-hosting isn't free. It just swaps a money bill for a responsibility one.
There is one control plane, so no high availability for the brain. Apps that are already running survive if it goes down, because the workers keep their containers alive on their own, but you can't schedule, self-heal, or change anything until it's back. Lose its datastore disk and the cluster's definition goes with it.
Postgres and MinIO are single instances on node-local disks. Lose the wrong node and that service is down until the node comes back. There's no quiet replica waiting to take over. Nightly Postgres backups with a tested restore and a MinIO mirror soften that, but they don't erase it.
And it is one building. We did what we could about that: we arranged with the building's power people to keep our circuit on the generator when the mains go. That covers a power cut. It does not cover the internet line, or the room itself. This setup shrugs off a dead disk or a dead laptop. It does not shrug off the address. A managed platform eats that whole class of problem for you. Here it's yours to think about.
Against all that you get control, a bill that is mostly electricity, and a cluster you actually understand, because you built every layer of it.
Would we do it again?
For internal and non-critical stuff, on hardware we already own, with a team that wants to genuinely understand its own infrastructure? Yeah. The learning alone paid for it, and it costs us almost nothing to keep running.
For anything that needs real high availability, hands-off ops, and scale on demand, a managed platform still wins, and it isn't close. We knew the trade we were making: more to look after, in exchange for more control and a lot less spend.
A Mac mini and a few ordinary laptops made a perfectly capable cluster, one we can grow by opening another lid. The surprise was how little the hardware mattered. The work, and the reason it was worth doing, lived in everything wrapped around it: the networking, the data migration, the build discipline, the failure modes you only think about once you own them.