Operations
mstdn.ro: Operating a Public Mastodon Instance
Running community infrastructure on the fediverse: a full Rails stack, three separate Sidekiq tiers, pooled Postgres connections, and a monthly upgrade discipline — on a deliberately modest footprint.
- Discipline
- Operations
- Built
- Live at
- mstdn.ro
- Reading time
- 3 min read
- Operations
- Mastodon
- Self-hosted
- Ruby on Rails
Context
mstdn.ro is a public Romanian Mastodon instance we operate. Running a fediverse server is a different kind of responsibility from running your own website: there are real people with real accounts on it, and federation means the server exchanges traffic with thousands of other instances on their schedule, not ours. Posts arrive whether we are watching or not.
It is also one of the better operational teachers we have. Mastodon is a full-sized Rails application — web workers, background queues, a streaming API, Postgres, Redis, Elasticsearch — and every scaling lesson written about it had to be applied, not just read.
Constraints
A modest footprint
The instance shares a box with the rest of our estate. Community infrastructure does not come with a community budget, so efficiency is a requirement, not an optimization.Traffic we do not control
Federation ingest is bursty. Another instance's viral moment becomes our queue backlog.Upstream's release cadence
Mastodon ships releases with migrations and breaking changes on its own schedule. Falling behind is a security problem; upgrading carelessly is an outage.
Approach
The deployment is the full Mastodon architecture, each part a separate container:
Three Sidekiq tiers, not one
This is the load-bearing decision. Federation ingest runs in its own Sidekiq service, separate from the default queue and from scheduled work. When a federation burst lands, it queues in the ingress tier — and the jobs users actually feel, like their own posts fanning out, keep flowing through the default tier. One shared queue is the classic Mastodon failure mode: the timeline feels dead while the server is busy digesting someone else's traffic.PgBouncer in front of Postgres
Rails web workers plus three Sidekiq services multiply into more database connections than Postgres should hold open. PgBouncer pools them; Postgres sees a sane number of connections regardless of how concurrency is tuned above it.The rest of the stack, separated by job
Redis for queues and cache, Elasticsearch for full-text search, and the streaming API as its own service — long-lived WebSocket connections have a different lifecycle than request/response web workers and restart independently.Monthly-tagged, self-built images
The instance runs images we build ourselves, tagged by month. No:latest, no surprise upgrades. A new month's image is built, release notes are read first, migrations are run deliberately, and the previous tag remains on disk as the rollback. Self-building also means patches can be carried when needed without waiting on upstream packaging.
What was hard
Queue starvation was the first real lesson — the three-tier Sidekiq layout exists because the single-queue default eventually punishes any instance with active federation. Connection arithmetic was the second: every concurrency knob in Rails and Sidekiq multiplies into Postgres connections, and PgBouncer is the difference between tuning freely and tuning fearfully.
The quieter, ongoing work is the discipline: reading release notes before every upgrade, treating migrations as changes to rehearse rather than apply, and being a good citizen of a federated network — moderation obligations included — where our server's behavior is visible to every server it talks to.
Where it stands
The instance runs today as described. Our measure of success for community infrastructure is that it is boring: upgrades are uneventful, queues drain, and nobody thinks about the server — which is exactly how the people using it should experience it.
If you run community infrastructure — or infrastructure that should be this boring — talk to us.
Want one like this?
We build and run these ourselves. Tell us what you need and we will quote it.