The server · Operations handbook

Server, on the home box

One script, run as root on the home box. It asks what the cloud is called, who the first account is, where it goes, which addresses it answers on, which database it keeps its records in and which standard quills it has, then clones the repo, builds a venv, writes the config and the systemd unit, generates the sealing key, starts the service, creates that account as administrator and gives the server an agent of its own.

curl -fsSL https://raw.githubusercontent.com/Cloudmorrow/cloudmorrow/main/deploy/install-server.sh | sudo sh

The questions are asked on the terminal, not stdin, so the script can arrive through a pipe. Every answer has a flag, and with all three given nothing is asked, which is what a script or a re-run wants:

sudo sh deploy/install-server.sh \
  --name "The Larsens" \
  --public-url https://cloud.example.com \
  --user alice                    # password from $CLOUDMORROW_ADMIN_PASSWORD, or asked

Add --dry-run to read what it is about to do without doing any of it, or asking anything. --repo deploys a fork; from a checkout it defaults to that checkout's origin, and on its own to the public repository.

Installing from a private fork instead: give --repo an ssh URL and the script does the deploy-key dance — the first run generates a read-only key and stops, you add it under the repository's deploy keys, and the second run does the install. (Cloning by hand first cannot work: that is the access the key is about to grant.)

What it leaves behind:

path
`/opt/cloudmorrow/src`	the git checkout the service runs from
`/opt/cloudmorrow/venv`	its virtualenv, installed editable so updates are a pull
`/etc/cloudmorrow/server.toml`	the config — yours to edit, never rewritten
`/etc/systemd/system/cloudmorrow.service`	runs as the `cloudmorrow` system user
`/usr/local/bin/cloudmorrow-update`	the updater, below
`/usr/local/bin/cm`	the terminal app, here on the server too (`cloudmorrow` beside it is the same command)
`/etc/sudoers.d/cloudmorrow`	lets `--admin` run exactly what the updater needs: the update as the service user, the restart, and linking `cm`
`/var/lib/cloudmorrow/.ssh/`	the deploy key the service pulls with, if the repo needed one
`/etc/cloudmorrow/cloudmorrow.key`	the key everything is sealed with at rest
`/etc/cloudmorrow/postgres.pgpass`	the database password, when the installer ran PostgreSQL in a container

Re-running the script is safe: it puts the checkout on the remote branch, reinstalls, and leaves your config, notes, database and accounts alone. It reads the name, address and database back out of the existing config rather than asking again, and asks for an account only while the server has none. On a server that is running and answering it asks first whether to update it, and Enter says yes; --update skips the question, and without a terminal a server that is well is left as it is.

The terminal app is on the server too. cm at the box opens it, you sign in with your own name and password, and an administrator has Administration on the top bar, as on any other machine. It needs no address: a client on the server with no config of its own reads where the server listens off /etc/cloudmorrow/server.toml (or wherever CLOUDMORROW_SERVER_CONFIG says) and reaches it there — loopback when it answers there or on every address, else the first address it was given — whichever addresses were chosen at install, and whether or not a proxy and a public_url sit in front. Plain http to an address the server here listens on is this machine, so nothing has to be allowed. cm login --server, or cm config set api_url, points it somewhere else as on any machine, and then that config wins. Signing in on the server enrols no agent: the box has the server's own already, and it is the machine's.

The database

The records, the accounts and everything else that is not a file live in one database, and the installer asks which: the built-in one, a SQLite file at /var/lib/cloudmorrow/cloudmorrow.db, which is right for a household or a company and needs nothing else on the machine; or PostgreSQL. --database container runs PostgreSQL beside the server as the container cloudmorrow-postgres (docker or podman), with a password made once and kept at /etc/cloudmorrow/postgres.pgpass, and a dump of it written into /var/lib/cloudmorrow/postgres/ every day by cloudmorrow-pgdump.timer, so whatever backs up the data directory carries the database with it. --database postgresql://… is a server you already run, reached once before anything is written. Either way the answer is database_url in server.toml, blank for the built-in one; the server reads CLOUDMORROW_DATABASE_URL in a container. The choice is made at install: the records do not move from one database to the other. HOSTING.md, The database, says what each is for.

The cloud's name

name in /etc/cloudmorrow/server.toml — what the installer's first question wrote — is on the sign-in screen, the install page, the phone's home screen (through the manifest and the Apple title), the page title, and /api/health. It is text, never markup, and blank means "Cloudmorrow". Change it by editing the config and sudo systemctl restart cloudmorrow; a phone that already has the app on its home screen keeps the name it was added under until it is added again.

First boot in the browser

Skip the account question, or run the installer with no terminal, and the server is set up on its first visit instead. While it has no accounts, /, /install and /app all redirect to /setup: one form for the cloud's name and the first account, posted to /api/setup, which makes that account the administrator and sends the browser to the app. The moment any account exists the page redirects to /app and the endpoint answers 409, so there is nothing to find afterwards. /api/health says "setup": true until then, for a client that wants to send somebody there.

The name set this way lives in the database, and wins over the one in the config file: a running service cannot write /etc, and a hosted tenant has no file to edit. An administrator changes it later with PATCH /api/server/settings and {"name": "…"}; anybody signed in may GET it. HOSTING.md is where this leads: a tenant somebody bought is "open it and fill in the setup page".

As a container

deploy/docker/ builds the server into one image, with /data and /keys as its two volumes and every config key an environment variable. compose.yml puts Caddy in front of it for the certificate, and the first visit is the setup page above. The database is the built-in one in the data volume unless the postgres profile is on, which runs PostgreSQL as a third container on a volume of its own (COMPOSE_PROFILES=postgres POSTGRES_PASSWORD=… docker compose up -d), or CLOUDMORROW_DATABASE_URL names a server of your own. Updating is replacing the image, so allow_api_update is off inside it, and FORWARDED_ALLOW_IPS=* (the same as trusted_proxies = ["*"]) lets uvicorn believe a proxy that is another container rather than the loopback. The wheel the image was built from is copied into /data/dist on every start, so /install.sh hands out the code that is running.

Accounts can also be made from the server at any time; the first one on a server is its administrator whatever flags it is given:

sudo -u cloudmorrow /opt/cloudmorrow/venv/bin/cloudmorrow-server user create sam

--password-stdin reads the password from stdin instead of prompting, for a script, and user list --count prints how many accounts there are, which is how the installer knows whether to ask.

Moving an install from BramCloud

Everything was renamed at once — the package, the commands, the paths, the environment variables, the units — so an existing install carries on as BramCloud until you move it. Do not run the old bramcloud update server against the renamed code: it would pull it and then restart a unit whose command no longer exists. Instead, on the server:

sudo sh deploy/migrate-from-bramcloud.sh --public-url https://cm.hl.bramlabs.io --admin jimmi

It stops the old units, moves /etc, /var/lib, /opt and the database to their new names, renames the system user, rewrites the paths and public_url in the config, fetches the renamed code and hands over to install-server.sh, which writes the new unit, updater and sudoers file and rebuilds the venv. The signing, secrets and VAPID keys move with the data directory, so accounts, tokens, secrets and push subscriptions survive; notes stay where notes_dir points.

Then, on each machine, sh deploy/migrate-client.sh https://cm.hl.bramlabs.io installs the cloudmorrow client, keeps your sign-in and project links, and retires the old bramcloud one. The phone app stores its token under a new key, so it asks you to sign in once more.

The URLs

Three places, and only the first two matter for a normal setup:

public_url in /etc/cloudmorrow/server.toml — https://cm.hl.bramlabs.io. The install page bakes this into the command it hands out, so it must be the address other machines use, not the bind address.
your reverse proxy — point it at whatever --host/--port the service listens on. On one box that is 127.0.0.1:8787:
```
cm.hl.bramlabs.io {
	reverse_proxy 127.0.0.1:8787
}
```
The server listens on every address by default, so http://<its IP>:8787 answers on the network too, and a proxy on a different machine reaches it directly (--host 127.0.0.1 keeps it to a proxy on the same box; --host "127.0.0.1, 192.168.1.5" answers on both, and the installer offers the machine's addresses as boxes to tick):
```
cm.hl.bramlabs.io {
	reverse_proxy 100.71.157.19:8787
}
```
A proxy on a different machine also has to be trusted, or the server takes everything through it for plain http — refused with 426 once public_url is https:
```
trusted_proxies = ["100.115.4.117"]   # the proxy's address, as the server sees it
```
Addresses or CIDR ranges, or "*"; the loopback is always trusted, so a proxy on the same box needs nothing. --trusted-proxy on the installer writes it. What a trusted proxy says in X-Forwarded-Proto and X-Forwarded-For is believed; from anyone else those headers are ignored. One thing needs neither: the address the install page shows, the script bakes in and cm signs in to is taken from X-Forwarded-Proto and X-Forwarded-Host (or Forwarded) whoever sent them, because they change only the address handed back to the one who asked. So an untrusted proxy still hands out an https:// command; it just cannot let the sign-in through until it is trusted or public_url is set.
each client — cloudmorrow config set api_url https://cm.hl.bramlabs.io, which the install script does for you.

Change public_url later by editing the config and sudo systemctl restart cloudmorrow.

Answering only the proxy

On a machine that also serves other things, a host firewall can be more trouble than it is worth. allowed_client_ips narrows the API instead:

allowed_client_ips = ["192.168.10.88"]   # just the reverse proxy

Addresses or CIDR ranges, matched against the real TCP peer — never X-Forwarded-For, which the caller sets and could therefore forge. Everything else gets a 403, including the install page and login. Empty (the default) allows anyone who can reach the port.

It does not replace the login: whatever can reach the proxy can still reach the API through it. It stops everything else on the LAN from talking to the port directly.

Updating after you push

Push, then deploy. From your laptop, if your account is an admin:

cloudmorrow update server

That goes through the API — no shell on the server, no ssh agent, nothing to type but the command. The server puts its own checkout on whatever the remote branch says, reinstalls in case dependencies moved, rebuilds the client wheel, and restarts onto the new code; the command waits for it to come back and prints the commit it came back on. With nothing new to pull it says so and stops.

So a release is two things:

python scripts/release.py   # tag this commit, push the code and the tag
cloudmorrow update server     # the server runs it, and republishes the wheel

cloudmorrow update all does that second line and the cloudmorrow update that usually follows it, in one go.

Versions

The version is a git tag. Nothing writes it down — pyproject.toml reads it back off the nearest vX.Y.Z behind HEAD — so there is no number in a file to go stale and no way for it to disagree with the history it claims to describe.

One bump per push: every push of new code is a release and takes the next minor, which is what scripts/release.py does. A major is the only thing anyone decides:

python scripts/release.py            # 0.6.0 → v0.7.0
python scripts/release.py --major    # 0.6.0 → v1.0.0
python scripts/release.py --dry-run  # say what it would do

It refuses on a dirty tree, and refuses to tag a commit that is already a release — a push with nothing new in it is not one. The tag and the commits go up together (git push --follow-tags), so the remote never holds code whose release has not arrived.

Between releases the version says so: a commit past v0.7.0 builds as 0.8.0.dev3+g1c91c8b — the release being worked towards, how far past, and which commit. Anything with .dev in it is not a release, and the deploy output names it by its commit rather than dressing it up as one.

The commit hash has not gone away, and should not: a version only changes when a tag does, so it cannot prove a process restarted onto new code. That is what /api/health still reports a commit for, and what cloudmorrow update server compares when it waits for the server to come back. The version is what you read; the commit is what is checked.

Nothing on the server watches git, by design — a push on its own deploys nothing, and the deploy is a thing you do at a moment you choose.

What restarts it, given the service has no privileges. The service user owns /opt/cloudmorrow and the deploy key, so the pull and the reinstall are just it working on its own files. The restart it genuinely cannot do — so it does not try: it stops itself, and systemd starts it again on the new code. That is what Restart=always in the unit is for, and the server checks the unit says so before it stops — otherwise a deploy would take the API down until someone walked over to the box. systemctl stop cloudmorrow still stops it for good; systemd knows the difference between that and an exit.

What it trusts. An admin token can change the code the server runs. That is a real step up from what a token could do before, and worth deciding deliberately rather than inheriting. What bounds it is git: the deploy is git reset --hard origin/<branch>, so it can only run a commit that is already pushed to the repository — not code of the caller's choosing. If that is still more than you want a bearer token to do:

allow_api_update = false    # in /etc/cloudmorrow/server.toml

and the endpoint answers 403 for everyone.

The way in when the API is the problem. A bad deploy is exactly when you cannot use the API to fix it, so the ssh path stays:

cloudmorrow update server --ssh

or, from your own account on the server, no sudo password and no root shell:

cloudmorrow-update

Those drive the same update over ssh instead, authorised by your ssh access plus the sudoers rule the installer wrote rather than by a token. --host takes any ssh target when the API hostname is not the one you ssh to (an ssh alias, a tailnet name); make it permanent with cloudmorrow config set server_host <target>.

Either way it refuses to run if the checkout has uncommitted changes or if history has diverged, rather than merging on your behalf; --force overrides the first of those, --branch deploys something other than the current branch, and --no-restart leaves the new code on disk with the old code still serving.

Under the hood the ssh path is cloudmorrow-server update, which also takes --source if you need it. The checkout belongs to the service user, and git refuses to touch a checkout owned by somebody else ("dubious ownership"), so cloudmorrow-server update run as root does what cloudmorrow-update does: the update as the owner, only the restart as root. Run as any other user it stops and says whose checkout it is. Do not add the safe.directory exception git suggests: files root writes there are files the service user cannot update past.

Upgrading a server installed before any of this existed. The endpoint and the Restart=always unit both arrive in an update, so the first one still goes over ssh:

cloudmorrow update server --ssh     # deploys the code that adds the endpoint

then re-run deploy/install-server.sh --update (it rewrites the unit and is safe to run again) or edit Restart= in /etc/systemd/system/cloudmorrow.service by hand and sudo systemctl daemon-reload. After that, cloudmorrow update server is the API path. Until the unit is right the deploy still lands — the server just reports that it did not restart itself, and says why.

cm on the server arrives the same way. An update installs the terminal app into the server's venv, and puts it on the PATH beside cloudmorrow-server when it may; from the API it runs as the service user, which owns the venv and not /usr/local/bin, so it says cm installed on the server, but not on its PATH instead. Then either re-run deploy/install-server.sh --update, which links it along with everything else, or make the one link yourself:

sudo ln -s /opt/cloudmorrow/venv/bin/cm /usr/local/bin/cm

Any run of cloudmorrow-server as root does the same, quietly, the first time.

Where the client comes from

Cloudmorrow is not on PyPI, so the server is where machines get it: the installer builds a wheel from the deployed checkout into <data_dir>/dist, serves it at /dist/…, and /install.sh installs from there. Every cloudmorrow-update rebuilds it, so a machine installing tomorrow gets today's code.

cloudmorrow-server publish rebuilds it by hand; pass a .whl to publish one built elsewhere.

The wheel keeps the same version number between commits, so /install.sh installs with --force-reinstall — otherwise pip would decide the old copy already satisfied it.

From docs/MANUAL.md in the Cloudmorrow repository. Something wrong? A pull request there fixes it here.