mptyl/ThothII
1
0
Fork 0
Code Issues 25 Pull Requests Actions Packages Projects Releases 1 Wiki Activity

fix: make server operations executable

Browse Source
This commit is contained in:
marcopan
2026-08-05 11:06:34 +02:00
parent a707fb442c
commit 96fe5bfa79
11 changed files with 1299 additions and 43 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/install/server.md
+123 -25
View File
@@ -43,19 +43,29 @@ sudo useradd --system --uid 10001 --user-group --home-dir /srv/thothii \
--create-home --shell /usr/sbin/nologin thothii
```
The human operator who runs `thothctl` needs Docker access. On many installations membership in
the `docker` group is effectively host-root access; grant it only according to site policy. The
non-login `thothii` account owns application data and secrets but does not itself need Docker
access.
Use a dedicated `thothii-ops` group for the small set of human operators. A human who runs
`thothctl` must be in both `thothii-ops` (to traverse operator paths and read declared secret files
for output redaction) and the host `docker` group (to invoke Docker). Docker-group membership is
effectively host-root access; grant both memberships only to reviewed administrators. The
non-login `thothii` account owns files and writable data but does not need Docker access.
```sh
sudo groupadd --system thothii-ops
sudo usermod --append --groups thothii-ops,docker "$USER"
```
Log out and back in before continuing; `id` must show both groups. Do not run `thothctl` through
`sudo -u thothii`: that account deliberately lacks Docker access. Do not grant the human direct
write access to runtime bind trees.
Create explicit directories. `source` contains the clone; `operator` contains untracked path-only
configuration; the three writable trees are bind-mounted into `core`; `secrets` contains regular
files only. Backups are separate from live data.
```sh
sudo install -d -o 10001 -g 10001 -m 0750 /srv/thothii/source
sudo install -d -o 10001 -g 10001 -m 0700 /srv/thothii/operator
sudo install -d -o 10001 -g 10001 -m 0700 /srv/thothii/secrets
sudo install -d -o 10001 -g thothii-ops -m 2750 /srv/thothii/source
sudo install -d -o 10001 -g thothii-ops -m 2750 /srv/thothii/operator
sudo install -d -o 10001 -g thothii-ops -m 2750 /srv/thothii/secrets
sudo install -d -o 10001 -g 10001 -m 0750 /srv/thothii/data
sudo install -d -o 10001 -g 10001 -m 0750 /srv/thothii/pi-state
sudo install -d -o 10001 -g 10001 -m 0750 /srv/thothii/workspace-registry
@@ -103,8 +113,50 @@ services:
```
Use `host.docker.internal` in the endpoint binding. The `host-gateway` mapping supplies routing;
it does not bundle or trust the target service. Keep the target port bound/firewalled for Docker
host access only. A stable internal DNS record is the preferred alternative.
it does not bundle or trust the target service. A host service listening only on host
`127.0.0.1` is **not reachable** through this mapping. Bind that service to the ThothII Docker
bridge gateway address or to a dedicated private host interface—never to `0.0.0.0` merely to make
the check pass. A stable internal DNS record routed through an authenticated private listener is
the preferred alternative.
After the first bounded start attempt, copy the exact core container name from `thothctl status`
into `CORE_NAME`, then derive—not guess—the network ID, Linux bridge interface, gateway, and
subnet. Compose networks normally use `br-<first-12-network-id>`; an explicit
`com.docker.network.bridge.name` option takes precedence:
```sh
CORE_NAME=replace-with-exact-core-container-name
NETWORK_ID=$(docker inspect --format '{{range .NetworkSettings.Networks}}{{.NetworkID}}{{end}}' "$CORE_NAME")
NETWORK_NAME=$(docker network inspect --format '{{.Name}}' "$NETWORK_ID")
BRIDGE=$(docker network inspect --format '{{index .Options "com.docker.network.bridge.name"}}' "$NETWORK_ID")
test -n "$BRIDGE" || BRIDGE="br-${NETWORK_ID%${NETWORK_ID#????????????}}"
GATEWAY=$(docker network inspect --format '{{(index .IPAM.Config 0).Gateway}}' "$NETWORK_ID")
SUBNET=$(docker network inspect --format '{{(index .IPAM.Config 0).Subnet}}' "$NETWORK_ID")
printf 'network=%s bridge=%s gateway=%s subnet=%s\n' "$NETWORK_NAME" "$BRIDGE" "$GATEWAY" "$SUBNET"
ip address show dev "$BRIDGE"
```
Bind the co-resident service to `$GATEWAY`. In the host firewall `INPUT` chain, allow its exact
TCP port only when source is `$SUBNET`, input interface is `$BRIDGE`, and destination is
`$GATEWAY`; reject other sources to that listener and persist the rules using the distribution's
firewall manager. Docker's `DOCKER-USER` chain governs forwarded/published traffic and does not
replace this host-input rule. Ask the firewall administrator to implement the equivalent policy
with nftables when iptables is not the site's source of truth.
For an iptables-managed host, replace the port before applying these reviewed rules; the second
rule prevents any other interface/source from reaching that gateway listener:
```sh
EXTERNAL_PORT=replace-with-exact-service-port
sudo iptables -I INPUT 1 -i "$BRIDGE" -s "$SUBNET" -d "$GATEWAY" -p tcp --dport "$EXTERNAL_PORT" -j ACCEPT
sudo iptables -I INPUT 2 -d "$GATEWAY" -p tcp --dport "$EXTERNAL_PORT" -j REJECT
```
Confirm reachability with `thothctl pi test` for the configured LLM/Pi path and with the
authenticated Workspace Diagnostics action for DWH, vector collection/embedding pairing, and
embedding endpoints. A timeout paired with `ss -lntp`, `ip address show dev "$BRIDGE"`, and the
firewall counters distinguishes a loopback bind from a subnet/interface rule failure. Do not add
a shell to the browser or mount the Docker socket into core for this diagnostic.
Configure each boundary independently:
@@ -136,7 +188,9 @@ Copy the path-only server environment and installation descriptor:
sudo -u thothii cp deploy/env/server.env.example /srv/thothii/operator/server.env
sudo -u thothii cp docs/install/examples/thothii-installation.server.yaml \
/srv/thothii/operator/thothii-installation.yaml
sudo chmod 0600 /srv/thothii/operator/server.env \
sudo chown 10001:thothii-ops /srv/thothii/operator/server.env \
/srv/thothii/operator/thothii-installation.yaml
sudo chmod 0640 /srv/thothii/operator/server.env \
/srv/thothii/operator/thothii-installation.yaml
```
@@ -146,15 +200,16 @@ session-server overlay and generated connector-secret override. Optional host-ga
image overrides go after them.
Create each credential as an independent regular file in `/srv/thothii/secrets`, owned by
UID/GID 10001 and mode `0600`. The operator environment records only absolute `*_FILE` or
UID 10001, group `thothii-ops`, and mode `0640`. Owner access lets the UID 10001 container read a
file mounted under `/run/secrets`; group access lets the reviewed human run `thothctl`. The
operator environment records only absolute `*_FILE` or
`*_SOURCE` paths. Compose mounts application and connector targets read-only under `/run/secrets`;
the frontend receives none. Do not print file contents while testing permissions.
```sh
sudo find /srv/thothii/secrets -type f -exec chown 10001:10001 {} +
sudo find /srv/thothii/secrets -type f -exec chmod 0600 {} +
sudo find /srv/thothii/secrets -type f ! -user thothii -print
sudo find /srv/thothii/secrets -type f ! -perm 0600 -print
sudo find /srv/thothii/secrets -type f -exec chown 10001:thothii-ops {} +
sudo find /srv/thothii/secrets -type f -exec chmod 0640 {} +
sudo find /srv/thothii/secrets -type f \( ! -user thothii -o ! -group thothii-ops -o ! -perm 0640 \) -print
```
Add `THT_WORKSPACE_BINDINGS_ENV_FILE=/srv/thothii/operator/workspace-bindings.env` and the matching
@@ -185,13 +240,18 @@ services:
core:
build: !reset null
image: registry.example.com/thothii/core@sha256:<64-lowercase-hex-digits>
session-migrate:
build: !reset null
image: registry.example.com/thothii/core@sha256:<64-lowercase-hex-digits>
frontend:
build: !reset null
image: registry.example.com/thothii/frontend@sha256:<64-lowercase-hex-digits>
```
Add that absolute file last in `overrides`. Both images must come from one compatible release; the
core image must retain the declared Pi version labels checked by `thothctl pi doctor`. Pull access
Add that absolute file last in `overrides`. `core` and `session-migrate` must use the exact same
core digest; neither may retain a local build or `:local` image. Frontend uses its own exact digest.
Both images must come from one compatible release; the core image must retain the declared Pi
version labels checked by `thothctl pi doctor`. Pull access
belongs in the host Docker credential store, not in Compose or the installation descriptor.
## Install thothctl
@@ -201,7 +261,7 @@ Build the operator binaries with Docker. No Go installation or Go knowledge is r
```sh
cd /srv/thothii/source/ThothII
bash scripts/build-thothctl.sh
sudo install -o 10001 -g 10001 -m 0755 dist/thothctl/thothctl-linux-amd64 \
sudo install -o root -g thothii-ops -m 0750 dist/thothctl/thothctl-linux-amd64 \
/srv/thothii/operator/thothctl
```
@@ -211,7 +271,7 @@ not source `server.env` as shell code:
```sh
THTCTL=/srv/thothii/operator/thothctl
INSTALLATION=/srv/thothii/operator/thothii-installation.yaml
"$THTCTL" --installation "$INSTALLATION" --help
"$THTCTL" --help
"$THTCTL" --installation "$INSTALLATION" update --check-only
```
@@ -221,7 +281,24 @@ profile, overrides, project identity, and durable current-image selector. The ge
## Start and verify readiness
Keep the TLS proxy stopped or firewalled during bootstrap:
Keep the TLS proxy stopped or firewalled during bootstrap. First stop the app, run the
installation-aware session migration, and inspect its pristine JSON. The command activates only
the `session-migrate` profile/service with `--no-deps --no-TTY`; it derives the migrator image from the
selected core image after all installation overrides, so this procedure is identical for source
and pinned modes. It exits nonzero unless both arrays are empty:
```sh
"$THTCTL" --installation "$INSTALLATION" stop
"$THTCTL" --installation "$INSTALLATION" sessions migrate --yes
```
Successful output has this shape (the `applied` list may contain versions on first use):
```json
{"applied":[],"drifted":[],"pending":[]}
```
Only after seeing `"pending":[]` and `"drifted":[]`, start and verify:
```sh
"$THTCTL" --installation "$INSTALLATION" start
@@ -303,7 +380,7 @@ BACKUP=/srv/thothii-backups/2026-08-05
sudo install -d -o root -g root -m 0700 "$BACKUP"
sudo tar --numeric-owner --xattrs --acls -C /srv/thothii -czf "$BACKUP/runtime-data.tgz" \
data pi-state workspace-registry
sudo sha256sum "$BACKUP/runtime-data.tgz" >"$BACKUP/SHA256SUMS"
sudo sh -ceu 'cd "$1"; sha256sum runtime-data.tgz > SHA256SUMS; sha256sum --check SHA256SUMS' sh "$BACKUP"
```
Back up the installation descriptor, path-only environment, generated overrides, source revision,
@@ -318,7 +395,7 @@ the restored set. This keeps the previous state recoverable:
```sh
RESTORE=/srv/thothii-restore-2026-08-05
sudo install -d -o root -g root -m 0700 "$RESTORE"
sudo sha256sum --check /srv/thothii-backups/2026-08-05/SHA256SUMS
sudo sh -ceu 'cd "$1"; sha256sum --check SHA256SUMS' sh /srv/thothii-backups/2026-08-05
sudo tar --numeric-owner --xattrs --acls -C "$RESTORE" \
-xzf /srv/thothii-backups/2026-08-05/runtime-data.tgz
sudo test -d "$RESTORE/workspace-registry/repo"
@@ -357,9 +434,30 @@ external service identity; and the proxy/identity provider for login failures.
## Data-preserving uninstall
Drain and stop through `thothctl`, take and verify one final backup, disable the TLS proxy route,
and remove only this installation's stopped `frontend` and `core` containers and optional images
by their exact Compose project labels. Keep `/srv/thothii/data`, `pi-state`,
Drain and stop through `thothctl`, take and verify one final backup, and disable the TLS proxy
route. Set `THT_BACKUP_ROOT=/srv/thothii-backups` in `server.env`; the removal command verifies the
filesystem identity of that backup root, all three bind trees, and every declared secret before
and after removing anything.
First run without confirmation. It displays the exact installation project, service, container
name, container ID, and stopped state, then exits without mutation. Check every target:
```sh
"$THTCTL" --installation "$INSTALLATION" stop
"$THTCTL" --installation "$INSTALLATION" remove
```
If and only if both targets are the expected stopped `frontend` and `core` containers, confirm:
```sh
"$THTCTL" --installation "$INSTALLATION" remove --yes exact-core-id exact-frontend-id
```
Replace both example IDs with the values from the immediately preceding dry-run. The command
refuses confirmation if the current target set differs. The confirmed operation passes only those
previously displayed immutable container IDs to Docker,
uses no force or volume option, rejects running/replaced containers, and proves the preservation
paths still identify the same filesystem objects. Keep `/srv/thothii/data`, `pi-state`,
`workspace-registry`, `operator`, protected secrets, database backups, and the installation
descriptor if reinstallation is possible. Do not prune global Docker data.