Self-hosting operations

Self-hosted Radioso has two durable data surfaces:

• PostgreSQL, including accounts, workspaces, settings, conversations, audit events, documents, chunks, vectors, and worker queues
• uploaded source-file storage, either the local document-storage volume or the configured object-storage bucket

Back up both surfaces together. A database backup without source files can leave document records whose original uploads cannot be reprocessed. Source files without the database are not enough to restore accounts, settings, embeddings, or chat history.

Minimum operating baseline

For a small private deployment, the minimum baseline is:

• run the backend, frontend, document worker, crawler worker, PostgreSQL, and document storage as separate services or containers
• put the frontend and backend behind HTTPS
• keep .env and secret values outside source control
• back up PostgreSQL and document storage every day
• test restore before relying on the backup
• pin the deployed image tag or Git commit
• apply upgrades manually after reading the migration list

Backup

For Docker Compose deployments, back up:

• the PostgreSQL database with pg_dump or a physical volume snapshot
• the radioso_document_storage volume when DOCUMENT_STORAGE_DRIVER=local
• the deployed .env or equivalent secret inventory, stored in a secure password manager or secrets system

For cloud deployments, back up:

• Cloud SQL or the managed PostgreSQL database
• the GCS bucket or object-storage bucket used by DOCUMENT_STORAGE_BUCKET
• Secret Manager values or the external secret inventory

Keep database and source-file backups from the same time window. That makes document reprocessing predictable after restore.

Restore

Upgrade

Use this order for self-hosted upgrades:

1. Read the release notes or inspect backend/src/db/migrations/ for new SQL migrations.
2. Back up PostgreSQL and document storage.
3. Build or pull the new backend, frontend, and worker images.
4. Start the backend and let migrations run.
5. Start the document worker and crawler worker.
6. Start or refresh the frontend.
7. Upload a small document and confirm it becomes searchable.
8. Ask one grounded question and verify citations or retrieval trace data.

Do not auto-pull main into production without a backup. A failed application upgrade is usually reversible; a partially applied schema or mismatched storage restore is harder to recover.

The backend runs SQL migrations during startup before it opens the HTTP port. The document worker and crawler worker only check for pending migrations. Start the backend first, then start workers after the backend has completed migrations and /health responds.

Migration startup metadata checks have their own timeout controls:

• DB_MIGRATION_LOCK_TIMEOUT_MS
• DB_MIGRATION_STATEMENT_TIMEOUT_MS

Keep these values shorter than your platform startup-probe window. That way a blocked migration metadata check appears as a clear application log instead of a silent port-listen timeout. Large migration SQL bodies, such as index builds or backfills, are not capped by these local metadata timeouts.

Migration startup incidents

If a new backend revision fails before /health is reachable, check the backend logs first. A migration metadata lock or statement timeout should name startup migrations as the failing phase.

If the worker starts but the backend does not, that is a useful signal. Workers use a read-only pending-migration check, while the backend is the runtime that applies SQL migrations.

To inspect blocking sessions in PostgreSQL, use a privileged database console and check locks around the migration metadata table:

SELECT a.pid, a.state, a.wait_event_type, l.mode, l.granted, left(a.query, 140) AS query
FROM pg_locks l
JOIN pg_class c ON c.oid = l.relation
JOIN pg_stat_activity a ON a.pid = l.pid
WHERE c.relname = 'schema_migrations';
 
SELECT pid, pg_blocking_pids(pid) AS blocked_by, state, left(query, 140) AS query
FROM pg_stat_activity
WHERE cardinality(pg_blocking_pids(pid)) > 0;

If a stale session is clearly blocking a rollout, terminate only that backend session:

SELECT pg_terminate_backend(<pid>);

Use this carefully. Confirm the session is stale or belongs to the failed rollout before terminating it. Restarting the database is the broader fallback when session-level recovery is not clear.

Worker incidents

If uploads succeed but new documents do not become searchable, treat it as a worker-path incident first.

Check in this order:

1. The backend can accept uploads and write document records.
2. The document worker is running and can connect to PostgreSQL.
3. Source-file storage is reachable from the worker.
4. The embedding provider credentials and model settings are valid.
5. The queue is draining rather than growing.
6. Failed jobs have enough logs to identify parser, storage, embedding, or timeout failures.

For local polling deployments, WORKER_DISPATCH_DRIVER=noop means the worker claims durable jobs from PostgreSQL. For queue-backed deployments, also check Cloud Tasks or AMQP delivery and retry configuration.

Secrets

Rotate these values deliberately and keep historical impact in mind:

• SESSION_COOKIE_SECRET affects browser sessions.
• WORKSPACE_TOKEN_SECRET affects workspace API tokens.
• PUBLIC_CHAT_SESSION_SECRET affects anonymous public-chat sessions and website embeds.
• CONNECTOR_ENCRYPTION_KEY protects both connector secrets and per-workspace LLM provider API keys at rest. The value must be 32 random bytes, base64-encoded — generate one with openssl rand -base64 32. The bootstrap command generates a key automatically when .env does not already have one. Workspaces that have stored an API key in the UI cannot read it back after the key is changed; operators must coordinate a rotation by clearing or re-entering each affected provider credential.
• provider API keys affect chat, rewrite, rerank, and embedding calls.

Changing encryption or token secrets without a migration or rotation plan can invalidate existing sessions, tokens, or saved connector credentials.

Health checks

At minimum, monitor:

• backend /health
• frontend reachability
• worker process liveness
• PostgreSQL connection errors
• document-processing failures
• embedding provider failures
• disk or bucket storage errors
• HTTP 429 and 5xx rates for public chat and website embeds

Read next

• Deployment — the production contract, required secrets, and rollout checklist this runbook assumes.
• Enterprise usage limits — cap indexed storage, monthly content, and monthly answers per account on Enterprise deployments.