You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 828e13c
Browse filesBrowse the repository at this point in the historyBrowse files
dbbackup: discover every database on every server over ssh (#64)
* feat(dbbackup): discover every database on every server over ssh
`diskpush dbbackup discover [SERVER...] [--save]` looks on each saved
connection and ~/.ssh/config host with one read-only probe per server and
finds Postgres/Supabase/MySQL in Docker or native, SQLite files, and the
app env files that use them. --save keeps them; running it again adds only
what is new and names saved ones a reachable server no longer has.
Databases can now live on a server: the dump runs there (docker exec
pg_dump / mysqldump, sudo -u postgres pg_dump, sqlite3 .backup +
quick_check) and only the bytes come back over `ssh -o BatchMode=yes`,
through the same gzip + completion-line check. No database port opened,
no password copied, no local pg_dump needed.
- plugin-api: optional, additive BaseContext.servers (ServerRef list,
known_hosts files, ssh_config path); wired in the CLI and desktop hosts
- ssh-core: toServerRef()
- git forges and duplicate logins are skipped; aliases that reach the
same machine (/etc/machine-id) are listed once
- desktop task "Find databases on every server"; setting sqliteRoots
- docs/plugins.md, README
Live-tested read-only against the fleet (370 -> ~180 real databases after
noise filters on 7 of 8 servers) and end to end on dev2: a Supabase stack,
a shared-cluster app database, a plain Postgres container and a SQLite
volume backed up over ssh and verified.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(dbbackup): discovery skips hidden, backups and tmp folders
Vienna tenant homes held .retired/ app copies and backups/ full of dated
copies; the live fleet count drops from 175 to 142 real databases.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
diskpush dbbackup run --tags nightly # back them all up
133
+
```
134
+
135
+
On each server one read-only shell script, sent in a single ssh call, finds:
136
+
137
+
| What | How it is found | How it is backed up |
138
+
| --- | --- | --- |
139
+
| Postgres / Supabase in Docker |`docker ps` (images `postgres`, `postgis`, `supabase/postgres`, `timescale`, `pgvector`…; a compose project ending `-supabase` makes it Supabase), then each container's databases with `psql` inside it, signing in the way the image allows (`POSTGRES_USER`, `postgres`, `supabase_admin`, over the socket or `127.0.0.1`) |`docker exec <container> pg_dump …` on the server |
140
+
| MySQL / MariaDB in Docker |`docker ps`, then `mysql -uroot` inside the container with the container's own `MYSQL_ROOT_PASSWORD`|`docker exec <container> mysqldump …`, with that same password from the container's environment |
141
+
| Native Postgres | every online cluster (`pg_lsclusters`), as the `postgres` OS user |`sudo -n -u postgres pg_dump -p <port> …`|
142
+
| Native MySQL / MariaDB | root over the local socket |`mysqldump` as root |
143
+
| SQLite | files under `/srv`, `/opt`, `/var/www`, `/root`, `/home` (setting `sqliteRoots`) whose first bytes are SQLite's header, skipping dependency, cache, source-checkout, worktree, `backups` and `tmp` folders, hidden folders and files, `*.bak*`/replica copies and `opencode.db`; files over 4 GB are listed as skipped (`--max-sqlite-mb`) | SQLite's online backup on the server, `PRAGMA quick_check`, then the file itself comes back |
144
+
145
+
**The dump runs on the server; only its bytes come back over ssh.** No
146
+
database port is opened to this machine, no `pg_dump` of the right version is
147
+
needed here, and **no database password is copied anywhere**: a container is
148
+
entered with `docker exec`, a native install through its socket. A database
149
+
that discovery lists is one that it reached exactly the way the backup will.
150
+
The same completion-line rule applies, so a dump cut short by a dropped
151
+
connection is still thrown away.
152
+
153
+
Each database is named after its server and what a person would call it
154
+
(`dev2-crawlproof`, `dev2-nichedb-supabase`, `bbs-agentbbs-bbs`), and app
155
+
env files (`~/www/*/app.env`, `*/.env`) that point at it are shown beside it,
156
+
passwords masked on the server before they are read. Running `discover` again
157
+
adds only what is new, and names anything saved that a reachable server no
158
+
longer has.
159
+
160
+
Git forges (`github.com`, `gitlab.com`, …) and second aliases of the same
161
+
`user@host:port` are not looked on, and two aliases that turn out to reach the
162
+
same machine (same `/etc/machine-id`) are listed once, under the one with the
163
+
most access. A server that cannot be reached is reported and the rest carry on.
164
+
Probing uses `ssh -o BatchMode=yes`: an unknown host key fails rather than
165
+
asks, unless you pass `--accept-new`. The login needs to be root, in the
166
+
`docker` group, or have passwordless `sudo` for what it should see; the report
167
+
says when it cannot get in somewhere. In the desktop, *Find databases on every
168
+
server* under Plugins… does the same and saves what it finds.
169
+
121
170
**File names are the catalogue.**`shop--20261017T031500Z--nightly+prod.sql.gz`
122
171
is database `shop`, taken at that UTC time, tagged `nightly` and `prod`. Each
123
172
database gets its own folder under the backup folder (`~/Backups/diskpush`
@@ -174,7 +223,7 @@ the file really is SQLite), and *Verify backup* on a backup. *Back up every
174
223
database now* is under Plugins… in the desktop.
175
224
176
225
**Settings:**`backupDir`, `keepLast`, `keepDays`, `push`, `rsh` (the ssh
177
-
command for pushes), and `pg_dumpPath` / `mysqldumpPath` / `psqlPath` /
226
+
command for pushes), `sqliteRoots` (where discovery looks for SQLite files), and `pg_dumpPath` / `mysqldumpPath` / `psqlPath` /
178
227
`mysqlPath` / `sqlite3Path` to pin a tool.
179
228
180
229
## Writing a plugin
@@ -237,6 +286,7 @@ it (`ctx.surface` says which: `cli`, `tui` or `desktop`).
237
286
|`openUrl(url)`| the user's browser; http and https only |
238
287
|`signal`| aborted on Ctrl+C, esc, or Cancel. Stop, save what you have, return |
239
288
|`env`| environment variables |
289
+
|`servers`| optional: the user's servers (`list()` → name, host, port, username, keyPath, jumpHost, sshConfigAlias, passwordAuth; never a password), plus `knownHostsFiles` and `sshConfigFile` to hand to `ssh`. Absent where a host has none to offer |
240
290
|`print`, `warn`, `printJson`, `json`, `prompt`, `cwd`| commands only |
241
291
242
292
An action returns `{ ok, message, changed? }`. `changed: true` makes the file
0 commit comments