Correction to the patch-0002 report: after a reboot the Hub icon asks for a password AND the correct free (desktop) password is rejected there, while desktop sign-in accepts it. Patch 0002/0003 explain the icon prompt (legacy hub_manual_logout cookie; fixed in 0003), but the rejection itself means verify_system_password() never gets a conclusive crypt answer in the Hub unit and its PAM fallback cannot load either: on NixOS a bare libcrypt.so.1/libpam.so.0 resolves through LD_LIBRARY_PATH or not at all (or to glibc's failure-token stub), and until now every stage failed silently into a generic 'Incorrect password'.
password-model.nix now puts pkgs.libxcrypt and pkgs.pam on sovran-hub-web's LD_LIBRARY_PATH so both the crypt fallbacks and PAM can actually load. verify_system_password() now logs, journal-visible and never containing a password or hash: the shadow hash scheme, every libcrypt source tried and whether it loaded, and PAM's service + return code when it has to decide.
1) Clicking the desktop Hub icon demanded the Hub password after any sign-out: api_logout set a hub_manual_logout cookie for 400 days and /auto-login honoured it. The icon is a trusted local entry point (its user is already signed in to the OS as free), so /auto-login now ignores and clears that legacy cookie and logout no longer sets it. Sign-out still destroys the session, and LAN clients -- who never get auto-login -- must still enter the password.
2) The Administrator (root) row in System Passwords showed the free/Hub hint verbatim. Root has no desktop or Hub sign-in and nothing refreshes its display copy, so it now gets a root-specific hint (sudo passwd root to set it; Hub Security Reset to make the tile show one again).
3) Brave prompted for a locked 'Default keyring' on Hub launch: the password cleanup removed configuration.nix's keyrings/default pointer and password-model.nix never re-seeded it, so gnome-keyring fell back to an encrypted keyring named Default. password-model.nix now seeds default -> login (the blank keyring) via tmpfiles, and its activation migration restores the pointer and moves any encrypted default.keyring aside (kept, never deleted).
verify_system_password() trusted the first loadable libcrypt. A library that cannot speak the shadow hash's algorithm (yescrypt, NixOS's default) answers with the '*0' sentinel, which was compared to the hash and reported as a wrong password -- even though the same password still signed in at the desktop. Following the System Passwords tile's 'sign out and back in' hint then locked the user out of the Hub, asking for a 'new' Hub password that does not exist: Hub login is the free account password, verified against /etc/shadow.
Try every loadable libcrypt, treat None/'*' replies as inconclusive rather than wrong, and when no library can decide, verify via PAM -- the same check GDM performs. Also replaces the sign-out hint (server + JS fallback) with wording that says the Hub has no separate password and not to sign out unless you know the current desktop password.
Recovery for an already locked-out user whose desktop login still works: on the machine itself, clear the localhost:8937 cookies and open http://localhost:8937/auto-login (localhost-only), or run 'sudo change-free-password' in a terminal to set a known free/Hub password.
The passwords update made the System Passwords tile the only tile whose credential resolution calls system_password.verify_system_password(). That loaded libcrypt with no error handling: an unloadable LIBCRYPT (e.g. a stale Nix store path) raised OSError through _resolve_credential(), so GET /api/service-detail/root-password-setup.service returned 500 and the Hub showed only 'Could not load service details.'
Verification now fails closed, never fatal: _crypt() falls back through LIBCRYPT / find_library / libcrypt.so.1/.so.2, verify_system_password() returns False on any load or crypt failure, _resolve_credential() degrades that one credential to the 'Changed outside the Hub' hint instead of 500-ing the modal, and the login-time display-copy heal is best-effort.
Adds tests/test_system_password.py regression tests.
- Ensure bitcoind enables rest=1 and electrs.service omits the removed
--daemon-p2p-addr flag even before Sovran_Bitcoin flake input is re-locked.
- Clean up pre-0.12.0 (FixedPrefix.8) RocksDB directories in electrs
preStart so bindex 0.12.0 reindexes cleanly.
The Mullvad VPN app and its daemon now ship with Desktop Only, as the stock
NixOS service with nothing configured on top of it: no local-network boot
service, no DNS or firewall edits of our own, no Hub tile, no Feature Manager
card, no toggles. The account, the relay, the kill switch and the app's other
switches belong to the app.
modules/mullvad.nix is the whole change to the system: a role gate around
services.mullvad-vpn = {
enable = true; # daemon, pkgs.mullvad
gui.enable = true; # app, pkgs.mullvad-vpn
};
Server + Desktop and Bitcoin Node are machines that serve; a VPN client with its
own routing and DNS policy is not something their services asked for, so only
roles.desktop builds it. Every other option stays at the distribution's default
(early boot blocking off, the mullvad-exclude wrapper on), and the only
consequences are the ones the NixOS module already defines — the tun module, and
systemd-resolved with NetworkManager pointed at it while the client is enabled.
The Hub and the desktop shell are untouched: no tile, no feature card, no
service description, no icon, no GNOME folder edit. An app that is simply
installed has no decision to put in front of the user, and no state to report.
README, SECURITY.md (what the app's own kill switch and DNS behaviour mean, so
the change in network behaviour is not a surprise), CHANGELOG, the custom.nix
template (the mkForce line that removes it) and the installer's Desktop card
document it as an app rather than a service.
tests/test_mullvad_desktop_only.py guards the gate and the restraint: the client
builds on Desktop and nowhere else, the module adds no services, no DNS or
firewall edits and no packages, the Hub and desktop shell never mention it, and
it ships no tile icon.
Checked by evaluating the real module against the revision flake.lock pins
(3d23ea05a8a3) with eval-config.nix, four cases: Desktop -> daemon and GUI on,
one unit (mullvad-daemon), mullvad + mullvad-vpn in systemPackages, every other
option at the distribution default; Bitcoin Node and Server + Desktop -> nothing
at all, not even the tun module or resolved; Desktop with
`services.mullvad-vpn.enable = lib.mkForce false;` -> no unit, no package.
The word-picking one-liner used `eval echo \${...}`, but in a Nix
indented string a backslash does not escape `${` -- it starts
interpolation, so evaluation failed with "syntax error, unexpected
invalid token". Replace the eval/positional-parameter trick with a
plain `shuf` pipeline that contains no `${` at all.
Found by nixos-rebuild boot on staging-dev. The shell logic itself was
tested; the file had not been parse-checked by Nix. Re-scanned the
whole module: every remaining `${` is an intended interpolation and
every `''` is a string delimiter.
The Hub, GDM and sudo now share /etc/shadow as the single source of
truth. /var/lib/secrets/{free,root}-password become root-only display
copies, written only by system_password.py and verified against shadow
before being shown. The GNOME login keyring is decoupled from the login
password (blank master password, pam_gnome_keyring off), so nothing
needs re-keying or wiping when the password changes and the keyring
never pops an unlock prompt.
- add app/sovran_systemsos_web/system_password.py: the single module
that verifies / sets / displays the free + root passwords and drives
the legacy-migration onboarding step (same endpoints and UI)
- add modules/password-model.nix: PAM gnome-keyring off, blank
login.keyring seed with a one-time migration of old encrypted
keyrings, first-boot-only password application (no more boot-revert),
LIBCRYPT for the Hub
- server.py: login verifies against shadow and heals stale display
copies; change-password takes the current password; security reset
sets both passwords through the same writer (root is shown as a
password again, not a hash); all keyring wipes removed
- credentials.nix: drop root-password-setup, free-password-setup and
free-password-migration (replaced by password-model.nix) and the
keyring wipe in change-free-password
- configuration.nix: drop enableGnomeKeyring and the keyring tmpfiles
hack (replaced by password-model.nix)
- service-detail.js: "Current password" field; credentials changed
outside the Hub render with a hint instead of a stale value
Installed machines keep their exact passwords: the first-boot services
no-op once an account has a real hash, and the upgrade never writes
/etc/shadow. The legacy-migration onboarding step and "reset all
passwords" keep working.
sshd-localhost.nix turns sshd on for every role so that "ssh root@localhost"
works, and binds it to 127.0.0.1 only ("zero network exposure", as its
own comment says, and as sshd.nix expects: the sshd feature "extends this
to 0.0.0.0 and opens port 22 on the firewall when the user enables remote
SSH").
It never turned off services.openssh.openFirewall, which NixOS defaults
to true and applies whether or not sshd listens on the ports. So port 22
was open in the firewall on every role, Desktop Only included, with
nothing behind it. No listener means no live exposure today; it does mean
the firewall was not saying what the documentation says, and the day
something does bind 22 on a wider address (a listenAddresses change, a
second daemon) it would be reachable without anyone having opened it.
openFirewall is now mkDefault false there. Nothing that wants SSH
published loses it: sshd.nix (feature sshd) and remote-deploy.nix both
add 22 explicitly, and only when enabled. Found by evaluating the real
module set; grepping for allowedTCPPorts does not see a NixOS default.
Evaluated with nix eval, TCP firewall ports:
before after
Server + Desktop 22 80 443 3051 8937 80 443 3051 8937
Bitcoin Node Only 22 3051 8937 60847 3051 8937 60847
Desktop Only 22 (none)
Desktop + features.sshd 22 22
Desktop + deploy.enable 22 3389 22 3389
and sshd's listen addresses are unchanged: 127.0.0.1 by default,
127.0.0.1 and 0.0.0.0 with the sshd feature. Desktop Only now opens no
TCP port at all; UDP 5353 (mDNS) is the only port open there, and
SECURITY.md says so.
Add tests/test_ssh_exposure.py.
allowedUDPPorts was set to [ 3051 ] alongside allowedTCPPorts, which looks
like it was copied from the line above. Caddy serves Ride The Lightning
over TCP on 3051; nothing listens for UDP there, so the rule only opened a
port for no reason.
The comment above the pair also said "Hub management port"; 3051 is RTL.
Evaluated with nix eval, UDP firewall ports per role: Server + Desktop
80 443 3051 5353 -> 80 443 5353, Bitcoin Node Only 3051 5353 -> 5353,
Desktop Only 5353. (80 and 443 are Caddy's; 5353 is mDNS.)
generate_diceware_password() built the password from 3 words out of a 96
word list plus a single digit: 96^3 x 10 = 8,847,360 combinations, about
23 bits.
That one password is the desktop login, the 'free' account password, and
the only thing standing in front of the Hub, which runs as root and
displays the root password, the SSH passphrase, the RTL password and the
Vaultwarden admin token. 23 bits is thin for something that valuable, and
the rate limiting in front of it was weaker than intended (see "hub: make
the login lockout that LOGIN_FAIL_MAX described").
Now 4 words plus 2 digits: 96^4 x 100 = 8,493,465,600, about 33 bits,
for the cost of one more word to write down.
- iso/installer.py: generate_diceware_password().
- modules/credentials.nix: the three fallback generators in
root-password-setup, free-password-setup and free-password-migration,
so a machine provisioned without the installer gets the same strength.
Affects new installs only; existing passwords are untouched.
Checked by running the real thing. The installer function was exercised
2000 times: 96 words in the list, always word-word-word-word-NN, 33.0
bits. For the three services, the generator lines were taken from the
script the evaluated module really produces (nix eval on the nixpkgs
revision flake.lock pins) and run 1500 times each: 96 words in each
list, always word-word-word-word-NN, every two-digit suffix from 00 to 99
seen, and no repeated password.
c33457f put an address check (sovran_lan_only) on the Hub, RTL and
Mempool sites. It was written for ports 80/443 being forwarded and a
Host header selecting a site, and that only ever applied to the Hub. RTL
and Mempool are sites on ports of their own (:3051, :60847): a request
on 80/443 cannot select them, whatever Host it carries.
Checked with Caddy 2.9.1 and the Caddyfile this module generates for
Server + Desktop with every domain configured, serving the public sites
on a stand-in port: Host: sovransystemsos.local, localhost:8937,
127.0.0.1 and x:3051 all get an empty 200, and Host: matrix.example.org
gets the Synapse stand-in. With the Hub off Caddy the guard has nothing
left to guard, and it could not be made right for the two sites that
remain:
- IPv6. A laptop's global address on the LAN looks exactly like a
stranger's. The choice was between letting all of 2000::/3 through,
which is the whole IPv6 internet and is what c33457f does, and
refusing every LAN device that connects over a global address unless
the operator copies the ISP's prefix into a Nix option.
- They do not need it. RTL has a random 20-character password
(pwgen -s 20, about 119 bits) and, since 0.15.12, which Sovran_Bitcoin
pins, a 30-minute lockout keyed on the client address. Mempool shows
public chain data. If someone forwards 3051 or 60847 that is the same
exposure as any other port on the machine, and SECURITY.md says not to.
Remove the snippet and its two imports, and tests/test_caddy_lan_only.py
with them. What is still worth pinning moves to test_hub_direct.py: no
address filter anywhere in caddy.nix, the two sites are plain proxies to
their loopback ports, and no option for a declared prefix is left
half-wired.
Behaviour change: RTL and Mempool answer any client that can reach :3051
or :60847, as they did before c33457f. In practice that is the local
network, because nothing asks you to forward those ports.
Checked with the real generator and Caddy 2.9.1: Node Only generates
`auto_https off` and the two plain sites and validates. Run live next to
the real Hub, a LAN client gets the Hub, RTL and Mempool; a stranger's
address gets RTL and Mempool (by design) and a 403 from the Hub; port 80
is not listening on Node Only.
Caddy fronted the Hub at http://sovransystemsos.local, but the Hub
already listens on 0.0.0.0:8937 itself, and nothing Caddy added is
something it needs:
- Not the name. That is avahi's: mDNS advertises a hostname, not a port,
so the name resolves wherever the Hub listens.
- Not TLS (the site was plain http), not authentication, not cache
headers. The header block duplicated NoCacheMiddleware, and its
Clear-Site-Data ("cache") overrode the app's stronger ("cache",
"storage").
- Not access control, and this is the point. With ports 80/443 forwarded
for public services, a Host header on those ports reached the Hub. That
second door is how the reported bug happened, and c33457f guards it
with an address check instead of closing it.
The Hub is now served on port 8937 only, at
http://sovransystemsos.local:8937, and Caddy has no site for it. The
only thing Caddy answers on 80/443 is the public sites. Caddy keeps
Ride The Lightning (:3051) and Mempool (:60847), because those do need
it: Sovran_Bitcoin binds both to 127.0.0.1 and RTL's unit is sandboxed to
loopback besides, so Caddy is how the local network reaches them.
- caddy.nix: no Hub site. Caddy runs wherever RTL and Mempool do, which
includes Bitcoin Node Only. There it did not run at all (enable was
needsHttpsPorts || extraVhosts != ""), so :3051 and :60847 were open
in the firewall with nothing listening. Ports 80/443 still follow
needsHttpsPorts alone, so Node Only does not open them. The two sites
are written only where their service exists; they were unconditional.
- sovran-hub.nix: 8937 follows the new hub.directPort, 60847 follows
Mempool. It used to be `[ 8937 60847 ]` on every role, Desktop Only
included.
- roles.nix: hub.directPort defaults to !roles.desktop: open on Server +
Desktop and Bitcoin Node Only, closed on Desktop Only, where the Hub is
reached from the machine itself through the desktop window on
localhost.
- The bind stays 0.0.0.0, which is IPv4 only: with that bind [::1]:8937
is refused and "localhost" falls back to 127.0.0.1. That is on purpose
and is now said in the comment. An IPv6 listener would let in clients
whose global addresses the Hub cannot tell from a stranger's, which is
the question the previous commit declines to answer by guessing.
- README, SECURITY.md and two strings in index.html give the new URL.
Behaviour changes: the Hub's address gains :8937, and http://sovransystemsos.local
on port 80 no longer reaches it. Bitcoin Node Only now runs Caddy.
Evaluated with nix eval (nixpkgs as flake.lock pins it, Sovran_Bitcoin at
the locked revision), firewall TCP ports per role:
c33457f this commit
Server + Desktop 22 80 443 3051 8937 60847 22 80 443 3051 8937
Bitcoin Node Only 22 3051 8937 60847 22 3051 8937 60847 (Caddy now runs)
Desktop Only 22 8937 60847 22
Port 22 is open on every role although sshd listens on loopback only;
the last commit of this series deals with that.
The Caddyfile the module really generates (the evaluated generator
script, run, then `caddy validate` with Caddy 2.9.1): Node Only gets the
two sites and nothing else; Server + Desktop with every domain
configured gets the seven domain sites plus :3051 and :60847 and no
mention of the Hub; with Bitcoin off there are no local-network sites;
Node Only with Bitcoin off and no domains leaves Caddy off.
Add tests/test_hub_direct.py and keep tests/test_caddy_lan_only.py for
the two sites it still covers.
The reported bug was the Hub being reachable from outside the local
network when Server + Desktop is active. c33457f guards the Caddy site
for sovransystemsos.local, but the Hub is an application that also
listens on a port of its own (8937), and a check in Caddy does nothing
for a client that never goes through Caddy. Whether a client could reach
the Hub depended on which door it used.
The Hub now asks the question itself. LanOnlyMiddleware is registered
outermost, so a client that is not on this computer or the local network
gets a bare 403 before authentication is considered; it never sees the
login page.
- Local means loopback, 10/8, 172.16/12, 192.168/16, 100.64/10
(Tailscale and other CGNAT/VPN ranges) and 169.254/16 over IPv4, and
::1, fc00::/7 and fe80::/10 over IPv6.
- IPv6 global addresses are not on the list. A global address belonging
to a laptop on the LAN cannot be told apart from a stranger's by the
address alone, and 2000::/3 is every public IPv6 address there is.
- ::ffff:a.b.c.d is read as the IPv4 address inside it.
- sovran_systemsOS.hub.extraLanNetworks adds networks (IPv4 or IPv6 CIDR)
for setups whose own devices use addresses outside those ranges. It is
checked at build time. The app ignores an entry it cannot parse and
refuses 0.0.0.0/0 and ::/0: it must never widen its policy by
guessing, and "everyone" is hub.lanOnly = false, asked for by name.
- sovran_systemsOS.hub.lanOnly (default true) turns the check off.
- The first refusal from each address is logged, naming the option to
change, so an operator whose own device is refused can find out why.
The list is capped so a scanner cannot fill the journal or memory.
Behind Caddy the policy applies to the real client, not to Caddy: uvicorn
takes the address from X-Forwarded-For only when the peer is 127.0.0.1.
Checked, not only reviewed:
- The real app with the config.json the module really generates (nix
eval on the nixpkgs revision flake.lock pins, read back from the
derivation), on a real socket with the source address chosen per
request: 127.0.0.1, 192.168/16, 100.64/10 and a declared extra
network are served; 203.0.113.9, 8.8.4.4 and an address just outside
the declared /28 get 403 on /login, / and /api/ping. /auto-login still
answers 303 to loopback and 403 to everyone else.
- With c33457f's Caddy guard in front, an IPv6 client at 2001:db8::9
passes Caddy (it is inside 2000::/3) and is refused by the Hub; an
IPv4 stranger has the connection closed by Caddy; a LAN client is
served.
- hub.extraLanNetworks accepts 203.0.113.0/28, 2001:db8:abcd::/48 and
bare hosts, and fails the build for /33, 300.1.1.1/8, 0.0.0.0/0, ::/0,
2001:db8::/129 and junk, with a message that says what to write.
Add tests/test_lan_policy.py and tests/test_hub_lan_only.py, and a note
in SECURITY.md.
LOGIN_FAIL_MAX was declared as "max failures in window before extra
delay" and then never read anywhere in the file. _record_failure() only
ever slept a flat LOGIN_FAIL_DELAY. So the Hub's entire defence against
online guessing was a constant 2 second pause per wrong password: no
escalation, no lockout, no ban, and fail2ban is configured for SSH only.
Worse, the old 60 second window could not have worked even if the
constant had been wired up. With a delay per attempt, reaching 10
failures takes about 80 seconds, so the earliest failures aged out of the
window before the count could ever reach the limit.
- security_helpers.py: new LoginThrottle. The delay ramps with the
failure count (2s, 4s, ... capped at 10s), and once LOGIN_FAIL_MAX
failures land inside the window the address is refused outright for
LOGIN_LOCKOUT_SECONDS (5 minutes). A successful login clears the
address, so an operator who fumbles a password is not penalised later.
The sleep is never taken under the lock, so one slow client cannot
stall every other login. Tracked addresses are evicted, so a
distributed sweep cannot grow the table without bound.
The window moves from 60s to 900s so the whole ramp fits inside it.
clock and sleep are injectable, which is what makes it testable.
- server.py: /api/login checks the lockout before the scrypt hash, so a
locked-out client costs almost nothing to reject, and answers 429 with
a human-readable wait instead of a bare 401.
- tests/test_login_throttle.py: covers the ramp, the cap, the lockout
firing and expiring, per-address isolation, clearing on success,
eviction, and that the limit is actually reachable inside the window.
Verified against the real app with TestClient: 10 wrong passwords return
401 and the 11th returns 429 "Too many failed attempts. Try again in
about 5 minute(s)." A correct password clears the counter.
The Hub (sovransystemsos.local), Ride The Lightning (:3051) and Mempool
(:60847) sites are meant for the home network. With ports 80/443
forwarded for public services, Caddy also receives requests from other
clients, so these sites now check the client address as well as the Host
header.
A new snippet, sovran_lan_only, closes the connection unless the client
is on this computer or the local network: private_ranges, 100.64.0.0/10
(Tailscale), 169.254.0.0/16, fe80::/10 and fc00::/7. IPv6 global
addresses (2000::/3) are not filtered: computers on the network often
connect over their own global address, which cannot be told apart from
one on the internet by the address alone. Only the three local sites
import the snippet; the domain sites for public services are unchanged.
Clients with a public IPv4 address on the local network are no longer
served on these sites. The Hub is still available on port 8937.
Checked with Caddy 2.11.4 and the Caddyfile the generator writes: public
IPv4 clients get the connection closed on all three sites, local clients
are served, and the public domain sites answer as before.
Add tests/test_caddy_lan_only.py and a note in SECURITY.md.
Server + Desktop publishes services under the operator's own domain, and
the DNS record for that domain points at the home connection, so anyone
can look up the home IP address. None of the places that offer Server +
Desktop said so.
- README: new section "Server + Desktop and your home IP address" (what
becomes public, what does not, the alternatives, and what happens
technically), plus a note on the role table and in the security
overview.
- SECURITY.md: a matching section, the consequence noted next to "Public
web services exposed only when enabled by the operator", and the
supported versions row no longer pins 1.0.x.
- ISO installer: the Server + Desktop role card ends with the warning.
- Hub: the domain setup text (onboarding, feature setup and domain
reconfiguration share renderDomainNeedsHtml) and the upgrade dialog
carry the same notice.
- Add tests/test_exposure_guards.py. It fails if one of these places
loses the notice or the README anchor stops resolving.
The Hub no longer asks a STUN server, a public DNS resolver or a "what
is my IP" service for the home IP address. The DDNS update asks Njal.la
to use the address the request comes from ("&auto"), reads back the
address Njal.la says it recorded and saves it to
/var/lib/secrets/external-ip. Njal.la is the only third party that
learns the address; it has to, to publish it.
- Add app/sovran_systemsos_web/ddns_update.py, installed as
/etc/sovran/ddns-update.py and run by sovran-ddns-update.service. It
runs curl --ipv4 without redirects, accepts only a public IPv4
address, and rewrites the file atomically and only when the address
changes. Stored "&a=${IP}" URLs are converted when they are used and
"&quiet" is dropped.
- Rewrite modules/core/njalla.nix around that runner and delete
modules/core/public-ip.nix. Setting a sovran_systemsOS.publicIP.*
option now fails with a message that says where the address comes
from. Activation removes the old scripts in /var/lib/sovran. The
existing external-ip file keeps working.
- server.py reads the saved address and starts
sovran-ddns-update.service after a domain is saved, instead of looking
the address up itself.
- Element calling uses sovran_systemsOS.elementCalling.externalIP if
set, otherwise the saved address, and fails with a clear message when
neither exists or the address is not public. It no longer falls back
to STUN. livekit-external-ip.path re-runs livekit-turn-setup and
starts LiveKit when the address changes.
- Add tests/test_ddns_update.py.
Follow-up to the previous two commits: the ALTER DATABASE ... SET
autovacuum_* commands fail at boot with
ERROR: parameter "autovacuum_vacuum_scale_factor" cannot be changed now
and take matrix-synapse-db-tune.service (and nextcloud-db-init.service)
down with them.
Root cause: ALTER DATABASE/ROLE ... SET validates through
set_config_option() with an interactive context, and guc.c rejects any
PGC_SIGHUP parameter set that way. All autovacuum_* GUCs are
SIGHUP-context, so per-database scoping is impossible for them — only
USERSET-level parameters (e.g. work_mem, statement_timeout) can be set
per-database.
Nothing is lost: the same values are already set cluster-wide in
configuration.nix, which covers both nextclouddb and matrix-synapse.
Remove the ALTERs from nextcloud-db-init and delete the now-purposeless
matrix-synapse-db-tune service.
Monolith Synapse spends most of its RAM on caches to avoid Postgres
round-trips, but Sovran ships stock cache settings (global_factor 0.5,
10K event cache, no autotuning) and a default 5-connection DB pool.
- caches.global_factor 4.0 + 100K event cache + autotuning capped at
2G (target 1G), with boosts for the /sync and room-join hot paths.
- DB pool cp_min 5 / cp_max 15, txn_limit 10000 (fewer reconnects).
- gc_thresholds raised to cut GC pauses on a 32 GB box.
- cache-memory extra for cache-size statistics.
- Per-database autovacuum (ALTER DATABASE, scoped to matrix-synapse)
matching the nextclouddb tuning.
Deliberately unchanged: presence and URL previews stay enabled
(disabling them is faster but user-visible), and no workers — monolith
is the right call under ~100 users. Workers would need Redis
replication, the redis extra, and Caddy reverse-proxy rework; revisit
if federation load ever justifies it.
Nextcloud 35's Database checks flag three Performance issues out of the
box: buffer cache hit ratio ~96% (wants 99%+), 100k+ dead tuples, and
million-plus sequential scans on oc_mail_tags / oc_guests_users.
Root causes in Sovran: stock 128MB shared_buffers, stock 60s autovacuum
naptime, APCu file locking, and db:add-missing-indices running exactly
once at install time (never on upgrades or app installs).
Size Postgres for the README's Server + Desktop recommendation (32 GB
RAM, NVMe): 2GB shared_buffers, 12GB effective_cache_size, 512MB
maintenance_work_mem, 32MB work_mem, 4GB max_wal_size, 30s autovacuum
naptime with 4 workers. shared_buffers stays below the 25% rule because
Postgres shares the box with bitcoind, Electrs, LND, MariaDB and PHP-FPM.
Scope the aggressive autovacuum to nextclouddb via ALTER DATABASE so the
shared matrix-synapse DB keeps the milder cluster defaults.
Add a local Redis (127.0.0.1:6379, Nextcloud only) and move
memcache.distributed/locking to Redis; migrate existing installs with a
one-shot since nextcloud-init never re-runs.
Add a weekly nextcloud-db-maintenance timer (VACUUM ANALYZE +
db:add-missing-*) so upgrades and later app installs can't regress the
checks again.
Note: shared_buffers needs one 'systemctl restart postgresql', which
briefly takes down both Nextcloud and Matrix. Everything else is
reload-only or scoped to nextclouddb.
Replace the pre-redesign Hub capture with the new welcome dashboard
introduced in v1.1.5 — the default view showing system status, Bitcoin
sync, and the update card at a glance.
The capture is rendered from the real Hub frontend (Server + Desktop
role, demo credentials/domains) at 1920x1080, and doubles as the hero
shot of the marketing kit. Also drops the asset from 391 KB to 59 KB
with no visible loss.
The welcome dashboard's updates card showed "Sovran_SystemsOS keeps
itself current" whenever no updates were pending. Nothing in the OS
auto-updates: the Hub only *checks* for updates (on load and every
30 minutes while the Hub is open); applying an update is a manual
"Update System" action followed by a reboot. The sub-line promised
behavior that does not exist.
Tie "up to date" to the last check instead ("Last check found no
updates · Click to check again"), matching the actionable phrasing
of the card's other states ("Click to review and update", "Click to
retry the update").
Also add a "Checking for updates" state for the first paint, so the
card never claims "up to date" before the first /api/updates/check
has returned.
Sovran_Bitcoin 0.15.12 serves the RTL UI under /rtl/ (upstream Angular
<base href="/rtl/"> + PathLocationStrategy); the package redirects / to
/rtl/ only for the exact root path.
Update the Hub's RTL tile so the Tor and Local Network credentials show
the canonical /rtl/ URLs (with trailing slash, which the redirect does
not cover) instead of relying on the root-path redirect. Bump the dev
versions.json fallback for rtl.service from 0.15.10 to 0.15.12 to match
the flake (deployed systems already read pkgs.sovran-bitcoin.rtl.version).
Arena Agent
naturallaw777
Sovran Systems
Sovran Systems
Sovran_SystemsOS contributor
naturallaw777
Sovran Systems
Security Fix
Arena.ai Agent
Sovran Contributor