Merge pull request #428 from naturallaw777/fix/hub-load-performance

perf: speed up Hub service status loading

CONTRIBUTING.md

@@ -30,7 +30,7 @@ The workflow is:
 Please note: GitHub `main` and Gitea `staging-dev` may contain new features and
 code not yet in `stable`, and that code may not be fully tested.
 
-Moreover, Sovran_SystemsOS has been improved with the help of AI. We have used Copilot and Arean.Ai to work through significant coding challenges and troubleshooting hurdles. We will continue to use AI to help keep Sovran_SystemsOS stable and maintained.
+Moreover, Sovran_SystemsOS has been improved with the help of AI. We have used Copilot and Arena.ai to work through significant coding challenges and troubleshooting hurdles. We will continue to use AI to help keep Sovran_SystemsOS stable and maintained.
 
 ## How Can I Contribute?
 
@@ -92,6 +92,13 @@ git push origin feature/your-feature-name
 - **Do not push directly to `main`.** Always use a feature branch and open a PR.
 - **Be patient.** PRs will be reviewed as soon as possible.
 
+## Security-Sensitive Changes
+
+- Report vulnerabilities privately as described in [`SECURITY.md`](SECURITY.md).
+- Pin upstream revisions and hashes; do not use floating source references.
+- Update [`THIRD_PARTY_NOTICES.md`](THIRD_PARTY_NOTICES.md) when importing code.
+- Add regression tests and describe any threat-model change in the pull request.
+
 ## Code Style
 
 - Follow the existing patterns in the codebase

README.md

@@ -507,14 +507,13 @@ setup, supported hardware, and Royal Membership.
 
 ## For developers
 
-Sovran_SystemsOS combines the reproducibility of [NixOS](https://nixos.org),
-the Bitcoin service modules of
-[nix-bitcoin](https://github.com/fort-nix/nix-bitcoin), the desktop Bitcoin
-packages provided by
+Sovran_SystemsOS combines [NixOS](https://nixos.org), an in-repository
+Bitcoin and Lightning stack, desktop packages from
 [btc-clients-nix](https://github.com/emmanuelrosa/btc-clients-nix), and the
-Sovran Hub into a complete Bitcoin operating system. The operating system
-configuration, installer, Hub, desktop integration, Bitcoin services, and
-optional self-hosting services are all maintained in this repository.
+Sovran Hub. The Bitcoin modules under `modules/bitcoin/` were adapted from
+[nix-bitcoin](https://github.com/fort-nix/nix-bitcoin) and are now maintained
+here. Builds no longer import or fetch nix-bitcoin. Legacy `nix-bitcoin.*`
+option names and `/etc/nix-bitcoin-secrets` remain for compatibility.
 
 ### Development workflow
 
@@ -533,9 +532,9 @@ optional self-hosting services are all maintained in this repository.
 ### Technology
 
 - [NixOS](https://nixos.org) and [Nix flakes](https://nixos.wiki/wiki/Flakes)
-  for reproducible system configuration
-- [nix-bitcoin](https://github.com/fort-nix/nix-bitcoin) for declarative
-  Bitcoin and Lightning services
+  for declarative, pinned system configuration
+- `modules/bitcoin/` for the in-repository Bitcoin and Lightning stack
+- `packages/` for Sovran-maintained package definitions and patches
 - [btc-clients-nix](https://github.com/emmanuelrosa/btc-clients-nix) for the
   Sparrow, Bisq, and Bisq 2 packages
 - [Python](https://www.python.org) and [FastAPI](https://fastapi.tiangolo.com)
@@ -625,12 +624,14 @@ sudo nixos-rebuild switch --rollback
 | `flake.nix` | Declares flake inputs, the running system, and installer outputs |
 | `flake.lock` | Pins dependencies for reproducible builds |
 | `configuration.nix` | Base host, boot, desktop, user, security, backup, and system configuration |
-| `modules/` | Core modules, Bitcoin services, self-hosted services, and optional features |
+| `modules/` | Core modules, self-hosted services, and optional features |
+| `modules/bitcoin/` | In-repository Bitcoin and Lightning service modules |
 | `modules/core/` | Roles, Hub integration, Caddy, desktop, support, and other core behavior |
 | `app/` | Sovran Hub backend, templates, static assets, scripts, and web interface |
 | `scripts/` | Automated release, build, and CDN upload utility scripts |
 | `iso/` | Installer configuration, installer code, and installer assets |
-| `packages/` | Custom package sources and patches (for example, Alby Hub) |
+| `packages/` | Sovran-maintained package definitions and patches |
+| `tests/` | Security and Nix integration checks |
 | `assets/` | Documentation images |
 | `custom.template.nix` | Template for local features and service overrides |
 
@@ -704,65 +705,35 @@ production environment.
 
 ## About Bitcoin wallet entropy
 
-When you create a Bitcoin wallet in Sparrow Wallet, you generate a **24-word
-seed phrase**. You should also apply a **128-bit BIP39 passphrase**. These are
-**two independent entropy sources** — both are required to recover your wallet,
-and neither one alone is sufficient.
+Wallet recovery words control the funds. Never share them with a website,
+support technician, cloud service, or chat application.
 
-**Create each entropy source on different hardware.**
+For meaningful balances, prefer a well-reviewed hardware signer and follow its
+verified backup process. A BIP39 passphrase is optional, advanced protection;
+it is not a replacement for the recovery words. If you use one, back it up
+separately—losing either item can make the wallet unrecoverable.
 
-The seed phrase is generated by Sparrow Wallet on your Sovran_SystemsOS machine.
-The 128-bit passphrase should be generated on a separate device — for example:
-
-- A **GrapheneOS** phone using Bitwarden's passphrase generator
-- A different computer running the **Diceware** program
-
-This way, no single device ever produces or holds both sources of randomness.
-If one device is ever compromised, the other entropy source remains unknown to
-an attacker. Your wallet cannot be recovered without **both**.
-
-This is not a recommendation against hardware wallets, dice, or any other
-well-tested method of creating entropy. Those are established and valuable
-approaches. If you use a hardware wallet, verify for yourself that its entropy
-generation — both the software and the hardware — is sound and follows best
-practices.
-
-The takeaway is simply this: **how your entropy is created matters, and
-generating your two entropy sources on two physically separate devices is a
-strong practice to understand before setting up a Bitcoin wallet** with the
-software included in Sovran_SystemsOS.
-
-**You must do your own research.** The information above is basic educational
-context to help you build your own best practices — not financial advice and not
-a substitute for your own due diligence.
+Keep durable offline backups in separate secure locations. Test recovery before
+relying on a wallet, and begin with a small amount.
 
 ---
 
 ## Security approach
 
-Sovran_SystemsOS is designed around local ownership and explicit control.
+Sovran_SystemsOS uses layered controls:
 
-Its security foundations include:
+- Pinned flake inputs and hash-pinned source archives
+- Bitcoin and Lightning modules maintained in this repository
+- Firewall enabled; public SSH and remote desktop disabled by default
+- Separate service users, systemd sandboxing, and loopback bindings where practical
+- Tor enforcement for supported Bitcoin services
+- Restricted, time-limited support access with scoped `sudo`
+- Operator-controlled public service exposure
 
-- Reproducible builds from pinned flake inputs
-- Firewall enabled by default
-- Public SSH disabled by default
-- Remote desktop disabled by default
-- Hub authentication
-- Local-network Hub access through `sovransystemsos.local`
-- Tor integration for the Bitcoin stack
-- User-controlled service exposure
-- Declarative system configuration
-- Restricted technical-support access
-- Auditable open-source code
-
-No operating system can guarantee complete security. Users should still apply
-updates, protect credentials, maintain backups, secure their local network,
-and review any services they choose to expose publicly.
-
-Bitcoin users must also securely back up wallet seed phrases, descriptors,
-channel backups, and other recovery information. Never store your only wallet
-backup on the same computer that holds the wallet.
+See [`SECURITY.md`](SECURITY.md) for the threat model, limitations, reporting,
+and operator guidance. No operating system can protect funds after recovery
+words, administrator credentials, or the root account are compromised. Apply
+updates and keep tested offline backups.
 
 ---
 
@@ -780,11 +751,15 @@ Sovran_SystemsOS would not have the same reliability, transparency, or reproduci
 
 ### nix-bitcoin
 
-Special thanks go to the [nix-bitcoin](https://github.com/fort-nix/nix-bitcoin) project and its [contributors](https://github.com/fort-nix/nix-bitcoin/graphs/contributors).
+The in-repository Bitcoin stack began with code adapted from
+[nix-bitcoin](https://github.com/fort-nix/nix-bitcoin), primarily from commit
+[`360e30f`](https://github.com/fort-nix/nix-bitcoin/commit/360e30fee5ba32f9fecc89bc35628195d9d2dbbe).
+It has since been narrowed to Sovran's supported services and is maintained in
+this repository. nix-bitcoin is no longer a flake input or build dependency.
 
-nix-bitcoin provides the declarative foundation for building and operating Bitcoin and Lightning services on NixOS. Its work makes it possible to configure complex Bitcoin infrastructure—including nodes, Electrs, Lightning, Tor integration, and related services—in a reproducible and auditable way.
-
-Sovran_SystemsOS builds upon that foundation to make this infrastructure approachable through an integrated desktop, installer, and Sovran Hub.
+We remain grateful to the nix-bitcoin contributors for the declarative and
+security-focused foundation. Its MIT notice is retained in
+[`THIRD_PARTY_NOTICES.md`](THIRD_PARTY_NOTICES.md).
 
 ### Emmanuel Rosa and btc-clients-nix
 
@@ -859,9 +834,10 @@ license.
 > Individual upstream applications, packages, artwork, fonts, and other
 > components included with or built by Sovran_SystemsOS may have their own
 > licenses and copyright holders. The AGPL-3.0 license for this repository does
-> not replace the licenses of independent upstream projects.
+> not replace those licenses.
 
-Read the complete license terms in [`LICENSE`](LICENSE).
+Read [`LICENSE`](LICENSE) and
+[`THIRD_PARTY_NOTICES.md`](THIRD_PARTY_NOTICES.md).
 
 ---
 

SECURITY.md

@@ -1,104 +1,101 @@
-# Security Policy (SECURITY.md)
+# Security Policy
 
-This document outlines the security policy, vulnerability disclosure process, threat model, and security best practices for **Sovran_SystemsOS**.
+## Supported versions
 
----
+| Release | Supported |
+|---|:---:|
+| Latest `1.0.x` stable release | Yes |
+| `main` / `staging-dev` | Development only |
+| Older than `1.0.0` | No |
 
-## 1. Supported Versions
+Install the newest stable point release to receive security fixes.
 
-We actively monitor and patch security vulnerabilities in Sovran_SystemsOS. Because the operating system is built on [NixOS](https://nixos.org), stable releases receive security updates backported from upstream Nixpkgs as well as our custom software updates.
+## Report a vulnerability
 
-| Version | Supported | Notes |
-|:---|:---:|:---|
-| **1.0.x** (Stable) |  Yes | Active stable release line. Patches are backported regularly. |
-| **staging-dev** |  Yes | Development line; updated frequently but may contain untested code. |
-| **Legacy (< 1.0.0)** |  No | Please upgrade to the latest stable release to ensure you have active security updates. |
+**Do not open a public issue or pull request.** Report privately through:
 
----
+- [GitHub Private Vulnerability Reporting](https://github.com/naturallaw777/Sovran_SystemsOS/security/advisories/new)
+- Email: [support@sovransystems.com](mailto:support@sovransystems.com)
 
-## 2. Reporting a Vulnerability
+Include the affected version, impact, reproduction steps, and a minimal proof of
+concept. Never send wallet recovery words, private keys, or live credentials.
 
-We take the security of self-custody systems, private clouds, and communications servers extremely seriously. If you discover a vulnerability, **please do not open a public issue, forum thread, or submit a public pull request.** 
+We aim to acknowledge reports within two business days. Please allow reasonable
+time for a fix and coordinated disclosure.
 
-Instead, report vulnerabilities privately through one of the following secure channels:
+## Security model
 
-1. **Email:** Send an encrypted or plain-text email to [support@sovransystems.com](mailto:support@sovransystems.com).
-   - If sending sensitive information, please contact us first to establish a secure PGP-encrypted communication channel.
-2. **GitHub Private Vulnerability Reporting:** If you are viewing this on GitHub, you can report vulnerabilities privately via the **Security** tab of this repository under **Vulnerability reporting**.
+### Local-first operation
 
-### What to Include in a Report:
-To help us triage and resolve the issue quickly, please include:
-- A clear description of the vulnerability and its potential impact.
-- Step-by-step instructions to reproduce the issue (including any scripts or configuration snippets).
-- The version of Sovran_SystemsOS you tested (found in the Hub header or `/etc/nixos/VERSION`).
-- Any potential remediation steps or suggestions you may have.
+The Hub and core data run on operator-owned hardware. The Hub is intended for a
+trusted local network and must not be port-forwarded to the internet. Public
+services, DDNS, software updates, and optional third-party relays require
+external networks and are outside a “fully offline” model.
 
-### Our Response and Disclosure Process:
-1. **Acknowledgment:** We will acknowledge receipt of your report within **24–48 hours** and provide a tracking reference.
-2. **Triage:** We will investigate and verify the vulnerability privately. We may reach out to you for clarifying details.
-3. **Remediation:** If verified, we will develop a patch on a private security branch.
-4. **Release:** We will coordinate a release date with you and publish the patch to Gitea `stable` and GitHub `main`.
-5. **Advisory:** A public security advisory will be published, giving full credit to you for the discovery (unless you request anonymity).
+The local Hub currently uses HTTP. Authentication does not encrypt local network
+traffic, so use a trusted LAN and avoid public or guest Wi-Fi.
 
-We ask that you practice **coordinated vulnerability disclosure**, giving us reasonable time to patch the vulnerability before disclosing it publicly to protect other operators' funds and data.
+### In-repository Bitcoin stack
 
----
+Bitcoin and Lightning modules are maintained under `modules/bitcoin/`. They were
+adapted from nix-bitcoin, but Sovran builds do not import or fetch nix-bitcoin.
+The `nix-bitcoin.*` option namespace and `/etc/nix-bitcoin-secrets` path remain
+only for upgrade compatibility.
 
-## 3. The Sovran_SystemsOS Security Architecture
+### Supply chain and integrity
 
-Understanding our underlying threat model and architectural decisions will help you evaluate the system's security.
+`flake.lock` pins flake inputs, and fetched source archives use fixed hashes.
+Builds still depend on pinned Nixpkgs, NixVim, btc-clients-nix, upstream source
+archives, and any configured binary cache. Keeping the Bitcoin modules in this
+repository reduces an external dependency; it does not remove supply-chain
+risk.
 
-### A. Local-First Threat Model
-- **No Cloud Dependency:** Your data, keys, and credentials live entirely on your physical hardware.
-- **Local network Hub Access:** The Sovran Hub is served directly on your local network (`sovransystemsos.local`). It is protected by authentication and **never** automatically exposed to the public internet.
-- **Localhost Auto-Login:** Auto-login to the Hub is restricted strictly to local connections from `127.0.0.1` or `::1` (e.g., when accessing it directly from the local GNOME desktop). LAN and WAN clients must authenticate via password.
+The Hub integrity check verifies Nix store contents and compares the running
+system with a build from local `/etc/nixos`. It does not authenticate the release
+publisher or protect against an attacker who already controls root and can
+change both the system and local configuration.
 
-### B. Declarative & Reproducible NixOS Foundation
-- **No Supply Chain Drift:** Dependencies, packages, and system components are strictly pinned using Nix flakes (`flake.lock`).
-- **Reproducible Integrity Checks:** Operators can verify system integrity at any time via the Hub (`/api/security/verify-integrity`), which runs `nix store verify --all` to check binary consistency and rebuilds the system state from local files to guarantee it matches the expected configuration.
-- **Default-Hardened Base:** Firewalls are enabled by default (`networking.firewall.enable = true`), public SSH and remote desktop are disabled by default, and legacy kernel modules like `rxrpc` are blacklisted to reduce the kernel's attack surface.
+### Access and service isolation
 
-### C. Restricted Tech Support ("Zero-Trust Support")
-To prevent support staff or malicious actors from accessing your private wallets, the operating system employs a unique **"Zero-Trust Support"** architecture:
-- **Restricted System Account:** Support sessions use a restricted, non-root account (`sovran-support`) with heavily scoped shell access and limited `sudo` privileges (restricted strictly to viewing logs, editing configuration, and rebuilding or restarting services).
-- **POSIX ACL Wallet Lockout:** Enabling support immediately triggers POSIX ACLs (`setfacl -m u:sovran-support:---`) to block the support user from reading, writing, or traversing sensitive directories (such as `/etc/nix-bitcoin-secrets`, `/var/lib/bitcoind`, `/var/lib/lnd`, and `/home`).
-- **User-Controlled Timed Unlock:** Support staff can *only* access wallet files if the operator explicitly grants a time-limited unlock from the Hub interface. This unlock automatically expires, re-locking the directories.
-- **Tamper-Evident Auditing:** All support actions (enabling, disabling, wallet unlocking, or locking) are logged in a tamper-evident audit trail (`/var/log/sovran-support-audit.log`) readable directly by the operator.
+- Firewall enabled by default
+- Public SSH and remote desktop disabled by default
+- Separate service users and systemd sandboxing where supported
+- Administrative service ports bound to loopback where practical
+- Tor enforced for supported Bitcoin traffic and onion services
+- Public web services exposed only when enabled by the operator
 
-### D. Native Cryptographic Isolation
-- **Tor Network Integration:** Upstream Bitcoin services (Knots/Core, Electrs, LND) utilize native Tor integration, preventing your home IP address from leaking to the public network or third-party node providers.
-- **Isolated Lightning Wallets:** Nostr Wallet Connect (NWC) uses isolated, sandboxed Lightning wallets with specific spending limits and access presets (e.g., receive-only), keeping your main node funds completely isolated from individual apps.
+Tor reduces network exposure for configured Bitcoin services. It is not a
+guarantee against every IP leak, application bug, or traffic-analysis attack.
 
----
+### Restricted support access
 
-## 4. Security Best Practices for Operators (Users)
+Support uses a per-session SSH key on the non-root `sovran-support` account.
+Sessions expire after 24 hours and have a small allowlist of `sudo` commands.
+Wallet paths receive deny ACLs unless the operator explicitly removes them.
+Disabling support removes the key and reapplies the ACLs.
 
-No software can provide absolute security. You are the ultimate sovereign of your machine. Follow these best practices to keep your system and assets secure:
+Support events are written to `/var/log/sovran-support-audit.log`. This is a
+local audit log, not a cryptographically tamper-evident record.
 
-### 🔑 1. Split Wallet Entropy
-When setting up a Bitcoin wallet, **generate your two entropy sources on separate, unconnected hardware**:
-- **Seed Phrase:** Generated by Sparrow Wallet on your Sovran_SystemsOS machine.
-- **BIP-39 Passphrase (128-bit):** Generated on a separate device (e.g., a GrapheneOS phone using a trusted password generator or offline Diceware dice).
-- *Why?* If either device is compromised, your wallet remains secure because an attacker needs **both** independent sources of randomness to recover it.
+## Out of scope
 
-### 💾 2. Implement a Robust Backup Strategy
-- **Off-Machine Backups:** Never store your only wallet backups, seed phrases, or descriptor backups on the same computer as your running wallet.
-- **Descriptor & Channel Backups:** If using Lightning (LND), regularly back up your Static Channel Backups (SCB) and store them off-site. A node hardware failure can permanently lock up channel funds without a fresh SCB.
-- **Ext4 Backups:** If using the Hub's Manual Backup utility, ensure you are backing up to a dedicated **ext4-formatted** external drive. This preserves Linux-native security metadata, permissions, and POSIX ACLs.
+Sovran_SystemsOS cannot protect against:
 
-### 🛡️ 3. Maintain Network and Access Hygiene
-- **Public SSH Off:** Keep SSH remote access disabled in the Hub unless you actively need it. If enabled, Fail2Ban is active, but you should still enforce a strong system password.
-- **Strong Diceware Passwords:** Ensure the system `free` and `root` users have strong passphrases. Avoid re-using passwords across services.
-- **Guest Networks:** Do not access the Sovran Hub over public or guest Wi-Fi networks. These networks often lack isolation, exposing local network traffic to other devices.
-- **Disable RDP When Done:** Turn off Remote Desktop (RDP) once your administrative session is finished.
+- Compromised root or administrator credentials
+- Stolen recovery words, private keys, or backups
+- Malicious or compromised hardware, firmware, or build infrastructure
+- Services the operator deliberately exposes or weakens
+- Physical access without appropriate disk and firmware protections
 
-### 🧑‍💻 4. Hardening Support Sessions
-If you require technical support from Sovran Systems:
-- **Enable SSH and Support on-demand:** Only enable SSH Remote Access and Support sessions when actively working with a technician, and **disable** them immediately when the session is over.
-- **Confirm Removal:** The Hub's support page includes a "Verify Removal" feature. Always use it to ensure public keys have been fully stripped from the system.
-- **Restrict Wallet Access:** Only unlock your wallet directories if absolutely necessary for the support session, and choose the shortest possible duration. Re-lock manually as soon as the troubleshooting steps are complete.
-- **Review Audit Logs:** Periodically inspect the support audit log via the Hub interface to review exactly when support sessions were toggled or unlocked.
+## Operator basics
 
----
+- Verify downloads and stop if the checksum does not match.
+- Apply stable security updates promptly.
+- Use unique passwords and keep SSH/RDP off when not needed.
+- Prefer a well-reviewed hardware signer for meaningful Bitcoin balances.
+- Keep tested, offline backups in separate secure locations.
+- Never share recovery words or private keys with support.
+- Disable support access when the session ends and review the audit log.
 
-Thank you for helping us maintain a highly secure, private, and sovereign computing operating system!
+No software can provide absolute security. Review your configuration and threat
+model before storing important funds or data.

THIRD_PARTY_NOTICES.md

@@ -0,0 +1,33 @@
+# Third-Party Notices
+
+## nix-bitcoin
+
+Portions of `modules/bitcoin/` and selected package definitions were adapted
+from [fort-nix/nix-bitcoin](https://github.com/fort-nix/nix-bitcoin), commit
+[`360e30fee5ba32f9fecc89bc35628195d9d2dbbe`](https://github.com/fort-nix/nix-bitcoin/commit/360e30fee5ba32f9fecc89bc35628195d9d2dbbe).
+The current Sovran build does not import or fetch nix-bitcoin.
+
+MIT License
+
+Copyright (c) 2019 nix-bitcoin developers
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
+
+Other packaged applications retain their own upstream licenses and copyright
+notices.

app/sovran_systemsos_web/server.py

@@ -84,6 +84,17 @@ _DOMAIN_REACHABILITY_STARTUP_DELAY = 5
 _domain_reachability_task: asyncio.Task | None = None
 _domain_reachability_task_lock = asyncio.Lock()
 
+# Short-lived local diagnostics caches.  These values are only used for the
+# dashboard's health hints, so a few seconds of staleness is preferable to
+# launching several subprocesses and DNS lookups on every five-second poll.
+_PORT_CACHE_TTL = 5
+_port_cache_lock = Lock()
+_listening_ports_cache: tuple[float, dict[str, set[int]]] = (0.0, {"tcp": set(), "udp": set()})
+_firewall_ports_cache: tuple[float, dict[str, set[int]]] = (0.0, {"tcp": set(), "udp": set()})
+_DOMAIN_DNS_CACHE_TTL = 60
+_domain_dns_cache_lock = Lock()
+_domain_dns_cache: dict[str, tuple[float, list[str]]] = {}
+
 # Units to start after the next successful rebuild (feature enable flow)
 _pending_service_starts: set[str] = set()
 _pending_service_starts_lock = Lock()
@@ -918,7 +929,7 @@ def _get_listening_ports() -> dict[str, set[int]]:
         try:
             proc = subprocess.run(
                 ["ss", flag],
-                capture_output=True, text=True, timeout=10,
+                capture_output=True, text=True, timeout=5,
             )
             logger.debug("ss %s rc=%s stderr=%r", flag, proc.returncode, proc.stderr.strip())
             logger.debug("ss %s output sample: %r", flag, "\n".join(proc.stdout.splitlines()[:8]))
@@ -969,7 +980,7 @@ def _get_firewall_allowed_ports() -> dict[str, set[int]]:
     try:
         proc = subprocess.run(
             ["nft", "list", "ruleset"],
-            capture_output=True, text=True, timeout=10,
+            capture_output=True, text=True, timeout=5,
         )
         logger.debug("nft list ruleset rc=%s stderr=%r", proc.returncode, proc.stderr.strip())
         logger.debug("nft output sample: %r", "\n".join(proc.stdout.splitlines()[:12]))
@@ -1006,7 +1017,7 @@ def _get_firewall_allowed_ports() -> dict[str, set[int]]:
     try:
         proc = subprocess.run(
             ["iptables", "-L", "-n"],
-            capture_output=True, text=True, timeout=10,
+            capture_output=True, text=True, timeout=5,
         )
         if proc.returncode == 0:
             for line in proc.stdout.splitlines():
@@ -1023,6 +1034,50 @@ def _get_firewall_allowed_ports() -> dict[str, set[int]]:
     return result
 
 
+def _get_listening_ports_cached() -> dict[str, set[int]]:
+    """Return listening ports from a short-lived cache."""
+    global _listening_ports_cache
+    now = time.monotonic()
+    with _port_cache_lock:
+        cached_at, cached_value = _listening_ports_cache
+        if now - cached_at < _PORT_CACHE_TTL:
+            return cached_value
+
+    value = _get_listening_ports()
+    with _port_cache_lock:
+        _listening_ports_cache = (now, value)
+    return value
+
+
+def _get_firewall_allowed_ports_cached() -> dict[str, set[int]]:
+    """Return firewall ports from a short-lived cache."""
+    global _firewall_ports_cache
+    now = time.monotonic()
+    with _port_cache_lock:
+        cached_at, cached_value = _firewall_ports_cache
+        if now - cached_at < _PORT_CACHE_TTL:
+            return cached_value
+
+    value = _get_firewall_allowed_ports()
+    with _port_cache_lock:
+        _firewall_ports_cache = (now, value)
+    return value
+
+
+def _resolve_all_addresses_cached(domain: str) -> list[str]:
+    """Resolve a domain once per cache window for dashboard health checks."""
+    now = time.monotonic()
+    with _domain_dns_cache_lock:
+        cached = _domain_dns_cache.get(domain)
+        if cached is not None and now - cached[0] < _DOMAIN_DNS_CACHE_TTL:
+            return cached[1]
+
+    addresses = _resolve_all_addresses(domain)
+    with _domain_dns_cache_lock:
+        _domain_dns_cache[domain] = (now, addresses)
+    return addresses
+
+
 def _port_range_to_ints(port_str: str) -> list[int]:
     """Convert a port string like ``"443"``, ``"30000-40000"`` to a list of ints."""
     port_str = port_str.strip()
@@ -2886,7 +2941,9 @@ def _get_bitcoin_version_info() -> dict | None:
             ["bitcoin-cli", f"-datadir={BITCOIN_DATADIR}", "getnetworkinfo"],
             capture_output=True,
             text=True,
-            timeout=10,
+            # This is a dashboard hint; do not make an unavailable RPC delay
+            # the entire service tile response.
+            timeout=3,
         )
         if result.returncode != 0:
             _btc_version_cache = (now, None)
@@ -2915,7 +2972,9 @@ def _get_bitcoin_deployment_info() -> dict | None:
             ["bitcoin-cli", f"-datadir={BITCOIN_DATADIR}", "getdeploymentinfo"],
             capture_output=True,
             text=True,
-            timeout=10,
+            # BIP-110 is supplementary tile metadata; an RPC timeout must not
+            # block the dashboard from rendering.
+            timeout=3,
         )
         if result.returncode != 0:
             _btc_deployment_cache = (now, None)
@@ -3114,7 +3173,9 @@ def _get_bitcoin_sync_info() -> dict | None:
             ["bitcoin-cli", f"-datadir={BITCOIN_DATADIR}", "getblockchaininfo"],
             capture_output=True,
             text=True,
-            timeout=10,
+            # Sync progress is optional tile metadata.  Keep a cold/unavailable
+            # node from delaying the initial dashboard response.
+            timeout=3,
         )
         if result.returncode != 0:
             _btc_sync_cache = (now, None)
@@ -3176,6 +3237,7 @@ async def api_bitcoin_bip110():
 
 @app.get("/api/services")
 async def api_services():
+    started_at = time.monotonic()
     cfg = load_config()
     services = cfg.get("services", [])
 
@@ -3188,32 +3250,115 @@ async def api_services():
 
     loop = asyncio.get_event_loop()
 
-    # Read runtime feature overrides from custom.nix Hub Managed section
+    # Read runtime feature overrides from custom.nix Hub Managed section and
+    # calculate each service's effective enabled state once.  The old code did
+    # this inside every status coroutine and then launched one systemctl process
+    # per tile; batching the state lookup keeps a slow D-Bus from multiplying
+    # the dashboard's first-render latency.
     overrides, *_ = await loop.run_in_executor(None, _read_hub_overrides)
-
-    # Cache port/firewall data once for the entire /api/services request
-    listening_ports, firewall_ports = await asyncio.gather(
-        loop.run_in_executor(None, _get_listening_ports),
-        loop.run_in_executor(None, _get_firewall_allowed_ports),
-    )
-
-    async def get_status(entry):
+    effective_entries: list[tuple[dict, bool]] = []
+    active_units_by_scope: dict[str, list[str]] = {}
+    domain_names: set[str] = set()
+    for entry in services:
         unit = entry.get("unit", "")
         scope = entry.get("type", "system")
         icon = entry.get("icon", "")
         enabled = entry.get("enabled", True)
 
-        # Overlay runtime feature state from custom.nix Hub Managed section
         feat_id = unit_to_feature.get(unit)
         if feat_id is None:
             feat_id = FEATURE_ICON_MAP.get(icon)
         if feat_id is not None and feat_id in overrides:
             enabled = overrides[feat_id]
 
+        effective_entries.append((entry, enabled))
+        if enabled and unit:
+            active_units_by_scope.setdefault(scope, []).append(unit)
+
         if enabled:
-            status = await loop.run_in_executor(
-                None, lambda: sysctl.is_active(unit, scope)
+            domain_key = SERVICE_DOMAIN_MAP.get(unit)
+            if domain_key:
+                domain_path = os.path.join(DOMAINS_DIR, domain_key)
+                try:
+                    with open(domain_path, "r") as f:
+                        domain = f.read(512).strip()
+                    if domain:
+                        domain_names.add(domain)
+                except OSError:
+                    pass
+
+    async def resolve_domain(domain: str) -> tuple[str, list[str] | None]:
+        """Resolve DNS off the event loop with a hard upper bound.
+
+        A broken DNS server must not hold the entire dashboard response hostage.
+        ``None`` means the lookup was inconclusive and the background checker can
+        fill in the health result later; an empty list means DNS definitively
+        returned no address.
+        """
+        try:
+            addresses = await asyncio.wait_for(
+                loop.run_in_executor(None, _resolve_all_addresses_cached, domain),
+                timeout=2,
             )
+            return domain, addresses
+        except (asyncio.TimeoutError, OSError):
+            return domain, None
+
+    active_states_future = loop.run_in_executor(
+        None, sysctl.active_states, active_units_by_scope
+    )
+    port_states_future = asyncio.gather(
+        loop.run_in_executor(None, _get_listening_ports_cached),
+        loop.run_in_executor(None, _get_firewall_allowed_ports_cached),
+    )
+    dns_states_future = asyncio.gather(
+        *(resolve_domain(domain) for domain in sorted(domain_names))
+    )
+    # Bitcoin sync and BIP-110 are supplementary tile metadata.  Fetch each
+    # once, concurrently with the other diagnostics, instead of doing duplicate
+    # RPC calls inside both bitcoind tile coroutines.
+    has_enabled_bitcoin = any(
+        entry.get("unit") == "bitcoind.service" and enabled
+        for entry, enabled in effective_entries
+    )
+    has_enabled_bip110 = any(
+        entry.get("icon") == "bip110" and enabled
+        for entry, enabled in effective_entries
+    )
+    bitcoin_sync_future = (
+        loop.run_in_executor(None, _get_bitcoin_sync_info)
+        if has_enabled_bitcoin
+        else asyncio.sleep(0, result=None)
+    )
+    bip110_future = (
+        loop.run_in_executor(None, _get_bip110_status)
+        if has_enabled_bip110
+        else asyncio.sleep(0, result=None)
+    )
+    (
+        active_states,
+        port_states,
+        dns_states,
+        bitcoin_sync_info,
+        bip110_status,
+    ) = await asyncio.gather(
+        active_states_future,
+        port_states_future,
+        dns_states_future,
+        bitcoin_sync_future,
+        bip110_future,
+    )
+    listening_ports, firewall_ports = port_states
+    resolved_domains = dict(dns_states)
+
+    async def get_status(item: tuple[dict, bool]):
+        entry, enabled = item
+        unit = entry.get("unit", "")
+        scope = entry.get("type", "system")
+        icon = entry.get("icon", "")
+
+        if enabled:
+            status = active_states.get((scope, unit), "unknown")
         else:
             status = "disabled"
 
@@ -3269,18 +3414,23 @@ async def api_services():
                     break
         has_domain_issues = False
         if needs_domain and domain and enabled:
-            addrs = _resolve_all_addresses(domain)
+            # DNS was resolved concurrently above.  A timed-out lookup is
+            # treated as unknown rather than as a hard failure; the background
+            # reachability checker will refresh the health state without
+            # delaying the first tile render.
+            addrs = resolved_domains.get(domain)
             dns_ok = True
-            if not addrs:
-                dns_ok = False
-            elif all(_is_loopback_address(a) for a in addrs):
-                # Intentional server-local /etc/hosts override — not a mismatch.
-                dns_ok = True
-            elif (
-                _cached_external_ip != "unavailable"
-                and not any(a == _cached_external_ip for a in addrs)
-            ):
-                dns_ok = False
+            if addrs is not None:
+                if not addrs:
+                    dns_ok = False
+                elif all(_is_loopback_address(a) for a in addrs):
+                    # Intentional server-local /etc/hosts override — not a mismatch.
+                    dns_ok = True
+                elif (
+                    _cached_external_ip != "unavailable"
+                    and not any(a == _cached_external_ip for a in addrs)
+                ):
+                    dns_ok = False
 
             if not dns_ok:
                 has_domain_issues = True
@@ -3304,14 +3454,13 @@ async def api_services():
                     health = "checking_reachability" if cached_reachable is None else "healthy"
                 else:
                     health = "healthy"
-            # Check Bitcoin IBD state
+            # Check Bitcoin IBD state from the single shared lookup above.
             if unit == "bitcoind.service" and enabled:
-                sync = await loop.run_in_executor(None, _get_bitcoin_sync_info)
-                if sync and sync.get("initialblockdownload"):
+                if bitcoin_sync_info and bitcoin_sync_info.get("initialblockdownload"):
                     health = "syncing"
-                    sync_progress = sync.get("verificationprogress", 0)
-                    sync_blocks = sync.get("blocks", 0)
-                    sync_headers = sync.get("headers", 0)
+                    sync_progress = bitcoin_sync_info.get("verificationprogress", 0)
+                    sync_blocks = bitcoin_sync_info.get("blocks", 0)
+                    sync_headers = bitcoin_sync_info.get("headers", 0)
                     sync_ibd = True
         elif status == "inactive":
             # For enabled services that are inactive (e.g. socket-activated PHP-FPM),
@@ -3356,8 +3505,8 @@ async def api_services():
                 btc_ver = _format_bitcoin_version(raw_ver, icon=icon)
                 service_data["bitcoin_version"] = btc_ver  # backwards compat
                 service_data["version"] = btc_ver
-            if icon == "bip110":
-                service_data["bip110"] = await loop.run_in_executor(None, _get_bip110_status)
+            if icon == "bip110" and bip110_status is not None:
+                service_data["bip110"] = bip110_status
         # ── Generic version for all services (Nix store path) ──────────
         if enabled and unit and "version" not in service_data:
             ver = await loop.run_in_executor(None, _get_service_version, unit)
@@ -3365,7 +3514,9 @@ async def api_services():
                 service_data["version"] = ver
         return service_data
 
-    results = await asyncio.gather(*[get_status(s) for s in services])
+    results = await asyncio.gather(*[get_status(item) for item in effective_entries])
+    elapsed = time.monotonic() - started_at
+    logger.info("/api/services: %d services in %.3fs", len(results), elapsed)
     return list(results)
 
 
@@ -4730,9 +4881,11 @@ async def api_domains_set(req: DomainSetRequest):
         # Replace trailing &auto with the IP placeholder used by _run_njalla_ddns
         if ddns_url.endswith("&auto"):
             ddns_url = ddns_url[:-5] + "&a=${IP}"
-        # Validate URL strictly — reject injection attempts before persisting
+        # Validate URL strictly — reject injection attempts before persisting.
+        # The placeholder ${IP} is replaced temporarily so the validator sees a
+        # real address; the original URL (with the placeholder) is kept for storage.
         try:
-            ddns_url = _validate_ddns_url(ddns_url)
+            _validate_ddns_url(ddns_url.replace("${IP}", "127.0.0.1"))
         except ValueError as exc:
             raise HTTPException(status_code=400, detail=f"Invalid DDNS URL: {exc}")
         # Persist the URL in the JSON store (never in executable shell source)

app/sovran_systemsos_web/static/css/tiles.css

@@ -1,5 +1,28 @@
 /* ── Service tile card (status-only) ─────────────────────────────── */
 
+.dashboard-loading {
+  min-height: 180px;
+  display: flex;
+  align-items: center;
+  justify-content: center;
+  gap: 10px;
+  color: var(--text-secondary);
+  font-size: 0.9rem;
+}
+
+.dashboard-loading-spinner {
+  width: 16px;
+  height: 16px;
+  border: 2px solid var(--border-color);
+  border-top-color: var(--accent-color);
+  border-radius: 50%;
+  animation: dashboard-loading-spin 0.8s linear infinite;
+}
+
+@keyframes dashboard-loading-spin {
+  to { transform: rotate(360deg); }
+}
+
 .service-tile {
   width: 160px;
   min-height: 130px;

app/sovran_systemsos_web/static/js/events.js

@@ -187,30 +187,39 @@ function showSecurityBanner() {
 // ── Init ──────────────────────────────────────────────────────────
 
 async function init() {
-  // Check onboarding status first — redirect to wizard if not complete
+  // These lightweight requests are independent.  Running them together avoids
+  // making the dashboard wait through three serial network round-trips before
+  // it can even start loading the service tiles.
+  var onboardingStatus = null;
+  var bannerData = null;
+  var cfg;
   try {
-    var onboardingStatus = await apiFetch("/api/onboarding/status");
-    if (!onboardingStatus.complete) {
-      window.location.href = "/onboarding";
-      return;
-    }
+    var startupResults = await Promise.all([
+      apiFetch("/api/onboarding/status").catch(function() { return null; }),
+      apiFetch("/api/security/banner-status").catch(function() { return null; }),
+      apiFetch("/api/config"),
+    ]);
+    onboardingStatus = startupResults[0];
+    bannerData = startupResults[1];
+    cfg = startupResults[2];
   } catch (_) {
-    // If we can't reach the endpoint, continue to normal dashboard
+    // If config cannot be loaded, continue with the normal service fallback.
+    cfg = null;
+  }
+
+  if (onboardingStatus && !onboardingStatus.complete) {
+    window.location.href = "/onboarding";
+    return;
   }
 
   // Show first-login security banner only for machines that went through onboarding
   // (legacy machines without the onboarding flag will never see this)
-  try {
-    var bannerData = await apiFetch("/api/security/banner-status");
-    if (bannerData && bannerData.show) {
-      showSecurityBanner();
-    }
-  } catch (_) {
-    // Non-fatal — silently ignore
+  if (bannerData && bannerData.show) {
+    showSecurityBanner();
   }
 
   try {
-    var cfg = await apiFetch("/api/config");
+    if (!cfg) throw new Error("Hub config unavailable");
     _currentRole = cfg.role || "server_plus_desktop";
     if (cfg.category_order) {
       for (var i = 0; i < cfg.category_order.length; i++) {

app/sovran_systemsos_web/systemctl.py

@@ -18,6 +18,45 @@ def is_active(unit: str, scope: Literal["system", "user"] = "system") -> str:
     return _run(["systemctl", f"--{scope}", "is-active", unit]) or "unknown"
 
 
+def active_states(units_by_scope: dict[str, list[str]], timeout: float = 5) -> dict[tuple[str, str], str]:
+    """Return active states for many units with one systemctl call per scope.
+
+    The dashboard polls service state frequently. Spawning one ``systemctl``
+    process per tile makes a slow or unavailable system bus multiply the delay
+    (and can make the first dashboard render wait through many timeouts).  The
+    ``is-active`` command accepts multiple units, so batch them and cap the
+    whole batch at a single timeout.
+    """
+    states: dict[tuple[str, str], str] = {}
+    valid_scopes = {"system", "user"}
+
+    for scope, units in units_by_scope.items():
+        unique_units = list(dict.fromkeys(unit for unit in units if unit))
+        if not unique_units:
+            continue
+        if scope not in valid_scopes:
+            for unit in unique_units:
+                states[(scope, unit)] = "unknown"
+            continue
+
+        try:
+            result = subprocess.run(
+                ["systemctl", f"--{scope}", "is-active", "--", *unique_units],
+                capture_output=True,
+                text=True,
+                timeout=timeout,
+            )
+            lines = result.stdout.splitlines()
+            for index, unit in enumerate(unique_units):
+                state = lines[index].strip() if index < len(lines) else ""
+                states[(scope, unit)] = state or "unknown"
+        except Exception:
+            for unit in unique_units:
+                states[(scope, unit)] = "unknown"
+
+    return states
+
+
 def is_enabled(unit: str, scope: Literal["system", "user"] = "system") -> str:
     return _run(["systemctl", f"--{scope}", "is-enabled", unit]) or "unknown"
 

app/sovran_systemsos_web/templates/index.html

@@ -55,7 +55,12 @@
       <div id="sidebar-support"></div>
       <div id="sidebar-features"></div>
     </aside>
-    <div id="tiles-area"></div>
+    <div id="tiles-area">
+      <div class="dashboard-loading" role="status" aria-live="polite">
+        <span class="dashboard-loading-spinner" aria-hidden="true"></span>
+        <span>Loading service status…</span>
+      </div>
+    </div>
   </main>
 
   <!-- Update modal -->

app/sovran_systemsos_web/versions.json

@@ -5,10 +5,10 @@
   "bitcoind.service": "27.1.0",
   "electrs.service": "0.10.6",
   "lnd.service": "0.18.0",
-  "rtl.service": "0.15.2",
+  "rtl.service": "0.15.8",
   "btcpayserver.service": "2.4.2",
   "albyhub.service": "1.8.0",
-  "mempool.service": "3.0.0",
+  "mempool.service": "3.2.1",
   "matrix-synapse.service": "1.115.0",
   "livekit.service": "1.5.2",
   "vaultwarden.service": "1.32.0",

modules/core/sovran-hub.nix

@@ -135,13 +135,16 @@ let
     "bitcoind.service" = if pkgs ? bitcoind-knots then pkgs.bitcoind-knots.version else (if pkgs ? bitcoind then pkgs.bitcoind.version else "27.1.0");
     "electrs.service" = if pkgs ? electrs then pkgs.electrs.version else "0.10.6";
     "lnd.service" = if pkgs ? lnd then pkgs.lnd.version else "0.18.0";
-    "rtl.service" = if pkgs ? clightning-rtl then pkgs.clightning-rtl.version else (if pkgs ? rtl then pkgs.rtl.version else "0.15.2");
+    # Keep the fallbacks aligned with the vendored packages used by the
+    # service modules (RTL 0.15.8 and Mempool 3.2.1).  The nixpkgs attrs are
+    # optional because these packages are built locally in this repository.
+    "rtl.service" = if pkgs ? clightning-rtl then pkgs.clightning-rtl.version else (if pkgs ? rtl then pkgs.rtl.version else "0.15.8");
     # BTCPay Server is intentionally sourced from pkgs.stable by the service
     # module.  Read the configured package here rather than pkgs.btcpayserver
     # (unstable), otherwise the Hub can advertise a version that is not running.
     "btcpayserver.service" = lib.getVersion config.services.btcpayserver.package;
     "albyhub.service" = if pkgs ? albyhub then pkgs.albyhub.version else "1.8.0";
-    "mempool.service" = if pkgs ? mempool then pkgs.mempool.version else "3.0.0";
+    "mempool.service" = if pkgs ? mempool then pkgs.mempool.version else "3.2.1";
     "matrix-synapse.service" = if pkgs ? matrix-synapse then pkgs.matrix-synapse.version else "1.115.0";
     "livekit.service" = if pkgs ? livekit then pkgs.livekit.version else "1.5.2";
     "vaultwarden.service" = if pkgs ? vaultwarden then pkgs.vaultwarden.version else "1.32.0";
@@ -320,7 +323,8 @@ let
 
   sovran-hub-web = pkgs.python3Packages.buildPythonApplication {
     pname   = "sovran-systemsos-hub-web";
-    version = "1.0.0";
+    # Keep the package metadata in lockstep with the version shown in the Hub.
+    version = sovranVersion;
     format  = "other";
 
     src = ../../app;