# Backend Health Monitoring

# Backend Health Monitoring

<details>
<summary>Relevant source files</summary>

The following files were used as context for generating this wiki page:

- [src/app.d.ts](src/app.d.ts)
- [src/lib/api-types.ts](src/lib/api-types.ts)
- [src/lib/app-connectivity-events.ts](src/lib/app-connectivity-events.ts)
- [src/lib/backend-availability.svelte.ts](src/lib/backend-availability.svelte.ts)
- [src/lib/backend-availability.ts](src/lib/backend-availability.ts)
- [src/lib/components/AppShell.svelte](src/lib/components/AppShell.svelte)
- [src/lib/components/AppSidebar.svelte](src/lib/components/AppSidebar.svelte)
- [src/lib/version.ts](src/lib/version.ts)

</details>



The backend health monitoring system provides real-time visibility into the availability of the Primebrick API and its critical dependencies (Database and Identity Provider). It implements a reactive polling mechanism that transitions the UI into degraded modes when connectivity is lost and automatically triggers recovery logic when services return.

## Core State Management

The system's state is centralized in `backendState`, a reactive Svelte 5 rune object. This ensures that UI components like the Sidebar Health Chip and the Global Error Boundary stay synchronized without the overhead of manual store subscriptions [src/lib/backend-availability.svelte.ts:19-30]().

### The `backendState` Object
| Property | Type | Description |
| :--- | :--- | :--- |
| `offline` | `boolean` | `true` if the API gateway/server is unreachable [src/lib/backend-availability.svelte.ts:20](). |
| `dbOk` | `boolean \| null` | Status of the Postgres/Database connection [src/lib/backend-availability.svelte.ts:22](). |
| `idpOk` | `boolean \| null` | Status of the OIDC/Identity Provider connection [src/lib/backend-availability.svelte.ts:24](). |
| `health` | `HealthPayload \| null` | The full JSON response from the `/api/v1/health` endpoint [src/lib/backend-availability.svelte.ts:26](). |
| `healthChip` | `HealthChipState` | Derived semantic state for UI rendering (`ok`, `db_offline`, etc.) [src/lib/backend-availability.svelte.ts:29](). |

### Health State Transitions
The following diagram illustrates how the system transitions between different health states based on API responses.

**Backend Health State Machine**
```mermaid
graph TD
    subgraph "State Management (backend-availability.svelte.ts)"
        START["Initial: loading"] --> PROBE["probeHealth()"]
        PROBE --> SUCCESS["200 OK / 503 Service Unavailable"]
        PROBE --> FAIL["Network Error / 502 / 504"]
        
        SUCCESS --> VALIDATE["isValidHealthPayload()"]
        VALIDATE -- "Valid JSON" --> UPD_STATE["Update backendState"]
        VALIDATE -- "Invalid" --> OFFLINE["setBackendOffline(true)"]
        
        UPD_STATE --> CHIP_OK["'ok'"]
        UPD_STATE --> CHIP_DB["'db_offline'"]
        UPD_STATE --> CHIP_IDP["'idp_offline'"]
        
        FAIL --> OFFLINE
        OFFLINE --> CHIP_BE["'backend_offline'"]
    end
```
Sources: [src/lib/backend-availability.svelte.ts:42-52](), [src/lib/backend-availability.svelte.ts:125-169](), [src/lib/api-types.ts:18-31]()

## Probing and Validation

### `probeHealth()`
The `probeHealth` function is the primary entry point for checking connectivity. It targets the `GET /api/v1/health` endpoint [src/lib/backend-availability.svelte.ts:125](). 
- **Concurrency Guard**: It uses an internal `inflight` promise to ensure multiple simultaneous calls (e.g., from different components) share a single request [src/lib/backend-availability.svelte.ts:126-132]().
- **Forced Re-verification**: The `force: true` option allows bypassing the inflight guard, which is used by `noteGatewayFailure` when a standard API call fails [src/lib/backend-availability.svelte.ts:171-180]().

### `isValidHealthPayload()`
To prevent false positives (such as an ISP redirect page or a proxy HTML error being parsed as valid), the system strictly validates the JSON structure using `isValidHealthPayload` [src/lib/api-types.ts:18-31](). It requires the presence of `ok: true`, a `service` string, and nested objects for `db` and `idp` [src/lib/api-types.ts:21-30]().

Sources: [src/lib/backend-availability.svelte.ts:125-169](), [src/lib/api-types.ts:18-31]()

## Connectivity Lifecycle

### Polling Loop
The `AppShell.svelte` component manages the health-check lifecycle. 
- **Initialization**: A probe is triggered on mount [src/lib/components/AppShell.svelte:23]().
- **Reactive Polling**: If the system detects it is offline, or if a dependency (DB/IDP) is down, an `$effect` starts a 5-second interval loop to monitor for recovery [src/lib/components/AppShell.svelte:49-58]().

### Recovery Events
When the system transitions from a degraded state (`backend_offline`, `db_offline`, or `idp_offline`) to `ok`, it dispatches a `primebrick:connectivity-restored` custom event [src/lib/app-connectivity-events.ts:4-13](). 
- **`onConnectivityRestored()`**: A utility function that allows components to subscribe to this recovery event [src/lib/app-connectivity-events.ts:18-36]().
- **Module Reloading**: `AppShell` uses this transition to re-attempt loading the navigation modules via `loadShellNav()` if they previously failed [src/lib/components/AppShell.svelte:61-71]().

Sources: [src/lib/components/AppShell.svelte:22-71](), [src/lib/app-connectivity-events.ts:1-36]()

## UI Components

### Health Chip
The `AppSidebar` renders a visual indicator of the backend status. The chip's label and color are derived directly from `backendState.healthChip` [src/lib/components/AppSidebar.svelte:121-123]().

| State | Color (Light/Dark) | Translation Key |
| :--- | :--- | :--- |
| `backend_offline` | Red | `shell.health.beOffline` |
| `db_offline` | Red | `shell.health.dbOffline` |
| `idp_offline` | Orange | `shell.health.idpOffline` |
| `ok` | Emerald | `shell.health.beOnline` |
| `loading` | Muted | `common.loading` |

Sources: [src/lib/components/AppSidebar.svelte:144-166]()

### AppShell Global Error Boundary
The `AppShell` acts as a global safety net by listening for unhandled window errors and promise rejections. These are captured and pushed into the `appErrors` store to be displayed in the UI [src/lib/components/AppShell.svelte:26-45]().

**Connectivity Monitoring Architecture**
```mermaid
graph LR
    subgraph "UI Layer (AppSidebar.svelte)"
        HC["Health Chip"]
    end

    subgraph "Core Logic (backend-availability.svelte.ts)"
        BS["backendState (Rune)"]
        PH["probeHealth()"]
        NGF["noteGatewayFailure()"]
    end

    subgraph "Lifecycle (AppShell.svelte)"
        POLL["Polling Interval (5s)"]
        EB["Error Boundary (window.onerror)"]
    end

    subgraph "API Layer (api.ts)"
        AF["apiFetch()"]
    end

    AF -- "502/503/504" --> NGF
    NGF -- "force: true" --> PH
    PH -- "Update" --> BS
    BS -- "Reactive Sync" --> HC
    POLL -- "Every 5s if Degraded" --> PH
    EB -- "pushAppError()" --> UI_ERRORS["ErrorsPanel"]
```
Sources: [src/lib/components/AppShell.svelte:26-58](), [src/lib/backend-availability.svelte.ts:171-180](), [src/lib/components/AppSidebar.svelte:121-123]()

---