> For the complete documentation index, see [llms.txt](https://docs.holoplot.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.holoplot.com/holoplot-control/maintaining-the-system/monitoring-device-health.md). # Monitoring device health HOLOPLOT Control monitors the health of every device in your system. It combines these signals into clear health statuses for Modules, Arrays, Controllers, Processors, and the overall system.Device List ### Health statuses Every device and every aggregated view reports one of five statuses.Statuses have a fixed severity order: `OK < Warning < Error < Critical`. When multiple inputs are combined, the highest severity wins. | | | | | ----------- | ------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------- | | 🟢 OK | Everything is functioning correctly. Sound can be played without issues. | Everything is functioning correctly. Operation is normal. | | 🟡 Warning | Sound can be played, there might be minor issues that compromise the system’s performance. Show can go on, but should be checked later on. | Controllers/Processors still function, but minor issues are observed. The show can go on, but it should be checked later on. | | 🟠 Error | Modules can still make sound, but with severe degradation to system’s performance. Should be checked as soon as possible. | Controllers/Processors impacted, and major issues are observed. It should be checked as soon as possible. | | 🔴 Critical | Modules are severely compromised and might not make sound. | Controllers/Processors cannot function properly. | | ⚪️ Unknown | Modules are not reporting any information, or we cannot accurately report its health. | Controllers/Processors are not reporting any information, or we cannot accurately report their health. | ## How health is calculated Health is calculated in layers. Each device first evaluates its own components. HOLOPLOT Control then aggregates those results into higher-level views. ### Per-device health * **Modules** combine Network and Hardware. Power status does not affect health. * **Controllers** combine Online, Network, and Hardware. * **Processors** combine Online, Software, Network, and Hardware. In every case, the overall device health is the worst status across its relevant components. ### Component conditions — Controllers and Processors Each component group takes the worst status across its parts. | Component | Condition | Status | | --------------------- | --------------------------------------------------------------------------------------------- | ----------- | | Drives | Up to 25% of drives failed | 🟠 Error | | Drives | More than 25% of drives failed, or drive data unavailable | 🔴 Critical | | Fans | Up to 25% of fans failed | 🟠 Error | | Fans | More than 25% of fans failed, or fan data unavailable | 🔴 Critical | | Power supplies (PSUs) | At least 50% failed | 🟡 Warning | | Power supplies (PSUs) | At least 60% failed | 🔴 Critical | | Network interface | A link is down, an expected IP or MAC address is missing, or the link speed is below expected | 🟠 Error | | NTP | Not synchronized to a time server | 🔴 Critical | | HOLOPLOT Cloud | No internet connectivity | 🟠 Error | | HOLOPLOT Cloud | Internet is available but the HOLOPLOT VPN is not connected | 🟡 Warning | Processors also evaluate PTP as part of Network, and software conditions as part of Software. On Controllers and Processors, loss of NTP synchronization is always 🔴 Critical. ### Component conditions — Modules A Module’s health is the worst of its Hardware and Network components. Percentages below apply to the module’s own parts, such as amplifiers and drivers in each frequency band. | Component | Condition | Status | Notes | | ------------------------ | ------------------------------------------------------------------------------------------- | ----------- | ------------------------------------------------------------------------------------------------------ | | Hardware self-report | Reports `Okay` | 🟢 OK | — | | Hardware self-report | Reports `Warning` | 🟡 Warning | — | | Hardware self-report | Reports `Error` or `Critical`, or is `Unknown` or `Initializing` | 🔴 Critical | — | | Amplifiers | At least 10% are in a fault state | 🟡 Warning | — | | Amplifiers | At least 30% are in a fault state | 🟠 Error | Healthy states are `OK`, `Idle`, and `Suspended`. Any other state counts toward the fault percentage. | | Amplifiers | At least 50% are in a fault state | 🔴 Critical | — | | Power supplies (PSUs) | At least 50% failed | 🟡 Warning | — | | Power supplies (PSUs) | At least 60% failed | 🔴 Critical | — | | Drivers (per HF/LF band) | At least 10% are out of tolerance | 🟡 Warning | Driver status comes from self-test results. Before a self-test runs, drivers report ⚪️ Unknown. | | Drivers (per HF/LF band) | At least 30% are out of tolerance | 🟠 Error | — | | Drivers (per HF/LF band) | At least 50% are out of tolerance | 🔴 Critical | — | | Sublinks (X1-MD80S only) | Exactly one sublink is up | 🟢 OK | Expected state. | | Sublinks (X1-MD80S only) | Both sublinks report a fault | 🟠 Error | — | | Network interface | A link is down, an expected IP or MAC address is missing, or the link speed is below 1 Gbps | 🟠 Error | — | | NTP | Not synchronized to a time server | 🟡 Warning | On Modules, loss of NTP synchronization is `Warning`. On Controllers and Processors, it is `Critical`. | | PTP (Ravenna modules) | Not synchronized | 🔴 Critical | — | For systems with redundant networking, a network interface group is 🟡 Warning while up to half of its interfaces are in error. It becomes 🔴 Critical once more than half are in error. For non-redundant systems, an error on the primary interface makes the group 🔴 Critical. Configure this in **Settings** › **System Settings**.

### Processor software conditions Processors also evaluate software status. | Component | Condition | Status | | ----------------------------------- | ----------------------------------------------------- | ----------- | | Audio engine (DSP) | Audio engine / ALSA error (DSP not processing) | 🔴 Critical | | Audio engine | Audio buffer under/overruns (xruns) detected (1 to 4) | 🟠 Error | | Audio engine | 5 or more xruns | 🔴 Critical | | Configuration | Configuration pending too long | 🔴 Critical | | Configuration | Configuration error | 🔴 Critical | | CPU usage (audio processing thread) | At or above 80% (and below 90%) | 🟡 Warning | | CPU usage (audio processing thread) | At or above 90% | 🟠 Error | ### Aggregated health When multiple devices are summarized, HOLOPLOT Control uses percentage thresholds. **Modules within an Array, and Arrays within the System**

| Aggregated status | ≥ % Critical | ≥ % Error | ≥ % Warning | ≥ % Unknown | | ----------------- | ------------ | --------- | ----------- | ----------- | | 🟡 Warning | 1% | 5% | 5% | 1% | | 🟠 Error | 10% | 30% | — | 30% | | 🔴 Critical | 30% | 50% | — | 50% | **Controllers**

| Aggregated status | ≥ % Critical | ≥ % Error | ≥ % Warning | ≥ % Unknown | | ----------------- | ------------ | --------- | ----------- | ----------- | | 🟡 Warning | 1% | 10% | 10% | 1% | | 🟠 Error | 10% | 30% | — | 30% | | 🔴 Critical | 50% | 80% | — | 50% | **Processors**

| Aggregated status | ≥ % Critical | ≥ % Error | ≥ % Warning | | ----------------- | ------------ | --------- | ----------- | | 🟡 Warning | 1% | 30% | 50% | | 🟠 Error | 10% | 50% | — | | 🔴 Critical | 50% | 80% | — | The overall system health is the worst of the aggregated Arrays, Controllers, and Processors statuses. A device that is offline counts as 🔴 Critical in aggregation. A device that is not reporting counts as ⚪️ Unknown. ## Device details in the Device List Use the side panels in the **Device List** to inspect component-level status for each device. ### Modules

Module health combines Network and Hardware. * **Power** is shown, but does not affect health. * **Network** includes network interfaces, NTP, Ravenna, and PTP. * **Hardware** includes amplifiers, PSUs, and drivers.

If Dante is selected as the AoIP protocol, Ravenna and PTP are not used in the health calculation. ### Controllers

Controller health combines Online, Network, and Hardware. * **Connectivity** includes HOLOPLOT Cloud and NTP. * **Networks** include the Audio Module Control Network, HOLOPLOT Control Network, Audio Network, and Intercontroller Network. * **Hardware** includes drives, fans, and PSUs.

### Processors

Processor health combines Online, Software, Network, and Hardware. * **Software** includes DSP processing, configuration status, audio engine state, and CPU usage. * **Network** includes HOLOPLOT Cloud, NTP, the Audio Module Control Network, Audio Bridge interfaces, and PTP. * **Hardware** includes drives, fans, and PSUs.

## Processor failover Processors can be configured as redundant pairs. One Processor runs as **Main**. The other stays ready as **Backup**. Configure this in **Settings** › **Processors**. Failover status does not affect Processor health. Health can trigger failover, but failover does not change the health calculation. ### Automatic failover When automatic failover is enabled and a backup is assigned, HOLOPLOT Control switches from the active Main Processor to its Backup when: * the active Processor health becomes 🟠 Error or 🔴 Critical * the active Processor stops streaming audio * the active Processor stops responding before the failover timeout expires A short timeout is applied before the switch. If the Main Processor recovers within that window, the pending failover is cancelled. {% hint style="warning" %} Automatic failover only switches from **Main** to **Backup**. {% endhint %} ### Manual failover Manual failover is available when a backup Processor is assigned. To switch manually, open **Settings** › **Processors**, open the three-dot menu for the pair, then select **Force failover**. Manual failover takes effect immediately. It can switch in either direction.

### Event Log Failovers are recorded as **System** events in the Event Log. To find them, filter the Event Log by the **System** category. ## Filtering and finding issues In the **Device List**, the right panel contains two cards: **Filter** and **Issue Tracker**. Use them together or separately.

### Filter The filter behaves differently depending on whether it is collapsed or expanded. When collapsed, it lets you monitor **Network** and **Hardware** statuses.

When expanded, it lets you filter by **Power**, **Network**, and **Hardware**. This helps you isolate issues quickly during troubleshooting.

### Issue Tracker When a network or hardware issue is active, it appears in the **Issue Tracker** in the format `Type – Issue description`. Select an issue to filter the affected modules.

--- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://docs.holoplot.com/holoplot-control/maintaining-the-system/monitoring-device-health.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.