> For the complete documentation index, see [llms.txt](https://docs.holoplot.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.holoplot.com/holoplot-control/maintaining-the-system/monitoring-device-health.md).

# Monitoring device health

Monitor current system health, identify affected devices, and investigate active issues.

HOLOPLOT Control monitors the health of every device in your system. It combines these signals into clear health statuses for Modules, Arrays, Controllers, Processors, and the overall system. These statuses are displayed on the Device List.

Use this page to investigate a **current** condition. Use the [Event Log](/holoplot-control/maintaining-the-system/device-events-list.md) to investigate changes over time.

### Investigate a health issue

1. Check the system, Array, or device status in the **Device List**.
2. Use **Filter** and **Issue Tracker** to find affected Modules.
3. Open the device side panel for component-level status.

Use the [Issue Tracker reference](/holoplot-control/maintaining-the-system/device-issues-list.md) when you need an exact issue name or its meaning.

### Health statuses <a href="#health-statuses" id="health-statuses"></a>

Every device and every aggregated view reports one of five statuses. Statuses have a fixed severity order: `OK < Warning < Error < Critical`. When multiple inputs are combined, the highest severity wins.​

| Status      | Modules                                                                                                                                    | Controllers / Processors                                                                                                     |
| ----------- | ------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------- |
| 🟢 OK       | Everything is functioning correctly. Sound can be played without issues.                                                                   | Everything is functioning correctly. Operation is normal.                                                                    |
| 🟡 Warning  | Sound can be played, there might be minor issues that compromise the system’s performance. Show can go on, but should be checked later on. | Controllers/Processors still function, but minor issues are observed. The show can go on, but it should be checked later on. |
| 🟠 Error    | Modules can still make sound, but with severe degradation to system’s performance. Should be checked as soon as possible.                  | Controllers/Processors impacted, and major issues are observed. It should be checked as soon as possible.                    |
| 🔴 Critical | Modules are severely compromised and might not make sound.                                                                                 | Controllers/Processors cannot function properly.                                                                             |
| ⚪️ Unknown  | Modules are not reporting any information, or we cannot accurately report its health.                                                      | Controllers/Processors are not reporting any information, or we cannot accurately report their health.                       |

### Device status

#### Modules

From HOLOPLOT OS 2.6.3 onwards, 'Power Status' is renamed to 'Status'. The following states are displayed for all Modules in the Device List:

<table><thead><tr><th>Status</th><th width="206.375">What it means</th><th width="100.421875">Audio</th><th>What you can do</th></tr></thead><tbody><tr><td>Online</td><td>The Module is reporting and its amplifiers are powered.</td><td>✅</td><td>Everything: sleep, reboot, blink and driver tests.</td></tr><tr><td>Waking up</td><td>The Module is powering its amplifiers back up after a wake command.</td><td>Not yet</td><td>Wait. Driver tests stay unavailable until the module is online.</td></tr><tr><td>Going to sleep</td><td>The Module is powering its amplifiers down after a sleep command.</td><td>Stopping</td><td>Wake to cancel.</td></tr><tr><td>Sleeping</td><td>Amplifiers are off. The Module stays on the network and keeps reporting health.</td><td>❌</td><td>Wake to bring it back online. Health readings remain available.</td></tr><tr><td>Rebooting</td><td>The Module is restarting.</td><td>❌</td><td>Wait. If stuck in a rebooting state, contact HOLOPLOT.</td></tr><tr><td>Offline</td><td>The Module is not reporting statuses to the controller. Check the power and network link.</td><td>❌</td><td>Nothing remotely. Needs a physical check.</td></tr></tbody></table>

#### Controllers and processors

<table><thead><tr><th width="228.65234375">Status</th><th>What it means</th><th>What you can do</th></tr></thead><tbody><tr><td>Online</td><td>The device is reporting to the system. Hardware, network and software health are read live.</td><td>Normal operation.</td></tr><tr><td>Offline</td><td>The device is unreachable. Its live health readings are cleared. It contributes to the Critical percentage for Controller or Processor aggregation.</td><td>Check the power and control network link.</td></tr><tr><td>Unknown</td><td>No status has arrived from this device yet; it may be starting up or may never have reported. In aggregation, it counts as Unknown.</td><td>Wait, then check the power and control network link.</td></tr></tbody></table>

***

## How health is calculated

Health is calculated in layers. Each device first evaluates its own components. HOLOPLOT Control then aggregates those results into higher-level views. Expand the sections below to learn how health statuses are calculated for each device type.

<details>

<summary>Per-device component conditions</summary>

### Per-device health

* **Modules** combine Network and Hardware.
* **Controllers** combine Online, Network, and Hardware.
* **Processors** combine Online, Software, Network, and Hardware.

In every case, the overall device health is the worst status across its relevant components.

### Component conditions — Controllers and Processors

Each component group takes the worst status across its parts.

| Component             | Condition                                                                                     | Status      |
| --------------------- | --------------------------------------------------------------------------------------------- | ----------- |
| Drives                | Up to 25% of drives failed                                                                    | 🟠 Error    |
| Drives                | More than 25% of drives failed, or drive data unavailable                                     | 🔴 Critical |
| Fans                  | Up to 25% of fans failed                                                                      | 🟠 Error    |
| Fans                  | More than 25% of fans failed, or fan data unavailable                                         | 🔴 Critical |
| Power supplies (PSUs) | At least 50% failed                                                                           | 🟡 Warning  |
| Power supplies (PSUs) | At least 60% failed                                                                           | 🔴 Critical |
| Network interface     | A link is down, an expected IP or MAC address is missing, or the link speed is below expected | 🟠 Error    |
| NTP                   | Not synchronized to a time server                                                             | 🔴 Critical |

Processors also evaluate PTP as part of Network, and software conditions as part of Software. On Controllers and Processors, loss of NTP synchronization is always 🔴 Critical.

{% hint style="info" %}
Starting with HOLOPLOT OS 2.6.3, the HOLOPLOT VPN connection does not affect Controller health calculations.
{% endhint %}

### Component conditions — Modules

A Module’s health is the worst of its Hardware and Network components. Percentages below apply to the module’s own parts, such as amplifiers and drivers in each frequency band.

| Component                        | Condition                                                                                   | Status      | Notes                                                                                                  |
| -------------------------------- | ------------------------------------------------------------------------------------------- | ----------- | ------------------------------------------------------------------------------------------------------ |
| Hardware self-report             | Reports `Okay`                                                                              | 🟢 OK       | —                                                                                                      |
| Hardware self-report             | Reports `Warning`                                                                           | 🟡 Warning  | —                                                                                                      |
| Hardware self-report             | Reports `Error` or `Critical`, or is `Unknown` or `Initializing`                            | 🔴 Critical | —                                                                                                      |
| Amplifiers                       | At least 10% are in a fault state                                                           | 🟡 Warning  | —                                                                                                      |
| Amplifiers                       | At least 30% are in a fault state                                                           | 🟠 Error    | Healthy states are `OK`, `Idle`, and `Suspended`. Any other state counts toward the fault percentage.  |
| Amplifiers                       | At least 50% are in a fault state                                                           | 🔴 Critical | —                                                                                                      |
| Power supplies (PSUs)            | At least 50% failed                                                                         | 🟡 Warning  | —                                                                                                      |
| Power supplies (PSUs)            | At least 60% failed                                                                         | 🔴 Critical | —                                                                                                      |
| Drivers (per HF/LF band)         | At least 10% are out of tolerance                                                           | 🟡 Warning  | Driver status comes from self-test results. Before a self-test runs, drivers report ⚪️ Unknown.        |
| Drivers (per HF/LF band)         | At least 30% are out of tolerance                                                           | 🟠 Error    | —                                                                                                      |
| Drivers (per HF/LF band)         | At least 50% are out of tolerance                                                           | 🔴 Critical | —                                                                                                      |
| Sublinks (X1-MD80-S/X1-S21 only) | Exactly one sublink is up                                                                   | 🟢 OK       | Expected state.                                                                                        |
| Sublinks (X1-MD80-S/X1-S21 only) | Both sublinks report a fault                                                                | 🟠 Error    | —                                                                                                      |
| Network interface                | A link is down, an expected IP or MAC address is missing, or the link speed is below 1 Gbps | 🟠 Error    | —                                                                                                      |
| NTP                              | Not synchronized to a time server                                                           | 🟡 Warning  | On Modules, loss of NTP synchronization is `Warning`. On Controllers and Processors, it is `Critical`. |
| PTP (Ravenna modules)            | Not synchronized                                                                            | 🔴 Critical | —                                                                                                      |

For redundant networks, errors on up to half of a network interface group's interfaces result in 🟡 Warning. Errors on more than half result in 🔴 Critical.

For non-redundant systems, an error on the primary interface makes the group 🔴 Critical. This is configured in **Settings** › **System Settings**.

### Processor software conditions

Processors also evaluate software status.

| Component                           | Condition                                             | Status      |
| ----------------------------------- | ----------------------------------------------------- | ----------- |
| Audio engine (DSP)                  | Audio engine / ALSA error (DSP not processing)        | 🔴 Critical |
| Audio engine                        | Audio buffer under/overruns (xruns) detected (1 to 4) | 🟠 Error    |
| Audio engine                        | 5 or more xruns                                       | 🔴 Critical |
| Configuration                       | Configuration pending too long                        | 🔴 Critical |
| Configuration                       | Configuration error                                   | 🔴 Critical |
| CPU usage (audio processing thread) | At or above 80% (and below 90%)                       | 🟡 Warning  |
| CPU usage (audio processing thread) | At or above 90%                                       | 🟠 Error    |

</details>

<details>

<summary>Aggregated health thresholds</summary>

### Aggregated health

When multiple devices are summarized, HOLOPLOT Control uses percentage thresholds.

**Modules within an Array, and Arrays within the System**

| Aggregated status | ≥ % Critical | ≥ % Error | ≥ % Warning | ≥ % Unknown |
| ----------------- | ------------ | --------- | ----------- | ----------- |
| 🟡 Warning        | 1%           | 5%        | 5%          | 1%          |
| 🟠 Error          | 10%          | 30%       | —           | 30%         |
| 🔴 Critical       | 30%          | 50%       | —           | 50%         |

**Controllers**

| Aggregated status | ≥ % Critical | ≥ % Error | ≥ % Warning | ≥ % Unknown |
| ----------------- | ------------ | --------- | ----------- | ----------- |
| 🟡 Warning        | 1%           | 10%       | 10%         | 1%          |
| 🟠 Error          | 10%          | 30%       | —           | 30%         |
| 🔴 Critical       | 50%          | 80%       | —           | 50%         |

**Processors**

<table><thead><tr><th width="148.34893798828125">Aggregated status</th><th>≥ % Critical</th><th>≥ % Error</th><th>≥ % Warning</th></tr></thead><tbody><tr><td>🟡 Warning</td><td>1%</td><td>30%</td><td>50%</td></tr><tr><td>🟠 Error</td><td>10%</td><td>50%</td><td>—</td></tr><tr><td>🔴 Critical</td><td>50%</td><td>80%</td><td>—</td></tr></tbody></table>

The overall system health is the worst of the aggregated Arrays, Controllers, and Processors statuses.

An offline Controller or Processor contributes to the 🔴 Critical percentage. The aggregate status still follows the thresholds above. For example, one offline Controller in a three-Controller system contributes 33% Critical, which produces 🟠 Error. A Controller or Processor with no reported status counts as ⚪️ Unknown.

</details>

***

## Device details in the Device List

Use the side panels in the **Device List** to inspect component-level status for each device.

### Modules

<figure><img src="https://1870956965-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FEWZvA6Ecdn48SRkbJ1C8%2Fuploads%2FbU6P7HvPQZIlp7NViTgP%2F2-6Maintenance_Modules_HealthTable.png?alt=media&amp;token=34a151c7-1fe1-4c8c-9669-5400852ff86a" alt=""><figcaption><p>Device List – Modules</p></figcaption></figure>

Module health combines Network and Hardware.

* **Network** includes network interfaces, NTP, Ravenna, and PTP.
* **Hardware** includes amplifiers, PSUs, and drivers.

Status (previously Power Status) is not used in the health calculation. If a module is offline, HOLOPLOT Control will not display any health information. However, if a module was previously paired and online, then goes offline, its health is calculated as critical.

{% hint style="info" %}
If Dante is selected as the AoIP protocol, RAVENNA and PTP are not used in the health calculation.
{% endhint %}

### Controllers

<figure><img src="https://1870956965-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FEWZvA6Ecdn48SRkbJ1C8%2Fuploads%2FsXXTlM3aZytbapdatTYe%2FMaintenance_DeviceList%E2%80%93Controllers.png?alt=media&amp;token=897991b2-27b1-423a-bf77-9ba55a26e19a" alt=""><figcaption><p>Device List – Controllers</p></figcaption></figure>

Controller health combines Online, Network, and Hardware.

* **Connectivity** includes NTP.
* **Networks** include the Audio Module Control Network, HOLOPLOT Control Network, Audio Network, and Intercontroller Network.
* **Hardware** includes drives, fans, and PSUs.

### Processors

<figure><img src="https://1870956965-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FEWZvA6Ecdn48SRkbJ1C8%2Fuploads%2FZkiHmCenSPhyxsQvQdmq%2FControl_Maintenance_Processor_DeviceList.png?alt=media&amp;token=ffd3041c-7398-4ccb-b088-348f9046dea5" alt=""><figcaption><p>Device List – Processor panel</p></figcaption></figure>

Processor health combines Online, Software, Network, and Hardware.

* **Software** includes DSP processing, configuration status, audio engine state, and CPU usage.
* **Network** includes NTP, the Audio Module Control Network, Audio Bridge interfaces, and PTP.
* **Hardware** includes drives, fans, and PSUs.

***

## Filtering and finding issues

In the **Device List**, the right panel contains two cards: **Filter** and **Issue Tracker**. Use them together or separately.

<figure><img src="https://1870956965-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FEWZvA6Ecdn48SRkbJ1C8%2Fuploads%2F2wOdbtUG3k8L0umPJ5du%2FControl_Maintenance_DeviceList_Filters.gif?alt=media&amp;token=7fe9b854-e806-49c1-a56f-6e5fff25c23d" alt=""><figcaption><p>Filters &#x26; Issue Tracker</p></figcaption></figure>

### Filter

The filter behaves differently depending on whether it is collapsed or expanded.

When collapsed, it lets you monitor **Status**, **Network**, and **Hardware** health.

<figure><img src="https://1870956965-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FEWZvA6Ecdn48SRkbJ1C8%2Fuploads%2F9dgVmIEV6VSww4jFJR7g%2F2-6Maintenace_Filter-Collapsed.png?alt=media&amp;token=72638946-9840-4d7d-86be-618c47bb9ee0" alt=""><figcaption><p>Collapsed Filters</p></figcaption></figure>

When expanded, it lets you filter by **Status**, **Network**, and **Hardware**. This helps you isolate issues quickly during troubleshooting.

<figure><img src="https://1870956965-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FEWZvA6Ecdn48SRkbJ1C8%2Fuploads%2F9hrXmd0LmByy35772aji%2F2-6Maintenace_Filter-Expanded.png?alt=media&amp;token=57faea8f-138a-442b-ae80-70baa4298ee5" alt=""><figcaption><p>Expanded Filters</p></figcaption></figure>

### Issue Tracker

When a network or hardware issue is active, it appears in the **Issue Tracker** in the format `Type – Issue description`.

Select an issue to filter the affected Modules. Every issue that can appear here is described in the [Issue Tracker reference](/holoplot-control/maintaining-the-system/device-issues-list.md).

<figure><img src="https://1870956965-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FEWZvA6Ecdn48SRkbJ1C8%2Fuploads%2FQfCU43VzrdIFSnT1F6oH%2FCleanShot%202024-12-10%20at%2014.40.04%402x.png?alt=media&amp;token=220984ee-cf23-409f-9aaa-ec8f9d54b92d" alt=""><figcaption><p>Issue Tracker</p></figcaption></figure>


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.holoplot.com/holoplot-control/maintaining-the-system/monitoring-device-health.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.