aboutsummaryrefslogtreecommitdiffstats
path: root/README.md
diff options
context:
space:
mode:
authorTim Keller <tjk@tjkeller.xyz>2026-09-02 21:01:46 -0500
committerTim Keller <tjk@tjkeller.xyz>2026-09-02 21:01:46 -0500
commite88a880435ebdda6e50bf16a493abb85b4e80d15 (patch)
treecc59b2810868bcc90bb2772380945fc024f7f5ee /README.md
parenta74c2877571afbfd04584b8821c754e9050b9e72 (diff)
downloadtelemetry-flake-flake-modules.tar.xz
telemetry-flake-flake-modules.zip
add NVMe support, and give the self-test collector the device listHEADmasterflake-modules
Two configurations the modules claimed to support and did not. NVMe drives had no per-drive section. smartctl:bus routes a drive by metric presence -- ATA attribute table, or SCSI grown defect list -- and NVMe emits neither, confirmed against a live smartctl_exporter 0.14.0 on real hardware. With no bus label the smartctl:row join dropped them, so they appeared in Fleet Overview and All Drives but could not be opened. Adds an nvme arm keyed on smartctl_device_percentage_used, verdict rules over the SMART/Health Information log, and a third repeating section. The verdict is BAD on a critical warning, media errors, or available spare below the drive's own threshold, and CAUTION at 90% of rated endurance. num_err_log_entries is deliberately not in it: the NVMe drive in this machine is healthy and sits at 2200, because it counts every command that ever returned an error. smart.devices reached the exporter but not the self-test collector, which always ran its own `smartctl --scan` -- exactly what cannot see drives behind a RAID controller, which is the only reason to set the option. The collector now takes the same PATH[;TYPE] syntax and passes -d through. That means it also has to reproduce the exporter's device label, which is not the basename: /dev/bus/0 as megaraid,1 is bus_0_megaraid_1. Ported buildDeviceLabel with its /dev/disk/by-{id,path}/ prefix stripping and the comma rule, and the scan-time scsi -> auto fallback. Getting this wrong fails silently -- every join in rules/smartctl.yml simply misses. Verified: promtool; a Prometheus fixture covering all three transports plus a megaraid-labelled device, asserting bus and verdict per drive and resolving $ata_device/$sas_device/$nvme_device the way Grafana does; the grid layout; the ExecStart with and without smart.devices; and the collector against this machine's NVMe drive. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JKczN1f66kmLxNBLX37voQ
Diffstat (limited to 'README.md')
-rw-r--r--README.md10
1 files changed, 5 insertions, 5 deletions
diff --git a/README.md b/README.md
index e86edd3..92ed419 100644
--- a/README.md
+++ b/README.md
@@ -18,8 +18,8 @@ What *has* been checked:
- every dashboard query executed against a real Prometheus fed fixture data
- `promtool` over the recording rules and the generated scrape config
- `nix flake check`: both modules evaluate, and both dashboard generators run
-- the collectors run against mock `smartctl`/`zpool`/`zfs`, and partially against real
- hardware
+- the collectors run against mock `smartctl`/`zpool`/`zfs`, and against a real NVMe drive
+- the NVMe metric names the rules depend on, checked against a live `smartctl_exporter`
What has **not**:
@@ -156,7 +156,7 @@ host and the whole page swaps to it.
| dashboard | picker(s) |
|---|---|
-| Drive Health | **Host** → SATA drive / SAS drive |
+| Drive Health | **Host** → SATA drive / SAS drive / NVMe drive |
| ZFS Pools | **Host** → Pool |
| Node Exporter Full | Job → Nodename → Instance |
| Smartctl Exporter | Node |
@@ -233,8 +233,8 @@ Shipped in `rules/alerts.yml`, loaded unless you set `alerts.defaultRules = fals
| | condition | severity |
|---|---|---|
-| `DriveFailing` | SMART self-assessment failed, an attribute is below its threshold, or uncorrectable errors | critical |
-| `DriveCaution` | reallocated/pending sectors, or a non-empty SAS grown defect list | warning |
+| `DriveFailing` | SMART self-assessment failed, an attribute is below its threshold, uncorrectable errors, or an NVMe critical warning | critical |
+| `DriveCaution` | reallocated/pending sectors, a non-empty SAS grown defect list, or 90%+ NVMe endurance used | warning |
| `SelfTestFailed` | last short or extended self-test did not pass | critical |
| `DriveOverheating` | above 55 °C for 30 minutes | warning |
| `PoolNotOnline` | pool is DEGRADED, FAULTED, … | critical |