add NVMe support, and give the self-test collector the device listHEADmasterflake-modules

Two configurations the modules claimed to support and did not.

NVMe drives had no per-drive section. smartctl:bus routes a drive by metric
presence -- ATA attribute table, or SCSI grown defect list -- and NVMe emits
neither, confirmed against a live smartctl_exporter 0.14.0 on real hardware.
With no bus label the smartctl:row join dropped them, so they appeared in Fleet
Overview and All Drives but could not be opened. Adds an nvme arm keyed on
smartctl_device_percentage_used, verdict rules over the SMART/Health
Information log, and a third repeating section.

The verdict is BAD on a critical warning, media errors, or available spare
below the drive's own threshold, and CAUTION at 90% of rated endurance.
num_err_log_entries is deliberately not in it: the NVMe drive in this machine
is healthy and sits at 2200, because it counts every command that ever
returned an error.

smart.devices reached the exporter but not the self-test collector, which
always ran its own `smartctl --scan` -- exactly what cannot see drives behind a
RAID controller, which is the only reason to set the option. The collector now
takes the same PATH[;TYPE] syntax and passes -d through.

That means it also has to reproduce the exporter's device label, which is not
the basename: /dev/bus/0 as megaraid,1 is bus_0_megaraid_1. Ported
buildDeviceLabel with its /dev/disk/by-{id,path}/ prefix stripping and the
comma rule, and the scan-time scsi -> auto fallback. Getting this wrong fails
silently -- every join in rules/smartctl.yml simply misses.

Verified: promtool; a Prometheus fixture covering all three transports plus a
megaraid-labelled device, asserting bus and verdict per drive and resolving
$ata_device/$sas_device/$nvme_device the way Grafana does; the grid layout;
the ExecStart with and without smart.devices; and the collector against this
machine's NVMe drive.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JKczN1f66kmLxNBLX37voQ

1 files changed, 39 insertions, 1 deletions

diff --git a/rules/smartctl.yml b/rules/smartctl.yml
index 3982e06..60a2caa 100644
--- a/rules/smartctl.yml
+++ b/rules/smartctl.yml
@@ -59,6 +59,36 @@ groups:
           + sum by (host, device) (smartctl_write_total_uncorrected_errors)
           or (count by (host, device) (smartctl_device_smart_status) * 0)
 
+      # --- NVMe -----------------------------------------------------------------
+      # NVMe has neither an ATA attribute table nor SCSI error counters. Its health is
+      # in the SMART/Health Information log, which smartctl_exporter emits only when the
+      # interface is nvme.
+      #
+      # Each arm is reduced to {host, device} *before* the `or`, so the union
+      # deduplicates per drive rather than emitting one series per metric name -- the
+      # count is then "did anything fire", which is all smartctl:health_code asks.
+      - record: smartctl:nvme_failing:count
+        expr: |
+          (
+            count by (host, device) (smartctl_device_critical_warning > 0)
+            or
+            count by (host, device) (smartctl_device_media_errors > 0)
+            or
+            count by (host, device) (
+              smartctl_device_available_spare
+              < on(host, device) smartctl_device_available_spare_threshold
+            )
+          )
+          or (count by (host, device) (smartctl_device_smart_status) * 0)
+
+      # percentage_used is the controller's own estimate of consumed write endurance,
+      # where 100 means the drive has reached its rated lifetime. Drives routinely run
+      # well past it, so this is a warning and not a failure.
+      - record: smartctl:nvme_caution:count
+        expr: |
+          count by (host, device) (smartctl_device_percentage_used >= 90)
+          or (count by (host, device) (smartctl_device_smart_status) * 0)
+
       # --- Verdict --------------------------------------------------------------
       # 0 good, 1 caution, 2 bad. The inner clamp_max on each group keeps two
       # simultaneous caution signals from adding up to a false "bad".
@@ -69,22 +99,30 @@ groups:
                   (max by (host, device) (smartctl_device_smart_status) == bool 0)
                 + (smartctl:attr_failing:count > bool 0)
                 + (smartctl:scsi_uncorrected:count > bool 0)
+                + (smartctl:nvme_failing:count > bool 0)
               , 1)
             + clamp_max(
                   (smartctl:attr_caution:count > bool 0)
                 + (smartctl:scsi_defects:count > bool 0)
+                + (smartctl:nvme_caution:count > bool 0)
               , 1)
           , 2)
 
       # --- Row labels -----------------------------------------------------------
       # Which section of the SMART dashboard a drive belongs in. Keyed on metric
       # presence rather than the `protocol` label: only ATA drives report an attribute
-      # table, and only SCSI drives report a grown defect list.
+      # table, only SCSI drives report a grown defect list, and smartctl_exporter emits
+      # percentage_used only for nvme. The three are disjoint by construction, so the
+      # order of the `or` arms does not matter -- but a drive matching none of them gets
+      # no bus, drops out of the smartctl:row join, and silently loses its dashboard
+      # section, which is what NVMe used to do.
       - record: smartctl:bus
         expr: |
           label_replace(count by (host, device) (smartctl_device_attribute), "bus", "ata", "", "")
           or
           label_replace(count by (host, device) (smartctl_scsi_grown_defect_list), "bus", "sas", "", "")
+          or
+          label_replace(count by (host, device) (smartctl_device_percentage_used), "bus", "nvme", "", "")
 
       # The identity string from collectors/smart-selftest.py, made total so a drive
       # still gets a row if that collector has not written its file yet. `unless` (not