|
The repeated per-drive sections are driven by $ata_device/$sas_device, which
were label_values(smartctl:row{...}, row). label_values resolves against
/api/v1/label/<l>/values over the *dashboard's* time range -- 24h by default --
so it returns every row line seen in that window, not the ones that exist now.
Replace a drive and the window still holds the line the old drive produced.
Both lines carry the same variable value, because the regex reduces them to the
bare device name that the panels select on, and Grafana's
metricNamesToVariableValues ends in uniqBy(options, 'value'), which keeps
whichever series Prometheus returned first. The dashboard then renders one row
headed with the removed drive's model and serial, no row for its replacement,
and panels underneath showing the new drive's numbers.
query_result() is the only variable form the Prometheus datasource evaluates as
an instant query (metric_find_query.ts: queryResultQuery hits /api/v1/query at
range.to), so switch both device variables to it and re-anchor the regex on the
row= label in the line Grafana renders for a vector result.
Verified against a real Prometheus loading rules/smartctl.yml, with a fixture
that replaces a SAS drive and a SATA drive in place: the old form headed sde
with OLDBAD99 and sdz with the 870 EVO it no longer contains, the new form
lists exactly the drives in the machine.
The host variable deliberately stays on label_values -- its window covering the
time range is what keeps an unreachable machine selectable.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JKczN1f66kmLxNBLX37voQ
|
|
Replaces the single lgtm.nix with a flake exposing two NixOS modules under
services.telemetry, so the web services can move into a container declared in
the consuming config rather than one hardcoded here.
modules/collectors.nix host-side: node + smartctl exporters, the ZFS and
SMART textfile collector timers, Alloy journal
shipping. Must run on the host -- reads /proc, /sys,
raw block devices and ZFS ioctls.
modules/server.nix Prometheus, Loki, Grafana, optional Caddy.
modules/common.nix ports, shared by both halves.
modules/dashboards.nix jq fixup and grafana.com fetches.
modules/packages.nix the two collectors as ordinary packages.
Modules are exported as paths, not functions, so they evaluate with the
consumer's nixpkgs; the flake's own input only affects packages/checks/devShells.
Multi-host support. collectorHosts is a list, both scrape jobs relabel
__address__ into a `host` label, and all 133 queries in the two generated
dashboards are scoped to host=~"$host" behind a single-select Host picker.
`instance` could not serve as that key: one machine is scraped twice on
different ports (textfile metrics on :9100, smartctl_* on :9633) and the
recording rules join across that boundary.
Grafana's secret key becomes a plain secretKeyFile path wired through
LoadCredential, with an assertion -- nixpkgs 26.05 dropped the default. No
sops-nix input.
nix flake check runs promtool over the rules, diffs the committed dashboard
JSON against what the generators produce, and evaluates both modules.
Also fixes smart-selftest emitting a placeholder identity line when smartctl
cannot open a device. smartctl exits 0 with well-formed but empty JSON in that
case, so "SSD - unknown model - no serial" was reaching a dashboard row title;
it now emits nothing and lets the smartctl:ident fallback apply.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JKczN1f66kmLxNBLX37voQ
|
