| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
The repeated per-drive sections are driven by $ata_device/$sas_device, which
were label_values(smartctl:row{...}, row). label_values resolves against
/api/v1/label/<l>/values over the *dashboard's* time range -- 24h by default --
so it returns every row line seen in that window, not the ones that exist now.
Replace a drive and the window still holds the line the old drive produced.
Both lines carry the same variable value, because the regex reduces them to the
bare device name that the panels select on, and Grafana's
metricNamesToVariableValues ends in uniqBy(options, 'value'), which keeps
whichever series Prometheus returned first. The dashboard then renders one row
headed with the removed drive's model and serial, no row for its replacement,
and panels underneath showing the new drive's numbers.
query_result() is the only variable form the Prometheus datasource evaluates as
an instant query (metric_find_query.ts: queryResultQuery hits /api/v1/query at
range.to), so switch both device variables to it and re-anchor the regex on the
row= label in the line Grafana renders for a vector result.
Verified against a real Prometheus loading rules/smartctl.yml, with a fixture
that replaces a SAS drive and a SATA drive in place: the old form headed sde
with OLDBAD99 and sdz with the 870 EVO it no longer contains, the new form
lists exactly the drives in the machine.
The host variable deliberately stays on label_values -- its window covering the
time range is what keeps an unreachable machine selectable.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JKczN1f66kmLxNBLX37voQ
|
|
The JSON is machine-written and the generator beside it is the only source of
truth, so committing both only created a way for them to drift -- which is what
the dashboards-current check existed to police. modules/dashboards.nix now runs
gen_zfs.py / gen_smart.py in the derivation, so a stale dashboard cannot be
built at all and the check is gone.
Note there is no third option: flakes copy only git-tracked files into the
store, so a gitignored dashboards/*.json cannot be read from disk either. It is
generate, or commit.
Verified equivalent -- both generated dashboards are byte-identical to the JSON
this commit deletes.
Adds a dashboards-nonempty check. Factoring the shared jq invocation out first
introduced a bug it caught: the multi-line string ended in a newline, so the
callers' `< $src > "$out"` landed on a line of its own as a separate command,
truncating every dashboard to zero bytes while the build still succeeded. All
four provisioned dashboards were empty. jqFixup is now a single line, and the
check asserts each dashboard is non-empty and has a title, uid and panels.
Both failure modes are exercised: an empty dashboard and one with no panels each
fail the check. It also asserts it found four files, because the first version
globbed *.json against linkFarm entries named without the suffix and passed
vacuously.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JKczN1f66kmLxNBLX37voQ
|
|
Replaces the single lgtm.nix with a flake exposing two NixOS modules under
services.telemetry, so the web services can move into a container declared in
the consuming config rather than one hardcoded here.
modules/collectors.nix host-side: node + smartctl exporters, the ZFS and
SMART textfile collector timers, Alloy journal
shipping. Must run on the host -- reads /proc, /sys,
raw block devices and ZFS ioctls.
modules/server.nix Prometheus, Loki, Grafana, optional Caddy.
modules/common.nix ports, shared by both halves.
modules/dashboards.nix jq fixup and grafana.com fetches.
modules/packages.nix the two collectors as ordinary packages.
Modules are exported as paths, not functions, so they evaluate with the
consumer's nixpkgs; the flake's own input only affects packages/checks/devShells.
Multi-host support. collectorHosts is a list, both scrape jobs relabel
__address__ into a `host` label, and all 133 queries in the two generated
dashboards are scoped to host=~"$host" behind a single-select Host picker.
`instance` could not serve as that key: one machine is scraped twice on
different ports (textfile metrics on :9100, smartctl_* on :9633) and the
recording rules join across that boundary.
Grafana's secret key becomes a plain secretKeyFile path wired through
LoadCredential, with an assertion -- nixpkgs 26.05 dropped the default. No
sops-nix input.
nix flake check runs promtool over the rules, diffs the committed dashboard
JSON against what the generators produce, and evaluates both modules.
Also fixes smart-selftest emitting a placeholder identity line when smartctl
cannot open a device. smartctl exits 0 with well-formed but empty JSON in that
case, so "SSD - unknown model - no serial" was reaching a dashboard row title;
it now emits nothing and lets the smartctl:ident fallback apply.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JKczN1f66kmLxNBLX37voQ
|
|
|
