A Zigbee2MQTT networkmap on a 200+ device mesh takes minutes to build.
Two separate failures fell out of that:
- POST /zigbee/import held the HTTP request open for the whole MQTT
round-trip, so any reverse proxy in front of the API cut it first
(Cloudflare returns a 524 at 120 s) and the browser never saw the map.
It now registers a job, fetches in the background and answers 202; the
client polls GET /zigbee/import/{job_id} until the payload is ready.
Job results are transient and live in memory with a 15 min TTL — the
same single-worker assumption the scheduler already makes. A failed
fetch replays the status the synchronous route used to raise, so a bad
broker is still a 502 and a slow mesh still a 504.
- The networkmap wait was hard-coded at 300 s with no way to raise it.
It now reads ZIGBEE_NETWORKMAP_TIMEOUT, and the shared MQTT round-trip
used by the Z-Wave import reads MQTT_RESPONSE_TIMEOUT. Both default to
300 s, fall back to that if misconfigured to a non-positive value, and
name themselves in the timeout message.
Also corrects the route and doc claims that the wait was 60 s.
The /import tests changed with the contract they cover, not to pass.
Fixes#380
ha-relevant: yes
Every node type persisted its size in `nodes.width` / `nodes.height` except
`groupRect`, which stashed it inside the `custom_colors` JSON next to its
colours. The columns already existed and were simply unused for zones, so
this was an inconsistency rather than a missing-column workaround, and it
put geometry in a blob that otherwise holds style.
The serializer now writes the columns for a zone too, and strips the legacy
`width`/`height` keys out of the blob so the two cannot drift apart and
leave an older canvas reading a stale size.
No data is lost on upgrade:
- `_backfill_zone_size` copies the blob geometry into the columns at
startup. It only fills a column that is still NULL, so it cannot overwrite
a size set since; it parses the JSON in Python rather than with
`json_extract`, so it does not depend on the SQLite build carrying JSON1;
and an unreadable row is skipped without costing the others their size.
Re-running it is a no-op.
- the reader still falls back to the blob, covering a payload the backfill
has not reached — an older server, or an import.
Standalone mode is unaffected: it stores React Flow nodes verbatim, so the
size was always on `node.width` / `node.height` there.
The four serializer tests that pinned the size to the blob now assert the
columns, since that is the behaviour being changed.
ha-relevant: yes
A scan runs on a background thread inside the API process. If that process
dies mid-scan — an OOM kill, docker stop, a crash — the ScanRun row stays
"running" for ever, because nothing is left alive to finish it.
That row is not just cosmetic clutter in Scan History: the trigger endpoints
reject a new scan while one is "running" for the same target, so a single kill
locks that range out permanently.
Nothing can legitimately be "running" the moment we boot, so lifespan() now
marks every such row "error" — the same word run_scan and run_device_scan
write when they fail themselves — with finished_at and an explanatory message.
Reported alongside the OOM itself in #374, which is what produced the orphans.
Fixes#374
ha-relevant: maybe
Both HTTP paths buffered the whole response body before looking at it. An
endpoint that streams without end and sends no Content-Length — a Freebox
bandwidth-test port, an MJPEG camera, a log tail — grew the backend until the
cgroup limit killed it, every 5-10 minutes at a constant ~1.04 GB RSS.
httpx's timeout does not help: it applies per network operation, not to the
total time spent draining a socket that keeps delivering data.
- status_checker._http_get only needs the status line, so it now uses
client.stream() and never reads the body at all.
- http_probe._probe_scheme needs at most _MAX_BODY_BYTES to hunt for <title>,
so it streams and stops there. The cap existed already but was applied to
resp.text, after the full body had been downloaded.
Regression tests serve an endless, Content-Length-less body through a
MockTransport and assert the read stays bounded. Both fail on the old code.
The existing mocks patched httpx.AsyncClient.get, which neither path calls
now; they are rebuilt on MockTransport — a real client over a fake network —
with every original assertion kept.
Fixes#375
ha-relevant: yes
Three findings from the PR review.
A finishing deep rescan threw away an edit in progress. The reset effect in
InventoryDeviceModal was keyed on the `device` object, and the parent hands
down a fresh one whenever the row is refreshed — including from the poll's
own onSaved. Same device, new object, so the effect reset the form and left
edit mode, minutes into a scan the user was waiting on. It is keyed on the
id now: a refreshed row is not a different device, and only a different
device is a reason to throw the form away. The fresh row still reaches the
canvas and the grid — gating that behind edit mode would discard the scan
result instead, and a save unions the services server-side anyway.
A rescan tagged every device it touched as "arp"-discovered, so a Proxmox
guest, a rack mount or a hand-added host started answering the network
source filter. It carries its own source through instead.
The two background wrappers wrote status "failed" where the scanners write
"error" for the same condition. Harmonized on "error" — the one Scan
History filters and colours; "failed" showed up unlabelled. The frontend
union keeps 'failed' for rows already recorded.
That last one meant updating an existing assertion in test_scan_run.py: it
encoded the old spelling.
ha-relevant: yes
merge_services did {**existing, **incoming}, so the fingerprint's guess at
an icon overwrote the one the user chose — and on a port no signature
covers it wrote None, clearing it outright. Since 3.3.0 the inventory row
is the only copy of a device's services, so every "Scan network" repainted
the service on every canvas drawing that device at once.
The scanner now merges with discovered=True: it still adds services and
refreshes what it knows, but leaves an established icon and category alone.
A user edit from the modal or a canvas changes them as before.
Blank incoming values no longer clear established ones either, on both
paths — an absent field is silence, not a reset. Same rule merge_properties
already follows.
ha-relevant: yes
The Deep scan link on a device now opens a small dialog instead of firing
straight away. It is prefilled with the full 1-65535 range — that is still
the point of the feature — but a user who knows where a service lives can
narrow it and get an answer in seconds instead of minutes.
The dialog takes an nmap-style spec: a port, a range, or a comma list
(80,443,8000-9000). It shows the port count live, refuses to start on
something nmap could not use, and carries three presets (all / 1-1024 /
1-10000).
Backend:
- _parse_port_spec / _port_chunks generalize the slicing that used to be
full-range only. Ranges are merged before slicing, so an overlapping
spec is never scanned twice, and small ranges are packed into one nmap
call instead of one call each. _deep_port_chunks still yields the same
eight slices as before.
- run_device_scan takes ports=; it wins over full_ports.
- The retry-free flags (--max-retries 0 --min-rate 2000) now key on the
total port count rather than on full_ports. They pay for themselves over
thousands of ports on a lossy host; over a handful they only cost
accuracy.
- RescanDeviceRequest.ports validates the spec — 422 rather than handing
nmap a bad -p. Blank means the full sweep.
Frontend utils/portSpec.ts mirrors the backend parser so a typo is caught
before the request; the backend validates again because it is the one
calling nmap.
The existing rescan tests now go through the dialog: the click path
changed, so the start is two steps.
ha-relevant: yes
A deep rescan of a slow host came back with nothing at all: the run took
its full 600s ceiling and the device's services were unchanged, so a
service deleted by hand was never rediscovered.
nmap answers --host-timeout with "Skipping host <ip> due to host timeout"
and discards every port it had already found — the ceiling turned a slow
scan into one that reports nothing. What costs the time is a host that
drops packets: 8188 of 8192 ports filtered, each waiting out its probe.
- No --host-timeout on the deep discovery pass, ever.
- The full range runs as 8 slices of 8192 ports, one nmap call each,
unioning the open ports. A slice that overruns costs its own ports, not
all of them, and the loop has somewhere to notice a stop request.
- `scanner_deep_host_timeout` is now a total budget checked between
slices (default 2700s), not an nmap flag. The first slice always runs.
- A partial sweep is reported rather than passed off as complete: the run
finishes `done` carrying "Scanned 3/8 port ranges …", and the modal
toasts a warning instead of success.
- Deep slices use --max-retries 0 --min-rate 2000. Measured against a
dropping host, 8192 ports took 329s at --max-retries 1 and 164s at 0,
finding the same ports; capping the RTT changed nothing. The range scan
keeps nmap's default retries on its curated port list.
ha-relevant: yes
Devices added before the scanner knew a service showed an empty Services
section with no way to refresh it (#350). The detail modal now starts a
full-port scan of that single device.
- `process_host` lifted out of `run_scan` so the range scan and the new
single-device scan share the same match / merge / dedupe rules — a
rescan unions services, it never replaces what the user added by hand.
- `run_device_scan`: no ping sweep, no mDNS, straight to the phase-2 nmap
pass on the device IP over all 65535 TCP ports.
- `POST /scan/pending/{id}/rescan` records a normal ScanRun
(`kind=device`, `ranges=["<ip>/32"]`), so stop, progress and Scan
History work unchanged. One run per device at a time — a second request
while the first is scanning is a 409. 404 unknown, 409 no-IP or hidden.
- `GET /scan/runs/{id}` so a caller can poll the run it started.
- Deep scan button in the Services section of the device detail, hidden
without an IP and for Zigbee. Swaps to a stop control while running,
folds the fresh services back in on completion (never over an edit in
progress).
ha-relevant: yes
A same-origin deployment behind a reverse proxy needs no CORS, so
CORS_ORIGINS is left at its localhost default — but browsers still send
Origin on unsafe methods, and OIDCCSRFMiddleware validated it against
CORS_ORIGINS alone. Every POST/PUT/DELETE came back 403 while reads
worked, so creating a canvas failed with no usable error.
The OIDC callback URL is served by this app, so the origin of
OIDC_REDIRECT_URI is the app's own and is always a legitimate CSRF
origin. origin_is_allowed now accepts it in addition to CORS_ORIGINS,
and settings validation warns at startup when CORS_ORIGINS omits it so
the misconfiguration is visible rather than silent.
Fixes#356
ha-relevant: no
Since 3.3.0 the inventory row is the only copy of a device's services and
every canvas drawing it reads that list, but the scanner still assigned
`keep.services = fingerprint_ports(...)` — the pre-split behaviour, when a
node held its own copy. One re-scan therefore deleted every service the user
added by hand, on every canvas at once, and brought back the ones they had
deleted. It unions now, like every other writer of that field.
Two more things #347 turned up:
* A node whose view matches nothing the row still holds drew nothing at all —
the row had been replaced under it, so every key was gone and every key was
new, and `apply_view` hid the lot. Such a view says nothing about the list
that replaced it, so it is treated as no view: the row is drawn. An empty
view is untouched, being a real answer ("this canvas draws none of them").
* The 3.3.3 view seed recovers a node's arrangement from the pre-3.3.0 backup,
which is 3.2.0-era and cannot know about a property added afterwards. On
3.3.0-3.3.2 the row was the only place to add one and every canvas drew it,
so seeding strictly from the backup took it off all of them at once. Those
are appended visible, keeping the recovered order and hidden flags for
everything the backup does know. Services keep the strict recovery: holding
back what a scan fingerprinted is the whole point of the view.
ha-relevant: maybe
The seed selected any node with a `device_id` and no view. Deleting a device
leaves that column dangling — SQLite runs with foreign keys off, so the
`ON DELETE SET NULL` never fires on an existing table — and such a node can be
seeded from nothing, so it stayed selected. The query matched it again on every
later start, re-opening the pre-upgrade backup and logging a recovery that
recovered nothing.
Narrowing the query to nodes whose row actually exists restores the invariant
the seed is documented to have: it runs on one boot, and never opens a file
again.
ha-relevant: no
Since 3.3.0 the inventory row owns a device's services and properties, and
every node drawing that device rendered the row wholesale. One row shared by
several canvases meant one rendering: a service the scanner fingerprinted
appeared on every canvas at once (users reported Uptime Kuma and Synology DSM
on hosts running neither — both are port-only signatures), and a property added
on one schematic showed up on all the others.
Order and visibility are presentation, so they move to the node. `display_view`
records, per node, which of the row's services and properties it draws and in
what order, keyed by `port|protocol|name` and by lowercased property key so the
view survives an edit to the fact itself. The facts stay on the row: hiding is
per node, deleting is still device-wide.
An item the view does not list is reported hidden rather than dropped, so what
a scan finds next is one toggle away instead of pushed onto every canvas.
The wire shape is unchanged. A client already sends its lists in display order
with their `visible` flags, so the view is read back out of them rather than
asking for a second field, and `visible` is only stamped when something is
hidden.
Upgrades keep what each canvas showed:
* from 3.2.0, the backfill seeds each node's view from its own legacy columns
before they are dropped;
* from 3.3.0-3.3.2 those columns are gone and the row holds the union of every
canvas, so the layout is recovered from the backup `_backup_db` took before
the 3.3.0 migration — the newest one whose `nodes` table still has the
columns, read read-only and matched by node id;
* with no usable backup the row is the seed, so every canvas keeps showing
exactly what it shows today and only later additions are held back.
Refs #347
ha-relevant: maybe
3.3.1 kept `nodes` writable when the backfill failed, but the backfill still
failed — so a canvas upgraded from 3.2.0 shows every device stripped of its ip,
services, hostname, notes and hardware. The facts are intact in the legacy
columns; nothing was ever reading them across.
The backfill reads those columns with raw SQL, so SQLite hands `last_seen` and
`last_scan` back as text. Writing text into a DateTime column raises
`TypeError: SQLite DateTime type only accepts Python datetime and date objects`,
which is neither IntegrityError nor OperationalError — so it went straight past
the per-node savepoint and aborted the whole run, exactly as before.
- Parse the legacy timestamps back into datetimes, the way `_decode_json`
already handles the JSON columns. An unparseable stamp becomes None rather
than an error: the status checker refreshes both within a minute.
- Catch every exception per node, not two chosen classes. Picking the exception
types was the defect; a node that fails for any reason must cost only itself.
The same run would also have destroyed data. A canvas saved while an earlier
migration was stuck minted a blank inventory row from a UI that had no facts to
show and linked the node to it. The backfill only looked at unlinked nodes, so
it skipped that one, counted the canvas migrated, and dropped the columns
holding its only copy of the ip and services.
- Read linked nodes too, and fill only what their row is missing. A row that
already holds a fact is never overwritten, so an edit made after the migration
survives, and the pass stays a no-op once everything has moved across.
Verified end to end against a rebuilt 3.2.0 database stuck the way the reports
describe: one boot restores ip, services, hostname, mac, notes, check method,
hardware, properties, status and last_seen, then drops the legacy columns.
Refs #347, #348, #351
ha-relevant: no
Upgrading 3.2.0 -> 3.3.0 could leave a database where approving a device —
or creating any node — failed with `NOT NULL constraint failed: nodes.status`.
The 3.3.0 migration moves the device facts off `nodes` onto `device_inventory`
and then drops the columns. The drop is skipped when a node is still unlinked,
because those columns are the only remaining copy of its facts. But a 3.2.0
`nodes` declares `status`, `services`, `properties` and `show_hardware` NOT NULL
with no server-side default, and the 3.3.0 model no longer writes them — so a
skipped drop bricks every later INSERT.
It skipped because one node killed the whole backfill: the loop shared a single
session, so an IntegrityError raised by autoflush lost every link made before it.
- Give each node its own savepoint. A node whose merge violates a constraint is
skipped and logged on its own; the rest still link. `skipped` joins the stats
reported at boot.
- Relax NOT NULL on the retained legacy columns when the drop is skipped, so the
database stays writable while later boots retry the backfill. Values are kept;
only the constraint goes. Guarded on the PRAGMA notnull flag, so it runs once.
- Share one `_rebuild_nodes` between the drop and the relax: it restores
`PRAGMA foreign_keys = ON` in a finally (the connection returns to the pool)
and clears a `nodes_new` left by a failed attempt.
- Match IEEE addresses case-insensitively, and never write one another inventory
row already owns — `device_inventory.ieee_address` is UNIQUE, and a duplicate
is one way the backfill was raising in the first place.
Tests build the real v3.2.0 schema from the tag: the existing legacy-migration
fixture declares `status` nullable, which is why this reached a release.
Fixes#351
ha-relevant: no
Three unrelated fixes from the review of this branch.
`create_pending` merging into a hidden row left it hidden, so adding a device
by hand whose IP collides with one hidden weeks ago answered 201 while nothing
appeared in the inventory — the add read as a no-op. An explicit add outranks
the earlier hide, exactly like restore. Only `hidden` is lifted; an approved
row is not walked back down the lifecycle.
Standalone stored a node's live reachability in the inventory row's `status`,
which is the pending/approved/hidden lifecycle — the backend keeps the two
apart as `status` vs `status_live`. Reachability now goes to `status_live`, and
`normalizeLifecycle` repairs rows already written the old way: the node still
reads its status on load, and the next save rewrites the row correctly.
Status checks stay device-scoped and keep covering a device no canvas draws.
That is deliberate — the inventory row is what's monitored, so a rack mount or
an inventory-only entry reports state without being drawn anywhere, and
deleting a node no longer silently stops monitoring the host. `hide`, or
clearing the check method, is what ends the checks. Documented and pinned with
tests rather than changed.
ha-relevant: maybe
A canvas node carries a full copy of its Device Inventory row, hydrated when
the canvas loads. The save routed all of it back, so a save made for nothing
but a moved node rewrote the row from a snapshot that could be hours old —
silently reverting an edit made meanwhile in the inventory modal, on another
canvas, or by the scanner.
Diffing the payload against the row server-side cannot fix this: it can't tell
"I edited this" from "the row moved on since I loaded it". Only the client
holds the baseline.
- canvasStore keeps `factsBaseline` — the device facts as received — set on
load, refreshed on save, rebased per field by `applyDeviceFacts`.
- `serializeNode` sends `changed_facts`: what this canvas actually edited.
`link_facts(changed_fields=)` writes nothing outside that list, even where
the values differ. Absent (older client, YAML import, MCP) keeps the previous
full-write behaviour.
- `changed_facts()` additionally drops facts already equal to the row, so a
no-op save writes nothing. Identity matching still uses the full payload.
- Live fields (status, last_seen…) bypass the filter — the status checker owns
reachability, and a save only ever fills a row never checked.
An inventory edit also lands on the canvases already on screen:
`applyDeviceFacts` pushes the saved row onto every node drawing it, without
marking the canvas unsaved. A fact the canvas has edited but not saved is left
alone — work in progress wins locally and still saves.
ha-relevant: maybe
`_slim_canvas` filtered `n["data"]`, the React Flow shape the frontend store
holds. `GET /api/v1/canvas` has never sent that: a node is reported flat, so
the filter matched nothing and `get_canvas` returned an id and a node type per
node — no label, no address, no services. The three tests covering it fed a
hand-built nested payload, so they passed against a shape the backend does not
produce.
Fold `data` over the node and read both, so either form slims the same way.
`type` leaves `NODE_KEEP` — flat it is the node type, not a device fact, and it
is already reported as `node_type`; `parentId` becomes `parent_id`, the key the
API uses, normalized from either spelling.
The new tests are built on a full `NodeResponse` copied from a real response,
and the backend now pins that wire shape from its side so the two move together.
ha-relevant: no
`merge_facts_into_device` applied one `replace_lists` flag to properties
and services alike, taking each wholesale from `facts`. But a PATCH only
carries the keys the client sent, so `{"properties": [...]}` replaced
services with `[]` — losing them for every canvas drawing the device and
stopping the scheduler from service-checking it. Symmetric the other way.
Gate the replace per list: a list absent from `facts` falls through to the
merge, which is a no-op against `None`. An explicit `[]` still clears, and
the canvas save path is unchanged since it always sends both keys.
ha-relevant: no
Completes the split: `nodes` now holds only how a device is drawn on one
canvas, and every device fact reaches the API from the inventory row.
- The device columns are removed from `nodes` (SQLite table rebuild, the
same shape as the existing device_inventory and canvas_state rebuilds).
The drop is skipped, and logged, while any non-furniture node is still
unlinked: those columns are the last copy of that node's facts. The
backfill therefore reads them with raw SQL — by the time it runs, the
model no longer declares them.
- The status checker iterates devices, not nodes: one check per device
however many canvases draw it, writing `status_live` / `last_seen` /
`response_time_ms` on the row. `/ws/status` messages carry `device_id`
and the node ids they light up. Hidden devices are not probed.
- Readers repointed: scanner (last_scan lands on the row), proxmox (its
node tier collapses into the inventory tier, keeping only the cluster
handles), zigbee/zwave (one property refresh serves every canvas), rack
inventory, liveview, stats, node dedupe.
- `POST /scan/pending` merges into the row that already describes the host
instead of minting a second one — one device is one row, whichever way it
was documented.
- Standalone keeps parity: the canvas blob gains `devices`, split on save
and hydrated on load. A blob written before the split still reads.
The rack inventory had a related bug: a mount that names a node explicitly
printed the mount's device rather than the pinned node's. It now reads the
node's own row.
Tests that built a node with device columns are ported to the link; where a
behaviour genuinely moved (properties refresh once on the row, last_scan is
the device's) the assertion moved with it rather than being dropped.
ha-relevant: yes
A device drawn on three canvases was three independent copies of the same
facts. Point every node at the Device Inventory row it draws, and let that
row own what the device *is* — the node keeps only how it is drawn.
- nodes.device_id -> device_inventory.id, ON DELETE SET NULL. NULL for
canvas furniture (group / groupRect / text), which describes nothing
physical. Deleting a node never deletes the row.
- services/inventory_sync holds the shared rules: matching by ieee > ip >
mac (per token, so 10.0.0.4 never matches 10.0.0.40), a property union on
key, a service union on (port, protocol, name), and the backfill that
links every pre-existing node.
- The backfill is non-destructive by construction: it writes device_id and
fills the row, and deletes nothing. Nodes are visited oldest-edit-first,
so where two canvases disagree on a scalar the most recently edited wins,
while properties and services stay unioned — nothing any canvas recorded
is lost. A second boot finds nothing to do.
- The wire shape does not change: GET /canvas hydrates the device fields
from the row, and a save routes them back to it. Editing a node's IP on
one canvas now shows on every other canvas holding that device.
- approve / bulk-approve set device_id instead of owning a copy, and a new
canvas node joins (or mints) its row. Rows minted this way are tagged
with a new `canvas` discovery source and get their own inventory filter.
- DetailPanel offers "Open in inventory"; standalone, which has no
inventory, is not offered it.
test_racks' "reports what the canvas node knows" seeded a second inventory
row for a host that already had one — a state a node create can no longer
produce. Its seeding order is swapped so the node links to the row; every
assertion is unchanged.
ha-relevant: yes
The Device Inventory was a read-only discovery log: rows arrived from a
scan or an import and went stale. Everything a user curates — properties,
services, notes, hardware, check method — could only be edited on a canvas
node, and never came back.
Give the inventory row the fields a node carries and a way to write them:
- device_inventory gains label, type, notes, cpu_count/cpu_model/ram_gb/
disk_gb, show_hardware, check_method/check_target, last_seen, last_scan,
response_time_ms and updated_at. Live reachability goes in a new
status_live: `status` already means the pending/approved/hidden lifecycle
and the two must not be conflated.
- PATCH /api/v1/scan/pending/{id} applies only what the client sends, so
editing one field never clears the rest. Lifecycle and discovery
bookkeeping stay owned by the approve/hide routes and the importers.
- InventoryDeviceCreate carries the same fields, so a hand-made entry needs
no create-then-PATCH round trip.
- InventoryDeviceModal becomes show *and* edit, reusing the canvas editors
(PropertyList, ServiceModal) rather than growing a second implementation.
- InventoryEntry moves to types/index.ts, its home; the modal re-exports it
so existing call sites are untouched. NODE_TYPE_GROUPS moves to
utils/nodeTypeGroups so both type pickers share one vocabulary.
ha-relevant: maybe
The frontend takes the first host of a comma-separated override
(`splitFirstHost`), the backend took the whole string — so a two-host
override checked a hostname the UI never links to. `_parse_override`
now splits the same way.
The WebSocket service-status entry also declares `host` (nullable, as
the backend sends it), so the overlay key stops relying on an
undeclared field.
ha-relevant: yes
Follow-up to the per-service host override: the status checker still
probed the node host, and the live overlay keyed services on
node/port/protocol only — so several vhosts sharing port 443 all read
one status. `check_service` now resolves the override (scheme, port,
IPv6 literal), echoes it back in the payload, and `serviceStatusKey`
takes the host as part of the key.
Also guard `new URL()` in `getServiceUrl`: a hand-typed "https://"
threw mid-render instead of resolving to no URL.
ha-relevant: yes
Adds `kvm` to NodeType, with a label, a MonitorCog icon, a per-theme
accent and a registered node component, plus the Hardware group of the
node and custom-style pickers and the MCP NODE_TYPES list.
Scanner autodetect: port-agnostic signatures for PiKVM, TinyPilot,
JetKVM and NanoKVM, matched on the HTTP probe's title/headers, and `kvm`
placed ahead of `server` in the suggest_node_type priority list.
suggest_node_type called match_port, which drops the HTTP-probe signals,
so no `port: null` signature could influence a suggested type — Unraid,
OpenMediaVault and Cockpit included. It now calls match_service with the
probe signals, which is what makes the KVM signatures reachable.
Closes#314
ha-relevant: yes
POST /api/v1/settings persisted the new status check interval and echoed it
back, but never rescheduled the running job: reschedule_status_checks() is
defined in app/core/scheduler.py and has no caller anywhere. The UI and
scan_config.json show the new value while the scheduler keeps firing at the
old one until the backend restarts.
On our install this went unnoticed for three weeks: 3600s was configured but
30s stayed in effect, so ~120 nodes were pinged 120x more often than intended
with no visible symptom.
Also add ge=10 to interval_seconds, mirroring the floor already enforced by
reschedule_status_checks(). Without it a value below 10 is written to
scan_config.json first and only then raises, which surfaces as a 500 and
leaves the invalid value to be reloaded on the next boot.
Both new tests fail without the fix: the first with AttributeError (the route
does not import the function), the second with 200 instead of 422.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The rack's link picker read `/nodes`, so it only ever offered devices
someone had already approved onto a logical canvas — two rows on a homelab
holding 74 inventory entries. A device on no canvas is still the record of
a real box, and is exactly what a rack is built out of.
`DevicePickerModal` replaces `NodePickerModal` and lists the Device
Inventory itself. Picking an entry calls the new `relinkDevice`, which
repoints the mount's `deviceId`, adopts that entry's node, status and —
unless the user renamed the plate — its label. One entry, one mount: a row
another plate stands for is not offered, and the store refuses it anyway.
The placeholder a rack-created plate left behind is dropped through the new
`DELETE /api/v1/scan/pending/{id}`, which refuses a device a rack still
mounts (409): foreign keys are off at runtime, so the mount would be left
naming a row that no longer exists.
`LinkedDevicePanel` becomes "Linked device" and now prints what discovery
found even when nothing on a canvas answers for the device; only the
canvas-side rows go missing, under a "Not on a logical canvas." note.
Also renames `pending_devices` to `device_inventory` (and
`pending_device_links` to `device_inventory_links`), with the Python and
TypeScript names that followed it. "Pending devices" was the scanner's word
for a queue of finds awaiting approval; the rows outlive approval, are
edited by hand and are what a rack mounts. Routes, payload keys and MCP
tool names are a published contract and are unchanged — `/scan/pending/*`
and the `pending_devices` key in `/stats` stay as they are.
The rename migration runs before `create_all`, or an empty new table would
be created beside the populated old one and every scanned device would read
as gone; it repairs that state too, for anyone whose app already started
mid-upgrade. Foreign keys are switched on for the rename so SQLite rewrites
the `REFERENCES` clause in `rack_devices`.
ha-relevant: maybe
SQLite has no timezone type: an aware `last_seen` reads back naive, and
Pydantic then put it on the wire as `2026-08-08T10:00:00`. To the browser
that is *local* time, so the Logical view panel printed a timestamp
shifted by the viewer's UTC offset.
ha-relevant: no
The Edit Device modal left the column under the port list empty, while
the logical canvas already held every technical fact about the same box.
A mount that stands for a Device Inventory entry now prints them there:
canvas name, type, hostname, IP, MAC, OS, the status check the node runs,
the canvas it is drawn on, when it was last seen, and the services
discovery fingerprinted on it. Read-only — the logical view owns them.
`/racks/inventory` ships the node half as `node_*` alongside the
inventory row's own mac/hostname/os/services, and the panel prefers the
node value: the node is what the user curates, the inventory row is what
discovery last saw and goes stale after a rename or a DHCP move. Rows
neither side can fill are dropped, so a device on no canvas still shows
what was reported and an accessory shows no panel at all.
The new API fields are optional client-side, so an older backend reads
back as "not on a canvas" rather than an empty node.
ha-relevant: maybe
The number input's min/max are hints the browser does not enforce on a
typed value, and updateRack wrote the patch straight through. Two ways
out of a usable canvas:
- Shrinking below a mounted device left the plate drawn above the
chassis, outside the React Flow node, with nothing in the UI able to
drag it back. freeUnits only counts 1..uHeight, so the modal's "used
of" label under-counted it in silence.
- A height over 100 made RackSave reject every save with a 422, so both
the explicit Save and each autosave tick reported "Save failed" with
no cause.
updateRack now clamps to [MIN_RACK_U, MAX_RACK_U] and relocates the
mounts a shrink pushes past the top rail, one at a time so two never
land on the same slot. It returns false — changing nothing — when one
has nowhere to go, and the modal says so.
Two server-side guards that would have contained it: RackSaveRequest
cross-checks every mount against the rack it names, and RackDeviceSave
checks col_start + col_span against the grid, which each field passing
its own bounds never caught.
Also normalizes the MAC on POST /scan/pending. Every other write path
canonicalizes it and dedup compares by equality, so a hand-typed
AA-BB-CC-11-22-33 never matched the scanned aa:bb:cc:11:22:33 and
approve built a duplicate node.
ha-relevant: maybe
Clicking a cable — in patch mode or out of it — selects it and opens a
right rail. It carries the physical facts (type, colour, label) and
`properties`: the same NodeProperty records the logical canvas already
uses for nodes, each with its own "show on canvas" eye. What is ticked
visible is drawn on a small plate at the midpoint of the run, so an
export carries its lengths and VLANs.
The property editor moves out of DetailPanel into
components/common/PropertyList so both canvases share one implementation
rather than growing a second.
Persistence: rack_cables gains label_visible and properties, added by an
idempotent migration in _try_migrate; the response schema coerces the
NULLs legacy rows read back as.
Also drops the 40 % plate fade that came with cables-on. It let the rail
strips and the U grid show through the mounted gear, which read as a
rendering bug — cables are drawn above the plates anyway.
ha-relevant: maybe
A mount's Status select gains "Check device": instead of a colour frozen at
mount time, it follows the status check already configured on the matching
logical-canvas node (ping, http, ssh…). The rack runs no checker of its own,
so `auto` resolves through the Device Inventory's `node_status`, falls back to
`unknown` when nothing is behind it, and is only offered when the mount — or
the entry being picked — resolves to a node. Losing that link drops the mount
back to `unknown` rather than leaving it on a status nothing can answer.
`useAutoStatusRefresh` polls the inventory every 60s while at least one mount
follows a node, and only then; it writes inventory only, so autosave stays
quiet.
A device created from the rack canvas also lands in the Device Inventory with
a type now. Nothing discovers those rows, so the plate they wear is the only
thing that says what they are — without it they had no icon, no role badge and
no place in the type filter. Passive gear uses the rack-only kinds
`patch_panel` and `pdu`; the latter is deliberately not `socket`, which is
excluded from the rack inventory and would make a PDU vanish on creation.
ha-relevant: maybe
The rack device modal listed inventory entries in a flat <select>: one line per
device, no search, no filters, nothing to tell two look-alike hosts apart. It
now opens the Device Inventory modal itself in a picker mode — a card click
returns the device instead of approving it, and the bulk/clear controls (plus
the s/a/Enter shortcuts) are hidden, since a picker has no business deleting
anything.
That list also carried VMs, containers and mesh devices, which no rack can
hold. A new "Rackable" filter drops them, armed by default when the rack opens
the modal. It is an exclusion list, not an allow list, so new hardware types
show up by default and an unclassified device (a bare ARP hit) stays visible.
`utils/rackable.ts` mirrors `_UNRACKABLE_TYPES` in racks.py — the list the
inventory endpoint already filtered on — and test_rackable_sync.py fails the
backend suite on drift.
The rack side keeps the last word: a device missing from its own inventory, or
already mounted in this design, is refused by name rather than mounted as a
ghost. Nothing is preselected anymore either, so submitting without picking
says so instead of quietly mounting whichever entry happened to be first.
Standalone keeps the <select>: the inventory modal reads pending_devices over
REST and there is no backend there.
The modal itself was max-w-md and one column, which pushed the port list below
the fold. It now matches NodeModal's width with the fields on the left and the
ports on the right.
ha-relevant: maybe
Four more defects from the branch review.
`POST /racks/save` fetched every row by primary key alone and then wrote
`design_id` from the payload, so a save on design B carrying an id that belongs
to design A moved A's rack — devices and cables with it — into B, with no error
and no way to notice. Ids come from the client, so a copied design or a stale
tab is enough. `_owned` now refuses a row owned by another design with a 409.
`commitEdit` dropped `applyFaceplate`'s return. Picking a plate the rack has no
room for, then shrinking the height by hand, let `updateDevice` succeed on its
own and `setPorts` write the new plate's ports onto the old faceplate — the
device ended up wearing a plate it did not have, silently. The failure is now
reported and nothing is committed.
Two smaller ones. The inventory select held a dead id after the list was
refetched, so submitting reported "No free slot in this rack" when the real
cause was an entry that had been racked elsewhere or deleted; it now falls back
to its placeholder and says so. And both create paths patched the form's own
geometry over the slot `findSlot` had just chosen, undoing a relocation the user
was never shown — the mount is handed the whole geometry up front, and only the
overrides are patched afterwards.
ha-relevant: maybe
Four defects found reviewing the last two commits.
Reverting a faceplate choice took every cable on the device down with it.
Browsing to another plate and back left `plateChanged` set with freshly seeded
port ids, while `commitEdit` skipped `applyFaceplate` because the id matched —
so `setPorts` received ids the device never had and dropped every cable
attached to the old ones, with no warning, since from the user's side the plate
had not changed. Coming back to the device's own plate now restores its ports,
ids included.
The inventory select preselects the first unracked entry, but only
`pickInventory` seeded the status from it, so submitting without touching the
select wrote `unknown` over the status the mount had just copied. Status now
seeds from that same entry.
Creating a device POSTs to the Device Inventory before the mount is attempted,
so a rack with no free slot stranded the row — and stranded another on every
retry. The rack is checked for a slot before anything is created.
And `_service_name` accepted a service with no port, where the inventory
requires one (`None not in _COMMON_PORTS` is True): a portless service named a
device in the rack picker but not in the Device Inventory, the exact mismatch
the naming fix set out to remove.
ha-relevant: maybe
The rack picker listed "192.168.1.63 · 192.168.1.63" where the inventory
showed "jellyfin", so a device the user picked out by one name turned up here
under another.
Two causes. The inventory route labelled an entry
`friendly_name or hostname or ip or id`, skipping the app-name step: `_device_label`
now mirrors `deviceLabel` in PendingDevicesModal — friendly name, host, the app
its ports say it runs (ssh/http/https ignored, generic web deprioritized), IP,
IEEE, id. And the option line appended the IP unconditionally, doubling it
whenever the label already was the IP; it now adds the IP only when it says
something new, plus the type as the tiebreaker between look-alikes.
Also fixes the port row: the name field and the type select both carried
`w-full` from the shared input class, so the select took the row and the name
field collapsed into an unlabelled box.
ha-relevant: maybe
The rack canvas loses its right rail: a mount is now edited in its own dialog,
like a network node, and the canvas keeps the full width.
Devices:
- `RackDeviceModal` adds and edits a mount — label, faceplate, U position,
height, column, width, status, colour, ports, unmount. Adding offers three
sources: an existing Device Inventory entry, a new device created here, or a
rack-only accessory. Opened by `+ Device` in the left rail or a double-click
on a plate.
- `RackSettingsModal` takes over rack chrome (name, location, capacity, 19"/10",
numbering, colours, delete), on a double-click of the chassis.
- The left rail's `+ Rack` becomes `+ Device`; `+ Rack` stays in the header.
Inventory:
- Gear created from a rack canvas lands in the Device Inventory with
`discovery_source: "rack"`, so it inherits the existing search, filters, hide
and delete instead of a parallel list in the sidebar. New "Rack devices"
source filter and badge.
- It is never placed on a logical canvas: approve returns 409, bulk-approve
skips it with `match: "rack"`, and the UI drops the Approve button and reports
the skip.
- `InventoryTray` becomes `AccessoryTray` — blanks, shelves and cable managers
only, still a drag source.
Fixes:
- A taller faceplate (or a hand-typed height) no longer silently keeps the old
geometry when it collides: the device relocates to the nearest slot that takes
the new size, and only a rack with no such slot refuses the edit. This is why
a 1U and a 2U plate rendered at the same height and the field looked locked.
- The empty-state buttons sat under `.react-flow__renderer` (z-index 4), so the
pane swallowed their clicks as a canvas drag.
Tests: rack device/settings modals, empty-state stacking, relocate-vs-refuse in
the store, rack source bucketing, inventory approve gating (frontend + backend).
ha-relevant: maybe
The modal-based editors and the rack store are portable; the inventory tagging
rides on the standalone FastAPI scan routes. Discuss before porting.
The rack prototype lived behind /racklab with no backend. It now becomes a
third design_type alongside network and electrical, sharing the app shell:
same header, same left rail, same right panel, same design switcher. App
branches the renderer on design_type.
Backend
- racks / rack_devices / rack_cables, all design_id-scoped.
- rack_devices.device_id -> pending_devices is the primary link. Those rows
survive approval and node deletion, so unracking never removes the device
from the Device Inventory. node_id -> nodes is a second, optional link used
only for live status and for matching network-link imports. Both are
ON DELETE SET NULL and label is denormalized, so a rack keeps rendering
after an inventory purge.
- GET /racks, POST /racks/save (full-state upsert + prune, same contract as
canvas/save), GET /racks/inventory.
- POST /scan/pending adds an inventory entry by hand, for hardware no scan
can discover.
- design_type is now validated and actually branched on; the schema comment
calling it vestigial is gone.
- Design delete unwinds rack rows explicitly (SQLite does not always honour
the cascade); design copy duplicates racks, mounts and patches with fresh
ids and re-points the cables at the copies.
Frontend
- /racklab and RackLab are removed; no dual maintenance.
- Rack types lifted into @/types; rackSerializer maps the snake_case wire
shapes to them, narrowing every enum on the way in.
- The store gains load/save, dirty tracking and an editSeq, so racks get the
same explicit Save and the same opt-in autosave as the logical canvas.
- Header, sidebar and inspector adapt in rack mode: canvas-only actions are
hidden, Add Rack / Patch / cable visibility / type filter / Import links
take their place, the sidebar body becomes the inventory tray, and the
footer counts racks, mounts, cables and free U.
- rackTheme derives the rack palette from the active app theme rather than
declaring one per theme, so a new theme works here for free. Per-rack
chrome stays user-editable; only its default comes from the theme.
- New Canvas gains a Kind picker (hidden when copying, since a copy inherits
the source kind).
Tests: 24 backend, 5 frontend files (store, persistence, networkLinks,
rackTheme, rackSerializer) plus DesignModal coverage for the Kind picker.
ha-relevant: maybe
Frontend rack renderer is portable; the FastAPI routes and SQLAlchemy models
are standalone-only by the persistence/transport split. Discuss before porting.
Add a label substring filter to GET /api/v1/nodes, plus MCP tools
list_node_summaries (id/label/type/status only) and get_node
(by id -> single object, by label -> array) so an LLM client can look
up nodes without paying the token cost of the full node payload.
Cleared canvas re-showed the demo, and backend errors silently fell back to
demo — hiding real outages and forcing users to wipe demo nodes before bulk
edits. Now:
- backend load reports `initialized` (CanvasState row exists = ever saved)
- decideCanvasLoad() picks real | empty | demo; empty is kept empty
- backend down/error shows an error banner + toast, never the demo
- data-new-user flag reserved as the Getting Started walkthrough hook
ha-relevant: yes
nmap -sV sends plaintext probes to TLS ports (e.g. Proxmox 8006). They
stall until the host crosses --host-timeout, at which point nmap skips
the whole host and discards ports it already found — a reachable host is
returned with 0 open ports (issue #277).
Split Phase 2 into two passes: Pass A (-sS/-sT --open) discovers ports
fast with no version detection; its ports are authoritative. Pass B
(-sV --host-timeout, scoped to the found ports) enriches banners
best-effort — on timeout/failure the discovered ports are kept with
empty banners instead of losing the host.
Healthy hosts still get full-intensity -sV banners (no quality drop).
Version host-timeout is now configurable via SCANNER_VERSION_HOST_TIMEOUT
(default 60s).
ha-relevant: yes
CI mypy flagged the shared payload/background locals in _run_mesh_sync
as incompatibly typed across the zigbee/zwave branches. Annotate them
with the union / generic Callable and alias the per-branch
env_import_request imports so the two bindings don't clash.
Mirror the Proxmox auto-sync pattern for the Zigbee2MQTT and Z-Wave JS
UI mesh imports. Connection config + MQTT credentials live in .env only,
are never persisted to scan_config.json, and are never returned by any
API or shown in the UI (single source of truth).
- config: ZIGBEE_* / ZWAVE_* env settings; only sync_enabled+interval
are persisted, connection/credentials stay env-only
- routes: GET/POST /config, POST /sync-now; auto-sync reuses the exact
manual _background_*_import + _persist_pending_import path (fresh
import when empty, update-in-place when nodes exist, ScanRun trace)
- scheduler: zigbee_sync / zwave_sync jobs with live enable + reschedule
- frontend: reusable MeshAutoSync section in Settings (Zigbee, Z-Wave)
- .env.example: documented both blocks
- tests: scheduler jobs, router config/sync-now/auth, credential-never-
persisted, SettingsModal sections
Manual Zigbee/Z-Wave import behaviour is unchanged.
ha-relevant: no
The scheduled auto-sync job called _persist_pending_import directly and
never created a ScanRun, so auto-imports left no trace in Scan history
(unlike manual /sync-now and /import-pending, which both record a run).
Create a ScanRun(kind=proxmox) and delegate to the shared
_background_proxmox_import flow — fetch, persist, mark the run
done/error, and broadcast the inventory-reload signal.
ha-relevant: maybe
Add backend API tests for the two issue #265 features that shipped
untested: auto-positioning root nodes into a free grid slot (first at
origin, collision avoidance, child origin default, explicit coords and
explicit zero preserved) and auto-assigning edge handles from absolute
canvas Y (source above/below/equal target, partial handle fill, child
abs-Y resolved through parent).
Drop the max(0, round()) clamp in _find_free_position: it folded
negative-positioned nodes onto cell (0,0), falsely blocking or freeing
the origin slot. Negative nodes now keep their true cells and never
intersect the positive search space.
Audit of the heavy-mock suites (scanner/status_checker/scheduler)
confirmed the mocking is sound layered seam-mocking, not brittle
internal patching. It did hide real gaps where a primitive was always
mocked and never exercised:
- status_checker._http_get: add status-code interpretation tests
(2xx/4xx online, 5xx offline) + service-check exception -> offline.
Module coverage 95% -> 100%.
- scheduler.reschedule_status_checks / reschedule_service_checks were
wholly untested; add validation + guard tests. Add _run_proxmox_sync
happy-path and error-swallow tests. Module coverage 76% -> 91%.
639 passed.
ha-relevant: no
Group the three upgrade-path suites (legacy->designs, hardware->
properties, pre-migration DB backup) under tests/migrations/ with
clearer names. Pure relocation via git mv; each keeps its own
self-contained fixtures. No behavior change; 627 passed.
ha-relevant: no
Pouzor