merge_services did {**existing, **incoming}, so the fingerprint's guess at
an icon overwrote the one the user chose — and on a port no signature
covers it wrote None, clearing it outright. Since 3.3.0 the inventory row
is the only copy of a device's services, so every "Scan network" repainted
the service on every canvas drawing that device at once.
The scanner now merges with discovered=True: it still adds services and
refreshes what it knows, but leaves an established icon and category alone.
A user edit from the modal or a canvas changes them as before.
Blank incoming values no longer clear established ones either, on both
paths — an absent field is silence, not a reset. Same rule merge_properties
already follows.
ha-relevant: yes
Since 3.3.0 the inventory row is the only copy of a device's services and
every canvas drawing it reads that list, but the scanner still assigned
`keep.services = fingerprint_ports(...)` — the pre-split behaviour, when a
node held its own copy. One re-scan therefore deleted every service the user
added by hand, on every canvas at once, and brought back the ones they had
deleted. It unions now, like every other writer of that field.
Two more things #347 turned up:
* A node whose view matches nothing the row still holds drew nothing at all —
the row had been replaced under it, so every key was gone and every key was
new, and `apply_view` hid the lot. Such a view says nothing about the list
that replaced it, so it is treated as no view: the row is drawn. An empty
view is untouched, being a real answer ("this canvas draws none of them").
* The 3.3.3 view seed recovers a node's arrangement from the pre-3.3.0 backup,
which is 3.2.0-era and cannot know about a property added afterwards. On
3.3.0-3.3.2 the row was the only place to add one and every canvas drew it,
so seeding strictly from the backup took it off all of them at once. Those
are appended visible, keeping the recovered order and hidden flags for
everything the backup does know. Services keep the strict recovery: holding
back what a scan fingerprinted is the whole point of the view.
ha-relevant: maybe
The seed selected any node with a `device_id` and no view. Deleting a device
leaves that column dangling — SQLite runs with foreign keys off, so the
`ON DELETE SET NULL` never fires on an existing table — and such a node can be
seeded from nothing, so it stayed selected. The query matched it again on every
later start, re-opening the pre-upgrade backup and logging a recovery that
recovered nothing.
Narrowing the query to nodes whose row actually exists restores the invariant
the seed is documented to have: it runs on one boot, and never opens a file
again.
ha-relevant: no
Since 3.3.0 the inventory row owns a device's services and properties, and
every node drawing that device rendered the row wholesale. One row shared by
several canvases meant one rendering: a service the scanner fingerprinted
appeared on every canvas at once (users reported Uptime Kuma and Synology DSM
on hosts running neither — both are port-only signatures), and a property added
on one schematic showed up on all the others.
Order and visibility are presentation, so they move to the node. `display_view`
records, per node, which of the row's services and properties it draws and in
what order, keyed by `port|protocol|name` and by lowercased property key so the
view survives an edit to the fact itself. The facts stay on the row: hiding is
per node, deleting is still device-wide.
An item the view does not list is reported hidden rather than dropped, so what
a scan finds next is one toggle away instead of pushed onto every canvas.
The wire shape is unchanged. A client already sends its lists in display order
with their `visible` flags, so the view is read back out of them rather than
asking for a second field, and `visible` is only stamped when something is
hidden.
Upgrades keep what each canvas showed:
* from 3.2.0, the backfill seeds each node's view from its own legacy columns
before they are dropped;
* from 3.3.0-3.3.2 those columns are gone and the row holds the union of every
canvas, so the layout is recovered from the backup `_backup_db` took before
the 3.3.0 migration — the newest one whose `nodes` table still has the
columns, read read-only and matched by node id;
* with no usable backup the row is the seed, so every canvas keeps showing
exactly what it shows today and only later additions are held back.
Refs #347
ha-relevant: maybe
3.3.1 kept `nodes` writable when the backfill failed, but the backfill still
failed — so a canvas upgraded from 3.2.0 shows every device stripped of its ip,
services, hostname, notes and hardware. The facts are intact in the legacy
columns; nothing was ever reading them across.
The backfill reads those columns with raw SQL, so SQLite hands `last_seen` and
`last_scan` back as text. Writing text into a DateTime column raises
`TypeError: SQLite DateTime type only accepts Python datetime and date objects`,
which is neither IntegrityError nor OperationalError — so it went straight past
the per-node savepoint and aborted the whole run, exactly as before.
- Parse the legacy timestamps back into datetimes, the way `_decode_json`
already handles the JSON columns. An unparseable stamp becomes None rather
than an error: the status checker refreshes both within a minute.
- Catch every exception per node, not two chosen classes. Picking the exception
types was the defect; a node that fails for any reason must cost only itself.
The same run would also have destroyed data. A canvas saved while an earlier
migration was stuck minted a blank inventory row from a UI that had no facts to
show and linked the node to it. The backfill only looked at unlinked nodes, so
it skipped that one, counted the canvas migrated, and dropped the columns
holding its only copy of the ip and services.
- Read linked nodes too, and fill only what their row is missing. A row that
already holds a fact is never overwritten, so an edit made after the migration
survives, and the pass stays a no-op once everything has moved across.
Verified end to end against a rebuilt 3.2.0 database stuck the way the reports
describe: one boot restores ip, services, hostname, mac, notes, check method,
hardware, properties, status and last_seen, then drops the legacy columns.
Refs #347, #348, #351
ha-relevant: no
Upgrading 3.2.0 -> 3.3.0 could leave a database where approving a device —
or creating any node — failed with `NOT NULL constraint failed: nodes.status`.
The 3.3.0 migration moves the device facts off `nodes` onto `device_inventory`
and then drops the columns. The drop is skipped when a node is still unlinked,
because those columns are the only remaining copy of its facts. But a 3.2.0
`nodes` declares `status`, `services`, `properties` and `show_hardware` NOT NULL
with no server-side default, and the 3.3.0 model no longer writes them — so a
skipped drop bricks every later INSERT.
It skipped because one node killed the whole backfill: the loop shared a single
session, so an IntegrityError raised by autoflush lost every link made before it.
- Give each node its own savepoint. A node whose merge violates a constraint is
skipped and logged on its own; the rest still link. `skipped` joins the stats
reported at boot.
- Relax NOT NULL on the retained legacy columns when the drop is skipped, so the
database stays writable while later boots retry the backfill. Values are kept;
only the constraint goes. Guarded on the PRAGMA notnull flag, so it runs once.
- Share one `_rebuild_nodes` between the drop and the relax: it restores
`PRAGMA foreign_keys = ON` in a finally (the connection returns to the pool)
and clears a `nodes_new` left by a failed attempt.
- Match IEEE addresses case-insensitively, and never write one another inventory
row already owns — `device_inventory.ieee_address` is UNIQUE, and a duplicate
is one way the backfill was raising in the first place.
Tests build the real v3.2.0 schema from the tag: the existing legacy-migration
fixture declares `status` nullable, which is why this reached a release.
Fixes#351
ha-relevant: no
A canvas node carries a full copy of its Device Inventory row, hydrated when
the canvas loads. The save routed all of it back, so a save made for nothing
but a moved node rewrote the row from a snapshot that could be hours old —
silently reverting an edit made meanwhile in the inventory modal, on another
canvas, or by the scanner.
Diffing the payload against the row server-side cannot fix this: it can't tell
"I edited this" from "the row moved on since I loaded it". Only the client
holds the baseline.
- canvasStore keeps `factsBaseline` — the device facts as received — set on
load, refreshed on save, rebased per field by `applyDeviceFacts`.
- `serializeNode` sends `changed_facts`: what this canvas actually edited.
`link_facts(changed_fields=)` writes nothing outside that list, even where
the values differ. Absent (older client, YAML import, MCP) keeps the previous
full-write behaviour.
- `changed_facts()` additionally drops facts already equal to the row, so a
no-op save writes nothing. Identity matching still uses the full payload.
- Live fields (status, last_seen…) bypass the filter — the status checker owns
reachability, and a save only ever fills a row never checked.
An inventory edit also lands on the canvases already on screen:
`applyDeviceFacts` pushes the saved row onto every node drawing it, without
marking the canvas unsaved. A fact the canvas has edited but not saved is left
alone — work in progress wins locally and still saves.
ha-relevant: maybe
`_slim_canvas` filtered `n["data"]`, the React Flow shape the frontend store
holds. `GET /api/v1/canvas` has never sent that: a node is reported flat, so
the filter matched nothing and `get_canvas` returned an id and a node type per
node — no label, no address, no services. The three tests covering it fed a
hand-built nested payload, so they passed against a shape the backend does not
produce.
Fold `data` over the node and read both, so either form slims the same way.
`type` leaves `NODE_KEEP` — flat it is the node type, not a device fact, and it
is already reported as `node_type`; `parentId` becomes `parent_id`, the key the
API uses, normalized from either spelling.
The new tests are built on a full `NodeResponse` copied from a real response,
and the backend now pins that wire shape from its side so the two move together.
ha-relevant: no
`merge_facts_into_device` applied one `replace_lists` flag to properties
and services alike, taking each wholesale from `facts`. But a PATCH only
carries the keys the client sent, so `{"properties": [...]}` replaced
services with `[]` — losing them for every canvas drawing the device and
stopping the scheduler from service-checking it. Symmetric the other way.
Gate the replace per list: a list absent from `facts` falls through to the
merge, which is a no-op against `None`. An explicit `[]` still clears, and
the canvas save path is unchanged since it always sends both keys.
ha-relevant: no
Completes the split: `nodes` now holds only how a device is drawn on one
canvas, and every device fact reaches the API from the inventory row.
- The device columns are removed from `nodes` (SQLite table rebuild, the
same shape as the existing device_inventory and canvas_state rebuilds).
The drop is skipped, and logged, while any non-furniture node is still
unlinked: those columns are the last copy of that node's facts. The
backfill therefore reads them with raw SQL — by the time it runs, the
model no longer declares them.
- The status checker iterates devices, not nodes: one check per device
however many canvases draw it, writing `status_live` / `last_seen` /
`response_time_ms` on the row. `/ws/status` messages carry `device_id`
and the node ids they light up. Hidden devices are not probed.
- Readers repointed: scanner (last_scan lands on the row), proxmox (its
node tier collapses into the inventory tier, keeping only the cluster
handles), zigbee/zwave (one property refresh serves every canvas), rack
inventory, liveview, stats, node dedupe.
- `POST /scan/pending` merges into the row that already describes the host
instead of minting a second one — one device is one row, whichever way it
was documented.
- Standalone keeps parity: the canvas blob gains `devices`, split on save
and hydrated on load. A blob written before the split still reads.
The rack inventory had a related bug: a mount that names a node explicitly
printed the mount's device rather than the pinned node's. It now reads the
node's own row.
Tests that built a node with device columns are ported to the link; where a
behaviour genuinely moved (properties refresh once on the row, last_scan is
the device's) the assertion moved with it rather than being dropped.
ha-relevant: yes
A device drawn on three canvases was three independent copies of the same
facts. Point every node at the Device Inventory row it draws, and let that
row own what the device *is* — the node keeps only how it is drawn.
- nodes.device_id -> device_inventory.id, ON DELETE SET NULL. NULL for
canvas furniture (group / groupRect / text), which describes nothing
physical. Deleting a node never deletes the row.
- services/inventory_sync holds the shared rules: matching by ieee > ip >
mac (per token, so 10.0.0.4 never matches 10.0.0.40), a property union on
key, a service union on (port, protocol, name), and the backfill that
links every pre-existing node.
- The backfill is non-destructive by construction: it writes device_id and
fills the row, and deletes nothing. Nodes are visited oldest-edit-first,
so where two canvases disagree on a scalar the most recently edited wins,
while properties and services stay unioned — nothing any canvas recorded
is lost. A second boot finds nothing to do.
- The wire shape does not change: GET /canvas hydrates the device fields
from the row, and a save routes them back to it. Editing a node's IP on
one canvas now shows on every other canvas holding that device.
- approve / bulk-approve set device_id instead of owning a copy, and a new
canvas node joins (or mints) its row. Rows minted this way are tagged
with a new `canvas` discovery source and get their own inventory filter.
- DetailPanel offers "Open in inventory"; standalone, which has no
inventory, is not offered it.
test_racks' "reports what the canvas node knows" seeded a second inventory
row for a host that already had one — a state a node create can no longer
produce. Its seeding order is swapped so the node links to the row; every
assertion is unchanged.
ha-relevant: yes
Pouzor