Compare commits

..
10 Commits
Author SHA1 Message Date
Pouzor ad0b1b427d chore: bump version to 3.3.4
ha-relevant: no
2026-08-17 19:19:25 +02:00
Pouzor 4c460cec1c fix(scan): stop a re-scan from deleting a device's hand-added services
Since 3.3.0 the inventory row is the only copy of a device's services and
every canvas drawing it reads that list, but the scanner still assigned
`keep.services = fingerprint_ports(...)` — the pre-split behaviour, when a
node held its own copy. One re-scan therefore deleted every service the user
added by hand, on every canvas at once, and brought back the ones they had
deleted. It unions now, like every other writer of that field.

Two more things #347 turned up:

* A node whose view matches nothing the row still holds drew nothing at all —
  the row had been replaced under it, so every key was gone and every key was
  new, and `apply_view` hid the lot. Such a view says nothing about the list
  that replaced it, so it is treated as no view: the row is drawn. An empty
  view is untouched, being a real answer ("this canvas draws none of them").

* The 3.3.3 view seed recovers a node's arrangement from the pre-3.3.0 backup,
  which is 3.2.0-era and cannot know about a property added afterwards. On
  3.3.0-3.3.2 the row was the only place to add one and every canvas drew it,
  so seeding strictly from the backup took it off all of them at once. Those
  are appended visible, keeping the recovered order and hidden flags for
  everything the backup does know. Services keep the strict recovery: holding
  back what a scan fingerprinted is the whole point of the view.

ha-relevant: maybe
2026-08-17 19:05:13 +02:00
Pouzor 1aa88de491 chore: bump version to 3.3.3
ha-relevant: no
2026-08-17 14:09:01 +02:00
Pouzor d0679ad72b fix(inventory): stop the view seed from re-reading the backup every boot
The seed selected any node with a `device_id` and no view. Deleting a device
leaves that column dangling — SQLite runs with foreign keys off, so the
`ON DELETE SET NULL` never fires on an existing table — and such a node can be
seeded from nothing, so it stayed selected. The query matched it again on every
later start, re-opening the pre-upgrade backup and logging a recovery that
recovered nothing.

Narrowing the query to nodes whose row actually exists restores the invariant
the seed is documented to have: it runs on one boot, and never opens a file
again.

ha-relevant: no
2026-08-17 14:06:09 +02:00
Pouzor 5df70ce128 feat(canvas): give each node its own view of a device's services and properties
Since 3.3.0 the inventory row owns a device's services and properties, and
every node drawing that device rendered the row wholesale. One row shared by
several canvases meant one rendering: a service the scanner fingerprinted
appeared on every canvas at once (users reported Uptime Kuma and Synology DSM
on hosts running neither — both are port-only signatures), and a property added
on one schematic showed up on all the others.

Order and visibility are presentation, so they move to the node. `display_view`
records, per node, which of the row's services and properties it draws and in
what order, keyed by `port|protocol|name` and by lowercased property key so the
view survives an edit to the fact itself. The facts stay on the row: hiding is
per node, deleting is still device-wide.

An item the view does not list is reported hidden rather than dropped, so what
a scan finds next is one toggle away instead of pushed onto every canvas.

The wire shape is unchanged. A client already sends its lists in display order
with their `visible` flags, so the view is read back out of them rather than
asking for a second field, and `visible` is only stamped when something is
hidden.

Upgrades keep what each canvas showed:

* from 3.2.0, the backfill seeds each node's view from its own legacy columns
  before they are dropped;
* from 3.3.0-3.3.2 those columns are gone and the row holds the union of every
  canvas, so the layout is recovered from the backup `_backup_db` took before
  the 3.3.0 migration — the newest one whose `nodes` table still has the
  columns, read read-only and matched by node id;
* with no usable backup the row is the seed, so every canvas keeps showing
  exactly what it shows today and only later additions are held back.

Refs #347

ha-relevant: maybe
2026-08-17 14:06:09 +02:00
Pouzor a58725c2ee chore: bump version to 3.3.2
ha-relevant: no
2026-08-17 11:37:54 +02:00
Pouzor 466df59b0c fix(inventory): finish the 3.3.0 backfill on databases carrying timestamps
3.3.1 kept `nodes` writable when the backfill failed, but the backfill still
failed — so a canvas upgraded from 3.2.0 shows every device stripped of its ip,
services, hostname, notes and hardware. The facts are intact in the legacy
columns; nothing was ever reading them across.

The backfill reads those columns with raw SQL, so SQLite hands `last_seen` and
`last_scan` back as text. Writing text into a DateTime column raises
`TypeError: SQLite DateTime type only accepts Python datetime and date objects`,
which is neither IntegrityError nor OperationalError — so it went straight past
the per-node savepoint and aborted the whole run, exactly as before.

- Parse the legacy timestamps back into datetimes, the way `_decode_json`
  already handles the JSON columns. An unparseable stamp becomes None rather
  than an error: the status checker refreshes both within a minute.
- Catch every exception per node, not two chosen classes. Picking the exception
  types was the defect; a node that fails for any reason must cost only itself.

The same run would also have destroyed data. A canvas saved while an earlier
migration was stuck minted a blank inventory row from a UI that had no facts to
show and linked the node to it. The backfill only looked at unlinked nodes, so
it skipped that one, counted the canvas migrated, and dropped the columns
holding its only copy of the ip and services.

- Read linked nodes too, and fill only what their row is missing. A row that
  already holds a fact is never overwritten, so an edit made after the migration
  survives, and the pass stays a no-op once everything has moved across.

Verified end to end against a rebuilt 3.2.0 database stuck the way the reports
describe: one boot restores ip, services, hostname, mac, notes, check method,
hardware, properties, status and last_seen, then drops the legacy columns.

Refs #347, #348, #351

ha-relevant: no
2026-08-17 11:36:32 +02:00
Pouzor 7eec0b3f9a chore: bump version to 3.3.1
ha-relevant: no
2026-08-17 01:58:26 +02:00
Pouzor b76519b0dd fix(db): keep nodes insertable when the 3.3.0 backfill cannot finish
Upgrading 3.2.0 -> 3.3.0 could leave a database where approving a device —
or creating any node — failed with `NOT NULL constraint failed: nodes.status`.

The 3.3.0 migration moves the device facts off `nodes` onto `device_inventory`
and then drops the columns. The drop is skipped when a node is still unlinked,
because those columns are the only remaining copy of its facts. But a 3.2.0
`nodes` declares `status`, `services`, `properties` and `show_hardware` NOT NULL
with no server-side default, and the 3.3.0 model no longer writes them — so a
skipped drop bricks every later INSERT.

It skipped because one node killed the whole backfill: the loop shared a single
session, so an IntegrityError raised by autoflush lost every link made before it.

- Give each node its own savepoint. A node whose merge violates a constraint is
  skipped and logged on its own; the rest still link. `skipped` joins the stats
  reported at boot.
- Relax NOT NULL on the retained legacy columns when the drop is skipped, so the
  database stays writable while later boots retry the backfill. Values are kept;
  only the constraint goes. Guarded on the PRAGMA notnull flag, so it runs once.
- Share one `_rebuild_nodes` between the drop and the relax: it restores
  `PRAGMA foreign_keys = ON` in a finally (the connection returns to the pool)
  and clears a `nodes_new` left by a failed attempt.
- Match IEEE addresses case-insensitively, and never write one another inventory
  row already owns — `device_inventory.ieee_address` is UNIQUE, and a duplicate
  is one way the backfill was raising in the first place.

Tests build the real v3.2.0 schema from the tag: the existing legacy-migration
fixture declares `status` nullable, which is why this reached a release.

Fixes #351

ha-relevant: no
2026-08-17 01:55:51 +02:00
Pouzor a2870a6e29 docs(changelog): match the house style for contributor credit
ha-relevant: no
2026-08-15 02:48:06 +02:00
21 changed files with 2028 additions and 56 deletions
+32 -1
View File
@@ -5,6 +5,37 @@ All notable changes to **Homelable** are documented here.
The format is loosely based on [Keep a Changelog](https://keepachangelog.com/),
and this project adheres to [Semantic Versioning](https://semver.org/).
## [3.3.4] - 2026-08-17
### Fixes
- A network scan no longer deletes the services you added by hand. The scanner replaced a device's whole service list with what it fingerprinted, so one scan wiped manual entries on every canvas at once and brought back the ones you had deleted. It merges now. (#347)
- A node whose services were all replaced under it no longer renders empty: when nothing a node's view names is left on the device, the device's own list is drawn instead of hiding everything. (#347)
- Properties added while running 3.3.0–3.3.2 survive the 3.3.3 upgrade. The view recovered from the pre-3.3.0 backup could not know about them, so they were hidden on every canvas; they are kept visible, and the per-canvas arrangement the backup does know is still restored. (#347)
## [3.3.3] - 2026-08-17
### Features
- Each node now keeps its own view of a device's services and properties: order and visibility are per node, so a service the scanner fingerprinted no longer appears on every canvas drawing that device, and a property added on one schematic stays there. The facts still live on the inventory row — hiding is per node, deleting is device-wide. Upgrades keep what each canvas showed today. (#357)
### Fixes
- The one-off view seed no longer re-opens the pre-upgrade backup on every boot: a node pointing at a deleted device kept matching the seed query and logged a recovery that recovered nothing. (#357)
## [3.3.2] - 2026-08-17
### Fixes
- Devices upgraded from 3.2.0 keep their ip, services, hostname, notes and hardware. The 3.3.0 migration aborted on timestamps SQLite returns as text, so nothing was carried across to the Device Inventory and every device read back blank. (#347, #348)
- A canvas saved while an earlier migration was stuck no longer loses its device facts: the migration now fills the blank inventory row that save created, instead of skipping the node and dropping the columns holding its only copy. (#347)
## [3.3.1] - 2026-08-17
### Fixes
- Approving a device no longer fails after an upgrade from 3.2.0: a node the 3.3.0 inventory backfill could not link kept the old device columns on `nodes`, and their NOT NULL constraint rejected every new node. (#352)
## [3.3.0] - 2026-08-15
### Features
@@ -20,7 +51,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/).
### Fixes
- Alignment guides show for nodes inside groups and containers. (#337)
- The status check interval is applied to the running scheduler. (#319) — thanks @JaG-v2
- The status check interval is applied to the running scheduler. Thanks @JaG-v2. (#319)
- Bumped undici and ip-address to patch high severity advisories. (#342)
### Docs
+1 -1
View File
@@ -1 +1 @@
3.3.0
3.3.4
+170 -18
View File
@@ -1,12 +1,14 @@
import json as _json
import logging
import shutil
import sqlite3
import uuid as _uuid_mod
from collections.abc import AsyncGenerator
from contextlib import suppress
from contextlib import closing, suppress
from pathlib import Path
from typing import Any
from sqlalchemy.exc import OperationalError
from sqlalchemy.exc import IntegrityError, OperationalError
from sqlalchemy.ext.asyncio import AsyncConnection, AsyncSession, async_sessionmaker, create_async_engine
from sqlalchemy.orm import DeclarativeBase
@@ -467,6 +469,10 @@ async def init_db() -> None:
"nodes.device_id.index",
"CREATE INDEX IF NOT EXISTS ix_nodes_device_id ON nodes(device_id)",
),
# Which of the row's services and properties this canvas shows, and
# in what order. Seeded from the row further down, once the backfill
# has had its say — see `_seed_node_views`.
("nodes.display_view", "ALTER TABLE nodes ADD COLUMN display_view JSON"),
):
await _try_migrate(conn, sql, label=label)
for label, sql in (
@@ -487,6 +493,7 @@ async def init_db() -> None:
await _backfill_node_devices()
await _drop_legacy_node_columns()
await _seed_node_views()
@@ -507,6 +514,7 @@ _NODE_COLUMNS_SQL = (
"label VARCHAR NOT NULL,"
"design_id VARCHAR REFERENCES designs(id) ON DELETE SET NULL,"
"device_id VARCHAR REFERENCES device_inventory(id) ON DELETE SET NULL,"
"display_view JSON,"
"pos_x FLOAT,"
"pos_y FLOAT,"
"parent_id VARCHAR REFERENCES nodes(id) ON DELETE CASCADE,"
@@ -525,19 +533,70 @@ _NODE_COLUMNS_SQL = (
)
_NODE_KEPT = (
"id, type, label, design_id, device_id, pos_x, pos_y, parent_id, container_mode, "
"id, type, label, design_id, device_id, display_view, pos_x, pos_y, parent_id, container_mode, "
"custom_colors, custom_icon, show_port_numbers, width, height, bottom_handles, "
"top_handles, left_handles, right_handles, created_at, updated_at"
)
async def _relax_legacy_node_columns(conn: AsyncConnection, info: list[Any]) -> None:
"""Make the retained legacy `nodes` columns nullable — SQLite table rebuild.
The 3.2.0 schema declares `status`, `services`, `properties` and
`show_hardware` NOT NULL with no server-side default. The 3.3.0 model no
longer maps them, so every INSERT omits them and SQLite rejects the row —
approving a device, creating a node, importing a canvas all fail with
``NOT NULL constraint failed: nodes.status``. Dropping the columns is the
real fix, but it waits on a complete backfill; until then they have to stop
blocking writes. Values are preserved: only the constraint goes.
"""
kept = [row for row in info if row[1] in _LEGACY_NODE_COLUMNS]
if not any(row[3] for row in kept): # PRAGMA `notnull`
return # Already relaxed, or never constrained.
logger.info("Relaxing NOT NULL on the retained legacy node columns")
legacy_defs = ",".join(f"{row[1]} {row[2] or 'VARCHAR'}" for row in kept)
legacy_names = ", ".join(row[1] for row in kept)
await _rebuild_nodes(
conn,
columns_sql=f"{_NODE_COLUMNS_SQL},{legacy_defs}",
copied=f"{_NODE_KEPT}, {legacy_names}",
what="nodes legacy-column relax",
)
async def _rebuild_nodes(conn: AsyncConnection, *, columns_sql: str, copied: str, what: str) -> None:
"""Recreate `nodes` with a new column list — SQLite cannot alter constraints.
Never fatal: a rebuild that fails leaves the table it could not replace, and
the boot carries on. Foreign keys go off for the swap, because `edges` and
`rack_devices` point at `nodes`, and back on in every case — the pragma is
per connection and this one returns to the pool.
"""
try:
await conn.exec_driver_sql("PRAGMA foreign_keys = OFF")
# A previous attempt that failed after the create would block this one.
await conn.exec_driver_sql("DROP TABLE IF EXISTS nodes_new")
await conn.exec_driver_sql(f"CREATE TABLE nodes_new ({columns_sql})")
await conn.exec_driver_sql(f"INSERT INTO nodes_new ({copied}) SELECT {copied} FROM nodes")
await conn.exec_driver_sql("DROP TABLE nodes")
await conn.exec_driver_sql("ALTER TABLE nodes_new RENAME TO nodes")
await conn.exec_driver_sql("CREATE INDEX IF NOT EXISTS ix_nodes_device_id ON nodes(device_id)")
except (OperationalError, IntegrityError) as exc:
logger.warning("%s failed: %s", what, exc)
finally:
await conn.exec_driver_sql("PRAGMA foreign_keys = ON")
async def _drop_legacy_node_columns() -> None:
"""Remove the device columns from `nodes` (3.3.0) — SQLite table rebuild.
Runs only after the backfill has linked every device node to its inventory
row. If any non-furniture node is still unlinked the drop is skipped and
logged: the columns are the only remaining copy of that node's facts, and
losing them is not recoverable.
losing them is not recoverable. The skip then relaxes their NOT NULL instead,
because the 3.3.0 model no longer writes them and the database must stay
insertable while the backfill is retried on later boots.
"""
async with engine.begin() as conn:
info = (await conn.exec_driver_sql("PRAGMA table_info(nodes)")).fetchall()
@@ -557,23 +616,16 @@ async def _drop_legacy_node_columns() -> None:
"The backfill must link every device node before they can be dropped.",
unlinked,
)
await _relax_legacy_node_columns(conn, list(info))
return
logger.info("Migrating nodes: the device columns move to device_inventory")
try:
await conn.exec_driver_sql("PRAGMA foreign_keys = OFF")
await conn.exec_driver_sql(f"CREATE TABLE nodes_new ({_NODE_COLUMNS_SQL})")
await conn.exec_driver_sql(
f"INSERT INTO nodes_new ({_NODE_KEPT}) SELECT {_NODE_KEPT} FROM nodes"
)
await conn.exec_driver_sql("DROP TABLE nodes")
await conn.exec_driver_sql("ALTER TABLE nodes_new RENAME TO nodes")
await conn.exec_driver_sql(
"CREATE INDEX IF NOT EXISTS ix_nodes_device_id ON nodes(device_id)"
)
await conn.exec_driver_sql("PRAGMA foreign_keys = ON")
except OperationalError as exc:
logger.warning("nodes device-column drop failed: %s", exc)
await _rebuild_nodes(
conn,
columns_sql=_NODE_COLUMNS_SQL,
copied=_NODE_KEPT,
what="nodes device-column drop",
)
async def _backfill_node_devices() -> None:
@@ -596,10 +648,110 @@ async def _backfill_node_devices() -> None:
"Inventory backfill: linked %d node(s) — %d device(s) created, %d merged",
stats["linked"], stats["created"], stats["merged"],
)
if stats.get("skipped"):
logger.warning(
"Inventory backfill: %d node(s) could not be linked; they keep their "
"legacy columns and are retried on the next start.",
stats["skipped"],
)
except Exception as exc: # pragma: no cover - defensive, boot must not die
logger.warning("Inventory backfill failed: %s", exc)
def _pre_split_backup() -> Path | None:
"""The newest backup still holding the per-node device columns, if any.
`_backup_db` copies the database *before* the migrations of each new
version, so a user who upgraded to 3.3.0 has a `homelab.db.back-3.3.0`
carrying the last state in which `nodes` still owned its own services and
properties. That copy is the only record of which canvas showed what, since
3.3.0's backfill unioned them all onto one inventory row. Newest first: it is
the state closest to the upgrade, so it is what the user last saw.
"""
db_path = Path(settings.sqlite_path)
candidates = sorted(
db_path.parent.glob(f"{db_path.name}.back-*"),
key=lambda p: p.stat().st_mtime,
reverse=True,
)
for path in candidates:
try:
with closing(sqlite3.connect(f"file:{path}?mode=ro", uri=True)) as conn:
cols = {row[1] for row in conn.execute("PRAGMA table_info(nodes)")}
except sqlite3.Error:
continue
if {"services", "properties"} <= cols:
return path
return None
def _views_from_backup() -> dict[str, dict[str, Any]]:
"""What each node drew, read out of the pre-3.3.0 backup. Empty when there is none.
Keyed by node id, which is a uuid and stable across every version. A backup
that cannot be opened, or holds nodes this database no longer has, simply
contributes nothing — the caller falls back to the inventory row.
"""
path = _pre_split_backup()
if path is None:
return {}
try:
with closing(sqlite3.connect(f"file:{path}?mode=ro", uri=True)) as conn:
rows = conn.execute("SELECT id, services, properties FROM nodes").fetchall()
except sqlite3.Error as exc:
logger.warning("Could not read the pre-3.3.0 node lists from %s: %s", path.name, exc)
return {}
out: dict[str, dict[str, Any]] = {}
for node_id, services, properties in rows:
drawn: dict[str, Any] = {}
for kind, raw in (("services", services), ("properties", properties)):
if isinstance(raw, str | bytes):
with suppress(ValueError):
decoded = _json.loads(raw)
if isinstance(decoded, list):
drawn[kind] = decoded
if drawn:
out[node_id] = drawn
if out:
logger.info("Recovering the per-canvas service/property layout from %s", path.name)
return out
async def _seed_node_views() -> None:
"""Give pre-existing nodes an explicit view of their inventory row (3.3.3).
Order and visibility for services and properties moved to the node, so one
device drawn on two canvases can be rendered two ways. A node from before
that has no view, and where it comes from decides whether the user gets
their arrangement back:
* upgrading from 3.2.0, the backfill has already seeded it from the node's
own columns — nothing here to do;
* upgrading from 3.3.0-3.3.2, those columns are gone and the row holds the
union of every canvas, so the view is recovered from the backup taken
before the 3.3.0 migration. That is the difference between a canvas coming
back as the user left it and coming back showing every other canvas'
properties;
* with no usable backup, the row itself is the seed: what shows today keeps
showing, and only what the row gains *later* is held back.
Never fatal: without a view the node simply shows the whole row, which is
the behaviour it has now.
"""
# Imported here: app.services imports app.db.models, which imports this module.
from app.services.inventory_sync import seed_node_views
try:
async with AsyncSessionLocal() as session:
seeded = await seed_node_views(session, drawn=_views_from_backup)
if seeded:
await session.commit()
logger.info("Seeded the service/property view of %d node(s)", seeded)
except Exception as exc: # pragma: no cover - defensive, boot must not die
logger.warning("Seeding the node service/property views failed: %s", exc)
async def get_db() -> AsyncGenerator[AsyncSession, None]:
async with AsyncSessionLocal() as session:
yield session
+9
View File
@@ -49,6 +49,15 @@ class Node(Base):
device_id: Mapped[str | None] = mapped_column(
String, ForeignKey("device_inventory.id", ondelete="SET NULL"), index=True, nullable=True
)
# How this canvas renders the device's list facts: which services and which
# properties it shows, and in what order. The facts themselves stay on the
# inventory row, so the same device drawn on two canvases can show two
# different subsets — a scanner-guessed service on one, none on the other.
# {"services": [{"key": "443|tcp|https", "visible": true}, …],
# "properties": [{"key": "rack", "visible": false}, …]}
# NULL only for canvas furniture and for a node with no inventory row yet;
# `inventory_sync.link_facts` fills both lists as soon as there is one.
display_view: Mapped[dict[str, Any] | None] = mapped_column(JSON, nullable=True)
pos_x: Mapped[float] = mapped_column(Float, default=0)
pos_y: Mapped[float] = mapped_column(Float, default=0)
parent_id: Mapped[str | None] = mapped_column(String, ForeignKey("nodes.id", ondelete="CASCADE"))
+421 -22
View File
@@ -15,10 +15,11 @@ from __future__ import annotations
import json
import logging
from collections.abc import Mapping
from collections.abc import Callable, Mapping
from datetime import datetime
from typing import Any
from sqlalchemy import or_, select, text
from sqlalchemy import func, or_, select, text
from sqlalchemy.ext.asyncio import AsyncSession
from app.db.models import InventoryDevice, Node
@@ -66,6 +67,50 @@ def _blank(value: Any) -> bool:
return value is None or value == ""
def _same_ieee(left: str | None, right: str | None) -> bool:
"""IEEE addresses compare case-insensitively — the same radio either way."""
if _blank(left) or _blank(right):
return False
return str(left).lower() == str(right).lower()
async def _ieee_owner(db: AsyncSession, ieee: str, *, other_than: str | None) -> InventoryDevice | None:
"""The row already holding ``ieee``, if it is not ``other_than``.
``device_inventory.ieee_address`` is UNIQUE, so writing an address a second
row owns raises. Callers ask first and leave the address where it is.
"""
stmt = select(InventoryDevice).where(func.lower(InventoryDevice.ieee_address) == ieee.lower())
if other_than is not None:
# `id != NULL` is NULL in SQL and would match nothing — only narrow when
# there is a row to exclude.
stmt = stmt.where(InventoryDevice.id != other_than)
return (await db.execute(stmt)).scalars().first()
async def _drop_taken_ieee(
db: AsyncSession, facts: Mapping[str, Any], device: InventoryDevice | None
) -> Mapping[str, Any]:
"""``facts`` without an ``ieee_address`` another inventory row already owns.
Returned unchanged in the common case. The column is UNIQUE, so writing a
duplicate raises mid-merge — and during the backfill that would cost the node
its link. The address belongs to whichever row holds it; identity for *this*
device was established by ip or mac.
"""
ieee = facts.get("ieee_address")
if _blank(ieee) or (device is not None and _same_ieee(device.ieee_address, ieee)):
return facts
owner = await _ieee_owner(db, str(ieee), other_than=device.id if device else None)
if owner is None:
return facts
logger.warning(
"IEEE %s is already held by device %s — leaving it off %s",
ieee, owner.id, device.id if device else "the new row",
)
return {k: v for k, v in facts.items() if k != "ieee_address"}
async def find_device_for(
db: AsyncSession,
*,
@@ -83,7 +128,10 @@ async def find_device_for(
ip_toks = _ip_tokens(ip)
conds = []
if ieee:
conds.append(InventoryDevice.ieee_address == ieee)
# Case-insensitive: `0x00124B00…` and `0x00124b00…` are the same radio,
# and an exact comparison here would mint a second row for it — one the
# UNIQUE index then refuses.
conds.append(func.lower(InventoryDevice.ieee_address) == ieee.lower())
for tok in ip_toks:
# Narrow with a substring match, then confirm per token below — an exact
# comparison misses rows holding several addresses.
@@ -98,7 +146,7 @@ async def find_device_for(
).scalars().all()
for device in candidates:
if ieee and device.ieee_address == ieee:
if ieee and _same_ieee(device.ieee_address, ieee):
return device
for device in candidates:
# "1.2.3.4" must not match "1.2.3.40" — confirm the token, don't trust
@@ -155,6 +203,161 @@ def _service_key(svc: Any) -> Any:
return (svc.get("port"), svc.get("protocol"), (svc.get("service_name") or "").lower())
# --- Per-node view of the device's list facts -----------------------------
#
# The row owns the services and the properties; a node owns which of them it
# shows and in what order. Both are keyed by a stable string so the view
# survives an edit to a service's path or a property's value.
VIEW_LISTS = ("services", "properties")
def _service_view_key(svc: Any) -> str:
port, protocol, name = _service_key(svc) if isinstance(svc, dict) else (None, None, repr(svc))
return f"{port}|{protocol}|{name}"
def _property_view_key(prop: Any) -> str:
if not isinstance(prop, dict):
return repr(prop)
return str(prop.get("key") or "").lower()
_VIEW_KEY = {"services": _service_view_key, "properties": _property_view_key}
def view_entries(items: list[Any] | None, kind: str) -> list[dict[str, Any]]:
"""One list of device facts as a node's view of it: order plus visibility.
A service carries no ``visible`` of its own — one that reached a canvas was
always drawn — so it defaults to shown. A property carries an explicit flag
and keeps it. Duplicate keys collapse: the view addresses the row, and the
row holds one entry per key.
"""
key_of = _VIEW_KEY[kind]
out: list[dict[str, Any]] = []
seen: set[str] = set()
for item in items or []:
key = key_of(item)
if key in seen:
continue
seen.add(key)
visible = bool(item.get("visible", True)) if isinstance(item, dict) else True
out.append({"key": key, "visible": visible})
return out
def view_from_facts(facts: Mapping[str, Any]) -> dict[str, list[dict[str, Any]]]:
"""The view a node payload implies — only for the lists it actually sent.
The wire shape has no separate view: a client sends its services and its
properties in display order, each with its ``visible`` flag, exactly as it
draws them. That *is* the view, so it is read back out here rather than
asking clients for a second field.
"""
return {kind: view_entries(facts[kind], kind) for kind in VIEW_LISTS if kind in facts}
def view_of_device(device: InventoryDevice) -> dict[str, list[dict[str, Any]]]:
"""A view showing everything the row currently holds — the seed for a new node."""
return {
"services": view_entries(device.services, "services"),
"properties": view_entries(device.properties, "properties"),
}
def next_view(
current: Mapping[str, Any] | None,
incoming: Mapping[str, Any],
device: InventoryDevice | None,
*,
strict: bool = False,
) -> dict[str, Any] | None:
"""This node's view after a write: what it sent, then the row for the rest.
A list the write did not carry keeps the view it had. A node linked to a row
always ends up with both lists, so "not in the view" can mean one thing
only: this canvas does not show it. That is what keeps a service a later scan
discovers off every canvas until someone turns it on.
A node getting its *first* view is the exception: an empty list there means
the writer had nothing to say about it, not that the user hid everything —
creating a node for an already-scanned device sends no services and must
still draw the ones the row holds. ``strict`` turns that off for the one
caller whose empty list is a real answer: the legacy backfill, where the
node's own columns are the whole of what that canvas used to show.
"""
if device is None:
return dict(current) if current else None
out: dict[str, Any] = dict(current or {})
first_view = current is None
seed = view_of_device(device)
for kind in VIEW_LISTS:
entries = incoming.get(kind)
if entries or (entries is not None and (strict or not first_view)):
out[kind] = entries
elif kind not in out:
out[kind] = seed[kind]
return out
def apply_view(items: list[Any] | None, entries: Any, kind: str) -> list[Any]:
"""The row's facts as one node draws them: its order, its visibility.
Without a view — furniture, or a node whose row was linked by an older
version — everything shows, in the row's own order. With one, an item the
view does not list is appended hidden rather than dropped, so a service a
scan added is one toggle away instead of invisible.
A *non-empty* view that matches nothing the row still holds is treated as
having no view at all. It means the row was replaced wholesale under the
node — every key gone, every key new — and hiding the lot would leave a node
drawing nothing while its view claims otherwise. An empty view is different:
it is a real answer ("this canvas draws none of them") and keeps hiding
everything.
"""
key_of = _VIEW_KEY[kind]
facts = list(items or [])
if not isinstance(entries, list):
return facts
by_key: dict[str, Any] = {}
for item in facts:
by_key.setdefault(key_of(item), item)
out: list[Any] = []
taken: set[str] = set()
for entry in entries:
if not isinstance(entry, dict):
continue
key = str(entry.get("key"))
item = by_key.get(key)
if item is None or key in taken:
continue # Deleted from the row since — the view catches up on write.
taken.add(key)
out.append(_stamped(item, bool(entry.get("visible", True))))
if entries and not taken:
return facts
for item in facts:
if key_of(item) not in taken:
out.append(_stamped(item, False))
return out
def _stamped(item: Any, visible: bool) -> Any:
"""``item`` carrying this node's verdict on whether it is drawn.
A shown item that never had a ``visible`` key does not gain one: services
have always travelled without it, and readers treat its absence as shown.
Only hiding is news, and properties keep the explicit flag they arrived with.
"""
if not isinstance(item, dict):
return item
if visible and "visible" not in item:
return dict(item)
return {**item, "visible": visible}
def merge_services(base: list[Any] | None, incoming: list[Any] | None) -> list[Any]:
"""Union two service lists on (port, protocol, name); incoming wins."""
out: list[Any] = [dict(s) if isinstance(s, dict) else s for s in (base or [])]
@@ -330,6 +533,7 @@ async def link_facts(
replace_lists: bool = False,
only_changed: bool = False,
changed_fields: list[str] | None = None,
strict_view: bool = False,
) -> InventoryDevice | None:
"""Point one node at its inventory row, creating or merging as needed.
@@ -360,6 +564,12 @@ async def link_facts(
db, ip=facts.get("ip"), mac=facts.get("mac"), ieee=facts.get("ieee_address")
)
# The IEEE is UNIQUE across the inventory. Where another row already owns the
# one these facts carry, drop it rather than write a duplicate: identity has
# been resolved to *this* device by ip or mac, and the address stays with the
# row holding it.
facts = await _drop_taken_ieee(db, facts, device)
if device is None:
device = device_from_facts(facts)
db.add(device)
@@ -380,6 +590,13 @@ async def link_facts(
device.discovery_sources = add_source(device.discovery_sources, CANVAS_SOURCE)
node.device_id = device.id
# Order and visibility are this node's, not the device's, so they are taken
# from the full payload — never from the `changed_fields`/`only_changed`
# narrowing above, which exists to protect the *shared* row from a stale
# snapshot. A node's own view has no other writer to collide with.
node.display_view = next_view(
node.display_view, view_from_facts(facts), device, strict=strict_view
)
return device
@@ -390,10 +607,16 @@ def node_columns(payload: Mapping[str, Any]) -> dict[str, Any]:
inventory row now, so they are dropped here and applied through
:func:`link_facts` instead.
"""
allowed = {c.name for c in Node.__table__.columns}
allowed = {c.name for c in Node.__table__.columns} - _NODE_DERIVED_COLUMNS
return {k: v for k, v in payload.items() if k in allowed}
# Node columns the server derives rather than accepts: `display_view` is read
# back out of the services and properties a payload carries (see
# :func:`view_from_facts`), so a client sending one directly is ignored.
_NODE_DERIVED_COLUMNS = frozenset({"display_view"})
# Fields that are both a node column and a device fact: the node keeps a copy so
# a half-migrated database still renders, but the row is the truth.
_SHARED_FIELDS = ("label", "type")
@@ -448,6 +671,9 @@ def hydrated_node(node: Node, device: InventoryDevice | None) -> dict[str, Any]:
payload: dict[str, Any] = {
c.name: getattr(node, c.name) for c in node.__table__.columns
}
# An implementation detail of the split, not part of the wire shape: the
# view is reported *through* the services and properties it orders.
view = payload.pop("display_view", None) or {}
if device is None:
return payload
@@ -455,8 +681,8 @@ def hydrated_node(node: Node, device: InventoryDevice | None) -> dict[str, Any]:
payload[field] = getattr(device, field, None)
payload["label"] = device.label or node.label
payload["type"] = device.type or node.type
payload["services"] = device.services or []
payload["properties"] = device.properties or []
payload["services"] = apply_view(device.services, view.get("services"), "services")
payload["properties"] = apply_view(device.properties, view.get("properties"), "properties")
payload["show_hardware"] = bool(device.show_hardware)
payload["ieee_address"] = device.ieee_address
payload["status"] = device.status_live or "unknown"
@@ -494,6 +720,139 @@ def _decode_json(value: Any) -> Any:
return value
# Legacy `nodes` columns SQLite stores as DATETIME. Read through raw SQL they come
# back as text, and writing text into a DateTime column raises
# ``TypeError: SQLite DateTime type only accepts Python datetime and date objects``.
_LEGACY_DATETIME_COLUMNS = ("last_seen", "last_scan")
def _decode_dt(value: Any) -> datetime | None:
"""Raw SQL hands back DATETIME columns as text; the ORM would have parsed them.
An unparseable stamp becomes ``None`` rather than an error: `last_seen` and
`last_scan` are observations the status checker refreshes within a minute,
and losing one must not cost a node its inventory row.
"""
if value is None or isinstance(value, datetime):
return value
if not isinstance(value, str) or not value.strip():
return None
try:
return datetime.fromisoformat(value.strip())
except ValueError:
logger.debug("Ignoring an unparseable legacy timestamp: %r", value)
return None
async def _row_is_missing_facts(db: AsyncSession, device_id: str, facts: Mapping[str, Any]) -> bool:
"""Does the linked row lack a fact the node's legacy columns still hold?
True only where the node has something to give and the row has nothing in
that field, so a node whose facts already made it across is left alone and
the backfill stays a no-op on later boots.
"""
device = await db.get(InventoryDevice, device_id)
if device is None:
return True # The row is gone; the node needs a new one.
for field in (*DEVICE_SCALARS, "label", "type", "ieee_address"):
if not _blank(facts.get(field)) and _blank(getattr(device, field, None)):
return True
return any(
facts.get(field) and not getattr(device, field, None)
for field in ("services", "properties")
)
def _with_later_properties(
view: dict[str, Any] | None, device: InventoryDevice
) -> dict[str, Any] | None:
"""``view``, plus the row's properties the pre-3.3.0 backup never saw.
They are what the user added while running 3.3.0-3.3.2 — the row was the
only place to add them then, and every canvas drew them. Recovering the old
view alone would take them off every canvas at once, which reads as data
loss. Appended in the row's order, after everything the backup placed.
"""
if view is None:
return None
entries = view.get("properties")
if not isinstance(entries, list):
return view
listed = {str(e.get("key")) for e in entries if isinstance(e, dict)}
later = [e for e in view_entries(device.properties, "properties") if e["key"] not in listed]
if not later:
return view
return {**view, "properties": [*entries, *later]}
async def seed_node_views(
db: AsyncSession, *, drawn: Callable[[], Mapping[str, Mapping[str, Any]]] | None = None
) -> int:
"""Give every linked node with no view one it can be held to.
Runs once, on the boot that adds `nodes.display_view`. Without it every
pre-existing node would fall through to "no view, show everything" and the
next scan would push a newly fingerprinted service onto all of them at once —
the leak this column exists to stop.
``drawn`` returns a map of node id to the services and properties that node
itself carried before 3.3.0 unioned them onto the row — the caller reads
them out of the pre-upgrade backup, and is only asked to when there is
actually a node to seed. Where it has an answer that answer wins, restoring
the arrangement the user made; where it does not, the row is the seed and
the canvas keeps showing exactly what it shows today. Does not commit.
The backup is 3.2.0-era, so it can only speak for what existed then. A
*property* the row has gained since — one the user added by hand while
running 3.3.0-3.3.2, when the row was all a canvas had — appears in no
backup entry, and seeding strictly from the backup would hide work the user
has been looking at for days. Those are appended visible, keeping the
recovered order and hidden flags for everything the backup does know.
Services are not treated that way: what a row gained since 3.3.0 is mostly a
scan's fingerprint, and holding it back is the whole point of the view.
"""
nodes = list(
(
await db.execute(
select(Node).where(
# A `device_id` naming a row that no longer exists cannot be
# seeded from anything, and leaving it in would match this
# query on every later boot — re-reading the backup file and
# logging a recovery that seeds nothing. Deleting a device
# leaves exactly that: SQLite runs with foreign keys off, so
# the `ON DELETE SET NULL` never fires on an existing table.
Node.device_id.in_(select(InventoryDevice.id)),
Node.display_view.is_(None),
)
)
)
.scalars()
.all()
)
if not nodes:
return 0
devices = await load_devices_for(db, nodes)
# Read the backup only now: on every later boot this function returns above
# and no file is opened at all.
was_drawn = drawn() if drawn is not None else {}
seeded = 0
for node in nodes:
device = devices.get(node.device_id or "")
if device is None: # pragma: no cover - the query already excluded these
continue
was = was_drawn.get(node.id)
# `strict`: an empty list in the backup is this canvas' real answer —
# it drew no service — and must not be read as "say nothing, show all".
if was:
view = next_view(None, view_from_facts(was), device, strict=True)
node.display_view = _with_later_properties(view, device)
else:
node.display_view = view_of_device(device)
seeded += 1
await db.flush()
return seeded
async def backfill_node_devices(db: AsyncSession) -> dict[str, int]:
"""Link every pre-3.3.0 canvas node to a Device Inventory row.
@@ -507,44 +866,84 @@ async def backfill_node_devices(db: AsyncSession) -> dict[str, int]:
Reads the legacy columns with raw SQL because the model no longer declares
them — this runs on the boot that drops them, once. Returns counts for the
boot log. Does not commit.
boot log, ``skipped`` among them: a node whose merge violates a constraint is
left unlinked instead of aborting the run. Does not commit.
"""
legacy = await _legacy_columns_present(db)
if not legacy:
# Already migrated: the columns are gone, so nothing can be left to read.
return {"linked": 0, "created": 0, "merged": 0}
return {"linked": 0, "created": 0, "merged": 0, "skipped": 0}
placeholders = ", ".join(legacy)
furniture = ", ".join(f"'{t}'" for t in sorted(FURNITURE_TYPES))
# Linked nodes are read too, not just unlinked ones. A canvas saved while an
# earlier migration was stuck minted a *blank* inventory row from a UI that
# had no facts to show, and linked the node to it. Skipping those would count
# them as migrated and drop the columns that still hold their only copy of
# the ip, services and notes. A linked node is filled, never overwritten.
rows = (
await db.execute(
text(
f"SELECT id, label, type, design_id, {placeholders} FROM nodes "
f"WHERE device_id IS NULL AND type NOT IN ({furniture}) "
f"SELECT id, label, type, design_id, device_id, {placeholders} FROM nodes "
f"WHERE type NOT IN ({furniture}) "
"ORDER BY updated_at, created_at, id"
)
)
).mappings().all()
if not rows:
return {"linked": 0, "created": 0, "merged": 0}
return {"linked": 0, "created": 0, "merged": 0, "skipped": 0}
created = merged = 0
linked = created = merged = skipped = 0
for row in rows:
facts: dict[str, Any] = {c: row[c] for c in legacy}
facts["services"] = _decode_json(facts.get("services")) or []
facts["properties"] = _decode_json(facts.get("properties")) or []
for field in _LEGACY_DATETIME_COLUMNS:
if field in facts:
facts[field] = _decode_dt(facts[field])
facts["label"] = row["label"]
facts["type"] = row["type"]
existing = await find_device_for(
db, ip=facts.get("ip"), mac=facts.get("mac"), ieee=facts.get("ieee_address")
)
node = await db.get(Node, row["id"])
if node is None: # pragma: no cover - the row was just read
continue
device = await link_facts(db, node, facts, overwrite_scalars=True)
if device is None:
# One savepoint per node: a row the merge cannot write is dropped on its
# own rather than taking the whole backfill with it. Every node it does
# not link keeps its legacy columns, so nothing is lost and the next boot
# retries. The catch is deliberately broad — a node that cannot be read
# or written for *any* reason must not cost the others their link.
try:
async with db.begin_nested():
node = await db.get(Node, row["id"])
if node is None: # pragma: no cover - the row was just read
continue
if node.device_id and not await _row_is_missing_facts(db, node.device_id, facts):
continue # Already migrated, and its row holds the facts.
existing = await find_device_for(
db, ip=facts.get("ip"), mac=facts.get("mac"), ieee=facts.get("ieee_address")
)
# A node that is already linked only has its gaps filled: whatever
# is on the row was written after the migration and is newer.
device = await link_facts(
db,
node,
facts,
overwrite_scalars=node.device_id is None,
# The node's own columns are exactly what this canvas drew
# before the upgrade, empty lists included — so they define
# its view outright. Anything else the row carries (a service
# a scan fingerprinted, a property another canvas added)
# stays hidden here rather than appearing on every canvas.
strict_view=True,
)
if device is None:
continue
await db.flush()
except Exception as exc:
skipped += 1
logger.warning(
"Inventory backfill: node %s (%s) skipped — %s", row["id"], row["label"], exc
)
continue
linked += 1
if existing is None:
created += 1
logger.info(
@@ -558,4 +957,4 @@ async def backfill_node_devices(db: AsyncSession) -> dict[str, int]:
)
await db.flush()
return {"linked": len(rows), "created": created, "merged": merged}
return {"linked": linked, "created": created, "merged": merged, "skipped": skipped}
+9 -1
View File
@@ -19,6 +19,7 @@ from app.db.models import InventoryDevice, ScanRun
from app.services.discovery_sources import add_source
from app.services.fingerprint import fingerprint_ports, suggest_node_type
from app.services.http_probe import probe_open_ports
from app.services.inventory_sync import merge_services
from app.services.mac_utils import normalize_mac
logger = logging.getLogger(__name__)
@@ -591,7 +592,14 @@ async def run_scan(
keep.mac = norm_mac or keep.mac
keep.hostname = host.get("hostname") or keep.hostname
keep.os = host.get("os") or keep.os
keep.services = services
# Union, never replace. Since 3.3.0 the row is the only copy of
# a device's services — every canvas drawing it reads this list
# — so overwriting it with the fingerprint would delete services
# the user added by hand, on every canvas at once. What a scan
# finds is added; what it no longer sees stays. A service the
# user does not want on a canvas is hidden by that node's view,
# which is where "I deleted this" belongs.
keep.services = merge_services(keep.services, services)
# Don't downgrade a Proxmox-typed guest (vm/lxc) to the generic
# scan guess; the importer knows the true type.
if not (keep.ieee_address or "").startswith("pve-"):
@@ -0,0 +1,423 @@
"""The 3.2.0 → 3.3.0 upgrade of the `nodes` table.
3.3.0 moved the device facts off `nodes` onto `device_inventory` and dropped the
columns. A real 3.2.0 database declares several of them NOT NULL with no
server-side default (`status`, `services`, `properties`, `show_hardware`) — the
schema here is `create_all`'s output at tag v3.2.0, not a hand-simplified copy,
because that constraint is exactly what breaks: the 3.3.0 model no longer writes
those columns, so if they survive the upgrade every INSERT fails with
``NOT NULL constraint failed: nodes.status`` and approving a device is dead.
"""
import os
import shutil
os.environ.setdefault("SECRET_KEY", "test-only-secret-key-not-for-production")
import pytest
from sqlalchemy.exc import IntegrityError
from sqlalchemy.ext.asyncio import create_async_engine
import app.db.database as database
import app.services.inventory_sync as inventory_sync
# `create_all` under v3.2.0. NOT NULL is reproduced exactly where it was.
_NODES_320 = (
"CREATE TABLE nodes ("
"id VARCHAR NOT NULL PRIMARY KEY, type VARCHAR NOT NULL, label VARCHAR NOT NULL, "
"design_id VARCHAR, hostname VARCHAR, ip VARCHAR, mac VARCHAR, os VARCHAR, "
"status VARCHAR NOT NULL, check_method VARCHAR, check_target VARCHAR, "
"services JSON NOT NULL, notes TEXT, pos_x FLOAT NOT NULL, pos_y FLOAT NOT NULL, "
"parent_id VARCHAR, container_mode BOOLEAN NOT NULL, custom_colors JSON, "
"custom_icon VARCHAR, cpu_count INTEGER, cpu_model VARCHAR, ram_gb FLOAT, "
"disk_gb FLOAT, show_hardware BOOLEAN NOT NULL, show_port_numbers BOOLEAN NOT NULL, "
"properties JSON NOT NULL, width FLOAT, height FLOAT, "
"bottom_handles INTEGER NOT NULL, top_handles INTEGER NOT NULL, "
"left_handles INTEGER NOT NULL, right_handles INTEGER NOT NULL, "
"ieee_address VARCHAR, last_seen DATETIME, last_scan DATETIME, "
"response_time_ms INTEGER, created_at DATETIME NOT NULL, updated_at DATETIME NOT NULL)"
)
_DESIGNS_320 = (
"CREATE TABLE designs (id VARCHAR NOT NULL PRIMARY KEY, name VARCHAR NOT NULL, "
"design_type VARCHAR NOT NULL, icon VARCHAR, created_at DATETIME NOT NULL, "
"updated_at DATETIME NOT NULL)"
)
def _node_sql(node_id: str, label: str, node_type: str, ip: str | None) -> str:
ip_sql = f"'{ip}'" if ip else "NULL"
return (
"INSERT INTO nodes (id, type, label, design_id, ip, status, services, pos_x, pos_y, "
"container_mode, show_hardware, show_port_numbers, properties, bottom_handles, "
"top_handles, left_handles, right_handles, created_at, updated_at) VALUES "
f"('{node_id}', '{node_type}', '{label}', 'd1', {ip_sql}, 'online', '[]', 0, 0, "
"0, 0, 0, '[]', 1, 1, 0, 0, '2026-01-01 00:00:00', '2026-01-01 00:00:00')"
)
@pytest.fixture
def db_320(tmp_path, monkeypatch):
"""A v3.2.0 database, with the module-global engine pointed at it."""
db_path = tmp_path / "v320.db"
monkeypatch.setattr(database.settings, "sqlite_path", str(db_path))
engine = create_async_engine(f"sqlite+aiosqlite:///{db_path}")
monkeypatch.setattr(database, "engine", engine)
monkeypatch.setattr(
database, "AsyncSessionLocal", database.async_sessionmaker(engine, expire_on_commit=False)
)
return db_path, engine
async def _build_320(engine) -> None:
async with engine.begin() as conn:
await conn.exec_driver_sql(_DESIGNS_320)
await conn.exec_driver_sql(_NODES_320)
await conn.exec_driver_sql(
"INSERT INTO designs (id, name, design_type, icon, created_at, updated_at) "
"VALUES ('d1', 'Network Topology', 'network', 'dashboard', "
"'2026-01-01 00:00:00', '2026-01-01 00:00:00')"
)
await conn.exec_driver_sql(_node_sql("n1", "OpnSense", "firewall", "192.168.1.1"))
await conn.exec_driver_sql(_node_sql("n2", "NAS", "nas", "192.168.1.2"))
await conn.exec_driver_sql(_node_sql("n3", "a note", "text", None))
async def _insert_node(engine) -> None:
"""What approve does: insert a node the way the 3.3.0 model writes one."""
from app.db.models import Node
session_factory = database.async_sessionmaker(engine, expire_on_commit=False)
async with session_factory() as session:
session.add(Node(id="new", type="firewall", label="Approved", design_id="d1"))
await session.commit()
async def test_upgrade_drops_the_device_columns_and_leaves_nodes_insertable(db_320):
db_path, engine = db_320
await _build_320(engine)
await database.init_db()
check = create_async_engine(f"sqlite+aiosqlite:///{db_path}")
try:
async with check.begin() as conn:
cols = {c[1] for c in (await conn.exec_driver_sql("PRAGMA table_info(nodes)")).fetchall()}
assert "status" not in cols
assert "ip" not in cols
assert "device_id" in cols
# Every device node linked; the facts moved to the inventory row.
linked = (await conn.exec_driver_sql(
"SELECT label, device_id FROM nodes ORDER BY id"
)).fetchall()
assert [row[0] for row in linked] == ["OpnSense", "NAS", "a note"]
assert linked[0][1] and linked[1][1]
assert linked[2][1] is None # canvas furniture keeps no row
ips = (await conn.exec_driver_sql(
"SELECT ip FROM device_inventory ORDER BY ip"
)).fetchall()
assert [row[0] for row in ips] == ["192.168.1.1", "192.168.1.2"]
await _insert_node(engine)
finally:
await check.dispose()
await engine.dispose()
async def test_a_node_the_backfill_cannot_link_still_leaves_nodes_insertable(db_320, monkeypatch):
"""The regression: a failed backfill must not brick every later INSERT.
Before the fix, one unlinkable node kept `nodes.status` NOT NULL while the
3.3.0 model stopped writing it, so approving *any* device raised
``NOT NULL constraint failed: nodes.status`` (issue #351).
"""
db_path, engine = db_320
await _build_320(engine)
real_link_facts = inventory_sync.link_facts
async def link_facts(db, node, facts, **kwargs):
if node.id == "n2":
raise IntegrityError("boom", None, Exception("UNIQUE constraint failed"))
return await real_link_facts(db, node, facts, **kwargs)
monkeypatch.setattr(inventory_sync, "link_facts", link_facts)
await database.init_db()
check = create_async_engine(f"sqlite+aiosqlite:///{db_path}")
try:
async with check.begin() as conn:
info = (await conn.exec_driver_sql("PRAGMA table_info(nodes)")).fetchall()
cols = {c[1]: c for c in info}
# The columns stay — they are the only remaining copy of n2's facts.
assert "status" in cols
# But nothing is NOT NULL any more, so the model can insert again.
assert not any(c[3] for name, c in cols.items() if name in database._LEGACY_NODE_COLUMNS)
# Values preserved by the rebuild.
kept = (await conn.exec_driver_sql(
"SELECT status, ip FROM nodes WHERE id='n2'"
)).fetchone()
assert kept == ("online", "192.168.1.2")
# The node that did link is linked; the failed one is not.
linked = dict((await conn.exec_driver_sql(
"SELECT id, device_id FROM nodes"
)).fetchall())
assert linked["n1"] is not None
assert linked["n2"] is None
await _insert_node(engine)
# A boot where the failure persists changes nothing further: the relax is
# a one-off, not a rebuild on every start.
await database.init_db()
async with check.begin() as conn:
kept = (await conn.exec_driver_sql(
"SELECT status, ip FROM nodes WHERE id='n2'"
)).fetchone()
assert kept == ("online", "192.168.1.2")
assert (await conn.exec_driver_sql(
"SELECT COUNT(*) FROM nodes"
)).scalar() == 4 # the three seeded, plus the one just approved
# A later boot retries the backfill: with the failure gone, n2 links and
# the columns finally drop.
monkeypatch.setattr(inventory_sync, "link_facts", real_link_facts)
await database.init_db()
async with check.begin() as conn:
cols = {c[1] for c in (await conn.exec_driver_sql("PRAGMA table_info(nodes)")).fetchall()}
assert "status" not in cols
finally:
await check.dispose()
await engine.dispose()
async def test_upgrade_keeps_each_canvas_drawing_its_own_services(db_320):
"""3.3.3: order and visibility become the node's, and the upgrade freezes them.
Two canvases drew the same host with different service lists, and a scan had
already fingerprinted a third the user put on neither. All three converge on
one inventory row — so each node must come out of the upgrade still drawing
what it drew, with the scanner's guess hidden on both.
"""
db_path, engine = db_320
await _build_320(engine)
ssh = '[{"port": 22, "protocol": "tcp", "service_name": "ssh"}]'
both = '[{"port": 22, "protocol": "tcp", "service_name": "ssh"}, ' \
'{"port": 443, "protocol": "tcp", "service_name": "https"}]'
async with engine.begin() as conn:
await conn.exec_driver_sql(f"UPDATE nodes SET services = '{ssh}' WHERE id = 'n1'")
await conn.exec_driver_sql(f"UPDATE nodes SET services = '{both}' WHERE id = 'n2'")
await database.init_db()
from app.db.models import InventoryDevice, Node
session_factory = database.async_sessionmaker(engine, expire_on_commit=False)
async with session_factory() as session:
drawn = {}
for node_id in ("n1", "n2"):
node = await session.get(Node, node_id)
device = await session.get(InventoryDevice, node.device_id)
payload = inventory_sync.hydrated_node(node, device)
drawn[node_id] = [s["service_name"] for s in payload["services"] if s.get("visible", True)]
# Seeded, not left to "show the whole row".
assert node.display_view is not None
assert drawn == {"n1": ["ssh"], "n2": ["ssh", "https"]}
# What a scan finds next lands on the row, and on no canvas.
node = await session.get(Node, "n1")
device = await session.get(InventoryDevice, node.device_id)
device.services = [
*device.services,
{"port": 3001, "protocol": "tcp", "service_name": "Uptime Kuma"},
]
await session.commit()
payload = inventory_sync.hydrated_node(node, device)
assert [(s["service_name"], s.get("visible", True)) for s in payload["services"]] == [
("ssh", True), ("Uptime Kuma", False),
]
await engine.dispose()
async def _wind_back_to_3_3_2(engine, extra_service: dict) -> None:
"""A database that already took the 3.3.0 upgrade, leak included.
Both nodes point at one row holding the union of what each drew plus what a
scan found, and neither has a view — which is every 3.3.0-3.3.2 install.
"""
from app.db.models import InventoryDevice, Node
async with engine.begin() as conn:
await conn.exec_driver_sql("UPDATE nodes SET display_view = NULL")
session_factory = database.async_sessionmaker(engine, expire_on_commit=False)
async with session_factory() as session:
node = await session.get(Node, "n1")
one = await session.get(InventoryDevice, node.device_id)
other = await session.get(InventoryDevice, (await session.get(Node, "n2")).device_id)
for device in (one, other):
device.services = [
{"port": 22, "protocol": "tcp", "service_name": "ssh"},
{"port": 443, "protocol": "tcp", "service_name": "https"},
extra_service,
]
await session.commit()
async def test_a_database_already_on_3_3_recovers_its_layout_from_the_backup(db_320):
"""The second upgrade path: 3.3.0-3.3.2, where the legacy columns are gone.
3.3.0 unioned every canvas' services onto one row, so the row can no longer
say who drew what — but the backup taken before that migration still can.
Recovering from it is the difference between the user getting their canvases
back and getting each canvas showing every other canvas' services.
"""
_, engine = db_320
await _build_320(engine)
ssh = '[{"port": 22, "protocol": "tcp", "service_name": "ssh"}]'
both = '[{"port": 22, "protocol": "tcp", "service_name": "ssh"}, ' \
'{"port": 443, "protocol": "tcp", "service_name": "https"}]'
async with engine.begin() as conn:
await conn.exec_driver_sql(f"UPDATE nodes SET services = '{ssh}' WHERE id = 'n1'")
await conn.exec_driver_sql(f"UPDATE nodes SET services = '{both}' WHERE id = 'n2'")
await database.init_db() # 3.2.0 -> 3.3.x, and the backup that predates it.
kuma = {"port": 3001, "protocol": "tcp", "service_name": "Uptime Kuma"}
await _wind_back_to_3_3_2(engine, kuma)
await database.init_db()
from app.db.models import InventoryDevice, Node
session_factory = database.async_sessionmaker(engine, expire_on_commit=False)
async with session_factory() as session:
drawn = {}
for node_id in ("n1", "n2"):
node = await session.get(Node, node_id)
device = await session.get(InventoryDevice, node.device_id)
payload = inventory_sync.hydrated_node(node, device)
drawn[node_id] = [s["service_name"] for s in payload["services"] if s.get("visible", True)]
assert drawn == {"n1": ["ssh"], "n2": ["ssh", "https"]}
await engine.dispose()
async def test_a_property_added_while_on_3_3_survives_the_recovery(db_320):
"""The backup is 3.2.0-era and cannot know what the user added afterwards.
On 3.3.0-3.3.2 the row was the only place to add a property, and every
canvas drew it. Recovering the view strictly from the backup would hide it
on all of them at once — the "my properties disappeared" half of #347. The
scanner's service find is still held back: only properties are appended.
"""
_, engine = db_320
await _build_320(engine)
rack = '[{"key": "Rack", "value": "A", "icon": null, "visible": true}]'
ssh = '[{"port": 22, "protocol": "tcp", "service_name": "ssh"}]'
async with engine.begin() as conn:
await conn.exec_driver_sql(
f"UPDATE nodes SET properties = '{rack}', services = '{ssh}' WHERE id = 'n1'"
)
await database.init_db() # 3.2.0 -> 3.3.x, and the backup that predates it.
kuma = {"port": 3001, "protocol": "tcp", "service_name": "Uptime Kuma"}
await _wind_back_to_3_3_2(engine, kuma)
from app.db.models import InventoryDevice, Node
session_factory = database.async_sessionmaker(engine, expire_on_commit=False)
async with session_factory() as session:
node = await session.get(Node, "n1")
device = await session.get(InventoryDevice, node.device_id)
# What the user re-added by hand while running 3.3.x.
device.properties = [
*device.properties,
{"key": "Ports", "value": "8", "icon": None, "visible": True},
]
await session.commit()
await database.init_db()
async with session_factory() as session:
node = await session.get(Node, "n1")
device = await session.get(InventoryDevice, node.device_id)
payload = inventory_sync.hydrated_node(node, device)
assert [(p["key"], p.get("visible", True)) for p in payload["properties"]] == [
("Rack", True), ("Ports", True),
]
# And the scan's find is still off this canvas.
assert [s["service_name"] for s in payload["services"] if s.get("visible", True)] == ["ssh"]
await engine.dispose()
async def test_without_a_usable_backup_the_row_is_the_seed(db_320):
"""No backup to recover from: keep showing what the canvas shows today.
Nothing is taken away — the user simply keeps the merged list they have been
looking at since 3.3.0, and only what the row gains *after* this boot is
held back.
"""
db_path, engine = db_320
await _build_320(engine)
await database.init_db()
kuma = {"port": 3001, "protocol": "tcp", "service_name": "Uptime Kuma"}
await _wind_back_to_3_3_2(engine, kuma)
for backup in db_path.parent.glob(f"{db_path.name}.back-*"):
backup.unlink()
await database.init_db()
from app.db.models import InventoryDevice, Node
session_factory = database.async_sessionmaker(engine, expire_on_commit=False)
async with session_factory() as session:
node = await session.get(Node, "n1")
device = await session.get(InventoryDevice, node.device_id)
payload = inventory_sync.hydrated_node(node, device)
assert [s["service_name"] for s in payload["services"]] == ["ssh", "https", "Uptime Kuma"]
assert all(s.get("visible", True) for s in payload["services"])
# From here on the node is pinned: the next scan's find is held back.
device.services = [*device.services, {"port": 5001, "protocol": "tcp", "service_name": "Synology DSM HTTPS"}]
await session.commit()
payload = inventory_sync.hydrated_node(node, device)
assert [s["service_name"] for s in payload["services"] if not s.get("visible", True)] == [
"Synology DSM HTTPS"
]
await engine.dispose()
async def test_the_newest_backup_that_still_has_the_columns_is_the_one_read(db_320):
"""A 3.3.2 install has several backups; only some can answer.
`homelab.db.back-3.3.1` and `-3.3.2` were taken *after* the split and hold
nothing per node; `-3.3.0` was taken before it. Reading the wrong one would
either say nothing or resurrect a much older canvas.
"""
db_path, engine = db_320
await _build_320(engine)
async with engine.begin() as conn:
await conn.exec_driver_sql(
"UPDATE nodes SET services = "
"'[{\"port\": 22, \"protocol\": \"tcp\", \"service_name\": \"ssh\"}]' WHERE id = 'n1'"
)
# Ancient: same shape, but a canvas the user has long since moved on from.
old = db_path.parent / f"{db_path.name}.back-3.1.0"
shutil.copy2(db_path, old)
os.utime(old, (1, 1))
async with engine.begin() as conn:
await conn.exec_driver_sql(
"UPDATE nodes SET services = "
"'[{\"port\": 443, \"protocol\": \"tcp\", \"service_name\": \"https\"}]' WHERE id = 'n1'"
)
pre_split = db_path.parent / f"{db_path.name}.back-3.3.0"
shutil.copy2(db_path, pre_split)
await database.init_db() # drops the columns, and backs up under this version
# Post-split backups: newer, and unable to say who drew what.
for name in ("back-3.3.1", "back-3.3.2"):
shutil.copy2(db_path, db_path.parent / f"{db_path.name}.{name}")
assert database._pre_split_backup() == pre_split
assert database._views_from_backup()["n1"]["services"] == [
{"port": 443, "protocol": "tcp", "service_name": "https"}
]
await engine.dispose()
+614 -4
View File
@@ -17,8 +17,11 @@ from app.services.inventory_sync import (
backfill_node_devices,
changed_facts,
find_device_for,
hydrated_node,
link_facts,
merge_properties,
merge_services,
seed_node_views,
)
@@ -178,6 +181,60 @@ class TestFindDeviceFor:
found = await find_device_for(db_session, ip="10.0.0.5", mac=None, ieee=None)
assert found is not None and found.id == "d-hidden"
@pytest.mark.asyncio
async def test_matches_an_ieee_written_in_another_case(self, db_session):
"""One radio, one row: `ieee_address` is UNIQUE, so a case-sensitive
match here would mint a second row the index then refuses."""
db_session.add(InventoryDevice(id="d-1", ieee_address="0x00124b0022334455"))
await db_session.commit()
found = await find_device_for(
db_session, ip=None, mac=None, ieee="0x00124B0022334455"
)
assert found is not None and found.id == "d-1"
class TestIeeeCollisions:
"""`device_inventory.ieee_address` is UNIQUE — a merge must never duplicate it."""
@pytest.mark.asyncio
async def test_an_address_another_row_owns_is_left_where_it_is(self, db_session):
"""The node is already linked, so identity is settled — the IEEE is not
a reason to move it, and writing it here would violate the index."""
design = await _design(db_session)
db_session.add_all([
InventoryDevice(id="d-radio", ieee_address="0xAAA"),
InventoryDevice(id="d-host", ip="10.0.0.5"),
])
node = _node(design, device_id="d-host")
db_session.add(node)
await db_session.commit()
device = await link_facts(
db_session, node, {"ip": "10.0.0.5", "ieee_address": "0xAAA"}, overwrite_scalars=True
)
await db_session.commit()
assert device is not None and device.id == "d-host"
assert device.ieee_address is None
radio = await db_session.get(InventoryDevice, "d-radio")
assert radio is not None and radio.ieee_address == "0xAAA"
@pytest.mark.asyncio
async def test_the_row_that_already_holds_it_keeps_writing_it(self, db_session):
design = await _design(db_session)
db_session.add(InventoryDevice(id="d-1", ip="10.0.0.5", ieee_address="0xAAA"))
node = _node(design)
db_session.add(node)
await db_session.commit()
device = await link_facts(
db_session, node, {"ip": "10.0.0.5", "ieee_address": "0xAAA"}, overwrite_scalars=True
)
await db_session.commit()
assert device is not None and device.ieee_address == "0xAAA"
# --- the backfill -----------------------------------------------------------
@@ -238,7 +295,7 @@ class TestBackfill:
db_session.add(_node(design))
await db_session.commit()
assert await backfill_node_devices(db_session) == {"linked": 0, "created": 0, "merged": 0}
assert await backfill_node_devices(db_session) == {"linked": 0, "created": 0, "merged": 0, "skipped": 0}
@pytest.mark.asyncio
async def test_links_a_node_to_its_existing_row(self, db_session):
@@ -251,7 +308,7 @@ class TestBackfill:
stats = await backfill_node_devices(db_session)
await db_session.commit()
assert stats == {"linked": 1, "created": 0, "merged": 1}
assert stats == {"linked": 1, "created": 0, "merged": 1, "skipped": 0}
node = await db_session.get(Node, node_id)
assert node is not None and node.device_id == "d-1"
@@ -266,7 +323,7 @@ class TestBackfill:
stats = await backfill_node_devices(db_session)
await db_session.commit()
assert stats == {"linked": 1, "created": 1, "merged": 0}
assert stats == {"linked": 1, "created": 1, "merged": 0, "skipped": 0}
node = await db_session.get(Node, node_id)
assert node is not None
device = await db_session.get(InventoryDevice, node.device_id)
@@ -330,6 +387,133 @@ class TestBackfill:
assert stats["linked"] == 0
assert (await db_session.execute(select(InventoryDevice))).scalars().all() == []
@pytest.mark.asyncio
async def test_carries_over_timestamps_stored_as_text(self, db_session):
"""The legacy columns are read with raw SQL, so SQLite hands the DATETIME
ones back as strings. Writing one straight into the row raises
``TypeError: SQLite DateTime type only accepts Python datetime and date
objects`` — which used to abort the whole backfill (issue #347)."""
await self._legacy_nodes_table(db_session)
design = await _design(db_session)
node_id = await self._legacy_node(
db_session,
design,
ip="10.0.0.5",
label="Dockerhost",
status="online",
last_seen="2026-08-15 04:10:22.123456",
last_scan="2026-08-15 04:00:00",
)
stats = await backfill_node_devices(db_session)
await db_session.commit()
assert stats == {"linked": 1, "created": 1, "merged": 0, "skipped": 0}
node = await db_session.get(Node, node_id)
assert node is not None
device = await db_session.get(InventoryDevice, node.device_id)
assert device is not None
assert device.last_seen == datetime(2026, 8, 15, 4, 10, 22, 123456)
assert device.last_scan == datetime(2026, 8, 15, 4, 0, 0)
assert device.status_live == "online"
@pytest.mark.asyncio
async def test_a_stamp_it_cannot_parse_costs_only_the_stamp(self, db_session):
await self._legacy_nodes_table(db_session)
design = await _design(db_session)
await self._legacy_node(db_session, design, ip="10.0.0.5", last_seen="not a date")
stats = await backfill_node_devices(db_session)
await db_session.commit()
assert stats["linked"] == 1
device = (await db_session.execute(select(InventoryDevice))).scalars().one()
assert device.last_seen is None
assert device.ip == "10.0.0.5"
@pytest.mark.asyncio
async def test_one_unwritable_node_does_not_cost_the_others_their_link(self, db_session, monkeypatch):
"""Whatever a single node raises — not only a constraint violation — the
rest of the canvas still gets linked."""
await self._legacy_nodes_table(db_session)
design = await _design(db_session)
await self._legacy_node(db_session, design, ip="10.0.0.5", label="good")
await self._legacy_node(db_session, design, ip="10.0.0.6", label="bad")
import app.services.inventory_sync as module
real = module.link_facts
async def link_facts(db, node, facts, **kwargs):
if facts.get("ip") == "10.0.0.6":
raise TypeError("SQLite DateTime type only accepts Python datetime")
return await real(db, node, facts, **kwargs)
monkeypatch.setattr(module, "link_facts", link_facts)
stats = await module.backfill_node_devices(db_session)
await db_session.commit()
assert stats["linked"] == 1
assert stats["skipped"] == 1
devices = (await db_session.execute(select(InventoryDevice))).scalars().all()
assert [d.ip for d in devices] == ["10.0.0.5"]
@pytest.mark.asyncio
async def test_fills_a_blank_row_a_stuck_canvas_save_created(self, db_session):
"""A save made while an earlier migration was stuck linked the node to a
row minted from a blank UI. Those facts only exist in the legacy columns,
and the drop is about to remove them — so fill the row before it does."""
await self._legacy_nodes_table(db_session)
design = await _design(db_session)
db_session.add(InventoryDevice(id="d-blank", ip="", label="Dockerhost", services=[]))
await db_session.commit()
node_id = await self._legacy_node(
db_session, design, label="Dockerhost", ip="192.168.0.42", hostname="clara.lan",
notes="in the cupboard", services=[{"port": 443, "protocol": "tcp", "service_name": "https"}],
)
node = await db_session.get(Node, node_id)
node.device_id = "d-blank"
await db_session.commit()
stats = await backfill_node_devices(db_session)
await db_session.commit()
assert stats["linked"] == 1
device = await db_session.get(InventoryDevice, "d-blank")
assert device is not None
assert device.ip == "192.168.0.42"
assert device.hostname == "clara.lan"
assert device.notes == "in the cupboard"
assert [s["service_name"] for s in device.services] == ["https"]
@pytest.mark.asyncio
async def test_leaves_a_linked_row_that_already_has_the_facts_alone(self, db_session):
"""The mirror of the above: a row a user has since edited is not reverted
to what the node's abandoned columns still say."""
await self._legacy_nodes_table(db_session)
design = await _design(db_session)
db_session.add(InventoryDevice(
id="d-1", ip="192.168.0.99", hostname="renamed.lan",
label="Dockerhost", type="docker_host",
))
await db_session.commit()
node_id = await self._legacy_node(
db_session, design, ip="192.168.0.42", hostname="clara.lan", label="Dockerhost"
)
node = await db_session.get(Node, node_id)
node.device_id = "d-1"
await db_session.commit()
stats = await backfill_node_devices(db_session)
await db_session.commit()
assert stats["linked"] == 0
device = await db_session.get(InventoryDevice, "d-1")
assert device is not None
assert device.ip == "192.168.0.99"
assert device.hostname == "renamed.lan"
@pytest.mark.asyncio
async def test_is_a_no_op_on_a_second_run(self, db_session):
await self._legacy_nodes_table(db_session)
@@ -342,10 +526,65 @@ class TestBackfill:
await db_session.commit()
assert first["linked"] == 1
assert second == {"linked": 0, "created": 0, "merged": 0}
assert second == {"linked": 0, "created": 0, "merged": 0, "skipped": 0}
rows = (await db_session.execute(select(InventoryDevice))).scalars().all()
assert len(rows) == 1
@pytest.mark.asyncio
async def test_each_canvas_keeps_showing_what_it_showed(self, db_session):
"""The upgrade must not redraw a canvas.
Two canvases drew the same host with different service lists, and the
scanner's row for it holds a third the user never put on either. They
converge on one row — so each node keeps a view of the subset it drew,
and the scanner's guess appears on neither.
"""
await self._legacy_nodes_table(db_session)
ssh = {"port": 22, "protocol": "tcp", "service_name": "ssh"}
https = {"port": 443, "protocol": "tcp", "service_name": "https"}
kuma = {"port": 3001, "protocol": "tcp", "service_name": "Uptime Kuma"}
db_session.add(InventoryDevice(id="d-1", ip="10.0.0.5", services=[kuma]))
await db_session.commit()
one, two = await _design(db_session, "Net"), await _design(db_session, "Rack")
a = await self._legacy_node(db_session, one, ip="10.0.0.5", services=[ssh])
b = await self._legacy_node(db_session, two, ip="10.0.0.5", services=[ssh, https])
await backfill_node_devices(db_session)
await db_session.commit()
device = await db_session.get(InventoryDevice, "d-1")
assert {s["service_name"] for s in device.services} == {"Uptime Kuma", "ssh", "https"}
drawn = {}
for node_id in (a, b):
node = await db_session.get(Node, node_id)
payload = hydrated_node(node, device)
drawn[node_id] = [s["service_name"] for s in payload["services"] if s.get("visible", True)]
assert drawn[a] == ["ssh"]
assert drawn[b] == ["ssh", "https"]
@pytest.mark.asyncio
async def test_a_node_that_drew_no_service_keeps_drawing_none(self, db_session):
"""An empty legacy list is an answer: that canvas showed no services."""
await self._legacy_nodes_table(db_session)
db_session.add(
InventoryDevice(
id="d-1", ip="10.0.0.5",
services=[{"port": 3001, "protocol": "tcp", "service_name": "Uptime Kuma"}],
)
)
await db_session.commit()
design = await _design(db_session)
node_id = await self._legacy_node(db_session, design, ip="10.0.0.5", services=[])
await backfill_node_devices(db_session)
await db_session.commit()
node = await db_session.get(Node, node_id)
device = await db_session.get(InventoryDevice, "d-1")
payload = hydrated_node(node, device)
assert [s.get("visible", True) for s in payload["services"]] == [False]
# --- routes -----------------------------------------------------------------
@@ -798,3 +1037,374 @@ class TestRoutesKeepTheLinkInStep:
assert res.status_code == 200
node = await db_session.get(Node, res.json()["node_id"])
assert node is not None and node.device_id == "d-1"
class TestPerNodeView:
"""Order and visibility belong to the node, the facts to the row.
The same device drawn on two canvases is one inventory row, so without a
per-node view every canvas showing it inherits every service a scan ever
fingerprinted and every property any other canvas added.
"""
async def _node_on(self, client, headers, design_id: str, **payload) -> dict:
body = {"type": "nas", "label": "NAS", "ip": "10.0.0.5", "design_id": design_id, "force": True}
body.update(payload)
res = await client.post("/api/v1/nodes", json=body, headers=body.pop("headers", None) or headers)
assert res.status_code == 201
return res.json()
@pytest.mark.asyncio
async def test_a_new_node_shows_what_the_row_already_holds(
self, client: AsyncClient, headers, db_session
):
"""An empty payload list means "I have nothing to say", not "hide it all"."""
db_session.add(
InventoryDevice(
id="d-1",
ip="10.0.0.5",
services=[{"port": 22, "protocol": "tcp", "service_name": "ssh"}],
properties=[{"key": "Rack", "value": "A1", "icon": None, "visible": True}],
)
)
await db_session.commit()
design = await _design(db_session)
node = await self._node_on(client, headers, design)
assert [s["service_name"] for s in node["services"]] == ["ssh"]
assert all(s.get("visible", True) for s in node["services"])
assert node["properties"][0]["visible"] is True
@pytest.mark.asyncio
async def test_hiding_a_service_on_one_node_leaves_the_other_alone(
self, client: AsyncClient, headers, db_session
):
db_session.add(
InventoryDevice(
id="d-1",
ip="10.0.0.5",
services=[
{"port": 22, "protocol": "tcp", "service_name": "ssh"},
{"port": 3001, "protocol": "tcp", "service_name": "Uptime Kuma"},
],
)
)
await db_session.commit()
one, two = await _design(db_session, "Net"), await _design(db_session, "Rack")
node_a = await self._node_on(client, headers, one)
node_b = await self._node_on(client, headers, two)
hidden = [
{"port": 22, "protocol": "tcp", "service_name": "ssh"},
{"port": 3001, "protocol": "tcp", "service_name": "Uptime Kuma", "visible": False},
]
res = await client.patch(
f"/api/v1/nodes/{node_a['id']}", json={"services": hidden}, headers=headers
)
assert res.status_code == 200
assert [(s["service_name"], s.get("visible", True)) for s in res.json()["services"]] == [
("ssh", True), ("Uptime Kuma", False),
]
other = (await client.get(f"/api/v1/nodes/{node_b['id']}", headers=headers)).json()
assert [s.get("visible", True) for s in other["services"]] == [True, True]
# The service itself is untouched — hiding is not deleting.
device = await db_session.get(InventoryDevice, "d-1")
await db_session.refresh(device)
assert len(device.services) == 2
@pytest.mark.asyncio
async def test_the_order_is_per_node_too(self, client: AsyncClient, headers, db_session):
db_session.add(
InventoryDevice(
id="d-1",
ip="10.0.0.5",
properties=[
{"key": "Rack", "value": "A1", "icon": None, "visible": True},
{"key": "Owner", "value": "me", "icon": None, "visible": True},
],
)
)
await db_session.commit()
one, two = await _design(db_session, "Net"), await _design(db_session, "Rack")
node_a = await self._node_on(client, headers, one)
node_b = await self._node_on(client, headers, two)
flipped = [
{"key": "Owner", "value": "me", "icon": None, "visible": True},
{"key": "Rack", "value": "A1", "icon": None, "visible": True},
]
res = await client.patch(
f"/api/v1/nodes/{node_a['id']}", json={"properties": flipped}, headers=headers
)
assert [p["key"] for p in res.json()["properties"]] == ["Owner", "Rack"]
other = (await client.get(f"/api/v1/nodes/{node_b['id']}", headers=headers)).json()
assert [p["key"] for p in other["properties"]] == ["Rack", "Owner"]
@pytest.mark.asyncio
async def test_a_service_the_row_gains_later_stays_off_the_canvas(
self, client: AsyncClient, headers, db_session
):
"""The leak this column exists to stop: a scan must not redraw every canvas."""
db_session.add(
InventoryDevice(
id="d-1", ip="10.0.0.5",
services=[{"port": 22, "protocol": "tcp", "service_name": "ssh"}],
)
)
await db_session.commit()
design = await _design(db_session)
node = await self._node_on(client, headers, design)
device = await db_session.get(InventoryDevice, "d-1")
device.services = [
*device.services,
{"port": 3001, "protocol": "tcp", "service_name": "Uptime Kuma"},
]
await db_session.commit()
after = (await client.get(f"/api/v1/nodes/{node['id']}", headers=headers)).json()
assert [(s["service_name"], s.get("visible", True)) for s in after["services"]] == [
("ssh", True), ("Uptime Kuma", False),
]
@pytest.mark.asyncio
async def test_removing_a_service_removes_it_from_the_device(
self, client: AsyncClient, headers, db_session
):
"""Delete is device-wide — hiding is what a single canvas does."""
db_session.add(
InventoryDevice(
id="d-1", ip="10.0.0.5",
services=[
{"port": 22, "protocol": "tcp", "service_name": "ssh"},
{"port": 3001, "protocol": "tcp", "service_name": "Uptime Kuma"},
],
)
)
await db_session.commit()
one, two = await _design(db_session, "Net"), await _design(db_session, "Rack")
node_a = await self._node_on(client, headers, one)
node_b = await self._node_on(client, headers, two)
await client.patch(
f"/api/v1/nodes/{node_a['id']}",
json={"services": [{"port": 22, "protocol": "tcp", "service_name": "ssh"}]},
headers=headers,
)
other = (await client.get(f"/api/v1/nodes/{node_b['id']}", headers=headers)).json()
assert [s["service_name"] for s in other["services"]] == ["ssh"]
@pytest.mark.asyncio
async def test_a_canvas_save_records_the_view_even_when_no_fact_changed(
self, client: AsyncClient, headers, db_session
):
"""`changed_facts` guards the shared row, not the node's own view.
A canvas save reporting no edited fact is exactly what hiding a service
looks like from the row's side — nothing about the device changed. The
view still has to land, or the toggle would not survive the save.
"""
db_session.add(
InventoryDevice(
id="d-1", ip="10.0.0.5",
services=[
{"port": 22, "protocol": "tcp", "service_name": "ssh"},
{"port": 3001, "protocol": "tcp", "service_name": "Uptime Kuma"},
],
)
)
await db_session.commit()
design = await _design(db_session)
node = await self._node_on(client, headers, design)
res = await client.post(
"/api/v1/canvas/save",
json={
"design_id": design,
"nodes": [{
"id": node["id"], "type": "nas", "label": "NAS", "ip": "10.0.0.5",
"device_id": "d-1", "changed_facts": [], "pos_x": 10.0, "pos_y": 20.0,
"services": [
{"port": 3001, "protocol": "tcp", "service_name": "Uptime Kuma", "visible": False},
{"port": 22, "protocol": "tcp", "service_name": "ssh"},
],
}],
"edges": [],
"viewport": {"x": 0, "y": 0, "zoom": 1},
},
headers=headers,
)
assert res.status_code == 200
loaded = (await client.get(f"/api/v1/canvas?design_id={design}", headers=headers)).json()
assert [(s["service_name"], s.get("visible", True)) for s in loaded["nodes"][0]["services"]] == [
("Uptime Kuma", False), ("ssh", True),
]
device = await db_session.get(InventoryDevice, "d-1")
await db_session.refresh(device)
assert [s["service_name"] for s in device.services] == ["ssh", "Uptime Kuma"]
@pytest.mark.asyncio
async def test_seeding_freezes_what_a_pre_upgrade_node_showed(self, db_session):
"""`display_view` arrives NULL on every existing node; the seed fills it."""
design = await _design(db_session)
db_session.add(
InventoryDevice(
id="d-1", ip="10.0.0.5",
services=[{"port": 22, "protocol": "tcp", "service_name": "ssh"}],
)
)
node = _node(design, device_id="d-1")
db_session.add(node)
await db_session.commit()
assert await seed_node_views(db_session) == 1
await db_session.commit()
assert node.display_view == {
"services": [{"key": "22|tcp|ssh", "visible": True}],
"properties": [],
}
# Idempotent: a second boot finds nothing without a view.
assert await seed_node_views(db_session) == 0
@pytest.mark.asyncio
async def test_a_node_whose_device_was_deleted_does_not_keep_the_seed_alive(self, db_session):
"""Deleting a device leaves `nodes.device_id` dangling — foreign keys are off.
Such a node can be seeded from nothing, so it must not keep matching the
query: it would re-open the pre-upgrade backup on every single boot and
log a recovery that recovers nothing.
"""
design = await _design(db_session)
db_session.add(_node(design, device_id="d-gone"))
await db_session.commit()
reads = []
def drawn():
reads.append(1)
return {}
assert await seed_node_views(db_session, drawn=drawn) == 0
assert reads == [] # the backup was never opened
@pytest.mark.asyncio
async def test_seeding_leaves_furniture_alone(self, db_session):
design = await _design(db_session)
node = _node(design, type="groupRect")
db_session.add(node)
await db_session.commit()
assert await seed_node_views(db_session) == 0
assert node.display_view is None
@pytest.mark.asyncio
async def test_a_view_matching_nothing_left_on_the_row_shows_the_row(self, db_session):
"""The row was replaced wholesale under the node — don't draw a blank node.
A view whose every key is gone tells us nothing about the list that
replaced it, and hiding all of it leaves a node showing no service while
"Show services" is on (#347). An *empty* view is a real answer and still
hides everything — that case is covered below.
"""
device = InventoryDevice(
id="d-1",
services=[
{"port": 8080, "protocol": "tcp", "service_name": "HTTP Alt"},
{"port": 5001, "protocol": "tcp", "service_name": "Synology DSM HTTPS"},
],
)
node = _node(
await _design(db_session),
device_id="d-1",
display_view={"services": [{"key": "9000|tcp|portainer", "visible": True}]},
)
assert [s["service_name"] for s in hydrated_node(node, device)["services"]] == [
"HTTP Alt", "Synology DSM HTTPS",
]
assert all(s.get("visible", True) for s in hydrated_node(node, device)["services"])
@pytest.mark.asyncio
async def test_an_empty_view_still_hides_everything(self, db_session):
""""This canvas draws none of them" is an answer, not a missing view."""
device = InventoryDevice(
id="d-1", services=[{"port": 22, "protocol": "tcp", "service_name": "ssh"}]
)
node = _node(await _design(db_session), device_id="d-1", display_view={"services": []})
assert hydrated_node(node, device)["services"] == [
{"port": 22, "protocol": "tcp", "service_name": "ssh", "visible": False}
]
@pytest.mark.asyncio
async def test_seeding_keeps_a_property_added_after_the_backup(self, db_session):
"""The backup is 3.2.0-era: it cannot know a property added since.
Adding one while on 3.3.0-3.3.2 wrote it to the row, and every canvas
drew it. Seeding strictly from the backup would take it off all of them
at once (#347), so it is appended visible after what the backup places.
"""
design = await _design(db_session)
db_session.add(
InventoryDevice(
id="d-1", ip="10.0.0.5",
properties=[
{"key": "Rack", "value": "A", "icon": None, "visible": True},
{"key": "Ports", "value": "8", "icon": None, "visible": True},
],
)
)
node = _node(design, device_id="d-1")
db_session.add(node)
await db_session.commit()
# The backup only knew "Rack", and this canvas had it hidden.
drawn = {node.id: {
"services": [],
"properties": [{"key": "Rack", "value": "A", "icon": None, "visible": False}],
}}
assert await seed_node_views(db_session, drawn=lambda: drawn) == 1
await db_session.commit()
assert node.display_view["properties"] == [
{"key": "rack", "visible": False}, # recovered, still hidden here
{"key": "ports", "visible": True}, # added since the backup, kept
]
@pytest.mark.asyncio
async def test_seeding_still_holds_back_a_service_found_since_the_backup(self, db_session):
"""Services are not treated that way: a scan's find stays off the canvas."""
design = await _design(db_session)
db_session.add(
InventoryDevice(
id="d-1", ip="10.0.0.5",
services=[
{"port": 22, "protocol": "tcp", "service_name": "ssh"},
{"port": 5001, "protocol": "tcp", "service_name": "Synology DSM HTTPS"},
],
)
)
node = _node(design, device_id="d-1")
db_session.add(node)
await db_session.commit()
drawn = {node.id: {
"services": [{"port": 22, "protocol": "tcp", "service_name": "ssh"}],
"properties": [],
}}
assert await seed_node_views(db_session, drawn=lambda: drawn) == 1
await db_session.commit()
assert node.display_view["services"] == [{"key": "22|tcp|ssh", "visible": True}]
@pytest.mark.asyncio
async def test_a_node_without_a_view_still_shows_everything(self, db_session):
"""No view — furniture, or a node an older version linked — is not "hide all"."""
device = InventoryDevice(
id="d-1", services=[{"port": 22, "protocol": "tcp", "service_name": "ssh"}]
)
node = _node(await _design(db_session), device_id="d-1")
assert hydrated_node(node, device)["services"] == [
{"port": 22, "protocol": "tcp", "service_name": "ssh"}
]
+42
View File
@@ -750,6 +750,48 @@ async def test_run_scan_merges_proxmox_row_by_mac(mem_db):
assert set(row.discovery_sources) == {"proxmox", "arp"} # both filters
@pytest.mark.asyncio
async def test_run_scan_keeps_services_the_fingerprint_cannot_see(mem_db):
"""A re-scan unions its fingerprint onto the row — it never replaces it.
Since 3.3.0 the row is the only copy of a device's services and every canvas
drawing it reads that list, so overwriting it with what nmap happened to
match would delete hand-added services everywhere at once (#347).
"""
from app.services.scanner import run_scan
run_id = _make_run_id()
async with mem_db() as session:
session.add(_make_scan_run(run_id))
session.add(InventoryDevice(
id="row-1", ip="192.168.1.60", status="approved",
discovery_source="arp", discovery_sources=["arp"],
services=[
{"port": 9000, "protocol": "tcp", "service_name": "Portainer", "path": "/#!/home"},
{"port": 22, "protocol": "tcp", "service_name": "ssh"},
],
))
await session.commit()
nmap_hosts = [{"ip": "192.168.1.60", "hostname": "docker.lan", "mac": None, "os": None,
"open_ports": [{"port": 22, "protocol": "tcp", "banner": "OpenSSH 9.2"}]}]
async with mem_db() as session:
with patch("app.services.scanner._nmap_scan", return_value=nmap_hosts), \
patch("app.services.scanner._mdns_discover", new_callable=AsyncMock, return_value=[]), \
patch("app.api.routes.status.broadcast_scan_update", new_callable=AsyncMock):
await run_scan(["192.168.1.0/24"], session, run_id)
async with mem_db() as session:
row = await session.get(InventoryDevice, "row-1")
by_port = {s["port"]: s for s in row.services}
assert by_port[9000]["service_name"] == "Portainer" # hand-added, untouched
assert by_port[9000]["path"] == "/#!/home"
assert 22 in by_port # what the scan saw is still there
assert row.status == "approved"
@pytest.mark.asyncio
async def test_run_scan_mdns_skipped_if_already_in_nmap(mem_db):
"""If nmap and mDNS both find the same IP, it should not be double-counted."""
+2 -2
View File
@@ -1,12 +1,12 @@
{
"name": "frontend",
"version": "3.3.0",
"version": "3.3.4",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "frontend",
"version": "3.3.0",
"version": "3.3.4",
"dependencies": {
"@base-ui/react": "^1.2.0",
"@dagrejs/dagre": "^2.0.4",
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "frontend",
"private": true,
"version": "3.3.0",
"version": "3.3.4",
"type": "module",
"scripts": {
"dev": "vite",
@@ -40,7 +40,9 @@ export function BaseNode({ id, data, selected, icon: typeIcon, width, height }:
const colors = resolveNodeColors(data, activeTheme)
const statusColor = theme.colors.statusColors[data.status]
const isOnline = data.status === 'online'
const services = data.services ?? []
// Hiding a service is a property of this node, not of the device: another
// canvas drawing the same device answers for itself.
const services = (data.services ?? []).filter((svc) => svc.visible !== false)
const showServices = data.custom_colors?.show_services === true
const serviceHost = data.ip ? primaryIp(data.ip) : data.hostname
@@ -0,0 +1,63 @@
/**
* A service hidden on this node stays on the device.
*
* Order and visibility are the node's, the facts are the inventory row's, so
* the same device drawn on two canvases can show two different service lists.
* The node card only draws what its own copy says is visible.
*/
import { describe, it, expect, vi } from 'vitest'
import { render, screen } from '@testing-library/react'
import { ReactFlowProvider } from '@xyflow/react'
import { Server } from 'lucide-react'
import { BaseNode } from '../BaseNode'
import type { NodeData, ServiceInfo } from '@/types'
vi.mock('@/stores/canvasStore', async () => {
const actual = await vi.importActual<typeof import('@/stores/canvasStore')>('@/stores/canvasStore')
return {
...actual,
useCanvasStore: (selector: (s: Record<string, unknown>) => unknown) =>
selector({ hideIp: false, serviceStatuses: {} }),
}
})
const ssh: ServiceInfo = { port: 22, protocol: 'tcp', service_name: 'ssh' }
const kuma: ServiceInfo = { port: 3001, protocol: 'tcp', service_name: 'Uptime Kuma' }
function renderNode(services: ServiceInfo[]) {
const data: NodeData = {
label: 'NAS',
type: 'nas',
status: 'online',
services,
// Services only render when the node is set to show them.
custom_colors: { show_services: true },
}
return render(
<ReactFlowProvider>
<BaseNode
{...({ id: 'n1', data, selected: false, icon: Server } as React.ComponentProps<typeof BaseNode>)}
/>
</ReactFlowProvider>,
)
}
describe('BaseNode services', () => {
it('draws a service that carries no visible flag', () => {
renderNode([ssh, kuma])
expect(screen.getByText('ssh')).toBeInTheDocument()
expect(screen.getByText('Uptime Kuma')).toBeInTheDocument()
})
it('leaves out one this node hid, keeping the rest', () => {
renderNode([ssh, { ...kuma, visible: false }])
expect(screen.getByText('ssh')).toBeInTheDocument()
expect(screen.queryByText('Uptime Kuma')).not.toBeInTheDocument()
})
it('draws them in the order the node carries', () => {
const { container } = renderNode([kuma, ssh])
const names = [...container.querySelectorAll('span.font-medium')].map((n) => n.textContent)
expect(names.filter((n) => n === 'ssh' || n === 'Uptime Kuma')).toEqual(['Uptime Kuma', 'ssh'])
})
})
@@ -523,7 +523,10 @@ export function InventoryDeviceModal({ device, onClose, onApprove, onHide, onIgn
<PropertyList
properties={properties}
onChange={setProperties}
visibleLabel="Show on node"
// Whether a canvas draws a property is that node's own
// answer; the flag here only decides what a node drawing
// this device from now on starts out showing.
visibleLabel="Show on new nodes"
// Hardware is a property like any other; these are the keys
// the Proxmox import and the YAML import already mint.
suggestions={['CPU Model', 'CPU Cores', 'RAM', 'Disk']}
+23 -1
View File
@@ -134,6 +134,16 @@ export function DetailPanel({ onEdit, onOpenInventory }: DetailPanelProps) {
if (openSvcModal?.index === index) setSvcModal(null)
}
// Per node, not per device: the same device drawn on two canvases can show a
// service on one and hide it on the other. Only the order and the flag are the
// node's — the service itself still belongs to the inventory row.
const handleToggleService = (index: number) => {
snapshotHistory()
updateNode(node.id, {
services: services.map((svc, i) => (i === index ? { ...svc, visible: svc.visible === false } : svc)),
})
}
const handleStartEdit = (index: number) => {
const svc = services[index]
if (!svc) return
@@ -260,6 +270,7 @@ export function DetailPanel({ onEdit, onOpenInventory }: DetailPanelProps) {
setDragSvcIndex(null)
setDragOverSvcIndex(null)
}}
onToggleVisible={() => handleToggleService(i)}
onEdit={() => handleStartEdit(i)}
onRemove={() => handleRemoveService(i)}
/>
@@ -594,7 +605,7 @@ const CATEGORY_COLORS: Record<string, string> = {
web: '#00d4ff', database: '#a855f7', monitoring: '#39d353', storage: '#e3b341', security: '#f85149', remote: '#8b949e',
}
function ServiceBadge({ svc, host, status, draggable, isDragging, isDragOver, onDragStart, onDragEnter, onDragEnd, onDrop, onEdit, onRemove }: {
function ServiceBadge({ svc, host, status, draggable, isDragging, isDragOver, onDragStart, onDragEnter, onDragEnd, onDrop, onToggleVisible, onEdit, onRemove }: {
svc: ServiceInfo
host?: string
status?: ServiceStatus
@@ -605,6 +616,7 @@ function ServiceBadge({ svc, host, status, draggable, isDragging, isDragOver, on
onDragEnter: () => void
onDragEnd: () => void
onDrop: () => void
onToggleVisible: () => void
onEdit: () => void
onRemove: () => void
}) {
@@ -615,6 +627,8 @@ function ServiceBadge({ svc, host, status, draggable, isDragging, isDragOver, on
// A live offline service overrides the category colour with red.
const color = status === 'offline' ? '#f85149' : categoryColor
const pathLabel = svc.path?.trim() ? svc.path.trim() : ''
// Absent means shown: a service only carries the flag once it has been hidden.
const shown = svc.visible !== false
return (
<div
@@ -693,6 +707,14 @@ function ServiceBadge({ svc, host, status, draggable, isDragging, isDragOver, on
<span className="w-2.5" />
)}
<button
onClick={(e) => { e.preventDefault(); e.stopPropagation(); onToggleVisible() }}
className="text-[#8b949e] hover:text-[#00d4ff] ml-0.5 cursor-pointer transition-colors"
title={shown ? 'Hide on node' : 'Show on node'}
>
{shown ? <Eye size={10} /> : <EyeOff size={10} />}
</button>
<button
onClick={(e) => { e.preventDefault(); e.stopPropagation(); onEdit() }}
className="opacity-0 group-hover:opacity-100 transition-opacity text-[#8b949e] hover:text-[#00d4ff] ml-0.5 cursor-pointer"
@@ -546,6 +546,64 @@ describe('DetailPanel', () => {
expect(payload.services.map((s: { service_name: string }) => s.service_name)).toEqual(['C', 'A', 'B'])
})
it('hides a service on this node without touching the service itself', () => {
// Visibility is the node's answer, not the device's: the rest of the
// service record goes back unchanged so other canvases keep drawing it.
const updateNode = vi.fn()
vi.mocked(canvasStore.useCanvasStore).mockImplementation(
((sel?: (s: Record<string, unknown>) => unknown) => {
const state = {
nodes: [makeNode({ services: [{ port: 3001, protocol: 'tcp' as const, service_name: 'Uptime Kuma' }] })],
selectedNodeId: 'n1',
selectedNodeIds: [],
setSelectedNode: vi.fn(),
deleteNode: vi.fn(),
updateNode,
snapshotHistory: vi.fn(),
createGroup: vi.fn(),
ungroup: vi.fn(),
setNodeSize: vi.fn(),
serviceStatuses: {},
}
return sel ? sel(state) : state
}) as unknown as typeof canvasStore.useCanvasStore,
)
render(<DetailPanel onEdit={vi.fn()} />)
fireEvent.click(screen.getByTitle('Hide on node'))
expect(updateNode.mock.calls[0][1].services).toEqual([
{ port: 3001, protocol: 'tcp', service_name: 'Uptime Kuma', visible: false },
])
})
it('shows a hidden service again', () => {
const updateNode = vi.fn()
vi.mocked(canvasStore.useCanvasStore).mockImplementation(
((sel?: (s: Record<string, unknown>) => unknown) => {
const state = {
nodes: [makeNode({
services: [{ port: 3001, protocol: 'tcp' as const, service_name: 'Uptime Kuma', visible: false }],
})],
selectedNodeId: 'n1',
selectedNodeIds: [],
setSelectedNode: vi.fn(),
deleteNode: vi.fn(),
updateNode,
snapshotHistory: vi.fn(),
createGroup: vi.fn(),
ungroup: vi.fn(),
setNodeSize: vi.fn(),
serviceStatuses: {},
}
return sel ? sel(state) : state
}) as unknown as typeof canvasStore.useCanvasStore,
)
render(<DetailPanel onEdit={vi.fn()} />)
fireEvent.click(screen.getByTitle('Show on node'))
expect(updateNode.mock.calls[0][1].services[0].visible).toBe(true)
})
it('is not draggable with a single service', () => {
setupStore({ services: [{ port: 80, protocol: 'tcp', service_name: 'A' }] })
render(<DetailPanel onEdit={vi.fn()} />)
@@ -96,3 +96,49 @@ describe('canvasStore — applyDeviceFacts', () => {
expect(changedFactFields(nodes[0].data, factsBaseline.n1)).toEqual(['label'])
})
})
describe('canvasStore — applyDeviceFacts keeps each node\'s arrangement', () => {
beforeEach(resetStore)
const ssh = { port: 22, protocol: 'tcp' as const, service_name: 'ssh' }
const kuma = { port: 3001, protocol: 'tcp' as const, service_name: 'Uptime Kuma' }
const drawnAs = (id: string, services: typeof ssh[]) =>
makeNode(id, makeNodeData({ device_id: 'd-1', label: 'NAS', services }))
it('does not unhide a service the node hid', () => {
useCanvasStore.getState().loadCanvas([drawnAs('n1', [ssh, { ...kuma, visible: false }])], [])
// The inventory sends the row's own copy, which carries no such flag.
useCanvasStore.getState().applyDeviceFacts('d-1', { services: [ssh, kuma] })
expect(useCanvasStore.getState().nodes[0].data.services).toEqual([
ssh,
{ ...kuma, visible: false },
])
})
it('keeps this node\'s order while taking the row\'s values', () => {
useCanvasStore.getState().loadCanvas([drawnAs('n1', [kuma, ssh])], [])
useCanvasStore.getState().applyDeviceFacts('d-1', {
services: [{ ...ssh, path: '/admin' }, kuma],
})
expect(useCanvasStore.getState().nodes[0].data.services).toEqual([
kuma,
{ ...ssh, path: '/admin' },
])
})
it('brings a service the row gained in hidden', () => {
useCanvasStore.getState().loadCanvas([drawnAs('n1', [ssh])], [])
useCanvasStore.getState().applyDeviceFacts('d-1', { services: [ssh, kuma] })
expect(useCanvasStore.getState().nodes[0].data.services).toEqual([
ssh,
{ ...kuma, visible: false },
])
})
it('drops one the row lost', () => {
useCanvasStore.getState().loadCanvas([drawnAs('n1', [ssh, kuma])], [])
useCanvasStore.getState().applyDeviceFacts('d-1', { services: [ssh] })
expect(useCanvasStore.getState().nodes[0].data.services).toEqual([ssh])
})
})
+16 -1
View File
@@ -15,7 +15,13 @@ import { normalizeHandle, removedHandleIds, handleCountField, sideDefault, handl
import { applyOpacity } from '@/utils/colorUtils'
import { readHideIp, writeHideIp } from '@/utils/ipDisplay'
import { CONTAINER_MODE_TYPES } from '@/utils/virtualEdgeParent'
import { changedFactFields, factsBaselineOf, factsBaselines, type FactsBaseline } from '@/utils/deviceFacts'
import {
changedFactFields,
factsBaselineOf,
factsBaselines,
listArrangedForNode,
type FactsBaseline,
} from '@/utils/deviceFacts'
type HistoryEntry = { nodes: Node<NodeData>[]; edges: Edge<EdgeData>[] }
type Clipboard = { nodes: Node<NodeData>[]; edges: Edge<EdgeData>[] }
@@ -1012,6 +1018,15 @@ export const useCanvasStore = create<CanvasState>((rawSet) => {
if (Object.keys(applied).length === 0) return n
changed = true
const data = { ...n.data, ...applied }
// The row owns the facts, this node owns how they are laid out: an
// inventory edit updates the values in place rather than replacing the
// arrangement, and a service the row just gained arrives hidden.
if (Array.isArray(applied.services)) {
data.services = listArrangedForNode(n.data.services, applied.services)
}
if (Array.isArray(applied.properties)) {
data.properties = listArrangedForNode(n.data.properties, applied.properties)
}
// Rebase only what was applied: an untouched pending edit stays reported
// as this canvas' change so the next save still writes it.
const rebased = factsBaselineOf(data)
+4
View File
@@ -97,6 +97,10 @@ export interface ServiceInfo {
* several domains. Same accepted shapes as a node `ip`/`hostname`
* (`host`, `host:port`, `https://host/…`). */
host?: string
/** Whether the node drawing this device shows the service. Per node, not per
* device — the same device on another canvas keeps its own answer. Absent
* means shown: the flag only appears once the service has been hidden. */
visible?: boolean
}
export type ServiceStatus = 'online' | 'offline' | 'unknown'
@@ -48,6 +48,35 @@ describe('changedFactFields', () => {
expect(changedFactFields(data({ properties: [] }), base)).toEqual(['properties'])
})
it('does not call hiding a service an edit to the device', () => {
const services = [
{ port: 22, protocol: 'tcp' as const, service_name: 'ssh' },
{ port: 3001, protocol: 'tcp' as const, service_name: 'Uptime Kuma' },
]
const base = factsBaselineOf(data({ services }))
const hidden = [services[0], { ...services[1], visible: false }]
// Visibility belongs to this node — pushing it as a device edit would hide
// the service on every other canvas drawing the same row.
expect(changedFactFields(data({ services: hidden }), base)).toEqual([])
})
it('does not call reordering an edit to the device', () => {
const props = [
{ key: 'Rack', value: 'A1', icon: null, visible: true },
{ key: 'Owner', value: 'me', icon: null, visible: true },
]
const base = factsBaselineOf(data({ properties: props }))
expect(changedFactFields(data({ properties: [props[1], props[0]] }), base)).toEqual([])
})
it('still reports a real edit to a list item', () => {
const props = [{ key: 'Rack', value: 'A1', icon: null, visible: true }]
const base = factsBaselineOf(data({ properties: props }))
expect(changedFactFields(data({ properties: [{ ...props[0], value: 'B2' }] }), base)).toEqual([
'properties',
])
})
it('treats every fact as changed with no baseline — a new node has nothing to revert', () => {
expect(changedFactFields(data(), undefined)).toEqual([...DEVICE_FACT_FIELDS])
})
+58 -2
View File
@@ -40,9 +40,35 @@ export type FactsBaseline = Record<string, string>
const encode = (value: unknown): string => JSON.stringify(value ?? null)
/**
* Which services and properties the node carries, ignoring how it draws them.
*
* Order and visibility belong to the node, not to the inventory row, so hiding
* a service or dragging a property into place is not an edit to the device and
* must not be reported as one — otherwise a rearranged canvas would push its
* arrangement onto every other canvas showing the same device.
*/
const keyOf = (item: Record<string, unknown>): string =>
'key' in item
? String(item.key ?? '').toLowerCase()
: `${item.port ?? ''}|${item.protocol ?? ''}|${String(item.service_name ?? '').toLowerCase()}`
const encodeFacts = (field: DeviceFactField, value: unknown): string => {
if (field !== 'services' && field !== 'properties') return encode(value)
const items = (Array.isArray(value) ? value : []).filter(
(item): item is Record<string, unknown> => typeof item === 'object' && item !== null,
)
const bare = items.map((item) => {
const rest = { ...item }
delete rest.visible
return rest
})
return encode([...bare].sort((a, b) => keyOf(a).localeCompare(keyOf(b))))
}
export function factsBaselineOf(data: Partial<NodeData>): FactsBaseline {
const out: FactsBaseline = {}
for (const field of DEVICE_FACT_FIELDS) out[field] = encode(data[field])
for (const field of DEVICE_FACT_FIELDS) out[field] = encodeFacts(field, data[field])
return out
}
@@ -64,7 +90,37 @@ export function changedFactFields(
baseline: FactsBaseline | undefined,
): DeviceFactField[] {
if (!baseline) return [...DEVICE_FACT_FIELDS]
return DEVICE_FACT_FIELDS.filter((field) => encode(data[field]) !== baseline[field])
return DEVICE_FACT_FIELDS.filter((field) => encodeFacts(field, data[field]) !== baseline[field])
}
/**
* The row's list, arranged the way this node already draws it.
*
* The facts are the row's, the order and the visibility are the node's, so an
* edit made in the Device Inventory must not reshuffle a canvas or unhide what
* it hid. Same rule the backend applies on read: what the node already lists
* keeps its place and its flag, what the row gained since is appended hidden,
* and what the row lost disappears.
*/
export function listArrangedForNode<T extends Record<string, unknown>>(
current: T[] | undefined,
incoming: T[],
): T[] {
if (!current?.length) return incoming
const seen = new Set<string>()
const by = new Map(incoming.map((item) => [keyOf(item), item]))
const out: T[] = []
for (const item of current) {
const key = keyOf(item)
const fresh = by.get(key)
if (!fresh || seen.has(key)) continue
seen.add(key)
out.push('visible' in item ? { ...fresh, visible: item.visible } : (fresh as T))
}
for (const item of incoming) {
if (!seen.has(keyOf(item))) out.push({ ...item, visible: false })
}
return out
}
/**