Compare commits

...
665 Commits
Author SHA1 Message Date
Pouzor 390a712f14 chore: bump version to 3.3.5
ha-relevant: no
2026-08-21 12:56:17 +02:00
Pouzor 9daea4c4d0 fix(scan): keep an edit, a discovery source and one name for a failed run
Three findings from the PR review.

A finishing deep rescan threw away an edit in progress. The reset effect in
InventoryDeviceModal was keyed on the `device` object, and the parent hands
down a fresh one whenever the row is refreshed — including from the poll's
own onSaved. Same device, new object, so the effect reset the form and left
edit mode, minutes into a scan the user was waiting on. It is keyed on the
id now: a refreshed row is not a different device, and only a different
device is a reason to throw the form away. The fresh row still reaches the
canvas and the grid — gating that behind edit mode would discard the scan
result instead, and a save unions the services server-side anyway.

A rescan tagged every device it touched as "arp"-discovered, so a Proxmox
guest, a rack mount or a hand-added host started answering the network
source filter. It carries its own source through instead.

The two background wrappers wrote status "failed" where the scanners write
"error" for the same condition. Harmonized on "error" — the one Scan
History filters and colours; "failed" showed up unlabelled. The frontend
union keeps 'failed' for rows already recorded.

That last one meant updating an existing assertion in test_scan_run.py: it
encoded the old spelling.

ha-relevant: yes
2026-08-21 12:49:48 +02:00
Pouzor 2e642815f4 fix(scan): stop a scan from repainting hand-picked service icons
merge_services did {**existing, **incoming}, so the fingerprint's guess at
an icon overwrote the one the user chose — and on a port no signature
covers it wrote None, clearing it outright. Since 3.3.0 the inventory row
is the only copy of a device's services, so every "Scan network" repainted
the service on every canvas drawing that device at once.

The scanner now merges with discovered=True: it still adds services and
refreshes what it knows, but leaves an established icon and category alone.
A user edit from the modal or a canvas changes them as before.

Blank incoming values no longer clear established ones either, on both
paths — an absent field is silence, not a reset. Same rule merge_properties
already follows.

ha-relevant: yes
2026-08-21 12:49:48 +02:00
Pouzor c6d6b525ea feat(scan): choose the port range before a deep scan
The Deep scan link on a device now opens a small dialog instead of firing
straight away. It is prefilled with the full 1-65535 range — that is still
the point of the feature — but a user who knows where a service lives can
narrow it and get an answer in seconds instead of minutes.

The dialog takes an nmap-style spec: a port, a range, or a comma list
(80,443,8000-9000). It shows the port count live, refuses to start on
something nmap could not use, and carries three presets (all / 1-1024 /
1-10000).

Backend:
- _parse_port_spec / _port_chunks generalize the slicing that used to be
  full-range only. Ranges are merged before slicing, so an overlapping
  spec is never scanned twice, and small ranges are packed into one nmap
  call instead of one call each. _deep_port_chunks still yields the same
  eight slices as before.
- run_device_scan takes ports=; it wins over full_ports.
- The retry-free flags (--max-retries 0 --min-rate 2000) now key on the
  total port count rather than on full_ports. They pay for themselves over
  thousands of ports on a lossy host; over a handful they only cost
  accuracy.
- RescanDeviceRequest.ports validates the spec — 422 rather than handing
  nmap a bad -p. Blank means the full sweep.

Frontend utils/portSpec.ts mirrors the backend parser so a typo is caught
before the request; the backend validates again because it is the one
calling nmap.

The existing rescan tests now go through the dialog: the click path
changed, so the start is two steps.

ha-relevant: yes
2026-08-21 12:49:48 +02:00
Pouzor fbb660504e fix(scan): slice the deep rescan instead of timing out the host
A deep rescan of a slow host came back with nothing at all: the run took
its full 600s ceiling and the device's services were unchanged, so a
service deleted by hand was never rediscovered.

nmap answers --host-timeout with "Skipping host <ip> due to host timeout"
and discards every port it had already found — the ceiling turned a slow
scan into one that reports nothing. What costs the time is a host that
drops packets: 8188 of 8192 ports filtered, each waiting out its probe.

- No --host-timeout on the deep discovery pass, ever.
- The full range runs as 8 slices of 8192 ports, one nmap call each,
  unioning the open ports. A slice that overruns costs its own ports, not
  all of them, and the loop has somewhere to notice a stop request.
- `scanner_deep_host_timeout` is now a total budget checked between
  slices (default 2700s), not an nmap flag. The first slice always runs.
- A partial sweep is reported rather than passed off as complete: the run
  finishes `done` carrying "Scanned 3/8 port ranges …", and the modal
  toasts a warning instead of success.
- Deep slices use --max-retries 0 --min-rate 2000. Measured against a
  dropping host, 8192 ports took 329s at --max-retries 1 and 164s at 0,
  finding the same ports; capping the RTT changed nothing. The range scan
  keeps nmap's default retries on its curated port list.

ha-relevant: yes
2026-08-21 12:49:48 +02:00
Pouzor 1be96d1045 feat(scan): deep-rescan one device from the inventory detail
Devices added before the scanner knew a service showed an empty Services
section with no way to refresh it (#350). The detail modal now starts a
full-port scan of that single device.

- `process_host` lifted out of `run_scan` so the range scan and the new
  single-device scan share the same match / merge / dedupe rules — a
  rescan unions services, it never replaces what the user added by hand.
- `run_device_scan`: no ping sweep, no mDNS, straight to the phase-2 nmap
  pass on the device IP over all 65535 TCP ports.
- `POST /scan/pending/{id}/rescan` records a normal ScanRun
  (`kind=device`, `ranges=["<ip>/32"]`), so stop, progress and Scan
  History work unchanged. One run per device at a time — a second request
  while the first is scanning is a 409. 404 unknown, 409 no-IP or hidden.
- `GET /scan/runs/{id}` so a caller can poll the run it started.
- Deep scan button in the Services section of the device detail, hidden
  without an IP and for Zigbee. Swaps to a stop control while running,
  folds the fresh services back in on completion (never over an edit in
  progress).

ha-relevant: yes
2026-08-21 12:49:48 +02:00
Pouzor 9bc02d61c2 fix(canvas): give a valueless property label the full node width
The property line caps its label at max-w-15 so a long key cannot crowd
out the value drawn beside it. When the value is empty there is nothing
to protect, but the cap still applied, truncating the label against
empty space.

Drop the cap when the value is blank and let the label truncate against
the node width instead. Same fix in BaseNode and ProxmoxGroupNode.

Closes #361

ha-relevant: yes
2026-08-17 20:26:29 +02:00
Pouzor 21a64e52ab fix(auth): accept the app's own origin in the OIDC CSRF check
A same-origin deployment behind a reverse proxy needs no CORS, so
CORS_ORIGINS is left at its localhost default — but browsers still send
Origin on unsafe methods, and OIDCCSRFMiddleware validated it against
CORS_ORIGINS alone. Every POST/PUT/DELETE came back 403 while reads
worked, so creating a canvas failed with no usable error.

The OIDC callback URL is served by this app, so the origin of
OIDC_REDIRECT_URI is the app's own and is always a legitimate CSRF
origin. origin_is_allowed now accepts it in addition to CORS_ORIGINS,
and settings validation warns at startup when CORS_ORIGINS omits it so
the misconfiguration is visible rather than silent.

Fixes #356

ha-relevant: no
2026-08-17 20:13:04 +02:00
Pouzor ad0b1b427d chore: bump version to 3.3.4
ha-relevant: no
2026-08-17 19:19:25 +02:00
Pouzor 4c460cec1c fix(scan): stop a re-scan from deleting a device's hand-added services
Since 3.3.0 the inventory row is the only copy of a device's services and
every canvas drawing it reads that list, but the scanner still assigned
`keep.services = fingerprint_ports(...)` — the pre-split behaviour, when a
node held its own copy. One re-scan therefore deleted every service the user
added by hand, on every canvas at once, and brought back the ones they had
deleted. It unions now, like every other writer of that field.

Two more things #347 turned up:

* A node whose view matches nothing the row still holds drew nothing at all —
  the row had been replaced under it, so every key was gone and every key was
  new, and `apply_view` hid the lot. Such a view says nothing about the list
  that replaced it, so it is treated as no view: the row is drawn. An empty
  view is untouched, being a real answer ("this canvas draws none of them").

* The 3.3.3 view seed recovers a node's arrangement from the pre-3.3.0 backup,
  which is 3.2.0-era and cannot know about a property added afterwards. On
  3.3.0-3.3.2 the row was the only place to add one and every canvas drew it,
  so seeding strictly from the backup took it off all of them at once. Those
  are appended visible, keeping the recovered order and hidden flags for
  everything the backup does know. Services keep the strict recovery: holding
  back what a scan fingerprinted is the whole point of the view.

ha-relevant: maybe
2026-08-17 19:05:13 +02:00
Pouzor 1aa88de491 chore: bump version to 3.3.3
ha-relevant: no
2026-08-17 14:09:01 +02:00
Pouzor d0679ad72b fix(inventory): stop the view seed from re-reading the backup every boot
The seed selected any node with a `device_id` and no view. Deleting a device
leaves that column dangling — SQLite runs with foreign keys off, so the
`ON DELETE SET NULL` never fires on an existing table — and such a node can be
seeded from nothing, so it stayed selected. The query matched it again on every
later start, re-opening the pre-upgrade backup and logging a recovery that
recovered nothing.

Narrowing the query to nodes whose row actually exists restores the invariant
the seed is documented to have: it runs on one boot, and never opens a file
again.

ha-relevant: no
2026-08-17 14:06:09 +02:00
Pouzor 5df70ce128 feat(canvas): give each node its own view of a device's services and properties
Since 3.3.0 the inventory row owns a device's services and properties, and
every node drawing that device rendered the row wholesale. One row shared by
several canvases meant one rendering: a service the scanner fingerprinted
appeared on every canvas at once (users reported Uptime Kuma and Synology DSM
on hosts running neither — both are port-only signatures), and a property added
on one schematic showed up on all the others.

Order and visibility are presentation, so they move to the node. `display_view`
records, per node, which of the row's services and properties it draws and in
what order, keyed by `port|protocol|name` and by lowercased property key so the
view survives an edit to the fact itself. The facts stay on the row: hiding is
per node, deleting is still device-wide.

An item the view does not list is reported hidden rather than dropped, so what
a scan finds next is one toggle away instead of pushed onto every canvas.

The wire shape is unchanged. A client already sends its lists in display order
with their `visible` flags, so the view is read back out of them rather than
asking for a second field, and `visible` is only stamped when something is
hidden.

Upgrades keep what each canvas showed:

* from 3.2.0, the backfill seeds each node's view from its own legacy columns
  before they are dropped;
* from 3.3.0-3.3.2 those columns are gone and the row holds the union of every
  canvas, so the layout is recovered from the backup `_backup_db` took before
  the 3.3.0 migration — the newest one whose `nodes` table still has the
  columns, read read-only and matched by node id;
* with no usable backup the row is the seed, so every canvas keeps showing
  exactly what it shows today and only later additions are held back.

Refs #347

ha-relevant: maybe
2026-08-17 14:06:09 +02:00
Pouzor a58725c2ee chore: bump version to 3.3.2
ha-relevant: no
2026-08-17 11:37:54 +02:00
Pouzor 466df59b0c fix(inventory): finish the 3.3.0 backfill on databases carrying timestamps
3.3.1 kept `nodes` writable when the backfill failed, but the backfill still
failed — so a canvas upgraded from 3.2.0 shows every device stripped of its ip,
services, hostname, notes and hardware. The facts are intact in the legacy
columns; nothing was ever reading them across.

The backfill reads those columns with raw SQL, so SQLite hands `last_seen` and
`last_scan` back as text. Writing text into a DateTime column raises
`TypeError: SQLite DateTime type only accepts Python datetime and date objects`,
which is neither IntegrityError nor OperationalError — so it went straight past
the per-node savepoint and aborted the whole run, exactly as before.

- Parse the legacy timestamps back into datetimes, the way `_decode_json`
  already handles the JSON columns. An unparseable stamp becomes None rather
  than an error: the status checker refreshes both within a minute.
- Catch every exception per node, not two chosen classes. Picking the exception
  types was the defect; a node that fails for any reason must cost only itself.

The same run would also have destroyed data. A canvas saved while an earlier
migration was stuck minted a blank inventory row from a UI that had no facts to
show and linked the node to it. The backfill only looked at unlinked nodes, so
it skipped that one, counted the canvas migrated, and dropped the columns
holding its only copy of the ip and services.

- Read linked nodes too, and fill only what their row is missing. A row that
  already holds a fact is never overwritten, so an edit made after the migration
  survives, and the pass stays a no-op once everything has moved across.

Verified end to end against a rebuilt 3.2.0 database stuck the way the reports
describe: one boot restores ip, services, hostname, mac, notes, check method,
hardware, properties, status and last_seen, then drops the legacy columns.

Refs #347, #348, #351

ha-relevant: no
2026-08-17 11:36:32 +02:00
Pouzor 7eec0b3f9a chore: bump version to 3.3.1
ha-relevant: no
2026-08-17 01:58:26 +02:00
Pouzor b76519b0dd fix(db): keep nodes insertable when the 3.3.0 backfill cannot finish
Upgrading 3.2.0 -> 3.3.0 could leave a database where approving a device —
or creating any node — failed with `NOT NULL constraint failed: nodes.status`.

The 3.3.0 migration moves the device facts off `nodes` onto `device_inventory`
and then drops the columns. The drop is skipped when a node is still unlinked,
because those columns are the only remaining copy of its facts. But a 3.2.0
`nodes` declares `status`, `services`, `properties` and `show_hardware` NOT NULL
with no server-side default, and the 3.3.0 model no longer writes them — so a
skipped drop bricks every later INSERT.

It skipped because one node killed the whole backfill: the loop shared a single
session, so an IntegrityError raised by autoflush lost every link made before it.

- Give each node its own savepoint. A node whose merge violates a constraint is
  skipped and logged on its own; the rest still link. `skipped` joins the stats
  reported at boot.
- Relax NOT NULL on the retained legacy columns when the drop is skipped, so the
  database stays writable while later boots retry the backfill. Values are kept;
  only the constraint goes. Guarded on the PRAGMA notnull flag, so it runs once.
- Share one `_rebuild_nodes` between the drop and the relax: it restores
  `PRAGMA foreign_keys = ON` in a finally (the connection returns to the pool)
  and clears a `nodes_new` left by a failed attempt.
- Match IEEE addresses case-insensitively, and never write one another inventory
  row already owns — `device_inventory.ieee_address` is UNIQUE, and a duplicate
  is one way the backfill was raising in the first place.

Tests build the real v3.2.0 schema from the tag: the existing legacy-migration
fixture declares `status` nullable, which is why this reached a release.

Fixes #351

ha-relevant: no
2026-08-17 01:55:51 +02:00
Pouzor a2870a6e29 docs(changelog): match the house style for contributor credit
ha-relevant: no
2026-08-15 02:48:06 +02:00
Pouzor d6308a5573 chore: bump version to 3.3.0
ha-relevant: no
2026-08-15 02:39:49 +02:00
Pouzor fafbaeffcd feat(sidebar): link the version row to the public changelog
ha-relevant: maybe
2026-08-15 02:32:40 +02:00
Pouzor 824aa5d044 docs(install): document the pre-built GHCR images
The quick starts already pull ready-made images through install.sh, but
nothing said so — issue #310 asked for something that already ships.

Add a "Pre-built Docker images" section to INSTALLATION.md listing the
four GHCR images, their tag scheme and a manual docker-compose.prebuilt.yml
setup, and note that the root docker-compose.yml builds from source.

Closes #310

ha-relevant: no
2026-08-15 02:18:57 +02:00
Pouzor 416b5fd1cb feat(properties): make the property value optional
A property may now carry a label alone: the add and edit forms require
only the key, and every surface that prints one — the badge list, node
and Proxmox group renderers, rack cable annotations — drops the "· value"
separator when the value is empty.

ha-relevant: yes
2026-08-15 02:10:02 +02:00
Pouzor fcc2927b69 feat(canvas): let a zone parent the nodes dropped inside it
Dropping a node onto a groupRect zone now asks to add it to the zone and
sets a real React Flow parentId, so moving the zone moves its contents.

Zone children are deliberately not extent-clamped, unlike group and
container children: a zone has no side panel to release a child from, so
dragging the node back out of the zone is what detaches it.

Deleting a zone now releases its children onto the canvas in absolute
coordinates instead of cascade-deleting them; groups and containers keep
the cascade. Collapse counts and hides both parented and merely
overlapping nodes, and the deserializer restores zone parenting on load.

ha-relevant: yes
2026-08-14 20:09:11 +02:00
Pouzor be46c21dd4 fix(deps): bump undici and ip-address to patch high severity advisories
Resolves the two high Dependabot alerts on the frontend lockfile:

- undici 7.28.0 -> 7.29.0 (transitive via jsdom): cross-user information
  disclosure and parse-time crash via degenerate private cache directives
- ip-address 10.2.0 -> 10.5.0 (transitive via shadcn ->
  @modelcontextprotocol/sdk -> express-rate-limit): leading-zero octets
  decoded as decimal, allowing SSRF and trust-boundary bypass

Also closes the moderate advisories on the same two packages. Lockfile only,
both are dev dependencies.

ha-relevant: no
2026-08-14 18:36:36 +02:00
Pouzor 1c9d5791b7 fix(inventory): resolve the remaining PR review findings
Three unrelated fixes from the review of this branch.

`create_pending` merging into a hidden row left it hidden, so adding a device
by hand whose IP collides with one hidden weeks ago answered 201 while nothing
appeared in the inventory — the add read as a no-op. An explicit add outranks
the earlier hide, exactly like restore. Only `hidden` is lifted; an approved
row is not walked back down the lifecycle.

Standalone stored a node's live reachability in the inventory row's `status`,
which is the pending/approved/hidden lifecycle — the backend keeps the two
apart as `status` vs `status_live`. Reachability now goes to `status_live`, and
`normalizeLifecycle` repairs rows already written the old way: the node still
reads its status on load, and the next save rewrites the row correctly.

Status checks stay device-scoped and keep covering a device no canvas draws.
That is deliberate — the inventory row is what's monitored, so a rack mount or
an inventory-only entry reports state without being drawn anywhere, and
deleting a node no longer silently stops monitoring the host. `hide`, or
clearing the check method, is what ends the checks. Documented and pinned with
tests rather than changed.

ha-relevant: maybe
2026-08-14 18:02:02 +02:00
Pouzor 0c7fd3c127 fix(canvas): stop a canvas save from reverting a device edited elsewhere
A canvas node carries a full copy of its Device Inventory row, hydrated when
the canvas loads. The save routed all of it back, so a save made for nothing
but a moved node rewrote the row from a snapshot that could be hours old —
silently reverting an edit made meanwhile in the inventory modal, on another
canvas, or by the scanner.

Diffing the payload against the row server-side cannot fix this: it can't tell
"I edited this" from "the row moved on since I loaded it". Only the client
holds the baseline.

- canvasStore keeps `factsBaseline` — the device facts as received — set on
  load, refreshed on save, rebased per field by `applyDeviceFacts`.
- `serializeNode` sends `changed_facts`: what this canvas actually edited.
  `link_facts(changed_fields=)` writes nothing outside that list, even where
  the values differ. Absent (older client, YAML import, MCP) keeps the previous
  full-write behaviour.
- `changed_facts()` additionally drops facts already equal to the row, so a
  no-op save writes nothing. Identity matching still uses the full payload.
- Live fields (status, last_seen…) bypass the filter — the status checker owns
  reachability, and a save only ever fills a row never checked.

An inventory edit also lands on the canvases already on screen:
`applyDeviceFacts` pushes the saved row onto every node drawing it, without
marking the canvas unsaved. A fact the canvas has edited but not saved is left
alone — work in progress wins locally and still saves.

ha-relevant: maybe
2026-08-14 18:02:02 +02:00
Pouzor 19758b2ba1 Revert "chore: bump version to 3.3.0"
This reverts commit 779e09ecb8.

The bump does not belong to this PR. 3.3.0 ships several PRs, so the version
files and the release notes are raised once, in a release PR of their own,
covering the whole diff against main.

ha-relevant: no
2026-08-14 18:02:02 +02:00
Pouzor 1f6812ef2a chore: bump version to 3.3.0
The Device Inventory row becomes the source of a device's facts, and the
device columns are dropped from `nodes` in a one-way migration on first start,
so the release notes lead with a Breaking section: what moved, what the
migration does, when it refuses to drop, and that `homelab.db.back-3.3.0` —
written before any migration runs — is the way back.

ha-relevant: no
2026-08-14 18:02:02 +02:00
Pouzor 9db8b2a7a9 test(standalone): cover a canvas stored in the pre-split shape
An install that predates the split holds canvases whose nodes carry the device
facts inline, with no `devices` key at all. Reading one already worked; nothing
pinned that the next save converts it in place rather than writing the old
shape back.

Covers the load-then-save round trip with every fact intact, the second save
not minting a duplicate row, furniture staying out of the conversion, and the
legacy bare-key canvas carried through `ensureSeed`.

ha-relevant: no
2026-08-14 18:02:02 +02:00
Pouzor d40f73b85c fix(mcp): read a canvas node's facts where the API puts them
`_slim_canvas` filtered `n["data"]`, the React Flow shape the frontend store
holds. `GET /api/v1/canvas` has never sent that: a node is reported flat, so
the filter matched nothing and `get_canvas` returned an id and a node type per
node — no label, no address, no services. The three tests covering it fed a
hand-built nested payload, so they passed against a shape the backend does not
produce.

Fold `data` over the node and read both, so either form slims the same way.
`type` leaves `NODE_KEEP` — flat it is the node type, not a device fact, and it
is already reported as `node_type`; `parentId` becomes `parent_id`, the key the
API uses, normalized from either spelling.

The new tests are built on a full `NodeResponse` copied from a real response,
and the backend now pins that wire shape from its side so the two move together.

ha-relevant: no
2026-08-14 18:02:02 +02:00
Pouzor 71e29362aa fix(inventory): stop a partial node update from blanking the other list
`merge_facts_into_device` applied one `replace_lists` flag to properties
and services alike, taking each wholesale from `facts`. But a PATCH only
carries the keys the client sent, so `{"properties": [...]}` replaced
services with `[]` — losing them for every canvas drawing the device and
stopping the scheduler from service-checking it. Symmetric the other way.

Gate the replace per list: a list absent from `facts` falls through to the
merge, which is a no-op against `None`. An explicit `[]` still clears, and
the canvas save path is unchanged since it always sends both keys.

ha-relevant: no
2026-08-14 18:02:02 +02:00
Pouzor 7a5762e1a0 fix(inventory): read a device's curated type on its inventory card
A device documented straight on a canvas carries a `label` and a `type`
and no discovery guess at all — no scan ever saw it. The card read
`suggested_type` alone, so a printer drawn on a canvas fell back to
'generic': unnamed, filed under no type, and drawn with a plain circle.

Two faults compounded:

- The card decided the kind from `suggested_type`, ignoring the curated
  `type`. That also meant retyping a scanned device in the detail modal
  changed nothing here, and approving it still built a node of the type
  the scanner had guessed.
- It carried its own table of eleven icons, unrelated to the ~40
  NodeType values, so anything outside that list drew a circle even with
  the right type in hand.

`deviceType()` now resolves `type` before `suggested_type` for the icon,
the badge, the type filter and both approve paths, and the icons come
from the shared `NODE_TYPE_DEFAULT_ICONS` — one map, the same one the
canvas and the detail modal use. `deviceLabel` prefers the curated
`label` for the same reason, otherwise a canvas-created device shows as
its IP.

ha-relevant: maybe
2026-08-14 18:02:02 +02:00
Pouzor be2c5ce7e2 feat(inventory): group the device sheet by what a reader is after
The services sat in the curation column, beside the properties, while the
addresses that reach them were two columns away. They move under
Identity: what answers on a host belongs with how to reach it, and the
curation column now carries the properties alone.

Every section heading takes an icon — identity, services, monitoring,
activity, notes, properties, make & model — so a column is scannable
without reading each title. `Section` requires it, so a new section
cannot be added without one. The properties editor keeps its own plain
heading: it comes from the shared `PropertyList`, which the rack cable
panel also renders.

The three view columns carry a data-testid, which is what lets the
layout be asserted rather than eyeballed.

ha-relevant: maybe
2026-08-14 18:02:02 +02:00
Pouzor b4b22cc68f feat(inventory): make the device detail modal readable
The detail modal was a narrow column: a 15px icon, eighteen key-value
rows in one grey block, and a form that grew the dialog to `max-w-lg`
while the header and the buttons scrolled away with the content.

It reads as a device sheet now, following NodeModal's shape:

- One wide dialog with three fixed regions — header, scrolling body,
  footer — so the title and the actions stay put.
- The header identifies the row at a glance: a type-coloured icon tile
  (the accent the node wears on the canvas), then a badge per discovery
  source, the type, live reachability with a dot, and how many canvases
  draw the device.
- Three columns instead of one: identity, operations, curation. A
  section stays put when it has nothing to show and says why, rather
  than disappearing and leaving the reader unsure whether the field is
  absent or unsupported.
- Timestamps read relative with the exact value on hover, matching the
  inventory cards. IP / hostname / MAC / IEEE offer a copy button.
- The form uses the same control sizing as the canvas modals, and gains
  inputs for `friendly_name` and `device_subtype` — both were already
  submitted, so they could not be corrected here.

Hardware is a property like any other, so the modal no longer edits it:
nothing draws `cpu_*` / `ram_gb` / `disk_gb` / `show_hardware`, and the
CPU / RAM / Disk keys are property suggestions instead. The columns are
still sent back untouched — Proxmox and the YAML import write them and
the YAML export reads them.

Two assertions in the edit tests covered the removed section: the
curated-fields test drops the hardware line, and the numeric-payload
test stops editing a field that no longer exists while still checking
the values leave as numbers.

ha-relevant: maybe
2026-08-14 18:02:02 +02:00
Pouzor 21465eee65 fix(canvas): import YAML hardware specs as properties
A node draws its `properties`; `cpu_model` / `cpu_count` / `ram_gb` /
`disk_gb` are inventory data that nothing renders — BaseNode reads them
only when `properties` is absent, which the API never allows (it returns
`device.properties or []`, and the model defaults to a list).

The YAML import filled those fields and set `show_hardware`, so an
imported node did show its specs — until the first save, when the
serializer sent `properties: []`, the fallback died and the specs
silently vanished.

It now mints the same four properties the 3.x migration and the Proxmox
import already create (CPU Model, CPU Cores, RAM, Disk), visible, so the
specs survive the save. The structured fields stay written: the YAML
export reads them back.

`show_hardware` is no longer set — it drives nothing.

The two assertions that covered the old behaviour move with it: the
scalar-field test drops `show_hardware`, and "sets show_hardware only
when hardware fields present" becomes three tests over the minted
properties.

ha-relevant: yes
2026-08-14 18:02:02 +02:00
Pouzor 9b77e43ca1 feat(inventory): drop the device columns from nodes
Completes the split: `nodes` now holds only how a device is drawn on one
canvas, and every device fact reaches the API from the inventory row.

- The device columns are removed from `nodes` (SQLite table rebuild, the
  same shape as the existing device_inventory and canvas_state rebuilds).
  The drop is skipped, and logged, while any non-furniture node is still
  unlinked: those columns are the last copy of that node's facts. The
  backfill therefore reads them with raw SQL — by the time it runs, the
  model no longer declares them.
- The status checker iterates devices, not nodes: one check per device
  however many canvases draw it, writing `status_live` / `last_seen` /
  `response_time_ms` on the row. `/ws/status` messages carry `device_id`
  and the node ids they light up. Hidden devices are not probed.
- Readers repointed: scanner (last_scan lands on the row), proxmox (its
  node tier collapses into the inventory tier, keeping only the cluster
  handles), zigbee/zwave (one property refresh serves every canvas), rack
  inventory, liveview, stats, node dedupe.
- `POST /scan/pending` merges into the row that already describes the host
  instead of minting a second one — one device is one row, whichever way it
  was documented.
- Standalone keeps parity: the canvas blob gains `devices`, split on save
  and hydrated on load. A blob written before the split still reads.

The rack inventory had a related bug: a mount that names a node explicitly
printed the mount's device rather than the pinned node's. It now reads the
node's own row.

Tests that built a node with device columns are ported to the link; where a
behaviour genuinely moved (properties refresh once on the row, last_scan is
the device's) the assertion moved with it rather than being dropped.

ha-relevant: yes
2026-08-14 18:02:02 +02:00
Pouzor e53197d2cb feat(inventory): make the inventory row the source of a node's facts
A device drawn on three canvases was three independent copies of the same
facts. Point every node at the Device Inventory row it draws, and let that
row own what the device *is* — the node keeps only how it is drawn.

- nodes.device_id -> device_inventory.id, ON DELETE SET NULL. NULL for
  canvas furniture (group / groupRect / text), which describes nothing
  physical. Deleting a node never deletes the row.
- services/inventory_sync holds the shared rules: matching by ieee > ip >
  mac (per token, so 10.0.0.4 never matches 10.0.0.40), a property union on
  key, a service union on (port, protocol, name), and the backfill that
  links every pre-existing node.
- The backfill is non-destructive by construction: it writes device_id and
  fills the row, and deletes nothing. Nodes are visited oldest-edit-first,
  so where two canvases disagree on a scalar the most recently edited wins,
  while properties and services stay unioned — nothing any canvas recorded
  is lost. A second boot finds nothing to do.
- The wire shape does not change: GET /canvas hydrates the device fields
  from the row, and a save routes them back to it. Editing a node's IP on
  one canvas now shows on every other canvas holding that device.
- approve / bulk-approve set device_id instead of owning a copy, and a new
  canvas node joins (or mints) its row. Rows minted this way are tagged
  with a new `canvas` discovery source and get their own inventory filter.
- DetailPanel offers "Open in inventory"; standalone, which has no
  inventory, is not offered it.

test_racks' "reports what the canvas node knows" seeded a second inventory
row for a host that already had one — a state a node create can no longer
produce. Its seeding order is swapped so the node links to the row; every
assertion is unchanged.

ha-relevant: yes
2026-08-14 18:02:02 +02:00
Pouzor 4393ac92c6 feat(inventory): show and edit a device from the inventory
The Device Inventory was a read-only discovery log: rows arrived from a
scan or an import and went stale. Everything a user curates — properties,
services, notes, hardware, check method — could only be edited on a canvas
node, and never came back.

Give the inventory row the fields a node carries and a way to write them:

- device_inventory gains label, type, notes, cpu_count/cpu_model/ram_gb/
  disk_gb, show_hardware, check_method/check_target, last_seen, last_scan,
  response_time_ms and updated_at. Live reachability goes in a new
  status_live: `status` already means the pending/approved/hidden lifecycle
  and the two must not be conflated.
- PATCH /api/v1/scan/pending/{id} applies only what the client sends, so
  editing one field never clears the rest. Lifecycle and discovery
  bookkeeping stay owned by the approve/hide routes and the importers.
- InventoryDeviceCreate carries the same fields, so a hand-made entry needs
  no create-then-PATCH round trip.
- InventoryDeviceModal becomes show *and* edit, reusing the canvas editors
  (PropertyList, ServiceModal) rather than growing a second implementation.
- InventoryEntry moves to types/index.ts, its home; the modal re-exports it
  so existing call sites are untouched. NODE_TYPE_GROUPS moves to
  utils/nodeTypeGroups so both type pickers share one vocabulary.

ha-relevant: maybe
2026-08-14 18:02:02 +02:00
Pouzor 59079d90ca fix(canvas): seed a group child near its group and refuse nested groups
Follow-ups to the grouping fix. `handleAddNode` still gated its
"position near the parent" branch on `container_mode`, so a node added
straight into a group got a viewport-centred coordinate that `addNode`
then made group-relative — it landed far outside the box.

`isValidParentNode` also accepted a group or a zone as the child of a
group, which the drag-onto-a-group path refuses; both now agree.

ha-relevant: yes
2026-08-13 19:16:47 +02:00
Pouzor 26e18dafdd fix(canvas): keep a node inside its group when editing it
Editing any field of a grouped node dropped it out of the group on the
next save. NodeModal validated the parent against the type rules
(proxmox/vm/lxc/docker_host) or `container_mode`, and a `group` node is
neither — so it cleared `parent_id` on submit, `updateNode` detached the
node and the save persisted `parent_id: null`.

A shared `isValidParentNode` now treats a group as a valid parent for
any child type; the type rules only govern container nesting. Two
siblings of the same bug go with it: `updateNode` and `addNode` only
nested under a `container_mode` parent, so assigning or adding a node to
a group left it visually unparented until the next reload.

ha-relevant: yes
2026-08-13 19:16:47 +02:00
Pouzor c796cb39bd fix(services): split a comma-listed host override and type the wire host
The frontend takes the first host of a comma-separated override
(`splitFirstHost`), the backend took the whole string — so a two-host
override checked a hostname the UI never links to. `_parse_override`
now splits the same way.

The WebSocket service-status entry also declares `host` (nullable, as
the backend sends it), so the overlay key stops relying on an
undeclared field.

ha-relevant: yes
2026-08-13 19:16:47 +02:00
Pouzor a93b0408b8 fix(services): key and check a host-overridden service by its own host
Follow-up to the per-service host override: the status checker still
probed the node host, and the live overlay keyed services on
node/port/protocol only — so several vhosts sharing port 443 all read
one status. `check_service` now resolves the override (scheme, port,
IPv6 literal), echoes it back in the payload, and `serviceStatusKey`
takes the host as part of the key.

Also guard `new URL()` in `getServiceUrl`: a hand-typed "https://"
threw mid-render instead of resolving to no URL.

ha-relevant: yes
2026-08-13 19:16:47 +02:00
Pouzor e40ac25b20 feat(services): allow a per-service host override
A node can serve several domains, so a service now carries an optional
`host` that overrides the node ip/hostname when building its URL. Empty
falls back to the node host, and the override accepts the same shapes
(`host`, `host:port`, `https://host/…`). The canvas link and the detail
panel badge both resolve through `getServiceUrl`, so they follow it.

Closes #332

ha-relevant: yes
2026-08-13 19:16:47 +02:00
Pouzor e93aef3871 fix(canvas): show alignment guides for nodes inside groups and containers
Alignment guides and snapping were skipped for any node with a parentId, so
dragging a node inside a group or a container_mode node (Proxmox, docker host)
gave no visual help at all.

Boxes are now expressed in absolute canvas coordinates by walking the parentId
chain, so a child aligns against its siblings, its parent group, and top-level
nodes outside the group. A node whose ancestor is also being dragged follows
that ancestor: it is excluded from the snap (its parent-relative position would
otherwise be shifted twice) and from the candidate set.

Fixes #328

ha-relevant: yes
2026-08-13 00:58:58 +02:00
Pouzor 79b682148e feat(nodes): allow 0 connection points on top and bottom
Top and bottom were floored at 1 while left and right could go to 0.
MIN_HANDLES is now 0 for every side; sideDefault keeps its role as the
value used when a node carries no explicit count (1 for top/bottom,
0 for left/right), so existing canvases are unchanged.

YAML round-trip: export writes a count that differs from the side
default (not only one above it), and import tests for a number rather
than truthiness, so an explicit 0 survives.

ha-relevant: yes
2026-08-13 00:11:25 +02:00
Pouzor 5040b70073 feat(nodes): add KVM switch as a device type
Adds `kvm` to NodeType, with a label, a MonitorCog icon, a per-theme
accent and a registered node component, plus the Hardware group of the
node and custom-style pickers and the MCP NODE_TYPES list.

Scanner autodetect: port-agnostic signatures for PiKVM, TinyPilot,
JetKVM and NanoKVM, matched on the HTTP probe's title/headers, and `kvm`
placed ahead of `server` in the suggest_node_type priority list.

suggest_node_type called match_port, which drops the HTTP-probe signals,
so no `port: null` signature could influence a suggested type — Unraid,
OpenMediaVault and Cockpit included. It now calls match_service with the
probe signals, which is what makes the KVM signatures reachable.

Closes #314

ha-relevant: yes
2026-08-12 23:15:43 +02:00
1343391b0d fix: apply the status check interval to the running scheduler
POST /api/v1/settings persisted the new status check interval and echoed it
back, but never rescheduled the running job: reschedule_status_checks() is
defined in app/core/scheduler.py and has no caller anywhere. The UI and
scan_config.json show the new value while the scheduler keeps firing at the
old one until the backend restarts.

On our install this went unnoticed for three weeks: 3600s was configured but
30s stayed in effect, so ~120 nodes were pinged 120x more often than intended
with no visible symptom.

Also add ge=10 to interval_seconds, mirroring the floor already enforced by
reschedule_status_checks(). Without it a value below 10 is written to
scan_config.json first and only then raises, which surfaces as a 500 and
leaves the invalid value to be reloaded on the next boot.

Both new tests fail without the fix: the first with AttributeError (the route
does not import the function), the second with 200 instead of 422.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 11:02:12 +02:00
Pouzor ecc3cb8364 chore: bump version to 3.2.0
ha-relevant: no
2026-08-10 00:40:24 +02:00
Pouzor 9954fccf75 feat(rack): link a mount to any Device Inventory entry
The rack's link picker read `/nodes`, so it only ever offered devices
someone had already approved onto a logical canvas — two rows on a homelab
holding 74 inventory entries. A device on no canvas is still the record of
a real box, and is exactly what a rack is built out of.

`DevicePickerModal` replaces `NodePickerModal` and lists the Device
Inventory itself. Picking an entry calls the new `relinkDevice`, which
repoints the mount's `deviceId`, adopts that entry's node, status and —
unless the user renamed the plate — its label. One entry, one mount: a row
another plate stands for is not offered, and the store refuses it anyway.
The placeholder a rack-created plate left behind is dropped through the new
`DELETE /api/v1/scan/pending/{id}`, which refuses a device a rack still
mounts (409): foreign keys are off at runtime, so the mount would be left
naming a row that no longer exists.

`LinkedDevicePanel` becomes "Linked device" and now prints what discovery
found even when nothing on a canvas answers for the device; only the
canvas-side rows go missing, under a "Not on a logical canvas." note.

Also renames `pending_devices` to `device_inventory` (and
`pending_device_links` to `device_inventory_links`), with the Python and
TypeScript names that followed it. "Pending devices" was the scanner's word
for a queue of finds awaiting approval; the rows outlive approval, are
edited by hand and are what a rack mounts. Routes, payload keys and MCP
tool names are a published contract and are unchanged — `/scan/pending/*`
and the `pending_devices` key in `/stats` stay as they are.

The rename migration runs before `create_all`, or an empty new table would
be created beside the populated old one and every scanned device would read
as gone; it repairs that state too, for anyone whose app already started
mid-upgrade. Foreign keys are switched on for the rename so SQLite rewrites
the `REFERENCES` clause in `rack_devices`.

ha-relevant: maybe
2026-08-10 00:23:10 +02:00
Pouzor c263f839a8 fix(rack): stamp the linked node's last-seen with an offset
SQLite has no timezone type: an aware `last_seen` reads back naive, and
Pydantic then put it on the wire as `2026-08-08T10:00:00`. To the browser
that is *local* time, so the Logical view panel printed a timestamp
shifted by the viewer's UTC offset.

ha-relevant: no
2026-08-10 00:23:10 +02:00
Pouzor e22152d2d3 feat(rack): show the logical view of a mounted device
The Edit Device modal left the column under the port list empty, while
the logical canvas already held every technical fact about the same box.

A mount that stands for a Device Inventory entry now prints them there:
canvas name, type, hostname, IP, MAC, OS, the status check the node runs,
the canvas it is drawn on, when it was last seen, and the services
discovery fingerprinted on it. Read-only — the logical view owns them.

`/racks/inventory` ships the node half as `node_*` alongside the
inventory row's own mac/hostname/os/services, and the panel prefers the
node value: the node is what the user curates, the inventory row is what
discovery last saw and goes stale after a rename or a DHCP move. Rows
neither side can fill are dropped, so a device on no canvas still shows
what was reported and an accessory shows no panel at all.

The new API fields are optional client-side, so an older backend reads
back as "not on a canvas" rather than an empty node.

ha-relevant: maybe
2026-08-10 00:23:10 +02:00
Pouzor 57df3b9198 fix(rack): stop a blanked height field from shrinking the rack
Three from the third review pass, all in code this branch added.

Number('') is 0, so `Number(draft) || MIN_RACK_U` turned a select-all +
Delete into `uHeight: 1` on blur. A rack with mounts refused it and
showed a misleading toast; an empty one — the just-added case, 24U by
default — took it silently, with no undo. An empty or unparseable field
is now "no edit".

CableLayer returned null on `visibility === 'hidden'` before the
per-cable filter that deliberately keeps the selected run drawn ever
ran, so switching the header select to Hidden left RackCablePanel open
over a cable nothing showed. Hidden now still draws the selection —
and nothing else, not even the hover reveal.

PropertyList tracked the open edit form by array position while the
array belongs to the caller. Removing or reordering an earlier row
slid the form onto its neighbour and Save overwrote the wrong
property. The form closes on any mutation from outside it. This one
predates the branch — it moved verbatim out of DetailPanel — but the
extraction doubled its reach to the cable panel, and the component had
no tests. It has ten now.

ha-relevant: maybe
2026-08-09 20:35:23 +02:00
Pouzor 5617f0800f fix(rack): commit height and colour edits on blur, not per keystroke
The rack height field called updateRack on every character. Since that
relocates the mounts a shrink pushes past the top rail, and the rack
canvas has no undo, typing rearranged the rack behind the user:
backspacing "24" leaves "2", every mount is relocated to the bottom,
and typing "4" back restores the height but not the layout. On a full
rack it fired the refusal toast twice per edit instead.

The height and the cable colour hex now keep a local draft and commit
on blur or Enter. A colour that does not parse falls back to the cable
type's default, rather than handing the SVG stroke a half-typed "#39d"
and leaving the run invisible.

Three more from the same review:

- Delete rack asks first, naming how many mounts it takes with it. It
  drops every mount and every cable touching them, with no undo.
- save() takes the design id the caller means to write. The
  design-switch flow saves the old design while the store may already
  hold the new one; it now refuses instead of persisting the wrong one
  and dropping the other's edits.
- fromRackDevice clamps col_span against col_start rather than against
  the grid alone. 11 + 12 is two legal fields spanning to column 23 of
  12, which the new backend validator rejects — one device 422'd the
  whole save.

ha-relevant: maybe
2026-08-09 20:35:23 +02:00
Pouzor cd6402a680 fix(rack): keep a rack height edit from orphaning its mounts
The number input's min/max are hints the browser does not enforce on a
typed value, and updateRack wrote the patch straight through. Two ways
out of a usable canvas:

- Shrinking below a mounted device left the plate drawn above the
  chassis, outside the React Flow node, with nothing in the UI able to
  drag it back. freeUnits only counts 1..uHeight, so the modal's "used
  of" label under-counted it in silence.
- A height over 100 made RackSave reject every save with a 422, so both
  the explicit Save and each autosave tick reported "Save failed" with
  no cause.

updateRack now clamps to [MIN_RACK_U, MAX_RACK_U] and relocates the
mounts a shrink pushes past the top rail, one at a time so two never
land on the same slot. It returns false — changing nothing — when one
has nowhere to go, and the modal says so.

Two server-side guards that would have contained it: RackSaveRequest
cross-checks every mount against the rack it names, and RackDeviceSave
checks col_start + col_span against the grid, which each field passing
its own bounds never caught.

Also normalizes the MAC on POST /scan/pending. Every other write path
canonicalizes it and dedup compares by equality, so a hand-typed
AA-BB-CC-11-22-33 never matched the scanned aa:bb:cc:11:22:33 and
approve built a duplicate node.

ha-relevant: maybe
2026-08-09 20:35:23 +02:00
Pouzor 4d2128c494 feat(rack): document a cable and print it on the canvas
Clicking a cable — in patch mode or out of it — selects it and opens a
right rail. It carries the physical facts (type, colour, label) and
`properties`: the same NodeProperty records the logical canvas already
uses for nodes, each with its own "show on canvas" eye. What is ticked
visible is drawn on a small plate at the midpoint of the run, so an
export carries its lengths and VLANs.

The property editor moves out of DetailPanel into
components/common/PropertyList so both canvases share one implementation
rather than growing a second.

Persistence: rack_cables gains label_visible and properties, added by an
idempotent migration in _try_migrate; the response schema coerces the
NULLs legacy rows read back as.

Also drops the 40 % plate fade that came with cables-on. It let the rail
strips and the U grid show through the mounted gear, which read as a
rendering bug — cables are drawn above the plates anyway.

ha-relevant: maybe
2026-08-09 20:35:23 +02:00
Pouzor 84360ffc50 chore(deps): bump js-yaml to 4.3.1 for CVE-2026-59870
js-yaml 4.0.0 through 4.3.0 resolve !!omap in quadratic time
(GHSA-5p4m-2wfm-xmqj, high). The advisory landed after this branch's last
green run and fails `npm audit --omit=dev --audit-level=high` on main too.

4.3.1 is the patched 4.x, so the existing caret range covers it and the YAML
import/export code is untouched. Staying on 4.x avoids the 5.x API change,
which belongs in its own bump.

ha-relevant: no
2026-08-09 20:35:23 +02:00
Pouzor 56df462a8f ci: stop the secret scan reporting pytest names as Lob keys
The push that carried the whole branch through TruffleHog failed on a
"verified" Lob credential: `test_updates_an_existing_device_in_place`, a test
function in backend/tests/test_racks.py. Lob's detector matches any
`test_`-prefixed identifier of that length and reports it verified, because
Lob test-mode keys authenticate unconditionally — so verification proves
nothing here.

Lob is a direct-mail API this project does not use, and any pytest function
named that long trips it again, so the detector is excluded rather than the
test renamed.

ha-relevant: no
2026-08-09 20:35:23 +02:00
Pouzor 06ccd354c5 fix(rack): make Import links idempotent instead of one-shot
networkImportDone lived in memory only. loadDesign starts from emptyState and
the flag was in neither the save payload nor standaloneStorage, so a reload
re-armed the import: the second run found the original ports taken, walked on
to the next free pair, and drew a duplicate cable for every logical link — once
per reload.

The device pair is the guard now, and it survives a reload because the cables
do: a pair already patched is skipped whatever ports carry it. The flag and the
disabled button go with it, which also makes the feature better — run it again
after racking more gear and it adds only what is missing.

While in there, the import prefers a port whose type matches the link: a fibre
edge landing on two RJ45 jacks drew an amber run across copper. Any free port
is still used when the plate has no matching one.

Sidebar's rack branch gains the tests it shipped without: the view/action swap,
the hidden discovery actions, + Device opening the editor, the capacity footer,
and the unsaved badge reading the rack store rather than the canvas one.

ha-relevant: no
2026-08-09 20:35:23 +02:00
Pouzor c63bfd3859 fix(rack): keep Import links usable after a run that matched nothing
importCablesFromNetwork latched networkImportDone whatever the outcome. Click
Import links before racking anything and the toast says "No matching link found
— rack the devices first", then the button is disabled for the rest of the
session: it asks for a retry and forbids it. Only a design switch or a reload
cleared the flag.

It now latches on success only.

ha-relevant: no
2026-08-09 20:35:23 +02:00
Pouzor 2af6f728d7 docs(rack): document the rack canvas for users
The rack canvas shipped with a developer reference under frontend/src/rack and
nothing a user would find: the README did not contain the word rack, and
FEATURES.md jumped from Multiple Canvases to Customize Style.

docs/rack-canvas.md covers it end to end — creating the canvas, rack settings,
the three ways to mount something, the placement rules, the faceplate catalog,
ports and patching, the Check device status, saving, and the known limits.
FEATURES.md gains it as section 5, beside the other canvas features rather than
appended after Authentication, so sections 5 to 18 shift by one; the README gets
a nav entry and a short section pointing at the full page.

Marked as working without a backend, since a rack canvas persists to
localStorage in standalone; the inventory picker, the status check and the link
import stay flagged as full mode.

ha-relevant: no
2026-08-09 20:35:23 +02:00
Pouzor b1186bd7dd fix(rack): drop the cable type filter, and dress the visibility select like the header
Two native <select> elements sat in a header made of ghost buttons, and they
looked it: their own border and background, an OS chevron, an unthemed popup.

The type filter goes entirely — state, action and the CableLayer branch that
read it, plus CABLE_TYPE_LABELS which had no other consumer. Filtering the
cabling down to copper or fibre answered a question nobody asks of a rack
they are looking at; the type still follows the port a patch starts from,
which is where it matters.

The visibility control becomes the shared ui/select, with a trigger sized and
coloured like the buttons beside it — transparent until hovered — and an icon
per option: pointer for on-hover, eye for always, crossed eye for hidden.

Tests cover the trigger reflecting the active visibility, a pick writing it to
the store, and CableLayer drawing ethernet and fibre alike. base-ui commits a
selection on a real pointer sequence rather than a bare click, so that one goes
through userEvent.

ha-relevant: no
2026-08-09 20:35:23 +02:00
Pouzor a70fc248bf feat(walkthrough): introduce the rack canvas, and keep Next inside the card
The Getting Started tour stopped at the logical canvas, so the third design
type was invisible to anyone who had not gone looking for New Canvas → Kind.
A step between grouping and styling now points at the canvas switcher and
says what a rack canvas is for. It is `mode: 'all'` — a rack needs no backend,
it persists to localStorage in standalone like any other canvas — and it
anchors on the switcher rather than a rack control, since the tour runs on a
network canvas where none of the rack sidebar items exist.

That twelfth step also broke the card footer. The progress strip grows with
the step count and neither footer child could shrink (`min-width: auto`), so
at twelve dots the strip pushed the counter and both buttons past the right
edge of the 320px card — Next ended up outside it. The strip is now the part
that gives: it shrinks and clips, the controls hold their size, and Next stays
anchored whatever the step count.

ha-relevant: maybe
The overlay fix applies to any build of the frontend; the rack step itself is
pointless until the rack canvas is ported.
2026-08-09 20:35:23 +02:00
Pouzor ff740f7466 feat(rack): patch by dragging port to port, and select a cable before deleting it
Cabling was a two-click flow with no way back: the only way to remove a patch
was to click it, which deleted it outright, and clicking a port was the only way
to start one.

- Drag from a port to another to patch, with a dashed rubber band following the
  pointer. Clicking one port then another still works; Escape drops a
  half-drawn patch.
- A click on a cable now selects it (accent halo, fat invisible hit area);
  Delete/Backspace or the new header Unplug button removes it.
- Leaving patch mode restores the cable visibility it was entered with, instead
  of stranding the canvas on "always" with every plate faded until a reload.
- Draw every port at one fixed size — the 1U switch/patch-panel socket — rather
  than scaling it with plate height. Drops the now-unused template portSize.

ha-relevant: no
2026-08-09 20:35:23 +02:00
Pouzor 3138cda378 feat(rack): follow a node's status check, and type rack-created gear
A mount's Status select gains "Check device": instead of a colour frozen at
mount time, it follows the status check already configured on the matching
logical-canvas node (ping, http, ssh…). The rack runs no checker of its own,
so `auto` resolves through the Device Inventory's `node_status`, falls back to
`unknown` when nothing is behind it, and is only offered when the mount — or
the entry being picked — resolves to a node. Losing that link drops the mount
back to `unknown` rather than leaving it on a status nothing can answer.
`useAutoStatusRefresh` polls the inventory every 60s while at least one mount
follows a node, and only then; it writes inventory only, so autosave stays
quiet.

A device created from the rack canvas also lands in the Device Inventory with
a type now. Nothing discovers those rows, so the plate they wear is the only
thing that says what they are — without it they had no icon, no role badge and
no place in the type filter. Passive gear uses the rack-only kinds
`patch_panel` and `pdu`; the latter is deliberately not `socket`, which is
excluded from the rack inventory and would make a PDU vanish on creation.

ha-relevant: maybe
2026-08-09 20:35:23 +02:00
Pouzor 0640704678 fix(rack): draw desktop NAS towers at a size a disk actually is
The drive stack stretched to fill the plate, so the 2-bay box wore two doors
twice as wide as the 5-bay one's — a 3.5" disk is the same disk in every
chassis. The stack now grows with the bay count instead: `nasBays` sizes it
from a fixed per-door width, and scales that back through `colSpan`, since unit
coordinates are fractions of the plate and a narrower plate needs bigger
fractions to draw the same door.

Height went to 5U for all three. A U draws at 24px against a 456px inner width,
so the canvas squashes the vertical axis about 1.8x: a tower that is 3U in
metal renders as a squat box, and its doors read as squares. The picker looked
right all along because it draws a plate at its own aspect, not the rack's.

The 2-bay is a sixth of the rack rather than a quarter — 76x120px at 19", close
to the proportions of the real thing. That is a new plate width, so the modal's
Width select and the picker's width labels learned it; a colSpan with no
matching option leaves the select blank. A test now walks the catalog and
asserts every plate width is offered.

ha-relevant: maybe
2026-08-09 20:35:23 +02:00
Pouzor b129200929 feat(rack): pick a mount from the Device Inventory, filtered to rackable gear
The rack device modal listed inventory entries in a flat <select>: one line per
device, no search, no filters, nothing to tell two look-alike hosts apart. It
now opens the Device Inventory modal itself in a picker mode — a card click
returns the device instead of approving it, and the bulk/clear controls (plus
the s/a/Enter shortcuts) are hidden, since a picker has no business deleting
anything.

That list also carried VMs, containers and mesh devices, which no rack can
hold. A new "Rackable" filter drops them, armed by default when the rack opens
the modal. It is an exclusion list, not an allow list, so new hardware types
show up by default and an unclassified device (a bare ARP hit) stays visible.
`utils/rackable.ts` mirrors `_UNRACKABLE_TYPES` in racks.py — the list the
inventory endpoint already filtered on — and test_rackable_sync.py fails the
backend suite on drift.

The rack side keeps the last word: a device missing from its own inventory, or
already mounted in this design, is refused by name rather than mounted as a
ghost. Nothing is preselected anymore either, so submitting without picking
says so instead of quietly mounting whichever entry happened to be first.

Standalone keeps the <select>: the inventory modal reads pending_devices over
REST and there is no backend there.

The modal itself was max-w-md and one column, which pushed the port list below
the fold. It now matches NodeModal's width with the fields on the left and the
ports on the right.

ha-relevant: maybe
2026-08-09 20:35:23 +02:00
Pouzor e1f8850d1f fix(rack): guard rack rows against cross-design saves, and half-applied plates
Four more defects from the branch review.

`POST /racks/save` fetched every row by primary key alone and then wrote
`design_id` from the payload, so a save on design B carrying an id that belongs
to design A moved A's rack — devices and cables with it — into B, with no error
and no way to notice. Ids come from the client, so a copied design or a stale
tab is enough. `_owned` now refuses a row owned by another design with a 409.

`commitEdit` dropped `applyFaceplate`'s return. Picking a plate the rack has no
room for, then shrinking the height by hand, let `updateDevice` succeed on its
own and `setPorts` write the new plate's ports onto the old faceplate — the
device ended up wearing a plate it did not have, silently. The failure is now
reported and nothing is committed.

Two smaller ones. The inventory select held a dead id after the list was
refetched, so submitting reported "No free slot in this rack" when the real
cause was an entry that had been racked elsewhere or deleted; it now falls back
to its placeholder and says so. And both create paths patched the form's own
geometry over the slot `findSlot` had just chosen, undoing a relocation the user
was never shown — the mount is handed the whole geometry up front, and only the
overrides are patched afterwards.

ha-relevant: maybe
2026-08-09 20:35:23 +02:00
Pouzor 9bf90cbbed fix(rack): draw the desktop NAS plates like the boxes they are
The first cut put the drive bays in a squat block across the top half and left
the name across the middle, which looked nothing like a UGREEN DXP or a
Synology DS: their front is two to five tall tray doors filling the height, over
a thin strip carrying the badge, the LED and the sockets.

The name band was the blocker — it was nailed to mid-height for every plate, so
anything tall wore its name across its own artwork. `labelBox.y` (default 0.5,
so rack gear is untouched) now places the band, and the status LED rides it.
The NAS plates put it at 0.86 with the trays above and the ports on the same
strip.

Bay corners also scaled badly: a flat 1.5px radius reads as a square hole once
the cells are as large as a 3U tray, so the radius now follows the cell size.

ha-relevant: yes
2026-08-09 20:35:23 +02:00
Pouzor 7067bd76d4 feat(rack): add desktop NAS faceplates in 2, 4 and 5 bays
A UGREEN DXP or a Synology DS is not rack gear: it sits on a shelf, taking
about a third of the width and standing some 3U tall, with its drive trays side
by side rather than in a grid. The catalog had no shape for that — a homelab
NAS had to borrow the 2U rack-mount plate.

Trays fill the upper half, one column per disk. The name band and the ports
share the lower half side by side, since the name is always drawn at mid-height
whatever the U count. Ports follow the class: one RJ45 on the 2-bay, two on the
4-bay, two plus an SFP+ on the 5-bay.

`suggestFaceplate` still proposes the rack-mount `nas-2u` for a discovered NAS;
which of the two a scan should assume is a separate question.

ha-relevant: yes
2026-08-09 20:35:23 +02:00
Pouzor 8cafeee743 fix(rack): stop the device modal losing cables, status and inventory rows
Four defects found reviewing the last two commits.

Reverting a faceplate choice took every cable on the device down with it.
Browsing to another plate and back left `plateChanged` set with freshly seeded
port ids, while `commitEdit` skipped `applyFaceplate` because the id matched —
so `setPorts` received ids the device never had and dropped every cable
attached to the old ones, with no warning, since from the user's side the plate
had not changed. Coming back to the device's own plate now restores its ports,
ids included.

The inventory select preselects the first unracked entry, but only
`pickInventory` seeded the status from it, so submitting without touching the
select wrote `unknown` over the status the mount had just copied. Status now
seeds from that same entry.

Creating a device POSTs to the Device Inventory before the mount is attempted,
so a rack with no free slot stranded the row — and stranded another on every
retry. The rack is checked for a slot before anything is created.

And `_service_name` accepted a service with no port, where the inventory
requires one (`None not in _COMMON_PORTS` is True): a portless service named a
device in the rack picker but not in the Device Inventory, the exact mismatch
the naming fix set out to remove.

ha-relevant: maybe
2026-08-09 20:35:23 +02:00
Pouzor cfa296ef87 fix(rack): name inventory entries the way the Device Inventory does
The rack picker listed "192.168.1.63 · 192.168.1.63" where the inventory
showed "jellyfin", so a device the user picked out by one name turned up here
under another.

Two causes. The inventory route labelled an entry
`friendly_name or hostname or ip or id`, skipping the app-name step: `_device_label`
now mirrors `deviceLabel` in PendingDevicesModal — friendly name, host, the app
its ports say it runs (ssh/http/https ignored, generic web deprioritized), IP,
IEEE, id. And the option line appended the IP unconditionally, doubling it
whenever the label already was the IP; it now adds the IP only when it says
something new, plus the type as the tiebreaker between look-alikes.

Also fixes the port row: the name field and the type select both carried
`w-full` from the shared input class, so the select took the row and the name
field collapsed into an unlabelled box.

ha-relevant: maybe
2026-08-09 20:35:23 +02:00
Pouzor c6c7624084 feat(rack): pick faceplates from a visual catalog
A plate is a drawing, not a name — "Switch 24 ports + 2 SFP" in a <select>
tells you far less than seeing the plate. The Faceplate field now previews the
current plate and opens FaceplatePicker: every template drawn with the canvas
renderer, grouped by category, searchable by name or category, and filtered by
kind so accessories are offered only to accessories. Switching the device
source between device and accessory swaps the plate to a default of the other
kind.

Each tile scales the plate to the tile width and keeps its fraction of the rack
(a half-width plate draws half as wide), so sizes compare honestly; the meta
line states U height, width and port count.

Two layout notes worth keeping: DialogContent carries `sm:max-w-sm`, which is
media-scoped and beats a plain `max-w-*`, so the override has to be prefixed;
and an SVG is a flex item that shrinks but never grows, so the plate is drawn
at a nominal size and scaled up by CSS instead of sized in pixels.

ha-relevant: no
2026-08-09 20:35:23 +02:00
Pouzor e2da642554 feat(rack): edit devices in a modal, file rack gear in the Device Inventory
The rack canvas loses its right rail: a mount is now edited in its own dialog,
like a network node, and the canvas keeps the full width.

Devices:
- `RackDeviceModal` adds and edits a mount — label, faceplate, U position,
  height, column, width, status, colour, ports, unmount. Adding offers three
  sources: an existing Device Inventory entry, a new device created here, or a
  rack-only accessory. Opened by `+ Device` in the left rail or a double-click
  on a plate.
- `RackSettingsModal` takes over rack chrome (name, location, capacity, 19"/10",
  numbering, colours, delete), on a double-click of the chassis.
- The left rail's `+ Rack` becomes `+ Device`; `+ Rack` stays in the header.

Inventory:
- Gear created from a rack canvas lands in the Device Inventory with
  `discovery_source: "rack"`, so it inherits the existing search, filters, hide
  and delete instead of a parallel list in the sidebar. New "Rack devices"
  source filter and badge.
- It is never placed on a logical canvas: approve returns 409, bulk-approve
  skips it with `match: "rack"`, and the UI drops the Approve button and reports
  the skip.
- `InventoryTray` becomes `AccessoryTray` — blanks, shelves and cable managers
  only, still a drag source.

Fixes:
- A taller faceplate (or a hand-typed height) no longer silently keeps the old
  geometry when it collides: the device relocates to the nearest slot that takes
  the new size, and only a rack with no such slot refuses the edit. This is why
  a 1U and a 2U plate rendered at the same height and the field looked locked.
- The empty-state buttons sat under `.react-flow__renderer` (z-index 4), so the
  pane swallowed their clicks as a canvas drag.

Tests: rack device/settings modals, empty-state stacking, relocate-vs-refuse in
the store, rack source bucketing, inventory approve gating (frontend + backend).

ha-relevant: maybe
The modal-based editors and the rack store are portable; the inventory tagging
rides on the standalone FastAPI scan routes. Discuss before porting.
2026-08-09 20:35:23 +02:00
Pouzor a0d16e1454 feat(rack): integrate the rack canvas as a design type
The rack prototype lived behind /racklab with no backend. It now becomes a
third design_type alongside network and electrical, sharing the app shell:
same header, same left rail, same right panel, same design switcher. App
branches the renderer on design_type.

Backend
- racks / rack_devices / rack_cables, all design_id-scoped.
- rack_devices.device_id -> pending_devices is the primary link. Those rows
  survive approval and node deletion, so unracking never removes the device
  from the Device Inventory. node_id -> nodes is a second, optional link used
  only for live status and for matching network-link imports. Both are
  ON DELETE SET NULL and label is denormalized, so a rack keeps rendering
  after an inventory purge.
- GET /racks, POST /racks/save (full-state upsert + prune, same contract as
  canvas/save), GET /racks/inventory.
- POST /scan/pending adds an inventory entry by hand, for hardware no scan
  can discover.
- design_type is now validated and actually branched on; the schema comment
  calling it vestigial is gone.
- Design delete unwinds rack rows explicitly (SQLite does not always honour
  the cascade); design copy duplicates racks, mounts and patches with fresh
  ids and re-points the cables at the copies.

Frontend
- /racklab and RackLab are removed; no dual maintenance.
- Rack types lifted into @/types; rackSerializer maps the snake_case wire
  shapes to them, narrowing every enum on the way in.
- The store gains load/save, dirty tracking and an editSeq, so racks get the
  same explicit Save and the same opt-in autosave as the logical canvas.
- Header, sidebar and inspector adapt in rack mode: canvas-only actions are
  hidden, Add Rack / Patch / cable visibility / type filter / Import links
  take their place, the sidebar body becomes the inventory tray, and the
  footer counts racks, mounts, cables and free U.
- rackTheme derives the rack palette from the active app theme rather than
  declaring one per theme, so a new theme works here for free. Per-rack
  chrome stays user-editable; only its default comes from the theme.
- New Canvas gains a Kind picker (hidden when copying, since a copy inherits
  the source kind).

Tests: 24 backend, 5 frontend files (store, persistence, networkLinks,
rackTheme, rackSerializer) plus DesignModal coverage for the Kind picker.

ha-relevant: maybe
Frontend rack renderer is portable; the FastAPI routes and SQLAlchemy models
are standalone-only by the persistence/transport split. Discuss before porting.
2026-08-09 20:35:23 +02:00
Pouzor 33df37b691 feat(rack): front-only rack canvas prototype at /racklab
Physical counterpart to the logical canvas: racks hold hardware at a U
position, with drawn front panels and port-to-port patching.

- 12-column grid so 2 or 3 machines can share one U; collision and
  snap-to-free-slot handled in layout.ts
- 19 faceplate templates drawn as SVG in unit coordinates, so a plate
  scales with U height and rack width
- RJ45 / SFP port artwork only; ports stay hidden on non-patch gear
  until hover, selection or patch mode. Outlets are artwork, never cabled
- cables are their own relation, not React Flow edges, rendered in a
  ViewportPortal so they cross racks and follow pan/zoom
- unmounting a device never removes its inventory entry
- rack settings: size, 19"/10", numbering direction, colours, enclosure

Isolated behind its own entry point (like /view) — no API, no
persistence, no design_type change until the UX is settled.

ha-relevant: yes
2026-08-09 20:35:23 +02:00
Pouzor f07f43686e chore: bump version to 3.1.2
ha-relevant: no
2026-08-01 01:26:23 +02:00
Pouzor 3bde741fd2 chore(deps): clear the seven open dependabot alerts
Lockfile only — every advisory is transitive and no direct dependency range
had to move:

- postcss 8.5.14 -> 8.5.25 (high, GHSA-r28c-9q8g-f849, path traversal in
  source-map auto-loading), pulled in by vite and shadcn
- fast-uri 3.1.2 -> 3.1.5 (two high advisories), via shadcn's ajv
- hono 4.12.25 -> 4.12.33 and @hono/node-server 1.19.14 -> 2.0.12 (three
  moderate advisories), via shadcn's @modelcontextprotocol/sdk, itself bumped
  1.29.0 -> 1.30.0

`npm audit` now reports 0 vulnerabilities. Build, lint and the full frontend
suite (1542 tests) pass unchanged.

ha-relevant: no
2026-08-01 01:14:21 +02:00
NebeszandPouzor - Rémy Jardient e8ea6857bc feat(mcp): add lightweight node lookup tools
Add a label substring filter to GET /api/v1/nodes, plus MCP tools
list_node_summaries (id/label/type/status only) and get_node
(by id -> single object, by label -> array) so an LLM client can look
up nodes without paying the token cost of the full node payload.
2026-07-30 01:57:34 +02:00
Pouzor - Rémy JardientandGitHub 42774fc138 Merge pull request #311 from Nebesz/fix/mcp-node-type-enum
fix: sync MCP node type enum with frontend NodeType
2026-07-30 01:37:33 +02:00
Pouzor 8f273e0201 fix: sync MCP edge type enum with frontend and gate both enums in CI
The node type fix left the same stale-enum bug in place for edges:
create_edge only advertised ethernet/wifi/iot/vlan/virtual, while the
frontend EdgeType union also defines cluster, fibre and electrical.
The backend's EdgeBase.type is a plain str, so those three were always
accepted server-side — only the MCP schema rejected them.

- Extract EDGE_TYPES in mcp/app/tools.py and use it for create_edge.
- Add mcp/tests/test_edge_types_sync.py, mirroring the NodeType guard:
  parses EdgeType out of frontend/src/types/index.ts and asserts set
  equality with EDGE_TYPES.
- Add create_edge cases for the three previously-rejected types plus a
  schema test pinning the enum contents and the ethernet default.
- Add an `mcp` job to quality.yml. Both sync guards were unreachable
  from CI before: quality.yml only ran frontend and backend, so the
  enums could drift again with nothing to catch it. The job runs from a
  full checkout because the guards read the frontend sources, which are
  not in the mcp Docker build context.
- Pin mcp[cli]>=1.26,<2. The bound is load-bearing: SDK 2.0 dropped the
  low-level Server.list_resources/list_tools decorator API that
  app/resources.py and app/tools.py are built on, so an unbounded spec
  breaks both the tests and the Docker image at import time.
- Move pytest/pytest-anyio/trio into mcp/requirements.txt, matching how
  backend/requirements.txt carries its own test deps. trio is required
  because pytest-anyio parametrises every async test over both backends.

Tested: 114 passed in mcp/, both in the existing venv and in a clean
Python 3.13 venv installed the way the new CI job does.

ha-relevant: no
2026-07-30 01:05:48 +02:00
Pouzor - Rémy JardientandGitHub 3c8e463dd9 Merge pull request #313 from Pouzor/feat/service-brand-icons
feat: brand icon picker for services, in a dedicated add/edit modal
2026-07-28 22:12:27 +02:00
Pouzor c4017c7634 feat(services): brand icon picker and dedicated add/edit modal
Services can now carry a brand logo (homarr dashboard-icons) or a generic
lucide icon, the same way nodes already could — closes the request in
discussion #299 where services stand in for docker images.

The add/edit form moves out of the right-hand DetailPanel into a proper
ServiceModal, which gives the icon picker room to breathe and matches how
nodes are edited.

- ServiceModal: name, port, protocol, path, icon; validates the name and
  clamps the port, and spreads over the original service on save so fields
  it doesn't own (category) survive an edit
- IconPickerPanel: Generic/Brand picker extracted from NodeModal so both
  surfaces share one implementation
- ServiceIcon: renders a lucide or brand:<slug> icon in the DetailPanel
  badge and on the canvas node, and nothing when the key can't be resolved
- Legacy scanner signature keys (bar-chart-2) still don't resolve to an
  icon, but the modal shows them verbatim and preserves them on save

ServiceInfo.icon already existed and the backend stores services as
list[Any], so no schema or migration change.

ha-relevant: yes
2026-07-28 22:00:45 +02:00
Nebesz a88688170a fix: sync MCP node type enum with frontend NodeType
The hardcoded NODE_TYPES list in mcp/app/tools.py only had 11 entries,
causing create_node/update_node/approve_device to reject valid device
types (firewall, docker_container, ups, etc.) that the backend and
frontend already support. Expanded the list to match
frontend/src/types/index.ts's NodeType union, excluding group/groupRect/text
which are canvas annotations, not devices. Added a test that parses the
frontend union and fails if the two lists drift apart again.
2026-07-28 18:17:52 +02:00
Pouzor - Rémy JardientandGitHub aaccb50b00 Merge pull request #306 from autogpt-elendir/feat/oidc-auth
feat(auth): add generic OIDC browser sessions
2026-07-28 00:29:41 +02:00
Pouzor 34758c9850 fix(ui): use official Homelable logo on login page
The login page rendered lucide's generic Network glyph instead of the
brand mark. Use the Logo component there, and realign Logo with the
canonical assets in docs/logo:

- match icon.svg geometry exactly (bg glow gradient, node glow filter)
- fix inverted wordmark colors (Home #e6edf3 / lable #00d4ff)
- scope gradient/filter ids per instance so the Sidebar and Toolbar
  logos don't share defs ids

ha-relevant: maybe
2026-07-25 00:02:39 +02:00
ypso 4d2284dc37 fix(auth): harden OIDC callback validation 2026-07-24 11:12:57 +02:00
Pouzor 9f974f2fcf docs(auth): document OIDC setup and configuration
- FEATURES.md: add "Authentication (Local / OpenID Connect)" section + TOC
- docs/oidc-auth.md: flow overview, env var reference, fail-closed validation
  rules, provider setup (Authentik/Keycloak/Authelia/Google), reverse-proxy
  notes, session/logout caveats, troubleshooting, revert-to-local

ha-relevant: no
2026-07-21 23:35:50 +02:00
Pouzor 37ad9c6933 fix(ci): scope dependency-audit npm audit to runtime deps
The dependency-audit job failed on vulnerabilities from `shadcn` and its
transitive tree (hono, fast-uri, body-parser, @modelcontextprotocol/sdk).
`shadcn` is a build-time scaffolding CLI plus a CSS `@import`, never shipped
in the browser bundle, so it belongs in devDependencies.

- move `shadcn` to devDependencies (regenerated lockfile: dev flags only,
  no version changes)
- run `npm audit` with `--omit=dev`, matching the runtime-only intent of the
  pip-audit step

npm audit --omit=dev --audit-level=high now reports 0 vulnerabilities; the
production build still resolves the shadcn CSS import.

ha-relevant: no
2026-07-21 23:34:34 +02:00
ypso 99e7b9a343 ci(security): enforce unsuppressed Python audit 2026-07-21 00:37:32 +02:00
ypso f061db645b chore(deps): refresh vulnerable frontend packages 2026-07-21 00:36:06 +02:00
ypso fd96b7d6ae feat(auth): add OIDC browser session flow 2026-07-21 00:36:02 +02:00
ypso 0e95d99868 feat(auth): add generic OIDC backend sessions 2026-07-21 00:35:57 +02:00
Pouzor - Rémy JardientandGitHub d9f3b4c18a Merge pull request #303 from Pouzor/release/3.1.1
chore: bump version to 3.1.1
2026-07-19 22:23:57 +02:00
Pouzor 80ebbca804 chore: bump version to 3.1.1 2026-07-19 22:17:59 +02:00
Pouzor - Rémy JardientandGitHub 959cd4a3ae Merge pull request #302 from Isilla/feat/services-drag-reorder
feat(detail-panel): make services list draggable to reorder
2026-07-19 22:07:20 +02:00
Marianne JOSEPH 084a8e23e7 feat(detail-panel): make services list draggable to reorder
Mirrors the existing properties drag-and-drop (native HTML5 drag)
so services can be reordered the same way.
2026-07-19 21:59:16 +02:00
Pouzor - Rémy JardientandGitHub f5dd46a729 Merge pull request #301 from Pouzor/fix/yaml-export-port-numbers
fix: round-trip Show Port Numbers on YAML export/import
2026-07-19 21:12:18 +02:00
Pouzor 121873f41c test(settings): wait for async config before asserting Save payload
The Proxmox/Zigbee auto-sync Save tests clicked Save after only
waiting for an unrelated field, so under CI load getConfig could
resolve after the click and Save fired with the default
sync_enabled=false. Wait for the toggle to reflect the loaded
state first.
2026-07-19 15:34:19 +02:00
Pouzor 16d6348cec fix(yaml): round-trip show_port_numbers on export/import
YAML export wrote per-side handle counts but never the
show_port_numbers toggle, so the option was lost on re-import
while connection points survived.

ha-relevant: maybe

Fixes #272
2026-07-19 15:15:13 +02:00
Pouzor - Rémy JardientandGitHub 28fc4d8a7f Merge pull request #300 from Pouzor/feat/getting-started-walkthrough
feat(walkthrough): Getting Started tour + new-user canvas detection
2026-07-19 11:52:27 +02:00
Pouzor 5726289b18 fix(toaster): enable richColors so error surface actually applies
Setting --error-bg alone had no effect: sonner only honors per-type
surfaces when richColors is enabled, so error toasts kept using
--normal-bg and looked identical to a normal toast. Enable richColors
and pin success/info/warning back to the neutral popover surface so only
errors turn red.
2026-07-19 11:44:47 +02:00
Pouzor e85dab8cfc fix(toaster): render error toasts on a red surface
Error toasts previously used the same neutral popover background as
success toasts, so a failure (e.g. a save while the backend is down)
read like a normal 'Canvas saved'. Set sonner's per-type --error-bg/
--error-text/--error-border so errors stand out in red.
2026-07-19 11:33:54 +02:00
Pouzor deccff15cf changing wording 2026-07-19 11:05:43 +02:00
Pouzor a1bf54c417 fix(walkthrough): keep sidebar crisp behind modal steps
Neutralize the app modal backdrop's dim/blur while the tour runs so a
ringed sidebar/toolbar control behind an open modal (steps 2, 10) stays
readable — the overlay's SVG spotlight is now the single source of
dimming. Add a walkthrough-running body class toggled by the overlay and
a CSS rule stripping [data-slot=dialog-overlay] background/backdrop-filter.

Also refine coach-card placement: dialogCardPosition drops the card into
the empty bottom-right corner for large modals (inventory, scan history)
and beside small modals, so it never covers modal content (steps 3, 4).
2026-07-19 01:32:08 +02:00
Pouzor 8c56921375 feat(walkthrough): add Getting Started tour with first-run invite
Bespoke spotlight walkthrough (no new deps) that guides new users through the
main features. A version-stamped invite bubble offers the tour to new users and
re-offers it once to existing users after an upgrade; restartable from Settings.

- Multi-hole SVG-mask overlay keeps a ringed control and any open modal lit
- Steps: scan, scan history, inventory, nodes, edit, text/zone, grouping,
  style, imports, and a closing recap with a GitHub link
- Scan/history/inventory/import steps inject demo data (no backend calls) and
  are filtered out in standalone, which ends on a full-mode recap
- Persisted invite state via localStorage (utils/walkthrough.ts), live tour
  state via zustand (stores/walkthroughStore.ts)

ha-relevant: maybe
2026-07-19 00:40:08 +02:00
Pouzor bf8551015c feat(canvas): distinguish new user from cleared canvas; stop demo on backend error
Cleared canvas re-showed the demo, and backend errors silently fell back to
demo — hiding real outages and forcing users to wipe demo nodes before bulk
edits. Now:
- backend load reports `initialized` (CanvasState row exists = ever saved)
- decideCanvasLoad() picks real | empty | demo; empty is kept empty
- backend down/error shows an error banner + toast, never the demo
- data-new-user flag reserved as the Getting Started walkthrough hook

ha-relevant: yes
2026-07-18 23:02:44 +02:00
Pouzor 3f030959be docs(readme): rework header with logo, badges, nav links 2026-07-18 20:06:22 +02:00
Pouzor - Rémy JardientandGitHub 0b8b9760bc Merge pull request #297 from Pouzor/release/3.1.0
chore: bump version to 3.1.0
2026-07-18 02:27:33 +02:00
Pouzor f1fe82646d chore: bump version to 3.1.0 2026-07-18 02:22:29 +02:00
Pouzor - Rémy JardientandGitHub b7e03142e9 Merge pull request #296 from Pouzor/feat/reorder-node-properties
feat: drag to reorder node properties
2026-07-18 02:04:08 +02:00
Pouzor af3dc018c7 feat(properties): drag to reorder node properties
Add native HTML5 drag-and-drop reorder to the property list in the
detail panel. A grip handle appears when more than one property exists;
dragging updates property order (persisted with canvas save) and is
undoable via history snapshot.

Closes #292

ha-relevant: yes
2026-07-18 01:51:57 +02:00
Pouzor - Rémy JardientandGitHub 79bd0d4efb Merge pull request #294 from Floyddotnet/main
feat: search for nodes and properties
2026-07-18 01:26:17 +02:00
Pouzor a364a95cb5 fix(search): clean up node/property search matches
- add tests for notes, property key/value, hidden-property exclusion, matched-value display
- fix flatMap indentation and stray semicolon
- use stable match key instead of display_value (avoids dup-key collisions)
- keep label searchable but not repeated in matched-values column
2026-07-18 01:17:36 +02:00
Pouzor - Rémy JardientandGitHub 5b0c064e68 Merge pull request #295 from Pouzor/fix/scanner-two-stage-version-scan
fix(scanner): decouple port discovery from version detection (#277)
2026-07-18 00:50:46 +02:00
Pouzor 6f01d5550d fix(scanner): decouple port discovery from version detection
nmap -sV sends plaintext probes to TLS ports (e.g. Proxmox 8006). They
stall until the host crosses --host-timeout, at which point nmap skips
the whole host and discards ports it already found — a reachable host is
returned with 0 open ports (issue #277).

Split Phase 2 into two passes: Pass A (-sS/-sT --open) discovers ports
fast with no version detection; its ports are authoritative. Pass B
(-sV --host-timeout, scoped to the found ports) enriches banners
best-effort — on timeout/failure the discovered ports are kept with
empty banners instead of losing the host.

Healthy hosts still get full-intensity -sV banners (no quality drop).
Version host-timeout is now configurable via SCANNER_VERSION_HOST_TIMEOUT
(default 60s).

ha-relevant: yes
2026-07-18 00:35:27 +02:00
Peter Becker 9000f82213 add search for nodes and properties 2026-07-17 23:00:20 +02:00
Pouzor - Rémy JardientandGitHub 587f7f3e61 Merge pull request #293 from Pouzor/feat/zone-modal-two-column
feat: widen zone modal into two columns
2026-07-17 22:05:45 +02:00
Pouzor 2f91ca3dfd feat(canvas): widen zone modal into two columns
The add/edit zone modal grew tall enough to overflow the viewport.
Apply the same layout as NodeModal: widen to sm:max-w-3xl and split
fields into a Content column (label, font, text/label position, text
size) and a Style column (colors, border style/width, z-order).

ha-relevant: yes
2026-07-17 21:57:07 +02:00
Pouzor - Rémy JardientandGitHub 93e9b68821 Merge pull request #289 from nicolabottini/feat/autosave-canvas
feat: autosave canvas after configurable inactivity delay
2026-07-17 20:41:12 +02:00
Pouzor d63c9e6451 fix(canvas): drive autosave debounce off an edit counter
useAutosave reset its debounce whenever the nodes/edges array reference
changed. Live status polling (setNodeStatus) returns a fresh nodes array
on every message without dirtying the canvas, so during active
monitoring the status churn kept re-arming the timer faster than the
delay and the silent save could be starved indefinitely — the user
thinks autosave ran, but it never fired. Select-only changes reset it
the same way.

Add a monotonic editSeq counter that bumps only on real user edits. The
store's set() is wrapped to increment editSeq whenever an update flips
hasUnsavedChanges to true, so it stays centralised instead of touching
every mutating action. onNodesChange/onEdgesChange now set the dirty
flag only when a genuine edit occurred (not on select/measure), so those
no longer bump the counter. App keys the autosave changeSignals on
[editSeq] instead of [nodes, edges].

Add a regression test asserting editSeq bumps on add/update but not on
setNodeStatus, select, or initial dimensions measure.

ha-relevant: maybe
2026-07-17 20:29:48 +02:00
Pouzor 71ab3bf391 fix(canvas): stop non-edits from marking the canvas unsaved
Autosave saves whenever hasUnsavedChanges is set, so any source that
dirties the canvas without a user edit turns into a spurious silent
save. Two such sources:

1. Live status updates. useStatusPolling routed backend status pushes
   through updateNode, which always sets hasUnsavedChanges. With the
   60s status cycle, an idle opted-in tab saved itself every cycle and
   could clobber edits made elsewhere with its stale in-memory canvas.
   Add setNodeStatus, a live-overlay action that updates a node's
   status/response_time/last_seen without dirtying (mirrors
   setServiceStatuses), and route polling through it.

2. Initial dimensions measure. onNodesChange treated React Flow's
   first-measure `dimensions` change as an edit, marking a freshly
   loaded canvas dirty before any interaction — autosave then saved on
   every load/switch. Only count a `dimensions` change as an edit when
   it is a real resize (resizing === true).

Add regression tests: setNodeStatus updates status without dirtying;
onNodesChange ignores initial-measure dimensions and select-only
changes but still dirties on a user resize.

ha-relevant: maybe
2026-07-17 13:02:14 +02:00
Pouzor 9c0ed32485 Merge remote-tracking branch 'origin/main' into feat/autosave-canvas
# Conflicts:
#	frontend/src/App.tsx
2026-07-17 11:58:56 +02:00
Pouzor 8d752e0bd8 fix(canvas): gate autosave on canvas provenance, not selection
The previous guard compared the pinned design against the live
activeDesignId (the selection). On a design switch the selection flips
synchronously while the new canvas loads asynchronously, so the
on-screen nodes briefly still belong to the previous design — arming the
timer with the newly-selected id would save those stale nodes under the
wrong design. Track the design the canvas was actually loaded as
(loadedDesignId provenance) and gate autosave on that instead: pin it
when the timer is armed and re-check the live provenance at fire time,
skipping the save if a different canvas has since loaded.

Also validate the persisted `enabled` flag by type (mirror of `delay`),
and add the matching regression tests (provenance switch skip, enabled
type validation).

ha-relevant: maybe
2026-07-17 11:43:42 +02:00
Pouzor 310b9cb3fd fix(canvas): guard autosave against design-switch clobber, add tests
Extract the debounced autosave into a useAutosave hook. Pin the active
design id when the timer is armed and re-check it at fire time: if the
user switched designs while the timer was pending, the in-memory canvas
belongs to a different design, so skip the save instead of writing it
under the wrong design id. Also skip when there is no active design.

Add unit tests for autosaveSettings (persistence, validation, pub/sub)
and useAutosave (debounce, enable/unsaved/design guards, switch skip).

ha-relevant: maybe
2026-07-17 11:16:56 +02:00
Pouzor - Rémy JardientandGitHub fceefe6d4d Merge pull request #290 from Pouzor/fix/undo-auto-layout-and-yaml-import
fix: allow undo after Auto Layout and YAML import
2026-07-16 19:39:40 +02:00
Pouzor ba4f8b0be4 fix(canvas): allow undo after Auto Layout and YAML import
Auto Layout and YAML import routed through loadCanvas, which wipes
past/future history and clears the unsaved flag (design-switch
semantics). Undo was therefore impossible after either action (#280).

Add an in-place applyLayout store action that snapshots history, marks
the canvas unsaved and re-fits the view without discarding undo history,
and wire both Auto Layout and YAML import to it.

ha-relevant: yes
2026-07-16 19:29:41 +02:00
Nicola Bottini a7b7327ce5 feat: autosave canvas after configurable inactivity delay
Adds an opt-in autosave feature (disabled by default) that silently
saves the canvas after a configurable period of inactivity.

- frontend/src/utils/autosaveSettings.ts (new) — localStorage-backed
  setting {enabled, delay} with pub/sub CustomEvent pattern (mirrors
  alignmentSettings.ts). Key: homelable.autosave.
- frontend/src/App.tsx:
  * handleSave accepts optional { silent?: boolean } — skips the success
    toast when called from autosave to avoid noise on every periodic save
  * autosave state (useState + subscribeAutosaveSettings)
  * debounce useEffect: resets on nodes/edges change; when quiet for
    autosave.delay seconds and hasUnsavedChanges is true, calls
    handleSaveRef.current silently
- frontend/src/components/modals/SettingsModal.tsx:
  * Canvas section: Autosave canvas toggle + Save after delay selector
    (3/5/10/30/60 s). Changes persist immediately to localStorage via
    writeAutosaveSettings + propagate cross-tab via CustomEvent.

Default: disabled — no behaviour change for existing users.
Manual Ctrl+S and the Save button continue to work normally.
Errors always show a toast regardless of the silent flag.
2026-07-16 12:22:18 -04:00
Pouzor - Rémy JardientandGitHub 6ae8b91934 Merge pull request #288 from Pouzor/fix/edge-waypoints-follow-dragged-node
fix(canvas): move edge waypoints when the connected node or its container is dragged
2026-07-16 17:36:24 +02:00
Pouzor 3cab115a80 fix(canvas): move edge waypoints when the connected node or its container is dragged
Waypoints are stored as absolute canvas coords, so they stayed put when a
connected node was dragged, kinking a previously clean route. A dragged
container moved its children on screen too (their stored parent-relative
position is unchanged), so their edges were missed entirely.

onNodesChange now translates the waypoints of every edge touching a moved
node by that node's delta, propagating a container's delta to all
descendants and translating once when both endpoints ride along.

ha-relevant: yes

Fixes #279
2026-07-16 17:24:52 +02:00
Pouzor - Rémy JardientandGitHub 9e088ca7ac Merge pull request #281 from nicolabottini/feat/mcp-approve-delete-design
feat(mcp): design_id on approve_device + delete_design tool
2026-07-16 17:04:25 +02:00
Pouzor - Rémy JardientandGitHub 6cc8399dca Merge pull request #287 from Pouzor/fix/container-host-edit-height-reset
fix(canvas): keep container host height when editing an unrelated field
2026-07-16 17:02:18 +02:00
Pouzor 4956e8ab90 fix(canvas): keep container host height when editing an unrelated field
Editing any field on a container-mode vm/lxc/docker_host node (e.g. just
renaming it or adding a property) reset its manually-set height to
undefined, snapping the container back to auto-fit content size and
scrambling/overflowing its nested children.

Root cause: updateNode clears height on every 'properties' change and the
node modal always includes properties in its payload. Only proxmox was
excluded from the reset; vm/lxc/docker_host in container mode were not.

- export CONTAINER_MODE_TYPES from virtualEdgeParent (reuse, no re-declare)
- updateNode: skip the height reset for any container-mode host type;
  proxmox stays always-excluded (legacy), plain leaf nodes still auto-fit
- regression tests: manual height preserved for container-mode
  vm/lxc/docker_host and proxmox; still reset for a non-container vm

Fixes #278

ha-relevant: yes
2026-07-16 16:53:07 +02:00
Pouzor - Rémy JardientandGitHub d5cbdce870 Merge pull request #286 from nicolabottini/fix/container-host-edit-child-positions
fix(canvas): keep container children in place when editing a host node
2026-07-16 16:11:46 +02:00
Nicola Bottini 15b7ce4481 fix(canvas): keep container children in place when editing a host node
Editing any field on a container-mode host node (Proxmox/VM/LXC/docker)
collapsed all of its nested children into the top-left corner.

Root cause: the node modal always includes container_mode in its update
payload, and handleUpdateNode gated setProxmoxContainerMode on the field's
presence rather than an actual change. So every edit (e.g. changing the
icon) re-fired the container ON transition, which re-applied the
absolute->relative position conversion to children that were already
parent-relative -- piling them up in a corner.

- App.tsx: only call setProxmoxContainerMode when container_mode actually
  changes (compare against the pre-update value).
- canvasStore: make setProxmoxContainerMode per-child idempotent so a
  redundant ON/OFF call can never re-convert an already-nested child's
  position.
- add regression test for the idempotent ON transition.
2026-07-14 19:32:29 -04:00
Nicola Bottini 8df7f682e0 feat(mcp): add delete_design tool
Wires DELETE /api/v1/designs/{id} into the MCP layer.
The backend already implements the endpoint and prevents
deleting the last remaining design.

- mcp/app/tools.py: Tool schema + _dispatch case
- mcp/tests/test_tools.py: 2 new tests (dispatch + schema)
30/30 tests passing.
2026-07-13 15:15:10 -04:00
Nicola BottiniandNicola Bottini 30619caf0f feat(mcp): add design_id to approve_device tool
The backend approve endpoint already accepts design_id via NodeCreate,
but the MCP schema didn't expose the field so callers couldn't target
a specific canvas when approving a discovered device. Add _DESIGN_ID_FIELD
to approve_device so AI clients (and the MCP bridge) can pass it through.
2026-07-13 15:15:10 -04:00
Pouzor - Rémy JardientandGitHub 47e144ac1f Merge pull request #275 from Pouzor/fix/mcp-readme-transport-http
docs: fix MCP client transport to http (was sse)
2026-07-12 00:07:49 +02:00
Pouzor 5fada99e25 docs: fix MCP client transport to http (was sse)
Server uses Streamable HTTP (StreamableHTTPSessionManager), but README
told users to register the client with --transport sse / "type": "sse".
An SSE client waits for an endpoint event the server never emits, so
Claude Code reports "Failed to connect" despite a healthy container.

Switch all client config snippets to http transport and point at the
trailing-slash /mcp/ URL to avoid the 307 redirect that can drop the
auth header.

Fixes #274

ha-relevant: no
2026-07-12 00:04:59 +02:00
Pouzor - Rémy JardientandGitHub 2e4b353fcd Merge pull request #273 from MikeSviblov/fix/mcp-list-pending-status-filter
fix: list_pending_devices MCP tool leaked approved/hidden inventory rows
2026-07-11 23:54:46 +02:00
Pouzor d0e5c105ec feat: add inventory, hidden-device and restore MCP tools
The status filter on list_pending_devices closed the leak but left the
MCP surface unable to see anything beyond pending: approved inventory
and hidden devices were no longer reachable, and a wrongly hidden device
could not be recovered through the MCP at all.

Add three tools mirroring existing backend endpoints:
- list_inventory      GET  /scan/pending  (pending + approved, optional status filter)
- list_hidden_devices GET  /scan/hidden
- restore_device      POST /scan/pending/{id}/restore  (undo hide_device)

Bulk approve/hide/restore and scan-config endpoints are intentionally
left out: bulk mutation is the same blast radius that caused the mass-hide
incident this PR addresses.

Tests: 5 new (47 passed total).
2026-07-11 23:40:03 +02:00
Mike SviblovandClaude Fable 5 9a03097f80 fix: list_pending_devices MCP tool leaked approved/hidden inventory rows
The backend GET /scan/pending endpoint intentionally returns the whole
inventory — approved devices stay listed so the frontend can show the
canvas-presence badge. The MCP tool proxied that response verbatim while
its description promises devices "not yet approved or hidden", so MCP
clients saw every approved device as if it were still awaiting triage
(59 "pending" entries on a fully-triaged canvas, 58 of them approved).

Filter the tool response to status == "pending", keeping legacy rows
that predate the status field.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Udrvtwi3cqcxUNHZtBHkBg
2026-07-11 15:17:45 +03:00
Pouzor - Rémy JardientandGitHub 228f2a67e4 Merge pull request #271 from Pouzor/release/3.0.0
chore: bump version to 3.0.0
2026-07-10 17:37:43 +02:00
Pouzor 9331bc43cb chore: bump version to 3.0.0
Release 3.0.0. See CHANGELOG.md for full details.
2026-07-10 17:06:20 +02:00
Pouzor - Rémy JardientandGitHub f98a55dca0 Merge pull request #270 from Pouzor/feat/zigbee-zwave-autosync
feat: scheduled auto-sync for Zigbee & Z-Wave imports
2026-07-10 16:20:02 +02:00
Pouzor 5139daef90 fix: satisfy mypy in _run_mesh_sync branch typing
CI mypy flagged the shared payload/background locals in _run_mesh_sync
as incompatibly typed across the zigbee/zwave branches. Annotate them
with the union / generic Callable and alias the per-branch
env_import_request imports so the two bindings don't clash.
2026-07-10 15:58:47 +02:00
Pouzor d3728f4108 fix: widen Settings modal to two columns; env cleanup
- SettingsModal: widen to sm:max-w-3xl with max-h-[90vh] scroll and a
  two-column grid so it no longer overflows the viewport. Left column =
  status/service checks + canvas prefs; right column groups all auto-sync
  config (Zigbee, Z-Wave, Proxmox).
- .env.example: drop ZIGBEE_/ZWAVE_ SYNC_ENABLED + SYNC_INTERVAL — the
  auto-sync activation is configured in the Settings modal (persisted to
  scan_config.json), same as Proxmox; connection config stays env-only.

ha-relevant: no
2026-07-10 15:45:46 +02:00
Pouzor b99450db2f feat: scheduled auto-sync for Zigbee & Z-Wave imports
Mirror the Proxmox auto-sync pattern for the Zigbee2MQTT and Z-Wave JS
UI mesh imports. Connection config + MQTT credentials live in .env only,
are never persisted to scan_config.json, and are never returned by any
API or shown in the UI (single source of truth).

- config: ZIGBEE_* / ZWAVE_* env settings; only sync_enabled+interval
  are persisted, connection/credentials stay env-only
- routes: GET/POST /config, POST /sync-now; auto-sync reuses the exact
  manual _background_*_import + _persist_pending_import path (fresh
  import when empty, update-in-place when nodes exist, ScanRun trace)
- scheduler: zigbee_sync / zwave_sync jobs with live enable + reschedule
- frontend: reusable MeshAutoSync section in Settings (Zigbee, Z-Wave)
- .env.example: documented both blocks
- tests: scheduler jobs, router config/sync-now/auth, credential-never-
  persisted, SettingsModal sections

Manual Zigbee/Z-Wave import behaviour is unchanged.

ha-relevant: no
2026-07-10 14:51:01 +02:00
Pouzor - Rémy JardientandGitHub a90ca2f039 Merge pull request #269 from Pouzor/fix/proxmox-autosync-scan-history
fix: record ScanRun for scheduled Proxmox auto-sync
2026-07-10 13:42:16 +02:00
Pouzor ad4aa4aba4 fix: record ScanRun for scheduled Proxmox auto-sync
The scheduled auto-sync job called _persist_pending_import directly and
never created a ScanRun, so auto-imports left no trace in Scan history
(unlike manual /sync-now and /import-pending, which both record a run).

Create a ScanRun(kind=proxmox) and delegate to the shared
_background_proxmox_import flow — fetch, persist, mark the run
done/error, and broadcast the inventory-reload signal.

ha-relevant: maybe
2026-07-10 12:47:16 +02:00
Pouzor - Rémy JardientandGitHub c4cb44709d Merge pull request #268 from Pouzor/refactor/frontend-test-consolidation
test: consolidate and restructure frontend tests
2026-07-10 10:26:24 +02:00
Pouzor 24e755a097 docs: add frontend test infra README (factories, mocks, split conventions)
ha-relevant: yes
2026-07-10 03:04:47 +02:00
Pouzor 075f7f6d78 test: dedup sonner mocks and close Toolbar seam-mock gaps
- Replace 10 inline sonner toast mocks with the shared mockSonner() builder.
- Toolbar: add tests for the store-driven undo/redo disabled state and the
  unsaved-changes dot — behaviour the full-store mock previously left
  unasserted.

ha-relevant: yes
2026-07-10 02:59:38 +02:00
Pouzor 4124d6c5a4 test: split canvasStore monolith and relocate misnamed util test
- Split the 1430-line canvasStore.test.ts into a canvasStore/ subpackage by
  concern (nodes, containers, sizing, edges, selection, grouping, history,
  clipboard, customStyle, floorMap); each file <350 lines. Test bodies are
  unchanged; local makeNode/makeEdge replaced by the shared factories.
- Move panels/DetailPanel.test.ts -> utils/serviceUrl.test.ts: it tested
  getServiceUrl, not DetailPanel.

Same 1396 tests, all green.

ha-relevant: yes
2026-07-10 02:48:29 +02:00
Pouzor 64c48de1d1 test: add shared frontend test infra (factories, mocks, render)
Foundation for the frontend test consolidation (mirrors backend #267):
- src/test/factories.ts: canonical makeNode/makeEdge/makeNodeData/makeDesign
- src/test/mocks/: reusable mockSonner/mockReactFlow/makeUseCanvasStore builders
  consumed via async vi.mock dynamic-import to survive hoisting
- src/test/render.tsx: renderWithProviders wrapping Tooltip + ReactFlow providers

Migrate ScanConfigModal sonner mock as canary. No behavior change.

ha-relevant: yes
2026-07-10 02:15:49 +02:00
Pouzor - Rémy JardientandGitHub 012ba9cb97 Update README.md 2026-07-10 01:41:51 +02:00
Pouzor - Rémy JardientandGitHub 6644a208bb Update README.md 2026-07-10 01:39:42 +02:00
Pouzor - Rémy JardientandGitHub 54647413c4 Update README.md 2026-07-10 00:53:16 +02:00
Pouzor - Rémy JardientandGitHub 93a50cf144 Merge pull request #266 from nicolabottini/feat/mcp-design-targeting
feat: MCP design/canvas targeting + auto-position + auto-edge-handles
2026-07-10 00:46:53 +02:00
Pouzor b7985306cd test: cover auto-position and auto-edge-handle; fix negative-cell clamp
Add backend API tests for the two issue #265 features that shipped
untested: auto-positioning root nodes into a free grid slot (first at
origin, collision avoidance, child origin default, explicit coords and
explicit zero preserved) and auto-assigning edge handles from absolute
canvas Y (source above/below/equal target, partial handle fill, child
abs-Y resolved through parent).

Drop the max(0, round()) clamp in _find_free_position: it folded
negative-positioned nodes onto cell (0,0), falsely blocking or freeing
the origin slot. Negative nodes now keep their true cells and never
intersect the positive search space.
2026-07-10 00:16:32 +02:00
27e18f1c96 feat: auto-position nodes and auto-assign edge handles on create
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-10 00:02:11 +02:00
2d00be71bb feat: add create_design tool so the MCP can create canvases
Wrap POST /api/v1/designs so AI clients can create a new design (canvas) and get its id back for use as design_id. Completes the create-and-target canvas workflow alongside list_designs.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-09 23:58:28 +02:00
6e3d45fddc feat: let the MCP target a specific design/canvas
Add an optional design_id argument to create_node, create_edge and get_canvas so AI clients can read and populate a canvas other than the first design. Add a list_designs tool wrapping GET /api/v1/designs so those IDs are discoverable.

Backward compatible: omitting design_id preserves the existing first-design fallback in the backend (nodes.py / edges.py / canvas.py).

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-09 23:58:28 +02:00
Pouzor - Rémy JardientandGitHub fd3a3f1da6 Merge pull request #267 from Pouzor/refactor/unify-test-auth-fixture
test: refactor backend test suite for quality and consolidation
2026-07-09 23:39:27 +02:00
Pouzor 9205f9d36e test: close coverage gaps hidden by seam mocking
Audit of the heavy-mock suites (scanner/status_checker/scheduler)
confirmed the mocking is sound layered seam-mocking, not brittle
internal patching. It did hide real gaps where a primitive was always
mocked and never exercised:

- status_checker._http_get: add status-code interpretation tests
  (2xx/4xx online, 5xx offline) + service-check exception -> offline.
  Module coverage 95% -> 100%.
- scheduler.reschedule_status_checks / reschedule_service_checks were
  wholly untested; add validation + guard tests. Add _run_proxmox_sync
  happy-path and error-swallow tests. Module coverage 76% -> 91%.

639 passed.

ha-relevant: no
2026-07-09 22:14:36 +02:00
Pouzor 991169ba5a test: consolidate migration tests into a migrations/ subpackage
Group the three upgrade-path suites (legacy->designs, hardware->
properties, pre-migration DB backup) under tests/migrations/ with
clearer names. Pure relocation via git mv; each keeps its own
self-contained fixtures. No behavior change; 627 passed.

ha-relevant: no
2026-07-09 21:51:48 +02:00
Pouzor b513fa6f4e test: split monolithic test_scan into a scan/ subpackage
Break the 1897-line test_scan.py into topic modules under tests/scan/
(routes, approve, properties, run) sharing fixtures via a package
conftest.py and pure builders via helpers.py. Same 80 tests, no
behavior change; full suite 627 passed.

ha-relevant: no
2026-07-09 21:41:49 +02:00
Pouzor 3384a05932 test: unify auth headers into a single conftest fixture
Replace 9 duplicated per-file 'headers' fixtures and the awkward
'auth_headers' coroutine factory with one canonical 'headers' fixture
in conftest.py. Migrate liveview/media tests off the await-based
factory. No behavior change; 627 passed.

ha-relevant: no
2026-07-09 21:26:45 +02:00
Pouzor - Rémy JardientandGitHub 9051f6ca3e Merge pull request #264 from Pouzor/docs/changelog
docs: add CHANGELOG with full release history
2026-07-09 20:39:47 +02:00
Pouzor d9c6a4ea31 docs: add CHANGELOG with all release history
Adds CHANGELOG.md covering every GitHub release from v1.0.0 to v2.6.1,
grouped by Added/Changed/Fixed/Security in Keep a Changelog style.

ha-relevant: no
2026-07-09 20:35:02 +02:00
Pouzor - Rémy JardientandGitHub 160690bad8 Merge pull request #263 from Pouzor/feat/design-url-sync
feat: sync active design to URL for refresh/share
2026-07-09 17:09:27 +02:00
Pouzor 1aecb0a4d9 feat: sync active design to URL for refresh/share
Switching designs now reflects the active design id in the URL
(?design=<id>). A page refresh or shared link reopens that design;
an absent or unknown id falls back to the default design.

ha-relevant: no
2026-07-09 16:29:04 +02:00
Pouzor - Rémy JardientandGitHub 15b000d555 Merge pull request #262 from Pouzor/fix/258-match-device-by-ip-token-and-mac
fix: match scanned devices to canvas nodes by ip-token and mac (#258)
2026-07-09 15:12:13 +02:00
Pouzor 2d3b646e45 fix: match scanned devices to canvas nodes by ip-token and mac (#258)
Node.ip stores several comma-separated addresses once a user adds an
IPv6 (e.g. "fe80::1, 192.168.1.5"). All placement/inventory matching did
exact string equality on that field, so a device scanned as the plain
IPv4 looked absent from the canvas: canvas_count stayed 0, the "In N
canvas" badge and hide-on-canvas button vanished, and bulk-approve
re-placed a duplicate node.

Match per-address instead, and add MAC (a stable identifier immune to IP
edits) as a cumulative match key alongside ieee/ip:
- _canvas_correlation indexes ip per token + by_mac
- bulk_approve skip detection tokenizes ip + tracks mac
- find_duplicate_node narrows with Node.ip.contains then confirms per
  token in Python (guards the 10.0.0.4 / 10.0.0.40 substring false match)

ha-relevant: maybe
2026-07-09 14:41:53 +02:00
Pouzor - Rémy JardientandGitHub 2bebe8f42d Merge pull request #261 from Pouzor/feat/single-approve-duplicate-prompt
feat: prompt on duplicate device instead of silently blocking/merging approve
2026-07-09 13:54:56 +02:00
Pouzor 0792105b96 feat: prompt on duplicate device instead of silently blocking/merging approve
Single-device approve now guards duplicates per-design the same way bulk
approve does, and asks the user instead of failing or silently merging:

- create_node and approve_device reject a same-design duplicate (ieee, ip
  or mac) with 409 + the existing node; a force flag creates it anyway.
- Frontend shows a confirm dialog: go to existing node, add duplicate
  anyway, or cancel.
- approve_device no longer rejects a device already on another canvas
  (status is global, canvas membership is per-design) — it can be placed
  on a new design, matching bulk approve.
- IEEE (Zigbee/Z-Wave) devices now use the same prompt as ip/mac instead
  of auto-merging into the existing node.
- bulk approve reports which devices it skipped as duplicates.

Closes #260

ha-relevant: maybe
2026-07-09 13:05:19 +02:00
Pouzor - Rémy JardientandGitHub 8ebc716de8 Merge pull request #259 from Pouzor/feat/proxmox-resync-and-inventory-fix
feat: manual Proxmox re-sync + env-only connection config + inventory approve fix
2026-07-07 23:21:22 +02:00
Pouzor b09ebb5cd2 fix: keep approved devices in inventory after bulk approve
Bulk/single approve optimistically stripped approved rows from the local list,
but scanApi.pending() still returns them (on-canvas, with an 'In N canvas'
badge). The list went empty until the modal was reopened. Reload after approve
instead of stripping, so approved rows stay visible with a fresh canvas_count.

ha-relevant: yes
2026-07-07 21:23:28 +02:00
Pouzor 9437a74147 feat: manual Proxmox re-sync button + make connection config env-only
Add a 'Re-sync now' button to the Proxmox auto-sync settings section that
triggers an immediate inventory import (POST /proxmox/sync-now) using the
server env config — the manual counterpart to scheduled auto-sync.

Also fix a dual-source-of-truth bug: Proxmox connection config (host, port,
token, verify_tls) is now env-only and never persisted to scan_config.json.
Previously save_overrides() dumped host/port/verify alongside scan settings,
so saving an unrelated setting wrote an empty proxmox_host that load_overrides
then clobbered PROXMOX_HOST with on every boot. Only the auto-sync activation
(sync_enabled + sync_interval) stays user-editable and persisted.

ha-relevant: no
2026-07-07 21:20:13 +02:00
Pouzor - Rémy JardientandGitHub 4206d50c70 Merge pull request #257 from Pouzor/feat/canvas-copy-from-existing
feat: create a new canvas by copying an existing one
2026-07-07 16:23:51 +02:00
Pouzor 0b4bd5680d feat: create a new canvas by copying an existing one
Add a 'Copy from existing' option to the New Canvas modal. It lists every
canvas with node/group/text counts; picking one deep-copies its nodes, edges,
parent/child links and canvas state (viewport, custom style, floor plan) into
a fresh design.

Backend: POST /designs/{source_id}/copy remaps node ids, re-points edges and
parent links, and clones canvas state; GET /designs now returns per-design
counts for the picker. Standalone mode clones the localStorage canvas.

Closes #216

ha-relevant: maybe
2026-07-07 15:53:42 +02:00
Pouzor - Rémy JardientandGitHub 6b591cdd88 Merge pull request #256 from Pouzor/fix/media-path-injection
fix: harden media path handling against path injection
2026-07-07 14:55:27 +02:00
Pouzor 7db57bba3b fix: resolve media file by directory listing to kill path-injection taint
Match the validated filename against iterdir() entries with == instead of
building a path from the user string, so no tainted value ever reaches a
filesystem sink. Satisfies CodeQL py/path-injection.

ha-relevant: no
2026-07-07 14:35:44 +02:00
Pouzor b907c4b05e fix: use re.fullmatch for media filename allowlist so CodeQL recognizes barrier
ha-relevant: no
2026-07-07 14:24:19 +02:00
Pouzor 2d376c2bed fix: harden media path handling against path injection (CodeQL py/path-injection)
Resolve media filenames through a shared _resolve_media_path() barrier that
confirms the resolved path sits directly under the resolved media dir, so
CodeQL can trace the sanitization the regex already guaranteed.

ha-relevant: no
2026-07-07 11:50:12 +02:00
Pouzor - Rémy JardientandGitHub 738595be5a Merge pull request #255 from Pouzor/fix/yaml-export-connection-points
fix: preserve edge connection points in YAML export/import (#208)
2026-07-07 11:38:34 +02:00
Pouzor 7bc3e5d8a0 fix: preserve edge connection points in YAML export/import (#208)
YAML round-trip dropped every edge's handles: export never wrote them
and import hardcoded sourceHandle='bottom'/targetHandle='top-t'. After
import all connections collapsed onto slot 0 ("converge at a single
point"). Per-side handle counts were dropped too, so the extra bottom
slots did not even exist to attach to.

- yaml types: add sourceHandle/targetHandle to connections and
  top/bottom/left/rightHandles to nodes
- export: write each edge's real handles + per-side counts (only above
  the side default); orient parent-edge handles parent->child
- import: restore handle counts onto node data and use the stored
  handles, falling back to the legacy defaults for pre-existing YAML

Positions remain dagre-managed (unchanged); this restores connection
points only.

ha-relevant: yes
2026-07-07 11:16:29 +02:00
Pouzor - Rémy JardientandGitHub 65a1f49f93 Merge pull request #254 from Pouzor/fix/mesh-edges-second-canvas
fix: keep mesh/cluster links so edges resolve onto a second canvas
2026-07-07 10:45:21 +02:00
Pouzor 5b5eabf5db fix: keep mesh/cluster links so edges resolve onto a second canvas
Approving the same zigbee/zwave/proxmox devices onto a second design
placed the nodes but drew no edges. _resolve_pending_links_for_ieee
deleted each pending_device_link after materializing its edge, so the
first approve consumed the whole topology and later approves had nothing
to resolve.

Links are topology, not one-shot: every importer wipes+reinserts its
link set on each import, so they can safely persist across approvals.

- keep the link rows (drop both db.delete(link) calls)
- scope resolution to the target design (Node.ieee_address + design_id)
  so a re-approve links that canvas's nodes, not another's
- thread design_id through all three approve call sites

Existing links deleted by the old code do not come back on their own;
a re-import repopulates them.

ha-relevant: maybe
2026-07-07 10:32:47 +02:00
Pouzor - Rémy JardientandGitHub 601f731135 Merge pull request #253 from Pouzor/feat/proxmox-import
feat: import hosts/VMs/LXC from Proxmox VE with optional auto-sync
2026-07-07 09:48:05 +02:00
Pouzor 3d25fcaae2 update doc 2026-07-07 00:53:06 +02:00
Pouzor adf82f8f01 feat: merge IP-scanned and Proxmox-imported devices by MAC
Reconcile the same physical device discovered by both the nmap IP scan and
the Proxmox importer into a single inventory row, keyed on MAC. Previously
each path only deduped by IP, and the importer captured no MAC (and no IP for
stopped guests), so most guests double-listed.

Backend:
- mac_utils.normalize_mac: canonical MAC (lowercase, ':'-separated), the
  cross-source join key. Normalized on write and on compare.
- proxmox_service: capture the guest NIC MAC agent-free from the net0 config
  (qemu virtio=<MAC>, lxc hwaddr=<MAC>); works for stopped guests. Resolver
  now returns (ip, mac).
- proxmox persist: match existing Node/PendingDevice by ieee OR ip OR MAC;
  fill mac, keep the vm/lxc type, union sources.
- scanner persist: match PendingDevice by ip OR MAC; fill the IP a Proxmox
  import lacked, keep a pve row's type, union the scan source. Stamp query
  matches raw + normalized MAC (legacy-safe).
- Multi-source tags: new PendingDevice.discovery_sources JSON column so a
  merged device shows under both the IP and Proxmox filters. Idempotent
  migration backfills from discovery_source (legacy NULL-scalar rows with an
  IP become ["arp"]). _sources_after_merge preserves a scanned row's IP origin
  through the merge without tagging a pure Proxmox guest.
- Import now broadcasts a scan update on completion so an open inventory
  reloads without a manual refresh.

Frontend:
- pendingSources: sourceBuckets/orderedSources map discovery_sources to filter
  buckets; a device with ["arp","proxmox"] matches both filters and renders
  both badges. PendingDevicesModal filter + badges use them.

Tests: MAC normalization, config MAC capture, cross-source merge both
directions, legacy-row IP-tag preservation, no-false-IP-tag guard, refresh
broadcast, and the frontend bucket mapping.

ha-relevant: maybe
2026-07-06 20:11:03 +02:00
Pouzor 05ef746f22 fix: cluster edges render on left/right handles from approve flow
Cluster edges created via the pending -> approve path rendered on the top
handle instead of left/right, because the edge and its endpoints lost their
handle information on the way to the canvas.

- Approve resolver (scan.py) now returns each edge's type + source/target
  handle. Handle IDs are the bare slot-0 side names ('right'/'left'), the
  canonical stored form React Flow resolves to the correct side; a '-t' target
  id fails to resolve and falls back to the top handle.
- Frontend injectAutoEdges no longer hardcodes iot/bottom/top-t. It injects
  each edge with its real type + handles and bumps the referenced nodes'
  left/right handle counts (which default to 0) so the cluster endpoints exist.
  Logic extracted to a pure, tested util (applyAutoEdges).
- clusterEdges direct-import path uses the bare 'left' target to match.

Tests: new autoEdges unit tests; updated backend handle assertions.

ha-relevant: maybe
2026-07-06 11:13:32 +02:00
Pouzor 9670d0a86a feat: proxmox import diagnostics, node style fix, and cluster edges
Scan History:
- Proxmox is now a first-class scan kind (badge, filter chip, Server icon,
  completion toast) instead of being mislabeled as an IP scan.
- A done run carrying a non-fatal advisory renders amber (info) with a
  warning toast, distinct from red failures.

Import diagnostics:
- test-connection probes /access/permissions and warns when the API token
  has no ACL (VMs/LXC would be invisible) — points at the PVEAuditor grant.
- import surfaces an advisory when hosts import but no guests are visible
  (privilege-separated token whose rights are the intersection with the user),
  rather than a silent "done".

Node style:
- Proxmox container mode is now opt-in (container_mode === true), matching the
  rest of the codebase (App.tsx nesting logic). Imported proxmox nodes leave
  the flag unset and render like a manually-created node instead of an empty
  group container.

Cluster edges:
- Hosts from one import are chained with 'cluster' edges via left/right handles,
  distinct from the vertical host->guest 'virtual' edges. Wired for both the
  direct "Add to Canvas" path and the pending -> approve path (host<->host
  proxmox_cluster links, resolved to cluster edges on approve; cluster hosts
  get left/right handles).

Tests added on both sides.

ha-relevant: maybe
2026-07-06 10:26:52 +02:00
Pouzor abefc42fdf fix: tolerate legacy NULL properties on pending devices
The pending_devices.properties column is added by an idempotent migration, so
existing rows have properties = NULL. PendingDeviceResponse typed it as a list,
so GET /scan/pending 500'd on any pre-existing device.

- Coerce NULL/non-list properties to [] in PendingDeviceResponse.
- Backfill existing NULL rows to '[]' in init_db migrations.
- Regression test: /scan/pending returns 200 with a legacy NULL-properties row.

ha-relevant: maybe
2026-07-06 00:01:27 +02:00
Pouzor ab36ba6f81 feat: import hosts/VMs/LXC from Proxmox VE with optional auto-sync
Add a Proxmox VE importer that reads the /api2/json REST API with a read-only
API token and drops hosts (proxmox), VMs (vm) and LXC containers (lxc) onto the
canvas as typed nodes with run state and hardware specs (vCPU/RAM/disk).

- Backend: proxmox_service (httpx) + proxmox routes (test-connection, import,
  import-pending, config). Two-tier dedupe — merge onto an existing scanned node
  by IP, else synthetic pve-{host}-{vmid} identity. Update-in-place, never
  deletes. Host->guest rendered as a 'virtual' edge via the pending-link flow.
- Security: token is env-only (PROXMOX_TOKEN_*), never written to disk by the
  app, never returned by any endpoint; errors are credential-sanitized.
- Auto-sync: optional scheduled re-import into pending (APScheduler job).
- PendingDevice.properties carries specs through approve (+ migration).
- Frontend: ProxmoxImportModal, sidebar entry, pending inventory source filter,
  Settings auto-sync section, proxmoxApi client.
- Docs: docs/proxmox-import.md, README + FEATURES sections, .env.example keys.
- Tests: backend service/router/scheduler, frontend modal/client/pending.

ha-relevant: maybe
2026-07-05 18:58:12 +02:00
Pouzor - Rémy JardientandGitHub 1d40d70150 Merge pull request #252 from Pouzor/feat/configurable-edge-line-style
feat: configurable edge line style + width per type and per edge
2026-07-05 15:19:54 +02:00
Pouzor 485d2f2b04 feat: configurable edge line style + width per type and per edge
Edge render (solid/dashed/dotted) and stroke width were hardcoded per
edge type. Expose both as user settings.

- Custom Style modal (Edges): line-style buttons, 1-4x width slider,
  live preview; left-list swatch renders the actual line.
- Per-edge EdgeModal: same controls; line style follows the type preset
  live until overridden.
- Renderer applies line_style/width_mult over BASE_STYLES (width scales
  markers + animation overlays); unset keeps the type default look.
- Persist line_style/width_mult through serializer, canvas save, and the
  edges API (new nullable columns, idempotent migration).

ha-relevant: yes
2026-07-05 14:11:01 +02:00
Pouzor - Rémy JardientandGitHub 40ec26ab7e Merge pull request #250 from Pouzor/feat/edge-arrowheads
feat: edge endpoint marker shapes + fix parallel edges not rendering
2026-07-05 11:59:45 +02:00
Pouzor c95d104245 feat: selectable marker shapes per edge endpoint
Replace the on/off arrowhead toggle with a per-end shape picker. Each end
(start/end) independently selects: none, arrow, arrow-open, circle, diamond,
or square. Markers still recolor live from the resolved stroke color.

Frontend:
- MarkerShape type + edgeMarkers util (normalizeMarker, MARKER_GEOMETRY);
  legacy boolean coerces to 'arrow' on read.
- Per-shape <marker> inner geometry; symmetric shapes use fixed orient.
- MarkerShapePicker reused in EdgeModal and CustomStyleModal.
- Serializer normalizes to shape strings.

Backend:
- Edge marker columns Boolean -> String (default 'none'); TEXT migration.
- normalize_marker() + validators coerce legacy bool / unknown values.

ha-relevant: yes
2026-07-05 11:30:49 +02:00
Pouzor 1cf525844b feat: arrowhead endpoints for edges + fix parallel edges not rendering
Add optional filled-triangle arrowheads at either end of an edge,
independently toggleable per edge (EdgeModal) and as per-edge-type
defaults (CustomStyleModal). Arrowheads are custom inline <marker> defs
filled with the live stroke colour so they recolour reactively with
custom_color / vlan / selected state. Persisted frontend (serializer)
and backend (edge columns + schemas + runtime migration).

Also fix two dedupe layers that silently dropped legitimate parallel
links between the same two devices:
- store: React Flow addEdge() connectionExists dropped a second edge
  with matching source+target when handles were null/equal. Build the
  edge with a unique id and append directly.
- render: rewireEdgesForCollapse deduped ALL edges by src->tgt key even
  when nothing was collapsed, filtering real parallel edges out of the
  visible set. Restrict the anti-mesh dedupe to rewired collapse stubs.

Tests: marker render, per-edge/per-type UI, store apply, serializer
round-trip, backend edge/canvas persistence, parallel-edge regressions.

ha-relevant: yes
2026-07-05 01:16:26 +02:00
Pouzor - Rémy JardientandGitHub ae2d3e1eab Merge pull request #249 from Pouzor/feat/customizable-connection-points-per-side
feat: customizable connection points per side (#243)
2026-07-04 15:32:05 +02:00
Pouzor e8530ad3db feat: shortcut from node Appearance to canvas-wide type style
Adds a link under the colour swatches in the add/edit node modal that opens the
Custom Style editor with the node's type preselected, so a user can edit the
style for every node of that type without navigating Style -> Customize -> pick
type manually.

- CustomStyleModal: new optional `initialNodeType` prop; when set, the modal
  opens straight into that node type's editor instead of the empty placeholder.
- NodeModal: new optional `onEditTypeStyle(type)` prop renders the shortcut link
  (hidden for group/groupRect and when no handler is wired).
- App: `styleEditorType` state; both Add and Edit NodeModals forward the handler,
  and a standalone CustomStyleModal instance opens preselected (stacked over the
  node modal so in-progress node edits are preserved).

ha-relevant: yes
2026-07-04 15:14:33 +02:00
Pouzor e39ab7d530 fix: restore connection-handle magnet area (unclip node root)
The overflow-hidden added to the BaseNode root (to tame the oversized 16px
NodeResizer handles) also clipped the connection handles, which sit centred on
the node edge with half their box outside it. CSS overflow:hidden disables
pointer events in the clipped region, so the outer half of every handle — the
side an edge approaches from — became non-interactive, halving the magnet area.

- Remove overflow-hidden from the BaseNode root so handles stay fully grabbable.
- Shrink the NodeResizer handles 16px -> 8px (rounded): the reason the clip was
  added in the first place, so the resizer no longer looks oversized when zoomed
  out without relying on clipping.
- Enlarge the invisible target (magnet) hit area 12px -> 20px.
- Set connectionRadius={30} on ReactFlow for stronger distance-based snapping.

ha-relevant: yes
2026-07-04 12:59:08 +02:00
Pouzor 738eac3ebb fix: restore node overflow-hidden (oversized resize handles)
Removing overflow-hidden from the BaseNode root un-clipped the NodeResizer
handles, making them huge (especially when zoomed out). Clipping was never what
made left/right connection points fail — that was backend persistence (fixed
separately) — and top/bottom handles have always worked while clipped. Restore
overflow-hidden so the resizer looks correct; side handles still work.

ha-relevant: yes
2026-07-04 01:47:16 +02:00
Pouzor 1479c777d0 fix: replace Proxmox cluster handles with configurable side points
The Proxmox node had two always-on cluster-left/cluster-right handles that
rendered independently of the new per-side connection-point config — so a
configured side stacked on top of the forced handle (e.g. 0 config still showed
1; 2 config left the forced one in the middle).

- ProxmoxGroupNode: remove the always-on cluster handles in both container and
  non-container modes; cluster links now use the normal per-side points
- add migrateClusterHandles(nodes, edges): on load, remap any edge still bound
  to cluster-left/right onto the left/right slot-0 point and set that node's
  side count to at least 1, so existing links survive (edge 'cluster' type/style
  kept). Applied on API load, standalone load, LiveView, and YAML import
- App: new cluster links are made by choosing the Cluster edge type in the edge
  modal (drop the cluster-handle auto-default)

ha-relevant: yes
2026-07-04 01:35:28 +02:00
Pouzor 6302c43e06 fix: persist top/left/right connection-point counts (issue #243)
The backend only stored bottom_handles, so top/left/right_handles were dropped
on canvas save and reset to defaults on reload — a left/right snappoint would
vanish after reload.

- models.py: add top_handles (default 1), left_handles (0), right_handles (0)
- database.py: ALTER TABLE migrations for the three columns
- schemas: add fields to NodeBase, NodeUpdate, and the canvas-save node schema
- tests: save+reload round-trip and default-fallback coverage

ha-relevant: no
2026-07-04 00:04:24 +02:00
Pouzor 63e664efdd feat: widen node modal, two-column layout, unclip side handles
- NodeModal: widen to max-w-3xl; split into Information (left) and Design
  (right) columns with headers; Notes moved to left as a resizable textarea
- Connection Points: spatial cross of -/N/+ steppers around a node preview
  (replaces the four stacked sliders)
- BaseNode: drop overflow-hidden from the node root so left/right handles are
  no longer clipped to an ungrabbable sliver (inner sections keep their own
  clipping; rounded corners preserved)

ha-relevant: yes
2026-07-04 00:03:09 +02:00
Pouzor b708a08fd0 feat: customizable connection points per side (issue #243)
Generalize the bottom-only handle machinery to all four sides. Top/Bottom
keep their single-handle default; Left/Right are opt-in (default 0) so
existing diagrams render unchanged. Handle IDs stay keyed off the bare side
name at slot 0, keeping saved edges valid.

- handleUtils: side-generic API (handleId, handlePositions, clampHandles,
  sideDefault, removedHandleIds, sideHandleCount); legacy bottom fns kept as
  delegating aliases
- BaseNode + ProxmoxGroupNode render all sides via new shared SideHandles
- updateNode remaps edges on any side's shrink (fallback to slot-0 / bottom)
- serializer round-trips top/left/right_handles
- NodeModal: spatial cross of -/N/+ steppers around a node preview, replacing
  the four stacked sliders; seeds per-type defaults
- CustomStyleModal: per-type default connection points per side

ha-relevant: yes
2026-07-03 21:25:14 +02:00
Pouzor - Rémy JardientandGitHub c9a35b730d Merge pull request #248 from Pouzor/docs/features-md
docs: add user-facing FEATURES.md
2026-07-03 14:51:24 +02:00
Pouzor d573cffe0e docs: link README to FEATURES.md before install section 2026-07-03 14:44:37 +02:00
Pouzor b59476f2f9 docs: add user-facing FEATURES.md
Feature guide with table of contents: one line per feature plus how to
enable and use it. Covers zones, groups, multiple canvases, style,
floor plan, scanner, zigbee/zwave import, device inventory, live status,
export, live view, gethomepage, MCP, settings.

Untrack FEATURES.md (was ignored as internal meta) now that it ships as
public docs.

ha-relevant: no
2026-07-03 14:30:48 +02:00
Pouzor - Rémy JardientandGitHub 9b40ba0b7c Merge pull request #247 from Pouzor/fix/mesh-import-duplicate-nodes
fix: mesh (zigbee/zwave) import — duplicate-node crash + coordinator to pending
2026-07-03 12:16:47 +02:00
Pouzor 6f82c15c69 fix: backfill inventory row for on-canvas mesh devices missing one
A device already on a canvas (Node exists) but with no pending_devices row
never showed in the discovery inventory — the inventory lists pending_devices,
not nodes. This stranded legacy auto-placed coordinators: on a canvas yet
invisible in the inventory list.

Both mesh imports now, in the already-approved-node branch, ensure an
inventory row exists: create one as status="approved" when missing, or
refresh its metadata (preserving status) when present. New/unplaced devices
still land as status="pending"; hidden rows stay hidden.

Tests updated: approved-node path now backfills an approved inventory row
(zigbee + zwave), plus a regression that a hidden inventory row is not
revived.

ha-relevant: yes
2026-07-03 12:08:53 +02:00
Pouzor 1de6e91ba3 feat: colour device-inventory role badge by node-type accent
The role/type badge on each inventory tile was always flat grey
(bg-[#21262d] text-muted-foreground), which read as disabled and gave no
visual cue about the device kind. Colour it with the same per-type accent
the node uses on the canvas — resolved through the active theme / style
section via resolveNodeColors — so a zigbee_coordinator, zwave_router, etc.
gets its style-section colour (translucent background + solid text).

Applies to every source (IP, zigbee, zwave) since all node types carry an
accent in the theme.

Test: role badge renders with the node-type accent colour, not the grey
muted class.

ha-relevant: yes
2026-07-03 11:13:18 +02:00
Pouzor 3f6e9b00f7 fix: send zigbee/zwave coordinator to pending inventory, not auto-canvas
The coordinator was special-cased in both mesh imports to auto-create a
canvas Node, so it never appeared in the pending inventory and users could
not approve/hide/type it like every other device.

Remove the auto-placement: the coordinator now flows through the shared
pending path (upsert into pending_devices with suggested_type
zigbee_coordinator / zwave_coordinator). An already-approved coordinator
Node still gets its properties refreshed on re-import via the shared
approved-node path. The response's coordinator/coordinator_already_existed
fields are retained (now always unset) for backward-compatible shape; the
frontend already ignored them.

Tests updated: coordinator lands in pending (counts include it), no Node
auto-created, pending metadata carried, approved coordinator refresh + no
pending re-list.

ha-relevant: yes
2026-07-03 02:43:42 +02:00
Pouzor 60383bee64 fix: tolerate same device on multiple canvases in zigbee/zwave import
Zigbee and Z-Wave imports looked up canvas nodes by ieee_address with
scalar_one_or_none(), assuming one node per IEEE globally. A device placed
on two designs (one Node per canvas — a supported feature) made re-import
crash with MultipleResultsFound.

- Both mesh imports now refresh properties on every matching node instead
  of a single row (loop over .scalars().all()).
- approve_device guards against a true duplicate: same IEEE already on the
  SAME design reuses that node instead of inserting a second one.
- New node_dedupe service: loss-free repair keyed on (ieee, design_id).
  Collapses only genuine same-canvas duplicates (merges properties/services/
  missing fields, re-points edges + parent_id, drops self-loops/parallel
  edges). Cross-design placements preserved. Runs at start of both imports
  and bulk-approve. No-op on healthy DBs.
- IP correlation path already handled multiple nodes; left unchanged.

Tests: dedupe unit tests (collapse, cross-design preservation, edge/parent
re-point, idempotent), zigbee + zwave multi-canvas regression, approve
no-dupe guard.

ha-relevant: yes
2026-07-03 01:52:56 +02:00
Pouzor - Rémy JardientandGitHub b0cf0deab0 Merge pull request #207 from pranjal-joshi/pranjal/floorplan
feat: floor plan map, LQI edge coloring, zigbee path highlighting
2026-07-03 00:48:39 +02:00
Pouzor bdde03cdfa feat: floor plan selection handles, dbl-click edit, bottom z-index
- Resize handles only render when the plan is selected (click to select,
  click outside to deselect); previously always shown when unlocked.
- Double-clicking an unlocked plan opens the canvas edit modal (via a
  floorMapEditNonce signal the Sidebar watches).
- Floor plan always sits at the bottom of the canvas (z-index -1), behind
  nodes and edges, locked or not.
- Remount the edit modal on every open (key bump) so it re-seeds from the
  current floor plan; fixes Save clobbering a canvas-side resize/move with
  stale modal dimensions.
- Drop the no-op history snapshot on floor-plan edits (floorMap isn't part
  of undo history).
2026-07-03 00:22:53 +02:00
Pouzor 6160090919 revert: drop zigbee LQI edge coloring and path highlighting
Split out of this PR — will land separately with reworked zigbee edge
capture/storage. Keeps only the floor plan + generic media upload work.

- Remove getLqiColor + LQI-derived iot edge stroke (edges/index.tsx)
- Remove zigbee path highlight effect (DetailPanel.tsx)
- Remove highlightedPath/setHighlightedPath store state (canvasStore.ts)
- Delete utils/zigbeePathfinding.ts
- Revert associated test scaffolding and package-lock churn
2026-07-02 17:16:51 +02:00
Pouzor 1ed013bde2 feat: floor plan viewport rendering, per-canvas config, server media upload
- Render floor plan inside React Flow ViewportPortal so it pans/zooms with
  nodes (was screen-fixed, desynced on pan/zoom); zoom-stable resize handles.
- Move floor plan config from the left panel into the canvas (design) edit
  modal; attach per-design and fix cross-design bleed on load.
- Store images via a new generic backend media endpoint (POST/GET/DELETE
  /api/v1/media) on disk under <data_dir>/uploads, not base64 in the canvas.
- Disable floor plans in standalone mode (no backend to upload/serve); drop
  base64 localStorage persistence. See ADR-001 in CLAUDE.md.
- Tests: backend media route, DesignModal floor plan + upload, store floorMap.

ha-relevant: maybe
2026-07-02 16:36:25 +02:00
Pranjal Joshi 046c99e219 fix: null -> undefined for StandaloneCanvas.floorMap 2026-07-02 10:33:57 +05:30
Pranjal Joshi 8821b05b7b feat: floor plan map, LQI edge coloring, zigbee path highlighting, eslint/test fixes, vite audit fix 2026-07-02 02:00:04 +05:30
Pouzor c0c42d5f46 chore: gitignore docs/database-model.md 2026-06-30 14:54:15 +02:00
RémyandGitHub 332b6fb4f3 Merge pull request #246 from Pouzor/release/2.6.1
Release 2.6.1: Z-Wave manual node type + standalone multi-canvas
2026-06-30 14:45:53 +02:00
RémyandGitHub a8d365c1ee Merge pull request #245 from Pouzor/feat/standalone-multi-canvas
build: typecheck with tsc -b to match the production build
2026-06-30 14:45:36 +02:00
Pouzor 6e21218f05 chore: bump version to 2.6.1 2026-06-30 14:39:06 +02:00
Pouzor fb33e5cb16 feat: add Z-Wave node types to add/edit node modal
Z-Wave (controller/router/end device) types existed in the model, theme
accents, default icons and the custom-style editor, but were missing from
the node Add/Edit type selector, so they could not be placed manually like
Zigbee nodes. Add a Z-Wave group and default mesh-radio nodes to no status
check (not IP-reachable), matching Zigbee behaviour.
2026-06-30 14:38:02 +02:00
Pouzor 52de302f2b build: typecheck with tsc -b to match the production build
The `typecheck` script ran `tsc --noEmit` against the root tsconfig, which has
`files: []` and only project references — so without `-b` it checked nothing in
src and let real type errors (e.g. theme_id) reach Docker CI. Switch to `tsc -b`
(same as the build) so pre-commit and the Quality job catch them locally.

ha-relevant: no
2026-06-30 09:49:38 +02:00
RémyandGitHub f2162a663a Merge pull request #244 from Pouzor/feat/standalone-multi-canvas
feat: multi-canvas (designs) support in frontend-only standalone mode
2026-06-30 01:10:52 +02:00
Pouzor 1f2d298419 fix: type standalone canvas theme_id as ThemeId
The Docker CI build (tsc -b, stricter than the local tsc --noEmit) rejected
passing the storage theme_id (string) to setTheme/setCustomStyle which expect
ThemeId. Type StandaloneCanvas.theme_id as ThemeId so the standalone load path
matches the theme store API.

ha-relevant: no
2026-06-30 00:58:42 +02:00
Pouzor d1c052c9e8 feat: hide header View link in standalone mode
The "View" link opens the read-only live view of the canvas. In frontend-only
standalone mode the editor already renders the only (localStorage) copy, so the
live view adds nothing — hide the link. Kept in full mode.

ha-relevant: no
2026-06-30 00:48:56 +02:00
Pouzor 79cac7d80f feat: hide node stats footer in standalone mode
The Total/Online/Offline counts reflect live status-check results, which
require a backend scanner. In frontend-only standalone mode there is nothing to
populate them, so hide the footer (kept in full mode and live view).

ha-relevant: no
2026-06-30 00:33:42 +02:00
Pouzor c356a65a5f fix: repair live view + settings in standalone multi-canvas mode
Two gaps surfaced after adding per-design localStorage storage:

- Live view (/view) still read the legacy bare `homelable_canvas` key, which
  no longer exists once canvases are keyed per design — the read-only tab
  rendered empty. Now passes the active design id (?design=<id>) and reads that
  design's canvas, falling back to the first design.
- The Settings modal was gated out entirely in standalone, leaving the Settings
  button dead even though canvas prefs (snap, hide-IP) work without a backend.
  Mount it in standalone; only the backend status-check section stays hidden.

ha-relevant: no
2026-06-30 00:05:13 +02:00
Pouzor ca171089c2 feat: support multi-canvas (designs) in frontend-only standalone mode
Standalone mode (VITE_STANDALONE=true) skipped the designs system entirely:
the design list stayed empty (switcher hidden) and all canvases collapsed onto
a single localStorage key. Add a localStorage-backed design layer mirroring the
backend designs API.

- standaloneStorage util: list/create/update/delete designs + per-design canvas
  storage (homelable_designs + homelable_canvas:<id>). ensureSeed migrates a
  legacy single-canvas install into a default design so existing data survives.
- App.tsx: load/seed designs, save + switch canvases per design id in standalone.
- Sidebar.tsx: create/update/delete dispatch to standaloneStorage when standalone.
- Tests for the new storage util.

ha-relevant: no
2026-06-29 23:13:56 +02:00
RémyandGitHub e153b42b2e Merge pull request #241 from Pouzor/docs/zwave-import
docs: add Z-Wave import documentation
2026-06-29 02:10:46 +02:00
Pouzor 20dc9069d7 docs: add Z-Wave import section to README and full guide 2026-06-29 02:08:06 +02:00
RémyandGitHub c7b4db206b Merge pull request #240 from Pouzor/release/2.6.0
chore: bump version to 2.6.0
2026-06-29 01:51:18 +02:00
Pouzor a05c82b4f6 chore: bump version to 2.6.0 2026-06-29 01:37:58 +02:00
RémyandGitHub 9a066a3d6e Merge pull request #239 from Pouzor/fix/export-printing-165
fix: white background option for export + Firefox download (#165)
2026-06-29 01:17:48 +02:00
Pouzor f0af367c34 fix: force export background on the live react-flow element
The html-to-image style-option override did not reliably beat the
react-flow root's colorMode background, so white exports stayed black.
Set the background colour directly on the live element for the duration
of the capture (then restore it), which html-to-image reads when it
clones computed styles.

Refs #165
2026-06-29 01:13:55 +02:00
Pouzor 46185d187f fix: actually apply white export background over react-flow root
The .react-flow root paints its own opaque background via colorMode,
so html-to-image's backgroundColor option (which only fills transparent
areas) stayed black. Override the root background with an inline style
so the chosen export background is visible.

Refs #165
2026-06-29 01:02:00 +02:00
Pouzor fd2c50c1aa fix: white background option for export + Firefox download
Fixes two issues reported in #165:

- Export always produced a black background, making prints with a
  white page wasteful/unreadable. Adds a Dark/White background choice
  in the export modal, threaded through both PNG and SVG export.
- Firefox refused the download because the programmatic anchor was not
  attached to the document. The anchor is now appended before click and
  removed after, which Firefox requires.

Closes #165

ha-relevant: yes
2026-06-29 00:47:39 +02:00
RémyandGitHub 8b9972ca78 Merge pull request #238 from Pouzor/feat/svg-export-211
feat: add SVG export option to canvas export modal
2026-06-29 00:27:11 +02:00
Pouzor 313feb8ceb feat: add SVG export option to canvas export modal
Adds an SVG format choice below the Ultra quality option in the
export modal. PNG quality options still drive raster export; the
new SVG button exports a scalable vector via html-to-image toSvg.

Closes #211

ha-relevant: yes
2026-06-28 23:52:28 +02:00
RémyandGitHub 23673eb0ca Merge pull request #237 from Pouzor/fix/mcp-create-design-id
fix: attach MCP-created nodes/edges to a design (#225)
2026-06-28 15:52:39 +02:00
Pouzor cbc2bc03c2 fix: attach MCP-created nodes/edges to a design (#225)
create_node/create_edge persisted rows with design_id=null when the
client omitted it (the MCP write tools), so they existed in the DB but
never rendered on the canvas until a container restart reconciled them.
Both routes now fall back to the first design, matching bulk-approve.

Also fix MCP resource reads (homelable://canvas, homelable://edges):
the framework passes a pydantic AnyUrl, not a str, which raised
"'AnyUrl' object has no attribute 'startswith'". Coerce to str.

EdgeResponse now exposes design_id for symmetry with NodeResponse.

ha-relevant: no
2026-06-28 14:00:59 +02:00
RémyandGitHub 7e99d77edc Merge pull request #236 from Pouzor/fix/scan-stop-cancellation
fix: make scan stop interrupt in-flight nmap range (#218)
2026-06-28 11:24:27 +02:00
Pouzor da2c1c356a fix: make scan stop interrupt in-flight nmap range
Stop button had no effect: run_scan only checked the cancel flag between
CIDR ranges and between hosts, never inside the per-range nmap call. For a
single /24 the whole scan is one blocking call, so cancel was ignored for
minutes and the run status stayed 'running'.

- thread run_id into _nmap_scan/_ping_sweep/_nmap_port_scan; check cancel
  before each phase and skip queued hosts once cancelled
- flip ScanRun status to 'cancelled' eagerly in the stop endpoint so the UI
  reacts immediately instead of waiting for a checkpoint

Fixes #218

ha-relevant: yes
2026-06-28 11:13:14 +02:00
RémyandGitHub 3cedb40d17 Merge pull request #235 from Pouzor/fix/auto-layout-child-port-order
fix: order auto-layout children by parent port number
2026-06-28 01:09:13 +02:00
Pouzor 7fdce6af37 fix: order auto-layout children by parent port number
Dagre orders sibling nodes by node-insertion order and ignores handle
ids, so after auto-layout a host's children were frequently laid out in
the reverse of the ports they plug into. Add a post-pass that keeps
Dagre's X slots but reassigns which child sits in each, sorted by the
parent's bottom-port index, shifting each child's whole subtree by the
same delta so nested nodes follow their parent. Peer-group members are
skipped so the existing peer layout is untouched.

ha-relevant: yes
2026-06-28 01:03:38 +02:00
RémyandGitHub c1b0c42f0c Merge pull request #234 from Pouzor/feat/center-new-nodes-on-screen
feat: drop new nodes at centre of visible canvas
2026-06-28 00:37:51 +02:00
Pouzor 2a8c9d618b feat: drop new nodes at centre of visible canvas
New nodes previously landed at a fixed canvas origin, often off-screen,
forcing the user to drag them into view. Add a projector (registered by
CanvasContainer inside ReactFlowProvider) that maps the visible-canvas
centre into flow coordinates, and use it for add-node, group rect, text,
Zigbee/Z-Wave imports, and pending-device approval (single + bulk).

ha-relevant: yes
2026-06-28 00:27:18 +02:00
RémyandGitHub e811d83ceb Merge pull request #233 from Pouzor/feat/inventory-timestamps
feat: surface inventory timestamps on nodes
2026-06-27 23:39:32 +02:00
Pouzor c8d25c2383 fix: bulk-approve places every selected device onto the active canvas
Bulk-approve filtered status=='pending', so a device already approved onto
another canvas (status is global, canvas membership is per-design) — or whose
node was later deleted — was silently skipped. Selecting 64 devices on an
empty canvas produced only the ~28 still pending.

Approve now places a node on the target design for any selected, non-hidden
device that isn't already on that design (deduped by ip/ieee_address, including
within the batch). Returned device_ids/node_ids stay index-aligned so the
client places them all.

ha-relevant: yes
2026-06-27 19:02:28 +02:00
Pouzor 3020b400ed fix: declare transient node timestamp attrs on PendingDevice
mypy flagged the per-request node_* timestamp attributes as undefined on the
model. Declare them as class-level defaults (like canvas_count) and set
__allow_unmapped__ so SQLAlchemy 2.0 doesn't try to map the Optional[datetime]
annotations as columns.
2026-06-27 14:29:22 +02:00
Pouzor 8f8d9fe209 style: fit 3 inventory tiles per row
Move the 3-column grid down to the xl breakpoint (was 2xl), tighten the
inter-tile gap and card padding so three cards fit comfortably per row at
common widths instead of two.
2026-06-27 13:56:27 +02:00
Pouzor 612280e924 feat: show inventory timestamps on Device Inventory tiles
Surface the same lifecycle timestamps on the Device Inventory cards as on the
detail panel. Tiles for devices placed on a canvas show their linked node's
created / last scan / last modified / last seen (correlated by ip or
ieee_address, aggregated across matches: created = oldest, others = newest).
Devices not yet on any canvas fall back to their discovered_at.

Rendered as compact relative times ("2d ago") with the full date on hover, in
a tight two-column footer so the tile keeps its original footprint.

Backend: PendingDeviceResponse gains node_created_at / node_last_scan /
node_last_modified / node_last_seen; the canvas correlation now also pulls node
timestamps in the same single query. Frontend: shared timeFormat util
(absolute + relative), reused by the detail panel.

ha-relevant: maybe
2026-06-27 13:24:29 +02:00
Pouzor 19b7d38ec0 feat: surface inventory timestamps on nodes
Add creation_date, last_scan and last_modify to the node inventory and the
right-hand detail panel (last_seen already shown):

- creation_date  -> existing created_at column
- last_modify    -> existing updated_at column (bumped on any node change)
- last_scan      -> new column, stamped when a scan observes a node by IP/MAC
- last_seen      -> unchanged

Backend: new nodes.last_scan column + idempotent migration, NodeResponse
field, scanner stamps matching canvas nodes per scanned host. Frontend:
NodeData fields + DetailPanel rows with UTC-safe timestamp formatting.

ha-relevant: maybe
2026-06-27 03:13:49 +02:00
RémyandGitHub b3f7c38c67 Merge pull request #232 from Pouzor/docs/scanner-http-ranges
docs: document SCANNER_HTTP_RANGES port-spec format
2026-06-27 02:11:42 +02:00
Pouzor 5b20d4fd75 docs: document SCANNER_HTTP_RANGES port-spec format
Closes #229
2026-06-27 02:05:40 +02:00
RémyandGitHub 16e2dedab0 Merge pull request #219 from hooli/fix/markdown-copy
Fix: Markdow Copy now works over non-secure HTTP
2026-06-27 01:55:53 +02:00
Pouzor c8ed63712d refactor: extract copyToClipboard util with HTTP fallback + tests 2026-06-26 23:51:23 +02:00
Pouzor 10fdab52af Merge branch 'main' into fix/markdown-copy 2026-06-26 23:28:30 +02:00
RémyandGitHub eddf752b50 Merge pull request #224 from Pouzor/feat/zwave-scan
feat: Z-Wave network scan via MQTT
2026-06-26 23:03:49 +02:00
Pouzor 13420bead8 fix: approve devices onto the active design + Z-Wave node fields
Approve (single + bulk) created the canvas Node under the first design
instead of the design the user is viewing, so approved devices were
invisible on the active canvas and got wiped on the next save — and a
re-approve returned "0 approved" because the rows were already approved.

- bulk-approve now accepts design_id; the UI sends the active design.
- single approve already honoured design_id; the UI now sends it too.
- generalize the wireless branch (status=online, mesh props, no ICMP
  check) to Z-Wave as well as Zigbee, using build_zwave_properties.

ha-relevant: yes
2026-06-26 21:58:31 +02:00
Pouzor d41896fadf fix: render styled canvas nodes for zwave types
The zwave_* node types were not registered in the React Flow nodeTypes
map, so imported Z-Wave devices fell back to the default unstyled node
(no icon, no accent). Add ZwaveCoordinator/Router/EndDevice node
components and register them. Adds a registry guard test.

ha-relevant: yes
2026-06-26 20:42:52 +02:00
Pouzor f749b38edc fix: reset Custom Style draft on reopen
The modal is kept mounted by its parent (only `open` toggles), so Radix
onOpenChange never fires for a parent-driven open and the draft-reset
branch was dead code — abandoned edits leaked into the next open. Reset
on the `open` prop edge via effect instead. Also pass radix 10 to
parseInt for the size inputs. Adds a reopen-after-cancel regression test.

ha-relevant: yes
2026-06-26 16:20:30 +02:00
Pouzor ecf3cbdfe4 feat: group node types by category in Custom Style editor
The Custom Style modal listed node types as a flat list. It now groups
them under category headers (Hardware, Virtualization, IoT, Zigbee,
Z-Wave, Personal, Generic) like the Add/Edit Node modal, and exposes the
Z-Wave node types for styling.

ha-relevant: yes
2026-06-26 16:02:23 +02:00
Pouzor 5b08d57124 fix: add zwave node types to canvas theme maps
The 6 per-theme NodeType color maps in themes.ts were missing the three
zwave_* keys, breaking the production build (tsc -b) even though the
dev typecheck passed.

ha-relevant: yes
2026-06-26 13:58:30 +02:00
Pouzor 9b8f15bec3 feat: show Z-Wave scan runs in Scan History
Scan History now recognises kind=zwave: dedicated Z-Wave filter chip,
badge (RadioTower, orange) and import-done toast, instead of falling
back to the generic IP type.

ha-relevant: yes
2026-06-26 13:10:12 +02:00
Pouzor 8faf5c1c79 feat: add Z-Wave network scan via MQTT gateway
Import a Z-Wave JS UI (zwavejs2mqtt) network over the MQTT gateway API,
mirroring the existing Zigbee pipeline:

- New Z-Wave Import modal + sidebar entry (broker, prefix, gateway name)
- coordinator/router/end-device typing with mesh tree from node neighbors
- import to Pending section or straight to canvas
- Pending Devices gains a Z-Wave source filter
- shared mqtt_common helpers extracted from the zigbee service

ha-relevant: yes
2026-06-26 11:19:37 +02:00
RémyandGitHub c6076d133a Merge pull request #222 from Pouzor/feat/http-probe-deep-scan
feat: deep-scan HTTP probe + Device Inventory (#195)
2026-06-26 02:12:39 +02:00
Pouzor 662a240b05 fix: collapse duplicate inventory rows for the same device
A re-scan of an already-approved device created a fresh pending row each
time (the upsert only matched status=pending), so one device could show
several times in the inventory. Now the scanner keeps one non-hidden row
per IP, refreshing it in place and preserving an approved status, and
collapses any pre-existing duplicates at scan start.

ha-relevant: maybe
2026-06-26 01:58:53 +02:00
Pouzor dfeb981e40 feat: add 'With services' filter to Device Inventory
ha-relevant: maybe
2026-06-26 01:40:56 +02:00
Pouzor 430c14ed5e fix: rename left-panel nav item to Device Inventory
ha-relevant: maybe
2026-06-26 01:24:04 +02:00
Pouzor 234019c7b7 fix: declare transient canvas_count on PendingDevice for mypy
ha-relevant: maybe
2026-06-25 21:33:05 +02:00
Pouzor d7ab4ba49a feat: Device Inventory — show all scanned devices with canvas-presence
Reworks the "Pending Devices" panel into a "Device Inventory": scanned
devices already placed on a canvas are no longer suppressed — they stay
listed and badged with how many canvases they appear on.

- scanner: stop deleting/skipping on-canvas IPs (hidden still suppressed)
- scan API: /pending returns all non-hidden devices; compute canvas_count
  by correlating ip/ieee_address against nodes grouped by design
- frontend: rename to "Device Inventory", top-right canvas-count corner,
  toggle to show/hide on-canvas devices (default show)

ha-relevant: maybe
2026-06-25 18:05:13 +02:00
Pouzor 96107cc657 feat: seed port-agnostic http_regex signatures for ~50 popular homelab apps
Adds port:null signatures matched by HTTP page title for the most-used
self-hosted and home-automation apps (Jellyfin, Plex, *Arr, Home Assistant,
Node-RED, Zigbee2MQTT, ESPHome, Domoticz, Jeedom, Portainer, Grafana,
Uptime Kuma, Pi-hole, AdGuard Home, Nextcloud, Vaultwarden, Authelia,
OpenMediaVault, Unraid, etc.). These identify a service on any port once the
deep-scan HTTP probe is enabled. Existing port-keyed entries are unchanged.

ha-relevant: yes
2026-06-25 14:47:58 +02:00
Pouzor c68084b751 test: update client trigger assertion for deep-scan body
ha-relevant: yes
2026-06-24 10:40:03 +02:00
Pouzor d01630bf37 feat: add Deep Scan toggle to scan dialog
Collapsible Deep Scan section in ScanConfigModal exposes extra port ranges,
HTTP probe and TLS-verify switches. Pre-filled from saved defaults; edits are
passed to trigger() as a per-scan override and do not change the persisted
defaults (those live in the Options/scan config). scanApi.trigger now accepts
an optional deep-scan body.

ha-relevant: yes
2026-06-24 10:31:32 +02:00
Pouzor b6423c0115 feat: expose deep-scan settings via API and persisted config
Adds scanner_http_ranges / scanner_http_probe_enabled / scanner_http_verify_tls
to Settings (persisted in scan_config.json, Options page defaults). /scan/trigger
accepts an optional body to override these per-scan; /scan/config GET/POST read
and persist the defaults. Port ranges validated at the API boundary.

ha-relevant: yes
2026-06-24 10:04:31 +02:00
Pouzor 9d1ebd6c6e feat: wire deep-scan port ranges and HTTP probe into scanner
run_scan() accepts DeepScanOptions: user port ranges are validated and appended
to the nmap -p list; when http_probe_enabled, open ports are probed for HTTP
signals before fingerprinting. Defaults (no options) reproduce the standard
scan exactly — probe is never called, port list unchanged.

ha-relevant: yes
2026-06-23 14:04:15 +02:00
Pouzor 44e0029f2b feat: add async HTTP prober for deep-scan service identification
probe_port() GETs https:// then http:// for an open port and returns the page
<title> plus Server/X-Powered-By headers as identifying signals. Non-web ports
(SSH, DB, etc.) are skipped; body read is capped at 64KB; 3s timeout.
probe_open_ports() fans out over all open ports with a bounded semaphore.

ha-relevant: yes
2026-06-23 13:41:32 +02:00
Pouzor b20c47a0a5 feat: add tiered service matching with http_regex and port-agnostic signatures
Adds match_service() priority walk for fingerprinting:
1. port + http_regex confirmed
2. port + banner_regex confirmed
3. port:null + http_regex confirmed (custom-port services)
4. port-only fallback

http_regex is strict only once a probe has run; with no probe signals it
degrades to port-only matching, preserving pre-probe behaviour. match_port
kept as a probe-less alias.

ha-relevant: yes
2026-06-23 12:25:10 +02:00
RémyandGitHub be5771f7d0 Merge pull request #220 from Biztactix-Ryan/feat/oui-vendor-expansion
feat: expand MAC OUI database with router, switch, AP, NAS, and camera vendors
2026-06-23 11:37:36 +02:00
Pouzor 5d62df45f7 chore: bump zeroconf to 0.149.16 for CVE-2026-48487
ha-relevant: no
2026-06-23 11:28:59 +02:00
Pouzor 83b296747c chore: fix high-severity npm audit advisories (undici, babel, js-yaml)
ha-relevant: no
2026-06-23 11:03:53 +02:00
Ryan Tregea 77718a5239 feat: expand MAC OUI database with router, switch, AP, NAS, and camera vendors
Moves the OUI mapping from an inline dict in fingerprint.py into
data/oui_database.json grouped by vendor, matching the service_signatures
data-file pattern. Adds ~100 curated OUIs covering MikroTik, Ubiquiti,
Synology, QNAP, Cisco, Aruba, Juniper, Hikvision, Dahua, Reolink, Axis,
Raspberry Pi, Dell, Supermicro, and others. Existing IoT vendors and
hypervisor OUIs are preserved.

For multi-product vendors the OUI is tagged with the most common homelab
category (e.g. Ubiquiti -> ap) and port hints in suggest_node_type
continue to upgrade ambiguous matches (Ubiquiti + BGP -> router).
2026-06-22 04:11:37 +00:00
hooli 40f3edce68 Fix: Markdow Copy now works over non-secure HTTP 2026-06-20 01:22:22 +01:00
RémyandGitHub 1efc1c95ca Merge pull request #217 from Pouzor/chore/bump-2.5.1
chore: bump version to 2.5.1
2026-06-18 02:14:16 +02:00
Pouzor b70ee78d25 chore: bump version to 2.5.1
ha-relevant: no
2026-06-18 01:51:13 +02:00
RémyandGitHub 9efaaccb51 Merge pull request #210 from Pouzor/fix/node-resize-persist
fix: persist manual node sizes and add width/height inputs
2026-06-18 01:39:07 +02:00
Pouzor 6e5bda5860 chore: bump python-multipart to 0.0.31 for CVE fixes
pip-audit (dependency-audit CI step) flagged python-multipart 0.0.27
for CVE-2026-53538/53539/53540 (fixed in 0.0.30/0.0.31). Bump to
0.0.31. pip-audit clean and backend test suite green.

ha-relevant: no
2026-06-17 17:59:32 +02:00
Pouzor 0796c96fc1 chore: resolve high-severity npm audit advisories
The dependency-audit CI step (npm audit --audit-level=high) was failing
on pre-existing transitive advisories. Force patched versions via
overrides and bump vite to a patched 7.x:

- esbuild ^0.28.1  (GHSA-g7r4-m6w7-qqqr, high)
- form-data ^4.0.6 (GHSA-hmw2-7cc7-3qxx, high)
- vite ^7.3.5      (GHSA-v6wh-96g9-6wx3 / GHSA-fx2h-pf6j-xcff, high)

Audit now passes at --audit-level=high (only a low @babel/core and
moderate js-yaml remain, both below the gate and build/dev-time only;
js-yaml is a direct dep so it can't be overridden). Build + full test
suite (1130) green on the bumped toolchain.

ha-relevant: no
2026-06-17 16:49:25 +02:00
Pouzor 6fba0cdec4 fix: don't trap new child nodes under a non-container parent
Adding an LXC/VM with a parent_id pointed at a Proxmox node that is
NOT in container_mode set extent:'parent' anyway, confining the node
to the parent's ~140px bounding box with no way to drag it out.

Nesting is now gated on the parent's container_mode in both places:
App.handleAddNode no longer sets parentId/extent itself (it seeds an
absolute position and lets addNode decide), and addNode strips any
stray parentId/extent when it isn't nesting. New children of a real
container still land at the container's top-left.

ha-relevant: yes
2026-06-17 14:23:49 +02:00
Pouzor 2a4d109ee6 fix: persist manual node sizes and add width/height inputs
Nested vm/lxc/docker leaf nodes lost their size on reload: the
deserialize restore branch excluded those types entirely, so a
resized child snapped back to content-fit. Gate the restore on
container_mode instead of node type.

Also prefer explicit width/height over the DOM-measured value when
serializing, so a manual resize persists its exact target rather than
drifting to the fractional content-fit size.

Add a Size section to the detail panel with W/H number inputs
(setNodeSize store action, clamped to the resizer minimums). Inputs
resync live when the node is resized by corner drag, without
clobbering active keystrokes.

ha-relevant: yes
2026-06-17 11:55:55 +02:00
RémyandGitHub 96bb048228 Merge pull request #204 from Pouzor/chore/bump-2.5.0
chore: bump version to 2.5.0
2026-06-12 01:16:25 +02:00
Pouzor 88c006764a test: fix flaky per-service toggle assertion in SettingsModal
The toggle label renders before settingsApi.get() resolves, so reading
.checked immediately raced the async state update and failed on slower CI.
Wait for the resolved checked state instead.

ha-relevant: no
2026-06-12 01:05:48 +02:00
Pouzor 9c3dace12f chore: bump version to 2.5.0
ha-relevant: no
2026-06-12 00:54:52 +02:00
RémyandGitHub ec53363fdb Merge pull request #203 from Pouzor/feat/scan-history-modal
feat: move scan history into a modal
2026-06-12 00:40:19 +02:00
Pouzor 0e260ede73 feat: move scan history from sidebar panel into a modal
Replace the inline sidebar 'history' view with a dedicated ScanHistoryModal.
Same data and actions (refresh, auto-refresh while running, stop scan,
transition toasts) plus run duration, finished timestamp, and kind/status
filters. Scan/Zigbee start now surface a toast instead of force-switching
the sidebar view.

ha-relevant: yes
2026-06-12 00:11:16 +02:00
RémyandGitHub 60902167c0 Merge pull request #202 from Pouzor/feat/node-into-container
feat: drop node onto container node to nest it
2026-06-11 18:36:48 +02:00
Pouzor 592e7865c5 fix(deps): bump zeroconf to 0.149.12 for CVE-2026-48045
pip-audit (Security workflow) flagged zeroconf 0.149.7 as vulnerable to
CVE-2026-48045, fixed in 0.149.12.

ha-relevant: no
2026-06-11 18:24:59 +02:00
Pouzor f082c295fd feat: edit container parent in the node modal instead of detail panel
Replaces the detail-panel container selector with the existing "Parent
Container" dropdown in the edit modal (double-click). The selector now
shows for any node that is nested in — or can nest into — a container_mode
node, not just lxc/vm/docker_container, so a dragged-in node can be
re-targeted or detached ("None") from the modal.

- NodeModal: container_mode candidates count as valid parents; selector
  shown for nested nodes; submit/type-change keep valid container parents
- App: pass container_mode in parentCandidates
- revert detail-panel selector + setNodeParent store action (updateNode
  already handles parent attach/detach)

ha-relevant: yes
2026-06-11 17:25:44 +02:00
Pouzor 10b981ad1d feat: editable container selector to detach/re-parent a nested node
When a node is nested in a container, the detail panel now shows a
Container selector. Pick another container to move it, or "None" to
detach it back to the canvas (clears parentId).

- store: setNodeParent(childId, parentId|null) — attach/detach/re-parent
  via absolute coords, container_mode-only targets, history snapshot
- DetailPanel: Container <select>, shown only for nested nodes

ha-relevant: yes
2026-06-11 15:39:49 +02:00
Pouzor 2058e453ff feat: drop node onto container node to nest it
Dropping a top-level node over any container_mode node (proxmox,
docker_host, ...) now pops a confirm modal that nests it as a child
(sets parentId), mirroring the existing drop-onto-group flow.

- store: addToContainer(containerId, childId)
- CanvasContainer: detect container_mode intersection on drag stop
  (group still wins if both intersect)
- generalize ConfirmAddToGroupModal with a container variant

ha-relevant: yes
2026-06-11 14:59:25 +02:00
RémyandGitHub b776b6b8f1 Merge pull request #200 from Pouzor/feat/editable-node-groups
feat: editable node groups (add/remove members, description)
2026-06-11 02:05:34 +02:00
Pouzor fe3787f0cd fix: persist group description with Ctrl+S, not just on blur
The description field committed to the store only on textarea blur, so
saving via Ctrl+S (the primary shortcut, which never blurs the field)
dropped the edit. Make it controlled and commit on each change; snapshot
history once per edit session (on focus) so undo stays a single step.

ha-relevant: yes
2026-06-11 01:48:22 +02:00
Pouzor 7ea1fc2d69 feat: editable node groups (add/remove members, description)
Make Node Groups mutable instead of fixed-at-creation:
- removeFromGroup / addToGroup store actions (inverse pair, history-aware)
- right panel: per-member remove button + editable group description
  (reuses data.notes, no backend/serializer change)
- drag a node over a group → confirm modal to add it (getIntersectingNodes)

Tests: +21 (store, panel, canvas detection, modal).

ha-relevant: yes
2026-06-10 16:50:36 +02:00
RémyandGitHub 71cc183efb Merge pull request #199 from Pouzor/feat/multiline-edge-labels
feat: multi-line edge labels
2026-06-10 15:13:12 +02:00
Pouzor 7873f8b94f feat: multi-line edge labels
Connection labels now accept line breaks. The label field in EdgeModal
is a textarea (Enter inserts a newline) and the rendered edge label uses
whitespace-pre-line so newlines display as multiple lines.

Closes #183

ha-relevant: yes
2026-06-10 12:04:10 +02:00
RémyandGitHub b34b40794d Merge pull request #198 from Pouzor/fix/status-ping-service-color
feat: per-service status checks + ping/service-color fixes (#196)
2026-06-10 11:13:06 +02:00
Pouzor ed310f0d8d fix(status): keep non-HTTP services grey instead of red
Per-service checks now only probe HTTP(S)-reachable services. SSH (22) and
other non-web ports (DB, mail, DNS, raw TCP) stay 'unknown' (grey category
colour) rather than going red — an open TCP socket doesn't prove the service
is healthy, and a firewalled port flapped red misleadingly.

ha-relevant: yes
2026-06-10 10:48:03 +02:00
Pouzor 2db173cd56 fix(ws): idempotent connection removal, release slot on any error
The status WebSocket pool removed connections with list.remove(), which
raises ValueError on a double-remove (broadcast already dropped a dead
socket, then disconnect tries again), and only released a slot on
WebSocketDisconnect — any other error leaked the socket into the
broadcast pool. Centralise removal in an idempotent _drop() called from
a finally block and from _broadcast.

ha-relevant: yes
2026-06-10 09:58:06 +02:00
Pouzor d032103b7d fix(types): add type params to broadcast_service_status services arg
ha-relevant: yes
2026-06-10 02:44:06 +02:00
Pouzor bcc488993d feat: per-service status checks with offline colouring
Adds optional live status checking per service (not just per node),
requested as a follow-up to issue #196.

Backend:
- New check_service / check_services: HTTP(S) GET for web services, TCP
  connect otherwise; UDP and port-less non-web services stay 'unknown'.
- New scheduler job 'service_checks', independent interval (default 300s),
  added/removed live via set_service_checks_enabled.
- Settings gain service_check_enabled + service_check_interval (>=30s),
  persisted to scan_config.json. New WS message type 'service_status'.

Frontend:
- Live per-service status overlay in canvasStore (not persisted, so it
  never round-trips through canvas save), fed by the WS message.
- DetailPanel + canvas node service rows: offline service turns red
  (#f85149), otherwise keeps its category colour.
- SettingsModal: toggle + interval input (default 300s / 5 min).

Off by default — no behaviour change until enabled.

ha-relevant: yes
2026-06-10 02:22:48 +02:00
Pouzor aa0a97c245 fix: reduce status flapping, add IPv6 ping, color manual web services
Addresses three reports from issue #196:

- Ping now sends 2 probes with a ~2s timeout (was 1 probe / 1s) so a
  single dropped packet or a slow IoT/ESPHome device no longer flaps a
  node offline (#196.1, #196.2).
- IPv6-only devices (e.g. Alexa) are now pinged over IPv6: ping6 on
  macOS, -6 flag on Linux/Windows, detected via inet_pton (#196.3).
- Manually-added services carry no category and so always rendered grey
  even when they were reachable HTTP/HTTPS. A resolvable web URL now
  falls back to the web colour (#196.9).

ha-relevant: yes
2026-06-09 21:20:05 +02:00
RémyandGitHub a39d4d4406 Merge pull request #197 from Pouzor/some-fixes
Security fixes: Dependabot + code scanning alerts
2026-06-09 17:26:56 +02:00
Pouzor 7c2417f5a9 fix(security): resolve code scanning alerts
Add least-privilege 'permissions: contents: read' to quality, security
and docker-ci workflows (actions/missing-workflow-permissions).

Harden markdown table cell escaping in exportMarkdown: escape backslash
before pipe and collapse newlines so untrusted values can't break the
table (js/incomplete-sanitization). Add regression tests.

ha-relevant: maybe
2026-06-09 17:15:26 +02:00
Pouzor ea66e6c9c7 chore: bump hono to 4.12.21+ and ignore versioned db backups
Add npm override forcing hono>=4.12.21 to resolve 4 Dependabot alerts
(transitive via shadcn CLI, dev-only). Add *.db.back-* to .gitignore
to exclude versioned database backups.

ha-relevant: no
2026-06-09 16:29:52 +02:00
RémyandGitHub f78bac6e6b Merge pull request #192 from Pouzor/chore/bump-2.4.0
chore: bump version to 2.4.0
2026-06-05 16:24:23 +02:00
Pouzor a0bfb97517 chore: bump version to 2.4.0
ha-relevant: no
2026-06-05 16:21:43 +02:00
RémyandGitHub ff9c4735e0 Merge pull request #191 from Pouzor/fix/persist-show-port-numbers
fix(nodes): persist show_port_numbers across reloads
2026-06-05 16:04:27 +02:00
Pouzor 2f3e986d4a fix(nodes): persist show_port_numbers across reloads
The Show Port Numbers toggle reset on every reload because the backend
never stored it: the column was missing from the Node model and the field
was stripped by NodeSave/NodeBase/NodeUpdate schemas, so canvas/save
silently dropped it.

Add the show_port_numbers column (idempotent migration), and the field to
the node schemas so it round-trips through save/load.

Add regression tests covering persistence and the default-false case.

Fixes #184

ha-relevant: yes
2026-06-05 15:53:58 +02:00
RémyandGitHub 0a4bac30ac Merge pull request #190 from Pouzor/fix/save-button-event-arg
fix(canvas): Save button no-op due to leaked click event
2026-06-05 15:29:51 +02:00
Pouzor ba2f6564ec fix(canvas): Save button no-op due to leaked click event
The Toolbar and Sidebar Save buttons wired onClick={onSave} directly, so
React passed the MouseEvent as the first argument. handleSave treats its
first arg as a designIdOverride, corrupting design_id and making the save
fail silently. Ctrl+S worked because it calls handleSave() with no args.

Wrap both handlers as onClick={() => onSave()} so no event leaks through.

Add regression tests asserting onSave is called with zero arguments.

Fixes #186

ha-relevant: no
2026-06-05 15:22:13 +02:00
RémyandGitHub 754a79d21f Merge pull request #189 from Pouzor/feat/cross-design-copy-paste
feat(canvas): cross-design copy/paste + persisted Hide-IP in Settings
2026-06-05 14:24:33 +02:00
Pouzor b52bbc6d9f feat(settings): move Hide IP toggle into Settings modal, persist it
Hide-IP was a sidebar button held only in memory, so it reset on reload.
Moved it into the Settings modal Canvas section and persist it to
localStorage (new ipDisplay util); the canvas store now seeds hideIp from
storage and writes through on toggleHideIp/setHideIp. Settings is now also
reachable in standalone (no-backend) builds, with the backend-only status
interval guarded so the modal still works there.

ha-relevant: yes
2026-06-05 11:26:28 +02:00
Pouzor c67b1775a5 feat(canvas): copy/paste nodes across designs
Clipboard now holds nodes + internal edges and survives design switches
(loadCanvas no longer clears it), so a selection copied in one design can
be pasted into another. Copy pulls in children of selected groups/
containers; paste remaps node/edge/parent IDs and lands the bounding-box
center under the cursor (or viewport center). Shortcut handling moved into
CanvasContainer for flow-coordinate projection.

ha-relevant: yes
2026-06-05 10:52:44 +02:00
RémyandGitHub 5940e11860 Merge pull request #188 from Pouzor/refactor/settings-modal
refactor(settings): move sidebar settings into a modal
2026-06-05 10:17:22 +02:00
Pouzor fa8f949b45 refactor(settings): move sidebar settings into a modal
Clicking Settings now opens a dialog with the status-check interval and
canvas snap options plus Save/Cancel, replacing the inline left-panel view.
Removes the 'settings' Sidebar view and SettingsPanel component.

ha-relevant: maybe
2026-06-05 10:00:39 +02:00
RémyandGitHub 6e17a2afc6 Merge pull request #187 from Pouzor/fix/edge-anim-css
refactor(edges): CSS animations for snake/flow edges (drop SVG SMIL)
2026-06-05 02:03:06 +02:00
Pouzor 2375efa6b7 refactor(edges): use CSS animations for snake/flow edges instead of SVG SMIL
Snake and flow edge animations used SVG SMIL <animate repeatCount="indefinite">.
SMIL keeps running while a tab is hidden (CSS animations pause) and grows memory
in Chrome over long-lived background tabs. Convert both to CSS @keyframes
(homelable-snake / homelable-flow), matching the existing basic-dash pattern.

Identical visuals and timings; bidirectional snake yo-yos via `alternate`.
Adds a regression test asserting animated edges emit zero <animate> nodes and a
CSS animation instead.

ha-relevant: yes
2026-06-05 01:54:54 +02:00
RémyandGitHub 61fbebb13e Merge pull request #185 from Pouzor/feat/liveview-design-param
feat(liveview): header View link opens active design as read-only canvas
2026-06-04 21:35:08 +02:00
Pouzor 05fb67e7f0 feat(liveview): header View link opens active design as read-only canvas
Add an authenticated GET /api/v1/liveview/config endpoint exposing the
configured LIVEVIEW_KEY to logged-in admins, so the UI builds a
ready-to-use share link. The header View button opens
/view?key=...&design=<activeDesignId>; LiveView forwards ?design=<id>
to the public endpoint, which renders that design's canvas.

ha-relevant: maybe
2026-06-04 14:10:29 +02:00
RémyandGitHub 1a426f6301 Merge pull request #177 from pranjal-joshi/feat/electrical
feat: Multi Design Canvas + new node type for electrical devices
2026-06-04 13:20:22 +02:00
Pouzor f56dfea838 fix(liveview): allow zoom-out to 0.25 so large infra fits
View-only mode set no minZoom, so React Flow defaulted to 0.5 and big
canvases couldn't zoom out far enough to fit. Match the editor's bounds
(minZoom 0.25, maxZoom 2.5). Add a regression test asserting the props.

ha-relevant: maybe
2026-06-04 13:00:29 +02:00
Pouzor 6784b8f6be refactor(ui): harden pending-devices keyboard handler
Address review nits on the close/Enter fix:
- Add statusFilter to the keydown effect deps so Enter dispatches the
  correct bulk action regardless of whether the device list changes
- Drop redundant title="Close" on the X (aria-label is enough)

ha-relevant: maybe
2026-06-03 01:31:50 +02:00
Pouzor 01fce64b2c fix(ui): Enter restores hidden devices instead of approving them
In the Hidden view the only bulk action is Restore, but the Enter
shortcut in select mode always called handleBulkApprove — approving
hidden devices. Dispatch on statusFilter: restore when hidden, approve
when pending. Add regression tests for both views.

ha-relevant: maybe
2026-06-03 01:09:36 +02:00
Pouzor 79093fdcfb fix(ui): close pending/hidden devices modal via the X on Firefox
The close X used a raw <button onClick={onClose}>, bypassing Base UI's
dismissal. Outside-click worked (it goes through onOpenChange) but the
synthetic click on the X was dropped on Firefox/Windows. Route the X
through Base UI's DialogClose, the same path as outside-click and the
library's default close button. Add a regression test.

ha-relevant: maybe
2026-06-03 00:59:28 +02:00
Pouzor 4ce8a12720 test: guard legacy -> multi-design migration (no data loss on upgrade)
Builds a real pre-designs database (nodes/edges without design_id, integer
canvas_state PK) and asserts init_db() adopts everything into a single
default Network Topology design with the viewport preserved, and that the
migration is idempotent across reboots. The rest of the suite uses create_all
and never exercised this in-place upgrade path.

ha-relevant: yes
2026-06-02 16:52:42 +02:00
Pouzor cc2cc72482 Merge remote-tracking branch 'origin/main' into feat/electrical 2026-06-02 16:10:46 +02:00
Pouzor 12d527aad6 feat: manage canvases with custom name and icon
Make designs (canvases) fully user-manageable: create with a chosen name
and icon, rename, change icon, and delete. Replaces the hardcoded
"New Electrical Design" button with a generic "New Canvas" flow.

- Add Design.icon column + migration that backfills legacy rows
  (electrical -> zap, others -> dashboard)
- DesignModal: name input + curated lucide icon picker (create + edit)
- Sidebar switcher gains per-canvas edit/delete; delete guards the last
  canvas and confirms
- designStore: addDesign/updateDesign/removeDesign with active reassignment
- Fix data loss on design switch: abort load when the save fails and keep
  unsaved edits; skip the save-old step when the previous canvas was deleted
- designsApi create/update carry icon; design_type kept for back-compat

Tests: backend design CRUD (icon + cascade + last-canvas guard), designStore
actions, designIcons resolver, DesignModal create/edit/validation.

ha-relevant: yes
2026-06-02 15:47:57 +02:00
RémyandGitHub 73b479a408 Merge pull request #182 from Pouzor/chore/dependabot-qs-brace-expansion
chore(deps): fix Dependabot alerts (qs, brace-expansion)
2026-06-02 11:04:27 +02:00
Pouzor 16183285b5 chore(deps): bump qs to 6.15.2 and brace-expansion to 5.0.6
Fixes Dependabot alerts #1 (brace-expansion DoS) and #2 (qs stringify
DoS). Lockfile-only, transitive deps via shadcn/ts-morph/express.

ha-relevant: no
2026-06-02 10:10:04 +02:00
RémyandGitHub 3b3cc1eba0 Merge pull request #181 from Pouzor/chore/bump-2.3.0
chore: bump version to 2.3.0
2026-05-31 23:58:05 +02:00
Pouzor 3da5517996 chore: bump version to 2.3.0 2026-05-31 23:51:26 +02:00
RémyandGitHub cb26da3de5 Merge pull request #180 from Pouzor/feat/issue-174-mcp-full-node-schema
feat(mcp): expose full node schema in create_node/update_node (#174)
2026-05-31 23:14:31 +02:00
Pouzor 23a0a47a7f feat(mcp): expose full node schema in create_node/update_node (#174)
The MCP create_node/update_node inputSchema only advertised
type/label/ip/hostname/status (plus parent_id on update), so LLM
clients could not set documentation/hardware fields the backend
already validates and stores.

Mirror NodeBase/NodeUpdate by spreading a shared _NODE_FIELDS schema
into both tools: os, notes, mac, check_method, check_target, services,
cpu_count, cpu_model, ram_gb, disk_gb, show_hardware, container_mode,
custom_icon, properties. type stays an enum of canonical node types.
_dispatch already forwarded args verbatim, so no dispatch change.

Also extend _slim_canvas NODE_KEEP so get_canvas round-trips the new
documentation fields the LLM can now write.

Refactor tool definitions into a module-level TOOLS list for direct
schema assertions in tests.

ha-relevant: no
2026-05-31 22:53:00 +02:00
RémyandGitHub aac6c09a04 Merge pull request #178 from Pouzor/feat/issue-168-mac-in-properties
feat(scan): carry scanned MAC onto approved nodes (#168)
2026-05-31 16:39:50 +02:00
Pouzor bf90d6312b fix(canvas): include MAC property in approved node sent to canvas (#168)
The backend stored the MAC, but the frontend built the approved canvas
node locally with properties=[] (non-zigbee) and no mac field. On save,
canvas/save upserts every node field from the in-memory canvas, so the
empty properties overwrote the DB MAC — the address showed after a
no-save reload but vanished once saved.

Now both approve paths (single + bulk) attach a MAC property row and
the mac field to the node added to the canvas, mirroring the backend
build_mac_property. Hidden by default, toggleable from the right panel.

- new buildMacProperty util + unit tests
- bulk-approve modal test asserts MAC propagation (IP device) and
  absence for a zigbee device

ha-relevant: yes
2026-05-31 16:15:14 +02:00
Pouzor ec15c260e1 feat(scan): carry scanned MAC onto approved nodes (#168)
The network scan captures each device's MAC, but approving a pending
device dropped it: the new Node had mac=NULL and no MAC in its
properties pane. Now both approve paths (single + bulk) copy the
pending device's MAC to Node.mac and add a MAC row to the node
properties so it shows in the standard properties pane with toggleable
visibility (hidden by default, consistent with zigbee props).

- build_mac_property: MAC NodeProperty row (hidden by default)
- merge_mac_property: append MAC to user-supplied props, idempotent,
  preserves existing visibility
- single approve prefers the pending device MAC, falls back to payload

Adds unit tests for the helpers and API tests for both approve paths.

ha-relevant: yes
2026-05-31 14:47:15 +02:00
RémyandGitHub 47ab3a9a76 Merge pull request #176 from Pouzor/fix/issue-167-zigbee-pending-revival
fix(zigbee): revive orphaned approved devices on re-import (#167)
2026-05-31 14:21:24 +02:00
Pranjal Joshi eb7b0c6d38 test: fix canvasApi.load test to expect params arg 2026-05-31 15:07:29 +05:30
Pranjal Joshi be320c2ac4 fix: auto-create default design in save_canvas when none exists 2026-05-31 15:03:54 +05:30
Pranjal Joshi 60ce648a96 ci: fix remaining lint/type errors - react-compiler deps, None-indexable 2026-05-31 14:56:55 +05:30
Pranjal Joshi d5e35a885d ci: fix lint/type errors - missing electrical icons, unused vars, mypy issues 2026-05-31 14:53:38 +05:30
Pranjal Joshi 3a57d809a4 fix: compatibility with multi-design schema for scan/zigbee/liveview/CustomStyleModal
- liveview.py: replace hardcoded CanvasState PK lookup (get(1)) with
  design_id-aware query; filter nodes/edges by design_id
- scan.py: add design_id to bulk approve, single approve, and edge
  resolution Node/Edge constructors (fallback to first design)
- zigbee.py: add design_id to coordinator auto-approval Node constructor
- schemas/nodes.py: add design_id to NodeCreate and NodeResponse
- schemas/edges.py: add design_id to EdgeCreate
- CustomStyleModal.tsx: add 'electrical' to EDITABLE_EDGE_TYPES
2026-05-31 14:44:17 +05:30
Pranjal Joshi 46435605eb feat: multi-design canvas system with electrical nodes/edges
Backend: New Design model + designs table; design_id FK on nodes, edges, canvas_state; migration seeds default 'Network Topology' design; full CRUD API for designs; canvas load/save accept design_id.

Frontend: designStore (Zustand), design switcher in Sidebar, design-aware canvas load/save, auto-save on design switch.

Electrical node types (14): grid, ups, battery, generator, solar_panel, inverter, circuit_breaker, contactor, electrical_switch, socket, light, meter, transformer, load — icons, registrations, accent colors in all 6 themes.

Electrical edge type: registered in edgeTypes, BASE_STYLES, edgeColors, all theme edgeColors, EDGE_DEFAULT_COLORS.

Bug fixes: data corruption on design switch (stale closure), race condition on save-then-load, missing Zap import, missing Electrical group in NodeModal, missing electrical entries in custom theme edgeColors, inline imports hoisted.
2026-05-31 14:35:22 +05:30
Pouzor e8bcf04b46 fix(zigbee): revive orphaned approved devices on re-import (#167)
When a Zigbee device was approved (PendingDevice.status=approved + Node
created) and the user later deleted the canvas Node, the PendingDevice row
was left orphaned as approved. On re-import _persist_pending_import found
the existing row, refreshed its fields but left status=approved, so it
never reappeared in the Pending list (which filters status==pending) —
the import reported devices found but Pending stayed empty.

Reset such orphaned approved rows back to pending on re-import. Hidden
devices stay hidden (explicit user intent). Adds regression tests for
both revive-approved and keep-hidden paths.

ha-relevant: yes
2026-05-31 01:54:24 +02:00
RémyandGitHub 4ba04660c8 Merge pull request #175 from Pouzor/fix/zeroconf-cve-bump
fix(deps): bump zeroconf to 0.149.7 for CVE fixes
2026-05-31 01:33:12 +02:00
Pouzor 4b06ce6ef6 fix(deps): bump zeroconf 0.131.0 -> 0.149.7 for CVE fixes
Resolves CVE-2026-47180/47183/47184 flagged by Security workflow
pip-audit on main. Scanner mDNS API (AsyncServiceBrowser, async_request)
unchanged; scan tests pass.

ha-relevant: no
2026-05-31 01:26:42 +02:00
RémyandGitHub 6761f73c17 Merge pull request #172 from Pouzor/feat/switch-ports-64-fibre-edge
feat(canvas): switch port cap 64 + port numbers (#20) and fibre edge type (#21)
2026-05-31 01:09:36 +02:00
Pouzor 1431f5b19e feat(canvas): raise switch port cap to 64, add port numbers and fibre edge type
Issue #20: bump MAX_BOTTOM_HANDLES 48 -> 64 (covers 48+4 SFP switches) and add
a per-node "Show Port Numbers" toggle that labels each bottom connection point.

Issue #21: add `fibre` as a first-class edge/connection type (bright cyan with a
subtle glow) alongside ethernet/wifi/iot/vlan/virtual/cluster - selectable in the
edge modal, themeable, registered in the React Flow edgeTypes registry, and
round-tripped through YAML import/export.

Backport of homelable-hacs PR #23.

ha-relevant: yes
2026-05-29 17:56:51 +02:00
Pouzor b5628e18fa chore(release): bump version to 2.2.0 2026-05-29 10:42:19 +02:00
RémyandGitHub dc70805673 Merge pull request #166 from CodeWarrior4Life/pr-laptop-mobile-types
Add laptop and mobile node types
2026-05-29 10:25:35 +02:00
Pouzor 05c24d622b style(themes): align laptop/mobile keys + theme-coherent mobile accents
- Pad laptop:/mobile: to match each theme's existing key-column width
  (dark, light, custom were off by 1-4 spaces).
- Replace hardcoded #ec4899 mobile pink with palette-coherent accents:
  light  -> #db2777 (contrast on light bg)
  neon   -> #ff3399 (neon family)
  matrix -> #00cc66 (green palette)
  default/dark/custom keep #ec4899.
2026-05-29 10:12:04 +02:00
RémyandGitHub 541e25327b Merge pull request #158 from pranjal-joshi/feat/collapsible
feat: add collapsible/expandable zones for canvas decluttering
2026-05-29 09:38:20 +02:00
Pouzor 9823be9d78 fix(liveview): apply collapse filter in read-only canvas
LiveView.tsx passed raw nodes/edges from the store straight to
ReactFlow, bypassing the collapse pipeline that CanvasContainer applies
in the editor. Result: a group or zone marked collapsed in edit mode
still showed all its contents on /view?key=... and in standalone live
view. The flag was being persisted and read correctly — only the
read-only canvas ignored it.

Reuse the same memoized computeCollapseInfo + rewireEdgesForCollapse
pipeline. Add two regression tests that load a /liveview payload with
a collapsed group and assert the children never reach ReactFlow.
2026-05-29 09:27:45 +02:00
Pouzor b71c96897a fix(canvas): make collapse reachable + persist for every node type
Three connected bugs in PR #158's collapse feature:

1. Toggle wired to the wrong component
   The chevron was on GroupRectNode and computed children via React Flow
   parentId. But in this codebase parentId is set by createGroup() on
   type=group nodes, not on groupRect zones — zones are decorative
   rectangles. Result: childrenCount was always 0 on every zone and the
   button never rendered, so the feature was unreachable from the UI.

   Fix:
   - Add the same chevron toggle to GroupNode (the actual container).
     parentId children are already known there, so the existing BFS in
     computeCollapseInfo hides them when data.collapsed flips.
   - For GroupRectNode, switch childrenCount to spatial containment so
     drawn zones also work: hit-test other nodes' bbox centres against
     the zone bbox.

2. Visibility filter ignored spatial zones
   Extend computeCollapseInfo with a second pass that hides every node
   whose centre lies inside a collapsed groupRect, plus the parentId
   subtrees of those nodes (so a Proxmox host inside a collapsed zone
   takes its VMs/LXCs with it). Edge rewiring routes vanished endpoints
   to the same visible zone via a unified hiddenBy map populated by both
   passes.

3. Save dropped data.collapsed for every type except groupRect
   The DevTools payload was the smoking gun: for a type=group node the
   serializer wrote custom_colors: {show_border: true} with no collapsed
   key, so the backend stored a stale false on every save. Only the
   groupRect branch of serializeNode/deserializeApiNode stashed and
   hoisted the flag. Move the stash + hoist to the general branch too
   (backend's custom_colors is dict[str, Any] so no schema change).

Tests: 11 new cases for spatial containment + GroupNode toggle UI, and
4 round-trip cases for collapse on non-groupRect types.
2026-05-29 08:58:36 +02:00
Pouzor 20e1820a4e refactor(canvas): promote collapsed to first-class NodeData field + edge rewire
Three follow-ups to PR #158 review:

1. Promote collapsed to NodeData.collapsed
   The flag was previously stashed inside NodeData.custom_colors, which is
   a colors/style object — semantically wrong. Move it to a first-class
   boolean on NodeData. Persistence keeps the existing API shape: serialize
   writes it into the custom_colors blob (alongside width/height/z_order,
   matching how groupRect already stashes layout metadata), and deserialize
   hoists it back. Legacy saves from the original PR shape load correctly.

2. Re-route cross-zone edges to the collapsed ancestor
   Previously any edge touching a hidden node was dropped, so a Zigbee
   coordinator outside a collapsed mesh lost all visible links to it.
   rewireEdgesForCollapse now walks each endpoint up the parentId chain to
   its nearest visible ancestor, surfaces a single stub edge on the
   collapsed zone, de-dupes parallel rewires (a 20-device mesh becomes one
   stub, not twenty), and drops edges that would self-loop on a zone or
   reference an orphan.

3. Revert package-lock.json churn
   The 63-line diff from the original PR was npm-version drift (libc
   arrays stripped from optional deps), unrelated to the feature.

Tests:
- canvasStore.collapse: updated to assert on data.collapsed.
- collapseFilter: 8 cases for visibility + 7 for edge rewire, covering
  cross-boundary, nested collapse, sibling self-loop, mesh dedup, and
  orphan endpoints.
- canvasSerializer.collapse: round-trip + legacy-shape compat.
2026-05-29 01:34:21 +02:00
Pouzor 517486ff79 perf(canvas): memoize collapse visibility filter + add tests
Extract getVisibleNodeIds/filterVisibleEdges from CanvasContainer into
src/utils/collapseFilter.ts. Replace inline O(n^2) BFS (nested array
.find per node) with O(n) traversal backed by parentId->children and
id->node Maps, and wrap consumer calls in useMemo so visibility is
recomputed only when nodes/edges change rather than on every render.

Add 12 unit tests covering the filter logic that the original PR left
untested: single-level collapse, multi-level subtree hiding via
collapsed ancestor, sibling isolation when an inner zone is collapsed,
empty zones, missing custom_colors, insertion-order independence, and
edge filtering for hidden source/target.
2026-05-29 01:09:46 +02:00
Cyril Grosse III b5b1056ae6 Add laptop and mobile node types
Personal computing devices (laptops, phones, tablets) currently collapse
into the generic icon because the type vocabulary has no entries for
them. This adds two new NodeTypes with Lucide icons:

- laptop  -> Laptop icon (reuses the computer accent color per theme)
- mobile  -> Smartphone icon (new pink accent #ec4899 across themes)

Touches:
- types/index.ts                    NodeType union + NODE_TYPE_LABELS
- utils/nodeIcons.ts                Lucide import + ICON_REGISTRY +
                                    NODE_TYPE_DEFAULT_ICONS
- utils/themes.ts                   nodeAccents in all 6 themes
- canvas/nodes/index.tsx            LaptopNode + MobileNode wrappers
- canvas/nodes/nodeTypes.ts         register in react-flow nodeTypes
- modals/NodeModal.tsx              new "Personal" type group
- modals/CustomStyleModal.tsx       expose new types in style editor
- types/__tests__/types.test.ts     enumerate new types
- utils/__tests__/themes.test.ts    enumerate new types

Backwards-compatible: existing nodes typed as 'generic', 'server', etc.
keep rendering exactly as before. No data migration required.
2026-05-28 15:55:46 -04:00
RémyandGitHub 66a9a57861 Merge pull request #163 from Pouzor/feat/lxc-mcp-install-script
feat(scripts): LXC/bare-metal MCP install script
2026-05-28 00:50:49 +02:00
Pouzor 529c75a175 feat(scripts): lxc-mcp-install env-var overrides + repo clone fallback
- All prompted values overridable via env vars (MCP_API_KEY,
  MCP_SERVICE_KEY, BACKEND_URL, INSTALL_DIR, etc.).
- Clone the repo into INSTALL_DIR if it isn't already present, so the
  script can be fetched and run directly inside a fresh LXC created by
  the community-scripts/ProxmoxVE helper (no manual git clone first).
- README: clarify the Proxmox flow (community-scripts creates the LXC,
  user runs this script inside it).
2026-05-27 22:27:54 +02:00
Pouzor fc765fa255 ci(docker): publish homelable-mcp image to GHCR
Add mcp matrix entry in docker-publish workflow so the MCP server
image is built and pushed alongside backend/frontend. Also wire the
prebuilt compose file so users can pull and run it directly.

Closes #162
2026-05-27 21:32:55 +02:00
Pouzor 77159ce1cd feat(scripts): add LXC/bare-metal MCP install script
Adds scripts/lxc-mcp-install.sh for Proxmox LXC and other non-Docker
hosts. Creates a homelable-mcp systemd service, prompts for
MCP_API_KEY / MCP_SERVICE_KEY (auto-generated on Enter), and skips
prompts when mcp/.env already exists so user edits are preserved.

Closes #132
2026-05-26 15:13:14 +02:00
pranjal-joshiandCyberClaw 69aa8256f0 fix: add null-safe default for nodes array in GroupRectNode
- Use nullish coalescing operator to provide empty array default
- Prevents 'Cannot read properties of undefined' error when nodes is undefined
- Fixes failing GroupRectNode tests that don't provide mock nodes

Co-authored-by: CyberClaw <noreply@openclaw.ai>
2026-05-18 03:03:28 +00:00
pranjal-joshiandCyberClaw 78b43a300f fix: simplify collapse tests to focus on unit tests, avoid act() warnings
- Replace component render tests with unit tests on types and state logic
- Tests now verify: type definitions, optional properties, toggle logic, nesting support
- Removes complex mocking and React component testing that triggers act() warnings
- Full integration testing is covered by CanvasContainer tests
- Reduces test file from 142 lines to focused unit tests

Co-authored-by: CyberClaw <noreply@openclaw.ai>
2026-05-18 02:59:36 +00:00
pranjal-joshiandCyberClaw 995de26591 fix: remove any types and unused variables in test file
- Replace 'any' types with proper TypeScript types (unknown, jest.Mock, Record)
- Remove unused 'container' destructuring variable
- Use document.querySelector instead of container.querySelector
- Fixes ESLint errors: @typescript-eslint/no-explicit-any, @typescript-eslint/no-unused-vars

Co-authored-by: CyberClaw <noreply@openclaw.ai>
2026-05-18 02:40:05 +00:00
pranjal-joshiandCyberClaw 525dfe5ece feat: add collapsible/expandable zones for canvas decluttering
- Add collapsed state to NodeData.custom_colors (type=groupRect only)
- Implement toggleNodeCollapsed action in canvasStore (Zustand)
- Extend GroupRectNode UI with smooth chevron toggle button
  - Rotating chevron icon (↓ → when collapsed)
  - Shows '+N' badge when zone is hidden
  - Reduces zone opacity to 0.6 when collapsed
  - All transitions target 60 FPS (ease-out 200ms)
- Filter child nodes/edges in CanvasContainer based on parent collapse state
  - Breadth-first traversal handles multi-level nesting
  - Connecting edges to hidden nodes are automatically hidden
- Add comprehensive test coverage
  - Store: toggleNodeCollapsed state mutation, unsaved flag
  - Component: chevron rendering, click handlers, opacity transitions
- Persist collapsed state via YAML serialization (part of custom_colors)

Benefits:
- Declutter large Zigbee meshes, multi-building networks
- Preserve layout structure without deleting nodes
- Smooth 60 FPS transitions for UX polish

CONTRIBUTING.md compliance:
- Strict TypeScript types, no 'any'
- Zustand store pattern, no prop drilling
- Tests for store logic and component behavior
- Frontend linting requirements met

Co-authored-by: CyberClaw <noreply@openclaw.ai>
2026-05-17 19:37:04 +00:00
Pranjal JoshiandGitHub d066f37e88 Merge branch 'Pouzor:main' into main 2026-05-18 00:51:14 +05:30
Pranjal JoshiandGitHub 004623bae5 Merge pull request #1 from pranjal-joshi/feat/zigbee
feat: add Zigbee2MQTT network map importer
2026-05-18 00:50:30 +05:30
Pouzor f8635df1c5 bump: version 2.1.1 2026-05-17 00:30:21 +02:00
RémyandGitHub 1cc9b7c52f Merge pull request #155 from Pouzor/fix/liveview-parity
fix(liveview): nest docker children, apply theme & custom style
2026-05-16 22:28:38 +02:00
Pouzor fdf2b1f2be chore(liveview): include setTheme/setCustomStyle in effect deps 2026-05-16 21:52:52 +02:00
Pouzor 5630e7d202 fix(liveview): nest docker children, apply theme & custom style
- Live view rebuilt parent map from proxmox/group only, so docker_container
  (and any vm/lxc with container_mode=true) lost parentId and rendered
  outside its host. Match App.tsx logic: include any node with
  container_mode=true.
- Live view ignored viewport.theme_id and custom_style on load, so the
  shared canvas always rendered in default theme. Apply both like the
  authenticated canvas does.
- Backend /liveview response now includes custom_style from CanvasState.

Adds regression tests on both ends.
2026-05-16 21:49:55 +02:00
Pouzor be705f0cb9 bump: version 2.1.0 2026-05-16 18:18:46 +02:00
RémyandGitHub 1790d24ee7 Merge pull request #154 from Pouzor/feat/docker-container-lxc-parent
feat(node-modal): sanitize parent_id on type change and exclude descendants
2026-05-16 18:03:31 +02:00
Pouzor 8b97f578c6 feat(node-modal): sanitize parent_id on type change and exclude descendants
- App.tsx: filter descendants of edited node from parentCandidates to
  prevent picking a child as parent (would create a cycle).
- NodeModal.tsx: clear parent_id when switching to a type with no valid
  parents, and revalidate parent_id on submit against the type's allowlist.
- Tests: cover getValidParentTypes for lxc/vm/docker_container and types
  that disallow parents.

ha-relevant: yes
2026-05-16 17:41:58 +02:00
RémyandGitHub 48e7b3c690 Merge pull request #153 from Pouzor/feat/docker-container-lxc-parent
feat(docker): allow LXC as parent for docker_container
2026-05-16 17:11:50 +02:00
Pouzor 3a4df578ec feat(node-modal): restore Parent Container selector for child types
Adds back the Parent Container selector in NodeModal, shown when:
- child type is lxc or vm (parents: proxmox/vm/lxc/docker_host)
- child type is docker_container (parents: docker_host/lxc/vm/proxmox)

Selector hidden when no valid candidate node exists on the canvas.
Centralizes the parent-type allowlist in getValidParentTypes
(shared with the virtual-edge auto-parent helper).

ha-relevant: yes
2026-05-16 17:08:06 +02:00
Pouzor 43426e96ac feat(docker): allow vm and proxmox as parent for docker_container
Extend docker_container parent rule to accept vm and proxmox nodes
in addition to docker_host and lxc, covering nested virtualization
topologies (Docker on a VM, Docker directly on Proxmox host).

ha-relevant: yes
2026-05-16 16:47:14 +02:00
Pouzor 280d5a2ae1 feat(docker): allow LXC as parent for docker_container
Previously docker_container could only attach to a docker_host via
virtual edge. Extend the rule so an LXC node can also act as parent,
enabling docker_container nesting under LXC without requiring an
intermediate docker_host.

Extracts the virtual-edge parent rule into a pure helper
(resolveVirtualEdgeParent) with unit tests covering all type pairs.

ha-relevant: yes
2026-05-15 22:54:51 +02:00
RémyandGitHub cdc6091bd0 Merge pull request #152 from Pouzor/feat/group-edge-handles
feat(group): add 4 side handles for edge connections
2026-05-15 21:19:31 +02:00
Pouzor 14597ab86f feat(group): add 4 side handles for edge connections
Group nodes can now be source/target of edges via snap points on
each side (top/right/bottom/left). Each side exposes a source handle
plus an invisible target overlay, matching the pattern used by
BaseNode.
2026-05-15 21:12:10 +02:00
RémyandGitHub 14719a5c39 Merge pull request #136 from findthelorax/feat/multiple-ips
feat: multiple ips in detail panel
2026-05-15 14:40:47 +02:00
Pouzor ea451885af refactor(ip-parsing): centralize multi-delim splitting in splitIps
Extend splitIps to accept comma/semicolon/newline delimiters with dedupe,
and reuse it in DetailPanel for both the IP list and the service-URL host.
Fixes ServiceBadge links pointing at concatenated string when IPs were
entered with ';' or newline separators. Adds tests for new delimiters.
2026-05-15 13:36:15 +02:00
findtheloraxandPouzor 9cd93ef294 updated tests for multiple ips 2026-05-15 11:43:09 +02:00
findtheloraxandPouzor 8d937b2a0d removed comma and allow multiple in a row 2026-05-15 11:31:25 +02:00
findtheloraxandPouzor 60ea0fa11b made multiple ips all clickable separately 2026-05-15 11:31:25 +02:00
RémyandGitHub ff0d33128c Merge pull request #150 from Pouzor/feat/edge-endpoint-reconnect
feat(canvas): edge endpoint reconnect + proxmox container snap points
2026-05-15 11:10:47 +02:00
Pouzor 17d4ddd070 feat(canvas): edge endpoint reconnect + proxmox container snap points
- Selected edge shows interactive dots at both endpoints; drag onto any
  handle (different snap point or different node) to reconnect. Edge
  follows the cursor live during drag; drop on empty space leaves edge
  unchanged.
- Endpoint dots render in EdgeLabelRenderer above the node layer and walk
  the elementsFromPoint stack so the underlying handle wins over the
  dragged dot itself.
- ProxmoxGroupNode in container mode now renders bottom_handles snap
  points (was always single 'bottom' handle), matching BaseNode.
- New canvasStore.reconnectEdge action: swaps source/target/handles via
  normalizeHandle, snapshots history, marks dirty.
2026-05-15 03:05:38 +02:00
RémyandGitHub 4795dc40b8 Merge pull request #149 from Pouzor/feat/homepage-widget-stats
feat(api): stats endpoint for gethomepage widget
2026-05-15 01:16:48 +02:00
Pouzor 994ed9d77a feat(api): stats endpoint for gethomepage widget (closes #131)
Adds GET /api/v1/stats/summary, gated by HOMEPAGE_API_KEY env var and
X-API-Key header (hmac.compare_digest, matches the liveview pattern).

Payload:
  nodes / online / offline / unknown
  pending_devices (status='pending')
  zigbee_devices  (Node.ieee_address IS NOT NULL)
  last_scan_at    (max ScanRun.finished_at)

Disabled by default — endpoint returns 403 unless HOMEPAGE_API_KEY is set.
README documents activation and ships a ready-to-paste gethomepage
`customapi` widget snippet.

Tests cover: disabled-by-default, missing header, wrong key, empty DB,
and full aggregation across nodes/pending/zigbee/scan-runs.
2026-05-14 18:11:43 +02:00
RémyandGitHub 4d4befa916 Merge pull request #106 from findthelorax/fix/zone-styling
fix: zone styling
2026-05-14 15:58:45 +02:00
Pouzor 8f741691d0 fix(zone-modal): center slider thumb on track, drop dead code, cover a11y
- Center webkit slider thumb on 4px track (margin-top: -4.5px) so the
  opacity bullet no longer sits below the bar.
- Drop redundant inline accentColor; CSS module owns track color.
- Remove dead ::-ms-* prefix rules.
- Tests: assert aria-labels on grid/select triggers, cover getFontLabel
  happy path + unknown-value fallback via the Font selector DOM.
2026-05-14 15:35:19 +02:00
Pouzor c21fbac599 Merge remote-tracking branch 'origin/main' into fix/zone-styling 2026-05-14 14:03:46 +02:00
RémyandGitHub e9c66fceda Merge pull request #148 from Pouzor/feat/zigbee-properties
feat(zigbee): IEEE/Vendor/Model/LQI properties + security + tests
2026-05-14 13:47:12 +02:00
Pouzor 373960f6ea test(frontend): cover api client, ProxmoxGroupNode, CustomStyleModal 2026-05-14 13:25:41 +02:00
Pouzor aa17edf1d0 chore(security): drop passlib for direct bcrypt + reject CLI-flag check targets
- Replace passlib.CryptContext with bcrypt 4.2.1 directly (passlib is
  unmaintained and only emitted warnings when paired with bcrypt 4+).
  hash_password / verify_password now call bcrypt.checkpw / .hashpw, and
  verify_password is hardened against empty inputs without raising.
  scripts/hash_password.py + tests/conftest.py updated to match.
- status_checker.check_node() now rejects targets starting with '-' before
  any subprocess invocation, defending ping/tcp paths against arg-injection
  if an admin sets a check_target like '-O'.

Tests added:
- test_auth: expired JWT, malformed JWT, wrong-secret JWT, empty password
  vs empty server hash, verify_password edge cases.
- test_scan: _background_scan failure path marks ScanRun failed, leaves
  non-running terminal status alone, success path invokes run_scan.
- test_status_checker: ping/tcp invocations are bypassed when target or
  ip starts with '-'.

Backend coverage 89% → 90%.
2026-05-14 12:11:40 +02:00
Pouzor 5f87c64dcf fix(zigbee): default IEEE/Vendor/Model/LQI props to hidden, preserve user visibility
- New zigbee props (approve + first-time re-import) ship with visible=false
  so the canvas card stays clean. User opts in from the right panel.
- On re-import of an already-approved node, merge instead of overwrite:
  keys already present keep their visible flag (and any user-edited value
  is replaced with the freshly imported one), brand-new keys are appended
  hidden. Non-zigbee custom properties are preserved untouched.
2026-05-14 01:10:45 +02:00
Pouzor 5822a1483a fix(zigbee): inject IEEE/Vendor/Model/LQI props into approved nodes client-side
Backend already writes the properties on approve, but the frontend's local
addNode() call rebuilt NodeData without them — so newly approved zigbee
nodes showed an empty right panel until a full canvas reload. Build the
property list on the client from the PendingDevice fields so the canvas
state matches the DB row immediately. Single + bulk approve flows updated.
2026-05-14 00:45:58 +02:00
Pouzor 3a303a1376 feat(zigbee): populate IEEE/Vendor/Model/LQI properties on approve and re-import
- Approval (single + bulk) of zigbee pending devices now writes IEEE, Vendor,
  Model, LQI into Node.properties so they show in the right panel.
- Zigbee re-import refreshes properties on existing canvas Nodes and skips
  creating a pending row when the device was already approved — keeps
  approved devices out of pending/hidden modals.
- Coordinator Node also receives the same properties on first creation and
  on re-import.
- Remove the Parent Container selector from the add/edit node modal.
2026-05-14 00:17:13 +02:00
RémyandGitHub f5e0e68806 Update README.md 2026-05-13 11:49:14 +02:00
Pouzor 8541922386 bump: version 2.0.3 2026-05-13 11:37:06 +02:00
RémyandGitHub 3ed9cb0d4f Merge pull request #146 from Pouzor/fix/zigbee-no-check-method
fix(zigbee): hide check method in modal, force none/online for zigbee nodes
2026-05-13 11:35:10 +02:00
Pouzor fff11a4b6a fix(zigbee): hide check method in modal, force none/online for zigbee nodes
Zigbee nodes have no IP-based check — hide Check Method and Check Target
fields in NodeModal for all three zigbee types. Default and force
check_method='none' so the scheduler marks them always online.
Backend approve and zigbee import routes also set status='online' and
check_method='none' for zigbee node types.
2026-05-13 11:14:35 +02:00
Brett FerranteandGitHub 890463373a Merge branch 'Pouzor:main' into fix/zone-styling 2026-05-12 14:40:41 -04:00
Pouzor 0680566081 bump: 2.0.2 2026-05-11 19:55:21 +02:00
RemyandGitHub 2f5a90a00e Merge pull request #142 from Pouzor/fix/visual
Node modal polish + Smart Home and Brand icon pickers
2026-05-11 19:53:48 +02:00
Pouzor 928f63df0f fix(icons): narrow ICON_MAP lookup type for strict build
tsc -b (used in npm run build) flagged TS2774 because LucideIcon is a
function and therefore always truthy. Cast the lookup result to
LucideIcon | undefined so the falsy branch becomes meaningful.
2026-05-11 19:44:20 +02:00
Pouzor e84a4e0eb3 merge feat/brand-icons into fix/visual 2026-05-11 19:20:10 +02:00
Pouzor 3b0dbd7a8b feat(icons): add brand icon picker from dashboard-icons
Add a second tab in the node Icon picker to choose from the ~2250 brand
icons hosted by homarr-labs/dashboard-icons (Plex, Sonarr, Home
Assistant, etc.) served via jsDelivr CDN. The Generic tab keeps the
existing lucide picker unchanged.

Storage uses a 'brand:<slug>' prefix on custom_icon, so existing nodes
referencing lucide keys keep working with zero migration. A new
resolveCustomIcon helper returns a discriminated union (lucide | brand)
and a NodeIcon component centralizes rendering for both kinds.

Includes a manifest fetch script (scripts/fetch-dashboard-icons.mjs)
and a checked-in dashboardIcons.json snapshot.
2026-05-11 19:18:01 +02:00
Pouzor 3a9b3b2650 feat(icons): add Smart Home / Sensors icon category
Add 27 new icons covering common IoT/Zigbee endpoints: smart plug,
relay, energy meter, solar, door/window sensor, smart lock, smoke
detector, siren, motion radar, presence, vibration, water leak,
humidity, air quality, HVAC vent, fan, AC, smart light, blinds,
doorbell, speaker, remote, garage, valve, weather station, plus
voice assistant and webhook in the existing Automation category.
2026-05-11 17:08:11 +02:00
Pouzor ff02f3b5db fix(node-modal): cap height at 90vh with scroll
Modal grew taller than viewport when icon picker expanded, hiding
header and footer buttons. Constrain DialogContent to 90vh and add
overflow-y-auto so all controls stay reachable.
2026-05-11 16:54:33 +02:00
Pouzor a66e6aa906 bump: version 2.0.1 2026-05-11 16:11:51 +02:00
RemyandGitHub 8e59d9a0d1 Merge pull request #141 from Pouzor/fix/zigbee-node-types-in-modal
fix(node-modal): expose Zigbee device types in selector
2026-05-11 16:09:07 +02:00
Pouzor ee4136b506 fix(node-modal): expose Zigbee device types in selector
Add Zigbee section (coordinator, router, end device) to NodeModal
type group list, matching types used by zigbee2mqtt import.
2026-05-11 15:55:34 +02:00
Pouzor 8e8f01d6ca bump version 2.0.0 2026-05-11 02:23:48 +02:00
Pouzor aba4f34168 feat(style-modal): expose Zigbee node types for customization
Add zigbee_coordinator, zigbee_router, zigbee_enddevice to the
CustomStyleModal editable list with Radio/Zap/Lightbulb icons so users
can tweak color, opacity, and size like any other node type.
2026-05-11 01:44:24 +02:00
RemyandGitHub 6524efe126 Merge pull request #139 from Pouzor/feature/node-alignment-guides
feat(canvas): alignment guides + snap while dragging
2026-05-11 01:29:44 +02:00
Pouzor 062a08f116 fix(alignment): only snap top-level dragged nodes
Code review flagged silent corruption when dragging a mixed selection
of a parent + one of its children: the child's ids stayed in
pendingSnap even though nodeBox excluded it from the bbox. On drag
stop we shifted the child's parent-relative position by the same delta
the parent already moves by — double-snapping the child off-screen.

Restrict the pendingSnap id set to nodes that contributed a box
(top-level only). Children follow their parent's move automatically;
no extra shift is needed.
2026-05-11 01:23:37 +02:00
Pouzor 5e9963db4d refactor(alignment): defer snap to drag stop, stabilise callback deps
Code review flagged a drag race + deps issues:

- Race: setNodes during onNodeDrag fights React Flow's internal drag
  handler, which derives positions from the cursor offset captured at
  drag start. Snap delta could be overwritten on the next frame.
- Deps: useCallback closed over guides.length, causing recreation on
  every guide change and pulling stale references into onNodeDrag.

Fix: compute guides live during drag for visual feedback, store the
pending snap delta in a ref, and apply it via setNodes only on
onNodeDragStop. Replace the guides.length dep with a ref mirror updated
via useEffect (not during render).
2026-05-11 01:14:47 +02:00
Pouzor 82203f67d8 fix(alignment): use OnNodeDrag (NodeDragHandler doesn't exist in @xyflow/react)
CI build failed with TS2305: NodeDragHandler is not exported. Local
typecheck missed it because tsc-noEmit is more permissive than tsc -b.

Use the exported OnNodeDrag<NodeType> type and alias it with our
NodeData so the callback params are inferred (no implicit any).
2026-05-11 01:00:36 +02:00
Pouzor 004a8f19c1 feat(canvas): alignment guides + snap while dragging nodes
Draw.io / Figma style: while dragging a node, show dashed cyan guide
lines when its edges (left / center / right / top / middle / bottom)
align with another node's, and snap the position to the matched line
within a configurable threshold.

- utils/alignment.ts: pure snap math, returns delta + guide segments.
  Same-size boxes show all aligned guides simultaneously.
- canvas/AlignmentGuides.tsx: SVG overlay locked to the React Flow
  viewport (panned/zoomed correctly).
- hooks/useAlignmentGuides.ts: wires onNodeDrag/onNodeDragStop, applies
  snap via setNodes, listens for Alt to temporarily disable.
- utils/alignmentSettings.ts: localStorage-backed prefs (enabled,
  threshold 2-16px) with a tiny CustomEvent pub-sub so the SettingsPanel
  and the drag hook stay in sync without a global store.
- Sidebar settings panel: toggle + threshold slider.

Multi-selection drag uses the union bounding box. Children with
parentId are skipped for v1 to avoid mixing absolute and parent-relative
coordinates. Hold Alt to escape snap.
2026-05-11 00:35:50 +02:00
Pouzor 5ab0bdeb7f fix(text-node): persist text in label so it survives reload
text_content is not in the API serializer schema, so text node content
was dropped on save and the node came back empty after reload.

Store text in label instead — already persisted, and TextNode + the
edit modal already fall back to label, so existing data stays
compatible. Clear stale text_content on update.

Regression test added for the text-node save/load roundtrip.
2026-05-11 00:34:38 +02:00
Pouzor 3f9866e8a1 fix(detail): handle non-Z timezone offsets in Last Seen
Backend sends ISO timestamps with +00:00 offset. Code blindly appended
'Z' when the string didn't end with 'Z', producing '...+00:00Z' which
parses to Invalid Date.

Detect any timezone marker (Z, z, or numeric offset like +00:00) and
only append 'Z' for naive datetimes.

Co-authored-by:
2026-05-10 21:45:22 +02:00
Pouzor ebdf6cb55b fix(scan): default check_method='ping' for approved devices
Approving a pending device created a Node with check_method=NULL, so
the status scheduler silently skipped it (scheduler.py:64 filters
falsy check_method) and the bubble stayed grey forever.

Default to 'ping' when the device has an IP, in both single and bulk
approve paths. Also persist caller-supplied check_method/check_target
in the single-approve path.

Add regression tests asserting default check_method='ping' after
approval for both endpoints.
2026-05-10 20:33:49 +02:00
Pouzor ec8f1c87f1 fix(status): use ms timeout for ping on macOS
ping(8) -W flag is milliseconds on macOS, seconds on Linux. Code used
-W 1 for both, giving macOS a 1ms timeout that fails any host with
RTT >1ms (typical wifi/mesh is 2-10ms).

Use -W 1000 on darwin to match the intended 1s timeout. Add regression
test asserting the platform-specific value.
2026-05-10 19:45:36 +02:00
Pouzor 2b91f31807 fix(group): preserve children + size on edit, fix status WS proxy
- Skip NodeModal on double-click for 'group'/'groupRect' (use inline rename)
- Exempt 'group' from properties-clears-height rule in updateNode
  (was collapsing group + hiding children with extent: 'parent')
- Enable ws: true on Vite /api proxy so /api/v1/status/ws/status upgrades
  (status bubbles stayed grey because WS handshake failed)
- Add regression tests for group/groupRect height preservation
2026-05-10 19:30:04 +02:00
RemyandGitHub c6965b827b Merge pull request #138 from Pouzor/feature/text-node
feat: add Text node type to canvas
2026-05-10 16:58:41 +02:00
Pouzor afdea65d4e feat: add demo text nodes with onboarding hints
Top-right banner: 'This is demo canvas, start with fresh scan'.
Bottom-left hint: 'You can remove all nodes'.
2026-05-10 16:50:39 +02:00
Pouzor 32eb9fbe23 fix: add 'text' entry to nodeIcons + all themes
Build failed because Record<NodeType, ...> in nodeIcons.ts and
themes.ts requires every NodeType key. Add Type icon for text and
neutral border/icon palette per theme.
2026-05-10 16:31:38 +02:00
Pouzor 27fd78ece6 feat: add Text node type to canvas
New canvas object Text with edit/create modal: text content (single
or multi-line), font (Inter, JetBrains Mono, Serif, System Sans),
size, color, border (style + width + color), background. Double-click
on node opens edit modal. Sidebar gets "Add Text" entry below "Add Zone".

Refs #124
2026-05-10 15:58:35 +02:00
RemyandGitHub e672cb086a Merge pull request #107 from findthelorax/feature/services-node-toggle
feature: services node toggle
2026-05-10 03:29:31 +02:00
findtheloraxandPouzor 3cddcf5b81 added cursor pointer to edit and delete buttons on services 2026-05-10 03:15:57 +02:00
findtheloraxandPouzor 19791abf5b adjusted show services toggle for alignment 2026-05-10 03:15:57 +02:00
findtheloraxandPouzor d9d4be9a53 services styling in node and detail panel 2026-05-10 03:15:57 +02:00
findtheloraxandPouzor c165b8b016 rebase with new main branch 2026-05-10 03:15:57 +02:00
findtheloraxandPouzor b16e81cf29 fixed some styling inconsistencies, removed show services toggle from affecting custom colors 2026-05-10 03:15:57 +02:00
findtheloraxandPouzor 3028e5a3f7 fix: cleanup 2026-05-10 03:15:57 +02:00
findtheloraxandPouzor 340bd15099 fix: external link icon on the detail panel for non URL services 2026-05-10 03:15:57 +02:00
findtheloraxandPouzor d181cef4c1 fix: external link icon set to opacity 0 when no URL 2026-05-10 03:15:57 +02:00
findtheloraxandPouzor 4a048d0ab5 fix: styling 2026-05-10 03:15:57 +02:00
findtheloraxandPouzor f32c32f6a5 feature: added a toggle to show the services on a node 2026-05-10 03:15:57 +02:00
RemyandGitHub d1f170d5db Merge pull request #128 from pranjal-joshi/feat/zigbee
feat: add Zigbee2MQTT network map importer
2026-05-10 02:35:16 +02:00
Pouzor 0a680fb9f2 fix(deps): bump fast-uri to patch GHSA-q3j6-qgpj-74h6 / GHSA-v39h-62p7-jpjc
Resolves the npm audit high-severity findings that broke the
dependency-audit job in the Security workflow.
2026-05-10 02:16:52 +02:00
Pouzor 9db41e886a fix(pending): respect active filters when clearing pending devices
Trash button in PendingDevicesModal now removes only filtered devices
instead of wiping the whole pending list. Falls back to clearPending
when no filters are active.
2026-05-10 01:51:11 +02:00
Pouzor 0863c2db94 feat(zigbee): import as background scan run
Mirrors IP scan flow: POST /zigbee/import-pending now creates a
ScanRun(kind=zigbee, status=running) and returns immediately.
Networkmap fetch + pending upsert run in the background, status
transitions to done/error when finished.

Frontend: import modal closes on submit, scan history shows the
run with a ZIG/IP kind chip and toasts on completion. Pending
modal auto-refreshes when run finishes.

scan_runs.kind column added (default 'ip', idempotent migration).
Existing zigbee tests refactored to exercise _persist_pending_import
directly (background tasks don't see the test session); route test
verifies the run is created with kind=zigbee.
2026-05-10 00:45:21 +02:00
Pouzor b17299f531 fix(pending): drop dangling onNodeApproved call + add keydown deps
Single approve was calling a removed prop ref (caught by review).
Keydown effect now has explicit deps so it doesn't re-bind every render.
2026-05-09 23:53:33 +02:00
Pouzor 277879488f feat(pending): full-screen modal with grid cards, filters, bulk restore
Replaces sidebar pending/hidden panels with a wide modal showing
devices as cards. Adds search, segmented source/status filters,
type filter, select mode for bulk approve/hide/restore, and
keyboard shortcuts. Hidden cards click-to-restore (no approval
detour); approval no longer pops the edit modal.

Backend: new restore + bulk-restore endpoints (hidden -> pending).
2026-05-09 23:40:44 +02:00
Pouzor 456f26a85a fix(ci): null-safe pending IP in SearchModal + bump python-multipart
- SearchModal.tsx: same null-IP guard as SearchBar; also search by
  friendly_name and ieee_address. Fixes Docker CI typecheck.
- python-multipart 0.0.26 -> 0.0.27 (CVE-2026-42561). Fixes
  Security pip-audit.
2026-05-08 00:53:18 +02:00
Pouzor 2a6140a6b1 chore(db): log instead of silently suppress new Zigbee migrations
Pre-push code review flagged the inline `with suppress(OperationalError)`
ALTERs as silently eating real errors. The project still uses inline
init_db migrations (CLAUDE.md aspires to Alembic but the runtime path
is inline), so as a minimum: add a `_try_migrate` helper that
distinguishes already-applied (debug log) from genuine failures
(warning log). Apply it to the Zigbee-related migrations only — the
legacy ALTERs above remain unchanged in this commit's scope.
2026-05-07 23:57:52 +02:00
Pouzor 3ae159d8d6 feat(zigbee): import to pending section with edge persistence
Coordinator auto-approves to a canvas Node; routers/end devices land in
pending_devices keyed by IEEE. Discovered parent->child edges are stored
in pending_device_links so that approving a pending device later
auto-creates the Edge once both endpoints exist as canvas Nodes.

- new POST /api/v1/zigbee/import-pending (default mode in modal)
- new pending_device_links table; ieee_address on nodes + pending_devices
- pending_devices.ip migrated to nullable (table rebuild on existing DBs)
- approve / bulk-approve return auto-created edges; sidebar pushes them
  into the canvas store with bottom -> top-t handles
- ZigbeeImportModal: radio toggle pending vs canvas; reset on close
- PendingDeviceModal: zigbee badge, IEEE/LQI/vendor/model rows, services
  hidden for zigbee
- Sidebar pending row: ZIG source badge, LQI badge, friendly_name fallback
- SearchBar: null-safe IP, also searches friendly_name and ieee_address
- Tooltip trigger uses asChild to avoid nested-button hydration error
2026-05-07 23:18:29 +02:00
Pouzor 5e567d4628 feat(zigbee): add slow-fetch hint in modal + bump timeout to 300s
- Modal: small italic note under Test/Fetch buttons warning users that
  large meshes can take several minutes
- Service: _NETWORKMAP_TIMEOUT 180s -> 300s (5 min) for very large meshes
2026-05-07 00:27:48 +02:00
Pouzor 05db9a59f5 fix(zigbee): strict tree edges + auto-select imported nodes
Edges
- Z2M `links` is bidirectional and includes router mesh paths, which
  caused duplicate edges and edges entering the coordinator from the
  bottom. Walk `links` only to derive parent_id + LQI; build final
  edges strictly from the parent->child hierarchy (one edge per
  non-coordinator node). Result: parent bottom -> child top, every time.
- Tests: bidirectional pair collapses to one edge, router-mesh siblings
  dropped, coordinator never receives an edge.

Auto-select
- After import, deselect existing canvas nodes and mark only the
  freshly-imported ones as selected, so the user can drag the whole
  subtree as a group.
2026-05-06 23:53:31 +02:00
Pouzor a46e505505 fix(zigbee): parse real Z2M networkmap shape (data.value.nodes/links)
The previous parser read `data.routes` which is just an echo of the
`routes` request flag (a boolean). On real brokers this caused
`TypeError: 'bool' object is not iterable` and 500s during /import.

- Rewrite parse_networkmap to read data.value.nodes + data.value.links
  with fallback to data.{nodes,links} for legacy variants
- Defensive: drop links to unknown nodes, propagate lqi from link to
  target node, extract model/vendor from definition block
- Bump networkmap timeout 10s -> 180s (large meshes are slow)
- Tests: rewrite fixture builders + sample payload to real Z2M shape;
  add cases for legacy shape, routes:false echo (regression), malformed
  list, link to unknown node, lqi propagation, definition extraction
- Update docs to mention 60s+ wait window

53 backend tests pass, mypy + ruff clean.
2026-05-06 22:38:15 +02:00
5b5866d8d6 fix: update axios to ^1.15.2 and audit fix
- Update axios from ^1.13.6 to ^1.15.2 to fix high-severity vulnerabilities (SSRF, prototype pollution, CRLF injection)
- Run npm audit fix to patch transitive dependencies
- Resolves dependency-audit CI failure

Co-authored-by: CyberClaw <noreply@openclaw.ai>
2026-05-06 16:56:03 +02:00
Pouzorandpranjal-joshi 9970780e7a fix(zigbee): sanitize MQTT error messages to prevent credential leakage
aiomqtt/paho exception strings can include the broker URI with embedded
credentials (mqtt://user:pass@host) or auth detail. The 502 response
from /import and the message field on /test-connection echoed these
verbatim via str(exc).

- Add _sanitize_mqtt_error() that maps known patterns (auth, refused,
  DNS, TLS, timeout) to coarse, credential-free categories
- Original exception still logged at WARNING level for operator debug
- Drop hostname:port from TimeoutError messages
- /test-connection unexpected-error path no longer interpolates exc

Tests: 6 new (auth/refused/DNS/TLS/unknown sanitization, end-to-end
fetch_networkmap leak check).
2026-05-06 14:55:51 +00:00
Pouzor 53aeb82af1 bump: dependancy 2026-05-06 16:37:07 +02:00
Pouzor d49610163f feat(zigbee): add MQTT TLS support with optional cert verify skip
- Schema: mqtt_tls + mqtt_tls_insecure flags on import + test-connection
  requests; model_validator enforces insecure requires tls
- Service: _build_tls_context() using ssl.create_default_context();
  logger.warning when verification disabled; tls_context plumbed into
  aiomqtt.Client for both fetch_networkmap and test_mqtt_connection
- Route: passes tls flags through to service
- Frontend: TLS checkbox auto-toggles port 1883<->8883 unless user
  edited; insecure checkbox disabled until TLS on, red-tinted; password
  field marked autocomplete=new-password
- Tests: 7 new backend tests (TLS context build, client kwargs assertion,
  router happy path, insecure-without-tls 422)
2026-05-06 16:30:23 +02:00
Pouzor 517130a9ef fix(zigbee): add zigbee node types to icons and themes maps
TS build failed in smoke-and-integration: NODE_TYPE_DEFAULT_ICONS and
all theme nodeAccents were missing zigbee_coordinator, zigbee_router,
zigbee_enddevice entries required by NodeType union.
2026-05-06 15:41:29 +02:00
Brett FerranteandGitHub c90538b1d0 Refactor Select component for font selection 2026-05-04 20:26:05 -04:00
pranjal-joshiandCyberKeys 67b0620be2 fix: resolve all mypy errors in zigbee_service
- Add dict[str, Any] type parameters throughout (mypy type-arg)
- Fix str | None returns with explicit 'or None' coercion (no-any-return)
- Remove unused type: ignore[import] comment (unused-ignore)
- Decode message.payload bytes before json.loads (arg-type)
- Use X | Y syntax in isinstance check (ruff UP038)
- All 33 tests still passing, ruff + mypy clean

Co-authored-by: CyberKeys <noreply@openclaw.ai>
2026-05-04 14:25:15 +00:00
pranjal-joshiandCyberKeys cc9c010002 fix: adhere to CONTRIBUTING.md — ruff clean + all tests passing
- Move aiomqtt to module-level import (enables proper patch() in tests)
- Remove unused variable (description) in zigbee_service — ruff F841
- Split long line (132 chars) to fit 120 char limit — ruff E501
- Fix import sort order in test files — ruff I001
- Remove unused imports (asyncio, AsyncMock, MagicMock) — ruff F401
- Rename test_mqtt_connection import alias to _test_mqtt_connection
  to avoid pytest fixture name collision (ERROR at setup)
- All 33 backend tests now pass (21 service + 12 router)
- TypeScript typecheck: 0 errors

Co-authored-by: CyberKeys <noreply@openclaw.ai>
2026-05-04 14:12:38 +00:00
pranjal-joshiandCyberKeys 103e24e5fa feat: add Zigbee2MQTT network map importer
- Backend: async MQTT service (aiomqtt) to fetch Z2M networkmap via bridge API
- Backend: FastAPI router at /api/v1/zigbee with /import and /test-connection
- Backend: Pydantic v2 schemas for request/response validation
- Backend: coordinator → router → end-device parent_id hierarchy builder
- Frontend: ZigbeeImportModal with MQTT config form, Test Connection, Fetch Devices
- Frontend: device list grouped by type (coordinator/router/enddevice) with checkboxes
- Frontend: ZigbeeCoordinatorNode, ZigbeeRouterNode, ZigbeeEndDeviceNode canvas nodes
- Frontend: Zigbee Import button in sidebar alongside Scan Network
- Frontend: handleZigbeeAddToCanvas wires selected devices + edges onto canvas
- Tests: full unit test suite for parser, hierarchy builder, MQTT mocks
- Tests: API endpoint tests for /zigbee/import and /zigbee/test-connection
- Tests: Vitest component tests for ZigbeeImportModal
- Docs: docs/zigbee-import.md with full usage, MQTT config, troubleshooting guide
- Docs: README.md Zigbee2MQTT Import section

Co-authored-by: CyberKeys <noreply@openclaw.ai>
2026-05-04 13:58:58 +00:00
Brett FerranteandGitHub 8110ee075d Merge branch 'Pouzor:main' into fix/zone-styling 2026-05-03 19:34:44 -04:00
Brett FerranteandGitHub 2994f8653a Fix import statement for GroupRectModal.module.css 2026-05-03 19:16:49 -04:00
RemyandGitHub 5fb77ab00b Merge pull request #122 from Pouzor/1.12.1
1.12.1: firewall node + 1–48 bottom connection points
2026-05-03 22:50:09 +02:00
Pouzor 56cfbd1e76 bump version 1.13.0 2026-05-03 22:44:38 +02:00
Pouzor 43761c60cb feat(handles): allow 1-48 bottom connection points via slider
Replaces the 4-option dropdown with a 1..48 slider. Existing handle IDs
('bottom', 'bottom-2'..'bottom-4') are preserved and the hand-tuned pixel
positions for counts 1..4 are locked in (regression-tested) so saved
canvases render identically. Counts >=5 use uniform spacing; node card
auto-grows in width so handles stay clickable. Existing edge-remap on
shrink scales unchanged thanks to its index-based loop.

ha-relevant: yes
2026-05-03 21:09:14 +02:00
Pouzor ad958feabd fix(node-modal): keep modal open when delete confirm is cancelled
Bare if without braces meant onClose() ran unconditionally after the
window.confirm — clicking Cancel still closed the modal and dropped
unsaved edits. Wrap the confirm body so onClose only fires on accept.
2026-05-03 17:16:05 +02:00
Pouzor e3876e934c feat(nodes): add firewall node type
New 'firewall' NodeType with Flame icon. Wired through node registry,
icon registry, all six themes (red-tinted accent per palette), NodeModal
Hardware group, and CustomStyleModal editable types.
2026-05-03 16:48:15 +02:00
Pouzor dfa4a9c849 fix(sidebar): unfreeze view after scan triggers history panel
forceView prop was overriding local activeView on every render. Once parent
set sidebarForceView='history' after starting a scan, it never cleared, so
clicks on other nav items were ignored. Treat forceView as a one-shot
trigger via the previous-prop pattern; user clicks afterward control view.
2026-05-03 16:03:30 +02:00
RemyandGitHub 785be6a5dd Create FUNDING.yml 2026-04-28 14:52:25 +02:00
Brett FerranteandGitHub d5f4a9f729 cleaned up file structure 2026-04-26 23:13:25 -04:00
findthelorax 8859893e42 fixed font casing and increased slider thumb size 2026-04-24 21:47:07 -04:00
findthelorax 93b98f760c changed font casing, revamped opacity slider and improved accessibility with aria labels and keyboard navigation 2026-04-24 21:08:34 -04:00
Brett FerranteandGitHub 30ed78c240 Merge branch 'Pouzor:main' into main 2026-04-24 20:41:48 -04:00
RemyandGitHub 39f8d16ef1 Merge pull request #113 from jackspiering/patch-1
Fix typo in 'optional' section header
2026-04-25 02:14:55 +02:00
Jack SpieringandGitHub 0095bf8425 Fix typo in 'optional' section header 2026-04-24 22:43:26 +02:00
Brett FerranteandGitHub 69436c438c Merge branch 'Pouzor:main' into main 2026-04-24 13:27:12 -04:00
Pouzor 2cc97a6de9 bump: version 1.12.0 2026-04-24 15:33:09 +02:00
RemyandGitHub 29a2ef1b20 Merge pull request #94 from findthelorax/bug/bezier-label
fix: bezier label
2026-04-24 15:23:54 +02:00
Pouzor 0c836e0575 Merge remote-tracking branch 'origin/main' into bug/bezier-label 2026-04-24 15:04:05 +02:00
RemyandGitHub 6518eb313b Merge pull request #108 from findthelorax/fix/modal-accessibility
fix: modal accessibility
2026-04-24 14:48:52 +02:00
Pouzor 09a591f5f4 fix: move IP address comma-separated hint below input to prevent label wrapping 2026-04-24 12:08:56 +02:00
Pouzor 988b804b90 fix: subtle hover border (#8b949e) and remove redundant tabIndex on button elements 2026-04-24 12:01:30 +02:00
Pouzor 6fa0ada325 merge: resolve EdgeModal conflict — keep CSS module classes + SelectValue label fix 2026-04-24 11:50:30 +02:00
Pouzor 17613f42d1 fix: style apply-to-existing buttons as primary blue actions 2026-04-24 11:42:31 +02:00
Pouzor 0a44b69c4e fix: add default size hint (min 140x50, auto) in custom style editor 2026-04-24 11:42:31 +02:00
Pouzor 84235d81bf fix: show edge type label in select trigger after selection 2026-04-24 11:42:31 +02:00
Pouzor 31b61904ac fix: add setCustomStyle to useEffect dependency array 2026-04-24 11:42:31 +02:00
Pouzor babbcb1dc5 feat: add custom style theme with per-type node/edge style editor
- New 'custom' ThemeId with dedicated ThemeCard + pencil edit button
- CustomStyleModal: two-column editor for all node types (excl. groupRect/group) and edge types
  - Per node: border, background, icon color + opacity, default size (w/h)
  - Per edge: color + opacity, path style, animation
  - 'Apply to existing [Type]' per type, 'Apply All to Canvas' footer action
- canvasStore: applyTypeNodeStyle, applyTypeEdgeStyle, applyAllCustomStyles actions
- themeStore: customStyle state + setCustomStyle action
- Backend: custom_style JSON column on canvas_state (ALTER TABLE migration), saved/loaded with canvas
- App.tsx: custom_style included in save payload, restored on load (API + standalone)
- Tests: +8 frontend (themeStore + canvasStore), +3 backend (canvas API)
2026-04-24 11:42:31 +02:00
findthelorax 12fde681ba removed closeIconProps on edge modal 2026-04-23 22:57:36 -04:00
Brett FerranteandGitHub 63922f0841 Merge branch 'Pouzor:main' into bug/bezier-label 2026-04-23 20:06:50 -04:00
findthelorax 896cd4fa21 made a lot of accessibility fixes for the modalsincluding pointers, keyboard navigation and consistent border radius 2026-04-23 20:06:10 -04:00
Brett FerranteandGitHub 8c9f1a23e0 Merge branch 'Pouzor:main' into main 2026-04-23 19:40:14 -04:00
findtheloraxandRemy 18f9bb7bdf allow the enter key to apply the currently selected style 2026-04-24 00:42:00 +02:00
findtheloraxandRemy da287d459c allow for keyboard navigation and adjusted card height to match regardless of text inside 2026-04-24 00:42:00 +02:00
findtheloraxandRemy adb2088752 revert: restore workflow to upstream version 2026-04-24 00:42:00 +02:00
findtheloraxandRemy cd0e08fb91 fixed canvas style options to fit better and be max 50vw 2026-04-24 00:42:00 +02:00
Brett FerranteandRemy 3ccdde0bea Update Docker image references to use repository owner 2026-04-24 00:42:00 +02:00
Lucas Van VonderenandRemy d0a49d0a0d fix(mcp): mount session manager via Starlette Mount to avoid double response.start
StreamableHTTPSessionManager is an ASGI app — it sends its own
http.response.start and http.response.body messages via the scope/receive/send
triple. Wrapping it inside a @app.api_route FastAPI handler causes FastAPI
to try to finalize the response after the handler returns, emitting a
second http.response.start. uvicorn rejects this with:

    RuntimeError: Unexpected ASGI message 'http.response.start' sent,
    after response already completed.

Every POST /mcp raises, making the server unreachable from any MCP client
(tested with Claude Code 2.x against mcp==1.27.0).

Fix: mount the session manager as a Starlette Mount so it owns the
response cycle directly. Auth middleware still applies because
add_middleware attaches at the app level, wrapping all mounted sub-apps.
2026-04-24 00:34:55 +02:00
findthelorax 31b5bc4515 fixed accessibility and keyboard navigation for edit node and connection modals 2026-04-23 16:00:55 -04:00
Brett FerranteandGitHub b05d70663c Merge branch 'Pouzor:main' into bug/bezier-label 2026-04-22 20:49:11 -04:00
Brett FerranteandGitHub 4e68af7cac Merge branch 'Pouzor:main' into main 2026-04-22 20:48:47 -04:00
Pouzor 528c362633 bump: version 1.11.0 2026-04-23 02:38:31 +02:00
Pouzor bfe520cd49 fix: use absolute positioning for status dot to prevent IP truncation 2026-04-23 02:35:23 +02:00
Pouzor 29c97ae501 fix: align status dot to top and restore gap in ProxmoxGroupNode header 2026-04-23 02:35:23 +02:00
findtheloraxandRemy 4ecd241bf4 enlarged node resizer border for easier resizing 2026-04-23 02:35:23 +02:00
findtheloraxandRemy 96786f155b aligned all status icon styling 2026-04-23 02:35:23 +02:00
Pouzor 1d6127fed3 fix(detail-panel): replace pixel math and fixed tooltip in ServiceBadge
- Remove magic-number width calculations (maxTotalWidth, nameCharWidth)
- Remove useState tooltip + position:fixed custom tooltip div
- Use CSS flex/truncate for layout and Shadcn Tooltip for path hover
- Add ServiceBadge rendering tests (name, port, path, link vs plain text)
2026-04-23 00:55:14 +02:00
findtheloraxandRemy 26633f760d aligned services and path styling 2026-04-23 00:55:14 +02:00
findtheloraxandRemy 745002593f reduces clickable space to the external link icon and the service name and added a tooltip to show truncated paths at the pointer 2026-04-23 00:55:14 +02:00
findtheloraxandRemy 1a978c5e51 prioritize service name and port, truncate path or hide when necessary 2026-04-23 00:55:14 +02:00
findtheloraxandRemy 16adff5cff prioritize service name, truncate path or hide when necessary 2026-04-23 00:55:14 +02:00
findtheloraxandRemy 45b0965fb7 path max width set to 80px 2026-04-23 00:55:14 +02:00
findtheloraxandRemy f1bcd6ef78 restore: tooltip.tsx 2026-04-23 00:55:14 +02:00
findtheloraxandRemy 26be37f731 aligned services and path styling 2026-04-23 00:55:14 +02:00
Pouzor ebc1d41d5c feat: add docker_container node type and fix container mode for non-proxmox types
- Add docker_container node type (Package icon, sky-blue accent) as child of docker_host
- Parent selector for docker_container filters to docker_host only via nodeType field
- Virtual edge drag-connect syncs parent_id for docker_container <-> docker_host
- Fix setProxmoxContainerMode: remove proxmox-only guard so width/height are properly
  set/cleared for all container-capable types (docker_host, vm, lxc)
- Fix handleAddNode: only give group size when container_mode=true, making create
  and reload behavior consistent (was giving size unconditionally for CONTAINER_MODE_TYPES)
- Add regression tests for docker_host container mode toggle and docker_container nesting
2026-04-22 22:41:01 +02:00
findtheloraxandRemy fe5e3c9858 revert: restore workflow to upstream version 2026-04-22 22:41:01 +02:00
findtheloraxandRemy 78b47384da fixed the container mode toggle styling for backward compatability and proper padding to remain centered in it's parent element 2026-04-22 22:41:01 +02:00
findtheloraxandRemy 38e6604f50 align canvasStore expectations with container-mode behavior 2026-04-22 22:41:01 +02:00
findtheloraxandRemy 0e70b45e8a code aligned cleanup 2026-04-22 22:41:01 +02:00
findtheloraxandRemy 7312132767 adjusted test to be more exhaustive of all container and parent node types 2026-04-22 22:41:01 +02:00
findtheloraxandRemy cb5b1bd2e2 adjusting wording by replacing proxmox with generics since multiple node types can be a parent node 2026-04-22 22:41:01 +02:00
findtheloraxandRemy c9a402fa0a fix: shows the proper parent node names once selected in the edit node modal 2026-04-22 22:41:01 +02:00
findtheloraxandRemy 9f880395da fix: adjusted deserialize for all virtualization node types 2026-04-22 22:41:01 +02:00
findtheloraxandRemy 585df726e7 feature: added a Container Mode toggle to the Virtualization group of node types 2026-04-22 22:41:01 +02:00
Brett FerranteandRemy 1a3cde3a02 Update Docker image references to use repository owner 2026-04-22 22:41:01 +02:00
Brett FerranteandGitHub bfd7ccd36c Merge branch 'Pouzor:main' into bug/bezier-label 2026-04-21 10:47:39 -04:00
Brett FerranteandGitHub ada30311ed Merge branch 'Pouzor:main' into main 2026-04-21 10:11:55 -04:00
findtheloraxandRemy a8ca4f1cca revert: restore workflow to upstream version 2026-04-21 16:08:40 +02:00
findtheloraxandRemy feb05a1df5 updated node modal with consistent casing for type and check method following standard conventions 2026-04-21 16:08:40 +02:00
Brett FerranteandRemy 0204a7ddeb Update Docker image references to use repository owner 2026-04-21 16:08:40 +02:00
Brett FerranteandGitHub 6ead20125d Merge branch 'Pouzor:main' into main 2026-04-21 09:51:22 -04:00
RemyandGitHub 03e1e32af1 Create LICENSE 2026-04-21 12:29:18 +02:00
Pouzor c9cd6a08fc add CONTRIBUTING.md 2026-04-21 12:23:50 +02:00
Pouzor daa78a036a bump: version 1.10.2 2026-04-21 11:52:43 +02:00
Pouzor 3deb750441 fix: NaN guard on settings interval input and validate release URL scheme 2026-04-21 11:43:15 +02:00
Pouzor 88554ef952 fix: stop click propagation on pending device checkbox to prevent modal opening 2026-04-21 11:37:43 +02:00
Pouzor 110592f89e fix: checkbox onChange anti-pattern in PendingDevicesPanel 2026-04-21 11:28:55 +02:00
Pouzor 8b8da5584c feat: add logout button to sidebar 2026-04-21 11:23:52 +02:00
findthelorax 8626fb2ca4 revert: restore workflow to upstream version 2026-04-20 23:28:11 -04:00
findthelorax 5952274c27 adding in path geometry for the add point + 2026-04-20 23:26:25 -04:00
findthelorax 9c035e2be2 adding in path geometry for the add point + 2026-04-20 23:24:26 -04:00
findthelorax cad3add223 bug: fixed an issue with the label and a bezier path with multiple points 2026-04-20 23:24:26 -04:00
Brett FerranteandGitHub fc888629c3 Merge branch 'Pouzor:main' into bug/bezier-label 2026-04-20 23:17:05 -04:00
Brett FerranteandGitHub 6a21cc729e Merge pull request #16 from findthelorax/findthelorax-workflow-owner-patch
Update Docker image references to use repository owner
2026-04-20 09:36:24 -04:00
Brett FerranteandGitHub 06370529c6 Update Docker image references to use repository owner 2026-04-20 09:35:53 -04:00
Pouzor 4260a6582c Bimp version 1.10.1 2026-04-20 15:05:12 +02:00
Pouzor a0f18dd237 fix: use custom icon in proxmox container mode header 2026-04-20 14:57:45 +02:00
Pouzor 00edc32aeb fix: show visible properties on proxmox container mode node header 2026-04-20 14:45:37 +02:00
findtheloraxandRemy cd6a788f77 fix: test was missing leading / 2026-04-20 14:37:08 +02:00
findtheloraxandRemy 9cf6a48b04 fix: port input to text numeric and removed up/down arrows 2026-04-20 14:37:08 +02:00
findtheloraxandRemy 2c94616afa cleanup path examples 2026-04-20 14:37:08 +02:00
findtheloraxandRemy c7be851c34 feature: add support for services to use a path 2026-04-20 14:37:08 +02:00
RemyandGitHub fddfd0a769 Merge pull request #90 from Pouzor/feat/zone-color-opacity
feat: add opacity slider to zone color pickers (fixes #72)
2026-04-20 14:13:35 +02:00
Pouzor 074b49358b feat: add opacity slider to zone color pickers (fixes #72)
The native <input type="color"> only supports 6-digit hex, stripping alpha
and forcing background/border/text colors to be fully opaque on edit.

Each color field now shows an opacity slider (0–100%) below the swatch.
Values are stored as 8-digit hex (#rrggbbaa). Existing zones with 6-digit
colors are handled transparently (alpha defaults to 100%).

- colorUtils.ts: hexToRgba / rgbaToHex8 helpers
- GroupRectModal: opacity sliders for all three color fields
- 26 new tests across colorUtils and GroupRectModal
2026-04-20 14:05:06 +02:00
RemyandGitHub a47b7649f0 Merge pull request #89 from Pouzor/feat/export-quality
feat: PNG export quality selector (standard / high / ultra)
2026-04-20 13:50:42 +02:00
Pouzor 7e08a85f73 feat: add quality selector to PNG export (standard / high / ultra)
Clicking Export PNG now opens a modal with three quality presets:
- Standard (1× pixel ratio) — small file
- High (2×, default) — recommended for sharing
- Ultra (4×) — print quality

Adds ExportModal component, updates exportToPng() to accept a quality
param, and wires the modal into App.tsx replacing the direct export call.
2026-04-20 11:40:16 +02:00
RemyandGitHub c7c5183356 Merge pull request #87 from findthelorax/fix/reset-form-data
fix: resets form data after submission
2026-04-20 10:48:02 +02:00
Pouzor 7608d07255 fix: flush before reading node IDs in bulk/single approve; 404/409 guards; catch scan errors
- db.flush() ensures node.id is populated before reading — fixes bulk approve
  where node_ids were null, causing frontend to skip addNode for every device
- approve_device raises 404 on missing device, 409 on already-processed device
- _background_scan rollbacks dirty session then marks run as "failed"
- Explicit Node() field mapping instead of **model_dump() to prevent injection
- update_scan_config rolls back in-memory change if save_overrides() fails
- clear_pending uses bulk DELETE instead of N individual row deletes
2026-04-20 10:41:02 +02:00
RemyandGitHub 1bc6798d76 Merge pull request #84 from findthelorax/feature/property-icons
feature: added new icons for properties
2026-04-20 10:07:49 +02:00
findthelorax f6de7d1770 fix: removed setState within an effect, responsibility moved to parent key 2026-04-20 00:13:07 -04:00
findthelorax 9dddd00858 fix: resets form data after submission 2026-04-20 00:06:29 -04:00
findthelorax a5bf9c9db6 feature: added new icons for properties 2026-04-19 21:00:20 -04:00
RemyandGitHub a9c5c538b4 Merge pull request #82 from Pouzor/1.10.0
1.10.0
2026-04-19 23:49:38 +02:00
Pouzor a816faa0b9 fix: prevent node width expansion when content overflows after resize
Proxmox nodes with container_mode=false fell through both width conditions
in deserializeApiNode and got no explicit width on reload, causing RF to
auto-size to content width and ignoring the user's manual resize.

- canvasSerializer: unified width restore logic — saved width applies to all
  node types; proxmox container_mode defaults (300x200) only kick in when
  no saved width exists
- BaseNode: add overflow-hidden + min-w-0 to properties row so truncate
  actually clips long values instead of expanding the node
2026-04-19 23:43:15 +02:00
Pouzor fbfacec6dc fix: prevent node from expanding beyond resized width on reload 2026-04-19 22:58:10 +02:00
Pouzor b5eb8d1b74 fix: remove duplicate primaryIp export in maskIp.ts after rebase 2026-04-19 22:30:50 +02:00
Pouzor 0193f933ce feat: bulk approve/hide pending devices (#70)
- Backend: POST /scan/pending/bulk-approve and /scan/pending/bulk-hide endpoints (registered before dynamic routes to avoid conflict); bulk-approve response includes device_ids for frontend mapping
- Frontend: PendingDevicesPanel gains per-row checkboxes, select-all, and a bulk action bar (Approve N / Hide N) that appears when ≥1 device is selected
- Tests: 6 new backend API tests + 7 new frontend UI tests for bulk selection flows
2026-04-19 22:13:40 +02:00
Pouzor 5ad5eba58c feat: add connection handles to zone nodes (closes #58)
- GroupRectNode now renders source+target handles on all four sides
  (top, right, bottom, left) using IDs zone-{side} / zone-{side}-t
- Handles are hover-only: opacity 0 by default, fade in on mouse enter
- Handle color matches the zone border color (respects custom_colors)
- Zone↔zone and zone↔node connections both allowed; edge type picker
  (EdgeModal) opens on connect so user chooses ethernet/wifi/vlan/etc.
- Add GroupRectNode.test.tsx: verifies 8 handles rendered (4 source + 4 target)
- Fix @xyflow/react mocks in LiveView and CanvasContainer tests to include Position
2026-04-19 22:13:40 +02:00
Pouzor ef96cafcc8 feat: IPv6 support and multi-IP per node (closes #60)
- maskIp handles IPv6 addresses (masks second and last group)
- maskIp handles comma-separated IP strings (masks each address)
- Add splitIps() helper to parse comma-separated IP field
- Add primaryIp() helper used by status checker (first IP wins)
- BaseNode renders each IP on its own line when comma-separated
- NodeModal placeholder shows comma-separated example
- Backend status_checker uses only first IP for connectivity checks
- Expand maskIp test suite: IPv6, comma-separated, splitIps, primaryIp
2026-04-19 22:13:40 +02:00
Pouzor 6c9974b357 bump version 1.10 2026-04-19 22:13:40 +02:00
Pouzor ce5fc785e1 chore: bump version to 1.10.0 2026-04-19 22:13:40 +02:00
Pouzor 0019c086cf feat: automatic DB backup before migrations using VERSION file
- Add VERSION file at repo root as single source of truth for app version
- frontend/vite.config.ts reads VERSION file instead of package.json
- backend config.py exposes APP_VERSION read from VERSION (dev) or /app/VERSION (Docker)
- database.py backs up DB to homelab.db.back-{version} before running migrations
  (skipped if DB doesn't exist or backup already exists — fully idempotent)
- Dockerfile.backend and Dockerfile.frontend copy VERSION into the image
- Add test_db_backup.py with 4 tests covering create/skip/idempotent/version cases
2026-04-19 22:09:43 +02:00
RemyandGitHub 0eff7da46e Merge pull request #81 from Pouzor/fix/clickable-ip-multiip
fix: handle multi-IP for clickable IP link
2026-04-19 22:08:19 +02:00
Pouzor 2e6ee9dad2 fix: handle multi-IP and add tests for clickable IP link
Follow-up to #78:
- Use primaryIp() so href targets the first IP when data.ip is comma-separated (e.g. "192.168.1.1, 2001:db8::1")
- Add primaryIp() helper to maskIp.ts
- Add 4 tests covering single IP link, absent IP, multi-IP href, multi-IP display text
2026-04-19 22:04:37 +02:00
findthelorax 81b109f981 feat: make IP Address clickable in detail panel
- Display IP as a clickable link that opens http://<ip> in a new tab
- Match the existing Hostname link styling and behavior
- Add external link icon to indicate it's clickable
2026-04-19 10:08:09 -04:00
Brett FerranteandGitHub 73b16a7620 Merge pull request #15 from Pouzor/main
Merge updates from Pouzor main
2026-04-19 09:21:21 -04:00
RemyandGitHub a37bf101d2 Merge pull request #76 from findthelorax/bug/drag-from-title
Fix/drag from title
2026-04-19 11:41:41 +02:00
RemyandGitHub 5def6b7fbf Merge pull request #75 from dopp1e/fix-curl-healthcheck
fix: add curl to backend image to support the default healthcheck
2026-04-19 11:35:07 +02:00
Brett FerranteandGitHub 892710faac Merge branch 'Pouzor:main' into bug/bezier-label 2026-04-18 22:27:02 -04:00
Brett FerranteandGitHub eb235cb101 Merge branch 'Pouzor:main' into bug/drag-from-title 2026-04-18 22:14:00 -04:00
Brett FerranteandGitHub 04a1c63558 Merge pull request #14 from Pouzor/main
Merge with Pouzor main branch
2026-04-18 21:36:53 -04:00
doppie 88f0c03c57 fix: add curl to backend image to support the default healthcheck 2026-04-19 01:08:51 +02:00
RemyandGitHub 718aff5918 Merge pull request #73 from Pouzor/feature/extended-zoom
feat: extended zoom range + zoom-aware node borders
2026-04-18 23:10:24 +02:00
Pouzor 70311e6331 fix: propertyIcons mock, key stability, isValidConnection memoized
- Add resolvePropertyIcon mock to BaseNode tests
- Reset mockZoom in top-level beforeEach to prevent test bleed
- Use prop.key instead of index as list key in BaseNode properties
- Extract isValidConnection into useCallback in CanvasContainer
2026-04-18 23:03:21 +02:00
Pouzor 6a3da5aded fix: memoize onBeforeDelete and fix test mock selectors
Extract onBeforeDelete into useCallback to avoid new fn ref each render.
Fix useThemeStore/useCanvasStore mocks to call the selector fn rather
than ignoring it, and reset mockZoom in beforeEach.
2026-04-18 23:03:21 +02:00
Pouzor 35c3d00f17 fix: memoize borderWidth and sync boxShadow glow ring with zoom
- Wrap borderWidth in useMemo to avoid re-computing on unrelated renders
- Replace hardcoded 1px in boxShadow with borderWidth so glow ring
  matches border thickness at all zoom levels
- Add useViewport mock + 4 zoom-scaling tests to BaseNode.test.tsx
2026-04-18 23:01:27 +02:00
findtheloraxandPouzor 3a5cb0de21 feature: extend zoom out and border width math 2026-04-18 23:01:27 +02:00
RemyandGitHub f72d44d5e5 Merge pull request #71 from Pouzor/feature/double-click-to-edit
feat: double-click node to open edit modal
2026-04-18 21:55:01 +02:00
Pouzor a7b244502e fix: bump pytest-asyncio to 1.3.0 (first version supporting pytest 9) 2026-04-18 21:52:32 +02:00
Pouzor 72d5a51b44 fix: bump pytest-asyncio to 1.0.0 for pytest 9 compatibility
pytest-asyncio 0.26.0 requires pytest<9; 1.0.0 supports pytest 9.x
2026-04-18 21:45:10 +02:00
Pouzor 12f46715c1 fix: bump python-multipart, pytest, pytest-asyncio to fix CVEs
- python-multipart 0.0.22 → 0.0.26 (CVE-2026-40347)
- pytest 8.3.3 → 9.0.3 (CVE-2025-71176)
- pytest-asyncio 0.24.0 → 0.26.0 (pytest 9 compat)
2026-04-18 21:36:09 +02:00
Pouzor 62f674b15d fix: extract handleNodeDoubleClick into useCallback and add tests
Replace inline arrow with named useCallback handler to avoid creating
a new fn ref on every render. Add two CanvasContainer tests covering
the double-click callback and the no-op path when prop is omitted.
2026-04-18 21:09:49 +02:00
findthelorax b0a67744f5 bug: fixed to allow draging from the titlebar 2026-04-17 22:34:17 -04:00
findthelorax 431fb47498 bug: fixed an issue with the label and a bezier path with multiple points 2026-04-17 20:28:51 -04:00
findthelorax 04069e080a feature: double-click on a node to open an edit modal 2026-04-16 10:10:30 -04:00
Pouzor dd1f690892 feat: add logo assets in docs/logo
Icon (house + network nodes, dark bg) and horizontal logo (icon +
"Home"/"lable" color-split text) in multiple sizes:
- icon.svg + icon-16/32/64/128/256/512.svg
- logo.svg + logo-sm/md/lg.svg
2026-04-10 12:04:02 +02:00
RemyandGitHub 8b04deb608 Merge pull request #56 from Pouzor/1.9
v1.9.0 — Node properties, edge waypoints, version display
2026-04-09 19:17:14 +02:00
Pouzor 531fb12eab fix: replace SVG animate with CSS animation for basic edge to prevent bounce
SVG <animate> restarts on every React re-render (especially visible under
StrictMode double-invoke in dev), causing a visible bounce. CSS animation
runs on the compositor thread independently of React renders.
2026-04-09 16:52:43 +02:00
Pouzor e666abefad chore: remove db backup from tracking and add *.db.back to gitignore 2026-04-09 16:34:25 +02:00
Pouzor f1e9fd7cf8 test: verify standalone mode round-trips for properties, waypoints and animations
- Fix ApiEdge type: add 'basic' to animated union
- Add 5 standalone storage tests: properties, empty properties, waypoints,
  basic animation, and all three animation types round-trip through localStorage
2026-04-09 16:28:20 +02:00
Pouzor 9134812e32 feat: add Basic edge animation with consistent downward direction
Adds a "basic" animation type that uses React Flow's native moving-dash
style — the entire stroke is animated dashes with no solid line underneath.
Direction is always physically downward (top node → bottom node) regardless
of which end was the source when the edge was drawn.
2026-04-09 16:04:43 +02:00
Pouzor 4844576c3b feat: add Basic animation type for edges
Adds a new "basic" animation mode that uses React Flow's native animated
dashes — the entire stroke moves as dashes, with no solid line underneath.
Distinct from "snake" (moving dot) and "flow" (overlay on solid line).

- Frontend: new Basic button in EdgeModal, animMode mapping, passes
  animated={true} to BaseEdge when mode is "basic"
- Backend: normalize_animated accepts "basic" as a valid value
- Tests: EdgeModal + canvas round-trip tests for basic mode
2026-04-09 15:46:52 +02:00
Pouzor 4976f2e694 test: improve canvas route coverage to 100% and fix async tracing
- Add 22 new tests for canvas save/load: properties round-trip,
  edge waypoints, edge handles, animated, parent/child nodes,
  group nodes, all node fields, bottom_handles, services/notes,
  check_method/target, viewport update branches, edge update branch
- Fix coverage.py async tracing by setting core = "sysmon" in
  pyproject.toml (Python 3.12+ sys.monitoring API), resolving the
  sys.settrace drop-after-await issue
2026-04-09 15:17:58 +02:00
Pouzor d6a7b062f4 fix: clear node height when properties change to prevent overflow outside border 2026-04-09 14:34:01 +02:00
Pouzor 94c6ac7fa7 fix: add type parameters to bare list annotations for mypy compliance 2026-04-09 14:23:13 +02:00
Pouzor e0f96001e2 chore: bump version to 1.9.0 2026-04-09 14:06:24 +02:00
Pouzor 3d89ba3b6f feat: replace static hardware fields with dynamic node properties
Replaces the 4 fixed hardware columns (cpu_count, cpu_model, ram_gb,
disk_gb) with a flexible properties system. Each property has a key,
value, icon (from a curated Lucide picker), and a visibility toggle
that controls whether it appears on the canvas node card.

- Backend: add `properties` JSON column to Node model; data migration
  converts existing hardware rows to properties with correct icons
  (idempotent, old columns kept for safety)
- Backend: add `properties` to NodeBase, NodeUpdate, NodeSave schemas
  and canvasSerializer so values survive canvas save/load
- Frontend: add NodeProperty type; new propertyIcons.ts registry (20
  icons); BaseNode renders visible properties with legacy hardware
  fallback for unmigrated nodes
- Frontend: DetailPanel gains interactive properties section (add /
  edit / remove / toggle visibility / icon picker) replacing the
  read-only hardware block; hardware section removed from NodeModal
- Tests: 6 migration tests, 7 API tests, 8 DetailPanel property tests,
  6 BaseNode render/fallback tests, 9 propertyIcons util tests
2026-04-09 13:47:36 +02:00
Pouzor 3afc8ed3d8 fix: persist edge waypoints in backend
Add waypoints JSON column to edges table, include it in all edge
schemas (EdgeBase, EdgeUpdate, canvas CanvasEdge) and add the
idempotent ALTER TABLE migration so existing databases are upgraded
on next startup.
2026-04-08 23:31:53 +02:00
Pouzor 9e8bab5dec feat: add interactive edge waypoints with smooth path editing
- Drag waypoints to reshape edges; double-click a waypoint to remove it
- + handles at segment midpoints to insert new waypoints
- Bezier style: catmull-rom smooth curves through waypoints
- Smooth style: rounded-corner polyline with soft 45° snap (snaps within 15px)
  - First + handle biased to source axis for perpendicular node exit
  - snap45both: ray-intersection solver ensures both adjacent segments snap to 45° simultaneously
- Clear path button in EdgeModal when waypoints exist
- Waypoints serialised/deserialised with canvas state
2026-04-08 22:42:18 +02:00
Pouzor 9d9fdd61e9 Merge branch 'feat/version-display' into 1.9 2026-04-08 16:19:52 +02:00
Pouzor b0df8f389a fix: align dot grid to snap grid and fix node selection layout shift 2026-04-08 13:58:27 +02:00
Pouzor 75c7f25a30 feat: display app version in sidebar with GitHub release check 2026-04-08 12:22:39 +02:00
Pouzor 8bd1c48976 chore: bump version to 1.8.3 2026-04-07 01:04:10 +02:00
RemyandGitHub 05c98355a6 Update INSTALLATION.md 2026-04-07 00:51:25 +02:00
Pouzor 323dea6798 Remove custom proxmox script and doc 2026-04-07 00:49:52 +02:00
Pouzor 19cb4b71f5 chore: upgrade lucide-react to v1.7.0 2026-04-07 00:06:21 +02:00
Pouzor fd86c0f6ad chore: update frontend npm dependencies (patch/minor) 2026-04-07 00:02:33 +02:00
Pouzor 00d44abfad feat: reduce snap grid from 16px to 8px for finer node positioning 2026-04-06 23:54:56 +02:00
Pouzor 3bd18ab543 chore: bump version to 1.8.2 2026-04-05 15:01:57 +02:00
Pouzor 4d8bb246f1 fix: scan no longer triggers before user confirms in ScanConfigModal 2026-04-05 14:35:10 +02:00
Pouzor 07da498d18 fix: use correct ping args on Windows (-n/-w instead of -c/-W) 2026-04-05 14:08:36 +02:00
Pouzor 0e59f15608 chore: bump version to 1.8.1 2026-04-05 10:32:37 +02:00
Pouzor bd22891fab fix: correct UTC timestamp display and pre-existing code quality issues 2026-04-05 03:06:29 +02:00
Pouzor d96b502524 fix: correct UTC timestamp display in scan history, pending devices, and detail panel 2026-04-05 02:50:23 +02:00
Pouzor 9cb9d02459 feat: extend search (Ctrl+F and Ctrl+K) to include pending devices
Both SearchBar and SearchModal now fetch and search pending devices by
IP, hostname, and service name. Selecting a pending result opens the
sidebar to the Pending tab and highlights the matching device.
2026-04-05 02:40:12 +02:00
Pouzor 08a3a689c5 chore: bump version to 1.8.0 2026-04-04 23:29:20 +02:00
RemyandGitHub 034ba599f4 Merge pull request #40 from Pouzor/feat/bottom-handles
feat: configurable bottom handles, scanner rewrite, UI polish
2026-04-04 23:23:20 +02:00
Pouzor 38c5bcb606 feat: configurable bottom handles, scanner rewrite, UI polish
## New features
- Configurable bottom connection points per node (1–4 handles)
- Fit view on load
- LiveView improvements
- Node modal: inline Type/Icon picker, default icon in trigger
- Remove redundant Save button from ScanConfigModal

## Scanner fixes
- Phase 1: replace nmap ARP sweep with concurrent asyncio ping sweep
  (50 parallel pings, 1s timeout). Zero false positives, works in any
  Docker network mode. Supplements with /proc/net/arp for ICMP-blocked devices.
- Phase 2: explicit -sS (root) / -sT (non-root) scan type; bump
  host-timeout to 60s; gather(return_exceptions=True) so one failing
  host doesn't abort the batch
- Fix 404 on missing device in hide/ignore
- Validate CIDR ranges to prevent nmap injection
- Thread-safe cancel set, pre-fetch canvas/hidden IPs (no N+1 queries)
- Logging: attach StreamHandler to root logger so app.* logs are visible

## Tests
- 21 backend scanner tests (ping sweep, ARP cache, Phase 2 tolerance)
- Full NodeModal coverage (53 tests)
- LiveView, store, edge label tests
2026-04-04 23:15:47 +02:00
Pouzor 0a72f9be67 test: add tests for useStatusPolling hook and Sidebar component
- 12 tests for useStatusPolling: WebSocket lifecycle, auth guards,
  message routing (status/scan_device_found), error resilience, cleanup
- 21 tests for Sidebar: render, collapse/expand, node stats, action
  callbacks, view navigation, scan trigger, settings panel toggle
2026-04-03 22:55:56 +02:00
Pouzor 041583c9b4 fix: set correct edge handles on YAML import
Edges created by parseYamlToCanvas had no sourceHandle/targetHandle,
causing React Flow to connect everything at the top of nodes.

- Regular links: bottom → top-t
- Parent edges: bottom → top-t
- clusterR/clusterL: cluster-right → cluster-left
2026-04-03 00:50:34 +02:00
Pouzor 5a3e8ea0b1 chore: bump version to 1.7.1 2026-04-03 00:12:22 +02:00
Pouzor 70907e37bc fix: LXC/VM parent-child UX bugs with Proxmox container mode
- Don't create virtual edge when parent Proxmox is in container mode
  (containment is shown visually — edge was redundant and confusing)
- updateNode now syncs React Flow parentId/extent/position when
  data.parent_id changes, so nesting/un-nesting is immediate without
  requiring a save+reload
- Attaching to container: converts position to parent-relative coords
- Detaching from container: converts position back to absolute coords
  so the node escapes the container box immediately
- Ensure parent nodes precede children in array on attachment
  (React Flow rendering requirement)
2026-04-02 22:30:54 +02:00
RemyandGitHub e356d433cb Merge pull request #37 from Pouzor/feat/iot-discovery
feat: improve IoT device detection via two-phase scan and mDNS discovery
2026-04-02 17:56:34 +02:00
Pouzor cf7777e0af feat: improve IoT device detection via two-phase scan and mDNS discovery
- Two-phase nmap: ARP sweep first to find all alive hosts (incl. IoT with
  no open TCP ports), then port scan only alive hosts
- mDNS/Bonjour discovery via zeroconf for Shelly, ESPHome, HomeKit devices
- Add CoAP ports (5683, 5684, 4915) to port scan and IoT type hints
- Expand MAC OUI table with Shelly, Espressif, Sonoff, Tapo, Hue, IKEA, Tuya
- IoT vendor MAC takes precedence over generic HTTP port type hints
- Reorder suggest_node_type priority: iot now beats server
2026-04-02 17:41:51 +02:00
Pouzor 255443b8e1 fix: harden scheduler startup and interval validation
- Wrap shutdown() in try/except to prevent double-scheduler if teardown fails
- Guard reschedule_status_checks with interval_seconds >= 10 validation
2026-04-02 17:39:52 +02:00
Pouzor 7e24878077 fix: pass scalars to _check_single_node to prevent DetachedInstanceError
- Refactor _check_single_node to accept plain scalar args (node_id,
  check_method, check_target, ip) instead of a detached ORM Node object
- Extract scalars inside the session in _run_status_checks before it closes
- Capture datetime.now() once per check to keep DB and WebSocket timestamps consistent
2026-04-02 17:32:42 +02:00
Pouzor ff1bc7340d fix: resolve mypy errors in scheduler (dict type params + str | None arg) 2026-04-02 17:24:26 +02:00
Pouzor a7c9abbb9a npm audit fix 2026-04-02 11:48:37 +02:00
Pouzor e4bfab7e58 perf: run status checks concurrently to prevent scheduler overrun
Replace sequential node checks with asyncio.gather so all nodes are
checked in parallel. Add coalesce=True + max_instances=1 to suppress
APScheduler "maximum instances reached" log spam.
2026-04-02 11:41:08 +02:00
Pouzor d5069d9b96 chore: bump version to 1.7.0 2026-04-02 01:22:22 +02:00
Pouzor 7498a10c14 fix: drop -qq on apt-get update and add --fix-missing to handle stale mirror cache in LXC install 2026-04-02 00:24:18 +02:00
Pouzor a8dd41a156 fix: add group NodeType to all theme definitions 2026-04-02 00:20:32 +02:00
Pouzor bdf3b6ea40 feat: add Ctrl+F search bar to canvas
- Ctrl+F / Cmd+F opens floating search bar at top-center of canvas
- Filters nodes by label, IP, hostname and service name (case-insensitive)
- Shows match count and no-results message
- Click result selects node and flies camera to it with animation
- Escape or × closes the bar
- groupRect nodes excluded from results
- Handles grouped nodes with correct absolute position for navigation
2026-04-01 23:32:28 +02:00
Pouzor 0b89244317 feat: lasso selection, multi-select panel, and named groups
- Add lasso/box selection via selectionOnDrag (Space to pan, lasso by default)
- Add lasso/pan toggle button in canvas controls (bottom-left)
- Multi-select panel in right panel when 2+ nodes selected (including zones)
- Create named Group node from selected nodes with bounding box math
- Group node: resizable, inline rename, show/hide border toggle, status summary
- GroupDetailPanel: lists members, online/offline count, ungroup action
- Fix group persistence after save/reload (extend proxmox container map to include group nodes)
- Fix groupRect serialization to preserve parent_id
- Remove background color from group and proxmox container node wrappers
- Add tests for all new store actions, GroupNode, MultiSelectPanel, GroupDetailPanel
2026-04-01 23:15:52 +02:00
Pouzor 45a17b0254 fix: use node:20-slim in build stage to fix lightningcss musl binary error 2026-04-01 14:22:45 +02:00
Pouzor 057891f7d5 feat: add stop scan button in UI with backend cancellation support
- POST /scan/{run_id}/stop endpoint signals running scan to cancel
- Scanner checks cancellation flag between CIDR ranges and hosts, exits early
- Cancelled scans get status 'cancelled' instead of 'done'
- Stop button (red StopCircle) shown in Scan History panel for running scans
- 6 new backend tests, 5 new frontend tests
2026-04-01 14:18:44 +02:00
RemyandGitHub 5321070720 Update INSTALLATION.md 2026-03-31 14:17:14 +02:00
Pouzor 59e5a95912 docs: clarify bcrypt dollar-sign escaping for .env vs docker-compose.yml
Fixes #31
2026-03-31 14:13:22 +02:00
447 changed files with 74724 additions and 5201 deletions
+73 -2
View File
@@ -1,19 +1,49 @@
# Backend - server-side only (NEVER commit .env)
# Generate: python3 -c "import secrets; print(secrets.token_hex(32))"
SECRET_KEY=change_me_in_production
SQLITE_PATH=./data/homelab.db
# Uploaded media (floor plans) folder. Optional — defaults to <SQLITE_PATH dir>/uploads,
# which sits on the same persistent volume as the DB.
# UPLOAD_DIR=./data/uploads
# Set this to the URL(s) you use to access Homelable in your browser.
CORS_ORIGINS=["http://localhost:5173","http://localhost:3000"]
# Auth — default credentials: admin / admin
# Auth — local mode is the backward-compatible default.
AUTH_MODE=local
# Local credentials: admin / admin by default.
# ⚠️ Change before exposing on a network.
# Generate a new hash: python3 -c "from passlib.context import CryptContext; print(CryptContext(schemes=['bcrypt']).hash('yourpassword'))"
# Generate a new hash: python3 -c "import bcrypt; print(bcrypt.hashpw(b'yourpassword', bcrypt.gensalt()).decode())"
# ⚠️ Keep the single quotes around the hash — bcrypt hashes contain \$ which Docker misinterprets without them.
AUTH_USERNAME=admin
AUTH_PASSWORD_HASH='$2b$12$RtMbyw17l4N5UGzeXMNAWuzCaVV.XFBY7ZetWheQhxcBDcxahapkG'
# OpenID Connect (optional alternative to local auth).
# Set AUTH_MODE=oidc and configure every value below. The callback must be
# registered exactly at the identity provider. Keep CORS_ORIGINS restricted to
# the Homelable browser origin; wildcard CORS is rejected in OIDC mode. OIDC
# also requires SECRET_KEY to contain at least 32 bytes.
# AUTH_MODE=oidc
# OIDC_DISCOVERY_URL=https://idp.example/application/o/homelable/.well-known/openid-configuration
# OIDC_CLIENT_ID=homelable
# OIDC_CLIENT_SECRET=replace-with-a-secret
# OIDC_REDIRECT_URI=https://homelable.example/api/v1/auth/oidc/callback
# OIDC_SCOPES="openid profile email"
# OIDC_COOKIE_SECURE=true
# OIDC_SESSION_EXPIRE_MINUTES=480
# OIDC_TRANSACTION_EXPIRE_SECONDS=600
# Scanner — JSON array of CIDR ranges to scan
SCANNER_RANGES=["192.168.1.0/24"]
# Deep scan (optional) — extra nmap port ranges + HTTP probe for service ID on
# custom ports. Defaults below are overridable per-scan from the scan dialog.
# SCANNER_HTTP_RANGES: JSON array of port specs, each a single port "N" or an
# inclusive range "N-M" (1–65535, N <= M). Not CIDRs, not bare ints.
# Example: SCANNER_HTTP_RANGES=["8080","9000-9100"]
SCANNER_HTTP_RANGES=[]
SCANNER_HTTP_PROBE_ENABLED=false
SCANNER_HTTP_VERIFY_TLS=false
# Status checker interval in seconds
STATUS_CHECKER_INTERVAL=60
@@ -28,3 +58,44 @@ MCP_SERVICE_KEY=svc_changeme
# Off by default. Set to a random secret to enable.
# Generate: python3 -c "import secrets; print(secrets.token_urlsafe(32))"
# LIVEVIEW_KEY=
# Gethomepage widget — read-only stats at /api/v1/stats/summary
# Off by default. Set to a random secret to enable; clients must send
# the same value in the `X-API-Key` header.
# Generate: python3 -c "import secrets; print(secrets.token_urlsafe(32))"
# HOMEPAGE_API_KEY=
# Proxmox VE import — pull hosts/VMs/LXC from the Proxmox REST API.
# The token is a credential: kept in memory only, never written to disk by the
# app, never returned by any API. Create it under Datacenter → Permissions →
# API Tokens and grant the read-only PVEAuditor role at path "/".
# Token id format: user@realm!tokenname (e.g. root@pam!homelable).
# Required only for auto-sync; one-off imports can pass the token in the dialog.
# PROXMOX_TOKEN_ID=root@pam!homelable
# PROXMOX_TOKEN_SECRET=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
# PROXMOX_HOST=192.168.1.10
# PROXMOX_PORT=8006
# PROXMOX_VERIFY_TLS=true # set false only for self-signed certs
# Zigbee2MQTT auto-sync — pull the mesh from an MQTT broker on a schedule.
# MQTT credentials are secrets: kept in memory only, never written to disk by
# the app, never returned by any API. Manual imports (the Zigbee dialog) are
# unaffected — this block only powers Settings → Zigbee auto-sync and /sync-now.
# Required only for auto-sync; one-off imports pass their config in the dialog.
# ZIGBEE_MQTT_HOST=192.168.1.20
# ZIGBEE_MQTT_PORT=1883
# ZIGBEE_MQTT_USERNAME=mqttuser # optional
# ZIGBEE_MQTT_PASSWORD=mqttpass # optional
# ZIGBEE_BASE_TOPIC=zigbee2mqtt
# ZIGBEE_MQTT_TLS=false # true for TLS brokers (typically port 8883)
# ZIGBEE_MQTT_TLS_INSECURE=false # skip cert verify (self-signed only; requires TLS)
# Z-Wave JS UI (zwavejs2mqtt) auto-sync — same MQTT secret/env rules as Zigbee.
# ZWAVE_MQTT_HOST=192.168.1.20
# ZWAVE_MQTT_PORT=1883
# ZWAVE_MQTT_USERNAME=mqttuser # optional
# ZWAVE_MQTT_PASSWORD=mqttpass # optional
# ZWAVE_PREFIX=zwave
# ZWAVE_GATEWAY_NAME=zwavejs2mqtt
# ZWAVE_MQTT_TLS=false # true for TLS brokers (typically port 8883)
# ZWAVE_MQTT_TLS_INSECURE=false # skip cert verify (self-signed only; requires TLS)
+15
View File
@@ -0,0 +1,15 @@
# These are supported funding model platforms
github: # Replace with up to 4 GitHub Sponsors-enabled usernames e.g., [user1, user2]
patreon: # Replace with a single Patreon username
open_collective: # Replace with a single Open Collective username
ko_fi: pouzor
tidelift: # Replace with a single Tidelift platform-name/package-name e.g., npm/babel
community_bridge: # Replace with a single Community Bridge project-name e.g., cloud-foundry
liberapay: # Replace with a single Liberapay username
issuehunt: # Replace with a single IssueHunt username
lfx_crowdfunding: # Replace with a single LFX Crowdfunding project-name e.g., cloud-foundry
polar: # Replace with a single Polar username
buy_me_a_coffee: # Replace with a single Buy Me a Coffee username
thanks_dev: # Replace with a single thanks.dev username
custom: # Replace with up to 4 custom sponsorship URLs e.g., ['link1', 'link2']
+3
View File
@@ -6,6 +6,9 @@ on:
pull_request:
branches: [main]
permissions:
contents: read
jobs:
smoke-and-integration:
runs-on: ubuntu-latest
+9 -2
View File
@@ -16,14 +16,21 @@ jobs:
matrix:
include:
- image: ghcr.io/pouzor/homelable-backend
context: .
dockerfile: Dockerfile.backend
build_args: ""
- image: ghcr.io/pouzor/homelable-frontend
context: .
dockerfile: Dockerfile.frontend
build_args: ""
- image: ghcr.io/pouzor/homelable-frontend-standalone
context: .
dockerfile: Dockerfile.frontend
build_args: "VITE_STANDALONE=true"
- image: ghcr.io/pouzor/homelable-mcp
context: ./mcp
dockerfile: Dockerfile.mcp
build_args: ""
steps:
- uses: actions/checkout@v4
@@ -55,8 +62,8 @@ jobs:
- name: Build and push
uses: docker/build-push-action@v6
with:
context: .
file: ${{ matrix.dockerfile }}
context: ${{ matrix.context }}
file: ${{ matrix.context }}/${{ matrix.dockerfile }}
platforms: linux/amd64,linux/arm64
push: true
tags: ${{ steps.meta.outputs.tags }}
+20
View File
@@ -6,6 +6,9 @@ on:
pull_request:
branches: [main]
permissions:
contents: read
jobs:
lint-scripts:
runs-on: ubuntu-latest
@@ -57,3 +60,20 @@ jobs:
- run: ruff check .
- run: mypy app/
- run: pytest
# Also guards the NODE_TYPES / EDGE_TYPES enums in mcp/app/tools.py against
# drift from frontend/src/types/index.ts (test_node_types_sync.py,
# test_edge_types_sync.py) — those tests need the frontend sources, so this
# job runs from a full checkout rather than the mcp Docker context.
mcp:
runs-on: ubuntu-latest
defaults:
run:
working-directory: mcp
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: '3.13'
- run: pip install -r requirements.txt
- run: pytest
+14 -4
View File
@@ -8,6 +8,9 @@ on:
schedule:
- cron: '0 9 * * 1' # Weekly on Monday
permissions:
contents: read
jobs:
secrets-scan:
runs-on: ubuntu-latest
@@ -19,6 +22,12 @@ jobs:
uses: trufflesecurity/trufflehog@main
with:
path: ./
# Lob (a direct-mail API this project does not use) matches any
# `test_`-prefixed identifier of ~35 chars and reports it verified,
# because Lob test keys authenticate unconditionally. Every pytest
# function named at that length trips it — the first was
# `test_updates_an_existing_device_in_place`.
extra_args: --exclude-detectors=Lob
dependency-audit:
runs-on: ubuntu-latest
@@ -28,11 +37,12 @@ jobs:
with:
node-version: '20'
- name: NPM Audit
run: cd frontend && npm audit --audit-level=high
# --omit=dev: only ship-time deps gate the build. Dev-only tooling
# (shadcn CLI and its transitive tree: hono, fast-uri, …) is never
# bundled or served, so its advisories must not fail release CI.
run: cd frontend && npm audit --omit=dev --audit-level=high
- uses: actions/setup-python@v5
with:
python-version: '3.11'
- name: Pip audit
# CVE-2024-23342 (ecdsa): timing attack on ECDSA — not exploitable (app uses HS256/HMAC only).
# python-jose forces ecdsa as a dependency with no fixed version available.
run: pip install pip-audit && pip-audit -r backend/requirements.txt --ignore-vuln CVE-2024-23342
run: pip install pip-audit && pip-audit -r backend/requirements.txt
+8 -1
View File
@@ -1,6 +1,5 @@
# Claude / project meta — never commit
CLAUDE.md
FEATURES.md
.claude/
_project_specs/
@@ -45,7 +44,15 @@ htmlcov/
*.db
*.db-shm
*.db-wal
*.db.back
*.db.back-*
# Docker
.docker/
Ideas.md
# Local dev/test utilities (never commit)
scripts/zwave-mock-gateway.py
# Docs (local only)
docs/database-model.md
+765
View File
@@ -0,0 +1,765 @@
# Changelog
All notable changes to **Homelable** are documented here.
The format is loosely based on [Keep a Changelog](https://keepachangelog.com/),
and this project adheres to [Semantic Versioning](https://semver.org/).
## [3.3.5] - 2026-08-21
### Features
- Deep-scan a single device from its inventory detail: the Services section gets a **Deep scan** button that sweeps the device over a port range you pick, prefilled with the full 1-65535 range and narrowable when you know where the service lives. The run is a normal Scan Run, so stop, progress and Scan History work as usual, and the fresh services are merged in — never replacing what you added by hand. (#363)
### Fixes
- A deep rescan of a slow host no longer comes back empty. The run hit its timeout and nmap discarded every port it had already found; the full range is now scanned in slices, a partial sweep is reported as partial instead of passed off as complete, and the timeout is a total budget between slices rather than an nmap flag. (#363)
- A scan no longer repaints the service icons you picked by hand: the fingerprint's guess overwrote your choice, and cleared it outright on a port no signature covers. Since 3.3.0 the inventory row is the only copy, so one scan changed the service on every canvas at once. (#363)
- A rescan finishing while you edit a device no longer throws the edit away, and it no longer tags every device it touches as network-discovered — a Proxmox guest, a rack mount or a hand-added host keeps its own source. A failed run is also recorded as `error`, the status Scan History filters and colours. (#363)
- A property with no value gets the full node width for its label instead of being truncated against empty space. (#362)
- A same-origin OIDC deployment behind a reverse proxy no longer answers 403 to every write. The CSRF check validated the browser's `Origin` against `CORS_ORIGINS` alone; the app's own origin, taken from `OIDC_REDIRECT_URI`, is accepted too, and startup warns when `CORS_ORIGINS` omits it. (#360)
## [3.3.4] - 2026-08-17
### Fixes
- A network scan no longer deletes the services you added by hand. The scanner replaced a device's whole service list with what it fingerprinted, so one scan wiped manual entries on every canvas at once and brought back the ones you had deleted. It merges now. (#347)
- A node whose services were all replaced under it no longer renders empty: when nothing a node's view names is left on the device, the device's own list is drawn instead of hiding everything. (#347)
- Properties added while running 3.3.0–3.3.2 survive the 3.3.3 upgrade. The view recovered from the pre-3.3.0 backup could not know about them, so they were hidden on every canvas; they are kept visible, and the per-canvas arrangement the backup does know is still restored. (#347)
## [3.3.3] - 2026-08-17
### Features
- Each node now keeps its own view of a device's services and properties: order and visibility are per node, so a service the scanner fingerprinted no longer appears on every canvas drawing that device, and a property added on one schematic stays there. The facts still live on the inventory row — hiding is per node, deleting is device-wide. Upgrades keep what each canvas showed today. (#357)
### Fixes
- The one-off view seed no longer re-opens the pre-upgrade backup on every boot: a node pointing at a deleted device kept matching the seed query and logged a recovery that recovered nothing. (#357)
## [3.3.2] - 2026-08-17
### Fixes
- Devices upgraded from 3.2.0 keep their ip, services, hostname, notes and hardware. The 3.3.0 migration aborted on timestamps SQLite returns as text, so nothing was carried across to the Device Inventory and every device read back blank. (#347, #348)
- A canvas saved while an earlier migration was stuck no longer loses its device facts: the migration now fills the blank inventory row that save created, instead of skipping the node and dropping the columns holding its only copy. (#347)
## [3.3.1] - 2026-08-17
### Fixes
- Approving a device no longer fails after an upgrade from 3.2.0: a node the 3.3.0 inventory backfill could not link kept the old device columns on `nodes`, and their NOT NULL constraint rejected every new node. (#352)
## [3.3.0] - 2026-08-15
### Features
- The Device Inventory is now the source of truth for device data: a node reads and edits its facts from its inventory row, and the device sheet is grouped by what a reader is after. (#339)
- Per-service host override, so a service can point at a host other than the node's own. (#338)
- A zone now parents the nodes dropped inside it. (#343)
- Node properties accept an empty value. (#345)
- Nodes can be given 0 connection points on top and bottom. (#336)
- KVM switch added as a device type. (#335)
- The sidebar version row links to the public changelog.
### Fixes
- Alignment guides show for nodes inside groups and containers. (#337)
- The status check interval is applied to the running scheduler. Thanks @JaG-v2. (#319)
- Bumped undici and ip-address to patch high severity advisories. (#342)
### Docs
- The pre-built GHCR images are documented in the install guide.
## [3.2.0] - 2026-08-10
### Features
- Rack canvas: a third design type, next to network and electrical, with racks, mounted devices and port-to-port patching. (#315)
- Declarative SVG faceplates picked from a visual catalog, including desktop NAS enclosures in 2, 4 and 5 bays. (#315)
- Rack gear is filed in the Device Inventory, and a mount can be picked from the rackable entries. (#315)
- Cables carry a type, colour, label and properties, printed on the canvas at the midpoint of the run. (#315)
- Mounted devices follow the status check of the logical node they match, and links can be imported from the logical canvas. (#315)
- Getting Started walkthrough covers the rack canvas. (#315)
- Show the logical view of a mounted device, and link a mount to any Device Inventory entry. (#323)
### Fixes
- Guard rack rows against cross-design saves and half-applied faceplates. (#315)
- A rack height edit no longer orphans its mounts; height and colour commit on blur, and a blanked field no longer shrinks the rack. (#315)
- Import links is idempotent, and stays usable after a run that matched nothing. (#315)
- Stop the device modal losing cables, status and inventory rows. (#315)
- Name rack inventory entries the way the Device Inventory does. (#315)
- Stamp the linked node's last-seen with an offset. (#323)
- Bump js-yaml to 4.3.1 for CVE-2026-59870. (#315)
- CI: stop the secret scan reporting pytest names as Lob keys. (#315)
### Docs
- Document the rack canvas for users. (#315)
## [3.1.2] - 2026-08-01
### Features
- Generic OIDC authentication: backend sessions and browser login flow. Thanks @autogpt-elendir. (#306)
- Brand icon picker for services, in a dedicated add/edit modal. (#313)
- MCP: lightweight node lookup tools. Thanks @Nebesz. (#312)
### Fixes
- Sync the MCP node and edge type enums with the frontend, gated in CI against drift. Thanks @Nebesz. (#311)
- Harden OIDC callback validation. (#306)
- Use the official Homelable logo on the login page. (#306)
- Clear the seven open dependabot alerts and refresh vulnerable frontend packages. (#317, #306)
### Docs
- Document OIDC setup and configuration. (#306)
## [3.1.1] - 2026-07-19
### Features
- Drag to reorder services in the node detail panel. Thanks @Isilla. (#302)
- Getting Started walkthrough tour with first-run canvas detection. (#300)
### Fixes
- Round-trip Show Port Numbers on YAML export/import. (#301)
- Render error toasts on a red surface. (#300)
### Docs
- Rework README header with logo, badges, and nav links. (#300)
## [3.1.0] - 2026-07-18
### Features
- Drag to reorder node properties. (#296)
- Search for nodes and properties. Thanks @Floyddotnet. (#294)
- Widen zone modal into two columns. (#293)
- Autosave canvas after a configurable inactivity delay (opt-in). Thanks @nicolabottini. (#289)
- MCP: `design_id` on approve_device + `delete_design` tool. Thanks @nicolabottini. (#281)
### Fixes
- Scanner: decouple port discovery from version detection. (#295, #277)
- Allow undo after Auto Layout and YAML import. (#290)
- Move edge waypoints when the connected node or its container is dragged. (#288)
- Keep container host height when editing an unrelated field. (#287)
- Keep container children in place when editing a host node. Thanks @nicolabottini. (#286)
- `list_pending_devices` MCP tool leaked approved/hidden inventory rows. Thanks @MikeSviblov. (#273)
### Docs
- Fix MCP client transport to http (was sse). (#275)
## [3.0.0] - 2026-07-10
### Added
- Proxmox VE import: pull hosts, VMs and LXC into inventory, with optional scheduled auto-sync and a manual re-sync button (connection config is now env-only). (#253, #259)
- Scheduled auto-sync for Zigbee & Z-Wave imports. (#270)
- Floor plan map background, LQI-based edge coloring, and Zigbee path highlighting. Thanks @pranjal-joshi. (#207)
- Customizable connection points per node side. (#249)
- Configurable edge line style + width per type and per edge, plus selectable endpoint marker shapes. (#250, #252)
- Create a new canvas by copying an existing one. (#257)
- Prompt on duplicate device instead of silently blocking/merging on approve. (#261)
- Active design synced to URL for refresh/share. (#263)
- MCP can target a specific design/canvas, create canvases, and auto-position nodes with auto-assigned edge handles. Thanks @nicolabottini. (#266)
### Fixed
- Mesh (Zigbee/Z-Wave) import: duplicate-node crash and coordinator now routed to pending. (#247)
- Keep mesh/cluster links so edges resolve onto a second canvas. (#254)
- Preserve edge connection points in YAML export/import. (#255, #208)
- Harden media path handling against path injection. (#256)
- Match scanned devices to canvas nodes by ip-token and MAC. (#262, #258)
- Record ScanRun for scheduled Proxmox auto-sync. (#269)
### Docs
- User-facing FEATURES.md. (#248)
- CHANGELOG with full release history. (#264)
## [2.6.1] - 2026-06-30
### Added
- Z-Wave nodes can be added/edited manually. A Z-Wave group (Controller / Router / End Device) is now in the Add/Edit node type selector, matching Zigbee. Mesh-radio nodes default to no status check.
### Fixed
- Standalone (frontend-only) multi-canvas (designs) support, originally merged in #244 after the 2.6.0 tag with no version bump.
## [2.6.0] - 2026-06-28
Device Pending is gone — replaced by **Device Inventory**: devices stay in an inventory, can be placed in multiple canvases, rescanned and updated. New **Z-Wave** section: scan, import and store Z-Wave devices like Zigbee.
### Added
- SVG export option in the canvas export modal, alongside PNG (#238).
- Deep-scan HTTP probe + Device Inventory: scans probe HTTP services and surface a per-device inventory (#222, closes #195).
- Z-Wave network scan via the MQTT bridge (#224).
- Inventory timestamps on nodes — nodes show when their inventory was last collected (#233).
- New nodes drop at the centre of the visible canvas instead of off-screen (#234).
- Expanded MAC OUI database: router, switch, AP, NAS and camera vendor prefixes (#220).
### Fixed
- Optional white background on export and working downloads in Firefox (#239, closes #165).
- MCP-created nodes/edges attach to the active design instead of floating (#237, closes #225).
- Stopping a scan interrupts the running nmap range immediately (#236, closes #218).
- Auto-layout orders Proxmox children by parent port number (#235).
- Copy-to-clipboard for Markdown works on non-secure HTTP origins (#219).
### Docs
- Documented the `SCANNER_HTTP_RANGES` port-spec format (#232).
## [2.5.1] - 2026-06-18
### Fixed
- Manually resized nodes — including VM/LXC nodes nested inside a Proxmox container — keep their size on reload instead of snapping back to content-fit.
- Node detail panel has a Size section for pixel-exact width/height; fields resync live when dragging a corner handle.
- Adding an LXC/VM under a non-container-mode Proxmox node no longer confines it to a tiny box — it stays a free, draggable node.
### Security
- Resolved high-severity npm audit advisories (`esbuild`, `form-data`, `vite` → 7.3.5).
- Bumped `python-multipart` to 0.0.31 (CVE-2026-53538/53539/53540).
## [2.5.0] - 2026-06-11
### Added
- Scan History modal with run duration, finished timestamp, and kind/status filters (#203).
- Container nesting: drag a node onto a container node to nest it; detach/re-parent via an editable container selector (#202).
- Editable node groups: add/remove members and edit the group description directly (#200).
- Multi-line edge labels (#199).
- Per-service status checks with offline colouring (#198).
### Fixed
- Reduce status flapping, add IPv6 ping, colour manually-added web services (#198).
- Keep non-HTTP services grey instead of red (#198).
- Idempotent WebSocket connection removal — release the slot on any error (#198).
- Persist group description with Ctrl+S, not only on blur (#200).
### Security
- Resolve Dependabot and code-scanning alerts; bump hono to 4.12.21+ (#197).
- Bump zeroconf to 0.149.12 for CVE-2026-48045 (#202).
## [2.4.0] - 2026-06-05
### Added
- Multiple designs (canvases): build and switch between separate diagrams, plus a new electrical-device node type (#177).
- Copy & paste nodes between designs (#189).
- Switch port numbers (up to 64 ports) plus a new fibre-optic link style (#172).
- Read-only Live View link in the header for sharing or wall displays (#185).
- Scans keep discovered MAC addresses when a device is approved (#178).
- Richer AI/MCP control: assistants can set the full set of node fields on create/update (#180).
### Changed
- Settings moved into a dedicated window for a cleaner sidebar (#188).
- "Hide IP" option now lives in Settings and is remembered between sessions (#189).
- Smoother link animations for flowing/snake styles (#187).
### Fixed
- "Show Port Numbers" now persists after reload (#191).
- The Save button works again (previously only Ctrl/Cmd+S) (#190).
- Zigbee re-import correctly brings back already-approved devices (#176).
### Security
- Dependency updates: zeroconf, qs, brace-expansion (#175, #182).
## [2.3.0] - 2026-05-31
### Added
- Full node schema over MCP: `create_node` / `update_node` expose the complete backend schema (`os`, `notes`, `mac`, `services`, `cpu_count`, `cpu_model`, `ram_gb`, `disk_gb`, `properties`, …); `get_canvas` round-trips the same fields (#174).
- MAC carried onto approved nodes from a scan (#168).
- Richer switch & edge modeling: switch port cap raised to 64, port numbers shown, new fibre edge type.
### Fixed
- Canvas includes the MAC property on the approved node so it shows immediately after approval (#168).
- Zigbee: revive orphaned approved devices on re-import instead of dropping them (#167).
### Security
- Bump `zeroconf` 0.131.0 → 0.149.7 for CVE fixes.
## [2.2.0] - 2026-05-29
### Added
- Laptop & mobile node types with theme-coherent styling (#166).
- Collapsible / expandable zones — `collapsed` is now a first-class `NodeData` field with edge rewiring and read-only liveview support (#158).
- LXC / bare-metal MCP install script `lxc-mcp-install.sh` with env-var overrides and repo-clone fallback; MCP image published to GHCR (#163).
## [2.1.1] - 2026-05-16
### Fixed
- LiveView: nest Docker container children under their host node; apply saved theme & custom style on load.
## [2.1.0] - 2026-05-16
### Added
- Draggable edge endpoints — reconnect either end of an edge to another node/handle; Proxmox containers gain snap points (#150).
- Docker containers can live inside VMs, Proxmox hosts or LXCs; parent auto-cleaned on type change (#153, #154).
- Group node side handles (top/right/bottom/left) for proper edge attachment (#152).
- Multiple IPs per node — paste several IPs separated by comma/space/newline, each clickable individually (#136).
- Zigbee device properties (IEEE, Vendor, Model, LQI) auto-populated on approve / re-import (#148).
- Homepage widget stats endpoint: new `/stats` API for gethomepage integration (closes #131, #149).
- Zone modal polish: centered opacity thumb, fixed font casing, full keyboard + ARIA support (#106).
### Security
- Dropped `passlib` in favour of direct `bcrypt`; status-check targets reject CLI-flag injection.
## [2.0.3] - 2026-05-13
### Changed
- Remove check method for Zigbee nodes.
## [2.0.2] - 2026-05-11
### Added
- Activated dashboardicons icons — brand-new icons selectable for nodes.
### Fixed
- Various modal fixes.
## [2.0.1] - 2026-05-11
### Added
- New Zigbee nodes available in the new/edit node modal.
## [2.0.0] - 2026-05-11
A major milestone: Zigbee2MQTT integration, a new pending-devices modal, text nodes and alignment guides.
### Added
- **Zigbee2MQTT integration** (thanks @pranjal-joshi): import a full Zigbee network map from Z2M as a background scan run; three new node types (Coordinator, Router, End Device); MQTT TLS with optional cert-verify skip; imported devices flow through the pending section with edge persistence.
- **New pending devices modal**: full-screen grid with cards, filters (IP / Zigbee / all), search, multi-select and bulk restore.
- **Text node type** for free-form annotations (#138).
- **Alignment guides + snap** while dragging nodes, built on `OnNodeDrag` (#139).
- **Per-node services toggle** to show/hide the services list on the node (thanks @findthelorax) (#107).
- Zigbee node types exposed in the Custom Style editor.
### Fixed
- Status checker: ms timeout for ping on macOS.
- Scan: default `check_method='ping'` for approved devices.
- Detail panel: handle non-Z timezone offsets in Last Seen.
- Groups: preserve children + size on edit; fix status WebSocket proxy.
- Pending: drop dangling `onNodeApproved` call; null-safe pending IP in SearchModal.
- Text node: persist text in `label` across reloads.
### Security
- Bump `fast-uri` (GHSA-q3j6-qgpj-74h6, GHSA-v39h-62p7-jpjc), `axios` ^1.15.2, `python-multipart`.
- Sanitize MQTT error messages to prevent credential leakage.
## [1.13.0] - 2026-05-03
### Added
- New **Firewall** node type under Add Node → Hardware, with distinct flame icon and red accent.
- Up to **48 bottom connection points** per node (slider, previously capped at 4); the node card grows wider as handles are added.
### Fixed
- Sidebar no longer freezes on Scan History after starting a scan.
- Delete confirmation respects Cancel — clicking Cancel keeps the modal open.
## [1.12.0] - 2026-04-24
### Added
- **Custom Style Editor**: per-node-type border/background/icon colour + opacity, per-edge colour/opacity/path/animation, default size per type, "Apply to existing nodes" and "Apply All to Canvas"; saved with the canvas.
### Changed
- Accessibility overhaul: consistent hover/focus borders, pointer cursors, keyboard navigation and Enter-to-apply across all modals (#108).
- IP address hint moved below the field; hover border colour toned down.
### Fixed
- Edge labels and the `+` waypoint handle stay on the routed path when waypoints are used (#94).
- Edge type select displays its full label (e.g. "IoT / Zigbee") instead of the raw value.
- Status dot overlapping IP address in node cards.
- Canvas style modal layout on smaller screens.
- MCP session manager routing.
## [1.11.0] - 2026-04-23
### Added
- **Docker Host** node type with container mode support (visual group for containers).
- **Docker Container** node type, nestable inside a Docker Host.
- Container mode now works for `docker_host`, `vm` and `lxc` (was Proxmox-only).
### Changed
- Service badges prioritise the service name; path truncates gracefully with a hover tooltip.
- Node resizer handle hit area enlarged (8px → 16px).
### Fixed
- Container mode not persisting after save/reload for non-Proxmox types.
- Container mode toggle having no visual effect on `docker_host` nodes.
- Status dot overlapping node content; consistent top-right placement.
- Missing icon/label gap in ProxmoxGroupNode container header.
- IP shown unmasked in ProxmoxGroupNode when hide-IP is enabled.
## [1.10.2] - 2026-04-21
### Added
- **Logout button** at the bottom of the sidebar (normal mode only).
### Fixed
- Pending Devices checkbox click no longer opens the approval modal.
- NaN sent to the API when the status-check interval input is cleared.
- Potential open-redirect in the update badge (release URL scheme now validated).
## [1.10.1] - 2026-04-20
### Added
- PNG export quality selector: Standard (1×), High (2×, default), Ultra (4×) (#89).
- Zone colour opacity sliders for text/border/background (#72).
- Optional service paths (e.g. `/admin`) appended to the clickable URL; port optional when a path is set (thanks @findthelorax) (#86).
### Fixed
- Bulk-approved nodes appear on canvas immediately (node IDs were null before DB flush).
- Proxmox container mode: visible properties now shown; custom icon now applied.
- `approve_device` returns 404 on missing device, 409 on double-approve (was 200).
- Node form no longer retains previous-session values when reopened in Add mode (#87).
## [1.10.0] - 2026-04-19
### Added
- Bulk approve/hide pending devices (#70).
- IPv6 support & multiple comma-separated IPs per node (#60).
- Clickable IP addresses in the detail panel (#78).
- Connection handles on zone/group rect nodes (#58).
- Automatic DB backup before schema migrations.
- Drag group from its title (#76); double-click a node to open its edit modal (#65).
### Fixed
- Node width no longer expands when long content overflows after a user resize.
- Proxmox nodes with `container_mode=false` restore their saved width on reload.
- Added curl to the backend image for the default healthcheck.
## [1.9.0] - 2026-04-09
### Added
- **Node properties system**: dynamic key/value/icon/visible properties replacing static hardware fields; visible ones shown on the node card; 20 Lucide icons; existing hardware data auto-migrated.
- **Edge waypoints**: click `+` to add, drag to move, double-click to remove; Bezier and Smooth step supported; persisted to backend.
- **Basic edge animation** mode (native moving-dash), alongside Snake and Flow.
- App version shown in sidebar with a GitHub release update check.
### Fixed
- Node height no longer overflows when properties are added to a resized node.
- Dot grid alignment with the snap grid.
- Node selection layout shift.
## [1.8.3] - 2026-04-06
### Added
- Finer canvas grid: snap reduced from 16px to 8px.
### Changed
- Updated frontend npm dependencies; upgraded lucide-react; removed unused Proxmox/LXC install scripts.
## [1.8.2] - 2026-04-05
### Fixed
- Scan no longer starts before confirmation — "Scan Network" opens the config modal first.
- Windows ping compatibility (`-n 1 -w 1000` instead of Linux/macOS flags).
## [1.8.1] - 2026-04-05
### Added
- Search now includes pending devices in both the canvas search bar (Ctrl+F) and command palette (Ctrl+K).
### Fixed
- Timestamps (scan history, discovery time, Last Seen) now show local time instead of UTC.
## [1.8.0] - 2026-04-04
### Added
- Configurable 1–4 bottom connection points per node.
- Fit view on load.
- Inline Type and Icon pickers in the node modal.
### Changed
- **Scanner rewrite**: Phase 1 concurrent asyncio ping sweep (50 parallel pings) supplemented by `/proc/net/arp`; Phase 2 explicit `-sS`/`-sT` scan type with 60s host-timeout; resilient gather so one failing host no longer aborts the batch.
### Fixed
- Root logger StreamHandler so `app.*` logs are visible in Docker.
- CIDR validation on frontend and backend to prevent nmap argument injection.
- Pre-fetch canvas/hidden IPs before the scan loop (no N+1 queries); ARP table read off the event loop.
- Hide/ignore on a missing device returns 404 instead of 500.
## [1.7.1] - 2026-04-02
### Added
- Concurrent status checks via `asyncio.gather`, ending "maximum instances reached" spam.
- Improved IoT detection: two-phase nmap scan finds Shelly, Sonoff, Tapo devices with no open TCP ports.
- mDNS/Bonjour discovery (`_shelly._tcp`, `_esphomelib._tcp`, `_hap._tcp`, `_mqtt._tcp`, …) in parallel with nmap.
- 20+ IoT vendor MAC OUIs and CoAP ports added; IoT ranked above generic server.
### Fixed
- LXC/VM container mode: attaching to a Proxmox no longer creates a spurious edge; nesting is instant.
- Scheduler reliability: DetachedInstanceError, duplicate timestamps, shutdown handling, interval validation.
### Dependencies
- Added `zeroconf==0.131.0` for mDNS discovery.
## [1.7.0] - 2026-04-01
### Added
- **Lasso / box selection** — draw a rectangle to select multiple nodes; hold Space to pan; lasso/pan toggle in the controls.
- **Named groups** — group 2+ nodes into a resizable, renamable container with hide-border option and per-member navigation; persisted across save/reload.
- **Canvas search (Ctrl+F)** — filter nodes by label, IP, hostname, service; live match count; click to fly the camera.
### Fixed
- LXC install script: `apt-get update` runs verbosely with `--fix-missing` to handle stale Debian mirrors (fixes #36).
- Group parent/child relationships restore after save + reload.
- Removed React Flow's default grey background from container node types.
## [1.6.0] - 2026-03-30
### Added
- Scan deduplication — pending devices already on canvas or hidden are skipped on rescan; stale entries purged at scan start.
- Dedicated Settings panel (status-check interval moved out of the Scan Config modal).
- Live interval update without a server restart.
- Separate `/api/v1/settings` endpoint for the check interval.
### Fixed
- DEL key deletes selected nodes (previously Backspace only).
- Node deletion via shortcut/detail panel is undoable with Ctrl+Z.
- APScheduler guards against double-start and unguarded reschedule calls.
## [1.5.0] - 2026-03-29
### Added
- Inline service editing in the detail panel (pencil icon).
- Edge animation modes: None, Snake, Flow — fully persisted.
- Zone improvements: rename Rectangle → Zone, border-width selector, label position, text-size options.
### Fixed
- Edge animation not saved on new connections.
- Backend 422 when saving canvas with string animation values.
- Snake and Flow animations rendering identically in production.
- Login returned 500 instead of 401 when a bcrypt hash had `$` stripped by the shell (#21).
- Hardcoded CORS in docker-compose; clearer login error messages.
## [1.4.0] - 2026-03-28
### Added
- **Live View** — share a read-only view of the canvas on your network, no login required.
- Resizable nodes with persisted width/height.
### Fixed
- QEMU arm64 crash during Docker image build (`npm ci` illegal instruction on ARM64).
- nginx `reload-or-start` fallback broken on fresh LXC installs.
### CI
- ShellCheck + hadolint linting; Docker smoke tests and full-stack integration tests.
## [1.3.3] - 2026-03-27
### Fixed
- Scan failures on fresh Docker installs — `service_signatures.json` moved into the app package so the volume mount no longer overwrites it.
- Missing `ping` on Docker (`iputils-ping` added).
- Thread-safe signature-file loading; clear error if the file is missing.
- CORS restricted to the HTTP methods/headers the frontend uses.
## [1.3.2] - 2026-03-27
### Fixed
- Ping check method on fresh Docker installs — `iputils-ping` added to the backend image (`python:3.13-slim` ships without `ping`).
## [1.3.1] - 2026-03-27
### Added
- **YAML import/export** of the full canvas topology (nodes, edges, hardware specs, connections); import merges without overwriting; Dagre auto-layout on import; toolbar Import/Export buttons.
### Fixed
- ESLint 10 incompatibility from `npm audit fix` — pinned back to ESLint 9.x.
### Docs
- Split installation into `INSTALLATION.md`; documented the network scanner and dev mode.
## [1.3.0] - 2026-03-21
### Added
- Hardware specs per node (CPU model, cores, RAM, Disk) with "Show on node" and GB → TB formatting.
- `Docker Host` node type (Anchor icon), themed across all 5 themes.
- Group rectangle border style: Solid, Dashed, Dotted, Double, None.
- Categorized node-type selector (Hardware / Virtualization / IoT / Generic).
## [1.2.2] - 2026-03-17
### Added
- `scripts/update.sh` for fast in-place LXC updates (never touches `.env` or DB).
### Fixed
- `crypto.randomUUID` crash on HTTP/LXC installs — polyfill fallback.
- WebSocket failure on LXC (hardcoded port 8000 bypassing Nginx); added `/api/v1/status/ws/` upgrade block.
- Production build crash on LXC (test files in `tsconfig.app.json`).
### Security
- JWT no longer exposed in the WebSocket URL — sent as the first message after connect.
## [1.2.1] - 2026-03-16
### Added
- **MCP server** with HTTP/SSE transport for AI integration (Claude Code compatible); `parent_id` exposed in `update_node`; service-key auth for MCP → backend.
### Fixed
- "Add node" appeared broken with an empty Label — native validation doesn't render in Radix Dialog portals; now shows an inline error.
- SSE streaming crash; reduced `get_canvas` token usage.
## [1.2.0] - 2026-03-13
### Added
- Edge flow animation (per-edge toggle); Proxmox cluster edges animate bidirectionally.
- Keyboard shortcuts: Undo/Redo, Ctrl+K search, copy/paste, Ctrl+S save, `?` reference.
- Canvas history (50-entry undo/redo stack) with toolbar buttons.
- Node search spotlight (Ctrl+K) with fuzzy match and fly-to.
- Copy/paste nodes (Ctrl+C / Ctrl+V) with 50px offset and fresh IDs.
- Markdown table export of the node inventory.
- Clickable hostname in the detail panel.
- Shortcuts reference modal.
## [1.1.1] - 2026-03-11
### Added
- New logo and favicon; page title updated to Homelable.
- **Hide IPs** toggle in the sidebar — masks the last two octets.
### Fixed
- Auto-layout: peer nodes placed on the same row (no staircase); correct left-to-right ordering; child nodes always below their parent.
## [1.1.0] - 2026-03-10
### Added
- **Group rectangles** — decorative resizable zones with configurable label/font/position/colours and z-order; saved with the canvas.
- Add/remove services manually in the detail panel.
- All TCP services now clickable (HTTPS auto-detected for 443/8443; non-web ports excluded).
## [1.0.0] - 2026-03-09
First stable public release of **Homelable**, a self-hosted homelab visualization tool.
### Added
- **Canvas**: interactive React Flow diagram; 11 node types; 5 edge types; Proxmox nested nodes; Dagre auto-layout; zoom/pan; snap-to-grid; PNG export.
- **Network discovery**: nmap scanner over CIDR ranges; pending-device queue (approve/hide/ignore); service fingerprinting; MAC OUI detection for QEMU/Proxmox/VMware.
- **Status monitoring**: per-node ping/http/https/tcp/ssh/prometheus/health checks; live WebSocket updates; scheduled background checks.
- **Auth & persistence**: single-user JWT auth (bcrypt in `.env`); SQLite canvas state with an explicit Save button.
- **Standalone mode**: backend-free diagram editor with localStorage persistence.
- **Install options**: Docker Compose, Proxmox LXC, manual Debian/Ubuntu script.
[2.6.1]: https://github.com/Pouzor/homelable/compare/v2.6.0...v2.6.1
[2.6.0]: https://github.com/Pouzor/homelable/compare/v2.5.1...v2.6.0
[2.5.1]: https://github.com/Pouzor/homelable/compare/v2.5.0...v2.5.1
[2.5.0]: https://github.com/Pouzor/homelable/compare/v2.4.0...v2.5.0
[2.4.0]: https://github.com/Pouzor/homelable/compare/v2.3.0...v2.4.0
[2.3.0]: https://github.com/Pouzor/homelable/compare/v2.2.0...v2.3.0
[2.2.0]: https://github.com/Pouzor/homelable/compare/v2.1.1...v2.2.0
[2.1.1]: https://github.com/Pouzor/homelable/compare/v2.1.0...v2.1.1
[2.1.0]: https://github.com/Pouzor/homelable/compare/v2.0.3...v2.1.0
[2.0.3]: https://github.com/Pouzor/homelable/compare/v2.0.2...v2.0.3
[2.0.2]: https://github.com/Pouzor/homelable/compare/v2.0.1...v2.0.2
[2.0.1]: https://github.com/Pouzor/homelable/compare/v2.0.0...v2.0.1
[2.0.0]: https://github.com/Pouzor/homelable/compare/v1.13.0...v2.0.0
[1.13.0]: https://github.com/Pouzor/homelable/compare/v1.12.0...v1.13.0
[1.12.0]: https://github.com/Pouzor/homelable/compare/v1.11.0...v1.12.0
[1.11.0]: https://github.com/Pouzor/homelable/compare/v1.10.2...v1.11.0
[1.10.2]: https://github.com/Pouzor/homelable/compare/v1.10.1...v1.10.2
[1.10.1]: https://github.com/Pouzor/homelable/compare/v1.10.0...v1.10.1
[1.10.0]: https://github.com/Pouzor/homelable/compare/v1.9.0...v1.10.0
[1.9.0]: https://github.com/Pouzor/homelable/compare/v1.8.3...v1.9.0
[1.8.3]: https://github.com/Pouzor/homelable/compare/v1.8.2...v1.8.3
[1.8.2]: https://github.com/Pouzor/homelable/compare/v1.8.1...v1.8.2
[1.8.1]: https://github.com/Pouzor/homelable/compare/v1.8.0...v1.8.1
[1.8.0]: https://github.com/Pouzor/homelable/compare/v1.7.1...v1.8.0
[1.7.1]: https://github.com/Pouzor/homelable/compare/v1.7.0...v1.7.1
[1.7.0]: https://github.com/Pouzor/homelable/compare/v1.6.0...v1.7.0
[1.6.0]: https://github.com/Pouzor/homelable/compare/v1.5.0...v1.6.0
[1.5.0]: https://github.com/Pouzor/homelable/compare/v1.4.0...v1.5.0
[1.4.0]: https://github.com/Pouzor/homelable/compare/v1.3.3...v1.4.0
[1.3.3]: https://github.com/Pouzor/homelable/compare/v1.3.2...v1.3.3
[1.3.2]: https://github.com/Pouzor/homelable/compare/v1.3.1...v1.3.2
[1.3.1]: https://github.com/Pouzor/homelable/compare/v1.3.0...v1.3.1
[1.3.0]: https://github.com/Pouzor/homelable/compare/v1.2.2...v1.3.0
[1.2.2]: https://github.com/Pouzor/homelable/compare/v1.2.1...v1.2.2
[1.2.1]: https://github.com/Pouzor/homelable/compare/v1.2.0...v1.2.1
[1.2.0]: https://github.com/Pouzor/homelable/compare/v1.1.1...v1.2.0
[1.1.1]: https://github.com/Pouzor/homelable/compare/v1.1.0...v1.1.1
[1.1.0]: https://github.com/Pouzor/homelable/compare/v1.0.0...v1.1.0
[1.0.0]: https://github.com/Pouzor/homelable/releases/tag/v1.0.0
+276
View File
@@ -0,0 +1,276 @@
# Contributing to Homelable
Thanks for taking the time to contribute! This document covers everything you need to get started.
---
## Table of Contents
- [Ways to Contribute](#ways-to-contribute)
- [Reporting Bugs](#reporting-bugs)
- [Suggesting Features](#suggesting-features)
- [Development Setup](#development-setup)
- [Project Structure](#project-structure)
- [Coding Standards](#coding-standards)
- [Testing](#testing)
- [Submitting a Pull Request](#submitting-a-pull-request)
- [Commit Message Format](#commit-message-format)
---
## Ways to Contribute
- Report bugs or unexpected behavior
- Suggest new features or improvements
- Fix open issues (check the [issue tracker](https://github.com/Pouzor/homelable/issues))
- Improve documentation
- Add service signatures to `service_signatures.json`
---
## Reporting Bugs
Before opening an issue, search existing ones to avoid duplicates.
When filing a bug, include:
- **Homelable version** (visible in the sidebar bottom-left)
- **Deployment method** (Docker Compose, Proxmox LXC, source)
- **Steps to reproduce**
- **Expected vs actual behavior**
- **Relevant logs** (`docker compose logs backend` / `docker compose logs frontend`)
- **Browser console errors** if it's a UI issue
---
## Suggesting Features
Open an issue with the `enhancement` label. Describe:
- The problem you're trying to solve
- Your proposed solution
- Any alternatives you considered
For large changes, discuss first before writing code — it avoids wasted effort.
---
## Development Setup
### Prerequisites
- **Node.js 20+** and **npm**
- **Python 3.11–3.13** (3.14 not yet supported by all dependencies)
- **nmap** installed on your system (required for scanner)
- **Docker + Docker Compose** (optional, for full-stack testing)
### 1. Clone the repo
```bash
git clone https://github.com/Pouzor/homelable.git
cd homelable
```
### 2. Backend
```bash
cd backend
python3.13 -m venv .venv
source .venv/bin/activate # Windows: .venv\Scripts\activate
pip install -r requirements.txt
# Copy and configure environment
cp .env.example .env # edit AUTH_PASSWORD_HASH, SECRET_KEY, etc.
# Start the backend (auto-reloads on change)
uvicorn app.main:app --reload --port 8000
```
API docs available at `http://localhost:8000/docs`.
### 3. Frontend
```bash
cd frontend
npm install
npm run dev # http://localhost:5173
```
Vite proxies `/api` to `localhost:8000` — the backend must be running.
### 4. Verify tooling
```bash
./scripts/verify-tooling.sh
```
---
## Project Structure
```
homelable/
├── frontend/src/
│ ├── components/
│ │ ├── canvas/ # React Flow canvas, custom nodes & edges
│ │ ├── panels/ # Sidebar, detail panel, toolbar
│ │ ├── modals/ # Add/edit node, scan config, pending devices
│ │ └── ui/ # Shadcn/ui base components
│ ├── stores/ # Zustand state (canvas, auth, scan)
│ ├── hooks/ # Custom React hooks
│ ├── types/ # TypeScript interfaces & enums
│ ├── api/ # Axios client & typed endpoints
│ └── utils/ # Layout, export, color helpers
│
├── backend/app/
│ ├── api/routes/ # FastAPI route handlers
│ ├── services/ # Scanner, status checker, canvas service
│ ├── db/ # SQLAlchemy models, Alembic migrations
│ ├── schemas/ # Pydantic request/response schemas
│ └── core/ # Config, JWT, scheduler
│
├── docker/ # Nginx configs
├── scripts/ # LXC bootstrap, dev helpers
└── mcp/ # MCP server (AI integration)
```
---
## Coding Standards
### General
- No untested code merged — every feature or fix must include tests
- Keep changes focused — one concern per PR
### Frontend (TypeScript + React)
- Strict TypeScript — no `any`, no type assertions unless truly necessary
- React Flow node domain fields go in `node.data`, never on the node root
- State management via Zustand stores — no prop drilling beyond 2 levels
- Styling via TailwindCSS utility classes — follow the existing [design system](#design-system)
- Run before committing:
```bash
cd frontend
npm run lint
npm run typecheck
npm test
```
### Backend (Python + FastAPI)
- Python 3.11+ syntax
- Pydantic v2 schemas for all request/response types
- SQLAlchemy async sessions — never block the event loop
- Scanner logic runs in a background thread — never in an async route directly
- All schema changes via Alembic migrations — never modify tables directly
- Run before committing:
```bash
cd backend
source .venv/bin/activate
ruff check .
pytest
```
### Design System
| Token | Value |
|---|---|
| Background | `#0d1117` |
| Surface | `#161b22` |
| Card | `#21262d` |
| Accent cyan | `#00d4ff` |
| Online | `#39d353` |
| Offline | `#f85149` |
| Pending | `#e3b341` |
| Font (UI) | Inter |
| Font (IPs/ports) | JetBrains Mono |
---
## Testing
Tests run automatically via a pre-commit hook when frontend or backend files are staged.
### Frontend
```bash
cd frontend
npm test # run all tests
npm run test:coverage # with coverage report
```
Test files live in `__tests__/` next to their module, named `*.test.ts(x)`.
**What to test:** Zustand store actions, utility functions, non-trivial component logic.
### Backend
```bash
cd backend
source .venv/bin/activate
pytest # run all tests
pytest -v tests/test_nodes.py # single file
```
Test files live in `backend/tests/test_*.py`.
**What to test:** all API routes (happy path + error cases), auth flows, service logic.
Use the `client` and `headers` fixtures from `conftest.py` — they provide an in-memory SQLite database so tests are isolated and fast.
---
## Submitting a Pull Request
1. **Fork** the repo and create a branch from `main`:
```bash
git checkout -b feat/my-feature
```
2. **Make your changes** — include tests.
3. **Run the full test suite** (frontend + backend) and make sure everything passes.
4. **Open a PR** against `main`:
- Use a clear title (see commit format below)
- Describe what changed and why
- Reference any related issues (`Closes #123`)
- Include screenshots for UI changes
5. Keep the PR focused — one feature or fix per PR. Large refactors should be discussed in an issue first.
---
## Commit Message Format
Follow [Conventional Commits](https://www.conventionalcommits.org/):
```
<type>: <short description>
[optional body]
```
| Type | When to use |
|---|---|
| `feat` | New feature |
| `fix` | Bug fix |
| `docs` | Documentation only |
| `refactor` | Code change with no behavior change |
| `test` | Adding or fixing tests |
| `chore` | Build, deps, tooling |
**Examples:**
```
feat: add logout button to sidebar
fix: stop click propagation on pending device checkbox
docs: add CONTRIBUTING.md
```
---
## Questions?
Open a [GitHub Discussion](https://github.com/Pouzor/homelable/discussions) or drop a comment on a relevant issue.
+3 -2
View File
@@ -2,13 +2,14 @@ FROM python:3.13-slim
WORKDIR /app
# Install nmap for network scanning + iputils-ping for ping-based status checks
RUN apt-get update && apt-get install -y --no-install-recommends nmap iputils-ping && rm -rf /var/lib/apt/lists/*
# Install nmap for network scanning + iputils-ping for ping-based status checks + curl for the health check
RUN apt-get update && apt-get install -y --no-install-recommends nmap iputils-ping curl && rm -rf /var/lib/apt/lists/*
COPY backend/requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY backend/ .
COPY VERSION /app/VERSION
# Create data directory (volume mount point)
RUN mkdir -p /app/data
+3 -1
View File
@@ -1,7 +1,8 @@
# Stage 1: build
# Use the native build platform so npm ci never runs under QEMU emulation.
# The build output (static HTML/JS/CSS) is platform-independent.
FROM --platform=$BUILDPLATFORM node:20-alpine AS builder
# node:20-slim (Debian/glibc) avoids lightningcss musl binary resolution issues on Alpine.
FROM --platform=$BUILDPLATFORM node:20-slim AS builder
ARG VITE_STANDALONE=false
ENV VITE_STANDALONE=$VITE_STANDALONE
@@ -11,6 +12,7 @@ COPY frontend/package*.json ./
RUN npm ci
COPY frontend/ .
COPY VERSION ../VERSION
RUN npm run build
# Stage 2: serve
+293
View File
@@ -0,0 +1,293 @@
# Homelable Features
Here's what Homelable can do. One line on what each feature is, then how to switch it on and use it.
> **Two modes.** Same UI, two ways to run it:
> - **Full mode**, with the backend (Docker/LXC). Everything works.
> - **Standalone mode** (`VITE_STANDALONE=true`), no backend, canvas lives in your browser's `localStorage`. Great for just drawing; anything that needs a server (scanning, imports, device inventory, floor-plan upload, live view) is hidden.
>
> Features marked 🔒 need Full mode.
---
## Table of Contents
1. [Zones](#1-zones)
2. [Groups & Nesting](#2-groups--nesting)
3. [Text Annotations](#3-text-annotations)
4. [Multiple Canvases](#4-multiple-canvases)
5. [Rack Canvas](#5-rack-canvas)
6. [Customize Style](#6-customize-style)
7. [Floor Plan](#7-floor-plan-)
8. [Network Scanner (IP import)](#8-network-scanner-ip-import-)
9. [Zigbee Import](#9-zigbee-import-)
10. [Z-Wave Import](#10-z-wave-import-)
11. [Proxmox VE Import](#11-proxmox-ve-import-)
12. [Device Inventory](#12-device-inventory-)
13. [Live Status Monitoring](#13-live-status-monitoring-)
14. [Export (PNG / SVG / YAML / Markdown)](#14-export)
15. [Live View (read-only public canvas)](#15-live-view-)
16. [Gethomepage Widget](#16-gethomepage-widget-)
17. [MCP Server (AI integration)](#17-mcp-server-)
18. [Settings & Shortcuts](#18-settings--shortcuts)
19. [Authentication (Local / OpenID Connect)](#19-authentication-local--openid-connect-)
---
## 1. Zones
**What:** Labeled boxes to group devices by area: "Living room", "Rack 1", "DMZ", whatever makes sense to you.
**Use:**
- Sidebar → **Add Zone**. Give it a title and a color, then drag it around and resize it.
- Drop nodes onto a zone and Homelable asks if you want to add them to it.
- Zones sit behind your nodes and move on their own. They're just there to keep things tidy.
---
## 2. Groups & Nesting
**What:** Some devices hold others, like a **Proxmox** host with its **VMs** and **LXCs** inside. Those show up as an expandable container.
**Use:**
- Drag a `vm` or `lxc` onto a `proxmox` node, confirm **Add to container**, and it becomes a child.
- Click the container header to fold it open or shut; it resizes itself around what's inside.
- The Zigbee and Z-Wave imports build the same kind of hierarchy for you (coordinator → routers → end devices).
---
## 3. Text Annotations
**What:** Loose text labels for notes, section titles, or anything you want to call out on the canvas.
**Use:** Sidebar → **Add Text**, type, drop it where you want, style it.
---
## 4. Multiple Canvases
**What:** More than one diagram in a single install, say "Network", "Home automation", "Rack layout", each with its own nodes, links, floor plan, and style.
**Use:**
- The **canvas switcher** is at the top of the sidebar. Click to jump between canvases, or **New Canvas** to start a fresh one.
- Hover a canvas to **Edit** it (name, icon, floor plan) or **Delete** it. You can't delete the last one.
- Each canvas saves on its own, so hit **Save Canvas** after you change something.
---
## 5. Rack Canvas
**What:** The physical side of the lab, as its own kind of canvas: racks, the gear mounted in them, and the patch cables between their ports. Where a diagram says what talks to what, a rack says what sits in which U, and which switch port the uplink lands on.
**Use:**
- **Canvas switcher** → **New Canvas** → Kind **Rack**. A canvas cannot change kind afterwards.
- **Add Rack** in the header drops a rack; double-click its frame for U height, 19"/10" width, numbering direction, colours and the enclosed/open look.
- **+ Device** in the sidebar mounts something: an entry from your **Device Inventory** 🔒, a **new device** (which joins the inventory tagged *Rack devices*), or a rack-only **accessory** (blank, shelf, cable manager). Double-click a plate to edit it, and **Unmount** to take it out — the device itself stays in the inventory.
- Gear occupies a U range and part of a 12-column width grid, so half- and third-width machines share a U. A drop snaps to the nearest free slot, an impossible one previews red, and growing a device relocates it to the nearest slot that fits.
- The **Faceplate** field opens a visual catalog — servers, switches, routers, patch panels, UPS and PDUs, desktop NAS towers, shelves and blanks — drawn as vector artwork that scales with the rack and follows your theme.
- **Patch**: drag from one port to another (or click both in turn) to cable them, across racks if you need to. Click a cable to select it, then **Delete** or **Unplug**. A header select controls whether cables show on hover, always, or not at all.
- **Status**: pin a mount's colour, or set **Check device** 🔒 so it follows the status check already configured on the matching diagram node.
- **Import links** 🔒 derives patches from the physical links already drawn on your diagrams. Run it again after racking more gear — pairs already cabled are left alone.
- Saving is explicit — **Save Rack** — like any other canvas.
> **Full documentation:** [docs/rack-canvas.md](./docs/rack-canvas.md)
---
## 6. Customize Style
**What:** Repaint the whole thing with a preset theme, or roll your own node and edge colors.
**Use:**
- Toolbar → **Style**. Pick a preset: **Default**, **Dark**, **Light**, **Neon**, or **Matrix**.
- Or pick **Custom** and hit its **Edit** button to set border/background colors per node type and link colors per edge type.
- Theme and custom colors are saved **per canvas** on your next **Save Canvas**.
---
## 7. Floor Plan 🔒
**What:** Put a background image (a house plan, an office layout, a rack diagram) behind a canvas and lay your devices out on top of it.
**Use:**
- Open the **canvas switcher** → **Edit** the active canvas (or double-click the floor plan already on the canvas).
- In the **Floor Plan** section, upload an image and set its size and lock state.
- The image lives on the backend and is loaded by URL, never baked into the canvas, so your canvases stay light. *(See ADR-001; floor plans are Full mode only.)*
---
## 8. Network Scanner (IP import) 🔒
**What:** Point `nmap` at your network, fingerprint the services it finds, and turn hosts into nodes.
**Use:**
1. Sidebar → **Scan Network**. Scan History opens and keeps refreshing until it's done.
2. Set the CIDR ranges you want in `SCANNER_RANGES` (`.env`), or override them per scan in the dialog.
3. Whatever it finds shows up in the **Device Inventory** (below) to approve, hide, or ignore.
**Deep scan (custom ports):** to catch services on odd ports, add them in `.env`:
```env
SCANNER_HTTP_RANGES=["8080","9000-9100"]
SCANNER_HTTP_PROBE_ENABLED=true
SCANNER_HTTP_VERIFY_TLS=false
```
**Root note:** SYN scans and OS detection need root. If a scan trips on permissions, run `scripts/run_scan.py` with `sudo`, or on Linux give nmap the `NET_RAW` capability. Full details in the [README](./README.md#network-scanner).
---
## 9. Zigbee Import 🔒
**What:** Pull your **Zigbee2MQTT** topology in over MQTT and drop every device on the canvas as a typed node.
**Use:**
1. Sidebar → **Zigbee Import**.
2. Enter broker host/port (default `1883`), any credentials, and the base topic (default `zigbee2mqtt`).
3. **Test Connection** → **Fetch Devices** → pick from the grouped list → **Add N to Canvas**.
Nodes come in as `zigbee_coordinator` / `zigbee_router` / `zigbee_enddevice`. The hierarchy (coordinator → routers → end devices) and **LQI** are filled in automatically. More: [docs/zigbee-import.md](./docs/zigbee-import.md).
---
## 10. Z-Wave Import 🔒
**What:** Same idea for **Z-Wave JS UI**, over the same MQTT broker.
**Use:**
1. Sidebar → **Z-Wave Import**.
2. Enter broker host/port, any credentials, the MQTT prefix (default `zwave`), and the gateway name (default `zwavejs2mqtt`).
3. **Test Connection** → send them to **Pending** or straight to the **Canvas** → import → pick devices → **Add N to Canvas**.
Nodes: `zwave_coordinator` / `zwave_router` / `zwave_enddevice`. The hierarchy comes from each node's neighbor list (Z-Wave has no LQI). More: [docs/zwave-import.md](./docs/zwave-import.md).
---
## 11. Proxmox VE Import 🔒
**What:** Pull your **Proxmox VE** inventory (hosts, VMs, LXC) in over the Proxmox REST API — typed, named nodes with run state and hardware specs. Optional scheduled **auto-sync**; guest IPs already found by a scan are merged, not duplicated.
**Use:**
1. Create a read-only API token in Proxmox (Datacenter → Permissions → API Tokens, role `PVEAuditor`).
2. Sidebar → **Proxmox Import**.
3. Enter host, port (default `8006`), and the token (`user@realm!tokenid` + secret) — or leave blank to use the server token.
4. **Test Connection** → send to **Pending** or the **Canvas** → import → pick devices → **Add N to Canvas**.
Nodes: `proxmox` (host) / `vm` / `lxc`, linked host→guest by a `virtual` edge. The token is env-only, never stored on disk, never returned by the API. Enable auto-sync from **Settings** once `PROXMOX_TOKEN_ID` / `PROXMOX_TOKEN_SECRET` are set. More: [docs/proxmox-import.md](./docs/proxmox-import.md).
---
## 12. Device Inventory 🔒
**What:** The holding pen for everything found by a scan or import that isn't on the canvas yet, plus a separate **Hidden Devices** list.
**Use:**
- Sidebar → **Device Inventory**. Each entry shows IP, MAC, hostname, and any OS and services detected.
- Per device: **Approve** to drop a typed node on the canvas, **Hide** to stash it (you can get it back), or **Ignore** to dismiss it.
- **Hidden Devices** is the sidebar entry where you review and restore anything you've hidden.
---
## 13. Live Status Monitoring 🔒
**What:** Keeps checking each node and shows its status (🟢 online / 🔴 offline / ⚫ unknown) right on the canvas.
**Use:**
- Pick a **check method** per node when you add or edit it:
| Method | Checks |
|--------|--------|
| `ping` | ICMP reachability |
| `http` | GET, OK if status < 500 |
| `https` | GET with TLS verify |
| `tcp` | TCP connect to `host:port` |
| `ssh` | TCP connect to port 22 |
| `prometheus` | GET `/metrics` |
| `health` | GET `/health` |
- Checks run on a timer (`STATUS_CHECKER_INTERVAL`, 60s by default) and stream to the UI over WebSocket, no refresh. The sidebar footer keeps a running online/offline tally.
---
## 14. Export
**What:** Get your canvas out as a picture or as structured data.
**Use (toolbar):**
- **PNG**, a snapshot of the canvas, quality of your choice. Works in standalone too.
- **SVG**, vector export, keeps fonts, icons, and colors crisp. Same dialog as PNG.
- **Export (YAML)**, the whole canvas (nodes + links) as YAML you can re-import.
- **Markdown**, copies your device inventory as a Markdown table, handy for docs or a README.
---
## 15. Live View 🔒
**What:** A read-only, no-login snapshot of a canvas you can share on your LAN. Off by default.
**Use:**
1. Add `LIVEVIEW_KEY=your-secret-key` to `.env`, then `docker compose restart backend`.
2. Open `http://<your-homelab-ip>/view?key=your-secret-key`.
Pan and zoom only, no editing. Click a node with an IP and it opens in a new tab.
---
## 16. Gethomepage Widget 🔒
**What:** A tiny JSON stats endpoint for [gethomepage](https://gethomepage.dev)'s `customapi` widget. Off by default.
**Use:**
1. Add `HOMEPAGE_API_KEY=your-secret-key` to `.env`, restart the backend.
2. `GET /api/v1/stats/summary` with header `X-API-Key: your-secret-key` returns node counts, online/offline, pending, zigbee, and last scan time.
Widget snippet lives in the [README](./README.md#gethomepage-widget-read-only-stats).
---
## 17. MCP Server 🔒
**What:** A [Model Context Protocol](https://modelcontextprotocol.io) server so an MCP client (Claude Code, Claude Desktop, Open WebUI…) can read and change your topology. Optional, runs as its own service.
**Use:**
1. Add the keys to `.env`:
```env
MCP_API_KEY=mcp_sk_changeme # AI client -> MCP server
MCP_SERVICE_KEY=svc_changeme # MCP server -> backend (internal only)
# generate: python3 -c "import secrets; print(secrets.token_hex(32))"
```
2. `docker compose up -d mcp` (listens on `:8001`). No Docker? `sudo bash scripts/lxc-mcp-install.sh`.
3. Point your client at `http://<your-homelab-ip>:8001/mcp` with header `X-API-Key: <your key>`.
The AI can list nodes/edges/canvas/pending/scans, add/update/delete nodes and edges, kick off scans, and approve or hide devices. Keep port 8001 firewalled to your LAN. Full setup in the [README](./README.md#mcp-server-ai-integration-optional).
---
## 18. Settings & Shortcuts
**What:** App config and keyboard shortcuts.
**Use:**
- Sidebar → **Settings** for app-level config.
- **Search** to find nodes fast.
- Open the **Shortcuts** modal for the full key list (Save `Ctrl/Cmd+S`, undo/redo, and the rest).
---
## 19. Authentication (Local / OpenID Connect) 🔒
**What:** Homelable protects the app behind a login. Two exclusive modes, set once in `.env` with `AUTH_MODE`:
- **`local`** (default) — a single username + bcrypt-hashed password. Nothing changes for existing installs.
- **`oidc`** — sign in through your own identity provider (Authentik, Keycloak, Authelia, Google, …) via **Authorization Code + PKCE**. Good for SSO, per-user accounts at the IdP, MFA, and central account management.
**Use (OIDC):**
1. Register Homelable as a **confidential** client at your IdP and set the redirect URI to `https://<your-host>/api/v1/auth/oidc/callback`.
2. In `.env` set `AUTH_MODE=oidc` and the `OIDC_*` values (discovery URL, client id/secret, redirect URI, scopes), keep `CORS_ORIGINS` pinned to your browser origin, and give `SECRET_KEY` at least 32 bytes.
3. Restart the backend. The login screen now shows **Sign in with OpenID Connect**.
**How it protects you:** provider tokens are exchanged server-side and never touch the browser — Homelable issues its own short-lived, `__Host-`, HttpOnly, `SameSite=Lax` session cookie. Discovery metadata, issuer, audience, signature, expiry, nonce, state and PKCE are all validated; cookie-authenticated writes require a CSRF token and an allowed `Origin`; the status WebSocket authenticates from the cookie and validates `Origin`. Local Bearer auth and the MCP service key keep working unchanged. Full config, provider examples and troubleshooting: [docs/oidc-auth.md](./docs/oidc-auth.md).
> OIDC is Full-mode only — the no-backend standalone build has no login.
---
*Installing (Docker, Proxmox LXC, source) is covered in [INSTALLATION.md](./INSTALLATION.md). Running Home Assistant? See [homelable-hacs](https://github.com/Pouzor/homelable-hacs).*
+66 -32
View File
@@ -11,9 +11,18 @@ Open **http://localhost:3000** — login with `admin` / `admin`.
> Change the password before exposing to a network: edit `.env` and update `AUTH_USERNAME` / `AUTH_PASSWORD_HASH`.
>
> Generate a new hash: `docker compose exec backend python -c "from passlib.context import CryptContext; print(CryptContext(schemes=['bcrypt']).hash('yourpassword'))"`
>
> ⚠️ Keep the single quotes around the hash value in `.env` — bcrypt hashes contain `$` characters that Docker Compose would otherwise misinterpret.
Generate a new hash:
```bash
docker compose exec backend python -c 'import bcrypt; print(bcrypt.hashpw(b"yourpassword", bcrypt.gensalt()).decode())'
```
⚠️ **bcrypt hashes contain `$` characters** — how to handle them depends on where you set the value:
- **`.env` file** (recommended): wrap the hash in single quotes → `AUTH_PASSWORD_HASH='$2b$12$...'`
- **`docker-compose.yml` `environment:` block**: escape every `$` as `$$` — use this command to generate a pre-escaped hash:
```bash
docker compose exec backend python -c 'import bcrypt; print(bcrypt.hashpw(b"yourpassword", bcrypt.gensalt()).decode().replace("$", "$$"))'
```
## Quick Start — Frontend only
@@ -31,8 +40,33 @@ curl -fsSL https://raw.githubusercontent.com/Pouzor/homelable/main/install.sh |
cd homelable && docker compose pull && docker compose up -d
```
## Pre-built Docker images
The quick starts above never build anything — `install.sh` writes `docker-compose.prebuilt.yml` (or `docker-compose.standalone.yml` with `--standalone`) as your `docker-compose.yml`, and both pull ready-made images. They are published to the GitHub Container Registry on every push to `main` and every `v*` tag, for `linux/amd64` and `linux/arm64`:
| Image | Contents |
|---|---|
| [`ghcr.io/pouzor/homelable-backend`](https://github.com/Pouzor/homelable/pkgs/container/homelable-backend) | FastAPI API, scanner, status checker |
| [`ghcr.io/pouzor/homelable-frontend`](https://github.com/Pouzor/homelable/pkgs/container/homelable-frontend) | React SPA behind nginx, proxying `/api` to the backend |
| [`ghcr.io/pouzor/homelable-frontend-standalone`](https://github.com/Pouzor/homelable/pkgs/container/homelable-frontend-standalone) | Same SPA built with `VITE_STANDALONE=true` — no backend, canvases in `localStorage` |
| [`ghcr.io/pouzor/homelable-mcp`](https://github.com/Pouzor/homelable/pkgs/container/homelable-mcp) | MCP server exposing the canvas to AI clients |
Tags: `latest` (tip of `main`), plus `X.Y.Z` and `X.Y` for releases.
To wire it up by hand instead of using `install.sh`:
```bash
curl -fsSLO https://raw.githubusercontent.com/Pouzor/homelable/main/docker-compose.prebuilt.yml
curl -fsSL https://raw.githubusercontent.com/Pouzor/homelable/main/.env.example -o .env
docker compose -f docker-compose.prebuilt.yml up -d
```
Update the same way: `docker compose -f docker-compose.prebuilt.yml pull && … up -d`.
## Build from source
`docker-compose.yml` at the repo root builds the images locally instead of pulling them — use it for development, or to run a patched tree.
```bash
git clone https://github.com/Pouzor/homelable.git
cd homelable
@@ -44,37 +78,13 @@ docker compose up -d
## Proxmox LXC Install
Run this **on the Proxmox host** — it creates a Debian 12 LXC container and installs Homelable inside automatically:
You can now install Homelable with community-scripts (proxmox-VE) :
`https://community-scripts.org/scripts/homelable`
```bash
bash <(curl -fsSL https://raw.githubusercontent.com/Pouzor/homelable/main/scripts/install-proxmox.sh)
```
Default container settings: 2 cores, 1 GB RAM, 8 GB disk, DHCP on `vmbr0`. Override before running:
```bash
CTID=150 RAM=2048 STORAGE=local-zfs bash <(curl -fsSL .../install-proxmox.sh)
```
The backend runs as a systemd service, the frontend is served via nginx on port 80.
> To install manually inside an existing Debian/Ubuntu machine or LXC:
> ```bash
> bash <(curl -fsSL https://raw.githubusercontent.com/Pouzor/homelable/main/scripts/lxc-install.sh)
> ```
### Update (LXC)
Run the update script inside the container (pulls latest code, rebuilds frontend, restarts services — `.env` and database are never touched):
```bash
sudo bash /opt/homelable/scripts/update.sh
```
Or directly from GitHub:
```bash
sudo bash <(curl -fsSL https://raw.githubusercontent.com/Pouzor/homelable/main/scripts/update.sh)
bash -c "$(curl -fsSL https://raw.githubusercontent.com/community-scripts/ProxmoxVE/main/ct/homelable.sh)"
```
---
@@ -84,6 +94,7 @@ sudo bash <(curl -fsSL https://raw.githubusercontent.com/Pouzor/homelable/main/s
All configuration is done via `.env` (copied from `.env.example`):
```env
# Generate with: python3 -c "import secrets; print(secrets.token_hex(32))"
SECRET_KEY=change_me_in_production
# Auth — default: admin / admin
@@ -97,6 +108,29 @@ SCANNER_RANGES=["192.168.1.0/24"]
STATUS_CHECKER_INTERVAL=60
```
### OpenID Connect (optional)
OIDC is an exclusive alternative to the local password. Existing installs stay
in `AUTH_MODE=local` unless explicitly changed.
```env
AUTH_MODE=oidc
CORS_ORIGINS=["https://homelable.example"]
OIDC_DISCOVERY_URL=https://idp.example/application/o/homelable/.well-known/openid-configuration
OIDC_CLIENT_ID=homelable
OIDC_CLIENT_SECRET=replace-with-a-secret
OIDC_REDIRECT_URI=https://homelable.example/api/v1/auth/oidc/callback
OIDC_SCOPES="openid profile email"
OIDC_COOKIE_SECURE=true
OIDC_SESSION_EXPIRE_MINUTES=480
```
Register `OIDC_REDIRECT_URI` exactly at the identity provider. Production OIDC
requires HTTPS, a non-wildcard `CORS_ORIGINS`, and a confidential client. The
backend uses Authorization Code with PKCE and keeps provider tokens out of the
browser. `SECRET_KEY` must contain at least 32 bytes in OIDC mode. Do not expose
`OIDC_CLIENT_SECRET` in Compose YAML or commit it to Git.
All settings are also editable in-app via the **Scan Network** button.
---
+21
View File
@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2026 Remy Jardinet
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
+240 -13
View File
@@ -1,13 +1,45 @@
# Homelable
<h1 align="center"><img src="docs/logo/icon-inline.svg" alt="Homelable" width="58" align="middle" />&nbsp;Homelable</h1>
Homelable is a self-hosted infrastructure visualization solution. It provides a network scanning feature to accelerate the identification of machines and services deployed on your local infrastructure.
<p align="center">
<strong>Self-hosted homelab infrastructure visualization, scanning &amp; live monitoring</strong>
</p>
Homelable also offers a healthcheck system (WIP) through multiple methods (ping/TCP, /health API, etc.) to get a global overview of online/offline services.
<p align="center">
<a href="https://github.com/Pouzor/homelable/releases/latest"><img src="https://img.shields.io/github/v/release/Pouzor/homelable" alt="Latest release" /></a>
<a href="https://github.com/Pouzor/homelable/actions/workflows/docker-ci.yml"><img src="https://img.shields.io/github/actions/workflow/status/Pouzor/homelable/docker-ci.yml?branch=main&amp;label=build" alt="Build status" /></a>
<a href="https://github.com/Pouzor/homelable/blob/main/LICENSE"><img src="https://img.shields.io/badge/license-MIT-blue.svg" alt="License" /></a>
<a href="https://github.com/Pouzor/homelable/issues"><img src="https://img.shields.io/github/issues/Pouzor/homelable" alt="Issues" /></a>
<a href="https://github.com/Pouzor/homelable/stargazers"><img src="https://img.shields.io/github/stars/Pouzor/homelable?style=social" alt="Stars" /></a>
<a href="https://github.com/Pouzor/homelable/network/members"><img src="https://img.shields.io/github/forks/Pouzor/homelable?style=social" alt="Forks" /></a>
</p>
<p align="center">
<a href="https://trendshift.io/repositories/24461?utm_source=trendshift-badge&amp;utm_medium=badge&amp;utm_campaign=badge-trendshift-24461" target="_blank" rel="noopener noreferrer"><img src="https://trendshift.io/api/badge/trendshift/repositories/24461/daily?language=TypeScript" alt="Pouzor%2Fhomelable | Trendshift" width="130" /></a>
</p>
<p align="center">
<a href="#screenshots">Screenshots</a> ·
<a href="#features">Features</a> ·
<a href="#installation">Installation</a> ·
<a href="#rack-canvas">Rack Canvas</a> ·
<a href="#network-scanner">Network Scanner</a> ·
<a href="#zigbee2mqtt-import">Zigbee / Z-Wave</a> ·
<a href="#proxmox-ve-import">Proxmox</a> ·
<a href="#live-view-read-only-public-canvas">Live View</a> ·
<a href="#mcp-server-ai-integration-optional">MCP Server</a>
</p>
## About
Homelable is a self-hosted infrastructure visualization solution. It provides a network/zigbee scanning feature to accelerate the identification of machines, devices and services deployed on your local infrastructure.
Homelable also offers a healthcheck system through multiple methods (ping/TCP, /health API, etc.) to get a global overview of online/offline services.
You can also select some pre-built design styles, or personalize each device in your diagram.
If you just like the design, you can only run the frontend and export your design as PNG.
If you are running <img width="22" height="22" align="top" alt="New_Home_Assistant_logo" src="https://github.com/user-attachments/assets/3bb17686-c706-40ce-a2d3-57e02378f37c" /> Homeassistant, check the [Homelable HA version](https://github.com/Pouzor/homelable-hacs) (via HACS)
---
@@ -15,16 +47,43 @@ If you just like the design, you can only run the frontend and export your desig
<p align="center">
<img src="docs/homelable1.png" alt="Homelable canvas overview" width="100%" />
<img src="docs/homelable2.png" alt="Homelable node detail" width="100%" />
<img src="docs/homelable3.png" alt="Homelable sidebar and scan" width="48%" />
<img src="docs/homelable4.png" alt="Homelable edit pannel" width="48%" />
<img alt="Homelable Device inventory" src="https://github.com/user-attachments/assets/f3903ac8-354d-4873-81ba-1914971890ed" />
<img width="49.5%" alt="Homelable Custom node" src="https://github.com/user-attachments/assets/813725b1-376b-4bad-bb1f-0985f3bc7546" />
<img width="49.5%" alt="Homelable Zigbee Network" src="https://github.com/user-attachments/assets/35e18d11-8363-498d-ae3d-642685cac76d" />
</p>
---
## Features
From one-click **network scans** and **Proxmox / Zigbee / Z-Wave** imports to **live status monitoring**, floor plans, **rack canvases** with port-to-port patching, multi-canvas layouts and an **MCP server** for AI assistants — Homelable maps and watches your whole homelab.
Every feature, with how to turn it on and use it, is described in **[FEATURES.md](./FEATURES.md)**.
---
## Installation
Docker, Proxmox LXC, build from source, configuration, and development setup are all covered in **[INSTALLATION.md](./INSTALLATION.md)**.
Docker (from the **[pre-built GHCR images](./INSTALLATION.md#pre-built-docker-images)**), Proxmox LXC, build from source, configuration, and development setup are all covered in **[INSTALLATION.md](./INSTALLATION.md)**.
---
## Rack Canvas
Next to the network diagram, Homelable draws the **physical** side of your lab: racks, the gear mounted in them, and the patch cables between their ports. It is a canvas kind of its own — create it from the canvas switcher, **New Canvas → Kind → Rack**.
### Usage
1. **Add Rack** in the header drops a rack; double-click its frame to set U height, 19"/10" width, numbering direction and colours
2. **+ Device** in the sidebar mounts something — an entry from your Device Inventory, a new device (which joins the inventory), or an accessory (blank, shelf, cable manager)
3. Pick a **faceplate** from the visual catalog: servers, switches, routers, patch panels, UPS and PDUs, desktop NAS towers, shelves and blanks
4. Click **Patch**, then drag from one port to another to cable them — across racks if you need to. Click a cable and press Delete to unplug it
5. **Save Rack** when you're happy; nothing is written behind your back
Gear sits in a U range and part of a 12-column width grid, so half- and third-width machines share a U; a drop snaps to the nearest free slot. A mount can follow the status check of its matching diagram node, and **Import links** derives patches from the links already drawn on your diagrams.
> **Full documentation:** [docs/rack-canvas.md](./docs/rack-canvas.md)
---
@@ -37,6 +96,20 @@ The scanner runs `nmap -sV --open` on your configured CIDR ranges and populates
To save you time when mapping your infrastructure, Homlable can scan your network and report all the services it detects. It can also identify them, saving you even more time.
Click **Scan Network** in the sidebar. The Scan History tab opens automatically and refreshes every 3 seconds until the scan completes.
### Deep scan (custom ports)
By default the scanner only probes nmap's standard port set. To fingerprint services on non-standard ports, enable the deep scan via `.env` (all options are overridable per-scan from the scan dialog):
```env
# JSON array of port specs — each entry is a single port "N" or an inclusive
# range "N-M" (1–65535, N <= M). These are ports, not CIDRs or bare integers.
SCANNER_HTTP_RANGES=["8080","9000-9100"]
SCANNER_HTTP_PROBE_ENABLED=true # send an HTTP probe to those ports for service ID
SCANNER_HTTP_VERIFY_TLS=false # verify TLS certs on the HTTP probe
```
The listed ports are appended to nmap's `-p` spec. Invalid entries (out-of-range, malformed, or reversed ranges) are silently skipped.
### macOS / root privileges
Some nmap scan types (SYN scan, OS detection) require root. If the scan fails with a permissions error, run it manually with sudo using the included script:
@@ -74,6 +147,100 @@ Homelable continuously monitors your nodes and displays their live status (onlin
---
## Zigbee2MQTT Import
Homelable can connect directly to your MQTT broker and import your Zigbee network topology from **Zigbee2MQTT**, placing each device on the canvas as a typed node.
### Prerequisites
- A running **MQTT broker** (e.g. Mosquitto) accessible from the Homelable host
- **Zigbee2MQTT** connected to the broker with at least one device paired
### Usage
1. Click **Zigbee Import** in the left sidebar (below "Scan Network")
2. Enter your broker host, port (default `1883`), optional credentials, and base topic (default `zigbee2mqtt`)
3. Click **Test Connection** to verify reachability, then **Fetch Devices**
4. Select the devices you want from the grouped list (Coordinator / Router / End Device)
5. Click **Add N to Canvas** — devices are placed in a grid with IoT edges
### Node Types
| Type | Z2M Device | Icon |
|------|-----------|------|
| `zigbee_coordinator` | Coordinator | Network hub |
| `zigbee_router` | Router (mains-powered) | Radio |
| `zigbee_enddevice` | End Device (battery) | Antenna |
Hierarchy is set automatically: coordinator → routers → end devices (`parent_id`).
LQI (Link Quality Indicator) is stored as a node property.
> **Full documentation:** [docs/zigbee-import.md](./docs/zigbee-import.md)
---
## Z-Wave Import
Homelable can also import your **Z-Wave** network from **Z-Wave JS UI** (formerly `zwavejs2mqtt`) over the same MQTT broker, dropping each node on the canvas as a typed node.
### Prerequisites
- A running **MQTT broker** (e.g. Mosquitto) accessible from the Homelable host
- **Z-Wave JS UI** connected to the broker with its MQTT gateway enabled and at least one node included
### Usage
1. Click **Z-Wave Import** in the left sidebar (below "Zigbee Import")
2. Enter your broker host, port (default `1883`), optional credentials, MQTT prefix (default `zwave`), and gateway name (default `zwavejs2mqtt`)
3. Click **Test Connection** to verify reachability
4. Choose a target — **Pending section** or **Canvas directly** — then **Import to Pending** / **Fetch Devices**
5. Select the devices you want from the grouped list (Controller / Router / End Device) and click **Add N to Canvas**
### Node Types
| Type | Z-Wave Role | Icon |
|------|-------------|------|
| `zwave_coordinator` | Controller | Network hub |
| `zwave_router` | Routing (mains-powered) node | Radio |
| `zwave_enddevice` | End Device (battery) | Antenna |
Hierarchy is set automatically: controller → routers → end devices (`parent_id`), derived from each node's neighbor list. Z-Wave has no LQI, so that property is omitted.
> **Full documentation:** [docs/zwave-import.md](./docs/zwave-import.md)
---
## Proxmox VE Import
Homelable can import your **Proxmox VE** inventory over the Proxmox REST API — hosts, VMs and LXC containers arrive as typed, named nodes with run state and hardware specs, and can auto-sync on a schedule. Guest IPs that were already found by a network scan are merged in place (no duplicates).
### Prerequisites
- A reachable **Proxmox VE** host (default API port `8006`)
- A **Proxmox API token** with the read-only **`PVEAuditor`** role (Datacenter → Permissions → API Tokens)
### Usage
1. Click **Proxmox Import** in the left sidebar (below "Z-Wave Import")
2. Enter the host, port (default `8006`), and API token (`user@realm!tokenid` + secret) — or leave the token blank to use the server-configured one
3. Click **Test Connection** to verify reachability + token
4. Choose a target — **Pending section** or **Canvas directly** — then **Import to Pending** / **Fetch Inventory**
5. Select the devices from the grouped list (Hosts / Virtual Machines / LXC Containers) and click **Add N to Canvas**
### Node Types
| Type | Proxmox object | Icon |
|------|----------------|------|
| `proxmox` | Host / cluster member | Layers |
| `vm` | QEMU virtual machine | Box |
| `lxc` | LXC container | Container |
Each host is linked to its guests with a `virtual` edge. vCPU / RAM / disk are imported as node properties (hidden by default). Enable **auto-sync** from Settings once a server token is configured (`PROXMOX_TOKEN_ID` / `PROXMOX_TOKEN_SECRET`).
> **Full documentation:** [docs/proxmox-import.md](./docs/proxmox-import.md)
---
## Live View (read-only public canvas)
Live View lets you share a read-only snapshot of your canvas with anyone on your network — no login required. It is disabled by default.
@@ -99,7 +266,61 @@ The page shows your canvas in pan/zoom-only mode — no editing, no credentials
---
## MCP Server (AI Integration) (optionnal)
## Gethomepage Widget (read-only stats)
Homelable can expose a small JSON stats endpoint that [gethomepage](https://gethomepage.dev) consumes through its built-in `customapi` widget. Disabled by default.
### Activation
Add `HOMEPAGE_API_KEY` to your `.env`:
`HOMEPAGE_API_KEY=your-secret-key`
Restart the backend (`docker compose restart backend`).
### Endpoint
`GET /api/v1/stats/summary` — requires header `X-API-Key: your-secret-key`. Returns:
```json
{
"nodes": 12,
"online": 9,
"offline": 2,
"unknown": 1,
"pending_devices": 3,
"zigbee_devices": 5,
"last_scan_at": "2026-05-14T10:00:00+00:00"
}
```
### gethomepage `services.yaml` snippet
```yaml
- Homelab:
- Homelable:
icon: mdi-lan
href: http://homelable.local:3000
widget:
type: customapi
url: http://homelable.local:8000/api/v1/stats/summary
method: GET
headers:
X-API-Key: your-secret-key
mappings:
- field: nodes ; label: Nodes
- field: online ; label: Online
- field: offline ; label: Offline
- field: pending_devices ; label: Pending
- field: zigbee_devices ; label: Zigbee
- field: last_scan_at ; label: Last scan
```
The backend port (`8000`) must be reachable from your gethomepage container.
---
## MCP Server (AI Integration) (optional)
Homelable can exposes a [Model Context Protocol](https://modelcontextprotocol.io) server so any MCP-compatible AI client (Claude Code, Claude Desktop, Open WebUI…) can read your homelab topology and act on it.
@@ -134,11 +355,17 @@ docker compose up -d mcp
# MCP server is now listening on http://<your-homelab-ip>:8001
```
> **Proxmox LXC / bare-metal (no Docker):** create the LXC via
> [community-scripts/ProxmoxVE](https://github.com/community-scripts/ProxmoxVE) (or any
> Debian/Ubuntu LXC), then inside it run `sudo bash scripts/lxc-mcp-install.sh`.
> Installs a `homelable-mcp` systemd service, prompts for `MCP_API_KEY` / `MCP_SERVICE_KEY`
> (auto-generated if you press Enter), and skips prompts if `mcp/.env` already exists.
**3. Configure your AI client:**
**Claude Code** — run this command in your terminal:
```bash
claude mcp add --transport sse homelable http://<your-homelab-ip>:8001/mcp \
claude mcp add --transport http homelable http://<your-homelab-ip>:8001/mcp/ \
--header "X-API-Key: mcp_sk_yourkey"
```
@@ -147,8 +374,8 @@ Or add it manually to `~/.claude.json`:
{
"mcpServers": {
"homelable": {
"type": "sse",
"url": "http://<your-homelab-ip>:8001/mcp",
"type": "http",
"url": "http://<your-homelab-ip>:8001/mcp/",
"headers": {
"X-API-Key": "mcp_sk_yourkey"
}
@@ -162,8 +389,8 @@ Or add it manually to `~/.claude.json`:
{
"mcpServers": {
"homelable": {
"type": "sse",
"url": "http://<your-homelab-ip>:8001/mcp",
"type": "http",
"url": "http://<your-homelab-ip>:8001/mcp/",
"headers": {
"X-API-Key": "mcp_sk_yourkey"
}
+1
View File
@@ -0,0 +1 @@
3.3.5
+45 -10
View File
@@ -1,31 +1,66 @@
import hmac
from dataclasses import dataclass
from fastapi import Depends, Header, HTTPException, Request, status
from fastapi.security import HTTPAuthorizationCredentials, HTTPBearer
from app.core.config import settings
from app.core.security import decode_token
from app.core.security import decode_token, get_oidc_session
bearer = HTTPBearer(auto_error=False)
def get_current_user(
@dataclass(frozen=True)
class AuthContext:
subject: str
display_name: str
auth_method: str
issuer: str | None = None
csrf_token: str | None = None
def get_auth_context(
request: Request,
credentials: HTTPAuthorizationCredentials | None = Depends(bearer),
x_mcp_service_key: str | None = Header(default=None),
) -> str:
) -> AuthContext:
# 1. MCP service key (Docker-internal only — backend port is not externally exposed)
if x_mcp_service_key is not None:
if not settings.mcp_service_key:
raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="MCP service key not configured")
if not hmac.compare_digest(x_mcp_service_key.encode(), settings.mcp_service_key.encode()):
raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Invalid MCP service key")
return "__mcp_service__"
return AuthContext(
subject="__mcp_service__",
display_name="MCP service",
auth_method="mcp",
)
# 2. Standard JWT bearer token
if credentials is None:
raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Not authenticated")
username = decode_token(credentials.credentials)
if not username:
raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Invalid token")
return username
if credentials is not None:
username = decode_token(credentials.credentials)
if not username:
raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Invalid token")
return AuthContext(
subject=username,
display_name=username,
auth_method="local",
)
# 3. OIDC session cookie (only active in exclusive OIDC mode)
if settings.auth_mode == "oidc":
oidc_session = get_oidc_session(request)
if oidc_session is not None:
return AuthContext(
subject=oidc_session.subject,
display_name=oidc_session.display_name,
auth_method="oidc",
issuer=oidc_session.issuer,
csrf_token=oidc_session.csrf_token,
)
raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Not authenticated")
def get_current_user(context: AuthContext = Depends(get_auth_context)) -> str:
return context.subject
+97 -2
View File
@@ -1,12 +1,27 @@
import hmac
import logging
from collections.abc import Mapping
from typing import cast
from fastapi import APIRouter, HTTPException, status
from authlib.integrations.base_client.errors import OAuthError
from authlib.jose.errors import JoseError
from fastapi import APIRouter, Depends, HTTPException, Request, Response, status
from pydantic import BaseModel
from starlette.responses import RedirectResponse
from app.api.deps import AuthContext, get_auth_context
from app.core.config import settings
from app.core.security import create_access_token, verify_password
from app.core.security import (
clear_oidc_session_cookie,
create_access_token,
create_oidc_session_token,
get_oidc_client,
set_oidc_session_cookie,
verify_password,
)
router = APIRouter()
logger = logging.getLogger(__name__)
class LoginRequest(BaseModel):
@@ -19,8 +34,36 @@ class TokenResponse(BaseModel):
token_type: str = "bearer"
class AuthConfigResponse(BaseModel):
mode: str
oidc_login_url: str | None = None
class CurrentUserResponse(BaseModel):
subject: str
display_name: str
auth_method: str
issuer: str | None = None
csrf_token: str | None = None
def _require_oidc_mode() -> None:
if settings.auth_mode != "oidc":
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="OIDC login is disabled")
@router.get("/config", response_model=AuthConfigResponse)
async def auth_config() -> AuthConfigResponse:
return AuthConfigResponse(
mode=settings.auth_mode,
oidc_login_url="/api/v1/auth/oidc/login" if settings.auth_mode == "oidc" else None,
)
@router.post("/login", response_model=TokenResponse)
async def login(body: LoginRequest) -> TokenResponse:
if settings.auth_mode != "local":
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Local login is disabled")
# Always run both checks to prevent timing-based username enumeration.
# hmac.compare_digest is constant-time; verify_password (bcrypt) always runs.
username_ok = hmac.compare_digest(body.username, settings.auth_username)
@@ -29,3 +72,55 @@ async def login(body: LoginRequest) -> TokenResponse:
raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Invalid credentials")
token = create_access_token(body.username)
return TokenResponse(access_token=token)
@router.get("/oidc/login")
async def oidc_login(request: Request) -> Response:
_require_oidc_mode()
client = get_oidc_client()
return cast(Response, await client.authorize_redirect(request, settings.oidc_redirect_uri))
@router.get("/oidc/callback")
async def oidc_callback(request: Request) -> Response:
_require_oidc_mode()
client = get_oidc_client()
try:
token = await client.authorize_access_token(request)
except (OAuthError, JoseError) as exc:
logger.warning("OIDC callback rejected: %s", exc.error)
request.session.clear()
raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="OIDC authentication failed") from None
userinfo = token.get("userinfo")
if not isinstance(userinfo, Mapping):
request.session.clear()
raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="OIDC authentication failed")
try:
session_token = create_oidc_session_token(userinfo)
except ValueError:
request.session.clear()
raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="OIDC authentication failed") from None
request.session.clear()
response = RedirectResponse(url="/", status_code=status.HTTP_303_SEE_OTHER)
set_oidc_session_cookie(response, session_token)
return response
@router.get("/me", response_model=CurrentUserResponse)
async def current_user(context: AuthContext = Depends(get_auth_context)) -> CurrentUserResponse:
return CurrentUserResponse(
subject=context.subject,
display_name=context.display_name,
auth_method=context.auth_method,
issuer=context.issuer,
csrf_token=context.csrf_token,
)
@router.post("/logout", status_code=status.HTTP_204_NO_CONTENT)
async def logout(_: AuthContext = Depends(get_auth_context)) -> Response:
response = Response(status_code=status.HTTP_204_NO_CONTENT)
clear_oidc_session_cookie(response)
return response
+85 -20
View File
@@ -1,79 +1,144 @@
import uuid
from datetime import datetime, timezone
from typing import Any
from fastapi import APIRouter, Depends
from fastapi import APIRouter, Depends, Query
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.api.deps import get_current_user
from app.db.database import get_db
from app.db.models import CanvasState, Edge, Node
from app.db.models import CanvasState, Design, Edge, Node
from app.schemas.canvas import CanvasSaveRequest, CanvasStateResponse
from app.schemas.edges import EdgeResponse
from app.schemas.nodes import NodeResponse
from app.services.inventory_sync import (
facts_from_payload,
hydrated_node,
link_facts,
load_devices_for,
node_columns,
)
router = APIRouter()
@router.get("", response_model=CanvasStateResponse)
async def load_canvas(db: AsyncSession = Depends(get_db), _: str = Depends(get_current_user)) -> CanvasStateResponse:
nodes = (await db.execute(select(Node))).scalars().all()
edges = (await db.execute(select(Edge))).scalars().all()
state = await db.get(CanvasState, 1)
async def load_canvas(
design_id: str | None = Query(None, description="Design ID to load; uses first design if omitted"),
db: AsyncSession = Depends(get_db),
_: str = Depends(get_current_user),
) -> CanvasStateResponse:
if design_id is None:
first = (await db.execute(select(Design).order_by(Design.created_at).limit(1))).scalar()
design_id = first.id if first else None
if design_id is None:
return CanvasStateResponse(nodes=[], edges=[], viewport={"x": 0, "y": 0, "zoom": 1}, custom_style=None)
nodes = list((await db.execute(select(Node).where(Node.design_id == design_id))).scalars().all())
edges = (await db.execute(select(Edge).where(Edge.design_id == design_id))).scalars().all()
state = await db.get(CanvasState, design_id)
viewport: dict[str, Any] = state.viewport if state else {"x": 0, "y": 0, "zoom": 1}
# The device facts live on the inventory row now; hydrate them back into the
# node so the wire shape the canvas consumes is unchanged.
devices = await load_devices_for(db, nodes)
return CanvasStateResponse(
nodes=[NodeResponse.model_validate(n) for n in nodes],
nodes=[NodeResponse.model_validate(hydrated_node(n, devices.get(n.device_id or ""))) for n in nodes],
edges=[EdgeResponse.model_validate(e) for e in edges],
viewport=viewport,
custom_style=state.custom_style if state else None,
# A CanvasState row exists only after a save (or explicit design create),
# so its presence marks an intentional canvas vs. a never-touched one.
initialized=state is not None,
)
@router.post("/save")
async def save_canvas(
body: CanvasSaveRequest, db: AsyncSession = Depends(get_db), _: str = Depends(get_current_user)
) -> dict[str, bool]:
) -> dict[str, bool | str]:
design_id = body.design_id
if design_id is None:
first = (await db.execute(select(Design).order_by(Design.created_at).limit(1))).scalar()
design_id = first.id if first else None
if design_id is None:
new_design = Design(id=str(uuid.uuid4()), name="Network Topology", design_type="network")
db.add(new_design)
await db.flush()
design_id = new_design.id
incoming_node_ids = {n.id for n in body.nodes}
incoming_edge_ids = {e.id for e in body.edges}
# Delete nodes removed from canvas
existing_nodes = (await db.execute(select(Node))).scalars().all()
# Delete nodes removed from canvas (only within this design)
existing_nodes = (await db.execute(select(Node).where(Node.design_id == design_id))).scalars().all()
for node in existing_nodes:
if node.id not in incoming_node_ids:
await db.delete(node)
# Delete edges removed from canvas
existing_edges = (await db.execute(select(Edge))).scalars().all()
# Delete edges removed from canvas (only within this design)
existing_edges = (await db.execute(select(Edge).where(Edge.design_id == design_id))).scalars().all()
for edge in existing_edges:
if edge.id not in incoming_edge_ids:
await db.delete(edge)
await db.flush()
# Upsert nodes
# Upsert nodes, then route their device facts to the inventory row that owns
# them — an edit made on this canvas is an edit to the device itself, and
# every other canvas showing it follows.
saved: list[tuple[Node, dict[str, Any], list[str] | None]] = []
for node_data in body.nodes:
db_node = await db.get(Node, node_data.id)
payload = node_data.model_dump()
payload["design_id"] = design_id
facts = facts_from_payload(payload, label=node_data.label, node_type=node_data.type)
columns = node_columns(payload)
if db_node:
for field, value in node_data.model_dump().items():
for field, value in columns.items():
setattr(db_node, field, value)
else:
db.add(Node(**node_data.model_dump()))
db_node = Node(**columns)
db.add(db_node)
saved.append((db_node, facts, node_data.changed_facts))
await db.flush()
for node, facts, changed in saved:
# The payload carries a full copy of each device, hydrated when this
# canvas loaded. Routing all of it back would rewrite the row from a
# stale snapshot — a save made for nothing but a moved node could revert
# an edit meanwhile in the inventory modal. `changed_facts` (when the
# client tracked it) plus the row diff narrow the write to this canvas'
# actual edit.
await link_facts(
db,
node,
facts,
overwrite_scalars=True,
replace_lists=True,
only_changed=True,
changed_fields=changed,
)
# Upsert edges
for edge_data in body.edges:
db_edge = await db.get(Edge, edge_data.id)
payload = edge_data.model_dump()
payload["design_id"] = design_id
if db_edge:
for field, value in edge_data.model_dump().items():
for field, value in payload.items():
setattr(db_edge, field, value)
else:
db.add(Edge(**edge_data.model_dump()))
db.add(Edge(**payload))
# Upsert viewport
state = await db.get(CanvasState, 1)
# Upsert viewport + custom style
state = await db.get(CanvasState, design_id)
if state:
state.viewport = body.viewport
state.custom_style = body.custom_style
state.saved_at = datetime.now(timezone.utc)
else:
db.add(CanvasState(id=1, viewport=body.viewport))
db.add(CanvasState(design_id=design_id, viewport=body.viewport, custom_style=body.custom_style))
await db.commit()
return {"saved": True}
+239
View File
@@ -0,0 +1,239 @@
import uuid
from fastapi import APIRouter, Depends, HTTPException
from sqlalchemy import func, select
from sqlalchemy.ext.asyncio import AsyncSession
from app.api.deps import get_current_user
from app.db.database import get_db
from app.db.models import CanvasState, Design, Edge, Node, Rack, RackCable, RackDevice
from app.schemas.designs import DesignCopy, DesignCreate, DesignResponse, DesignUpdate
router = APIRouter()
# Node.type values that are canvas annotations rather than real devices. Kept in
# sync with the frontend (Sidebar counts, canvasSerializer types).
_GROUP_TYPE = "groupRect"
_TEXT_TYPE = "text"
@router.get("", response_model=list[DesignResponse])
async def list_designs(
db: AsyncSession = Depends(get_db),
_: str = Depends(get_current_user),
) -> list[DesignResponse]:
designs = (await db.execute(select(Design).order_by(Design.created_at))).scalars().all()
# One grouped query for all designs → node/group/text counts per design.
rows = (
await db.execute(select(Node.design_id, Node.type, func.count()).group_by(Node.design_id, Node.type))
).all()
counts: dict[str, dict[str, int]] = {}
for design_id, node_type, count in rows:
if design_id is None:
continue
bucket = counts.setdefault(design_id, {"node": 0, "group": 0, "text": 0})
if node_type == _GROUP_TYPE:
bucket["group"] += count
elif node_type == _TEXT_TYPE:
bucket["text"] += count
else:
bucket["node"] += count
result = []
for d in designs:
resp = DesignResponse.model_validate(d)
c = counts.get(d.id, {"node": 0, "group": 0, "text": 0})
resp.node_count = c["node"]
resp.group_count = c["group"]
resp.text_count = c["text"]
result.append(resp)
return result
@router.post("", response_model=DesignResponse, status_code=201)
async def create_design(
body: DesignCreate,
db: AsyncSession = Depends(get_db),
_: str = Depends(get_current_user),
) -> DesignResponse:
design = Design(name=body.name, design_type=body.design_type, icon=body.icon)
db.add(design)
await db.flush()
# Create empty canvas state for the new design
db.add(CanvasState(design_id=design.id))
await db.commit()
await db.refresh(design)
return DesignResponse.model_validate(design)
@router.post("/{source_id}/copy", response_model=DesignResponse, status_code=201)
async def copy_design(
source_id: str,
body: DesignCopy,
db: AsyncSession = Depends(get_db),
_: str = Depends(get_current_user),
) -> DesignResponse:
"""Create a new design that deep-copies the source's nodes, edges and canvas state."""
source = await db.get(Design, source_id)
if not source:
raise HTTPException(404, "Source design not found")
new_design = Design(name=body.name, icon=body.icon, design_type=source.design_type)
db.add(new_design)
await db.flush()
src_nodes = (await db.execute(select(Node).where(Node.design_id == source_id))).scalars().all()
src_edges = (await db.execute(select(Edge).where(Edge.design_id == source_id))).scalars().all()
# New id per source node so edges and parent links can be re-pointed at the copy.
id_map = {n.id: str(uuid.uuid4()) for n in src_nodes}
# Columns we set explicitly or let the DB default — never copy verbatim.
node_skip = {"id", "design_id", "parent_id", "created_at", "updated_at"}
for n in src_nodes:
cols = {c.name: getattr(n, c.name) for c in Node.__table__.columns if c.name not in node_skip}
db.add(Node(id=id_map[n.id], design_id=new_design.id, parent_id=None, **cols))
await db.flush() # nodes must exist before we wire self-referential parent_id
# Second pass: re-point parent links inside the copy.
for n in src_nodes:
if n.parent_id and n.parent_id in id_map:
child = await db.get(Node, id_map[n.id])
if child:
child.parent_id = id_map[n.parent_id]
edge_skip = {"id", "design_id", "source", "target", "created_at"}
for e in src_edges:
# Skip edges whose endpoints aren't part of this design (dangling FKs).
if e.source not in id_map or e.target not in id_map:
continue
cols = {c.name: getattr(e, c.name) for c in Edge.__table__.columns if c.name not in edge_skip}
db.add(
Edge(
id=str(uuid.uuid4()),
design_id=new_design.id,
source=id_map[e.source],
target=id_map[e.target],
**cols,
)
)
# Copy the rack canvas, if the source has one. Racks, mounted gear and patches
# all get new ids; the inventory / node links are carried over untouched, since
# a copy documents the same hardware.
src_racks = (await db.execute(select(Rack).where(Rack.design_id == source_id))).scalars().all()
rack_id_map = {r.id: str(uuid.uuid4()) for r in src_racks}
rack_skip = {"id", "design_id", "created_at", "updated_at"}
for r in src_racks:
cols = {c.name: getattr(r, c.name) for c in Rack.__table__.columns if c.name not in rack_skip}
db.add(Rack(id=rack_id_map[r.id], design_id=new_design.id, **cols))
src_rack_devices = (
await db.execute(select(RackDevice).where(RackDevice.design_id == source_id))
).scalars().all()
rack_device_id_map = {d.id: str(uuid.uuid4()) for d in src_rack_devices}
rack_device_skip = {"id", "design_id", "rack_id", "created_at", "updated_at"}
for d in src_rack_devices:
if d.rack_id not in rack_id_map:
continue
cols = {
c.name: getattr(d, c.name)
for c in RackDevice.__table__.columns
if c.name not in rack_device_skip
}
db.add(
RackDevice(
id=rack_device_id_map[d.id],
design_id=new_design.id,
rack_id=rack_id_map[d.rack_id],
**cols,
)
)
src_cables = (await db.execute(select(RackCable).where(RackCable.design_id == source_id))).scalars().all()
cable_skip = {"id", "design_id", "from_device_id", "to_device_id", "created_at"}
for c_row in src_cables:
if c_row.from_device_id not in rack_device_id_map or c_row.to_device_id not in rack_device_id_map:
continue
cols = {
c.name: getattr(c_row, c.name) for c in RackCable.__table__.columns if c.name not in cable_skip
}
db.add(
RackCable(
id=str(uuid.uuid4()),
design_id=new_design.id,
from_device_id=rack_device_id_map[c_row.from_device_id],
to_device_id=rack_device_id_map[c_row.to_device_id],
**cols,
)
)
# Copy canvas state (viewport, custom style, and the floor plan carried in viewport).
src_state = await db.get(CanvasState, source_id)
db.add(
CanvasState(
design_id=new_design.id,
viewport=src_state.viewport if src_state else {},
custom_style=src_state.custom_style if src_state else None,
)
)
await db.commit()
await db.refresh(new_design)
return DesignResponse.model_validate(new_design)
@router.put("/{design_id}", response_model=DesignResponse)
async def update_design(
design_id: str,
body: DesignUpdate,
db: AsyncSession = Depends(get_db),
_: str = Depends(get_current_user),
) -> DesignResponse:
design = await db.get(Design, design_id)
if not design:
raise HTTPException(404, "Design not found")
if body.name is not None:
design.name = body.name
if body.icon is not None:
design.icon = body.icon
await db.commit()
await db.refresh(design)
return DesignResponse.model_validate(design)
@router.delete("/{design_id}", status_code=204)
async def delete_design(
design_id: str,
db: AsyncSession = Depends(get_db),
_: str = Depends(get_current_user),
) -> None:
design = await db.get(Design, design_id)
if not design:
raise HTTPException(404, "Design not found")
# Count remaining designs — prevent deleting the last one
count = (await db.execute(select(Design))).scalars().all()
if len(count) <= 1:
raise HTTPException(400, "Cannot delete the only design")
# Delete associated canvas state, rack rows, edges, nodes. SQLite does not
# always enforce the ON DELETE CASCADE, so unwind by hand — cables before
# devices before racks.
cs = await db.get(CanvasState, design_id)
if cs:
await db.delete(cs)
for cable in (await db.execute(select(RackCable).where(RackCable.design_id == design_id))).scalars().all():
await db.delete(cable)
for rack_device in (
await db.execute(select(RackDevice).where(RackDevice.design_id == design_id))
).scalars().all():
await db.delete(rack_device)
for rack in (await db.execute(select(Rack).where(Rack.design_id == design_id))).scalars().all():
await db.delete(rack)
edges = (await db.execute(select(Edge).where(Edge.design_id == design_id))).scalars().all()
for e in edges:
await db.delete(e)
nodes = (await db.execute(select(Node).where(Node.design_id == design_id))).scalars().all()
for n in nodes:
await db.delete(n)
await db.delete(design)
await db.commit()
+66 -2
View File
@@ -4,11 +4,56 @@ from sqlalchemy.ext.asyncio import AsyncSession
from app.api.deps import get_current_user
from app.db.database import get_db
from app.db.models import Edge
from app.db.models import Design, Edge, Node
from app.schemas.edges import EdgeCreate, EdgeResponse, EdgeUpdate
router = APIRouter()
# ---------------------------------------------------------------------------
# Auto-handle helpers
# ---------------------------------------------------------------------------
async def _abs_y(db: AsyncSession, node_id: str) -> float | None:
"""Resolve the approximate absolute canvas Y of a node.
Walks up the parent chain (up to 8 levels) and accumulates pos_y offsets so
that children inside containers are compared correctly against top-level nodes.
Returns None when the node is not found.
"""
node = await db.get(Node, node_id)
if node is None:
return None
y = node.pos_y
current = node
for _ in range(8):
if current.parent_id is None:
break
parent = await db.get(Node, current.parent_id)
if parent is None:
break
y += parent.pos_y
current = parent
return y
async def _auto_handles(
db: AsyncSession, source_id: str, target_id: str
) -> tuple[str, str]:
"""Return (source_handle, target_handle) that reflect the upstream/downstream
relationship between two nodes.
- Source above target (lower Y value) → downstream flow: exit bottom, enter top
- Source below target → upstream flow: exit top, enter bottom
- Equal or unknown → default to bottom/top-t (most common topology direction)
"""
src_y = await _abs_y(db, source_id)
tgt_y = await _abs_y(db, target_id)
if src_y is None or tgt_y is None or src_y <= tgt_y:
return "bottom", "top-t"
return "top", "bottom-t"
@router.get("", response_model=list[EdgeResponse])
async def list_edges(db: AsyncSession = Depends(get_db), _: str = Depends(get_current_user)) -> list[Edge]:
@@ -18,7 +63,26 @@ async def list_edges(db: AsyncSession = Depends(get_db), _: str = Depends(get_cu
@router.post("", response_model=EdgeResponse, status_code=status.HTTP_201_CREATED)
async def create_edge(body: EdgeCreate, db: AsyncSession = Depends(get_db), _: str = Depends(get_current_user)) -> Edge:
edge = Edge(**body.model_dump())
data = body.model_dump()
# Same reconciliation as nodes: clients omitting design_id (MCP write tools)
# would create design_id=null edges that never render until a restart.
# Fall back to the first design so the edge attaches to a canvas.
if data.get("design_id") is None:
first_design = (await db.execute(select(Design).order_by(Design.created_at).limit(1))).scalar()
data["design_id"] = first_design.id if first_design else None
# Auto-assign source/target handles when the caller omits them.
# Compares the canvas Y positions of both nodes so that the edge always exits
# the upstream node's bottom and enters the downstream node's top (or vice versa
# for reverse flows), matching the UI convention for top-to-bottom topologies.
if data.get("source_handle") is None or data.get("target_handle") is None:
auto_src, auto_tgt = await _auto_handles(db, data["source"], data["target"])
if data.get("source_handle") is None:
data["source_handle"] = auto_src
if data.get("target_handle") is None:
data["target_handle"] = auto_tgt
edge = Edge(**data)
db.add(edge)
await db.commit()
await db.refresh(edge)
+39 -5
View File
@@ -2,22 +2,46 @@ import hmac
from typing import Any
from fastapi import APIRouter, Depends, HTTPException, Query
from pydantic import BaseModel
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.api.deps import get_current_user
from app.core.config import settings
from app.db.database import get_db
from app.db.models import CanvasState, Edge, Node
from app.db.models import CanvasState, Design, Edge, Node
from app.schemas.canvas import CanvasStateResponse
from app.schemas.edges import EdgeResponse
from app.schemas.nodes import NodeResponse
from app.services.inventory_sync import hydrated_node, load_devices_for
router = APIRouter()
class LiveViewConfigResponse(BaseModel):
"""Whether live view is enabled, plus the key (admin-only) to build share links."""
enabled: bool
key: str | None = None
@router.get("/config", response_model=LiveViewConfigResponse)
async def liveview_config(
_: str = Depends(get_current_user),
) -> LiveViewConfigResponse:
"""Authenticated: expose the configured live view key so the UI can build a
ready-to-use share link (e.g. /view?key=...&design=<id>).
Only reachable by a logged-in user — the key is never exposed publicly.
"""
key = settings.liveview_key or None
return LiveViewConfigResponse(enabled=bool(key), key=key)
@router.get("", response_model=CanvasStateResponse)
async def liveview_canvas(
key: str | None = Query(default=None),
design_id: str | None = Query(default=None, description="Design to show; uses first if omitted"),
db: AsyncSession = Depends(get_db),
) -> CanvasStateResponse:
"""Read-only public canvas endpoint.
@@ -30,12 +54,22 @@ async def liveview_canvas(
if not key or not hmac.compare_digest(key, settings.liveview_key):
raise HTTPException(status_code=403, detail="Invalid live view key")
nodes = (await db.execute(select(Node))).scalars().all()
edges = (await db.execute(select(Edge))).scalars().all()
state = await db.get(CanvasState, 1)
if design_id is None:
first = (await db.execute(select(Design).order_by(Design.created_at).limit(1))).scalar()
design_id = first.id if first else None
if design_id is None:
return CanvasStateResponse(nodes=[], edges=[], viewport={"x": 0, "y": 0, "zoom": 1}, custom_style=None)
nodes = list((await db.execute(select(Node).where(Node.design_id == design_id))).scalars().all())
edges = (await db.execute(select(Edge).where(Edge.design_id == design_id))).scalars().all()
state = await db.get(CanvasState, design_id)
viewport: dict[str, Any] = state.viewport if state else {"x": 0, "y": 0, "zoom": 1}
custom_style: dict[str, Any] | None = state.custom_style if state else None
# Device facts come off the inventory row, same as the editable canvas.
devices = await load_devices_for(db, nodes)
return CanvasStateResponse(
nodes=[NodeResponse.model_validate(n) for n in nodes],
nodes=[NodeResponse.model_validate(hydrated_node(n, devices.get(n.device_id or ""))) for n in nodes],
edges=[EdgeResponse.model_validate(e) for e in edges],
viewport=viewport,
custom_style=custom_style,
)
+100
View File
@@ -0,0 +1,100 @@
"""Generic media upload/serve endpoint.
Images are stored on disk (see `Settings.media_dir`) with server-generated
UUID filenames — never in the DB, and the client filename is never trusted.
Upload/delete require auth; GET is public so plain <img> tags and the read-only
live view can load images (filenames are unguessable).
Currently used by the floor-plan feature; kept deliberately generic so future
raw-image uploads reuse the same endpoint.
"""
import re
import uuid
from pathlib import Path
from fastapi import APIRouter, Depends, HTTPException, UploadFile, status
from fastapi.responses import FileResponse
from app.api.deps import get_current_user
from app.core.config import settings
router = APIRouter()
# content-type → extension. Also the allowlist of accepted uploads.
ALLOWED_TYPES: dict[str, str] = {
"image/png": ".png",
"image/jpeg": ".jpg",
"image/webp": ".webp",
}
# Magic-byte signatures for defense-in-depth (don't trust content-type alone).
_MAGIC: dict[str, tuple[bytes, ...]] = {
".png": (b"\x89PNG\r\n\x1a\n",),
".jpg": (b"\xff\xd8\xff",),
".webp": (b"RIFF",), # RIFF....WEBP; RIFF prefix is enough to reject non-images
}
MAX_BYTES = 10 * 1024 * 1024 # 10 MB
# Only ever serve/delete files we created: 32 hex chars + known extension.
_NAME_RE = re.compile(r"[0-9a-f]{32}\.(png|jpg|webp)")
def _resolve_media_path(filename: str) -> Path:
"""Return the existing media file named `filename`, or raise 404.
`filename` is validated against `_NAME_RE` (no separators, no `..`), then
matched by name against the actual directory listing. The user string is
only ever compared with `==` against trusted `iterdir()` entries — it never
builds a path — so a crafted value cannot escape the media dir.
"""
if not _NAME_RE.fullmatch(filename):
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Not found")
base = settings.media_dir()
if base.is_dir():
for entry in base.iterdir():
if entry.name == filename and entry.is_file():
return entry
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Not found")
@router.post("/upload")
async def upload_media(file: UploadFile, _user: str = Depends(get_current_user)) -> dict[str, str]:
ext = ALLOWED_TYPES.get(file.content_type or "")
if ext is None:
raise HTTPException(
status_code=status.HTTP_415_UNSUPPORTED_MEDIA_TYPE,
detail="Unsupported media type — PNG, JPEG, or WebP only",
)
# Read one byte past the cap so we can detect oversize without loading more.
data = await file.read(MAX_BYTES + 1)
if len(data) > MAX_BYTES:
raise HTTPException(
status_code=status.HTTP_413_CONTENT_TOO_LARGE,
detail="File too large (max 10 MB)",
)
if not data:
raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail="Empty file")
if not any(data.startswith(sig) for sig in _MAGIC[ext]):
raise HTTPException(
status_code=status.HTTP_415_UNSUPPORTED_MEDIA_TYPE,
detail="File content does not match its type",
)
media_dir = settings.media_dir()
media_dir.mkdir(parents=True, exist_ok=True)
name = f"{uuid.uuid4().hex}{ext}"
(media_dir / name).write_bytes(data)
return {"filename": name, "url": f"/api/v1/media/{name}"}
@router.get("/{filename}")
async def get_media(filename: str) -> FileResponse:
# _resolve_media_path returns only an existing file, else raises 404.
return FileResponse(_resolve_media_path(filename))
@router.delete("/{filename}", status_code=status.HTTP_204_NO_CONTENT)
async def delete_media(filename: str, _user: str = Depends(get_current_user)) -> None:
_resolve_media_path(filename).unlink()
+134 -13
View File
@@ -1,50 +1,171 @@
from fastapi import APIRouter, Depends, HTTPException, status
from typing import Any
from fastapi import APIRouter, Depends, HTTPException, Query, status
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.api.deps import get_current_user
from app.db.database import get_db
from app.db.models import Node
from app.db.models import Design, InventoryDevice, Node
from app.schemas.nodes import NodeCreate, NodeResponse, NodeUpdate
from app.services.inventory_sync import (
facts_from_payload,
facts_from_update,
hydrated_node,
link_facts,
load_devices_for,
node_columns,
)
from app.services.node_dedupe import find_duplicate_node
router = APIRouter()
# ---------------------------------------------------------------------------
# Auto-positioning helpers
# ---------------------------------------------------------------------------
# Canvas grid used when placing nodes without explicit coordinates.
# Each slot is wide/tall enough that a normal node card fits without overlap.
_SLOT_W = 200.0
_SLOT_H = 100.0
_MAX_COLS = 7
async def _find_free_position(db: AsyncSession, design_id: str | None) -> tuple[float, float]:
"""Return (x, y) for a new root-level node that doesn't collide with existing ones.
Snaps existing root nodes to a virtual grid and returns the first unoccupied
cell, scanning left-to-right then top-to-bottom.
"""
result = await db.execute(
select(Node.pos_x, Node.pos_y).where(
Node.design_id == design_id,
Node.parent_id.is_(None),
)
)
positions = list(result.all())
if not positions:
return 0.0, 0.0
# Snap each existing node to its true grid cell (negatives kept as-is). The
# search below only visits col >= 0 / row >= 0, so nodes parked in negative
# space never falsely block — or falsely free — a positive slot.
occupied: set[tuple[int, int]] = set()
for (px, py) in positions:
col = round(px / _SLOT_W)
row = round(py / _SLOT_H)
occupied.add((col, row))
for row in range(10_000):
for col in range(_MAX_COLS):
if (col, row) not in occupied:
return col * _SLOT_W, row * _SLOT_H
return 0.0, 0.0 # unreachable in practice
@router.get("", response_model=list[NodeResponse])
async def list_nodes(db: AsyncSession = Depends(get_db), _: str = Depends(get_current_user)) -> list[Node]:
result = await db.execute(select(Node))
return list(result.scalars().all())
async def list_nodes(
label: str | None = Query(None, description="Case-insensitive substring filter on node label"),
db: AsyncSession = Depends(get_db),
_: str = Depends(get_current_user),
) -> list[Any]:
query = select(Node)
if label:
query = query.where(Node.label.ilike(f"%{label}%"))
nodes = list((await db.execute(query)).scalars().all())
devices = await load_devices_for(db, nodes)
return [hydrated_node(n, devices.get(n.device_id or "")) for n in nodes]
@router.post("", response_model=NodeResponse, status_code=status.HTTP_201_CREATED)
async def create_node(body: NodeCreate, db: AsyncSession = Depends(get_db), _: str = Depends(get_current_user)) -> Node:
node = Node(**body.model_dump())
async def create_node(body: NodeCreate, db: AsyncSession = Depends(get_db), _: str = Depends(get_current_user)) -> Any:
data = body.model_dump()
# `force` bypasses the duplicate guard below; it is not a Node column.
force = data.pop("force", False)
# Attach to a design so the node lands on a canvas. Clients that don't send a
# design_id (e.g. the MCP write tools) would otherwise create design_id=null
# nodes that exist in the DB but never render in the UI until a container
# restart reconciles them. Fall back to the first design, matching bulk-approve.
if data.get("design_id") is None:
first_design = (await db.execute(select(Design).order_by(Design.created_at).limit(1))).scalar()
data["design_id"] = first_design.id if first_design else None
# Reject a silent duplicate: a node with the same ip OR mac already on the
# target design. Scripts/MCP clients get a clear 409 (with the existing id)
# instead of a second card for the same host. Pass force=True to override.
if not force:
dup = await find_duplicate_node(db, data["design_id"], data.get("ip"), data.get("mac"))
if dup is not None:
raise HTTPException(status_code=status.HTTP_409_CONFLICT, detail=dup)
# Auto-position: when pos_x / pos_y are omitted (None), find a free canvas
# slot so the new node doesn't land on top of an existing one.
# Child nodes (parent_id set) use (0, 0) relative to their parent instead.
if data["pos_x"] is None or data["pos_y"] is None:
if data.get("parent_id") is None:
auto_x, auto_y = await _find_free_position(db, data["design_id"])
if data["pos_x"] is None:
data["pos_x"] = auto_x
if data["pos_y"] is None:
data["pos_y"] = auto_y
else:
if data["pos_x"] is None:
data["pos_x"] = 0.0
if data["pos_y"] is None:
data["pos_y"] = 0.0
node = Node(**node_columns(data))
db.add(node)
await db.flush()
# A new device node gets (or joins) its Device Inventory row — the canvas is
# one more way to document hardware, not a parallel store.
await link_facts(
db, node, facts_from_payload(data, label=data["label"], node_type=data["type"])
)
await db.commit()
await db.refresh(node)
return node
return await _hydrate(db, node)
async def _hydrate(db: AsyncSession, node: Node) -> dict[str, Any]:
"""Node as the API reports it, with the device facts read off its row."""
device = await db.get(InventoryDevice, node.device_id) if node.device_id else None
return hydrated_node(node, device)
@router.get("/{node_id}", response_model=NodeResponse)
async def get_node(node_id: str, db: AsyncSession = Depends(get_db), _: str = Depends(get_current_user)) -> Node:
async def get_node(node_id: str, db: AsyncSession = Depends(get_db), _: str = Depends(get_current_user)) -> Any:
node = await db.get(Node, node_id)
if not node:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Node not found")
return node
return await _hydrate(db, node)
@router.patch("/{node_id}", response_model=NodeResponse)
async def update_node(
node_id: str, body: NodeUpdate, db: AsyncSession = Depends(get_db), _: str = Depends(get_current_user)
) -> Node:
) -> Any:
node = await db.get(Node, node_id)
if not node:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Node not found")
for field, value in body.model_dump(exclude_unset=True).items():
sent = body.model_dump(exclude_unset=True)
for field, value in node_columns(sent).items():
setattr(node, field, value)
# The user edited the device, not just its drawing: push the facts down.
# Only what was sent counts — an omitted field must not clear the row, and
# lists are replaced only when the client actually sent them.
facts = facts_from_update(sent)
await link_facts(
db,
node,
facts,
overwrite_scalars=True,
replace_lists="properties" in sent or "services" in sent,
)
await db.commit()
await db.refresh(node)
return node
return await _hydrate(db, node)
@router.delete("/{node_id}", status_code=status.HTTP_204_NO_CONTENT)
+510
View File
@@ -0,0 +1,510 @@
"""FastAPI router for Proxmox VE import + auto-sync config.
Fetches hosts/VMs/LXC from the Proxmox REST API and upserts them into the
pending inventory (same review→approve flow as scans and mesh imports).
Credentials: the API token comes from the request body when provided, else
falls back to the server-configured env token (``settings.proxmox_token_*``).
The token is never persisted by the app and never returned by any endpoint.
"""
import logging
from datetime import datetime, timezone
from typing import Any
from fastapi import APIRouter, BackgroundTasks, Depends, HTTPException
from sqlalchemy import delete as sa_delete
from sqlalchemy import or_, select
from sqlalchemy.ext.asyncio import AsyncSession
from app.api.deps import get_current_user
from app.core.config import settings
from app.core.scheduler import reschedule_proxmox_sync, set_proxmox_sync_enabled
from app.db.database import AsyncSessionLocal, get_db
from app.db.models import InventoryDevice, InventoryDeviceLink, Node, ScanRun
from app.schemas.proxmox import (
ProxmoxConfig,
ProxmoxConnectionRequest,
ProxmoxEdgeOut,
ProxmoxImportPendingResponse,
ProxmoxImportResponse,
ProxmoxNodeOut,
ProxmoxSyncConfig,
ProxmoxTestConnectionResponse,
)
from app.schemas.scan import ScanRunResponse
from app.services.discovery_sources import add_source
from app.services.mac_utils import normalize_mac
from app.services.node_dedupe import dedupe_nodes_by_device
from app.services.proxmox_service import (
build_proxmox_cluster_links,
build_proxmox_properties,
fetch_proxmox_inventory,
merge_proxmox_properties,
test_proxmox_connection,
)
logger = logging.getLogger(__name__)
router = APIRouter()
# Discovery sources for the two proxmox link shapes. Host→guest links render as
# 'virtual' edges; host↔host cluster links render as 'cluster' edges.
_PROXMOX_GUEST_SOURCE = "proxmox"
_PROXMOX_CLUSTER_SOURCE = "proxmox_cluster"
def _resolve_credentials(payload: ProxmoxConnectionRequest) -> tuple[str, str]:
"""Pick the API token: request body first, else server env config.
Raises HTTP 400 when neither carries a token.
"""
token_id = payload.token_id or settings.proxmox_token_id
token_secret = payload.token_secret or settings.proxmox_token_secret
if not token_id or not token_secret:
raise HTTPException(
status_code=400,
detail="No Proxmox API token provided and none configured on the server.",
)
return token_id, token_secret
@router.post("/test-connection", response_model=ProxmoxTestConnectionResponse)
async def test_connection_endpoint(
payload: ProxmoxConnectionRequest,
_: str = Depends(get_current_user),
) -> ProxmoxTestConnectionResponse:
"""Validate host reachability + token before importing."""
token_id, token_secret = _resolve_credentials(payload)
connected, message = await test_proxmox_connection(
host=payload.host,
port=payload.port,
token_id=token_id,
token_secret=token_secret,
verify_tls=payload.verify_tls,
)
return ProxmoxTestConnectionResponse(connected=connected, message=message)
@router.post("/import", response_model=ProxmoxImportResponse)
async def import_proxmox(
payload: ProxmoxConnectionRequest,
_: str = Depends(get_current_user),
) -> ProxmoxImportResponse:
"""Fetch the inventory and return nodes + edges ready for canvas drop."""
token_id, token_secret = _resolve_credentials(payload)
try:
nodes_raw, edges_raw = await fetch_proxmox_inventory(
host=payload.host,
port=payload.port,
token_id=token_id,
token_secret=token_secret,
verify_tls=payload.verify_tls,
)
except ConnectionError as exc:
raise HTTPException(status_code=502, detail=str(exc)) from exc
except ValueError as exc:
raise HTTPException(status_code=422, detail=str(exc)) from exc
except Exception as exc:
logger.exception("Unexpected error during Proxmox import")
raise HTTPException(status_code=500, detail="Unexpected error during Proxmox import") from exc
nodes = [ProxmoxNodeOut(**n) for n in nodes_raw]
edges = [ProxmoxEdgeOut(**e) for e in edges_raw]
return ProxmoxImportResponse(nodes=nodes, edges=edges, device_count=len(nodes))
@router.post("/import-pending", response_model=ScanRunResponse)
async def import_proxmox_to_pending(
payload: ProxmoxConnectionRequest,
background_tasks: BackgroundTasks,
db: AsyncSession = Depends(get_db),
_: str = Depends(get_current_user),
) -> ScanRun:
"""Queue a Proxmox pending import as a background scan run (kind=proxmox)."""
token_id, token_secret = _resolve_credentials(payload)
run = ScanRun(
status="running",
kind="proxmox",
ranges=[f"{payload.host}:{payload.port}"],
)
db.add(run)
await db.commit()
await db.refresh(run)
background_tasks.add_task(
_background_proxmox_import,
run.id,
payload.host,
payload.port,
token_id,
token_secret,
payload.verify_tls,
)
return run
@router.post("/sync-now", response_model=ScanRunResponse)
async def sync_proxmox_now(
background_tasks: BackgroundTasks,
db: AsyncSession = Depends(get_db),
_: str = Depends(get_current_user),
) -> ScanRun:
"""Trigger an immediate Proxmox inventory sync using the server env config.
Same background flow as ``/import-pending`` but sources host + token from
``settings`` (env) rather than the request body — the manual counterpart to
the scheduled auto-sync job. Requires the env token to be configured.
"""
if not (settings.proxmox_host and settings.proxmox_token_id and settings.proxmox_token_secret):
raise HTTPException(
status_code=400,
detail="Cannot sync: no Proxmox host/token configured on the server.",
)
run = ScanRun(
status="running",
kind="proxmox",
ranges=[f"{settings.proxmox_host}:{settings.proxmox_port}"],
)
db.add(run)
await db.commit()
await db.refresh(run)
background_tasks.add_task(
_background_proxmox_import,
run.id,
settings.proxmox_host,
settings.proxmox_port,
settings.proxmox_token_id,
settings.proxmox_token_secret,
settings.proxmox_verify_tls,
)
return run
async def _background_proxmox_import(
run_id: str,
host: str,
port: int,
token_id: str,
token_secret: str,
verify_tls: bool,
) -> None:
async with AsyncSessionLocal() as db:
try:
nodes_raw, edges_raw = await fetch_proxmox_inventory(
host=host,
port=port,
token_id=token_id,
token_secret=token_secret,
verify_tls=verify_tls,
)
result = await _persist_pending_import(db, nodes_raw, edges_raw)
run = await db.get(ScanRun, run_id)
if run:
run.status = "done"
run.devices_found = result.device_count
run.finished_at = datetime.now(timezone.utc)
run.error = _guest_visibility_advisory(nodes_raw)
await db.commit()
# Nudge the frontend to reload the inventory (same signal the IP scan
# emits) so imported/merged devices appear without a manual refresh.
from app.api.routes.status import broadcast_scan_update
await broadcast_scan_update(run_id=run_id, devices_found=result.device_count)
except Exception as exc:
logger.exception("Proxmox import %s failed", run_id)
await db.rollback()
run = await db.get(ScanRun, run_id)
if run:
run.status = "error"
run.error = str(exc)[:500]
run.finished_at = datetime.now(timezone.utc)
await db.commit()
def _guest_visibility_advisory(nodes_raw: list[dict[str, Any]]) -> str | None:
"""Non-fatal advisory when hosts import but no VMs/LXC were visible.
A Proxmox API token that lacks ``VM.Audit`` sees empty ``qemu``/``lxc`` lists
(HTTP 200, no error), so only host nodes come through. The usual cause is a
privilege-separated token whose effective rights are the *intersection* with
the user's rights — granting PVEAuditor to the token alone is not enough when
the user has none. Surface that instead of a silent success.
"""
hosts = sum(1 for n in nodes_raw if n.get("type") == "proxmox")
guests = len(nodes_raw) - hosts
if hosts and not guests:
return (
f"Imported {hosts} host(s) but no VMs or LXC were visible to the API "
"token. Grant the PVEAuditor role at path '/' to BOTH the token and "
"the user (privilege-separated tokens get the intersection of token "
"and user rights), then re-import."
)
return None
async def _persist_pending_import(
db: AsyncSession,
nodes_raw: list[dict[str, Any]],
edges_raw: list[dict[str, Any]],
) -> ProxmoxImportPendingResponse:
"""Upsert Proxmox nodes/edges into device_inventory + device_inventory_links.
Two-tier identity (order matters):
1. Match an existing canvas Node or pending row by **IP** (merge into a
device previously found by a scan) — never duplicate.
2. Else match by synthetic ``ieee_address`` (``pve-...``).
Update-in-place only. Nothing is ever deleted; hidden rows stay hidden.
"""
await dedupe_nodes_by_device(db)
cluster_pairs = build_proxmox_cluster_links(nodes_raw)
cluster_members = {ieee for pair in cluster_pairs for ieee in pair}
pending_created = 0
pending_updated = 0
for n in nodes_raw:
ieee = n.get("ieee_address")
if not ieee:
continue
ip = n.get("ip")
mac = normalize_mac(n.get("mac"))
props = build_proxmox_properties(n)
# Match by ieee OR ip OR mac (the cross-source dedup key — a stopped VM
# has no IP but its configured NIC MAC still matches an ARP-scanned
# device). The inventory row owns the facts, so the import merges into
# it whether or not the device is drawn anywhere; a node only needs its
# cluster handles adjusted. Do NOT stomp user-set type/status.
pending = await _find_pending(db, ieee, ip, mac)
drawn = bool(
pending
and (
await db.execute(select(Node.id).where(Node.device_id == pending.id).limit(1))
).scalar_one_or_none()
)
if pending is None:
db.add(_new_pending(ieee, ip, mac, n, props, status="pending"))
pending_created += 1
else:
if drawn:
# Already on a canvas: refresh the facts but leave the lifecycle
# alone (_refresh_pending would revive it as pending).
await _ensure_inventory_row(db, ieee, ip, mac, n, props, approved=True)
else:
_refresh_pending(pending, ieee, ip, mac, n, props)
pending.cpu_count = pending.cpu_count or n.get("cpu_count")
pending.ram_gb = pending.ram_gb or n.get("ram_gb")
pending.disk_gb = pending.disk_gb or n.get("disk_gb")
pending_updated += 1
# A cluster host needs one left + one right handle for the cluster
# edge endpoints (both default to 0) — the one piece of this that is
# genuinely about how the node is drawn.
if ieee in cluster_members:
for en in (
await db.execute(select(Node).where(Node.device_id == pending.id))
).scalars().all():
en.left_handles = max(en.left_handles or 0, 1)
en.right_handles = max(en.right_handles or 0, 1)
links_recorded = await _replace_links(db, edges_raw, cluster_pairs)
await db.commit()
return ProxmoxImportPendingResponse(
pending_created=pending_created,
pending_updated=pending_updated,
links_recorded=links_recorded,
device_count=len(nodes_raw),
)
async def _find_pending(
db: AsyncSession, ieee: str, ip: str | None, mac: str | None
) -> InventoryDevice | None:
filters = [InventoryDevice.ieee_address == ieee]
if ip:
filters.append(InventoryDevice.ip == ip)
if mac:
filters.append(InventoryDevice.mac == mac)
return (
await db.execute(select(InventoryDevice).where(or_(*filters)))
).scalars().first()
def _new_pending(
ieee: str,
ip: str | None,
mac: str | None,
n: dict[str, Any],
props: list[dict[str, Any]],
status: str,
) -> InventoryDevice:
return InventoryDevice(
ieee_address=ieee,
ip=ip,
mac=mac,
hostname=n.get("hostname"),
friendly_name=n.get("label"),
suggested_type=n.get("type"),
vendor=n.get("vendor"),
model=n.get("model"),
properties=props,
# Specs the guest reports. They used to land only on the canvas node;
# the inventory row owns them now.
cpu_count=n.get("cpu_count"),
ram_gb=n.get("ram_gb"),
disk_gb=n.get("disk_gb"),
status=status,
discovery_source=_PROXMOX_GUEST_SOURCE,
discovery_sources=[_PROXMOX_GUEST_SOURCE],
)
def _sources_after_merge(row: InventoryDevice) -> list[str]:
"""Discovery sources for an inventory row after a Proxmox import merges in.
Must run BEFORE the ``pve-`` ieee is adopted onto the row, so it can tell
whether the row was originally a scanned device. Preserves the prior scan
origin — including legacy rows created before ``discovery_sources`` existed
(empty list) and possibly with a NULL ``discovery_source`` — so the IP tag
survives the merge. A row that carries an IP but was not itself a Proxmox
device (no ``pve-`` ieee) was found by a scan; keep an IP-scan source.
"""
sources = add_source(row.discovery_sources, row.discovery_source)
was_scanned = not (row.ieee_address or "").startswith("pve-")
if was_scanned and row.ip and not any(s in ("arp", "mdns") for s in sources):
sources = add_source(sources, "arp")
return add_source(sources, _PROXMOX_GUEST_SOURCE)
def _refresh_pending(
pending: InventoryDevice,
ieee: str,
ip: str | None,
mac: str | None,
n: dict[str, Any],
props: list[dict[str, Any]],
) -> None:
# Compute sources before adopting the pve ieee (needs the pre-merge origin).
pending.discovery_sources = _sources_after_merge(pending)
pending.ieee_address = pending.ieee_address or ieee
pending.ip = ip or pending.ip
pending.mac = pending.mac or mac
pending.hostname = n.get("hostname") or pending.hostname
pending.friendly_name = n.get("label") or pending.friendly_name
pending.suggested_type = n.get("type") or pending.suggested_type
pending.vendor = n.get("vendor") or pending.vendor
pending.model = n.get("model") or pending.model
pending.properties = merge_proxmox_properties(list(pending.properties or []), props)
if pending.status == "approved":
# Approved earlier but the canvas Node is gone — revive so it reappears.
pending.status = "pending"
# hidden stays hidden.
async def _ensure_inventory_row(
db: AsyncSession,
ieee: str,
ip: str | None,
mac: str | None,
n: dict[str, Any],
props: list[dict[str, Any]],
approved: bool,
) -> None:
"""Ensure an inventory row exists for a device already on a canvas, so it
shows in the inventory with an 'In N canvas' badge. Never changes status."""
inv = await _find_pending(db, ieee, ip, mac)
if inv is None:
db.add(_new_pending(ieee, ip, mac, n, props, status="approved" if approved else "pending"))
else:
# Compute sources before adopting the pve ieee (needs the pre-merge origin).
inv.discovery_sources = _sources_after_merge(inv)
inv.ieee_address = inv.ieee_address or ieee
inv.ip = ip or inv.ip
inv.mac = inv.mac or mac
inv.hostname = n.get("hostname") or inv.hostname
inv.suggested_type = n.get("type") or inv.suggested_type
inv.properties = merge_proxmox_properties(list(inv.properties or []), props)
async def _replace_links(
db: AsyncSession,
edges_raw: list[dict[str, Any]],
cluster_pairs: list[tuple[str, str]],
) -> int:
"""Wipe all proxmox-source links and re-insert the freshly discovered set.
Two link shapes: host→guest (``proxmox`` → 'virtual' edges) and host↔host
(``proxmox_cluster`` → 'cluster' edges).
"""
await db.execute(
sa_delete(InventoryDeviceLink).where(
InventoryDeviceLink.discovery_source.in_([_PROXMOX_GUEST_SOURCE, _PROXMOX_CLUSTER_SOURCE])
)
)
recorded = 0
seen: set[tuple[str, str]] = set()
def _add(src: str | None, tgt: str | None, source: str) -> None:
nonlocal recorded
if not src or not tgt or (src, tgt) in seen:
return
seen.add((src, tgt))
db.add(InventoryDeviceLink(source_ieee=src, target_ieee=tgt, discovery_source=source))
recorded += 1
for e in edges_raw:
_add(e.get("source"), e.get("target"), _PROXMOX_GUEST_SOURCE)
for src, tgt in cluster_pairs:
_add(src, tgt, _PROXMOX_CLUSTER_SOURCE)
return recorded
@router.get("/config", response_model=ProxmoxConfig)
async def get_proxmox_config(_: str = Depends(get_current_user)) -> ProxmoxConfig:
"""Return non-secret Proxmox config. Never includes the token — only whether
one is configured on the server."""
return ProxmoxConfig(
host=settings.proxmox_host,
port=settings.proxmox_port,
verify_tls=settings.proxmox_verify_tls,
sync_enabled=settings.proxmox_sync_enabled,
sync_interval=settings.proxmox_sync_interval,
token_configured=bool(settings.proxmox_token_id and settings.proxmox_token_secret),
)
@router.post("/config", response_model=ProxmoxConfig)
async def save_proxmox_config(
payload: ProxmoxSyncConfig,
_: str = Depends(get_current_user),
) -> ProxmoxConfig:
"""Persist the auto-sync activation (enabled + interval) and apply it live.
This is the ONLY Proxmox config the app writes. Connection settings
(host, port, token, verify_tls) are env-only and are never accepted or
persisted here — enabling auto-sync requires host + token already set in the
server env, since the scheduled job reads them from there.
"""
if payload.sync_enabled and not (
settings.proxmox_host and settings.proxmox_token_id and settings.proxmox_token_secret
):
raise HTTPException(
status_code=400,
detail="Cannot enable auto-sync: no Proxmox host/token configured in the server env.",
)
try:
settings.proxmox_sync_enabled = payload.sync_enabled
settings.proxmox_sync_interval = payload.sync_interval
settings.save_overrides()
set_proxmox_sync_enabled(payload.sync_enabled)
if payload.sync_enabled:
reschedule_proxmox_sync(payload.sync_interval)
except HTTPException:
raise
except Exception as exc:
raise HTTPException(status_code=500, detail=str(exc)) from exc
return await get_proxmox_config()
+354
View File
@@ -0,0 +1,354 @@
from datetime import datetime, timezone
from typing import Any, TypeVar
from fastapi import APIRouter, Depends, HTTPException, Query
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.api.deps import get_current_user
from app.db.database import get_db
from app.db.models import CanvasState, Design, InventoryDevice, Node, Rack, RackCable, RackDevice
from app.schemas.racks import (
RackCableResponse,
RackDeviceResponse,
RackInventoryItem,
RackInventoryResponse,
RackResponse,
RackSaveRequest,
RackServiceInfo,
RackStateResponse,
)
router = APIRouter()
# Device kinds that cannot be racked. An exclusion list rather than an allow list:
# new hardware types should show up in the tray by default, and only the obviously
# virtual / mesh / annotation kinds need to opt out.
_UNRACKABLE_TYPES = {
"vm",
"lxc",
"docker_container",
"mobile",
"laptop",
"light",
"socket",
"load",
"groupRect",
"group",
"text",
"zigbee_coordinator",
"zigbee_router",
"zigbee_enddevice",
"zwave_coordinator",
"zwave_router",
"zwave_enddevice",
}
_COMMON_PORTS = {22, 80, 443}
def _service_name(device: InventoryDevice) -> str | None:
"""Name a device after the app it runs, the way the Device Inventory does.
Ports everyone exposes (ssh, http, https) say nothing, and the generic web
category loses to a real match, so "jellyfin" wins over "http".
"""
candidates = [
s
for s in (device.services or [])
if isinstance(s, dict)
and s.get("category")
and s.get("service_name")
# A service with no port is not a fingerprint the inventory names a
# device after either — and `None not in _COMMON_PORTS` is True.
and s.get("port") is not None
and s.get("port") not in _COMMON_PORTS
]
if not candidates:
return None
named = next((s for s in candidates if str(s["category"]).lower() != "web"), candidates[0])
name = named.get("service_name")
return str(name) if name else None
def _device_label(device: InventoryDevice) -> str:
"""Inventory naming, mirrored: friendly name, host, app, IP, IEEE, id.
Must stay in step with `deviceLabel` in `InventoryDevicesModal.tsx` — a device
the user picked out of the inventory by one name should not turn up in the
rack picker under another.
"""
return (
device.friendly_name
or device.hostname
or _service_name(device)
or device.ip
or device.ieee_address
or device.id
)
# A mount prints its services as a short list, not a port scan dump.
_MAX_SERVICES = 12
def _services(raw: Any) -> list[RackServiceInfo]:
"""Fingerprinted services, deduped and trimmed for the rack's info panel."""
out: list[RackServiceInfo] = []
seen: set[tuple[int | None, str | None]] = set()
for entry in raw or []:
if not isinstance(entry, dict):
continue
raw_port = entry.get("port")
port = raw_port if isinstance(raw_port, int) else None
raw_name = entry.get("service_name") or entry.get("name")
name = str(raw_name) if raw_name else None
if port is None and name is None:
continue
if (port, name) in seen:
continue
seen.add((port, name))
out.append(RackServiceInfo(port=port, name=name))
if len(out) == _MAX_SERVICES:
break
return out
def _ip_tokens(ip: str | None) -> list[str]:
"""Split a device ``ip`` field into individual addresses.
Mirrors the scan routes: the canvas stores multiple addresses in one
comma-separated string, so matching must compare per-address.
"""
return [t.strip() for t in ip.split(",") if t.strip()] if ip else []
_Row = TypeVar("_Row", Rack, RackDevice, RackCable)
async def _owned(
db: AsyncSession, model: type[_Row], row_id: str, design_id: str
) -> _Row | None:
"""Fetch a rack row by id, refusing one that belongs to another design.
The upsert below sets `design_id` from the payload, so without this a save
carrying an id from design A would quietly move A's rack — devices and
cables included — into design B, and A would lose it with no error. Ids come
from the client, so a copied design or a stale tab is enough to collide.
"""
row = await db.get(model, row_id)
if row is not None and row.design_id != design_id:
raise HTTPException(409, f"{model.__name__} {row_id} belongs to another design")
return row
async def _require_design(db: AsyncSession, design_id: str) -> Design:
design = await db.get(Design, design_id)
if not design:
raise HTTPException(404, "Design not found")
return design
@router.get("", response_model=RackStateResponse)
async def load_racks(
design_id: str = Query(..., description="Rack design to load"),
db: AsyncSession = Depends(get_db),
_: str = Depends(get_current_user),
) -> RackStateResponse:
await _require_design(db, design_id)
racks = (await db.execute(select(Rack).where(Rack.design_id == design_id))).scalars().all()
devices = (
await db.execute(select(RackDevice).where(RackDevice.design_id == design_id))
).scalars().all()
cables = (await db.execute(select(RackCable).where(RackCable.design_id == design_id))).scalars().all()
state = await db.get(CanvasState, design_id)
return RackStateResponse(
racks=[RackResponse.model_validate(r) for r in racks],
devices=[RackDeviceResponse.model_validate(d) for d in devices],
cables=[RackCableResponse.model_validate(c) for c in cables],
viewport=state.viewport if state else {"x": 0, "y": 0, "zoom": 1},
)
@router.post("/save")
async def save_racks(
body: RackSaveRequest,
db: AsyncSession = Depends(get_db),
_: str = Depends(get_current_user),
) -> dict[str, bool]:
"""Persist the full rack state of one design: upsert what is sent, prune the rest."""
await _require_design(db, body.design_id)
rack_ids = {r.id for r in body.racks}
device_ids = {d.id for d in body.devices}
# A device must land in a rack that is part of the same payload, and a cable
# must join two devices in it — otherwise the prune below would orphan them.
for device in body.devices:
if device.rack_id not in rack_ids:
raise HTTPException(400, f"Device {device.id} references unknown rack {device.rack_id}")
for cable in body.cables:
if cable.from_device_id not in device_ids or cable.to_device_id not in device_ids:
raise HTTPException(400, f"Cable {cable.id} references a device outside this payload")
# Prune first (cables, then devices, then racks) so freed rows never collide
# with an upsert that reuses their geometry.
for existing_cable in (
await db.execute(select(RackCable).where(RackCable.design_id == body.design_id))
).scalars().all():
if existing_cable.id not in {c.id for c in body.cables}:
await db.delete(existing_cable)
for existing_device in (
await db.execute(select(RackDevice).where(RackDevice.design_id == body.design_id))
).scalars().all():
if existing_device.id not in device_ids:
await db.delete(existing_device)
for existing_rack in (
await db.execute(select(Rack).where(Rack.design_id == body.design_id))
).scalars().all():
if existing_rack.id not in rack_ids:
await db.delete(existing_rack)
await db.flush()
for rack_data in body.racks:
payload: dict[str, Any] = rack_data.model_dump()
payload["design_id"] = body.design_id
db_rack = await _owned(db, Rack, rack_data.id, body.design_id)
if db_rack:
for field, value in payload.items():
setattr(db_rack, field, value)
else:
db.add(Rack(**payload))
await db.flush() # racks must exist before devices point at them
for device_data in body.devices:
payload = device_data.model_dump()
payload["design_id"] = body.design_id
db_device = await _owned(db, RackDevice, device_data.id, body.design_id)
if db_device:
for field, value in payload.items():
setattr(db_device, field, value)
else:
db.add(RackDevice(**payload))
await db.flush() # devices must exist before cables point at them
for cable_data in body.cables:
payload = cable_data.model_dump()
payload["design_id"] = body.design_id
db_cable = await _owned(db, RackCable, cable_data.id, body.design_id)
if db_cable:
for field, value in payload.items():
setattr(db_cable, field, value)
else:
db.add(RackCable(**payload))
# Viewport lives on the design's shared CanvasState row, like the logical canvas.
state = await db.get(CanvasState, body.design_id)
if state:
state.viewport = body.viewport
state.saved_at = datetime.now(timezone.utc)
else:
db.add(CanvasState(design_id=body.design_id, viewport=body.viewport))
await db.commit()
return {"saved": True}
@router.get("/inventory", response_model=RackInventoryResponse)
async def rack_inventory(
design_id: str = Query(..., description="Rack design the tray is for"),
db: AsyncSession = Depends(get_db),
_: str = Depends(get_current_user),
) -> RackInventoryResponse:
"""Device Inventory entries that can be racked, flagged with what is already mounted.
Reads `device_inventory` — the inventory survives approval and node deletion,
so unracking or deleting a canvas node never removes the entry here.
"""
await _require_design(db, design_id)
devices = (
await db.execute(select(InventoryDevice).where(InventoryDevice.status != "hidden"))
).scalars().all()
mounts = (
await db.execute(
select(RackDevice.device_id, RackDevice.node_id).where(
RackDevice.design_id == design_id
)
)
).all()
mounted = {device_id for device_id, _ in mounts if device_id}
# A mount can name its canvas node itself, when the user linked one by hand
# in the rack. That beats the IEEE/IP guess below: the guess is what failed
# to find anything in the first place.
pinned = {device_id: node_id for device_id, node_id in mounts if device_id and node_id}
# Correlate against canvas nodes: a node names the inventory row it draws,
# so the IEEE/IP guessing this used to do is gone — the link is explicit.
nodes = (await db.execute(select(Node))).scalars().all()
by_id = {n.id: n for n in nodes}
by_device: dict[str, Node] = {}
for node in nodes:
if node.device_id:
by_device.setdefault(node.device_id, node)
# Which canvas the matched node lives on — the rack prints it so the user
# knows where the logical twin of a mount is drawn.
design_names = {
d.id: d.name for d in (await db.execute(select(Design))).scalars().all()
}
# A pinned node may draw a *different* device than the mount names, so its
# view is read off its own row.
devices_by_id = {d.id: d for d in devices}
items = []
for device in devices:
if device.suggested_type in _UNRACKABLE_TYPES:
continue
# A node the user linked by hand wins; a deleted one falls back to the
# node that draws this device, if any.
linked: Node | None = by_id.get(pinned.get(device.id) or "") or by_device.get(device.id)
# The logical view is whatever the linked node actually draws. Usually
# that is this same row — the node_* fields then simply mirror it.
node_device = None
if linked is not None:
node_device = devices_by_id.get(linked.device_id or "")
if node_device is None and linked.device_id:
node_device = await db.get(InventoryDevice, linked.device_id)
items.append(
RackInventoryItem(
id=device.id,
label=_device_label(device),
suggested_type=device.suggested_type,
ip=device.ip,
status=device.status,
discovery_source=device.discovery_source,
mac=device.mac or device.ieee_address,
hostname=device.hostname,
os=device.os,
services=_services(device.services),
node_id=linked.id if linked else None,
node_status=node_device.status_live if node_device else None,
node_label=(
(node_device.label if node_device else None) or linked.label
) if linked else None,
node_type=(
(node_device.type if node_device else None) or linked.type
) if linked else None,
node_ip=node_device.ip if node_device else None,
node_mac=(node_device.mac or node_device.ieee_address) if node_device else None,
node_hostname=node_device.hostname if node_device else None,
node_os=node_device.os if node_device else None,
node_check_method=node_device.check_method if node_device else None,
node_design_id=linked.design_id if linked else None,
node_design_name=(
design_names.get(linked.design_id) if linked and linked.design_id else None
),
node_last_seen=node_device.last_seen if node_device else None,
racked=device.id in mounted,
)
)
return RackInventoryResponse(items=items)
File diff suppressed because it is too large Load Diff
+26 -3
View File
@@ -1,20 +1,33 @@
"""App-level settings (status checker interval, etc.)."""
from fastapi import APIRouter, Depends, HTTPException
from pydantic import BaseModel
from pydantic import BaseModel, Field
from app.api.deps import get_current_user
from app.core.config import settings
from app.core.scheduler import (
reschedule_service_checks,
reschedule_status_checks,
set_service_checks_enabled,
)
router = APIRouter()
class AppSettings(BaseModel):
interval_seconds: int
# Mirrors the floor enforced by reschedule_status_checks(): rejecting here
# returns 422 instead of persisting the value and then failing with a 500.
interval_seconds: int = Field(ge=10)
service_check_enabled: bool = False
service_check_interval: int = Field(default=300, ge=30)
@router.get("", response_model=AppSettings)
async def get_settings(_: str = Depends(get_current_user)) -> AppSettings:
return AppSettings(interval_seconds=settings.status_checker_interval)
return AppSettings(
interval_seconds=settings.status_checker_interval,
service_check_enabled=settings.service_check_enabled,
service_check_interval=settings.service_check_interval,
)
@router.post("", response_model=AppSettings)
@@ -23,7 +36,17 @@ async def update_settings(
) -> AppSettings:
try:
settings.status_checker_interval = payload.interval_seconds
settings.service_check_enabled = payload.service_check_enabled
settings.service_check_interval = payload.service_check_interval
settings.save_overrides()
# Apply the status-check schedule live. Without this the new interval is
# persisted and shown in the UI, but the running job keeps the old one
# until the backend restarts.
reschedule_status_checks(payload.interval_seconds)
# Apply the service-check schedule live.
set_service_checks_enabled(payload.service_check_enabled)
if payload.service_check_enabled:
reschedule_service_checks(payload.service_check_interval)
return payload
except Exception as exc:
raise HTTPException(status_code=500, detail=str(exc)) from exc
+82
View File
@@ -0,0 +1,82 @@
import hmac
from fastapi import APIRouter, Depends, Header, HTTPException
from sqlalchemy import func, select
from sqlalchemy.ext.asyncio import AsyncSession
from app.core.config import settings
from app.db.database import get_db
from app.db.models import InventoryDevice, Node, ScanRun
router = APIRouter()
def _check_key(x_api_key: str | None) -> None:
if not settings.homepage_api_key:
raise HTTPException(status_code=403, detail="Stats endpoint is disabled")
if not x_api_key or not hmac.compare_digest(x_api_key, settings.homepage_api_key):
raise HTTPException(status_code=403, detail="Invalid API key")
@router.get("/summary")
async def summary(
x_api_key: str | None = Header(default=None, alias="X-API-Key"),
db: AsyncSession = Depends(get_db),
) -> dict[str, object]:
"""Read-only stats payload for the gethomepage `customapi` widget.
Disabled unless HOMEPAGE_API_KEY is set. Caller must send the same
value in the `X-API-Key` header.
"""
_check_key(x_api_key)
# A node's reachability lives on the Device Inventory row it draws, so the
# per-canvas counts come from the join. A node with no row (canvas
# furniture) counts as unknown.
status_rows = (
await db.execute(
select(InventoryDevice.status_live, func.count())
.select_from(Node)
.join(InventoryDevice, Node.device_id == InventoryDevice.id)
.group_by(InventoryDevice.status_live)
)
).all()
counts = {(row[0] or "unknown"): row[1] for row in status_rows}
unlinked = (
await db.execute(select(func.count()).select_from(Node).where(Node.device_id.is_(None)))
).scalar_one()
if unlinked:
counts["unknown"] = counts.get("unknown", 0) + unlinked
pending = (
await db.execute(
select(func.count())
.select_from(InventoryDevice)
.where(InventoryDevice.status == "pending")
)
).scalar_one()
zigbee = (
await db.execute(
select(func.count())
.select_from(Node)
.join(InventoryDevice, Node.device_id == InventoryDevice.id)
.where(InventoryDevice.ieee_address.isnot(None))
)
).scalar_one()
last_scan_at = (
await db.execute(select(func.max(ScanRun.finished_at)))
).scalar_one()
return {
"nodes": sum(counts.values()),
"online": counts.get("online", 0),
"offline": counts.get("offline", 0),
"unknown": counts.get("unknown", 0),
# Wire key kept from before the table was renamed to `device_inventory`:
# the stats payload is a published contract, and the MCP server reads it.
"pending_devices": pending,
"zigbee_devices": zigbee,
"last_scan_at": last_scan_at.isoformat() if last_scan_at else None,
}
+68 -16
View File
@@ -1,8 +1,15 @@
import contextlib
import json
from fastapi import APIRouter, WebSocket, WebSocketDisconnect
from app.core.security import decode_token
from app.core.config import settings
from app.core.security import (
decode_oidc_session_token,
decode_token,
oidc_session_cookie_name,
origin_is_allowed,
)
router = APIRouter()
@@ -10,30 +17,50 @@ router = APIRouter()
_connections: list[WebSocket] = []
def _drop(websocket: WebSocket) -> None:
"""Remove a connection if still present — idempotent, never raises."""
with contextlib.suppress(ValueError):
_connections.remove(websocket)
@router.websocket("/ws/status")
async def ws_status(websocket: WebSocket) -> None:
# Accept first so we can send a close frame with a reason code
await websocket.accept()
try:
# Expect the first message to be a JSON auth payload: {"token": "<jwt>"}
raw = await websocket.receive_text()
try:
payload = json.loads(raw)
token = payload.get("token", "")
except (json.JSONDecodeError, AttributeError):
token = ""
if not token or not decode_token(token):
if not origin_is_allowed(websocket.headers.get("origin")):
await websocket.close(code=1008) # Policy Violation
return
if settings.auth_mode == "oidc":
cookie_token = websocket.cookies.get(oidc_session_cookie_name(), "")
if not cookie_token or decode_oidc_session_token(cookie_token) is None:
await websocket.close(code=1008) # Policy Violation
return
except WebSocketDisconnect:
return
else:
# Local mode keeps the existing first-message bearer protocol.
try:
raw = await websocket.receive_text()
try:
payload = json.loads(raw)
token = payload.get("token", "")
except (json.JSONDecodeError, AttributeError):
token = ""
if not token or not decode_token(token):
await websocket.close(code=1008) # Policy Violation
return
except WebSocketDisconnect:
return
_connections.append(websocket)
try:
while True:
await websocket.receive_text()
except WebSocketDisconnect:
_connections.remove(websocket)
pass
finally:
# Any error (disconnect or otherwise) must release the slot, else the
# dead socket lingers in the broadcast pool.
_drop(websocket)
async def _broadcast(payload: str) -> None:
@@ -41,19 +68,44 @@ async def _broadcast(payload: str) -> None:
try:
await conn.send_text(payload)
except Exception:
_connections.remove(conn)
_drop(conn)
async def broadcast_status(node_id: str, status: str, checked_at: str, response_time_ms: int | None = None) -> None:
async def broadcast_status(
device_id: str,
node_ids: list[str],
status: str,
checked_at: str,
response_time_ms: int | None = None,
) -> None:
"""Report one device's reachability to every canvas drawing it.
A device is checked once, not once per canvas, so the result is addressed by
device and carries the nodes it lights up. ``node_ids`` is empty for a
device that is in the inventory but on no canvas.
"""
await _broadcast(json.dumps({
"type": "status",
"node_id": node_id,
"device_id": device_id,
"node_ids": node_ids,
"status": status,
"checked_at": checked_at,
"response_time_ms": response_time_ms,
}))
async def broadcast_service_status(
device_id: str, node_ids: list[str], services: list[dict[str, object]], checked_at: str
) -> None:
await _broadcast(json.dumps({
"type": "service_status",
"device_id": device_id,
"node_ids": node_ids,
"services": services,
"checked_at": checked_at,
}))
async def broadcast_scan_update(run_id: str, devices_found: int) -> None:
await _broadcast(json.dumps({
"type": "scan_device_found",
+372
View File
@@ -0,0 +1,372 @@
"""FastAPI router for Zigbee2MQTT import."""
import logging
from datetime import datetime, timezone
from typing import Any
from fastapi import APIRouter, BackgroundTasks, Depends, HTTPException
from sqlalchemy import delete as sa_delete
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.api.deps import get_current_user
from app.core.config import settings
from app.core.scheduler import reschedule_zigbee_sync, set_zigbee_sync_enabled
from app.db.database import AsyncSessionLocal, get_db
from app.db.models import InventoryDevice, InventoryDeviceLink, Node, ScanRun
from app.schemas.scan import ScanRunResponse
from app.schemas.zigbee import (
ZigbeeConfig,
ZigbeeCoordinatorOut,
ZigbeeEdgeOut,
ZigbeeImportPendingResponse,
ZigbeeImportRequest,
ZigbeeImportResponse,
ZigbeeNodeOut,
ZigbeeSyncConfig,
ZigbeeTestConnectionRequest,
ZigbeeTestConnectionResponse,
)
from app.services.node_dedupe import dedupe_nodes_by_device
from app.services.zigbee_service import (
build_zigbee_properties,
fetch_networkmap,
merge_zigbee_properties,
test_mqtt_connection,
)
logger = logging.getLogger(__name__)
async def _is_drawn(db: AsyncSession, device_id: str) -> bool:
"""True while at least one canvas node still draws this device."""
return (
await db.execute(select(Node.id).where(Node.device_id == device_id).limit(1))
).scalar_one_or_none() is not None
router = APIRouter()
@router.post("/import", response_model=ZigbeeImportResponse)
async def import_zigbee_network(
payload: ZigbeeImportRequest,
_: str = Depends(get_current_user),
) -> ZigbeeImportResponse:
"""Fetch the Zigbee2MQTT network map and return nodes + edges ready for canvas drop.
Connects to the specified MQTT broker, publishes a networkmap request to
``<base_topic>/bridge/request/networkmap``, and waits up to 60 s for the
response (large meshes can take 30 s+). The devices are returned as typed homelable nodes with a
coordinator → router → end-device hierarchy.
"""
try:
nodes_raw, edges_raw = await fetch_networkmap(
mqtt_host=payload.mqtt_host,
mqtt_port=payload.mqtt_port,
base_topic=payload.base_topic,
username=payload.mqtt_username,
password=payload.mqtt_password,
tls=payload.mqtt_tls,
tls_insecure=payload.mqtt_tls_insecure,
)
except ImportError as exc:
raise HTTPException(status_code=500, detail=str(exc)) from exc
except ConnectionError as exc:
raise HTTPException(status_code=502, detail=str(exc)) from exc
except TimeoutError as exc:
raise HTTPException(status_code=504, detail=str(exc)) from exc
except ValueError as exc:
raise HTTPException(status_code=422, detail=str(exc)) from exc
except Exception as exc:
logger.exception("Unexpected error during Zigbee import")
raise HTTPException(status_code=500, detail="Unexpected error during Zigbee import") from exc
nodes = [ZigbeeNodeOut(**n) for n in nodes_raw]
edges = [ZigbeeEdgeOut(**e) for e in edges_raw]
return ZigbeeImportResponse(nodes=nodes, edges=edges, device_count=len(nodes))
@router.post("/import-pending", response_model=ScanRunResponse)
async def import_zigbee_to_pending(
payload: ZigbeeImportRequest,
background_tasks: BackgroundTasks,
db: AsyncSession = Depends(get_db),
_: str = Depends(get_current_user),
) -> ScanRun:
"""Queue a Zigbee2MQTT pending import as a background scan run.
Returns the ScanRun row immediately so the UI can close the import
modal and surface progress under Scan History (kind=zigbee). The
actual MQTT fetch + pending upsert happens in the background.
"""
run = ScanRun(
status="running",
kind="zigbee",
ranges=[f"{payload.mqtt_host}:{payload.mqtt_port}"],
)
db.add(run)
await db.commit()
await db.refresh(run)
background_tasks.add_task(_background_zigbee_import, run.id, payload)
return run
def env_import_request() -> ZigbeeImportRequest:
"""Build an import request from the server env config (for auto-sync).
MQTT credentials live in the env only — never in the request body or any
API response. The scheduled auto-sync job and ``/sync-now`` both source
their connection settings here so there is a single source of truth."""
return ZigbeeImportRequest(
mqtt_host=settings.zigbee_mqtt_host,
mqtt_port=settings.zigbee_mqtt_port,
mqtt_username=settings.zigbee_mqtt_username or None,
mqtt_password=settings.zigbee_mqtt_password or None,
base_topic=settings.zigbee_base_topic,
mqtt_tls=settings.zigbee_mqtt_tls,
mqtt_tls_insecure=settings.zigbee_mqtt_tls_insecure,
)
@router.post("/sync-now", response_model=ScanRunResponse)
async def sync_zigbee_now(
background_tasks: BackgroundTasks,
db: AsyncSession = Depends(get_db),
_: str = Depends(get_current_user),
) -> ScanRun:
"""Trigger an immediate Zigbee import using the server env config.
Same background flow as ``/import-pending`` but sources the MQTT connection
from ``settings`` (env) rather than the request body — the manual
counterpart to the scheduled auto-sync job. Requires the env host to be set.
"""
if not settings.zigbee_mqtt_host:
raise HTTPException(
status_code=400,
detail="Cannot sync: no Zigbee MQTT host configured on the server.",
)
payload = env_import_request()
run = ScanRun(
status="running",
kind="zigbee",
ranges=[f"{payload.mqtt_host}:{payload.mqtt_port}"],
)
db.add(run)
await db.commit()
await db.refresh(run)
background_tasks.add_task(_background_zigbee_import, run.id, payload)
return run
async def _background_zigbee_import(run_id: str, payload: ZigbeeImportRequest) -> None:
async with AsyncSessionLocal() as db:
try:
nodes_raw, edges_raw = await fetch_networkmap(
mqtt_host=payload.mqtt_host,
mqtt_port=payload.mqtt_port,
base_topic=payload.base_topic,
username=payload.mqtt_username,
password=payload.mqtt_password,
tls=payload.mqtt_tls,
tls_insecure=payload.mqtt_tls_insecure,
)
result = await _persist_pending_import(db, nodes_raw, edges_raw)
run = await db.get(ScanRun, run_id)
if run:
run.status = "done"
run.devices_found = result.device_count
run.finished_at = datetime.now(timezone.utc)
await db.commit()
except Exception as exc:
logger.exception("Zigbee import %s failed", run_id)
await db.rollback()
run = await db.get(ScanRun, run_id)
if run:
run.status = "error"
run.error = str(exc)[:500]
run.finished_at = datetime.now(timezone.utc)
await db.commit()
async def _persist_pending_import(
db: AsyncSession,
nodes_raw: list[dict[str, Any]],
edges_raw: list[dict[str, Any]],
) -> ZigbeeImportPendingResponse:
"""Upsert nodes/edges into device_inventory + device_inventory_links.
Coordinator auto-approves to a canvas Node. Other devices upsert by IEEE.
All zigbee-source links are wiped and re-inserted from the new map.
"""
# Repair any pre-existing duplicate nodes (same IEEE) before upserting, so
# the by-IEEE lookups below resolve to a single row.
await dedupe_nodes_by_device(db)
# Coordinator is no longer auto-placed, so the response's coordinator fields
# stay unset — retained for backward-compatible response shape.
coordinator_out: ZigbeeCoordinatorOut | None = None
coordinator_existed = False
pending_created = 0
pending_updated = 0
for n in nodes_raw:
ieee = n.get("ieee_address")
if not ieee:
continue
props = build_zigbee_properties(
ieee, n.get("vendor"), n.get("model"), n.get("lqi")
)
# The coordinator is no longer auto-placed on the canvas — it flows to
# the pending inventory like every other device, so the user approves it
# explicitly. Only the shared paths below run for it.
# Properties belong to the inventory row now, so one upsert serves
# every canvas drawing this device — there is no per-node refresh left
# to do. The row's status is never touched here: an approved device
# stays approved, a hidden one stays hidden.
result = await db.execute(
select(InventoryDevice).where(InventoryDevice.ieee_address == ieee)
)
pending = result.scalar_one_or_none()
if pending is None:
db.add(
InventoryDevice(
ieee_address=ieee,
friendly_name=n.get("friendly_name"),
hostname=n.get("friendly_name"),
suggested_type=n.get("type"),
device_subtype=n.get("device_type"),
model=n.get("model"),
vendor=n.get("vendor"),
lqi=n.get("lqi"),
properties=props,
status="pending",
discovery_source="zigbee",
)
)
pending_created += 1
else:
pending.friendly_name = n.get("friendly_name") or pending.friendly_name
pending.suggested_type = n.get("type") or pending.suggested_type
pending.device_subtype = n.get("device_type") or pending.device_subtype
pending.model = n.get("model") or pending.model
pending.vendor = n.get("vendor") or pending.vendor
if n.get("lqi") is not None:
pending.lqi = n.get("lqi")
pending.properties = merge_zigbee_properties(list(pending.properties or []), props)
if pending.status == "approved" and not await _is_drawn(db, pending.id):
# Approved earlier but no canvas draws it any more — the node was
# deleted. Revive the row to "pending" so it reappears in the list
# on re-import instead of being silently swallowed. (Issue #167)
pending.status = "pending"
elif pending.status == "hidden":
# Re-imported a hidden device → leave it hidden, just refresh fields.
pass
pending_updated += 1
# Replace all zigbee-source links with the freshly discovered set.
await db.execute(
sa_delete(InventoryDeviceLink).where(InventoryDeviceLink.discovery_source == "zigbee")
)
links_recorded = 0
seen: set[tuple[str, str]] = set()
for e in edges_raw:
src = e.get("source")
tgt = e.get("target")
if not src or not tgt or (src, tgt) in seen:
continue
seen.add((src, tgt))
db.add(
InventoryDeviceLink(
source_ieee=src,
target_ieee=tgt,
discovery_source="zigbee",
)
)
links_recorded += 1
await db.commit()
return ZigbeeImportPendingResponse(
pending_created=pending_created,
pending_updated=pending_updated,
coordinator=coordinator_out,
coordinator_already_existed=coordinator_existed,
links_recorded=links_recorded,
device_count=len(nodes_raw),
)
@router.post("/test-connection", response_model=ZigbeeTestConnectionResponse)
async def test_zigbee_connection(
payload: ZigbeeTestConnectionRequest,
_: str = Depends(get_current_user),
) -> ZigbeeTestConnectionResponse:
"""Quick MQTT ping to validate broker connection before importing."""
try:
await test_mqtt_connection(
mqtt_host=payload.mqtt_host,
mqtt_port=payload.mqtt_port,
username=payload.mqtt_username,
password=payload.mqtt_password,
tls=payload.mqtt_tls,
tls_insecure=payload.mqtt_tls_insecure,
)
return ZigbeeTestConnectionResponse(connected=True, message="Connection successful")
except ImportError as exc:
raise HTTPException(status_code=500, detail=str(exc)) from exc
except (ConnectionError, TimeoutError) as exc:
return ZigbeeTestConnectionResponse(connected=False, message=str(exc))
except Exception:
logger.exception("Unexpected error during connection test")
return ZigbeeTestConnectionResponse(connected=False, message="Unexpected error")
@router.get("/config", response_model=ZigbeeConfig)
async def get_zigbee_config(_: str = Depends(get_current_user)) -> ZigbeeConfig:
"""Return non-secret Zigbee config. Never includes MQTT credentials — only
whether a host is configured on the server for auto-sync."""
return ZigbeeConfig(
mqtt_host=settings.zigbee_mqtt_host,
mqtt_port=settings.zigbee_mqtt_port,
base_topic=settings.zigbee_base_topic,
mqtt_tls=settings.zigbee_mqtt_tls,
sync_enabled=settings.zigbee_sync_enabled,
sync_interval=settings.zigbee_sync_interval,
host_configured=bool(settings.zigbee_mqtt_host),
)
@router.post("/config", response_model=ZigbeeConfig)
async def save_zigbee_config(
payload: ZigbeeSyncConfig,
_: str = Depends(get_current_user),
) -> ZigbeeConfig:
"""Persist the auto-sync activation (enabled + interval) and apply it live.
This is the ONLY Zigbee config the app writes. Connection settings
(host/port/credentials/topic/tls) are env-only and are never accepted or
persisted here — enabling auto-sync requires the MQTT host already set in
the server env, since the scheduled job reads it from there.
"""
if payload.sync_enabled and not settings.zigbee_mqtt_host:
raise HTTPException(
status_code=400,
detail="Cannot enable auto-sync: no Zigbee MQTT host configured in the server env.",
)
try:
settings.zigbee_sync_enabled = payload.sync_enabled
settings.zigbee_sync_interval = payload.sync_interval
settings.save_overrides()
set_zigbee_sync_enabled(payload.sync_enabled)
if payload.sync_enabled:
reschedule_zigbee_sync(payload.sync_interval)
except HTTPException:
raise
except Exception as exc:
raise HTTPException(status_code=500, detail=str(exc)) from exc
return await get_zigbee_config()
+364
View File
@@ -0,0 +1,364 @@
"""FastAPI router for Z-Wave JS UI (zwavejs2mqtt) import."""
import logging
from datetime import datetime, timezone
from typing import Any
from fastapi import APIRouter, BackgroundTasks, Depends, HTTPException
from sqlalchemy import delete as sa_delete
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.api.deps import get_current_user
from app.core.config import settings
from app.core.scheduler import reschedule_zwave_sync, set_zwave_sync_enabled
from app.db.database import AsyncSessionLocal, get_db
from app.db.models import InventoryDevice, InventoryDeviceLink, Node, ScanRun
from app.schemas.scan import ScanRunResponse
from app.schemas.zwave import (
ZwaveConfig,
ZwaveCoordinatorOut,
ZwaveEdgeOut,
ZwaveImportPendingResponse,
ZwaveImportRequest,
ZwaveImportResponse,
ZwaveNodeOut,
ZwaveSyncConfig,
ZwaveTestConnectionRequest,
ZwaveTestConnectionResponse,
)
from app.services.node_dedupe import dedupe_nodes_by_device
from app.services.zwave_service import (
build_zwave_properties,
fetch_zwave_network,
merge_zwave_properties,
test_zwave_connection,
)
logger = logging.getLogger(__name__)
async def _is_drawn(db: AsyncSession, device_id: str) -> bool:
"""True while at least one canvas node still draws this device."""
return (
await db.execute(select(Node.id).where(Node.device_id == device_id).limit(1))
).scalar_one_or_none() is not None
router = APIRouter()
@router.post("/import", response_model=ZwaveImportResponse)
async def import_zwave_network(
payload: ZwaveImportRequest,
_: str = Depends(get_current_user),
) -> ZwaveImportResponse:
"""Fetch the Z-Wave node list and return nodes + edges ready for canvas drop.
Connects to the broker, publishes a ``getNodes`` request to the Z-Wave JS UI
gateway, and waits for the response. Devices are returned as typed homelable
nodes with a coordinator → router → end-device hierarchy.
"""
try:
nodes_raw, edges_raw = await fetch_zwave_network(
mqtt_host=payload.mqtt_host,
mqtt_port=payload.mqtt_port,
prefix=payload.prefix,
gateway_name=payload.gateway_name,
username=payload.mqtt_username,
password=payload.mqtt_password,
tls=payload.mqtt_tls,
tls_insecure=payload.mqtt_tls_insecure,
)
except ImportError as exc:
raise HTTPException(status_code=500, detail=str(exc)) from exc
except ConnectionError as exc:
raise HTTPException(status_code=502, detail=str(exc)) from exc
except TimeoutError as exc:
raise HTTPException(status_code=504, detail=str(exc)) from exc
except ValueError as exc:
raise HTTPException(status_code=422, detail=str(exc)) from exc
except Exception as exc:
logger.exception("Unexpected error during Z-Wave import")
raise HTTPException(status_code=500, detail="Unexpected error during Z-Wave import") from exc
nodes = [ZwaveNodeOut(**n) for n in nodes_raw]
edges = [ZwaveEdgeOut(**e) for e in edges_raw]
return ZwaveImportResponse(nodes=nodes, edges=edges, device_count=len(nodes))
@router.post("/import-pending", response_model=ScanRunResponse)
async def import_zwave_to_pending(
payload: ZwaveImportRequest,
background_tasks: BackgroundTasks,
db: AsyncSession = Depends(get_db),
_: str = Depends(get_current_user),
) -> ScanRun:
"""Queue a Z-Wave pending import as a background scan run (kind=zwave)."""
run = ScanRun(
status="running",
kind="zwave",
ranges=[f"{payload.mqtt_host}:{payload.mqtt_port}"],
)
db.add(run)
await db.commit()
await db.refresh(run)
background_tasks.add_task(_background_zwave_import, run.id, payload)
return run
def env_import_request() -> ZwaveImportRequest:
"""Build an import request from the server env config (for auto-sync).
MQTT credentials live in the env only — never in the request body or any
API response. The scheduled auto-sync job and ``/sync-now`` both source
their connection settings here so there is a single source of truth."""
return ZwaveImportRequest(
mqtt_host=settings.zwave_mqtt_host,
mqtt_port=settings.zwave_mqtt_port,
mqtt_username=settings.zwave_mqtt_username or None,
mqtt_password=settings.zwave_mqtt_password or None,
prefix=settings.zwave_prefix,
gateway_name=settings.zwave_gateway_name,
mqtt_tls=settings.zwave_mqtt_tls,
mqtt_tls_insecure=settings.zwave_mqtt_tls_insecure,
)
@router.post("/sync-now", response_model=ScanRunResponse)
async def sync_zwave_now(
background_tasks: BackgroundTasks,
db: AsyncSession = Depends(get_db),
_: str = Depends(get_current_user),
) -> ScanRun:
"""Trigger an immediate Z-Wave import using the server env config.
Same background flow as ``/import-pending`` but sources the MQTT connection
from ``settings`` (env) rather than the request body — the manual
counterpart to the scheduled auto-sync job. Requires the env host to be set.
"""
if not settings.zwave_mqtt_host:
raise HTTPException(
status_code=400,
detail="Cannot sync: no Z-Wave MQTT host configured on the server.",
)
payload = env_import_request()
run = ScanRun(
status="running",
kind="zwave",
ranges=[f"{payload.mqtt_host}:{payload.mqtt_port}"],
)
db.add(run)
await db.commit()
await db.refresh(run)
background_tasks.add_task(_background_zwave_import, run.id, payload)
return run
async def _background_zwave_import(run_id: str, payload: ZwaveImportRequest) -> None:
async with AsyncSessionLocal() as db:
try:
nodes_raw, edges_raw = await fetch_zwave_network(
mqtt_host=payload.mqtt_host,
mqtt_port=payload.mqtt_port,
prefix=payload.prefix,
gateway_name=payload.gateway_name,
username=payload.mqtt_username,
password=payload.mqtt_password,
tls=payload.mqtt_tls,
tls_insecure=payload.mqtt_tls_insecure,
)
result = await _persist_pending_import(db, nodes_raw, edges_raw)
run = await db.get(ScanRun, run_id)
if run:
run.status = "done"
run.devices_found = result.device_count
run.finished_at = datetime.now(timezone.utc)
await db.commit()
except Exception as exc:
logger.exception("Z-Wave import %s failed", run_id)
await db.rollback()
run = await db.get(ScanRun, run_id)
if run:
run.status = "error"
run.error = str(exc)[:500]
run.finished_at = datetime.now(timezone.utc)
await db.commit()
async def _persist_pending_import(
db: AsyncSession,
nodes_raw: list[dict[str, Any]],
edges_raw: list[dict[str, Any]],
) -> ZwaveImportPendingResponse:
"""Upsert nodes/edges into device_inventory + device_inventory_links.
Coordinator auto-approves to a canvas Node. Other devices upsert by Z-Wave
identity. All zwave-source links are wiped and re-inserted from the new map.
"""
# Repair any pre-existing same-canvas duplicate nodes before upserting, so
# the by-IEEE lookups below resolve cleanly.
await dedupe_nodes_by_device(db)
# Coordinator is no longer auto-placed, so the response's coordinator fields
# stay unset — retained for backward-compatible response shape.
coordinator_out: ZwaveCoordinatorOut | None = None
coordinator_existed = False
pending_created = 0
pending_updated = 0
for n in nodes_raw:
ieee = n.get("ieee_address")
if not ieee:
continue
props = build_zwave_properties(ieee, n.get("vendor"), n.get("model"))
# The coordinator is no longer auto-placed on the canvas — it flows to
# the pending inventory like every other device, so the user approves it
# explicitly. Only the shared paths below run for it.
# Properties belong to the inventory row now, so one upsert serves
# every canvas drawing this device — there is no per-node refresh left
# to do. The row's status is never touched here: an approved device
# stays approved, a hidden one stays hidden.
result = await db.execute(
select(InventoryDevice).where(InventoryDevice.ieee_address == ieee)
)
pending = result.scalar_one_or_none()
if pending is None:
db.add(
InventoryDevice(
ieee_address=ieee,
friendly_name=n.get("friendly_name"),
hostname=n.get("friendly_name"),
suggested_type=n.get("type"),
device_subtype=n.get("device_type"),
model=n.get("model"),
vendor=n.get("vendor"),
lqi=n.get("lqi"),
properties=props,
status="pending",
discovery_source="zwave",
)
)
pending_created += 1
else:
pending.friendly_name = n.get("friendly_name") or pending.friendly_name
pending.suggested_type = n.get("type") or pending.suggested_type
pending.device_subtype = n.get("device_type") or pending.device_subtype
pending.model = n.get("model") or pending.model
pending.vendor = n.get("vendor") or pending.vendor
pending.properties = merge_zwave_properties(list(pending.properties or []), props)
if pending.status == "approved" and not await _is_drawn(db, pending.id):
# Approved earlier but no canvas draws it any more (the node was
# deleted) — revive to "pending" so it reappears in the list.
pending.status = "pending"
elif pending.status == "hidden":
pass
pending_updated += 1
# Replace all zwave-source links with the freshly discovered set.
await db.execute(
sa_delete(InventoryDeviceLink).where(InventoryDeviceLink.discovery_source == "zwave")
)
links_recorded = 0
seen: set[tuple[str, str]] = set()
for e in edges_raw:
src = e.get("source")
tgt = e.get("target")
if not src or not tgt or (src, tgt) in seen:
continue
seen.add((src, tgt))
db.add(
InventoryDeviceLink(
source_ieee=src,
target_ieee=tgt,
discovery_source="zwave",
)
)
links_recorded += 1
await db.commit()
return ZwaveImportPendingResponse(
pending_created=pending_created,
pending_updated=pending_updated,
coordinator=coordinator_out,
coordinator_already_existed=coordinator_existed,
links_recorded=links_recorded,
device_count=len(nodes_raw),
)
@router.post("/test-connection", response_model=ZwaveTestConnectionResponse)
async def test_connection_endpoint(
payload: ZwaveTestConnectionRequest,
_: str = Depends(get_current_user),
) -> ZwaveTestConnectionResponse:
"""Quick MQTT ping to validate broker connection before importing."""
try:
await test_zwave_connection(
mqtt_host=payload.mqtt_host,
mqtt_port=payload.mqtt_port,
username=payload.mqtt_username,
password=payload.mqtt_password,
tls=payload.mqtt_tls,
tls_insecure=payload.mqtt_tls_insecure,
)
return ZwaveTestConnectionResponse(connected=True, message="Connection successful")
except ImportError as exc:
raise HTTPException(status_code=500, detail=str(exc)) from exc
except (ConnectionError, TimeoutError) as exc:
return ZwaveTestConnectionResponse(connected=False, message=str(exc))
except Exception:
logger.exception("Unexpected error during connection test")
return ZwaveTestConnectionResponse(connected=False, message="Unexpected error")
@router.get("/config", response_model=ZwaveConfig)
async def get_zwave_config(_: str = Depends(get_current_user)) -> ZwaveConfig:
"""Return non-secret Z-Wave config. Never includes MQTT credentials — only
whether a host is configured on the server for auto-sync."""
return ZwaveConfig(
mqtt_host=settings.zwave_mqtt_host,
mqtt_port=settings.zwave_mqtt_port,
prefix=settings.zwave_prefix,
gateway_name=settings.zwave_gateway_name,
mqtt_tls=settings.zwave_mqtt_tls,
sync_enabled=settings.zwave_sync_enabled,
sync_interval=settings.zwave_sync_interval,
host_configured=bool(settings.zwave_mqtt_host),
)
@router.post("/config", response_model=ZwaveConfig)
async def save_zwave_config(
payload: ZwaveSyncConfig,
_: str = Depends(get_current_user),
) -> ZwaveConfig:
"""Persist the auto-sync activation (enabled + interval) and apply it live.
This is the ONLY Z-Wave config the app writes. Connection settings
(host/port/credentials/prefix/gateway/tls) are env-only and are never
accepted or persisted here — enabling auto-sync requires the MQTT host
already set in the server env, since the scheduled job reads it from there.
"""
if payload.sync_enabled and not settings.zwave_mqtt_host:
raise HTTPException(
status_code=400,
detail="Cannot enable auto-sync: no Z-Wave MQTT host configured in the server env.",
)
try:
settings.zwave_sync_enabled = payload.sync_enabled
settings.zwave_sync_interval = payload.sync_interval
settings.save_overrides()
set_zwave_sync_enabled(payload.sync_enabled)
if payload.sync_enabled:
reschedule_zwave_sync(payload.sync_interval)
except HTTPException:
raise
except Exception as exc:
raise HTTPException(status_code=500, detail=str(exc)) from exc
return await get_zwave_config()
+196 -2
View File
@@ -1,18 +1,43 @@
import json
import logging
from pathlib import Path
from typing import Literal
from urllib.parse import urlsplit
from pydantic import model_validator
from pydantic import Field, model_validator
from pydantic_settings import BaseSettings, SettingsConfigDict
logger = logging.getLogger(__name__)
def origin_of(url: str) -> str:
"""`scheme://host[:port]` of an absolute URL, or "" when it has none."""
parts = urlsplit(url)
if not parts.scheme or not parts.netloc:
return ""
return f"{parts.scheme}://{parts.netloc}"
def _read_version() -> str:
for candidate in [
Path(__file__).parent.parent.parent.parent / "VERSION", # repo root (dev)
Path("/app/VERSION"), # Docker image
]:
if candidate.exists():
return candidate.read_text().strip()
return "unknown"
APP_VERSION = _read_version()
class Settings(BaseSettings):
model_config = SettingsConfigDict(env_file=".env", env_file_encoding="utf-8")
secret_key: str # Required — set SECRET_KEY in .env
sqlite_path: str = "./data/homelab.db"
# Uploaded media (floor plans, and future raw image uploads) live on disk,
# not in the DB. Defaults to a `uploads/` folder next to the SQLite DB so it
# sits on the same persistent Docker volume. Override with UPLOAD_DIR.
upload_dir: str = ""
cors_origins: list[str] = ["http://localhost:5173", "http://localhost:3000"]
# JWT
@@ -20,11 +45,22 @@ class Settings(BaseSettings):
access_token_expire_minutes: int = 1440 # 24h
# Auth — set AUTH_USERNAME and AUTH_PASSWORD_HASH in .env
auth_mode: Literal["local", "oidc"] = "local"
auth_username: str = "admin"
auth_password_hash: str = ""
# OpenID Connect — required only when AUTH_MODE=oidc.
oidc_discovery_url: str = ""
oidc_client_id: str = ""
oidc_client_secret: str = ""
oidc_redirect_uri: str = ""
oidc_scopes: str = "openid profile email"
oidc_cookie_secure: bool = True
oidc_session_expire_minutes: int = Field(default=480, ge=5, le=1440)
oidc_transaction_expire_seconds: int = Field(default=600, ge=60, le=3600)
@model_validator(mode="after")
def check_password_hash(self) -> "Settings":
def validate_auth_settings(self) -> "Settings":
h = self.auth_password_hash
if h and not h.startswith("$2"):
logger.error(
@@ -32,14 +68,76 @@ class Settings(BaseSettings):
"bcrypt hashes contain '$' signs — wrap the value in single quotes "
"in your .env file: AUTH_PASSWORD_HASH='$2b$12$...'"
)
if self.auth_mode == "oidc":
required = {
"OIDC_DISCOVERY_URL": self.oidc_discovery_url,
"OIDC_CLIENT_ID": self.oidc_client_id,
"OIDC_CLIENT_SECRET": self.oidc_client_secret,
"OIDC_REDIRECT_URI": self.oidc_redirect_uri,
}
missing = [name for name, value in required.items() if not value]
if missing:
raise ValueError(f"AUTH_MODE=oidc requires: {', '.join(missing)}")
if len(self.secret_key.encode("utf-8")) < 32:
raise ValueError("SECRET_KEY must be at least 32 bytes when AUTH_MODE=oidc")
if "openid" not in self.oidc_scopes.split():
raise ValueError("OIDC_SCOPES must contain 'openid'")
for name, value in {
"OIDC_DISCOVERY_URL": self.oidc_discovery_url,
"OIDC_REDIRECT_URI": self.oidc_redirect_uri,
}.items():
if not value.startswith(("https://", "http://")):
raise ValueError(f"{name} must be an HTTP(S) URL")
if self.oidc_cookie_secure:
for name, value in {
"OIDC_DISCOVERY_URL": self.oidc_discovery_url,
"OIDC_REDIRECT_URI": self.oidc_redirect_uri,
}.items():
if not value.startswith("https://"):
raise ValueError(f"{name} must use HTTPS when OIDC_COOKIE_SECURE=true")
if "*" in self.cors_origins:
raise ValueError("CORS_ORIGINS cannot contain '*' when AUTH_MODE=oidc")
app_origin = origin_of(self.oidc_redirect_uri)
allowed = {value.rstrip("/") for value in self.cors_origins}
if app_origin and app_origin not in allowed:
logger.warning(
"CORS_ORIGINS does not list %s (the origin of OIDC_REDIRECT_URI). "
"The CSRF check accepts that origin anyway, but add it to CORS_ORIGINS "
"so browser requests are not rejected: CORS_ORIGINS=[\"%s\"]",
app_origin,
app_origin,
)
return self
# Scanner
scanner_ranges: list[str] = ["192.168.1.0/24"]
# Phase-2 version-detection (-sV) host timeout, seconds. Bounds the version
# pass so a stalling TLS port (e.g. Proxmox 8006) can't hang it. Discovered
# ports survive a timeout regardless; raise this on slow/overlay networks.
scanner_version_host_timeout: int = 60
# Per-device deep rescan: total time budget in seconds, checked between port
# slices. Unprivileged nmap falls back to a connect scan (-sT), and a host
# that drops packets makes every filtered port wait out its RTT — the whole
# range against such a host runs ~20-45 min. When the budget is spent the
# run keeps what it found and says how much it did not reach. This is NOT an
# nmap --host-timeout: that one discards a host's results wholesale.
scanner_deep_host_timeout: int = 2700
# Deep scan — persisted defaults (overridable per-scan from the scan dialog).
# http_ranges: extra nmap port ranges, opt-in, no default. Probe + TLS off by default.
scanner_http_ranges: list[str] = []
scanner_http_probe_enabled: bool = False
scanner_http_verify_tls: bool = False
# Status checker
status_checker_interval: int = 60
# Per-service status checker (independent of node checks). Off by default.
service_check_enabled: bool = False
service_check_interval: int = 300
# MCP service key — set MCP_SERVICE_KEY in .env
# Used by the MCP server to authenticate against the backend without a user password.
# Leave empty to disable MCP service key auth.
@@ -50,9 +148,60 @@ class Settings(BaseSettings):
# Leave unset (or empty) to keep the feature disabled (default).
liveview_key: str | None = None
# Homepage widget — optional read-only stats endpoint for gethomepage.
# Set to a random secret to enable /api/v1/stats/summary (X-API-Key header).
# Leave empty to keep the feature disabled (default).
homepage_api_key: str = ""
# Proxmox VE import.
# Token = a real credential → env/.env ONLY, never persisted by the app to
# scan_config.json and never returned by the API. token_id is
# 'user@realm!tokenname'; use a read-only PVEAuditor role.
proxmox_token_id: str = ""
proxmox_token_secret: str = ""
# Non-secret connection + auto-sync config (persisted via save_overrides).
proxmox_host: str = ""
proxmox_port: int = 8006
proxmox_verify_tls: bool = True
proxmox_sync_enabled: bool = False
proxmox_sync_interval: int = 3600 # seconds (floor 300 enforced on write)
# Zigbee2MQTT auto-sync import.
# MQTT credentials are secrets → env/.env ONLY, never persisted by the app to
# scan_config.json and never returned by the API. Only the auto-sync
# activation (enabled + interval) is persisted; connection config is env-only.
zigbee_mqtt_host: str = ""
zigbee_mqtt_port: int = 1883
zigbee_mqtt_username: str = ""
zigbee_mqtt_password: str = ""
zigbee_base_topic: str = "zigbee2mqtt"
zigbee_mqtt_tls: bool = False
zigbee_mqtt_tls_insecure: bool = False
zigbee_sync_enabled: bool = False
zigbee_sync_interval: int = 3600 # seconds (floor 300 enforced on write)
# Z-Wave JS UI (zwavejs2mqtt) auto-sync import. Same secret/env rules.
zwave_mqtt_host: str = ""
zwave_mqtt_port: int = 1883
zwave_mqtt_username: str = ""
zwave_mqtt_password: str = ""
zwave_prefix: str = "zwave"
zwave_gateway_name: str = "zwavejs2mqtt"
zwave_mqtt_tls: bool = False
zwave_mqtt_tls_insecure: bool = False
zwave_sync_enabled: bool = False
zwave_sync_interval: int = 3600 # seconds (floor 300 enforced on write)
def _override_path(self) -> Path:
return Path(self.sqlite_path).parent / "scan_config.json"
def media_dir(self) -> Path:
"""On-disk folder for uploaded media. Sits on the same persistent
volume as the SQLite DB unless UPLOAD_DIR is set."""
if self.upload_dir:
return Path(self.upload_dir)
return Path(self.sqlite_path).parent / "uploads"
def load_overrides(self) -> None:
"""Load runtime scan config overrides written by the API."""
try:
@@ -61,6 +210,35 @@ class Settings(BaseSettings):
self.scanner_ranges = data["scanner_ranges"]
if "status_checker_interval" in data:
self.status_checker_interval = int(data["status_checker_interval"])
if "service_check_enabled" in data:
self.service_check_enabled = bool(data["service_check_enabled"])
if "service_check_interval" in data:
self.service_check_interval = int(data["service_check_interval"])
if "scanner_http_ranges" in data:
self.scanner_http_ranges = list(data["scanner_http_ranges"])
if "scanner_http_probe_enabled" in data:
self.scanner_http_probe_enabled = bool(data["scanner_http_probe_enabled"])
if "scanner_http_verify_tls" in data:
self.scanner_http_verify_tls = bool(data["scanner_http_verify_tls"])
# Proxmox auto-sync activation only. Connection config (host, port,
# token, verify_tls) is env-only by design — never read from or
# written to this file. Persisting host here previously created a
# dual source of truth that silently clobbered PROXMOX_HOST.
if "proxmox_sync_enabled" in data:
self.proxmox_sync_enabled = bool(data["proxmox_sync_enabled"])
if "proxmox_sync_interval" in data:
self.proxmox_sync_interval = int(data["proxmox_sync_interval"])
# Zigbee/Z-Wave: only the auto-sync activation is persisted. MQTT
# connection config (host, port, credentials, topic, tls) is env-only
# by design — never read from or written to this file.
if "zigbee_sync_enabled" in data:
self.zigbee_sync_enabled = bool(data["zigbee_sync_enabled"])
if "zigbee_sync_interval" in data:
self.zigbee_sync_interval = int(data["zigbee_sync_interval"])
if "zwave_sync_enabled" in data:
self.zwave_sync_enabled = bool(data["zwave_sync_enabled"])
if "zwave_sync_interval" in data:
self.zwave_sync_interval = int(data["zwave_sync_interval"])
except Exception:
pass
@@ -70,6 +248,22 @@ class Settings(BaseSettings):
self._override_path().write_text(json.dumps({
"scanner_ranges": self.scanner_ranges,
"status_checker_interval": self.status_checker_interval,
"service_check_enabled": self.service_check_enabled,
"service_check_interval": self.service_check_interval,
"scanner_http_ranges": self.scanner_http_ranges,
"scanner_http_probe_enabled": self.scanner_http_probe_enabled,
"scanner_http_verify_tls": self.scanner_http_verify_tls,
# Proxmox: only the auto-sync activation is persisted. Connection
# config (host, port, token, verify_tls) is env-only and must never
# be written to disk — that is the single source of truth.
"proxmox_sync_enabled": self.proxmox_sync_enabled,
"proxmox_sync_interval": self.proxmox_sync_interval,
# Zigbee/Z-Wave: only the auto-sync activation is persisted. MQTT
# connection config (host/port/credentials/topic/tls) is env-only.
"zigbee_sync_enabled": self.zigbee_sync_enabled,
"zigbee_sync_interval": self.zigbee_sync_interval,
"zwave_sync_enabled": self.zwave_sync_enabled,
"zwave_sync_interval": self.zwave_sync_interval,
}))
+389 -26
View File
@@ -1,62 +1,325 @@
"""APScheduler setup for background scan and status check jobs."""
import asyncio
import logging
from collections.abc import Awaitable, Callable
from datetime import datetime, timezone
from typing import TYPE_CHECKING, Any
from apscheduler.schedulers.asyncio import AsyncIOScheduler
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.core.config import settings
from app.db.database import AsyncSessionLocal
from app.db.models import Node
from app.services.status_checker import check_node
from app.db.models import InventoryDevice, Node
from app.services.status_checker import check_node, check_services
if TYPE_CHECKING:
from app.schemas.zigbee import ZigbeeImportRequest
from app.schemas.zwave import ZwaveImportRequest
logger = logging.getLogger(__name__)
scheduler: AsyncIOScheduler = AsyncIOScheduler()
async def _run_status_checks() -> None:
"""Check all nodes and broadcast results via WebSocket."""
async def _nodes_by_device(db: AsyncSession) -> dict[str, list[str]]:
"""Map each device id to the nodes drawing it, across every canvas."""
rows = (await db.execute(select(Node.id, Node.device_id).where(Node.device_id.is_not(None)))).all()
out: dict[str, list[str]] = {}
for node_id, device_id in rows:
out.setdefault(device_id, []).append(node_id)
return out
async def _check_single_device(
device_id: str,
node_ids: list[str],
check_method: str,
check_target: str | None,
ip: str | None,
) -> tuple[str, dict[str, object] | None]:
"""Run a single device check; returns (device_id, result_or_None).
Accepts plain scalars — not an ORM object — so there is no risk of
DetachedInstanceError when the originating session has already closed.
"""
from app.api.routes.status import broadcast_status # avoid circular import
async with AsyncSessionLocal() as db:
result = await db.execute(select(Node))
nodes = result.scalars().all()
try:
check_result = await check_node(check_method, check_target, ip)
raw_ms = check_result["response_time_ms"]
response_ms = raw_ms if isinstance(raw_ms, int) else None
now = datetime.now(timezone.utc)
async with AsyncSessionLocal() as db:
device = await db.get(InventoryDevice, device_id)
if device:
device.status_live = str(check_result["status"])
device.response_time_ms = response_ms
if check_result["status"] == "online":
device.last_seen = now
await db.commit()
await broadcast_status(
device_id=device_id,
node_ids=node_ids,
status=str(check_result["status"]),
checked_at=now.isoformat(),
response_time_ms=response_ms,
)
return device_id, check_result
except Exception as exc:
logger.error("Status check failed for device %s: %s", device_id, exc)
return device_id, None
for node in nodes:
if not node.check_method:
continue
async def _run_status_checks() -> None:
"""Check every device once and broadcast the result to every canvas.
Device-scoped, not node-scoped: a host drawn on three canvases used to be
pinged three times and could report three different states. Hidden devices
are skipped — hiding one is how a user says "stop showing me this".
A device with no node at all is still checked, deliberately. The inventory
row owns the device and shows its live status in the list and the detail
modal, so a rack mount or an inventory-only entry reports state without
being drawn anywhere — and deleting a node no longer silently stops
monitoring the host. `hide` (or clearing the check method) is what ends the
checks; the broadcast simply carries an empty `node_ids`.
"""
async with AsyncSessionLocal() as db:
devices = (
await db.execute(
select(InventoryDevice).where(
InventoryDevice.check_method.is_not(None),
InventoryDevice.check_method != "",
InventoryDevice.status != "hidden",
)
)
).scalars().all()
node_map = await _nodes_by_device(db)
# Extract scalars while the session is open to avoid DetachedInstanceError
checkable = [
(d.id, node_map.get(d.id, []), d.check_method or "ping", d.check_target, d.ip)
for d in devices
]
if not checkable:
return
await asyncio.gather(*[
_check_single_device(device_id, node_ids, method, target, ip)
for device_id, node_ids, method, target, ip in checkable
])
def _node_host(ip: str | None, hostname: str | None) -> str | None:
"""Pick the address to probe services on: first IP, else hostname."""
if ip:
first = ip.split(",")[0].strip()
if first:
return first
return hostname or None
async def _run_service_checks() -> None:
"""Check every service of every device and broadcast per-service results.
Device-scoped for the same reason as the status check: the services belong
to the device, so one pass serves every canvas showing it.
"""
if not settings.service_check_enabled:
return
from app.api.routes.status import broadcast_service_status # avoid circular import
async with AsyncSessionLocal() as db:
devices = (
await db.execute(select(InventoryDevice).where(InventoryDevice.status != "hidden"))
).scalars().all()
node_map = await _nodes_by_device(db)
checkable = [
(d.id, node_map.get(d.id, []), _node_host(d.ip, d.hostname), list(d.services or []))
for d in devices
if d.services
]
now = datetime.now(timezone.utc).isoformat()
for device_id, node_ids, host, services in checkable:
try:
check_result = await check_node(node.check_method, node.check_target, node.ip)
async with AsyncSessionLocal() as db:
n = await db.get(Node, node.id)
if n:
n.status = check_result["status"]
n.response_time_ms = check_result["response_time_ms"]
n.last_seen = datetime.now(timezone.utc) if check_result["status"] == "online" else n.last_seen
await db.commit()
await broadcast_status(
node_id=node.id,
status=check_result["status"],
checked_at=datetime.now(timezone.utc).isoformat(),
response_time_ms=check_result["response_time_ms"],
statuses = await check_services(host, services)
await broadcast_service_status(
device_id=device_id, node_ids=node_ids, services=statuses, checked_at=now
)
except Exception as exc:
logger.error("Status check failed for node %s: %s", node.id, exc)
logger.error("Service checks failed for device %s: %s", device_id, exc)
async def _run_proxmox_sync() -> None:
"""Fetch the Proxmox inventory and upsert it into pending (auto-sync).
Records a ScanRun (kind=proxmox) so the scheduled sync shows in Scan
history, exactly like the manual /sync-now and /import-pending paths.
"""
if not settings.proxmox_sync_enabled:
return
if not (settings.proxmox_host and settings.proxmox_token_id and settings.proxmox_token_secret):
logger.warning("Proxmox auto-sync enabled but host/token not configured — skipping")
return
# Lazy import to avoid a circular import at module load.
from app.api.routes.proxmox import _background_proxmox_import
from app.db.models import ScanRun
async with AsyncSessionLocal() as db:
run = ScanRun(
status="running",
kind="proxmox",
ranges=[f"{settings.proxmox_host}:{settings.proxmox_port}"],
)
db.add(run)
await db.commit()
await db.refresh(run)
run_id = run.id
# Shares the manual-sync flow: fetch + persist + mark the run done/error +
# broadcast the inventory-reload signal.
await _background_proxmox_import(
run_id,
settings.proxmox_host,
settings.proxmox_port,
settings.proxmox_token_id,
settings.proxmox_token_secret,
settings.proxmox_verify_tls,
)
async def _run_mesh_sync(kind: str) -> None:
"""Shared auto-sync for the MQTT mesh imports (Zigbee / Z-Wave).
Records a ScanRun (kind=zigbee|zwave) so the scheduled sync shows in Scan
history, then delegates to the exact same background import the manual
/sync-now and /import-pending paths use. Connection config comes from the
server env only (credentials never leave it).
"""
from app.db.models import ScanRun
payload: ZigbeeImportRequest | ZwaveImportRequest
background: Callable[[str, Any], Awaitable[None]]
if kind == "zigbee":
if not settings.zigbee_sync_enabled:
return
if not settings.zigbee_mqtt_host:
logger.warning("Zigbee auto-sync enabled but MQTT host not configured — skipping")
return
from app.api.routes.zigbee import _background_zigbee_import
from app.api.routes.zigbee import env_import_request as _zigbee_env_request
host, port = settings.zigbee_mqtt_host, settings.zigbee_mqtt_port
payload = _zigbee_env_request()
background = _background_zigbee_import
else:
if not settings.zwave_sync_enabled:
return
if not settings.zwave_mqtt_host:
logger.warning("Z-Wave auto-sync enabled but MQTT host not configured — skipping")
return
from app.api.routes.zwave import _background_zwave_import
from app.api.routes.zwave import env_import_request as _zwave_env_request
host, port = settings.zwave_mqtt_host, settings.zwave_mqtt_port
payload = _zwave_env_request()
background = _background_zwave_import
async with AsyncSessionLocal() as db:
run = ScanRun(status="running", kind=kind, ranges=[f"{host}:{port}"])
db.add(run)
await db.commit()
await db.refresh(run)
run_id = run.id
await background(run_id, payload)
async def _run_zigbee_sync() -> None:
await _run_mesh_sync("zigbee")
async def _run_zwave_sync() -> None:
await _run_mesh_sync("zwave")
def _add_service_check_job() -> None:
scheduler.add_job(
_run_service_checks,
"interval",
seconds=settings.service_check_interval,
id="service_checks",
max_instances=1,
coalesce=True,
)
def _add_proxmox_sync_job() -> None:
scheduler.add_job(
_run_proxmox_sync,
"interval",
seconds=settings.proxmox_sync_interval,
id="proxmox_sync",
max_instances=1,
coalesce=True,
)
def _add_zigbee_sync_job() -> None:
scheduler.add_job(
_run_zigbee_sync,
"interval",
seconds=settings.zigbee_sync_interval,
id="zigbee_sync",
max_instances=1,
coalesce=True,
)
def _add_zwave_sync_job() -> None:
scheduler.add_job(
_run_zwave_sync,
"interval",
seconds=settings.zwave_sync_interval,
id="zwave_sync",
max_instances=1,
coalesce=True,
)
def start_scheduler() -> None:
global scheduler
if scheduler.running:
scheduler.shutdown(wait=False)
try:
scheduler.shutdown(wait=False)
except Exception as exc:
logger.warning("Failed to shut down previous scheduler instance: %s", exc)
scheduler = AsyncIOScheduler()
scheduler.add_job(_run_status_checks, "interval", seconds=settings.status_checker_interval, id="status_checks")
scheduler.add_job(
_run_status_checks,
"interval",
seconds=settings.status_checker_interval,
id="status_checks",
max_instances=1,
coalesce=True,
)
if settings.service_check_enabled:
_add_service_check_job()
if settings.proxmox_sync_enabled:
_add_proxmox_sync_job()
if settings.zigbee_sync_enabled:
_add_zigbee_sync_job()
if settings.zwave_sync_enabled:
_add_zwave_sync_job()
scheduler.start()
logger.info("Scheduler started — status checks every %ds", settings.status_checker_interval)
def reschedule_status_checks(interval_seconds: int) -> None:
"""Update the status check interval on the running scheduler."""
if interval_seconds < 10:
raise ValueError(f"interval_seconds must be >= 10, got {interval_seconds}")
if not scheduler.running:
logger.warning("Scheduler not running, skipping reschedule")
return
@@ -64,6 +327,106 @@ def reschedule_status_checks(interval_seconds: int) -> None:
logger.info("Status checks rescheduled to every %ds", interval_seconds)
def reschedule_service_checks(interval_seconds: int) -> None:
"""Update the service-check interval on the running scheduler (if enabled)."""
if interval_seconds < 30:
raise ValueError(f"interval_seconds must be >= 30, got {interval_seconds}")
if not scheduler.running:
logger.warning("Scheduler not running, skipping reschedule")
return
if scheduler.get_job("service_checks"):
scheduler.reschedule_job("service_checks", trigger="interval", seconds=interval_seconds)
logger.info("Service checks rescheduled to every %ds", interval_seconds)
def set_service_checks_enabled(enabled: bool) -> None:
"""Add or remove the service-check job on the running scheduler."""
if not scheduler.running:
return
job = scheduler.get_job("service_checks")
if enabled and not job:
_add_service_check_job()
logger.info("Service checks enabled — every %ds", settings.service_check_interval)
elif not enabled and job:
scheduler.remove_job("service_checks")
logger.info("Service checks disabled")
def reschedule_proxmox_sync(interval_seconds: int) -> None:
"""Update the Proxmox auto-sync interval on the running scheduler (if enabled)."""
if interval_seconds < 300:
raise ValueError(f"interval_seconds must be >= 300, got {interval_seconds}")
if not scheduler.running:
logger.warning("Scheduler not running, skipping reschedule")
return
if scheduler.get_job("proxmox_sync"):
scheduler.reschedule_job("proxmox_sync", trigger="interval", seconds=interval_seconds)
logger.info("Proxmox auto-sync rescheduled to every %ds", interval_seconds)
def set_proxmox_sync_enabled(enabled: bool) -> None:
"""Add or remove the Proxmox auto-sync job on the running scheduler."""
if not scheduler.running:
return
job = scheduler.get_job("proxmox_sync")
if enabled and not job:
_add_proxmox_sync_job()
logger.info("Proxmox auto-sync enabled — every %ds", settings.proxmox_sync_interval)
elif not enabled and job:
scheduler.remove_job("proxmox_sync")
logger.info("Proxmox auto-sync disabled")
def reschedule_zigbee_sync(interval_seconds: int) -> None:
"""Update the Zigbee auto-sync interval on the running scheduler (if enabled)."""
if interval_seconds < 300:
raise ValueError(f"interval_seconds must be >= 300, got {interval_seconds}")
if not scheduler.running:
logger.warning("Scheduler not running, skipping reschedule")
return
if scheduler.get_job("zigbee_sync"):
scheduler.reschedule_job("zigbee_sync", trigger="interval", seconds=interval_seconds)
logger.info("Zigbee auto-sync rescheduled to every %ds", interval_seconds)
def set_zigbee_sync_enabled(enabled: bool) -> None:
"""Add or remove the Zigbee auto-sync job on the running scheduler."""
if not scheduler.running:
return
job = scheduler.get_job("zigbee_sync")
if enabled and not job:
_add_zigbee_sync_job()
logger.info("Zigbee auto-sync enabled — every %ds", settings.zigbee_sync_interval)
elif not enabled and job:
scheduler.remove_job("zigbee_sync")
logger.info("Zigbee auto-sync disabled")
def reschedule_zwave_sync(interval_seconds: int) -> None:
"""Update the Z-Wave auto-sync interval on the running scheduler (if enabled)."""
if interval_seconds < 300:
raise ValueError(f"interval_seconds must be >= 300, got {interval_seconds}")
if not scheduler.running:
logger.warning("Scheduler not running, skipping reschedule")
return
if scheduler.get_job("zwave_sync"):
scheduler.reschedule_job("zwave_sync", trigger="interval", seconds=interval_seconds)
logger.info("Z-Wave auto-sync rescheduled to every %ds", interval_seconds)
def set_zwave_sync_enabled(enabled: bool) -> None:
"""Add or remove the Z-Wave auto-sync job on the running scheduler."""
if not scheduler.running:
return
job = scheduler.get_job("zwave_sync")
if enabled and not job:
_add_zwave_sync_job()
logger.info("Z-Wave auto-sync enabled — every %ds", settings.zwave_sync_interval)
elif not enabled and job:
scheduler.remove_job("zwave_sync")
logger.info("Z-Wave auto-sync disabled")
def stop_scheduler() -> None:
if scheduler.running:
scheduler.shutdown(wait=False)
+194 -9
View File
@@ -1,34 +1,219 @@
import hmac
import secrets
from collections.abc import Mapping
from dataclasses import dataclass
from datetime import datetime, timedelta, timezone
from functools import lru_cache
from typing import Any
from jose import JWTError, jwt
from passlib.context import CryptContext
import bcrypt
import jwt
from authlib.integrations.starlette_client import OAuth, StarletteOAuth2App
from fastapi import Request, Response, status
from jwt import InvalidTokenError
from starlette.middleware.base import BaseHTTPMiddleware, RequestResponseEndpoint
from starlette.responses import JSONResponse
from app.core.config import settings
from app.core.config import origin_of, settings
pwd_context = CryptContext(schemes=["bcrypt"], deprecated="auto")
ACCESS_TOKEN_USE = "access"
OIDC_SESSION_TOKEN_USE = "oidc_session"
SAFE_METHODS = frozenset({"GET", "HEAD", "OPTIONS", "TRACE"})
@dataclass(frozen=True)
class OIDCSession:
subject: str
issuer: str
display_name: str
csrf_token: str
def verify_password(plain: str, hashed: str) -> bool:
if not plain or not hashed:
return False
try:
return bool(pwd_context.verify(plain, hashed))
except ValueError:
return bcrypt.checkpw(plain.encode("utf-8"), hashed.encode("utf-8"))
except (ValueError, TypeError):
return False
def hash_password(password: str) -> str:
return str(pwd_context.hash(password))
return bcrypt.hashpw(password.encode("utf-8"), bcrypt.gensalt()).decode("utf-8")
def create_access_token(subject: str) -> str:
expire = datetime.now(timezone.utc) + timedelta(minutes=settings.access_token_expire_minutes)
payload = {"sub": subject, "exp": expire}
payload = {"sub": subject, "exp": expire, "token_use": ACCESS_TOKEN_USE}
return str(jwt.encode(payload, settings.secret_key, algorithm=settings.algorithm))
def decode_token(token: str) -> str | None:
try:
payload = jwt.decode(token, settings.secret_key, algorithms=[settings.algorithm])
if payload.get("token_use") not in (None, ACCESS_TOKEN_USE):
return None
sub = payload.get("sub")
return str(sub) if sub is not None else None
except JWTError:
except InvalidTokenError:
return None
def create_oidc_session_token(userinfo: Mapping[str, Any]) -> str:
subject = userinfo.get("sub")
issuer = userinfo.get("iss")
if not isinstance(subject, str) or not subject or not isinstance(issuer, str) or not issuer:
raise ValueError("OIDC userinfo must contain non-empty 'sub' and 'iss' claims")
display_name = next(
(
value
for value in (
userinfo.get("preferred_username"),
userinfo.get("name"),
userinfo.get("email"),
subject,
)
if isinstance(value, str) and value
),
subject,
)
now = datetime.now(timezone.utc)
payload = {
"sub": subject,
"oidc_issuer": issuer,
"name": display_name,
"csrf": secrets.token_urlsafe(32),
"iat": now,
"exp": now + timedelta(minutes=settings.oidc_session_expire_minutes),
"token_use": OIDC_SESSION_TOKEN_USE,
}
return str(jwt.encode(payload, settings.secret_key, algorithm=settings.algorithm))
def decode_oidc_session_token(token: str) -> OIDCSession | None:
try:
payload = jwt.decode(token, settings.secret_key, algorithms=[settings.algorithm])
if payload.get("token_use") != OIDC_SESSION_TOKEN_USE:
return None
subject = payload.get("sub")
issuer = payload.get("oidc_issuer")
display_name = payload.get("name")
csrf_token = payload.get("csrf")
if not isinstance(subject, str) or not subject:
return None
if not isinstance(issuer, str) or not issuer:
return None
if not isinstance(display_name, str) or not display_name:
return None
if not isinstance(csrf_token, str) or not csrf_token:
return None
return OIDCSession(
subject=subject,
issuer=issuer,
display_name=display_name,
csrf_token=csrf_token,
)
except InvalidTokenError:
return None
def oidc_session_cookie_name() -> str:
return "__Host-homelable-session" if settings.oidc_cookie_secure else "homelable-session"
def set_oidc_session_cookie(response: Response, token: str) -> None:
response.set_cookie(
key=oidc_session_cookie_name(),
value=token,
max_age=settings.oidc_session_expire_minutes * 60,
secure=settings.oidc_cookie_secure,
httponly=True,
samesite="lax",
path="/",
)
def clear_oidc_session_cookie(response: Response) -> None:
response.delete_cookie(
key=oidc_session_cookie_name(),
secure=settings.oidc_cookie_secure,
httponly=True,
samesite="lax",
path="/",
)
def get_oidc_session(request: Request) -> OIDCSession | None:
token = request.cookies.get(oidc_session_cookie_name())
return decode_oidc_session_token(token) if token else None
def origin_is_allowed(origin: str | None) -> bool:
if not origin:
return False
allowed_origins = {value.rstrip("/") for value in settings.cors_origins}
# A same-origin deployment behind a reverse proxy needs no CORS, so
# CORS_ORIGINS is often left at its localhost default — yet browsers still
# send Origin on unsafe methods. The OIDC callback URL is served by this app,
# so its origin is the app's own and is always a legitimate CSRF origin.
app_origin = origin_of(settings.oidc_redirect_uri)
if app_origin:
allowed_origins.add(app_origin)
return origin.rstrip("/") in allowed_origins
class OIDCCSRFMiddleware(BaseHTTPMiddleware):
async def dispatch(self, request: Request, call_next: RequestResponseEndpoint) -> Response:
if settings.auth_mode != "oidc" or request.method in SAFE_METHODS:
return await call_next(request)
# Explicit non-cookie credentials take precedence in get_current_user,
# so their requests keep the existing CSRF-free API behavior.
if request.headers.get("Authorization") or request.headers.get("X-MCP-Service-Key"):
return await call_next(request)
oidc_session = get_oidc_session(request)
if oidc_session is None:
return await call_next(request)
csrf_token = request.headers.get("X-Homelable-CSRF", "")
csrf_valid = hmac.compare_digest(csrf_token.encode(), oidc_session.csrf_token.encode())
if not csrf_valid or not origin_is_allowed(request.headers.get("Origin")):
return JSONResponse(
status_code=status.HTTP_403_FORBIDDEN,
content={"detail": "CSRF validation failed"},
)
return await call_next(request)
@lru_cache(maxsize=8)
def _build_oidc_client(
discovery_url: str,
client_id: str,
client_secret: str,
scopes: str,
) -> StarletteOAuth2App:
oauth = OAuth()
client = oauth.register(
name="oidc",
client_id=client_id,
client_secret=client_secret,
server_metadata_url=discovery_url,
client_kwargs={
"scope": scopes,
"code_challenge_method": "S256",
},
)
if client is None:
raise RuntimeError("Failed to register OIDC client")
return client
def get_oidc_client() -> StarletteOAuth2App:
return _build_oidc_client(
settings.oidc_discovery_url,
settings.oidc_client_id,
settings.oidc_client_secret,
settings.oidc_scopes,
)
+221
View File
@@ -0,0 +1,221 @@
[
{
"vendor": "Proxmox / QEMU / KVM",
"type": "vm",
"prefixes": ["52:54:00", "bc:24:11"]
},
{
"vendor": "VMware",
"type": "vm",
"prefixes": ["00:50:56", "00:0c:29", "00:05:69", "00:1c:14"]
},
{
"vendor": "VirtualBox",
"type": "vm",
"prefixes": ["08:00:27"]
},
{
"vendor": "Microsoft Hyper-V",
"type": "vm",
"prefixes": ["00:15:5d"]
},
{
"vendor": "Xen",
"type": "vm",
"prefixes": ["00:16:3e"]
},
{
"vendor": "MikroTik",
"type": "router",
"prefixes": [
"00:0c:42",
"08:55:31",
"18:fd:74",
"2c:c8:1b",
"48:8f:5a",
"4c:5e:0c",
"64:d1:54",
"6c:3b:6b",
"74:4d:28",
"b8:69:f4",
"c4:ad:34",
"cc:2d:e0",
"d4:ca:6d",
"dc:2c:6e",
"e4:8d:8c"
]
},
{
"vendor": "Ubiquiti",
"type": "ap",
"prefixes": [
"00:15:6d",
"00:27:22",
"04:18:d6",
"24:5a:4c",
"24:a4:3c",
"44:d9:e7",
"68:72:51",
"68:d7:9a",
"74:83:c2",
"78:8a:20",
"78:45:58",
"80:2a:a8",
"94:2a:6f",
"9c:05:d6",
"b4:fb:e4",
"dc:9f:db",
"e0:63:da",
"f0:9f:c2",
"fc:ec:da"
]
},
{
"vendor": "Ruckus Wireless",
"type": "ap",
"prefixes": ["00:13:92", "4c:b1:cd", "8c:7a:15", "f0:b0:52", "c0:8a:de"]
},
{
"vendor": "Aruba Networks (HPE)",
"type": "ap",
"prefixes": ["00:0b:86", "6c:f3:7f", "94:b4:0f", "9c:1c:12", "ac:a3:1e"]
},
{
"vendor": "Cisco Systems",
"type": "switch",
"prefixes": [
"00:00:0c",
"00:1b:0d",
"00:1c:f6",
"00:1e:13",
"00:23:04",
"00:24:13",
"00:25:45",
"00:50:0b",
"b0:00:b4",
"b8:38:61",
"f8:c0:01"
]
},
{
"vendor": "Juniper Networks",
"type": "switch",
"prefixes": ["00:14:f6", "2c:6b:f5", "b0:c6:9a", "f0:1c:2d"]
},
{
"vendor": "Zyxel",
"type": "switch",
"prefixes": ["00:13:49", "60:31:97", "ec:43:f6"]
},
{
"vendor": "Netgear",
"type": "router",
"prefixes": ["00:09:5b", "28:c6:8e", "c0:ff:d4", "2c:30:33", "a0:40:a0"]
},
{
"vendor": "TP-Link",
"type": "router",
"prefixes": ["14:eb:b6", "60:e3:27", "b0:4e:26", "c4:e9:0a", "ec:08:6b"]
},
{
"vendor": "Synology",
"type": "nas",
"prefixes": ["00:11:32", "00:f4:6f", "90:09:d0"]
},
{
"vendor": "QNAP Systems",
"type": "nas",
"prefixes": ["00:08:9b", "00:0e:23", "00:13:42", "04:f0:21", "24:5e:be"]
},
{
"vendor": "Asustor",
"type": "nas",
"prefixes": ["e8:9c:25"]
},
{
"vendor": "Hikvision",
"type": "camera",
"prefixes": ["28:57:be", "44:19:b6", "b4:a3:82", "bc:ad:28", "c0:51:7e", "c0:56:e3", "c4:2f:90"]
},
{
"vendor": "Dahua / Amcrest",
"type": "camera",
"prefixes": ["3c:ef:8c", "4c:11:bf", "90:02:a9", "bc:32:5f", "e0:50:8b"]
},
{
"vendor": "Reolink",
"type": "camera",
"prefixes": ["ec:71:db"]
},
{
"vendor": "Axis Communications",
"type": "camera",
"prefixes": ["00:40:8c", "ac:cc:8e"]
},
{
"vendor": "Raspberry Pi Foundation",
"type": "server",
"prefixes": ["28:cd:c1", "2c:cf:67", "b8:27:eb", "d8:3a:dd", "dc:a6:32", "e4:5f:01"]
},
{
"vendor": "Dell",
"type": "server",
"prefixes": ["00:14:22", "90:b1:1c", "b0:83:fe", "b8:ca:3a", "f8:b1:56"]
},
{
"vendor": "Supermicro",
"type": "server",
"prefixes": ["00:25:90", "0c:c4:7a", "ac:1f:6b"]
},
{
"vendor": "Shelly",
"type": "iot",
"prefixes": ["30:c6:f7", "34:94:54", "84:f3:eb", "ec:fa:bc"]
},
{
"vendor": "Espressif (ESP8266 / ESP32)",
"type": "iot",
"prefixes": [
"24:62:ab",
"30:ae:a4",
"3c:71:bf",
"8c:aa:b5",
"a0:20:a6",
"ac:67:b2",
"b4:e6:2d",
"cc:50:e3"
]
},
{
"vendor": "Sonoff / ITEAD",
"type": "iot",
"prefixes": ["dc:4f:22", "e8:db:84"]
},
{
"vendor": "TP-Link Tapo / Kasa",
"type": "iot",
"prefixes": ["10:27:f5", "1c:3b:f3", "50:c7:bf", "b0:a7:b9"]
},
{
"vendor": "Philips Hue",
"type": "iot",
"prefixes": ["00:17:88", "ec:b5:fa"]
},
{
"vendor": "IKEA Tradfri",
"type": "iot",
"prefixes": ["00:21:2e", "34:13:e8"]
},
{
"vendor": "Tuya / Smart Life",
"type": "iot",
"prefixes": ["68:57:2d", "d8:f1:5b"]
}
]
+70 -1
View File
@@ -142,5 +142,74 @@
{"port": 1194, "protocol": "udp", "banner_regex": null, "service_name": "OpenVPN", "icon": "shield", "category": "vpn", "suggested_node_type": "router"},
{"port": 500, "protocol": "udp", "banner_regex": null, "service_name": "IPsec IKE", "icon": "shield", "category": "vpn", "suggested_node_type": "router"},
{"port": 53, "protocol": "udp", "banner_regex": null, "service_name": "DNS", "icon": "search", "category": "network", "suggested_node_type": "router"},
{"port": 67, "protocol": "udp", "banner_regex": null, "service_name": "DHCP", "icon": "wifi", "category": "network", "suggested_node_type": "router"}
{"port": 67, "protocol": "udp", "banner_regex": null, "service_name": "DHCP", "icon": "wifi", "category": "network", "suggested_node_type": "router"},
{"port": null, "protocol": "tcp", "banner_regex": null, "http_regex": "Jellyfin", "service_name": "Jellyfin", "icon": "film", "category": "media", "suggested_node_type": "server"},
{"port": null, "protocol": "tcp", "banner_regex": null, "http_regex": "Plex", "service_name": "Plex", "icon": "play-circle", "category": "media", "suggested_node_type": "server"},
{"port": null, "protocol": "tcp", "banner_regex": null, "http_regex": "Emby", "service_name": "Emby", "icon": "film", "category": "media", "suggested_node_type": "server"},
{"port": null, "protocol": "tcp", "banner_regex": null, "http_regex": "Overseerr", "service_name": "Overseerr", "icon": "tv", "category": "media", "suggested_node_type": "server"},
{"port": null, "protocol": "tcp", "banner_regex": null, "http_regex": "Jellyseerr", "service_name": "Jellyseerr", "icon": "tv", "category": "media", "suggested_node_type": "server"},
{"port": null, "protocol": "tcp", "banner_regex": null, "http_regex": "Tautulli", "service_name": "Tautulli", "icon": "bar-chart", "category": "media", "suggested_node_type": "server"},
{"port": null, "protocol": "tcp", "banner_regex": null, "http_regex": "Navidrome", "service_name": "Navidrome", "icon": "music", "category": "media", "suggested_node_type": "server"},
{"port": null, "protocol": "tcp", "banner_regex": null, "http_regex": "[Aa]udiobookshelf", "service_name": "Audiobookshelf", "icon": "book-open", "category": "media", "suggested_node_type": "server"},
{"port": null, "protocol": "tcp", "banner_regex": null, "http_regex": "Immich", "service_name": "Immich", "icon": "camera", "category": "media", "suggested_node_type": "server"},
{"port": null, "protocol": "tcp", "banner_regex": null, "http_regex": "PhotoPrism", "service_name": "PhotoPrism", "icon": "camera", "category": "media", "suggested_node_type": "server"},
{"port": null, "protocol": "tcp", "banner_regex": null, "http_regex": "Calibre[- ]Web", "service_name": "Calibre-Web", "icon": "book", "category": "media", "suggested_node_type": "server"},
{"port": null, "protocol": "tcp", "banner_regex": null, "http_regex": "Sonarr", "service_name": "Sonarr", "icon": "tv", "category": "download", "suggested_node_type": "server"},
{"port": null, "protocol": "tcp", "banner_regex": null, "http_regex": "Radarr", "service_name": "Radarr", "icon": "film", "category": "download", "suggested_node_type": "server"},
{"port": null, "protocol": "tcp", "banner_regex": null, "http_regex": "Lidarr", "service_name": "Lidarr", "icon": "music", "category": "download", "suggested_node_type": "server"},
{"port": null, "protocol": "tcp", "banner_regex": null, "http_regex": "Readarr", "service_name": "Readarr", "icon": "book", "category": "download", "suggested_node_type": "server"},
{"port": null, "protocol": "tcp", "banner_regex": null, "http_regex": "Prowlarr", "service_name": "Prowlarr", "icon": "search", "category": "download", "suggested_node_type": "server"},
{"port": null, "protocol": "tcp", "banner_regex": null, "http_regex": "Bazarr", "service_name": "Bazarr", "icon": "subtitles", "category": "download", "suggested_node_type": "server"},
{"port": null, "protocol": "tcp", "banner_regex": null, "http_regex": "qBittorrent", "service_name": "qBittorrent", "icon": "download", "category": "download", "suggested_node_type": "server"},
{"port": null, "protocol": "tcp", "banner_regex": null, "http_regex": "SABnzbd", "service_name": "SABnzbd", "icon": "download", "category": "download", "suggested_node_type": "server"},
{"port": null, "protocol": "tcp", "banner_regex": null, "http_regex": "Homarr", "service_name": "Homarr", "icon": "home", "category": "web", "suggested_node_type": "server"},
{"port": null, "protocol": "tcp", "banner_regex": null, "http_regex": "Heimdall", "service_name": "Heimdall", "icon": "home", "category": "web", "suggested_node_type": "server"},
{"port": null, "protocol": "tcp", "banner_regex": null, "http_regex": "Dashy", "service_name": "Dashy", "icon": "home", "category": "web", "suggested_node_type": "server"},
{"port": null, "protocol": "tcp", "banner_regex": null, "http_regex": "Organizr", "service_name": "Organizr", "icon": "home", "category": "web", "suggested_node_type": "server"},
{"port": null, "protocol": "tcp", "banner_regex": null, "http_regex": "Portainer", "service_name": "Portainer", "icon": "box", "category": "containers", "suggested_node_type": "server"},
{"port": null, "protocol": "tcp", "banner_regex": null, "http_regex": "Dockge", "service_name": "Dockge", "icon": "box", "category": "containers", "suggested_node_type": "server"},
{"port": null, "protocol": "tcp", "banner_regex": null, "http_regex": "Yacht", "service_name": "Yacht", "icon": "box", "category": "containers", "suggested_node_type": "server"},
{"port": null, "protocol": "tcp", "banner_regex": null, "http_regex": "Home Assistant", "service_name": "Home Assistant", "icon": "home", "category": "automation", "suggested_node_type": "server"},
{"port": null, "protocol": "tcp", "banner_regex": null, "http_regex": "Node-RED", "service_name": "Node-RED", "icon": "share-2", "category": "automation", "suggested_node_type": "server"},
{"port": null, "protocol": "tcp", "banner_regex": null, "http_regex": "Zigbee2MQTT", "service_name": "Zigbee2MQTT", "icon": "radio", "category": "automation", "suggested_node_type": "iot"},
{"port": null, "protocol": "tcp", "banner_regex": null, "http_regex": "ESPHome", "service_name": "ESPHome", "icon": "cpu", "category": "automation", "suggested_node_type": "iot"},
{"port": null, "protocol": "tcp", "banner_regex": null, "http_regex": "openHAB", "service_name": "openHAB", "icon": "home", "category": "automation", "suggested_node_type": "server"},
{"port": null, "protocol": "tcp", "banner_regex": null, "http_regex": "Domoticz", "service_name": "Domoticz", "icon": "home", "category": "automation", "suggested_node_type": "server"},
{"port": null, "protocol": "tcp", "banner_regex": null, "http_regex": "Homebridge", "service_name": "Homebridge", "icon": "home", "category": "automation", "suggested_node_type": "server"},
{"port": null, "protocol": "tcp", "banner_regex": null, "http_regex": "Jeedom", "service_name": "Jeedom", "icon": "home", "category": "automation", "suggested_node_type": "server"},
{"port": null, "protocol": "tcp", "banner_regex": null, "http_regex": "Scrypted", "service_name": "Scrypted", "icon": "video", "category": "automation", "suggested_node_type": "server"},
{"port": null, "protocol": "tcp", "banner_regex": null, "http_regex": "Grafana", "service_name": "Grafana", "icon": "bar-chart-2", "category": "monitoring", "suggested_node_type": "server"},
{"port": null, "protocol": "tcp", "banner_regex": null, "http_regex": "Uptime Kuma", "service_name": "Uptime Kuma", "icon": "activity", "category": "monitoring", "suggested_node_type": "server"},
{"port": null, "protocol": "tcp", "banner_regex": null, "http_regex": "Netdata", "service_name": "Netdata", "icon": "activity", "category": "monitoring", "suggested_node_type": "server"},
{"port": null, "protocol": "tcp", "banner_regex": null, "http_regex": "Glances", "service_name": "Glances", "icon": "activity", "category": "monitoring", "suggested_node_type": "server"},
{"port": null, "protocol": "tcp", "banner_regex": null, "http_regex": "Dozzle", "service_name": "Dozzle", "icon": "terminal", "category": "monitoring", "suggested_node_type": "server"},
{"port": null, "protocol": "tcp", "banner_regex": null, "http_regex": "AdGuard Home", "service_name": "AdGuard Home", "icon": "shield", "category": "network", "suggested_node_type": "server"},
{"port": null, "protocol": "tcp", "banner_regex": null, "http_regex": "Pi-hole", "service_name": "Pi-hole", "icon": "shield", "category": "network", "suggested_node_type": "server"},
{"port": null, "protocol": "tcp", "banner_regex": null, "http_regex": "Nginx Proxy Manager", "service_name": "Nginx Proxy Manager", "icon": "share-2", "category": "network", "suggested_node_type": "server"},
{"port": null, "protocol": "tcp", "banner_regex": null, "http_regex": "Traefik", "service_name": "Traefik", "icon": "share-2", "category": "network", "suggested_node_type": "server"},
{"port": null, "protocol": "tcp", "banner_regex": null, "http_regex": "Vaultwarden|Bitwarden", "service_name": "Vaultwarden", "icon": "lock", "category": "auth", "suggested_node_type": "server"},
{"port": null, "protocol": "tcp", "banner_regex": null, "http_regex": "Authelia", "service_name": "Authelia", "icon": "lock", "category": "auth", "suggested_node_type": "server"},
{"port": null, "protocol": "tcp", "banner_regex": null, "http_regex": "[Aa]uthentik", "service_name": "Authentik", "icon": "lock", "category": "auth", "suggested_node_type": "server"},
{"port": null, "protocol": "tcp", "banner_regex": null, "http_regex": "Nextcloud", "service_name": "Nextcloud", "icon": "hard-drive", "category": "storage", "suggested_node_type": "server"},
{"port": null, "protocol": "tcp", "banner_regex": null, "http_regex": "Paperless", "service_name": "Paperless-ngx", "icon": "book", "category": "storage", "suggested_node_type": "server"},
{"port": null, "protocol": "tcp", "banner_regex": null, "http_regex": "Syncthing", "service_name": "Syncthing", "icon": "refresh-cw", "category": "storage", "suggested_node_type": "server"},
{"port": null, "protocol": "tcp", "banner_regex": null, "http_regex": "Gitea", "service_name": "Gitea", "icon": "git-branch", "category": "dev", "suggested_node_type": "server"},
{"port": null, "protocol": "tcp", "banner_regex": null, "http_regex": "[Pp]i[Kk]VM", "service_name": "PiKVM", "icon": "monitorcog", "category": "kvm", "suggested_node_type": "kvm"},
{"port": null, "protocol": "tcp", "banner_regex": null, "http_regex": "TinyPilot", "service_name": "TinyPilot", "icon": "monitorcog", "category": "kvm", "suggested_node_type": "kvm"},
{"port": null, "protocol": "tcp", "banner_regex": null, "http_regex": "JetKVM", "service_name": "JetKVM", "icon": "monitorcog", "category": "kvm", "suggested_node_type": "kvm"},
{"port": null, "protocol": "tcp", "banner_regex": null, "http_regex": "NanoKVM", "service_name": "NanoKVM", "icon": "monitorcog", "category": "kvm", "suggested_node_type": "kvm"},
{"port": null, "protocol": "tcp", "banner_regex": null, "http_regex": "openmediavault", "service_name": "OpenMediaVault", "icon": "hard-drive", "category": "nas", "suggested_node_type": "nas"},
{"port": null, "protocol": "tcp", "banner_regex": null, "http_regex": "Unraid", "service_name": "Unraid", "icon": "hard-drive", "category": "nas", "suggested_node_type": "nas"},
{"port": null, "protocol": "tcp", "banner_regex": null, "http_regex": "Cockpit", "service_name": "Cockpit", "icon": "monitor", "category": "nas", "suggested_node_type": "server"}
]
+691 -4
View File
@@ -1,12 +1,37 @@
import json as _json
import logging
import shutil
import sqlite3
import uuid as _uuid_mod
from collections.abc import AsyncGenerator
from contextlib import suppress
from contextlib import closing, suppress
from pathlib import Path
from typing import Any
from sqlalchemy.exc import OperationalError
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker, create_async_engine
from sqlalchemy.exc import IntegrityError, OperationalError
from sqlalchemy.ext.asyncio import AsyncConnection, AsyncSession, async_sessionmaker, create_async_engine
from sqlalchemy.orm import DeclarativeBase
from app.core.config import settings
from app.core.config import APP_VERSION, settings
logger = logging.getLogger(__name__)
async def _try_migrate(conn: AsyncConnection, sql: str, *, label: str) -> None:
"""Run an idempotent migration statement, logging any error.
Distinguishes 'already applied' errors (debug) from genuine failures
(warning) so silent corruption is avoided. Used for new in-commit
migrations; existing legacy ALTERs above remain wrapped in suppress.
"""
try:
await conn.exec_driver_sql(sql)
except OperationalError as exc:
msg = str(exc).lower()
if "duplicate column" in msg or "already exists" in msg:
logger.debug("Migration %s skipped (already applied): %s", label, exc)
else:
logger.warning("Migration %s failed: %s", label, exc)
# Ensure the data directory exists before SQLite tries to open the file
Path(settings.sqlite_path).parent.mkdir(parents=True, exist_ok=True)
@@ -23,8 +48,81 @@ class Base(DeclarativeBase):
pass
def _backup_db() -> None:
db_path = Path(settings.sqlite_path)
if not db_path.exists():
return
backup_path = db_path.with_suffix(f".db.back-{APP_VERSION}")
if backup_path.exists():
return
try:
shutil.copy2(db_path, backup_path)
logger.info("DB backup created: %s", backup_path.name)
except OSError:
logger.warning("Could not create DB backup at %s", backup_path)
# Tables renamed in 3.2.0. "Pending devices" was the scanner's word for a queue
# of finds awaiting approval; the same rows outlive approval, are edited by hand
# and are what a rack mounts, so the product calls them the Device Inventory.
# Only the names moved — every column, every row and every route is unchanged.
_RENAMED_TABLES: list[tuple[str, str]] = [
("pending_devices", "device_inventory"),
("pending_device_links", "device_inventory_links"),
]
async def _rename_legacy_tables(conn: AsyncConnection) -> None:
"""Rename the pre-3.2.0 inventory tables. Must run before `create_all`.
Order matters: `create_all` would otherwise create an empty
`device_inventory` beside the populated `pending_devices`, and every device
the user ever scanned would look lost.
Foreign keys are switched on for the rename so SQLite rewrites the
``REFERENCES pending_devices`` clause in `rack_devices` too; enforcement is
off at runtime, so a missed rewrite costs nothing today, but it would leave
the schema naming a table that no longer exists.
"""
for old, new in _RENAMED_TABLES:
rows = (
await conn.exec_driver_sql(
"SELECT name FROM sqlite_master WHERE type='table' AND name IN (?, ?)",
(old, new),
)
).fetchall()
names = {r[0] for r in rows}
if old not in names:
continue # Fresh install, or already migrated.
if new in names:
# A start that ran `create_all` before this migration existed left an
# empty new table beside the populated old one, and the app read the
# empty one. Clear it out of the way rather than stranding the data;
# a new table with rows in it means the rename already happened and
# something re-created the old one, which is not ours to resolve.
count = (await conn.exec_driver_sql(f"SELECT COUNT(*) FROM {new}")).scalar()
if count:
logger.warning(
"Both %s and %s hold rows; leaving them alone. Merge them by hand.", old, new
)
continue
logger.info("Dropping the empty %s left by an earlier start", new)
await _try_migrate(conn, f"DROP TABLE {new}", label=f"{new}.drop_empty")
logger.info("Migrating table %s -> %s", old, new)
with suppress(OperationalError):
await conn.exec_driver_sql("PRAGMA foreign_keys = ON")
await _try_migrate(conn, f"ALTER TABLE {old} RENAME TO {new}", label=f"{new}.rename")
# The index rides the rename under its old name; the model asks for the new
# one, so drop the stale duplicate rather than carrying both.
await _try_migrate(
conn, "DROP INDEX IF EXISTS ix_pending_devices_ieee_address", label="device_inventory.index",
)
async def init_db() -> None:
_backup_db()
async with engine.begin() as conn:
await _rename_legacy_tables(conn)
await conn.run_sync(Base.metadata.create_all)
# Add columns introduced after initial schema (idempotent)
with suppress(OperationalError):
@@ -43,6 +141,14 @@ async def init_db() -> None:
await conn.exec_driver_sql("ALTER TABLE edges ADD COLUMN target_handle TEXT")
with suppress(OperationalError):
await conn.exec_driver_sql("ALTER TABLE edges ADD COLUMN animated BOOLEAN NOT NULL DEFAULT 0")
with suppress(OperationalError):
await conn.exec_driver_sql("ALTER TABLE edges ADD COLUMN marker_start TEXT NOT NULL DEFAULT 'none'")
with suppress(OperationalError):
await conn.exec_driver_sql("ALTER TABLE edges ADD COLUMN marker_end TEXT NOT NULL DEFAULT 'none'")
with suppress(OperationalError):
await conn.exec_driver_sql("ALTER TABLE edges ADD COLUMN line_style TEXT")
with suppress(OperationalError):
await conn.exec_driver_sql("ALTER TABLE edges ADD COLUMN width_mult REAL")
with suppress(OperationalError):
await conn.exec_driver_sql("ALTER TABLE nodes ADD COLUMN cpu_count INTEGER")
with suppress(OperationalError):
@@ -53,16 +159,597 @@ async def init_db() -> None:
await conn.exec_driver_sql("ALTER TABLE nodes ADD COLUMN disk_gb REAL")
with suppress(OperationalError):
await conn.exec_driver_sql("ALTER TABLE nodes ADD COLUMN show_hardware BOOLEAN NOT NULL DEFAULT 0")
with suppress(OperationalError):
await conn.exec_driver_sql("ALTER TABLE nodes ADD COLUMN show_port_numbers BOOLEAN NOT NULL DEFAULT 0")
with suppress(OperationalError):
await conn.exec_driver_sql("ALTER TABLE nodes ADD COLUMN width REAL")
with suppress(OperationalError):
await conn.exec_driver_sql("ALTER TABLE nodes ADD COLUMN height REAL")
with suppress(OperationalError):
await conn.exec_driver_sql("ALTER TABLE nodes ADD COLUMN bottom_handles INTEGER NOT NULL DEFAULT 1")
with suppress(OperationalError):
await conn.exec_driver_sql("ALTER TABLE nodes ADD COLUMN top_handles INTEGER NOT NULL DEFAULT 1")
with suppress(OperationalError):
await conn.exec_driver_sql("ALTER TABLE nodes ADD COLUMN left_handles INTEGER NOT NULL DEFAULT 0")
with suppress(OperationalError):
await conn.exec_driver_sql("ALTER TABLE nodes ADD COLUMN right_handles INTEGER NOT NULL DEFAULT 0")
with suppress(OperationalError):
await conn.exec_driver_sql("ALTER TABLE device_inventory ADD COLUMN discovery_source TEXT")
with suppress(OperationalError):
await conn.exec_driver_sql("ALTER TABLE device_inventory ADD COLUMN properties JSON")
with suppress(OperationalError):
await conn.exec_driver_sql("UPDATE device_inventory SET properties = '[]' WHERE properties IS NULL")
with suppress(OperationalError):
await conn.exec_driver_sql("ALTER TABLE scan_runs ADD COLUMN kind TEXT NOT NULL DEFAULT 'ip'")
# --- Zigbee schema migrations (logged variant per CLAUDE.md feedback) ---
zigbee_migrations: list[tuple[str, str]] = [
("nodes.ieee_address", "ALTER TABLE nodes ADD COLUMN ieee_address TEXT"),
(
"nodes.ieee_address.index",
"CREATE INDEX IF NOT EXISTS ix_nodes_ieee_address ON nodes(ieee_address)",
),
("device_inventory.ieee_address", "ALTER TABLE device_inventory ADD COLUMN ieee_address TEXT"),
(
"device_inventory.ieee_address.index",
"CREATE INDEX IF NOT EXISTS ix_device_inventory_ieee_address "
"ON device_inventory(ieee_address)",
),
("device_inventory.friendly_name", "ALTER TABLE device_inventory ADD COLUMN friendly_name TEXT"),
("device_inventory.device_subtype", "ALTER TABLE device_inventory ADD COLUMN device_subtype TEXT"),
("device_inventory.model", "ALTER TABLE device_inventory ADD COLUMN model TEXT"),
("device_inventory.vendor", "ALTER TABLE device_inventory ADD COLUMN vendor TEXT"),
("device_inventory.lqi", "ALTER TABLE device_inventory ADD COLUMN lqi INTEGER"),
]
for label, sql in zigbee_migrations:
await _try_migrate(conn, sql, label=label)
# Drop NOT NULL on device_inventory.ip (Zigbee devices have no IP).
# SQLite can't ALTER column nullability — rebuild the table if needed.
try:
info = await conn.exec_driver_sql("PRAGMA table_info(device_inventory)")
cols = info.fetchall()
ip_col = next((c for c in cols if c[1] == "ip"), None)
# PRAGMA table_info row layout: (cid, name, type, notnull, dflt, pk)
if ip_col and ip_col[3] == 1:
logger.info("Migrating device_inventory: dropping NOT NULL on ip column")
await conn.exec_driver_sql("PRAGMA foreign_keys = OFF")
await conn.exec_driver_sql(
"CREATE TABLE device_inventory_new ("
"id VARCHAR PRIMARY KEY,"
"ip VARCHAR,"
"mac VARCHAR, hostname VARCHAR, os VARCHAR, services JSON,"
"suggested_type VARCHAR,"
"status VARCHAR,"
"discovery_source VARCHAR,"
"ieee_address VARCHAR,"
"friendly_name VARCHAR,"
"device_subtype VARCHAR,"
"model VARCHAR,"
"vendor VARCHAR,"
"lqi INTEGER,"
"properties JSON,"
"discovered_at DATETIME"
")"
)
await conn.exec_driver_sql(
"INSERT INTO device_inventory_new "
"(id, ip, mac, hostname, os, services, suggested_type, status, "
"discovery_source, ieee_address, friendly_name, device_subtype, "
"model, vendor, lqi, discovered_at) "
"SELECT id, ip, mac, hostname, os, services, suggested_type, status, "
"discovery_source, ieee_address, friendly_name, device_subtype, "
"model, vendor, lqi, discovered_at FROM device_inventory"
)
await conn.exec_driver_sql("DROP TABLE device_inventory")
await conn.exec_driver_sql(
"ALTER TABLE device_inventory_new RENAME TO device_inventory"
)
await conn.exec_driver_sql(
"CREATE INDEX IF NOT EXISTS ix_device_inventory_ieee_address "
"ON device_inventory(ieee_address)"
)
await conn.exec_driver_sql("PRAGMA foreign_keys = ON")
except OperationalError as exc:
logger.warning("device_inventory ip-nullable rebuild failed: %s", exc)
# --- end Zigbee schema migrations -------------------------------------
# --- Electrical designs schema migrations -----------------------------
# Create designs table (idempotent)
await _try_migrate(
conn,
"CREATE TABLE IF NOT EXISTS designs ("
"id VARCHAR PRIMARY KEY,"
"name VARCHAR NOT NULL,"
"design_type VARCHAR NOT NULL DEFAULT 'network',"
"created_at DATETIME,"
"updated_at DATETIME"
")",
label="designs.table",
)
# Add user-chosen icon to designs (idempotent), then backfill existing rows
# so legacy designs keep a sensible icon based on their original type.
await _try_migrate(
conn, "ALTER TABLE designs ADD COLUMN icon VARCHAR", label="designs.icon",
)
with suppress(OperationalError):
await conn.exec_driver_sql(
"UPDATE designs SET icon = 'zap' WHERE icon IS NULL AND design_type = 'electrical'"
)
with suppress(OperationalError):
await conn.exec_driver_sql(
"UPDATE designs SET icon = 'dashboard' WHERE icon IS NULL"
)
# Seed default Network Topology design if designs table is empty
_default_design_id = str(_uuid_mod.uuid4())
row = await conn.exec_driver_sql("SELECT COUNT(*) FROM designs")
count_row = row.fetchone()
count = count_row[0] if count_row else 0
if count == 0:
await conn.exec_driver_sql(
"INSERT INTO designs (id, name, design_type, icon, created_at, updated_at) "
"VALUES (?, 'Network Topology', 'network', 'dashboard', datetime('now'), datetime('now'))",
(_default_design_id,),
)
else:
row2 = await conn.exec_driver_sql("SELECT id FROM designs WHERE design_type = 'network' LIMIT 1")
default = row2.fetchone()
_default_design_id = default[0] if default else _default_design_id
# Add design_id to nodes
await _try_migrate(
conn, "ALTER TABLE nodes ADD COLUMN design_id VARCHAR REFERENCES designs(id)",
label="nodes.design_id",
)
# Assign existing nodes to default design
await conn.exec_driver_sql(
"UPDATE nodes SET design_id = ? WHERE design_id IS NULL", (_default_design_id,),
)
# Add design_id to edges
await _try_migrate(
conn, "ALTER TABLE edges ADD COLUMN design_id VARCHAR REFERENCES designs(id)",
label="edges.design_id",
)
# Assign existing edges to default design
await conn.exec_driver_sql(
"UPDATE edges SET design_id = ? WHERE design_id IS NULL", (_default_design_id,),
)
# Migrate canvas_state from id=1 to design_id PK (SQLite rebuild)
try:
info = await conn.exec_driver_sql("PRAGMA table_info(canvas_state)")
cols = info.fetchall()
has_design_id = any(c[1] == "design_id" for c in cols)
if not has_design_id:
logger.info("Migrating canvas_state: switching to design_id primary key")
await conn.exec_driver_sql("PRAGMA foreign_keys = OFF")
await conn.exec_driver_sql(
"CREATE TABLE canvas_state_new ("
"design_id VARCHAR PRIMARY KEY REFERENCES designs(id) ON DELETE CASCADE,"
"viewport JSON,"
"custom_style JSON,"
"saved_at DATETIME"
")"
)
# Copy existing row(s), mapping id=1 to default design_id
old_rows = await conn.exec_driver_sql("SELECT id, viewport, custom_style, saved_at FROM canvas_state")
for old in old_rows.fetchall():
cs_id, viewport, custom_style, saved_at = old
target_design = _default_design_id
await conn.exec_driver_sql(
"INSERT INTO canvas_state_new (design_id, viewport, custom_style, saved_at) "
"VALUES (?, ?, ?, ?)",
(target_design, viewport, custom_style, saved_at),
)
await conn.exec_driver_sql("DROP TABLE canvas_state")
await conn.exec_driver_sql("ALTER TABLE canvas_state_new RENAME TO canvas_state")
await conn.exec_driver_sql("PRAGMA foreign_keys = ON")
except OperationalError as exc:
logger.warning("canvas_state migration failed: %s", exc)
# --- end Electrical designs schema migrations --------------------------
# Rack cables gained editable annotations: a label the user can print on
# the canvas, plus the same free-form property records nodes carry.
rack_cable_migrations: list[tuple[str, str]] = [
(
"rack_cables.label_visible",
"ALTER TABLE rack_cables ADD COLUMN label_visible BOOLEAN NOT NULL DEFAULT 0",
),
("rack_cables.properties", "ALTER TABLE rack_cables ADD COLUMN properties JSON"),
]
for label, sql in rack_cable_migrations:
await _try_migrate(conn, sql, label=label)
with suppress(OperationalError):
await conn.exec_driver_sql(
"UPDATE rack_cables SET properties = '[]' WHERE properties IS NULL"
)
with suppress(OperationalError):
await conn.exec_driver_sql("ALTER TABLE edges ADD COLUMN waypoints JSON")
with suppress(OperationalError):
await conn.exec_driver_sql("ALTER TABLE nodes ADD COLUMN properties JSON")
# Migrate hardware columns → properties JSON (idempotent: only runs on nodes where properties IS NULL)
with suppress(OperationalError):
rows = await conn.exec_driver_sql(
"SELECT id, cpu_model, cpu_count, ram_gb, disk_gb, show_hardware "
"FROM nodes WHERE properties IS NULL"
)
for r in rows.fetchall():
node_id, cpu_model, cpu_count, ram_gb, disk_gb, show_hardware = r
props = []
visible = bool(show_hardware)
if cpu_model:
props.append({"key": "CPU Model", "value": str(cpu_model), "icon": "Cpu", "visible": visible})
if cpu_count is not None:
props.append({"key": "CPU Cores", "value": str(cpu_count), "icon": "Cpu", "visible": visible})
if ram_gb is not None:
props.append({"key": "RAM", "value": f"{ram_gb} GB", "icon": "MemoryStick", "visible": visible})
if disk_gb is not None:
props.append({"key": "Disk", "value": f"{disk_gb} GB", "icon": "HardDrive", "visible": visible})
await conn.exec_driver_sql(
"UPDATE nodes SET properties = ? WHERE id = ?",
(_json.dumps(props), node_id),
)
# Inventory timestamp: last time a scan observed this node (idempotent)
with suppress(OperationalError):
await conn.exec_driver_sql("ALTER TABLE nodes ADD COLUMN last_scan DATETIME")
# Migrate animated column from boolean (0/1) to string ('none'/'snake')
with suppress(OperationalError):
await conn.exec_driver_sql("UPDATE edges SET animated = 'snake' WHERE animated = '1' OR animated = 1")
with suppress(OperationalError):
sql = "UPDATE edges SET animated = 'none' WHERE animated = '0' OR animated = 0 OR animated IS NULL"
await conn.exec_driver_sql(sql)
# Multi-source discovery tags: a device found by both an IP scan and a
# Proxmox import carries every source. Backfill from the legacy single
# discovery_source so existing rows show under their filter.
with suppress(OperationalError):
await conn.exec_driver_sql("ALTER TABLE device_inventory ADD COLUMN discovery_sources JSON")
with suppress(OperationalError):
await conn.exec_driver_sql(
"UPDATE device_inventory SET discovery_sources = json_array(discovery_source) "
"WHERE discovery_sources IS NULL AND discovery_source IS NOT NULL"
)
# Legacy IP-scanned rows predating discovery_source have a NULL scalar
# but a real IP — treat them as an ARP scan so they keep the IP tag.
with suppress(OperationalError):
await conn.exec_driver_sql(
"UPDATE device_inventory SET discovery_sources = json_array('arp') "
"WHERE discovery_sources IS NULL AND discovery_source IS NULL AND ip IS NOT NULL"
)
with suppress(OperationalError):
await conn.exec_driver_sql(
"UPDATE device_inventory SET discovery_sources = '[]' WHERE discovery_sources IS NULL"
)
# Canonicalize stored MACs (lowercase, ':' separators) so cross-source
# dedup can match a Proxmox NIC MAC against an ARP-scanned one by equality.
with suppress(OperationalError):
await conn.exec_driver_sql(
"UPDATE device_inventory SET mac = lower(replace(mac, '-', ':')) WHERE mac IS NOT NULL"
)
with suppress(OperationalError):
await conn.exec_driver_sql(
"UPDATE nodes SET mac = lower(replace(mac, '-', ':')) WHERE mac IS NOT NULL"
)
# 3.3.0 — the inventory row owns the device facts, the canvas node only
# owns how it is drawn. These columns receive what used to live solely on
# `nodes`. Additive here; `nodes` keeps its copies until the backfill has
# run and been verified.
inventory_device_migrations: list[tuple[str, str]] = [
("device_inventory.label", "ALTER TABLE device_inventory ADD COLUMN label TEXT"),
("device_inventory.type", "ALTER TABLE device_inventory ADD COLUMN type TEXT"),
("device_inventory.notes", "ALTER TABLE device_inventory ADD COLUMN notes TEXT"),
("device_inventory.cpu_count", "ALTER TABLE device_inventory ADD COLUMN cpu_count INTEGER"),
("device_inventory.cpu_model", "ALTER TABLE device_inventory ADD COLUMN cpu_model TEXT"),
("device_inventory.ram_gb", "ALTER TABLE device_inventory ADD COLUMN ram_gb FLOAT"),
("device_inventory.disk_gb", "ALTER TABLE device_inventory ADD COLUMN disk_gb FLOAT"),
(
"device_inventory.show_hardware",
"ALTER TABLE device_inventory ADD COLUMN show_hardware BOOLEAN DEFAULT 0",
),
("device_inventory.check_method", "ALTER TABLE device_inventory ADD COLUMN check_method TEXT"),
("device_inventory.check_target", "ALTER TABLE device_inventory ADD COLUMN check_target TEXT"),
(
"device_inventory.status_live",
"ALTER TABLE device_inventory ADD COLUMN status_live TEXT DEFAULT 'unknown'",
),
("device_inventory.last_seen", "ALTER TABLE device_inventory ADD COLUMN last_seen DATETIME"),
("device_inventory.last_scan", "ALTER TABLE device_inventory ADD COLUMN last_scan DATETIME"),
(
"device_inventory.response_time_ms",
"ALTER TABLE device_inventory ADD COLUMN response_time_ms INTEGER",
),
("device_inventory.updated_at", "ALTER TABLE device_inventory ADD COLUMN updated_at DATETIME"),
]
for label, sql in inventory_device_migrations:
await _try_migrate(conn, sql, label=label)
# Backfill the columns that carry a non-NULL default in the model, so a
# legacy row round-trips through Pydantic without tripping the validators.
# The link itself: a node points at the inventory row it draws.
for label, sql in (
("nodes.device_id", "ALTER TABLE nodes ADD COLUMN device_id TEXT"),
(
"nodes.device_id.index",
"CREATE INDEX IF NOT EXISTS ix_nodes_device_id ON nodes(device_id)",
),
# Which of the row's services and properties this canvas shows, and
# in what order. Seeded from the row further down, once the backfill
# has had its say — see `_seed_node_views`.
("nodes.display_view", "ALTER TABLE nodes ADD COLUMN display_view JSON"),
):
await _try_migrate(conn, sql, label=label)
for label, sql in (
(
"device_inventory.status_live.backfill",
"UPDATE device_inventory SET status_live = 'unknown' WHERE status_live IS NULL",
),
(
"device_inventory.show_hardware.backfill",
"UPDATE device_inventory SET show_hardware = 0 WHERE show_hardware IS NULL",
),
(
"device_inventory.updated_at.backfill",
"UPDATE device_inventory SET updated_at = discovered_at WHERE updated_at IS NULL",
),
):
await _try_migrate(conn, sql, label=label)
await _backfill_node_devices()
await _drop_legacy_node_columns()
await _seed_node_views()
# Columns `nodes` carried before 3.3.0, when a node owned the device facts. They
# belong to `device_inventory` now; the backfill above copies them across, and
# this rebuild removes them.
_LEGACY_NODE_COLUMNS = (
"hostname", "ip", "mac", "os", "status", "check_method", "check_target",
"services", "notes", "cpu_count", "cpu_model", "ram_gb", "disk_gb",
"show_hardware", "properties", "ieee_address", "last_seen", "last_scan",
"response_time_ms",
)
# What a node keeps: how the device is drawn on one canvas.
_NODE_COLUMNS_SQL = (
"id VARCHAR PRIMARY KEY,"
"type VARCHAR NOT NULL,"
"label VARCHAR NOT NULL,"
"design_id VARCHAR REFERENCES designs(id) ON DELETE SET NULL,"
"device_id VARCHAR REFERENCES device_inventory(id) ON DELETE SET NULL,"
"display_view JSON,"
"pos_x FLOAT,"
"pos_y FLOAT,"
"parent_id VARCHAR REFERENCES nodes(id) ON DELETE CASCADE,"
"container_mode BOOLEAN,"
"custom_colors JSON,"
"custom_icon VARCHAR,"
"show_port_numbers BOOLEAN,"
"width FLOAT,"
"height FLOAT,"
"bottom_handles INTEGER,"
"top_handles INTEGER,"
"left_handles INTEGER,"
"right_handles INTEGER,"
"created_at DATETIME,"
"updated_at DATETIME"
)
_NODE_KEPT = (
"id, type, label, design_id, device_id, display_view, pos_x, pos_y, parent_id, container_mode, "
"custom_colors, custom_icon, show_port_numbers, width, height, bottom_handles, "
"top_handles, left_handles, right_handles, created_at, updated_at"
)
async def _relax_legacy_node_columns(conn: AsyncConnection, info: list[Any]) -> None:
"""Make the retained legacy `nodes` columns nullable — SQLite table rebuild.
The 3.2.0 schema declares `status`, `services`, `properties` and
`show_hardware` NOT NULL with no server-side default. The 3.3.0 model no
longer maps them, so every INSERT omits them and SQLite rejects the row —
approving a device, creating a node, importing a canvas all fail with
``NOT NULL constraint failed: nodes.status``. Dropping the columns is the
real fix, but it waits on a complete backfill; until then they have to stop
blocking writes. Values are preserved: only the constraint goes.
"""
kept = [row for row in info if row[1] in _LEGACY_NODE_COLUMNS]
if not any(row[3] for row in kept): # PRAGMA `notnull`
return # Already relaxed, or never constrained.
logger.info("Relaxing NOT NULL on the retained legacy node columns")
legacy_defs = ",".join(f"{row[1]} {row[2] or 'VARCHAR'}" for row in kept)
legacy_names = ", ".join(row[1] for row in kept)
await _rebuild_nodes(
conn,
columns_sql=f"{_NODE_COLUMNS_SQL},{legacy_defs}",
copied=f"{_NODE_KEPT}, {legacy_names}",
what="nodes legacy-column relax",
)
async def _rebuild_nodes(conn: AsyncConnection, *, columns_sql: str, copied: str, what: str) -> None:
"""Recreate `nodes` with a new column list — SQLite cannot alter constraints.
Never fatal: a rebuild that fails leaves the table it could not replace, and
the boot carries on. Foreign keys go off for the swap, because `edges` and
`rack_devices` point at `nodes`, and back on in every case — the pragma is
per connection and this one returns to the pool.
"""
try:
await conn.exec_driver_sql("PRAGMA foreign_keys = OFF")
# A previous attempt that failed after the create would block this one.
await conn.exec_driver_sql("DROP TABLE IF EXISTS nodes_new")
await conn.exec_driver_sql(f"CREATE TABLE nodes_new ({columns_sql})")
await conn.exec_driver_sql(f"INSERT INTO nodes_new ({copied}) SELECT {copied} FROM nodes")
await conn.exec_driver_sql("DROP TABLE nodes")
await conn.exec_driver_sql("ALTER TABLE nodes_new RENAME TO nodes")
await conn.exec_driver_sql("CREATE INDEX IF NOT EXISTS ix_nodes_device_id ON nodes(device_id)")
except (OperationalError, IntegrityError) as exc:
logger.warning("%s failed: %s", what, exc)
finally:
await conn.exec_driver_sql("PRAGMA foreign_keys = ON")
async def _drop_legacy_node_columns() -> None:
"""Remove the device columns from `nodes` (3.3.0) — SQLite table rebuild.
Runs only after the backfill has linked every device node to its inventory
row. If any non-furniture node is still unlinked the drop is skipped and
logged: the columns are the only remaining copy of that node's facts, and
losing them is not recoverable. The skip then relaxes their NOT NULL instead,
because the 3.3.0 model no longer writes them and the database must stay
insertable while the backfill is retried on later boots.
"""
async with engine.begin() as conn:
info = (await conn.exec_driver_sql("PRAGMA table_info(nodes)")).fetchall()
present = {row[1] for row in info}
if not (present & set(_LEGACY_NODE_COLUMNS)):
return # Already migrated.
unlinked = (
await conn.exec_driver_sql(
"SELECT COUNT(*) FROM nodes WHERE device_id IS NULL "
"AND type NOT IN ('group', 'groupRect', 'text')"
)
).scalar()
if unlinked:
logger.warning(
"Keeping the legacy node columns: %d node(s) have no inventory row. "
"The backfill must link every device node before they can be dropped.",
unlinked,
)
await _relax_legacy_node_columns(conn, list(info))
return
logger.info("Migrating nodes: the device columns move to device_inventory")
await _rebuild_nodes(
conn,
columns_sql=_NODE_COLUMNS_SQL,
copied=_NODE_KEPT,
what="nodes device-column drop",
)
async def _backfill_node_devices() -> None:
"""Link existing canvas nodes to their Device Inventory rows (3.3.0).
Runs outside the DDL connection because the merge is Python-side, not SQL.
Self-limiting: it only looks at nodes with no ``device_id``, so a second boot
finds nothing to do. Never fatal — a failure here leaves the canvas working
off its own columns, which still carry the data at this point.
"""
# Imported here: app.services imports app.db.models, which imports this module.
from app.services.inventory_sync import backfill_node_devices
try:
async with AsyncSessionLocal() as session:
stats = await backfill_node_devices(session)
if stats["linked"]:
await session.commit()
logger.info(
"Inventory backfill: linked %d node(s) — %d device(s) created, %d merged",
stats["linked"], stats["created"], stats["merged"],
)
if stats.get("skipped"):
logger.warning(
"Inventory backfill: %d node(s) could not be linked; they keep their "
"legacy columns and are retried on the next start.",
stats["skipped"],
)
except Exception as exc: # pragma: no cover - defensive, boot must not die
logger.warning("Inventory backfill failed: %s", exc)
def _pre_split_backup() -> Path | None:
"""The newest backup still holding the per-node device columns, if any.
`_backup_db` copies the database *before* the migrations of each new
version, so a user who upgraded to 3.3.0 has a `homelab.db.back-3.3.0`
carrying the last state in which `nodes` still owned its own services and
properties. That copy is the only record of which canvas showed what, since
3.3.0's backfill unioned them all onto one inventory row. Newest first: it is
the state closest to the upgrade, so it is what the user last saw.
"""
db_path = Path(settings.sqlite_path)
candidates = sorted(
db_path.parent.glob(f"{db_path.name}.back-*"),
key=lambda p: p.stat().st_mtime,
reverse=True,
)
for path in candidates:
try:
with closing(sqlite3.connect(f"file:{path}?mode=ro", uri=True)) as conn:
cols = {row[1] for row in conn.execute("PRAGMA table_info(nodes)")}
except sqlite3.Error:
continue
if {"services", "properties"} <= cols:
return path
return None
def _views_from_backup() -> dict[str, dict[str, Any]]:
"""What each node drew, read out of the pre-3.3.0 backup. Empty when there is none.
Keyed by node id, which is a uuid and stable across every version. A backup
that cannot be opened, or holds nodes this database no longer has, simply
contributes nothing — the caller falls back to the inventory row.
"""
path = _pre_split_backup()
if path is None:
return {}
try:
with closing(sqlite3.connect(f"file:{path}?mode=ro", uri=True)) as conn:
rows = conn.execute("SELECT id, services, properties FROM nodes").fetchall()
except sqlite3.Error as exc:
logger.warning("Could not read the pre-3.3.0 node lists from %s: %s", path.name, exc)
return {}
out: dict[str, dict[str, Any]] = {}
for node_id, services, properties in rows:
drawn: dict[str, Any] = {}
for kind, raw in (("services", services), ("properties", properties)):
if isinstance(raw, str | bytes):
with suppress(ValueError):
decoded = _json.loads(raw)
if isinstance(decoded, list):
drawn[kind] = decoded
if drawn:
out[node_id] = drawn
if out:
logger.info("Recovering the per-canvas service/property layout from %s", path.name)
return out
async def _seed_node_views() -> None:
"""Give pre-existing nodes an explicit view of their inventory row (3.3.3).
Order and visibility for services and properties moved to the node, so one
device drawn on two canvases can be rendered two ways. A node from before
that has no view, and where it comes from decides whether the user gets
their arrangement back:
* upgrading from 3.2.0, the backfill has already seeded it from the node's
own columns — nothing here to do;
* upgrading from 3.3.0-3.3.2, those columns are gone and the row holds the
union of every canvas, so the view is recovered from the backup taken
before the 3.3.0 migration. That is the difference between a canvas coming
back as the user left it and coming back showing every other canvas'
properties;
* with no usable backup, the row itself is the seed: what shows today keeps
showing, and only what the row gains *later* is held back.
Never fatal: without a view the node simply shows the whole row, which is
the behaviour it has now.
"""
# Imported here: app.services imports app.db.models, which imports this module.
from app.services.inventory_sync import seed_node_views
try:
async with AsyncSessionLocal() as session:
seeded = await seed_node_views(session, drawn=_views_from_backup)
if seeded:
await session.commit()
logger.info("Seeded the service/property view of %d node(s)", seeded)
except Exception as exc: # pragma: no cover - defensive, boot must not die
logger.warning("Seeding the node service/property views failed: %s", exc)
async def get_db() -> AsyncGenerator[AsyncSession, None]:
+226 -21
View File
@@ -16,39 +16,63 @@ def _uuid() -> str:
return str(uuid.uuid4())
class Design(Base):
__tablename__ = "designs"
id: Mapped[str] = mapped_column(String, primary_key=True, default=_uuid)
name: Mapped[str] = mapped_column(String, nullable=False)
design_type: Mapped[str] = mapped_column(String, nullable=False, default="network")
icon: Mapped[str | None] = mapped_column(String, nullable=True, default="dashboard")
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=_now)
updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=_now, onupdate=_now)
class Node(Base):
"""How a device is drawn on one canvas.
A node owns presentation only — position, size, colours, icon, handles,
nesting. What the device *is* (addresses, services, properties, notes,
hardware, check method, live status) belongs to the `device_inventory` row
named by `device_id`, so one device reads the same on every canvas. The API
still reports both together: see `services/inventory_sync.hydrated_node`.
"""
__tablename__ = "nodes"
id: Mapped[str] = mapped_column(String, primary_key=True, default=_uuid)
type: Mapped[str] = mapped_column(String, nullable=False)
label: Mapped[str] = mapped_column(String, nullable=False)
hostname: Mapped[str | None] = mapped_column(String)
ip: Mapped[str | None] = mapped_column(String)
mac: Mapped[str | None] = mapped_column(String)
os: Mapped[str | None] = mapped_column(String)
status: Mapped[str] = mapped_column(String, default="unknown")
check_method: Mapped[str | None] = mapped_column(String)
check_target: Mapped[str | None] = mapped_column(String)
services: Mapped[list[Any]] = mapped_column(JSON, default=list)
notes: Mapped[str | None] = mapped_column(Text)
design_id: Mapped[str | None] = mapped_column(String, ForeignKey("designs.id", ondelete="SET NULL"), nullable=True)
# The Device Inventory row this node draws. NULL for canvas furniture
# (group / groupRect / text), which represents nothing physical. Deleting a
# node never deletes the device — the inventory outlives every canvas.
device_id: Mapped[str | None] = mapped_column(
String, ForeignKey("device_inventory.id", ondelete="SET NULL"), index=True, nullable=True
)
# How this canvas renders the device's list facts: which services and which
# properties it shows, and in what order. The facts themselves stay on the
# inventory row, so the same device drawn on two canvases can show two
# different subsets — a scanner-guessed service on one, none on the other.
# {"services": [{"key": "443|tcp|https", "visible": true}, …],
# "properties": [{"key": "rack", "visible": false}, …]}
# NULL only for canvas furniture and for a node with no inventory row yet;
# `inventory_sync.link_facts` fills both lists as soon as there is one.
display_view: Mapped[dict[str, Any] | None] = mapped_column(JSON, nullable=True)
pos_x: Mapped[float] = mapped_column(Float, default=0)
pos_y: Mapped[float] = mapped_column(Float, default=0)
parent_id: Mapped[str | None] = mapped_column(String, ForeignKey("nodes.id", ondelete="CASCADE"))
container_mode: Mapped[bool] = mapped_column(Boolean, default=False)
custom_colors: Mapped[dict[str, Any] | None] = mapped_column(JSON, nullable=True)
custom_icon: Mapped[str | None] = mapped_column(String, nullable=True)
cpu_count: Mapped[int | None] = mapped_column(Integer, nullable=True)
cpu_model: Mapped[str | None] = mapped_column(String, nullable=True)
ram_gb: Mapped[float | None] = mapped_column(Float, nullable=True)
disk_gb: Mapped[float | None] = mapped_column(Float, nullable=True)
show_hardware: Mapped[bool] = mapped_column(Boolean, default=False)
show_port_numbers: Mapped[bool] = mapped_column(Boolean, default=False)
width: Mapped[float | None] = mapped_column(Float, nullable=True)
height: Mapped[float | None] = mapped_column(Float, nullable=True)
last_seen: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
response_time_ms: Mapped[int | None] = mapped_column(Integer)
bottom_handles: Mapped[int] = mapped_column(Integer, default=1)
top_handles: Mapped[int] = mapped_column(Integer, default=1)
left_handles: Mapped[int] = mapped_column(Integer, default=0)
right_handles: Mapped[int] = mapped_column(Integer, default=0)
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=_now)
updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=_now, onupdate=_now)
children: Mapped[list["Node"]] = relationship("Node", back_populates="parent")
parent: Mapped["Node | None"] = relationship("Node", back_populates="children", remote_side=[id])
@@ -59,37 +83,217 @@ class Edge(Base):
id: Mapped[str] = mapped_column(String, primary_key=True, default=_uuid)
source: Mapped[str] = mapped_column(String, ForeignKey("nodes.id", ondelete="CASCADE"))
target: Mapped[str] = mapped_column(String, ForeignKey("nodes.id", ondelete="CASCADE"))
design_id: Mapped[str | None] = mapped_column(String, ForeignKey("designs.id", ondelete="SET NULL"), nullable=True)
type: Mapped[str] = mapped_column(String, default="ethernet")
label: Mapped[str | None] = mapped_column(String)
vlan_id: Mapped[int | None] = mapped_column(Integer)
speed: Mapped[str | None] = mapped_column(String)
custom_color: Mapped[str | None] = mapped_column(String)
path_style: Mapped[str | None] = mapped_column(String)
line_style: Mapped[str | None] = mapped_column(String)
width_mult: Mapped[float | None] = mapped_column(Float)
animated: Mapped[str] = mapped_column(String, nullable=False, default='none')
marker_start: Mapped[str] = mapped_column(String, nullable=False, default='none')
marker_end: Mapped[str] = mapped_column(String, nullable=False, default='none')
source_handle: Mapped[str | None] = mapped_column(String)
target_handle: Mapped[str | None] = mapped_column(String)
waypoints: Mapped[list[dict[str, float]] | None] = mapped_column(JSON, nullable=True)
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=_now)
class CanvasState(Base):
__tablename__ = "canvas_state"
id: Mapped[int] = mapped_column(Integer, primary_key=True, default=1)
design_id: Mapped[str] = mapped_column(String, ForeignKey("designs.id", ondelete="CASCADE"), primary_key=True)
viewport: Mapped[dict[str, Any]] = mapped_column(JSON, default=dict)
custom_style: Mapped[dict[str, Any] | None] = mapped_column(JSON, nullable=True)
saved_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=_now)
class PendingDevice(Base):
__tablename__ = "pending_devices"
class Rack(Base):
"""A physical rack on a `design_type='rack'` canvas.
Racks live per design, like nodes and edges. Geometry is expressed in rack
units: `u_height` is the number of mountable U, and a device's `u_start` is
always counted from the bottom rail — `numbering` only changes the printed
labels.
"""
__tablename__ = "racks"
id: Mapped[str] = mapped_column(String, primary_key=True, default=_uuid)
ip: Mapped[str] = mapped_column(String, nullable=False)
design_id: Mapped[str] = mapped_column(String, ForeignKey("designs.id", ondelete="CASCADE"), nullable=False)
name: Mapped[str] = mapped_column(String, nullable=False)
u_height: Mapped[int] = mapped_column(Integer, default=42)
# "19" or "10" (inches). Drives the drawn inner width.
width_standard: Mapped[str] = mapped_column(String, default="19")
# "bottom-up" or "top-down" — printed U labels only, never storage order.
numbering: Mapped[str] = mapped_column(String, default="bottom-up")
location: Mapped[str | None] = mapped_column(String, nullable=True)
# Frame/rail/interior colours + showNumbers/enclosed flags.
style: Mapped[dict[str, Any]] = mapped_column(JSON, default=dict)
pos_x: Mapped[float] = mapped_column(Float, default=0)
pos_y: Mapped[float] = mapped_column(Float, default=0)
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=_now)
updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=_now, onupdate=_now)
class RackDevice(Base):
"""A piece of gear mounted in a rack.
Two independent, both-optional links back to the rest of the app:
* `device_id` — the Device Inventory entry (`device_inventory`). This is the
primary link: inventory rows survive approval *and* node deletion, so
unracking or deleting a canvas node never removes the inventory entry.
* `node_id` — the logical-canvas node, when one exists. Only used to resolve
live status and to seed cables from the network design's edges.
Both are ``SET NULL`` on delete and `label` is denormalized, so a rack keeps
rendering after an inventory purge.
"""
__tablename__ = "rack_devices"
id: Mapped[str] = mapped_column(String, primary_key=True, default=_uuid)
design_id: Mapped[str] = mapped_column(String, ForeignKey("designs.id", ondelete="CASCADE"), nullable=False)
rack_id: Mapped[str] = mapped_column(String, ForeignKey("racks.id", ondelete="CASCADE"), nullable=False)
device_id: Mapped[str | None] = mapped_column(
String, ForeignKey("device_inventory.id", ondelete="SET NULL"), nullable=True
)
node_id: Mapped[str | None] = mapped_column(String, ForeignKey("nodes.id", ondelete="SET NULL"), nullable=True)
label: Mapped[str] = mapped_column(String, nullable=False)
# 1-based, counted from the bottom rail.
u_start: Mapped[int] = mapped_column(Integer, default=1)
u_height: Mapped[int] = mapped_column(Integer, default=1)
# 12-column horizontal grid: full = 12, half = 6, third = 4, quarter = 3.
col_start: Mapped[int] = mapped_column(Integer, default=0)
col_span: Mapped[int] = mapped_column(Integer, default=12)
faceplate_id: Mapped[str] = mapped_column(String, nullable=False, default="blank-1u")
color: Mapped[str | None] = mapped_column(String, nullable=True)
status: Mapped[str] = mapped_column(String, default="unknown")
# [{id, label, type, x, y}] — positions are unit coordinates on the plate.
ports: Mapped[list[Any]] = mapped_column(JSON, default=list)
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=_now)
updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=_now, onupdate=_now)
class RackCable(Base):
"""A patch between two rack device ports.
Not an `Edge`: rack cables are port-to-port and may cross racks, so they are
their own relation rather than a canvas edge.
"""
__tablename__ = "rack_cables"
id: Mapped[str] = mapped_column(String, primary_key=True, default=_uuid)
design_id: Mapped[str] = mapped_column(String, ForeignKey("designs.id", ondelete="CASCADE"), nullable=False)
from_device_id: Mapped[str] = mapped_column(
String, ForeignKey("rack_devices.id", ondelete="CASCADE"), nullable=False
)
from_port_id: Mapped[str] = mapped_column(String, nullable=False)
to_device_id: Mapped[str] = mapped_column(
String, ForeignKey("rack_devices.id", ondelete="CASCADE"), nullable=False
)
to_port_id: Mapped[str] = mapped_column(String, nullable=False)
type: Mapped[str] = mapped_column(String, default="ethernet")
color: Mapped[str] = mapped_column(String, default="#39d353")
label: Mapped[str | None] = mapped_column(String, nullable=True)
# Print the label next to the run on the canvas.
label_visible: Mapped[bool] = mapped_column(Boolean, default=False)
# [{key, value, icon, visible}] — same records nodes carry; the visible ones
# are drawn beside the cable.
properties: Mapped[list[Any]] = mapped_column(JSON, default=list)
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=_now)
class InventoryDevice(Base):
__tablename__ = "device_inventory"
# Permit the plain (non-Mapped[]) annotations on the transient request-only
# attributes below; without this SQLAlchemy 2.0 tries to map them as columns.
__allow_unmapped__ = True
id: Mapped[str] = mapped_column(String, primary_key=True, default=_uuid)
ip: Mapped[str | None] = mapped_column(String, nullable=True)
mac: Mapped[str | None] = mapped_column(String)
hostname: Mapped[str | None] = mapped_column(String)
os: Mapped[str | None] = mapped_column(String)
services: Mapped[list[Any]] = mapped_column(JSON, default=list)
suggested_type: Mapped[str | None] = mapped_column(String)
status: Mapped[str] = mapped_column(String, default="pending")
# Origin/primary source (first discovery): "arp"/"mdns"/"zigbee"/"zwave"/
# "proxmox". Kept for back-compat; `discovery_sources` is the full set.
discovery_source: Mapped[str | None] = mapped_column(String)
# All sources that have observed this device. A device found by both an IP
# scan and a Proxmox import carries e.g. ["arp", "proxmox"] and shows under
# both inventory filters. Source of truth for the frontend source badges.
discovery_sources: Mapped[list[Any]] = mapped_column(JSON, default=list)
ieee_address: Mapped[str | None] = mapped_column(String, index=True, nullable=True, unique=True)
friendly_name: Mapped[str | None] = mapped_column(String, nullable=True)
device_subtype: Mapped[str | None] = mapped_column(String, nullable=True)
model: Mapped[str | None] = mapped_column(String, nullable=True)
vendor: Mapped[str | None] = mapped_column(String, nullable=True)
lqi: Mapped[int | None] = mapped_column(Integer, nullable=True)
# Display properties carried from discovery (e.g. Proxmox specs: CPU/RAM/Disk,
# VMID). Generic NodeProperty shape {key,value,icon,visible}; merged into the
# Node's properties on approve. Empty for scan/mesh sources that don't set it.
properties: Mapped[list[Any]] = mapped_column(JSON, default=list)
discovered_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=_now)
# --- Curated device facts (3.3.0) -------------------------------------
# The inventory row, not the canvas node, owns what the device *is*. A node
# only says how it is drawn. `label`/`type` supersede friendly_name/
# suggested_type for a device that reached a canvas; the older pair is kept
# so discovery imports and the source filters keep working unchanged.
label: Mapped[str | None] = mapped_column(String, nullable=True)
type: Mapped[str | None] = mapped_column(String, nullable=True)
notes: Mapped[str | None] = mapped_column(Text, nullable=True)
cpu_count: Mapped[int | None] = mapped_column(Integer, nullable=True)
cpu_model: Mapped[str | None] = mapped_column(String, nullable=True)
ram_gb: Mapped[float | None] = mapped_column(Float, nullable=True)
disk_gb: Mapped[float | None] = mapped_column(Float, nullable=True)
show_hardware: Mapped[bool] = mapped_column(Boolean, default=False)
check_method: Mapped[str | None] = mapped_column(String, nullable=True)
check_target: Mapped[str | None] = mapped_column(String, nullable=True)
# Live reachability from the status checker. NOT `status` — that column holds
# the inventory lifecycle (pending/approved/hidden) and the two must not be
# conflated.
status_live: Mapped[str] = mapped_column(String, default="unknown")
last_seen: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
last_scan: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
response_time_ms: Mapped[int | None] = mapped_column(Integer, nullable=True)
updated_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), default=_now, onupdate=_now
)
# Transient (not persisted): populated per-request by the scan routes to report
# how many canvases this device already appears on. Not a mapped column.
canvas_count: int = 0
# Transient (not persisted): timestamps from the linked canvas node(s),
# correlated by ip / ieee_address. None when the device is not on any canvas.
node_created_at: datetime | None = None
node_last_scan: datetime | None = None
node_last_modified: datetime | None = None
node_last_seen: datetime | None = None
class InventoryDeviceLink(Base):
"""Link between two Zigbee endpoints discovered during import.
Endpoints are addressed by IEEE (stable across re-imports). Either side may
already exist as a canvas Node (resolved via Node.ieee_address) or still be
a InventoryDevice. On approval, the matching Edge is auto-created when both
endpoints exist as canvas Nodes.
"""
__tablename__ = "device_inventory_links"
id: Mapped[str] = mapped_column(String, primary_key=True, default=_uuid)
source_ieee: Mapped[str] = mapped_column(String, nullable=False, index=True)
target_ieee: Mapped[str] = mapped_column(String, nullable=False, index=True)
lqi: Mapped[int | None] = mapped_column(Integer, nullable=True)
discovery_source: Mapped[str] = mapped_column(String, nullable=False, default="zigbee")
discovered_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=_now)
@@ -98,6 +302,7 @@ class ScanRun(Base):
id: Mapped[str] = mapped_column(String, primary_key=True, default=_uuid)
status: Mapped[str] = mapped_column(String, default="running")
kind: Mapped[str] = mapped_column(String, default="ip", server_default="ip")
ranges: Mapped[list[str]] = mapped_column(JSON, default=list)
devices_found: Mapped[int] = mapped_column(Integer, default=0)
started_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=_now)
+48 -3
View File
@@ -1,19 +1,48 @@
import logging
import logging.config
from collections.abc import AsyncGenerator
from contextlib import asynccontextmanager
from typing import Any
from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware
from starlette.middleware.sessions import SessionMiddleware
from app.api.routes import auth, canvas, edges, liveview, nodes, scan, status
from app.api.routes import (
auth,
canvas,
designs,
edges,
liveview,
media,
nodes,
proxmox,
racks,
scan,
stats,
status,
zigbee,
zwave,
)
from app.api.routes import settings as settings_routes
from app.core.config import settings
from app.core.scheduler import start_scheduler, stop_scheduler
from app.core.security import OIDCCSRFMiddleware
from app.db.database import init_db
@asynccontextmanager
async def lifespan(app: FastAPI) -> AsyncGenerator[None, None]:
# Ensure app logs are visible: attach a handler to the root logger if none
# exists (uvicorn only installs handlers on its own loggers, not the root).
root_logger = logging.getLogger()
if not root_logger.handlers:
handler = logging.StreamHandler()
handler.setFormatter(logging.Formatter("%(levelname)s:%(name)s:%(message)s"))
root_logger.addHandler(handler)
root_logger.setLevel(logging.INFO)
logging.getLogger("app").setLevel(logging.INFO)
logging.getLogger("app.services.scanner").setLevel(logging.INFO)
await init_db()
settings.load_overrides()
start_scheduler()
@@ -23,26 +52,42 @@ async def lifespan(app: FastAPI) -> AsyncGenerator[None, None]:
app = FastAPI(
title="Homelable API",
version="1.4.0",
version="1.9.0",
lifespan=lifespan,
)
app.add_middleware(OIDCCSRFMiddleware)
app.add_middleware(
SessionMiddleware,
secret_key=settings.secret_key,
session_cookie=("__Host-homelable-oidc-flow" if settings.oidc_cookie_secure else "homelable-oidc-flow"),
max_age=settings.oidc_transaction_expire_seconds,
same_site="lax",
https_only=settings.oidc_cookie_secure,
)
app.add_middleware(
CORSMiddleware,
allow_origins=settings.cors_origins,
allow_credentials=True,
allow_methods=["GET", "POST", "PUT", "PATCH", "DELETE"],
allow_headers=["Authorization", "Content-Type"],
allow_headers=["Authorization", "Content-Type", "X-Homelable-CSRF", "X-MCP-Service-Key"],
)
app.include_router(auth.router, prefix="/api/v1/auth", tags=["auth"])
app.include_router(nodes.router, prefix="/api/v1/nodes", tags=["nodes"])
app.include_router(edges.router, prefix="/api/v1/edges", tags=["edges"])
app.include_router(canvas.router, prefix="/api/v1/canvas", tags=["canvas"])
app.include_router(designs.router, prefix="/api/v1/designs", tags=["designs"])
app.include_router(racks.router, prefix="/api/v1/racks", tags=["racks"])
app.include_router(scan.router, prefix="/api/v1/scan", tags=["scan"])
app.include_router(status.router, prefix="/api/v1/status", tags=["status"])
app.include_router(settings_routes.router, prefix="/api/v1/settings", tags=["settings"])
app.include_router(liveview.router, prefix="/api/v1/liveview", tags=["liveview"])
app.include_router(zigbee.router, prefix="/api/v1/zigbee", tags=["zigbee"])
app.include_router(zwave.router, prefix="/api/v1/zwave", tags=["zwave"])
app.include_router(proxmox.router, prefix="/api/v1/proxmox", tags=["proxmox"])
app.include_router(stats.router, prefix="/api/v1/stats", tags=["stats"])
app.include_router(media.router, prefix="/api/v1/media", tags=["media"])
@app.get("/api/v1/health")
+36 -1
View File
@@ -4,13 +4,24 @@ from pydantic import BaseModel, field_validator
from app.schemas.edges import EdgeResponse
from app.schemas.nodes import NodeResponse
from app.schemas.utils import normalize_animated
from app.schemas.utils import normalize_animated, normalize_marker
class NodeSave(BaseModel):
id: str
type: str
label: str
# Round-tripped so a save keeps the node pointed at its inventory row. The
# device fields below are still accepted at their canvas values and routed
# to that row server-side.
device_id: str | None = None
# Which device facts this canvas edited since it loaded them, if the client
# tracked that. The payload always carries a *full* copy of the device — it
# was hydrated from the inventory row on load — so without this a save made
# for nothing but a moved node would rewrite the row from a snapshot that may
# be hours old. `None` (an older client, an import, the MCP server) keeps the
# previous behaviour: every fact is a candidate write.
changed_facts: list[str] | None = None
hostname: str | None = None
ip: str | None = None
mac: str | None = None
@@ -29,8 +40,14 @@ class NodeSave(BaseModel):
ram_gb: float | None = None
disk_gb: float | None = None
show_hardware: bool = False
show_port_numbers: bool = False
properties: list[Any] = []
width: float | None = None
height: float | None = None
bottom_handles: int = 1
top_handles: int = 1
left_handles: int = 0
right_handles: int = 0
pos_x: float = 0
pos_y: float = 0
@@ -45,23 +62,41 @@ class EdgeSave(BaseModel):
speed: str | None = None
custom_color: str | None = None
path_style: str | None = None
line_style: str | None = None
width_mult: float | None = None
animated: str = 'none'
marker_start: str = 'none'
marker_end: str = 'none'
source_handle: str | None = None
target_handle: str | None = None
waypoints: list[dict[str, float]] | None = None
@field_validator('animated', mode='before')
@classmethod
def validate_animated(cls, v: object) -> str:
return normalize_animated(v)
@field_validator('marker_start', 'marker_end', mode='before')
@classmethod
def validate_marker(cls, v: object) -> str:
return normalize_marker(v)
class CanvasSaveRequest(BaseModel):
nodes: list[NodeSave] = []
edges: list[EdgeSave] = []
viewport: dict[str, Any] = {}
custom_style: dict[str, Any] | None = None
design_id: str | None = None
class CanvasStateResponse(BaseModel):
nodes: list[NodeResponse]
edges: list[EdgeResponse]
viewport: dict[str, Any]
custom_style: dict[str, Any] | None = None
# True once this design's canvas has ever been persisted (a CanvasState row
# exists). Lets the frontend tell a brand-new user (show demo) apart from one
# who intentionally cleared their canvas (keep it empty). False also for a
# missing/uninitialized design.
initialized: bool = False
+51
View File
@@ -0,0 +1,51 @@
from datetime import datetime
from pydantic import BaseModel, field_validator
# Kept in sync with DesignType in frontend/src/types.
# "network" and "electrical" render the same React Flow canvas and differ only by
# palette and node set — the icon drives their presentation. "rack" is a genuinely
# different renderer (racks, mounted gear, port-to-port cables), so the frontend
# does branch on this field.
DESIGN_TYPES = {"network", "electrical", "rack"}
class DesignCreate(BaseModel):
name: str
icon: str = "dashboard"
design_type: str = "network"
@field_validator("design_type")
@classmethod
def _known_type(cls, v: str) -> str:
if v not in DESIGN_TYPES:
raise ValueError(f"design_type must be one of {sorted(DESIGN_TYPES)}")
return v
class DesignUpdate(BaseModel):
name: str | None = None
icon: str | None = None
class DesignCopy(BaseModel):
"""Create a new design by deep-copying an existing one's canvas."""
name: str
icon: str = "dashboard"
class DesignResponse(BaseModel):
id: str
name: str
design_type: str
icon: str | None = None
created_at: datetime
updated_at: datetime
# Populated by list_designs so the "copy from existing" picker can show what
# each canvas holds. None on create/update/copy responses (not computed there).
node_count: int | None = None
group_count: int | None = None
text_count: int | None = None
model_config = {"from_attributes": True}
+25 -2
View File
@@ -2,7 +2,7 @@ from datetime import datetime
from pydantic import BaseModel, field_validator
from app.schemas.utils import normalize_animated
from app.schemas.utils import normalize_animated, normalize_marker
class EdgeBase(BaseModel):
@@ -14,18 +14,28 @@ class EdgeBase(BaseModel):
speed: str | None = None
custom_color: str | None = None
path_style: str | None = None
line_style: str | None = None
width_mult: float | None = None
animated: str = 'none'
marker_start: str = 'none'
marker_end: str = 'none'
source_handle: str | None = None
target_handle: str | None = None
waypoints: list[dict[str, float]] | None = None
@field_validator('animated', mode='before')
@classmethod
def validate_animated(cls, v: object) -> str:
return normalize_animated(v)
@field_validator('marker_start', 'marker_end', mode='before')
@classmethod
def validate_marker(cls, v: object) -> str:
return normalize_marker(v)
class EdgeCreate(EdgeBase):
pass
design_id: str | None = None
class EdgeUpdate(BaseModel):
@@ -35,9 +45,14 @@ class EdgeUpdate(BaseModel):
speed: str | None = None
custom_color: str | None = None
path_style: str | None = None
line_style: str | None = None
width_mult: float | None = None
animated: str | None = None
marker_start: str | None = None
marker_end: str | None = None
source_handle: str | None = None
target_handle: str | None = None
waypoints: list[dict[str, float]] | None = None
@field_validator('animated', mode='before')
@classmethod
@@ -46,9 +61,17 @@ class EdgeUpdate(BaseModel):
return None
return normalize_animated(v)
@field_validator('marker_start', 'marker_end', mode='before')
@classmethod
def validate_marker(cls, v: object) -> str | None:
if v is None:
return None
return normalize_marker(v)
class EdgeResponse(EdgeBase):
id: str
design_id: str | None = None
created_at: datetime
model_config = {"from_attributes": True}
+28 -1
View File
@@ -27,12 +27,26 @@ class NodeBase(BaseModel):
ram_gb: float | None = None
disk_gb: float | None = None
show_hardware: bool = False
show_port_numbers: bool = False
properties: list[dict[str, Any]] = []
width: float | None = None
height: float | None = None
bottom_handles: int = 1
top_handles: int = 1
left_handles: int = 0
right_handles: int = 0
class NodeCreate(NodeBase):
pass
# Override pos_x/pos_y so callers can omit them; None signals "auto-place".
# The create_node route resolves None to a free grid slot before persisting.
pos_x: float | None = None # type: ignore[assignment]
pos_y: float | None = None # type: ignore[assignment]
design_id: str | None = None
# When a node with the same ip/mac already exists on the target design, the
# create/approve endpoints reject with 409 so the UI can ask the user. Set
# force=True to bypass that guard and create the duplicate deliberately.
force: bool = False
class NodeUpdate(BaseModel):
@@ -58,13 +72,26 @@ class NodeUpdate(BaseModel):
ram_gb: float | None = None
disk_gb: float | None = None
show_hardware: bool | None = None
show_port_numbers: bool | None = None
properties: list[dict[str, Any]] | None = None
width: float | None = None
height: float | None = None
bottom_handles: int | None = None
top_handles: int | None = None
left_handles: int | None = None
right_handles: int | None = None
class NodeResponse(NodeBase):
id: str
design_id: str | None = None
# The Device Inventory row this node draws; None for canvas furniture. The
# device fields above are hydrated from that row on read, so the wire shape
# is unchanged even though the row, not the node, owns them.
device_id: str | None = None
ieee_address: str | None = None
last_seen: datetime | None = None
last_scan: datetime | None = None
response_time_ms: int | None = None
created_at: datetime
updated_at: datetime
+87
View File
@@ -0,0 +1,87 @@
"""Pydantic v2 schemas for Proxmox VE import.
Token fields are accepted on requests only and are optional — when omitted the
backend falls back to the server-configured token (env). No response schema ever
carries a token; secrets are kept out of responses by structural omission.
"""
from pydantic import BaseModel, Field
class ProxmoxConnectionRequest(BaseModel):
host: str = Field(..., description="Proxmox VE host or IP")
port: int = Field(8006, ge=1, le=65535, description="Proxmox API port")
token_id: str | None = Field(
None, description="API token id 'user@realm!tokenname' (falls back to server env)"
)
token_secret: str | None = Field(
None, description="API token secret (falls back to server env)"
)
verify_tls: bool = Field(True, description="Verify the Proxmox TLS certificate")
class ProxmoxTestConnectionResponse(BaseModel):
connected: bool
message: str
class ProxmoxNodeOut(BaseModel):
"""A homelable-ready node representation of a Proxmox host / VM / LXC."""
id: str
label: str
type: str # proxmox | vm | lxc
ieee_address: str
hostname: str | None = None
ip: str | None = None
status: str
cpu_count: int | None = None
ram_gb: float | None = None
disk_gb: float | None = None
vendor: str | None = None
model: str | None = None
parent_ieee: str | None = None
class ProxmoxEdgeOut(BaseModel):
source: str
target: str
class ProxmoxImportResponse(BaseModel):
nodes: list[ProxmoxNodeOut]
edges: list[ProxmoxEdgeOut]
device_count: int
class ProxmoxImportPendingResponse(BaseModel):
"""Result of importing a Proxmox inventory into the pending section."""
pending_created: int
pending_updated: int
links_recorded: int
device_count: int
class ProxmoxConfig(BaseModel):
"""Non-secret Proxmox connection + auto-sync config (GET response).
Connection fields (host/port/verify_tls) are env-only and read-only here —
surfaced for display. ``token_configured`` reflects whether a server-side
token is present. Never carries the token itself."""
host: str = ""
port: int = Field(8006, ge=1, le=65535)
verify_tls: bool = True
sync_enabled: bool = False
sync_interval: int = Field(3600, ge=300)
token_configured: bool = False
class ProxmoxSyncConfig(BaseModel):
"""User-editable auto-sync config (POST body). The ONLY persisted Proxmox
settings. Connection fields (host/port/token/verify_tls) are env-only and
are deliberately not accepted here."""
sync_enabled: bool = False
sync_interval: int = Field(3600, ge=300)
+302
View File
@@ -0,0 +1,302 @@
from datetime import datetime, timezone
from typing import Any
from pydantic import BaseModel, field_validator, model_validator
# Kept in sync with the frontend rack model (frontend/src/types).
WIDTH_STANDARDS = {"19", "10"}
NUMBERINGS = {"bottom-up", "top-down"}
CABLE_TYPES = {"ethernet", "fiber"}
# 12-column horizontal grid: full = 12, half = 6, third = 4, quarter = 3.
RACK_COLUMNS = 12
class RackSave(BaseModel):
id: str
name: str
u_height: int = 42
width_standard: str = "19"
numbering: str = "bottom-up"
location: str | None = None
style: dict[str, Any] = {}
pos_x: float = 0
pos_y: float = 0
@field_validator("u_height")
@classmethod
def _positive_height(cls, v: int) -> int:
if not 1 <= v <= 100:
raise ValueError("u_height must be between 1 and 100")
return v
@field_validator("width_standard")
@classmethod
def _known_width(cls, v: str) -> str:
if v not in WIDTH_STANDARDS:
raise ValueError(f"width_standard must be one of {sorted(WIDTH_STANDARDS)}")
return v
@field_validator("numbering")
@classmethod
def _known_numbering(cls, v: str) -> str:
if v not in NUMBERINGS:
raise ValueError(f"numbering must be one of {sorted(NUMBERINGS)}")
return v
class RackDeviceSave(BaseModel):
id: str
rack_id: str
device_id: str | None = None
node_id: str | None = None
label: str
u_start: int = 1
u_height: int = 1
col_start: int = 0
col_span: int = RACK_COLUMNS
faceplate_id: str = "blank-1u"
color: str | None = None
status: str = "unknown"
ports: list[Any] = []
@field_validator("u_start", "u_height")
@classmethod
def _positive(cls, v: int) -> int:
if v < 1:
raise ValueError("u_start and u_height are 1-based and must be >= 1")
return v
@field_validator("col_start")
@classmethod
def _column_in_grid(cls, v: int) -> int:
if not 0 <= v < RACK_COLUMNS:
raise ValueError(f"col_start must be within the {RACK_COLUMNS}-column grid")
return v
@field_validator("col_span")
@classmethod
def _span_in_grid(cls, v: int) -> int:
if not 1 <= v <= RACK_COLUMNS:
raise ValueError(f"col_span must be between 1 and {RACK_COLUMNS}")
return v
@model_validator(mode="after")
def _fits_the_column_grid(self) -> "RackDeviceSave":
"""`col_start` and `col_span` are legal apart and still illegal together.
11 + 12 spans to column 23 of a 12-column grid — each field passes its
own check, and the frontend serializer clamps them independently, so
nothing caught it before this.
"""
if self.col_start + self.col_span > RACK_COLUMNS:
raise ValueError(
f"col_start + col_span must not exceed the {RACK_COLUMNS}-column grid"
)
return self
class RackCableSave(BaseModel):
id: str
from_device_id: str
from_port_id: str
to_device_id: str
to_port_id: str
type: str = "ethernet"
color: str = "#39d353"
label: str | None = None
label_visible: bool = False
# [{key, value, icon, visible}] — free-form, same shape as node properties.
properties: list[dict[str, Any]] = []
@field_validator("type")
@classmethod
def _known_type(cls, v: str) -> str:
if v not in CABLE_TYPES:
raise ValueError(f"type must be one of {sorted(CABLE_TYPES)}")
return v
@field_validator("properties", mode="before")
@classmethod
def _clean_properties(cls, v: Any) -> list[dict[str, Any]]:
"""Keep only usable records — a property with no key draws an empty badge."""
if not isinstance(v, list):
return []
return [p for p in v if isinstance(p, dict) and str(p.get("key", "")).strip()]
class RackSaveRequest(BaseModel):
"""Full rack state for one design — upserted, with anything missing pruned.
Same contract as ``POST /api/v1/canvas/save``: the client owns the state and
sends all of it on an explicit Save.
"""
design_id: str
racks: list[RackSave] = []
devices: list[RackDeviceSave] = []
cables: list[RackCableSave] = []
# Pan/zoom, stored on the design's shared CanvasState row like the logical canvas.
viewport: dict[str, Any] = {}
@model_validator(mode="after")
def _devices_fit_their_rack(self) -> "RackSaveRequest":
"""A mount must fit inside the rack it names.
`RackDeviceSave` validates each field in isolation and never sees the
rack, so `u_start: 40` in a 12U rack passed. A mount above the top rail
draws outside the chassis and nothing in the UI can drag it back, so the
server refuses to store it. Rack membership itself is checked in the
route, which knows what is already persisted.
"""
heights = {r.id: r.u_height for r in self.racks}
for device in self.devices:
u_height = heights.get(device.rack_id)
if u_height is None:
continue # unknown rack — the route reports that with a 400
if device.u_start + device.u_height - 1 > u_height:
raise ValueError(
f"Device {device.id} does not fit in rack {device.rack_id}: "
f"U {device.u_start}–{device.u_start + device.u_height - 1} "
f"of {u_height}U"
)
return self
class RackResponse(BaseModel):
id: str
design_id: str
name: str
u_height: int
width_standard: str
numbering: str
location: str | None = None
style: dict[str, Any] = {}
pos_x: float
pos_y: float
@field_validator("style", mode="before")
@classmethod
def _coerce_style(cls, v: Any) -> dict[str, Any]:
return v if isinstance(v, dict) else {}
model_config = {"from_attributes": True}
class RackDeviceResponse(BaseModel):
id: str
design_id: str
rack_id: str
device_id: str | None = None
node_id: str | None = None
label: str
u_start: int
u_height: int
col_start: int
col_span: int
faceplate_id: str
color: str | None = None
status: str
ports: list[Any] = []
@field_validator("ports", mode="before")
@classmethod
def _coerce_ports(cls, v: Any) -> list[Any]:
return v if isinstance(v, list) else []
model_config = {"from_attributes": True}
class RackCableResponse(BaseModel):
id: str
design_id: str
from_device_id: str
from_port_id: str
to_device_id: str
to_port_id: str
type: str
color: str
label: str | None = None
label_visible: bool = False
properties: list[dict[str, Any]] = []
@field_validator("properties", mode="before")
@classmethod
def _list_properties(cls, v: Any) -> list[dict[str, Any]]:
"""Rows written before the column existed read back as NULL."""
return v if isinstance(v, list) else []
@field_validator("label_visible", mode="before")
@classmethod
def _bool_label_visible(cls, v: Any) -> bool:
return bool(v)
model_config = {"from_attributes": True}
class RackStateResponse(BaseModel):
racks: list[RackResponse]
devices: list[RackDeviceResponse]
cables: list[RackCableResponse]
viewport: dict[str, Any] = {}
class RackServiceInfo(BaseModel):
"""One service fingerprinted on a device, as the rack view prints it."""
port: int | None = None
name: str | None = None
class RackInventoryItem(BaseModel):
"""A Device Inventory entry offered to the rack tray.
Carries the technical facts the logical canvas already knows about the same
hardware (`node_*`), so a mount can print them without the user leaving the
rack. Everything node-side is read-only here: the logical canvas owns it.
"""
id: str
label: str
suggested_type: str | None = None
ip: str | None = None
status: str
discovery_source: str | None = None
# Inventory-side technical detail, from discovery.
mac: str | None = None
hostname: str | None = None
os: str | None = None
services: list[RackServiceInfo] = []
# Live status of the matching canvas node, when the device is on one.
node_id: str | None = None
node_status: str | None = None
# What that node holds — the logical view of the same box.
node_label: str | None = None
node_type: str | None = None
node_ip: str | None = None
node_mac: str | None = None
node_hostname: str | None = None
node_os: str | None = None
node_check_method: str | None = None
node_design_id: str | None = None
node_design_name: str | None = None
node_last_seen: datetime | None = None
# True when this device is already mounted somewhere in this design.
racked: bool = False
@field_validator("node_last_seen")
@classmethod
def _utc(cls, v: datetime | None) -> datetime | None:
"""Stamp UTC on a naive timestamp before it goes on the wire.
SQLite has no timezone type: an aware datetime written to `last_seen`
reads back naive, and Pydantic then serializes it without an offset.
`new Date("2026-08-08T10:00:00")` in the browser is *local* time, so the
rack printed a last-seen shifted by the viewer's offset.
"""
if v is not None and v.tzinfo is None:
return v.replace(tzinfo=timezone.utc)
return v
class RackInventoryResponse(BaseModel):
items: list[RackInventoryItem]
+147 -3
View File
@@ -1,26 +1,170 @@
from datetime import datetime
from typing import Any
from pydantic import BaseModel
from pydantic import BaseModel, field_validator
class PendingDeviceResponse(BaseModel):
class InventoryDeviceResponse(BaseModel):
id: str
ip: str
ip: str | None
mac: str | None
hostname: str | None
os: str | None
services: list[Any]
suggested_type: str | None
status: str
discovery_source: str | None
# All sources that have observed this device (e.g. ["arp", "proxmox"]). Drives
# the inventory source filter + badges; falls back to [discovery_source].
discovery_sources: list[str] = []
ieee_address: str | None = None
friendly_name: str | None = None
device_subtype: str | None = None
model: str | None = None
vendor: str | None = None
lqi: int | None = None
# Display properties carried from discovery (e.g. Proxmox specs). Merged into
# the node on approve; empty for scan/mesh sources that don't set them.
properties: list[Any] = []
discovered_at: datetime
# Curated device facts (3.3.0). `label`/`type` supersede friendly_name/
# suggested_type once the user has named the device; the older pair stays for
# discovery imports and the source filters.
label: str | None = None
type: str | None = None
notes: str | None = None
cpu_count: int | None = None
cpu_model: str | None = None
ram_gb: float | None = None
disk_gb: float | None = None
show_hardware: bool = False
check_method: str | None = None
check_target: str | None = None
# Live reachability, distinct from `status` (the pending/approved/hidden
# lifecycle).
status_live: str = "unknown"
last_seen: datetime | None = None
last_scan: datetime | None = None
response_time_ms: int | None = None
updated_at: datetime | None = None
# Number of distinct canvases (designs) this device already appears on,
# correlated by ip / ieee_address against existing nodes. Computed per-request.
canvas_count: int = 0
# Timestamps from the linked canvas node(s), correlated by ip / ieee_address.
# Null when the device is not on any canvas yet. Aggregated across matches:
# created_at = oldest; last_scan / last_modified / last_seen = newest.
node_created_at: datetime | None = None
node_last_scan: datetime | None = None
node_last_modified: datetime | None = None
node_last_seen: datetime | None = None
@field_validator("properties", "discovery_sources", "services", mode="before")
@classmethod
def _coerce_list(cls, v: Any) -> list[Any]:
# Legacy rows (columns added by migration) have these = NULL.
return v if isinstance(v, list) else []
@field_validator("status_live", mode="before")
@classmethod
def _coerce_status_live(cls, v: Any) -> str:
return v if isinstance(v, str) and v else "unknown"
@field_validator("show_hardware", mode="before")
@classmethod
def _coerce_show_hardware(cls, v: Any) -> bool:
return bool(v)
model_config = {"from_attributes": True}
"""Sources a hand-made inventory entry may claim.
``rack`` marks gear created from a rack canvas. Those rows live in the Device
Inventory like any other, but they describe a physical mount rather than a host
on the network, so they are never placed on a logical canvas.
"""
MANUAL_SOURCES = {"manual", "rack"}
class InventoryDeviceCreate(BaseModel):
"""Manually add an entry to the Device Inventory.
Used when the user documents hardware no scan can find — a dumb switch, a
patch panel, a machine that is powered off. Lands with
`discovery_source="manual"` (or `"rack"`) so the inventory filters can tell
it apart.
"""
hostname: str
ip: str | None = None
mac: str | None = None
suggested_type: str | None = None
model: str | None = None
vendor: str | None = None
properties: list[Any] = []
discovery_source: str = "manual"
# Curated fields, so a hand-made entry can carry everything the edit modal
# shows rather than needing a create-then-PATCH round trip.
os: str | None = None
services: list[Any] = []
friendly_name: str | None = None
device_subtype: str | None = None
label: str | None = None
type: str | None = None
notes: str | None = None
cpu_count: int | None = None
cpu_model: str | None = None
ram_gb: float | None = None
disk_gb: float | None = None
show_hardware: bool = False
check_method: str | None = None
check_target: str | None = None
@field_validator("discovery_source")
@classmethod
def _known_source(cls, v: str) -> str:
if v not in MANUAL_SOURCES:
raise ValueError(f"discovery_source must be one of {sorted(MANUAL_SOURCES)}")
return v
class InventoryDeviceUpdate(BaseModel):
"""Partial edit of an inventory row — the write half of the detail modal.
Every field is optional and only what the client sends is applied
(`exclude_unset`), so a caller editing one field never clears the rest.
Lifecycle (`status`) and discovery bookkeeping (`discovery_source(s)`,
`discovered_at`, `ieee_address`) are deliberately not editable here: those
are owned by the approve/hide routes and the importers.
"""
ip: str | None = None
mac: str | None = None
hostname: str | None = None
os: str | None = None
label: str | None = None
type: str | None = None
suggested_type: str | None = None
friendly_name: str | None = None
device_subtype: str | None = None
model: str | None = None
vendor: str | None = None
services: list[Any] | None = None
properties: list[Any] | None = None
notes: str | None = None
cpu_count: int | None = None
cpu_model: str | None = None
ram_gb: float | None = None
disk_gb: float | None = None
show_hardware: bool | None = None
check_method: str | None = None
check_target: str | None = None
class ScanRunResponse(BaseModel):
id: str
status: str
kind: str = "ip"
ranges: list[str]
devices_found: int
started_at: datetime
+19 -1
View File
@@ -4,6 +4,24 @@ def normalize_animated(v: object) -> str:
return 'snake'
if v is False or v == 0 or v == '0' or v is None or v == 'none':
return 'none'
if v in ('snake', 'flow'):
if v in ('snake', 'flow', 'basic'):
return str(v)
return 'none'
MARKER_SHAPES = {'none', 'arrow', 'arrow-open', 'circle', 'diamond', 'square'}
def normalize_marker(v: object) -> str:
"""Normalize an edge endpoint marker to a shape string.
Legacy saves stored a boolean (True = filled arrow); coerce those and any
unknown value to a valid MarkerShape ('none' when off/unknown).
"""
if v is True or v == 1 or v == '1':
return 'arrow'
if v is False or v == 0 or v == '0' or v is None:
return 'none'
if isinstance(v, str) and v in MARKER_SHAPES:
return v
return 'none'
+121
View File
@@ -0,0 +1,121 @@
"""Pydantic v2 schemas for Zigbee2MQTT import."""
from pydantic import BaseModel, Field, model_validator
class ZigbeeImportRequest(BaseModel):
mqtt_host: str = Field(..., description="MQTT broker hostname or IP address")
mqtt_port: int = Field(1883, ge=1, le=65535, description="MQTT broker port")
mqtt_username: str | None = Field(None, description="MQTT username (optional)")
mqtt_password: str | None = Field(None, description="MQTT password (optional)")
base_topic: str = Field("zigbee2mqtt", description="Zigbee2MQTT base topic")
mqtt_tls: bool = Field(False, description="Enable TLS (typically port 8883)")
mqtt_tls_insecure: bool = Field(
False, description="Skip TLS certificate verification (self-signed only)"
)
@model_validator(mode="after")
def _insecure_requires_tls(self) -> "ZigbeeImportRequest":
if self.mqtt_tls_insecure and not self.mqtt_tls:
raise ValueError("mqtt_tls_insecure requires mqtt_tls=true")
return self
class ZigbeeTestConnectionRequest(BaseModel):
mqtt_host: str
mqtt_port: int = Field(1883, ge=1, le=65535)
mqtt_username: str | None = None
mqtt_password: str | None = None
mqtt_tls: bool = False
mqtt_tls_insecure: bool = False
@model_validator(mode="after")
def _insecure_requires_tls(self) -> "ZigbeeTestConnectionRequest":
if self.mqtt_tls_insecure and not self.mqtt_tls:
raise ValueError("mqtt_tls_insecure requires mqtt_tls=true")
return self
class ZigbeeDeviceData(BaseModel):
ieee_address: str
friendly_name: str
device_type: str # Coordinator, Router, EndDevice
model: str | None = None
vendor: str | None = None
description: str | None = None
lqi: int | None = None
last_seen: str | None = None
class ZigbeeNodeOut(BaseModel):
"""A homelable-ready node representation of a Zigbee device."""
id: str
label: str
type: str # zigbee_coordinator | zigbee_router | zigbee_enddevice
ieee_address: str
friendly_name: str
device_type: str
model: str | None = None
vendor: str | None = None
lqi: int | None = None
parent_id: str | None = None
class ZigbeeEdgeOut(BaseModel):
source: str
target: str
class ZigbeeImportResponse(BaseModel):
nodes: list[ZigbeeNodeOut]
edges: list[ZigbeeEdgeOut]
device_count: int
class ZigbeeTestConnectionResponse(BaseModel):
connected: bool
message: str
class ZigbeeCoordinatorOut(BaseModel):
id: str
label: str
ieee_address: str
class ZigbeeImportPendingResponse(BaseModel):
"""Result of importing a Z2M network into the pending section."""
pending_created: int
pending_updated: int
coordinator: ZigbeeCoordinatorOut | None = None
coordinator_already_existed: bool = False
links_recorded: int
device_count: int
class ZigbeeConfig(BaseModel):
"""Non-secret Zigbee connection + auto-sync config (GET response).
MQTT connection fields (host/port/base_topic/tls) are env-only and
read-only here — surfaced for display. ``host_configured`` reflects whether
a server-side MQTT host is set (required for auto-sync). MQTT credentials
(username/password) are never carried."""
mqtt_host: str = ""
mqtt_port: int = Field(1883, ge=1, le=65535)
base_topic: str = "zigbee2mqtt"
mqtt_tls: bool = False
sync_enabled: bool = False
sync_interval: int = Field(3600, ge=300)
host_configured: bool = False
class ZigbeeSyncConfig(BaseModel):
"""User-editable auto-sync config (POST body). The ONLY persisted Zigbee
settings. Connection fields (host/port/credentials/topic/tls) are env-only
and are deliberately not accepted here."""
sync_enabled: bool = False
sync_interval: int = Field(3600, ge=300)
+112
View File
@@ -0,0 +1,112 @@
"""Pydantic v2 schemas for Z-Wave JS UI (zwavejs2mqtt) import."""
from pydantic import BaseModel, Field, model_validator
class ZwaveImportRequest(BaseModel):
mqtt_host: str = Field(..., description="MQTT broker hostname or IP address")
mqtt_port: int = Field(1883, ge=1, le=65535, description="MQTT broker port")
mqtt_username: str | None = Field(None, description="MQTT username (optional)")
mqtt_password: str | None = Field(None, description="MQTT password (optional)")
prefix: str = Field("zwave", description="Z-Wave JS UI MQTT prefix")
gateway_name: str = Field("zwavejs2mqtt", description="Z-Wave JS UI gateway name")
mqtt_tls: bool = Field(False, description="Enable TLS (typically port 8883)")
mqtt_tls_insecure: bool = Field(
False, description="Skip TLS certificate verification (self-signed only)"
)
@model_validator(mode="after")
def _insecure_requires_tls(self) -> "ZwaveImportRequest":
if self.mqtt_tls_insecure and not self.mqtt_tls:
raise ValueError("mqtt_tls_insecure requires mqtt_tls=true")
return self
class ZwaveTestConnectionRequest(BaseModel):
mqtt_host: str
mqtt_port: int = Field(1883, ge=1, le=65535)
mqtt_username: str | None = None
mqtt_password: str | None = None
mqtt_tls: bool = False
mqtt_tls_insecure: bool = False
@model_validator(mode="after")
def _insecure_requires_tls(self) -> "ZwaveTestConnectionRequest":
if self.mqtt_tls_insecure and not self.mqtt_tls:
raise ValueError("mqtt_tls_insecure requires mqtt_tls=true")
return self
class ZwaveNodeOut(BaseModel):
"""A homelable-ready node representation of a Z-Wave device."""
id: str
label: str
type: str # zwave_coordinator | zwave_router | zwave_enddevice
ieee_address: str
friendly_name: str
device_type: str
model: str | None = None
vendor: str | None = None
lqi: int | None = None
parent_id: str | None = None
class ZwaveEdgeOut(BaseModel):
source: str
target: str
class ZwaveImportResponse(BaseModel):
nodes: list[ZwaveNodeOut]
edges: list[ZwaveEdgeOut]
device_count: int
class ZwaveTestConnectionResponse(BaseModel):
connected: bool
message: str
class ZwaveCoordinatorOut(BaseModel):
id: str
label: str
ieee_address: str
class ZwaveImportPendingResponse(BaseModel):
"""Result of importing a Z-Wave network into the pending section."""
pending_created: int
pending_updated: int
coordinator: ZwaveCoordinatorOut | None = None
coordinator_already_existed: bool = False
links_recorded: int
device_count: int
class ZwaveConfig(BaseModel):
"""Non-secret Z-Wave connection + auto-sync config (GET response).
MQTT connection fields (host/port/prefix/gateway_name/tls) are env-only and
read-only here — surfaced for display. ``host_configured`` reflects whether
a server-side MQTT host is set (required for auto-sync). MQTT credentials
(username/password) are never carried."""
mqtt_host: str = ""
mqtt_port: int = Field(1883, ge=1, le=65535)
prefix: str = "zwave"
gateway_name: str = "zwavejs2mqtt"
mqtt_tls: bool = False
sync_enabled: bool = False
sync_interval: int = Field(3600, ge=300)
host_configured: bool = False
class ZwaveSyncConfig(BaseModel):
"""User-editable auto-sync config (POST body). The ONLY persisted Z-Wave
settings. Connection fields (host/port/credentials/prefix/gateway/tls) are
env-only and are deliberately not accepted here."""
sync_enabled: bool = False
sync_interval: int = Field(3600, ge=300)
+19
View File
@@ -0,0 +1,19 @@
"""Helpers for the multi-valued ``InventoryDevice.discovery_sources`` set.
A device discovered by more than one path (e.g. an IP scan *and* a Proxmox
import) accumulates every source that has seen it, so it surfaces under each
matching inventory filter. Order is preserved (origin first) and duplicates are
dropped.
"""
from __future__ import annotations
from collections.abc import Iterable
def add_source(sources: Iterable[str] | None, source: str | None) -> list[str]:
"""Return ``sources`` with ``source`` appended if not already present."""
out = [s for s in (sources or []) if s]
if source and source not in out:
out.append(source)
return out
+133 -32
View File
@@ -6,6 +6,7 @@ from pathlib import Path
from typing import Any
_SIGNATURES: list[dict[str, Any]] | None = None
_OUI_MAP: dict[str, str] | None = None
_LOCK = threading.Lock()
@@ -26,25 +27,124 @@ def _load() -> list[dict[str, Any]]:
return _SIGNATURES
def match_port(port: int, protocol: str, banner: str | None = None) -> dict[str, Any] | None:
"""Return the first signature matching port+protocol, optionally banner."""
for sig in _load():
if sig["port"] != port or sig["protocol"] != protocol:
continue
if sig.get("banner_regex") and (not banner or not re.search(sig["banner_regex"], banner, re.IGNORECASE)):
continue
return sig
return None
def _load_oui() -> dict[str, str]:
"""Load OUI database and flatten to {prefix: node_type}."""
global _OUI_MAP
if _OUI_MAP is None:
with _LOCK:
if _OUI_MAP is None:
path = Path(__file__).parent.parent / "data" / "oui_database.json"
try:
with open(path) as f:
entries = json.load(f)
except FileNotFoundError as err:
raise FileNotFoundError(
f"oui_database.json not found at {path}. "
"This file should be bundled with the application."
) from err
_OUI_MAP = {
prefix.lower(): entry["type"]
for entry in entries
for prefix in entry["prefixes"]
}
return _OUI_MAP
def fingerprint_ports(open_ports: list[dict[str, Any]]) -> list[dict[str, Any]]:
def _http_regex_hit(sig: dict[str, Any], http_signals: dict[str, Any] | None) -> bool:
"""True when the signature's http_regex matches the probe's title/headers."""
rx = sig.get("http_regex")
if not rx or not http_signals:
return False
headers = http_signals.get("headers") or {}
haystack = " ".join(
s for s in (
http_signals.get("title"),
headers.get("Server"),
headers.get("X-Powered-By"),
) if s
)
return bool(haystack and re.search(rx, haystack, re.IGNORECASE))
def _service_tier(
sig: dict[str, Any],
port: int,
protocol: str,
banner: str | None,
http_signals: dict[str, Any] | None,
) -> int | None:
"""
Given a list of {port, protocol, banner?} dicts, return matched services.
Unknown ports are included as unknown_service.
Rank how well a signature matches (lower = stronger). None = not a match.
Tier 1: port match + http_regex confirmed
Tier 2: port match + banner_regex confirmed
Tier 3: port-agnostic (port: null) + http_regex confirmed
Tier 4: port match only (no regex, or http_regex with probe disabled)
When http_signals is None (probe not run) an http_regex entry degrades to
a port-only match — identical to pre-probe behaviour, no regression.
When http_signals is provided, http_regex is strict: a miss disqualifies.
"""
probe_ran = http_signals is not None
has_http = bool(sig.get("http_regex"))
# Port-agnostic entries (port: null) match purely on HTTP signals.
if sig.get("port") is None:
if has_http and _http_regex_hit(sig, http_signals):
return 3
return None
if sig["port"] != port or sig["protocol"] != protocol:
return None
# http_regex is authoritative once a probe has run.
if has_http and probe_ran:
return 1 if _http_regex_hit(sig, http_signals) else None
if sig.get("banner_regex"):
if banner and re.search(sig["banner_regex"], banner, re.IGNORECASE):
return 2
return None
# No regex constraint (or http_regex but probe disabled) → port-only guess.
return 4
def match_service(
port: int,
protocol: str,
banner: str | None = None,
http_signals: dict[str, Any] | None = None,
) -> dict[str, Any] | None:
"""Return the best signature for a port, walking tiers most-specific first."""
best: dict[str, Any] | None = None
best_tier = 99
for sig in _load():
tier = _service_tier(sig, port, protocol, banner, http_signals)
if tier is not None and tier < best_tier:
best, best_tier = sig, tier
if best_tier == 1:
break # strongest possible — stop early
return best
def match_port(port: int, protocol: str, banner: str | None = None) -> dict[str, Any] | None:
"""Back-compat alias: match without HTTP-probe signals."""
return match_service(port, protocol, banner)
def fingerprint_ports(
open_ports: list[dict[str, Any]],
) -> list[dict[str, Any]]:
"""
Given a list of {port, protocol, banner?, http_signals?} dicts, return
matched services. Unknown ports are included as unknown_service.
"""
results = []
for p in open_ports:
sig = match_port(p["port"], p.get("protocol", "tcp"), p.get("banner"))
sig = match_service(
p["port"], p.get("protocol", "tcp"), p.get("banner"), p.get("http_signals")
)
if sig:
results.append({
"port": p["port"],
@@ -65,23 +165,12 @@ def fingerprint_ports(open_ports: list[dict[str, Any]]) -> list[dict[str, Any]]:
return results
# Known OUI prefixes for virtual machines / hypervisors (lowercase, colon-separated)
_MAC_OUI_TYPES: dict[str, str] = {
"52:54:00": "vm", # QEMU/KVM (used by Proxmox VMs)
"bc:24:11": "vm", # Proxmox official OUI (VMs and LXC, Proxmox 7.3+)
"00:50:56": "vm", # VMware
"00:0c:29": "vm", # VMware Workstation / Fusion
"08:00:27": "vm", # VirtualBox
"00:15:5d": "vm", # Hyper-V
}
def suggest_type_from_mac(mac: str | None) -> str | None:
"""Return a suggested node type from MAC OUI, or None if unknown."""
if not mac:
return None
prefix = mac.lower()[:8]
return _MAC_OUI_TYPES.get(prefix)
return _load_oui().get(prefix)
_PORT_TYPE_HINTS: dict[int, str] = {
@@ -101,10 +190,13 @@ _PORT_TYPE_HINTS: dict[int, str] = {
37777: "camera", # Dahua
34567: "camera", # Amcrest
2020: "camera", # Tapo
# Smart-home / MQTT → iot
# Smart-home / MQTT / CoAP → iot
1883: "iot",
8883: "iot",
6052: "iot", # ESPHome
6052: "iot", # ESPHome dashboard
4915: "iot", # Shelly CoIoT
5683: "iot", # CoAP (Shelly Gen1, many IoT devices)
5684: "iot", # CoAP DTLS
# AP / wireless
8880: "ap", # UniFi HTTP
8443: "ap", # UniFi HTTPS
@@ -115,21 +207,30 @@ _PORT_TYPE_HINTS: dict[int, str] = {
def suggest_node_type(open_ports: list[dict[str, Any]], mac: str | None = None) -> str:
"""Suggest a node type based on matched signatures and MAC OUI."""
priority = ["proxmox", "nas", "router", "lxc", "vm", "server", "ap", "camera", "iot", "switch"]
"""Suggest a node type based on matched signatures, port hints, and MAC OUI."""
# IoT vendor MACs are a strong, unambiguous signal — don't let generic HTTP ports override
mac_type = suggest_type_from_mac(mac)
if mac_type == "iot":
return "iot"
priority = [
"proxmox", "nas", "kvm", "router", "lxc", "vm", "ap", "camera", "iot", "server", "switch",
]
found: set[str] = set()
for p in open_ports:
port = p["port"]
proto = p.get("protocol", "tcp")
sig = match_port(port, proto)
# Pass the HTTP-probe signals through: port-agnostic signatures (port: null,
# e.g. PiKVM, Unraid) match on title/headers alone and are invisible otherwise.
sig = match_service(port, proto, p.get("banner"), p.get("http_signals"))
if sig and sig.get("suggested_node_type"):
found.add(sig["suggested_node_type"])
if port in _PORT_TYPE_HINTS:
found.add(_PORT_TYPE_HINTS[port])
# MAC OUI is a lower-priority hint — only used if ports give no better answer
mac_type = suggest_type_from_mac(mac)
if mac_type:
found.add(mac_type)
for t in priority:
if t in found:
return t
+85
View File
@@ -0,0 +1,85 @@
"""HTTP probe: GET a discovered port and extract identifying signals.
Used by the optional deep-scan mode to confirm what service sits behind an
open port, regardless of port number. Returns the page <title> plus a small
set of identifying response headers, which fingerprint.match_service() then
matches against signature http_regex fields.
"""
import asyncio
import logging
import re
from typing import Any
import httpx
logger = logging.getLogger(__name__)
# Headers that commonly carry the application name.
_SIGNAL_HEADERS = ("Server", "X-Powered-By")
# Cap how much body we read when hunting for <title> — avoids large downloads.
_MAX_BODY_BYTES = 64 * 1024
_TITLE_RE = re.compile(r"<title[^>]*>(.*?)</title>", re.IGNORECASE | re.DOTALL)
_PROBE_TIMEOUT = 3.0
# Ports we never bother probing over HTTP (not web services).
_NON_HTTP_PORTS = frozenset({22, 21, 23, 25, 53, 110, 143, 161, 162, 179, 445, 3306, 5432, 6379})
def _extract_title(body: str) -> str | None:
m = _TITLE_RE.search(body)
if not m:
return None
title = re.sub(r"\s+", " ", m.group(1)).strip()
return title or None
async def _probe_scheme(client: httpx.AsyncClient, url: str) -> dict[str, Any] | None:
try:
resp = await client.get(url, follow_redirects=True)
except (httpx.HTTPError, OSError):
return None
headers = {h: resp.headers[h] for h in _SIGNAL_HEADERS if h in resp.headers}
body = resp.text[:_MAX_BODY_BYTES] if resp.text else ""
title = _extract_title(body)
if not title and not headers:
return None
return {"title": title, "headers": headers}
async def probe_port(
ip: str, port: int, verify_tls: bool = False
) -> dict[str, Any] | None:
"""
GET https:// then http:// for a port and return {title, headers} or None.
None means the port did not answer HTTP or yielded no usable signal.
"""
if port in _NON_HTTP_PORTS:
return None
async with httpx.AsyncClient(verify=verify_tls, timeout=_PROBE_TIMEOUT) as client:
for scheme in ("https", "http"):
result = await _probe_scheme(client, f"{scheme}://{ip}:{port}/")
if result is not None:
return result
return None
async def probe_open_ports(
ip: str,
open_ports: list[dict[str, Any]],
verify_tls: bool = False,
concurrency: int = 50,
) -> list[dict[str, Any]]:
"""
Probe every open port for HTTP signals (option 2: probe all, match after).
Returns the same port dicts, each enriched with an http_signals key
(None when the port gave no HTTP signal).
"""
sem = asyncio.Semaphore(concurrency)
async def _one(p: dict[str, Any]) -> dict[str, Any]:
async with sem:
signals = await probe_port(ip, p["port"], verify_tls)
return {**p, "http_signals": signals}
return await asyncio.gather(*(_one(p) for p in open_ports))
+983
View File
@@ -0,0 +1,983 @@
"""Keep a canvas :class:`Node` and its Device Inventory row in step.
The inventory row owns what a device *is* — addresses, services, properties,
notes, hardware, check method. A node owns only how that device is drawn on one
canvas. This module holds the matching and merging rules shared by:
* the one-off backfill that links pre-3.3.0 nodes to inventory rows,
* the approve / node-create paths, which link instead of copying,
* the canvas save write-through, which pushes a node edit back to the row.
Nothing here commits — the caller owns the transaction.
"""
from __future__ import annotations
import json
import logging
from collections.abc import Callable, Mapping
from datetime import datetime
from typing import Any
from sqlalchemy import func, or_, select, text
from sqlalchemy.ext.asyncio import AsyncSession
from app.db.models import InventoryDevice, Node
from app.services.discovery_sources import add_source
logger = logging.getLogger(__name__)
# Canvas furniture: annotations, not hardware. These never get an inventory row.
FURNITURE_TYPES = frozenset({"group", "groupRect", "text"})
# Source tag for a device that only ever existed as a canvas node — the backfill
# mints its inventory row.
CANVAS_SOURCE = "canvas"
# Scalar facts the inventory row owns. Order matters only for readability.
DEVICE_SCALARS = (
"hostname",
"ip",
"mac",
"os",
"notes",
"cpu_count",
"cpu_model",
"ram_gb",
"disk_gb",
"check_method",
"check_target",
)
def is_furniture(node_type: str | None) -> bool:
return (node_type or "") in FURNITURE_TYPES
def _ip_tokens(ip: str | None) -> list[str]:
"""Split an ``ip`` field into individual addresses.
A node or device may carry several comma-separated addresses, so identity
matching compares per token — same rule as ``node_dedupe._ip_tokens``.
"""
return [t.strip() for t in ip.split(",") if t.strip()] if ip else []
def _blank(value: Any) -> bool:
return value is None or value == ""
def _same_ieee(left: str | None, right: str | None) -> bool:
"""IEEE addresses compare case-insensitively — the same radio either way."""
if _blank(left) or _blank(right):
return False
return str(left).lower() == str(right).lower()
async def _ieee_owner(db: AsyncSession, ieee: str, *, other_than: str | None) -> InventoryDevice | None:
"""The row already holding ``ieee``, if it is not ``other_than``.
``device_inventory.ieee_address`` is UNIQUE, so writing an address a second
row owns raises. Callers ask first and leave the address where it is.
"""
stmt = select(InventoryDevice).where(func.lower(InventoryDevice.ieee_address) == ieee.lower())
if other_than is not None:
# `id != NULL` is NULL in SQL and would match nothing — only narrow when
# there is a row to exclude.
stmt = stmt.where(InventoryDevice.id != other_than)
return (await db.execute(stmt)).scalars().first()
async def _drop_taken_ieee(
db: AsyncSession, facts: Mapping[str, Any], device: InventoryDevice | None
) -> Mapping[str, Any]:
"""``facts`` without an ``ieee_address`` another inventory row already owns.
Returned unchanged in the common case. The column is UNIQUE, so writing a
duplicate raises mid-merge — and during the backfill that would cost the node
its link. The address belongs to whichever row holds it; identity for *this*
device was established by ip or mac.
"""
ieee = facts.get("ieee_address")
if _blank(ieee) or (device is not None and _same_ieee(device.ieee_address, ieee)):
return facts
owner = await _ieee_owner(db, str(ieee), other_than=device.id if device else None)
if owner is None:
return facts
logger.warning(
"IEEE %s is already held by device %s — leaving it off %s",
ieee, owner.id, device.id if device else "the new row",
)
return {k: v for k, v in facts.items() if k != "ieee_address"}
async def find_device_for(
db: AsyncSession,
*,
ip: str | None,
mac: str | None,
ieee: str | None,
) -> InventoryDevice | None:
"""Find the inventory row describing this host, or ``None``.
Precedence is ieee > ip > mac, matching ``find_duplicate_node`` and the
bulk-approve skip order so a device is identified the same way everywhere.
Hidden rows are eligible: a hidden device is still that device, and silently
minting a second row for it would resurrect the duplicate the user hid.
"""
ip_toks = _ip_tokens(ip)
conds = []
if ieee:
# Case-insensitive: `0x00124B00…` and `0x00124b00…` are the same radio,
# and an exact comparison here would mint a second row for it — one the
# UNIQUE index then refuses.
conds.append(func.lower(InventoryDevice.ieee_address) == ieee.lower())
for tok in ip_toks:
# Narrow with a substring match, then confirm per token below — an exact
# comparison misses rows holding several addresses.
conds.append(InventoryDevice.ip.contains(tok))
if mac:
conds.append(InventoryDevice.mac == mac)
if not conds:
return None
candidates = (
await db.execute(select(InventoryDevice).where(or_(*conds)).order_by(InventoryDevice.discovered_at))
).scalars().all()
for device in candidates:
if ieee and _same_ieee(device.ieee_address, ieee):
return device
for device in candidates:
# "1.2.3.4" must not match "1.2.3.40" — confirm the token, don't trust
# the SQL `contains`.
if ip_toks and set(_ip_tokens(device.ip)) & set(ip_toks):
return device
for device in candidates:
if mac and device.mac == mac:
return device
return None
def merge_properties(base: list[Any] | None, incoming: list[Any] | None) -> list[Any]:
"""Union two property lists on ``key`` (case-insensitive); incoming wins.
Order-stable: existing keys keep their position, new ones are appended, so a
user's arrangement survives a merge.
"""
out: list[Any] = [dict(p) if isinstance(p, dict) else p for p in (base or [])]
index: dict[str, int] = {}
for i, prop in enumerate(out):
if isinstance(prop, dict) and prop.get("key") is not None:
index[str(prop["key"]).lower()] = i
for prop in incoming or []:
if not isinstance(prop, dict) or prop.get("key") is None:
if prop not in out:
out.append(prop)
continue
key = str(prop["key"]).lower()
pos = index.get(key)
if pos is None:
out.append(dict(prop))
index[key] = len(out) - 1
continue
current = out[pos]
if not isinstance(current, dict):
out[pos] = dict(prop)
continue
merged = {**current, **{k: v for k, v in prop.items() if not _blank(v)}}
# Keys match case-insensitively but the display spelling is the user's —
# "rack" arriving must not rewrite their "Rack".
merged["key"] = current.get("key", prop["key"])
# `visible` is a real False, not an empty value — carry it explicitly.
if "visible" in prop:
merged["visible"] = prop["visible"]
out[pos] = merged
return out
def _service_key(svc: Any) -> Any:
if not isinstance(svc, dict):
return repr(svc)
return (svc.get("port"), svc.get("protocol"), (svc.get("service_name") or "").lower())
# --- Per-node view of the device's list facts -----------------------------
#
# The row owns the services and the properties; a node owns which of them it
# shows and in what order. Both are keyed by a stable string so the view
# survives an edit to a service's path or a property's value.
VIEW_LISTS = ("services", "properties")
def _service_view_key(svc: Any) -> str:
port, protocol, name = _service_key(svc) if isinstance(svc, dict) else (None, None, repr(svc))
return f"{port}|{protocol}|{name}"
def _property_view_key(prop: Any) -> str:
if not isinstance(prop, dict):
return repr(prop)
return str(prop.get("key") or "").lower()
_VIEW_KEY = {"services": _service_view_key, "properties": _property_view_key}
def view_entries(items: list[Any] | None, kind: str) -> list[dict[str, Any]]:
"""One list of device facts as a node's view of it: order plus visibility.
A service carries no ``visible`` of its own — one that reached a canvas was
always drawn — so it defaults to shown. A property carries an explicit flag
and keeps it. Duplicate keys collapse: the view addresses the row, and the
row holds one entry per key.
"""
key_of = _VIEW_KEY[kind]
out: list[dict[str, Any]] = []
seen: set[str] = set()
for item in items or []:
key = key_of(item)
if key in seen:
continue
seen.add(key)
visible = bool(item.get("visible", True)) if isinstance(item, dict) else True
out.append({"key": key, "visible": visible})
return out
def view_from_facts(facts: Mapping[str, Any]) -> dict[str, list[dict[str, Any]]]:
"""The view a node payload implies — only for the lists it actually sent.
The wire shape has no separate view: a client sends its services and its
properties in display order, each with its ``visible`` flag, exactly as it
draws them. That *is* the view, so it is read back out here rather than
asking clients for a second field.
"""
return {kind: view_entries(facts[kind], kind) for kind in VIEW_LISTS if kind in facts}
def view_of_device(device: InventoryDevice) -> dict[str, list[dict[str, Any]]]:
"""A view showing everything the row currently holds — the seed for a new node."""
return {
"services": view_entries(device.services, "services"),
"properties": view_entries(device.properties, "properties"),
}
def next_view(
current: Mapping[str, Any] | None,
incoming: Mapping[str, Any],
device: InventoryDevice | None,
*,
strict: bool = False,
) -> dict[str, Any] | None:
"""This node's view after a write: what it sent, then the row for the rest.
A list the write did not carry keeps the view it had. A node linked to a row
always ends up with both lists, so "not in the view" can mean one thing
only: this canvas does not show it. That is what keeps a service a later scan
discovers off every canvas until someone turns it on.
A node getting its *first* view is the exception: an empty list there means
the writer had nothing to say about it, not that the user hid everything —
creating a node for an already-scanned device sends no services and must
still draw the ones the row holds. ``strict`` turns that off for the one
caller whose empty list is a real answer: the legacy backfill, where the
node's own columns are the whole of what that canvas used to show.
"""
if device is None:
return dict(current) if current else None
out: dict[str, Any] = dict(current or {})
first_view = current is None
seed = view_of_device(device)
for kind in VIEW_LISTS:
entries = incoming.get(kind)
if entries or (entries is not None and (strict or not first_view)):
out[kind] = entries
elif kind not in out:
out[kind] = seed[kind]
return out
def apply_view(items: list[Any] | None, entries: Any, kind: str) -> list[Any]:
"""The row's facts as one node draws them: its order, its visibility.
Without a view — furniture, or a node whose row was linked by an older
version — everything shows, in the row's own order. With one, an item the
view does not list is appended hidden rather than dropped, so a service a
scan added is one toggle away instead of invisible.
A *non-empty* view that matches nothing the row still holds is treated as
having no view at all. It means the row was replaced wholesale under the
node — every key gone, every key new — and hiding the lot would leave a node
drawing nothing while its view claims otherwise. An empty view is different:
it is a real answer ("this canvas draws none of them") and keeps hiding
everything.
"""
key_of = _VIEW_KEY[kind]
facts = list(items or [])
if not isinstance(entries, list):
return facts
by_key: dict[str, Any] = {}
for item in facts:
by_key.setdefault(key_of(item), item)
out: list[Any] = []
taken: set[str] = set()
for entry in entries:
if not isinstance(entry, dict):
continue
key = str(entry.get("key"))
item = by_key.get(key)
if item is None or key in taken:
continue # Deleted from the row since — the view catches up on write.
taken.add(key)
out.append(_stamped(item, bool(entry.get("visible", True))))
if entries and not taken:
return facts
for item in facts:
if key_of(item) not in taken:
out.append(_stamped(item, False))
return out
def _stamped(item: Any, visible: bool) -> Any:
"""``item`` carrying this node's verdict on whether it is drawn.
A shown item that never had a ``visible`` key does not gain one: services
have always travelled without it, and readers treat its absence as shown.
Only hiding is news, and properties keep the explicit flag they arrived with.
"""
if not isinstance(item, dict):
return item
if visible and "visible" not in item:
return dict(item)
return {**item, "visible": visible}
# How a service looks is the user's call. A fingerprint only ever guesses it
# from a port number and a banner, so a scan that re-finds a known service must
# not repaint the icon someone picked by hand — that happened on every "Scan
# network", across every canvas drawing the device, at once.
_CURATED_SERVICE_FIELDS = ("icon", "category")
def merge_services(
base: list[Any] | None,
incoming: list[Any] | None,
*,
discovered: bool = False,
) -> list[Any]:
"""Union two service lists on (port, protocol, name); incoming wins.
``discovered`` marks ``incoming`` as scanner output rather than a user edit:
it still adds services and refreshes facts, but leaves an established icon
and category alone. Either way a blank incoming value never clears one that
is already set — an absent field is silence, not a reset.
"""
out: list[Any] = [dict(s) if isinstance(s, dict) else s for s in (base or [])]
index = {_service_key(s): i for i, s in enumerate(out)}
for svc in incoming or []:
key = _service_key(svc)
pos = index.get(key)
if pos is None:
out.append(dict(svc) if isinstance(svc, dict) else svc)
index[key] = len(out) - 1
elif isinstance(svc, dict) and isinstance(out[pos], dict):
merged = {**out[pos], **svc}
for field_name in _CURATED_SERVICE_FIELDS:
established = out[pos].get(field_name)
if established and (discovered or not svc.get(field_name)):
merged[field_name] = established
out[pos] = merged
else:
out[pos] = svc
return out
# Observations rather than edits: the checker and the scanner write these, so a
# client never lists them as changed and they survive a `changed_fields` filter.
_LIVE_FACT_FIELDS = frozenset({"status", "last_seen", "last_scan", "response_time_ms"})
def changed_facts(device: InventoryDevice, facts: Mapping[str, Any]) -> dict[str, Any]:
"""The subset of ``facts`` that actually differs from the row.
A canvas save sends a *full* copy of the device — the facts were hydrated
into the node when the canvas loaded — so a save triggered by nothing but a
node being dragged would otherwise rewrite the row from a snapshot that may
be minutes or hours old, silently reverting an edit made meanwhile in the
inventory modal, on another canvas, or by the scanner. Narrowing to what the
sender changed turns the write-through from "push my whole snapshot" into
"push my edit", so two writers only collide on the same field.
The comparison mirrors :func:`merge_facts_into_device`: a blank incoming
value is not a change (it never clears an established one), and a list
counts as changed only when it would actually be replaced by a different
one. Ambiguity is resolved toward reporting a change — a false positive is
the old behaviour for that field, a false negative would drop a real edit.
"""
out: dict[str, Any] = {}
for field in (*DEVICE_SCALARS, "label", "type"):
incoming = facts.get(field)
if _blank(incoming) or incoming == getattr(device, field, None):
continue
out[field] = incoming
if not _blank(facts.get("ieee_address")) and _blank(device.ieee_address):
out["ieee_address"] = facts["ieee_address"]
if facts.get("show_hardware") and not device.show_hardware:
out["show_hardware"] = facts["show_hardware"]
if "properties" in facts and list(facts["properties"] or []) != list(device.properties or []):
out["properties"] = facts["properties"]
if "services" in facts and list(facts["services"] or []) != list(device.services or []):
out["services"] = facts["services"]
# Live observations, not edits: carried through only where the merge would
# have used them — filling a row that has never been checked.
if facts.get("status") and device.status_live in (None, "", "unknown"):
out["status"] = facts["status"]
for field in ("last_seen", "last_scan", "response_time_ms"):
if facts.get(field) is not None:
out[field] = facts[field]
return out
def merge_facts_into_device(
device: InventoryDevice,
facts: Mapping[str, Any],
*,
overwrite_scalars: bool,
replace_lists: bool,
) -> None:
"""Fold one view of a device into its inventory row, in place.
``facts`` is a plain mapping — what a canvas save sent, or what a legacy
node's columns held — so this rule lives in one place regardless of where
the view came from.
Two independent knobs, because the callers need three combinations:
* ``overwrite_scalars`` — a non-blank incoming value replaces the row's.
True for the backfill (nodes are visited oldest-edit-first, so the most
recently edited canvas is the last writer and wins) and for a user's save.
False on approve, where the row was just discovered and the node is only a
placement. A blank *never* clears an established value in either mode.
* ``replace_lists`` — properties/services are taken wholesale rather than
unioned. True only for a user's save: otherwise a property they deleted
would come straight back on the next one. The backfill unions, so nothing
any canvas recorded is lost. It applies list by list: a list absent from
``facts`` is left alone even in replace mode, so a partial update never
clears the one it did not send.
"""
for field in (*DEVICE_SCALARS, "label", "type"):
incoming = facts.get(field)
if _blank(incoming):
continue
if overwrite_scalars or _blank(getattr(device, field, None)):
setattr(device, field, incoming)
if not _blank(facts.get("ieee_address")) and _blank(device.ieee_address):
# Identity, never overwritten — two IEEEs mean two devices.
device.ieee_address = facts["ieee_address"]
if facts.get("show_hardware") and not device.show_hardware:
device.show_hardware = True
# Per list, and only for one the caller actually sent: a partial update
# carrying properties alone must leave services as they were, not blank them.
if replace_lists and "properties" in facts:
device.properties = list(facts["properties"] or [])
else:
device.properties = merge_properties(device.properties, facts.get("properties"))
if replace_lists and "services" in facts:
device.services = list(facts["services"] or [])
else:
device.services = merge_services(device.services, facts.get("services"))
# Live status: keep the freshest observation rather than the last writer.
last_seen, status, last_scan = facts.get("last_seen"), facts.get("status"), facts.get("last_scan")
if last_seen and (device.last_seen is None or last_seen > device.last_seen):
device.last_seen = last_seen
device.status_live = status or device.status_live
device.response_time_ms = facts.get("response_time_ms")
elif device.status_live in (None, "", "unknown") and status:
device.status_live = status
if last_scan and (device.last_scan is None or last_scan > device.last_scan):
device.last_scan = last_scan
def device_from_facts(facts: Mapping[str, Any]) -> InventoryDevice:
"""Mint the inventory row for a device that has none.
Tagged ``canvas`` so the inventory filters can tell hand-drawn gear apart
from anything a scan or import found.
"""
return InventoryDevice(
label=facts.get("label"),
type=facts.get("type"),
hostname=facts.get("hostname"),
ip=facts.get("ip"),
mac=facts.get("mac"),
os=facts.get("os"),
ieee_address=facts.get("ieee_address"),
services=list(facts.get("services") or []),
properties=list(facts.get("properties") or []),
notes=facts.get("notes"),
cpu_count=facts.get("cpu_count"),
cpu_model=facts.get("cpu_model"),
ram_gb=facts.get("ram_gb"),
disk_gb=facts.get("disk_gb"),
show_hardware=bool(facts.get("show_hardware")),
check_method=facts.get("check_method"),
check_target=facts.get("check_target"),
suggested_type=facts.get("type"),
friendly_name=facts.get("label"),
# On a canvas already, so it is past the pending queue.
status="approved",
status_live=facts.get("status") or "unknown",
last_seen=facts.get("last_seen"),
last_scan=facts.get("last_scan"),
response_time_ms=facts.get("response_time_ms"),
discovery_source=CANVAS_SOURCE,
discovery_sources=[CANVAS_SOURCE],
)
async def link_facts(
db: AsyncSession,
node: Node,
facts: Mapping[str, Any],
*,
overwrite_scalars: bool = False,
replace_lists: bool = False,
only_changed: bool = False,
changed_fields: list[str] | None = None,
strict_view: bool = False,
) -> InventoryDevice | None:
"""Point one node at its inventory row, creating or merging as needed.
``facts`` is this node's view of the device — the fields a canvas save sent,
or a legacy node's columns during the backfill. Returns the row, or ``None``
for canvas furniture. Flushes so a freshly minted row has an id to link to,
but does not commit.
Two narrowings turn a save from "push my whole snapshot" into "push my edit",
and they compose. Identity matching always uses the full ``facts`` — the row
has to be found before it can be narrowed against.
* ``changed_fields`` — what the sender says it edited since it loaded the
device. Authoritative: a fact absent from the list is not written even when
it differs, because the difference means the *row* moved on, not the sender.
* ``only_changed`` — drop facts already equal to the row (see
:func:`changed_facts`), so a no-op save writes nothing at all.
"""
if is_furniture(node.type):
node.device_id = None
return None
device = None
if node.device_id:
device = await db.get(InventoryDevice, node.device_id)
if device is None:
device = await find_device_for(
db, ip=facts.get("ip"), mac=facts.get("mac"), ieee=facts.get("ieee_address")
)
# The IEEE is UNIQUE across the inventory. Where another row already owns the
# one these facts carry, drop it rather than write a duplicate: identity has
# been resolved to *this* device by ip or mac, and the address stays with the
# row holding it.
facts = await _drop_taken_ieee(db, facts, device)
if device is None:
device = device_from_facts(facts)
db.add(device)
await db.flush()
else:
merged = dict(facts)
if changed_fields is not None:
keep = set(changed_fields) | _LIVE_FACT_FIELDS
merged = {k: v for k, v in merged.items() if k in keep}
if only_changed:
merged = changed_facts(device, merged)
merge_facts_into_device(
device,
merged,
overwrite_scalars=overwrite_scalars,
replace_lists=replace_lists,
)
device.discovery_sources = add_source(device.discovery_sources, CANVAS_SOURCE)
node.device_id = device.id
# Order and visibility are this node's, not the device's, so they are taken
# from the full payload — never from the `changed_fields`/`only_changed`
# narrowing above, which exists to protect the *shared* row from a stale
# snapshot. A node's own view has no other writer to collide with.
node.display_view = next_view(
node.display_view, view_from_facts(facts), device, strict=strict_view
)
return device
def node_columns(payload: Mapping[str, Any]) -> dict[str, Any]:
"""The subset of a node payload that is still a `nodes` column.
The wire shape carries the device facts flat on the node; they belong to the
inventory row now, so they are dropped here and applied through
:func:`link_facts` instead.
"""
allowed = {c.name for c in Node.__table__.columns} - _NODE_DERIVED_COLUMNS
return {k: v for k, v in payload.items() if k in allowed}
# Node columns the server derives rather than accepts: `display_view` is read
# back out of the services and properties a payload carries (see
# :func:`view_from_facts`), so a client sending one directly is ignored.
_NODE_DERIVED_COLUMNS = frozenset({"display_view"})
# Fields that are both a node column and a device fact: the node keeps a copy so
# a half-migrated database still renders, but the row is the truth.
_SHARED_FIELDS = ("label", "type")
# Everything the inventory row owns, as it appears in a node payload.
DEVICE_FACT_FIELDS = (
*DEVICE_SCALARS, "services", "properties", "show_hardware", "status", *_SHARED_FIELDS,
)
def facts_from_update(sent: Mapping[str, Any]) -> dict[str, Any]:
"""The device facts inside a partial node update — only what was sent.
An omitted field must not clear the row, so unsent keys are simply absent.
"""
return {k: v for k, v in sent.items() if k in DEVICE_FACT_FIELDS}
def facts_from_payload(payload: Mapping[str, Any], *, label: str, node_type: str) -> dict[str, Any]:
"""The device facts inside a node save/create payload.
The wire shape still carries them flat on the node, so this is where they
are separated from the presentation fields the node itself keeps.
"""
facts: dict[str, Any] = {
field: payload.get(field)
for field in (*DEVICE_SCALARS, "services", "properties", "show_hardware", "status")
}
facts["label"] = label
facts["type"] = node_type
return facts
async def load_devices_for(db: AsyncSession, nodes: list[Node]) -> dict[str, InventoryDevice]:
"""Fetch the inventory rows a batch of nodes points at, keyed by device id."""
ids = {n.device_id for n in nodes if n.device_id}
if not ids:
return {}
rows = (
await db.execute(select(InventoryDevice).where(InventoryDevice.id.in_(ids)))
).scalars().all()
return {d.id: d for d in rows}
def hydrated_node(node: Node, device: InventoryDevice | None) -> dict[str, Any]:
"""A node as the API reports it: presentation from the node, facts from the row.
The device fields stay on the wire exactly where they have always been, so
every reader — the canvas, the live view, the MCP server — is unaffected by
the split. Canvas furniture has no row and simply reports the defaults.
"""
payload: dict[str, Any] = {
c.name: getattr(node, c.name) for c in node.__table__.columns
}
# An implementation detail of the split, not part of the wire shape: the
# view is reported *through* the services and properties it orders.
view = payload.pop("display_view", None) or {}
if device is None:
return payload
for field in DEVICE_SCALARS:
payload[field] = getattr(device, field, None)
payload["label"] = device.label or node.label
payload["type"] = device.type or node.type
payload["services"] = apply_view(device.services, view.get("services"), "services")
payload["properties"] = apply_view(device.properties, view.get("properties"), "properties")
payload["show_hardware"] = bool(device.show_hardware)
payload["ieee_address"] = device.ieee_address
payload["status"] = device.status_live or "unknown"
payload["last_seen"] = device.last_seen
payload["last_scan"] = device.last_scan
payload["response_time_ms"] = device.response_time_ms
return payload
# The device columns `nodes` carried before 3.3.0. They are read once, by the
# backfill, and then dropped — so they are named here as raw SQL rather than as
# model attributes that no longer exist.
_LEGACY_NODE_COLUMNS = (
"hostname", "ip", "mac", "os", "status", "check_method", "check_target",
"services", "notes", "cpu_count", "cpu_model", "ram_gb", "disk_gb",
"show_hardware", "properties", "ieee_address", "last_seen", "last_scan",
"response_time_ms",
)
async def _legacy_columns_present(db: AsyncSession) -> list[str]:
"""Which pre-3.3.0 device columns still exist on `nodes`."""
rows = (await db.execute(text("PRAGMA table_info(nodes)"))).all()
present = {r[1] for r in rows}
return [c for c in _LEGACY_NODE_COLUMNS if c in present]
def _decode_json(value: Any) -> Any:
"""Raw SQL hands back JSON columns as text; the ORM would have decoded them."""
if isinstance(value, str):
try:
return json.loads(value)
except json.JSONDecodeError:
return []
return value
# Legacy `nodes` columns SQLite stores as DATETIME. Read through raw SQL they come
# back as text, and writing text into a DateTime column raises
# ``TypeError: SQLite DateTime type only accepts Python datetime and date objects``.
_LEGACY_DATETIME_COLUMNS = ("last_seen", "last_scan")
def _decode_dt(value: Any) -> datetime | None:
"""Raw SQL hands back DATETIME columns as text; the ORM would have parsed them.
An unparseable stamp becomes ``None`` rather than an error: `last_seen` and
`last_scan` are observations the status checker refreshes within a minute,
and losing one must not cost a node its inventory row.
"""
if value is None or isinstance(value, datetime):
return value
if not isinstance(value, str) or not value.strip():
return None
try:
return datetime.fromisoformat(value.strip())
except ValueError:
logger.debug("Ignoring an unparseable legacy timestamp: %r", value)
return None
async def _row_is_missing_facts(db: AsyncSession, device_id: str, facts: Mapping[str, Any]) -> bool:
"""Does the linked row lack a fact the node's legacy columns still hold?
True only where the node has something to give and the row has nothing in
that field, so a node whose facts already made it across is left alone and
the backfill stays a no-op on later boots.
"""
device = await db.get(InventoryDevice, device_id)
if device is None:
return True # The row is gone; the node needs a new one.
for field in (*DEVICE_SCALARS, "label", "type", "ieee_address"):
if not _blank(facts.get(field)) and _blank(getattr(device, field, None)):
return True
return any(
facts.get(field) and not getattr(device, field, None)
for field in ("services", "properties")
)
def _with_later_properties(
view: dict[str, Any] | None, device: InventoryDevice
) -> dict[str, Any] | None:
"""``view``, plus the row's properties the pre-3.3.0 backup never saw.
They are what the user added while running 3.3.0-3.3.2 — the row was the
only place to add them then, and every canvas drew them. Recovering the old
view alone would take them off every canvas at once, which reads as data
loss. Appended in the row's order, after everything the backup placed.
"""
if view is None:
return None
entries = view.get("properties")
if not isinstance(entries, list):
return view
listed = {str(e.get("key")) for e in entries if isinstance(e, dict)}
later = [e for e in view_entries(device.properties, "properties") if e["key"] not in listed]
if not later:
return view
return {**view, "properties": [*entries, *later]}
async def seed_node_views(
db: AsyncSession, *, drawn: Callable[[], Mapping[str, Mapping[str, Any]]] | None = None
) -> int:
"""Give every linked node with no view one it can be held to.
Runs once, on the boot that adds `nodes.display_view`. Without it every
pre-existing node would fall through to "no view, show everything" and the
next scan would push a newly fingerprinted service onto all of them at once —
the leak this column exists to stop.
``drawn`` returns a map of node id to the services and properties that node
itself carried before 3.3.0 unioned them onto the row — the caller reads
them out of the pre-upgrade backup, and is only asked to when there is
actually a node to seed. Where it has an answer that answer wins, restoring
the arrangement the user made; where it does not, the row is the seed and
the canvas keeps showing exactly what it shows today. Does not commit.
The backup is 3.2.0-era, so it can only speak for what existed then. A
*property* the row has gained since — one the user added by hand while
running 3.3.0-3.3.2, when the row was all a canvas had — appears in no
backup entry, and seeding strictly from the backup would hide work the user
has been looking at for days. Those are appended visible, keeping the
recovered order and hidden flags for everything the backup does know.
Services are not treated that way: what a row gained since 3.3.0 is mostly a
scan's fingerprint, and holding it back is the whole point of the view.
"""
nodes = list(
(
await db.execute(
select(Node).where(
# A `device_id` naming a row that no longer exists cannot be
# seeded from anything, and leaving it in would match this
# query on every later boot — re-reading the backup file and
# logging a recovery that seeds nothing. Deleting a device
# leaves exactly that: SQLite runs with foreign keys off, so
# the `ON DELETE SET NULL` never fires on an existing table.
Node.device_id.in_(select(InventoryDevice.id)),
Node.display_view.is_(None),
)
)
)
.scalars()
.all()
)
if not nodes:
return 0
devices = await load_devices_for(db, nodes)
# Read the backup only now: on every later boot this function returns above
# and no file is opened at all.
was_drawn = drawn() if drawn is not None else {}
seeded = 0
for node in nodes:
device = devices.get(node.device_id or "")
if device is None: # pragma: no cover - the query already excluded these
continue
was = was_drawn.get(node.id)
# `strict`: an empty list in the backup is this canvas' real answer —
# it drew no service — and must not be read as "say nothing, show all".
if was:
view = next_view(None, view_from_facts(was), device, strict=True)
node.display_view = _with_later_properties(view, device)
else:
node.display_view = view_of_device(device)
seeded += 1
await db.flush()
return seeded
async def backfill_node_devices(db: AsyncSession) -> dict[str, int]:
"""Link every pre-3.3.0 canvas node to a Device Inventory row.
Non-destructive: it writes ``nodes.device_id`` and fills the inventory row,
and never deletes a node or a row. Two nodes on two canvases describing the
same host converge on one row — that convergence is the point.
Nodes are processed oldest-edit-first so that, where two canvases disagree
on a scalar, the most recently edited node is the last writer and wins;
properties and services are unioned, so nothing any canvas recorded is lost.
Reads the legacy columns with raw SQL because the model no longer declares
them — this runs on the boot that drops them, once. Returns counts for the
boot log, ``skipped`` among them: a node whose merge violates a constraint is
left unlinked instead of aborting the run. Does not commit.
"""
legacy = await _legacy_columns_present(db)
if not legacy:
# Already migrated: the columns are gone, so nothing can be left to read.
return {"linked": 0, "created": 0, "merged": 0, "skipped": 0}
placeholders = ", ".join(legacy)
furniture = ", ".join(f"'{t}'" for t in sorted(FURNITURE_TYPES))
# Linked nodes are read too, not just unlinked ones. A canvas saved while an
# earlier migration was stuck minted a *blank* inventory row from a UI that
# had no facts to show, and linked the node to it. Skipping those would count
# them as migrated and drop the columns that still hold their only copy of
# the ip, services and notes. A linked node is filled, never overwritten.
rows = (
await db.execute(
text(
f"SELECT id, label, type, design_id, device_id, {placeholders} FROM nodes "
f"WHERE type NOT IN ({furniture}) "
"ORDER BY updated_at, created_at, id"
)
)
).mappings().all()
if not rows:
return {"linked": 0, "created": 0, "merged": 0, "skipped": 0}
linked = created = merged = skipped = 0
for row in rows:
facts: dict[str, Any] = {c: row[c] for c in legacy}
facts["services"] = _decode_json(facts.get("services")) or []
facts["properties"] = _decode_json(facts.get("properties")) or []
for field in _LEGACY_DATETIME_COLUMNS:
if field in facts:
facts[field] = _decode_dt(facts[field])
facts["label"] = row["label"]
facts["type"] = row["type"]
# One savepoint per node: a row the merge cannot write is dropped on its
# own rather than taking the whole backfill with it. Every node it does
# not link keeps its legacy columns, so nothing is lost and the next boot
# retries. The catch is deliberately broad — a node that cannot be read
# or written for *any* reason must not cost the others their link.
try:
async with db.begin_nested():
node = await db.get(Node, row["id"])
if node is None: # pragma: no cover - the row was just read
continue
if node.device_id and not await _row_is_missing_facts(db, node.device_id, facts):
continue # Already migrated, and its row holds the facts.
existing = await find_device_for(
db, ip=facts.get("ip"), mac=facts.get("mac"), ieee=facts.get("ieee_address")
)
# A node that is already linked only has its gaps filled: whatever
# is on the row was written after the migration and is newer.
device = await link_facts(
db,
node,
facts,
overwrite_scalars=node.device_id is None,
# The node's own columns are exactly what this canvas drew
# before the upgrade, empty lists included — so they define
# its view outright. Anything else the row carries (a service
# a scan fingerprinted, a property another canvas added)
# stays hidden here rather than appearing on every canvas.
strict_view=True,
)
if device is None:
continue
await db.flush()
except Exception as exc:
skipped += 1
logger.warning(
"Inventory backfill: node %s (%s) skipped — %s", row["id"], row["label"], exc
)
continue
linked += 1
if existing is None:
created += 1
logger.info(
"Inventory backfill: node %s (%s) created device %s", node.id, row["label"], device.id
)
else:
merged += 1
logger.info(
"Inventory backfill: node %s (%s, design %s) merged into device %s",
node.id, row["label"], row["design_id"], device.id,
)
await db.flush()
return {"linked": linked, "created": created, "merged": merged, "skipped": skipped}
+18
View File
@@ -0,0 +1,18 @@
"""MAC-address normalization, shared by the scan + Proxmox persist paths.
Different discovery sources emit MACs in different casing/separators (ARP is
lowercase ``bc:24:11:..``, Proxmox config is often uppercase ``BC:24:11:..``).
Canonicalizing on write *and* on compare lets cross-source dedup match a device
by MAC with a plain ``==`` — the join key for merging an IP-scanned row with a
Proxmox-imported one.
"""
from __future__ import annotations
def normalize_mac(mac: str | None) -> str | None:
"""Canonical MAC: lowercase, ``-`` → ``:``, stripped. Blank/None → None."""
if not mac:
return None
normalized = mac.strip().lower().replace("-", ":")
return normalized or None
+168
View File
@@ -0,0 +1,168 @@
"""Shared MQTT helpers for the Zigbee and Z-Wave import services.
Holds the credential-safe error sanitizer, the TLS context builder, and a
generic request/response round-trip over MQTT used by gateway-style APIs
(publish a request topic, wait for a single response topic message).
"""
from __future__ import annotations
import asyncio
import json
import logging
import ssl
from typing import Any
logger = logging.getLogger(__name__)
try:
import aiomqtt
except ImportError: # pragma: no cover
aiomqtt = None # type: ignore[assignment]
_CONNECTION_TIMEOUT = 5.0 # seconds to verify broker reachability
_RESPONSE_TIMEOUT = 300.0 # seconds to wait for a gateway response (large meshes are slow)
def _sanitize_mqtt_error(exc: BaseException) -> str:
"""Return a generic, credential-free message for an MQTT error.
The raw aiomqtt/paho error string can include the broker URI with
embedded credentials (e.g. ``mqtt://user:pass@host``) or auth-related
detail that should not leak to API clients. Map known patterns to
coarse categories; default to a generic failure message. The original
exception is logged at WARNING level for operator debugging.
"""
logger.warning("MQTT error (sanitized for client): %r", exc)
raw = str(exc).lower()
if "not authoriz" in raw or "bad user" in raw or "bad username" in raw:
return "Authentication failed"
if "refused" in raw:
return "Connection refused by broker"
if "name or service not known" in raw or "getaddrinfo" in raw or "nodename nor servname" in raw:
return "Broker hostname could not be resolved"
if "ssl" in raw or "tls" in raw or "certificate" in raw:
return "TLS handshake failed"
if "timed out" in raw or "timeout" in raw:
return "Connection to broker timed out"
return "MQTT connection failed"
def _build_tls_context(insecure: bool) -> ssl.SSLContext:
"""Build an SSL context for MQTT TLS. If insecure, skip verification."""
ctx = ssl.create_default_context()
if insecure:
logger.warning(
"MQTT TLS certificate verification is DISABLED — "
"use only with self-signed brokers on trusted networks."
)
ctx.check_hostname = False
ctx.verify_mode = ssl.CERT_NONE
return ctx
async def request_response(
mqtt_host: str,
mqtt_port: int,
request_topic: str,
response_topic: str,
request_payload: dict[str, Any],
username: str | None = None,
password: str | None = None,
tls: bool = False,
tls_insecure: bool = False,
response_timeout: float = _RESPONSE_TIMEOUT,
) -> dict[str, Any]:
"""Publish ``request_payload`` to ``request_topic`` and return the first
JSON message received on ``response_topic`` as a dict.
Raises:
ImportError: if aiomqtt is not installed.
TimeoutError: if no response arrives in time.
ConnectionError: if the broker cannot be reached.
ValueError: if the response payload is not valid JSON / is empty.
"""
if aiomqtt is None: # pragma: no cover
raise ImportError(
"aiomqtt is required for MQTT import. "
"Install it with: pip install aiomqtt"
)
response_payload: dict[str, Any] = {}
tls_context = _build_tls_context(tls_insecure) if tls else None
try:
async with aiomqtt.Client(
hostname=mqtt_host,
port=mqtt_port,
username=username,
password=password,
timeout=_CONNECTION_TIMEOUT,
tls_context=tls_context,
) as client:
await client.subscribe(response_topic)
# Give the broker a brief window to register the subscription
# before we publish the request. Without this, brokers that
# race SUBACK with our PUBLISH may deliver the response before
# the subscription is active and we'd hang until timeout.
await asyncio.sleep(0.1)
await client.publish(request_topic, json.dumps(request_payload))
async def _wait_for_response() -> None:
async for message in client.messages:
if str(message.topic) != response_topic:
continue
raw = message.payload
try:
payload_str = (
raw.decode() if isinstance(raw, bytes | bytearray) else str(raw)
)
response_payload.update(json.loads(payload_str))
except (json.JSONDecodeError, TypeError) as exc:
raise ValueError(f"Malformed MQTT response: {exc}") from exc
return
await asyncio.wait_for(_wait_for_response(), timeout=response_timeout)
except aiomqtt.MqttError as exc:
raise ConnectionError(_sanitize_mqtt_error(exc)) from exc
except asyncio.TimeoutError as exc:
raise TimeoutError("Timed out waiting for MQTT response") from exc
if not response_payload:
raise ValueError("Empty MQTT response received")
return response_payload
async def test_connection(
mqtt_host: str,
mqtt_port: int,
username: str | None = None,
password: str | None = None,
tls: bool = False,
tls_insecure: bool = False,
) -> bool:
"""Attempt a quick MQTT connection to verify broker reachability.
Returns True on success, raises ConnectionError/TimeoutError on failure.
"""
if aiomqtt is None: # pragma: no cover
raise ImportError("aiomqtt is required")
tls_context = _build_tls_context(tls_insecure) if tls else None
try:
async with aiomqtt.Client(
hostname=mqtt_host,
port=mqtt_port,
username=username,
password=password,
timeout=_CONNECTION_TIMEOUT,
tls_context=tls_context,
):
return True
except aiomqtt.MqttError as exc:
raise ConnectionError(_sanitize_mqtt_error(exc)) from exc
except asyncio.TimeoutError as exc:
raise TimeoutError("Connection to broker timed out") from exc
+154
View File
@@ -0,0 +1,154 @@
"""Duplicate canvas nodes: the same device drawn twice on the *same* canvas.
A device legitimately appears on several canvases — one :class:`Node` per
design, all pointing at one ``device_inventory`` row. Two nodes drawing that row
on the *same* design are the corrupt case, and the only one collapsed here.
Identity is the device link, not the addresses: a node names the row it draws,
so "same device" is an id comparison rather than a guess across ieee/ip/mac.
The repair is idempotent — the oldest node stays, edges and ``parent_id``
references are re-pointed onto it, then the extras go. No device data is lost in
the process because none of it lives on the node any more.
"""
from __future__ import annotations
import logging
from typing import Any
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.db.models import Edge, Node
from app.services.inventory_sync import find_device_for
logger = logging.getLogger(__name__)
async def find_duplicate_node(
db: AsyncSession,
design_id: str | None,
ip: str | None,
mac: str | None,
ieee: str | None = None,
) -> dict[str, Any] | None:
"""Return conflict details if this device is already drawn on ``design_id``.
Identity is resolved once, by the inventory row the addresses point at
(``find_device_for``: ieee > ip > mac), and a duplicate is simply a second
node on the same design drawing that row. Scoped to one design on purpose:
the same device may legitimately appear on several canvases.
The create/approve endpoints turn this into a 409 so the UI can offer "go to
existing" vs "add duplicate anyway".
"""
device = await find_device_for(db, ip=ip, mac=mac, ieee=ieee)
if device is None:
return None
existing = (
await db.execute(
select(Node)
.where(Node.design_id == design_id, Node.device_id == device.id)
.order_by(Node.created_at, Node.id)
)
).scalars().first()
if existing is None:
return None
# Report which address identified the device, as the UI prints it, in the
# same precedence find_device_for used: ieee > ip > mac.
match: str
value: str | None
device_ips = {t.strip() for t in (device.ip or "").split(",") if t.strip()}
shared_ip = next((t.strip() for t in (ip or "").split(",") if t.strip() in device_ips), None)
if ieee and device.ieee_address == ieee:
match, value = "ieee", ieee
elif shared_ip:
match, value = "ip", shared_ip
elif mac and device.mac == mac:
match, value = "mac", mac
else:
match, value = "ip", ip
return {
"duplicate": True,
"existing_node_id": existing.id,
"existing_label": device.label or existing.label,
"match": match,
"value": value,
}
async def dedupe_nodes_by_device(db: AsyncSession) -> int:
"""Merge duplicate nodes drawing the same device on the same canvas.
Returns the number of nodes removed. Idempotent. Nodes drawing one device on
*different* designs are left alone — that is valid cross-canvas placement.
The device facts live on the inventory row, so nothing has to be merged out
of the extras: only edges and parent links are re-pointed before they go.
Does not commit — the caller owns the transaction.
"""
rows = (
await db.execute(
select(Node)
.where(Node.device_id.is_not(None))
.order_by(Node.device_id, Node.created_at, Node.id)
)
).scalars().all()
groups: dict[tuple[str, str | None], list[Node]] = {}
for node in rows:
groups.setdefault((node.device_id, node.design_id), []).append(node) # type: ignore[arg-type]
removed = 0
for (device_id, _design), nodes in groups.items():
if len(nodes) < 2:
continue
canonical, *dups = nodes # oldest first (ordered above)
dup_ids = {d.id for d in dups}
# Re-point edges + parents, then drop self-loops / duplicates.
edges = (
await db.execute(
select(Edge).where(Edge.source.in_(dup_ids) | Edge.target.in_(dup_ids))
)
).scalars().all()
for edge in edges:
if edge.source in dup_ids:
edge.source = canonical.id
if edge.target in dup_ids:
edge.target = canonical.id
children = (
await db.execute(select(Node).where(Node.parent_id.in_(dup_ids)))
).scalars().all()
for child in children:
child.parent_id = canonical.id
all_edges = (
await db.execute(
select(Edge).where((Edge.source == canonical.id) | (Edge.target == canonical.id))
)
).scalars().all()
seen_pairs: set[tuple[str, str, str]] = set()
for edge in all_edges:
if edge.source == edge.target:
await db.delete(edge)
continue
key = (edge.source, edge.target, edge.type)
if key in seen_pairs:
await db.delete(edge)
continue
seen_pairs.add(key)
await db.flush()
for dup in dups:
await db.delete(dup)
removed += 1
logger.info(
"Deduped device %s: merged %d duplicate node(s) into %s",
device_id, len(dups), canonical.id,
)
if removed:
await db.flush()
return removed
+424
View File
@@ -0,0 +1,424 @@
"""Proxmox VE inventory service: fetch hosts + VMs + LXC via the PVE REST API.
Mirrors the Zigbee/Z-Wave import pipeline, but talks to the Proxmox VE REST API
(``/api2/json``) over HTTPS with an API token instead of MQTT. It returns plain
homelable node dicts + parent→child edge hints; DB persistence lives in the
route layer (``app.api.routes.proxmox``).
Auth uses a Proxmox **API token** (never a password):
``Authorization: PVEAPIToken=<token_id>=<secret>`` where ``token_id`` looks like
``user@realm!tokenname``. A read-only ``PVEAuditor`` role is all that is needed.
"""
from __future__ import annotations
import logging
import re
from typing import Any
import httpx
from app.services.mac_utils import normalize_mac
from app.services.zigbee_service import merge_zigbee_properties
logger = logging.getLogger(__name__)
# Reuse the zigbee property-merge contract verbatim (same NodeProperty shape +
# visibility-preservation rules) for re-sync updates.
merge_proxmox_properties = merge_zigbee_properties
_CONNECT_TIMEOUT = 8.0
_READ_TIMEOUT = 20.0
_BYTES_PER_GB = 1024 ** 3
# net0 config line: "name=eth0,bridge=vmbr0,ip=192.168.1.5/24,gw=..."
_LXC_IP_RE = re.compile(r"(?:^|,)ip=([0-9]{1,3}(?:\.[0-9]{1,3}){3})(?:/\d+)?")
# NIC MAC inside a net0 string — qemu "virtio=BC:24:11:..,bridge=.." or lxc
# "..,hwaddr=BC:24:11:..,..". A bare 6-octet MAC match works for both forms.
_NET_MAC_RE = re.compile(r"([0-9a-fA-F]{2}(?::[0-9a-fA-F]{2}){5})")
def _sanitize_proxmox_error(exc: BaseException) -> str:
"""Return a generic, credential-free message for a Proxmox/HTTP error.
Raw httpx errors can echo the request URL and, worse, an
``Authorization: PVEAPIToken=...=<secret>`` header in some stacks. Map known
patterns to coarse categories so the token never reaches an API client. The
original exception is logged at WARNING for operators.
"""
logger.warning("Proxmox error (sanitized for client): %r", exc)
if isinstance(exc, httpx.HTTPStatusError):
code = exc.response.status_code
if code in (401, 403):
return "Authentication failed — check the API token and its permissions"
if code == 404:
return "Proxmox API path not found — is this a Proxmox VE host?"
return f"Proxmox API returned HTTP {code}"
raw = str(exc).lower()
if "name or service not known" in raw or "getaddrinfo" in raw or "nodename nor servname" in raw:
return "Proxmox host could not be resolved"
if "refused" in raw:
return "Connection refused by Proxmox host"
if "certificate" in raw or "ssl" in raw or "tls" in raw:
return "TLS verification failed — enable 'skip TLS verify' for self-signed certs"
if "timed out" in raw or "timeout" in raw:
return "Connection to Proxmox host timed out"
return "Proxmox connection failed"
def _auth_header(token_id: str, token_secret: str) -> dict[str, str]:
return {"Authorization": f"PVEAPIToken={token_id}={token_secret}"}
def _gb(value: Any) -> float | None:
"""Convert a byte count to GB (1 decimal). None/0 → None."""
try:
num = float(value)
except (TypeError, ValueError):
return None
if num <= 0:
return None
return round(num / _BYTES_PER_GB, 1)
def _int_or_none(value: Any) -> int | None:
try:
return int(value)
except (TypeError, ValueError):
return None
def _guest_type_to_homelable(kind: str) -> str:
"""qemu → vm, lxc → lxc (both existing homelable node types)."""
return "vm" if kind == "qemu" else "lxc"
def _extract_qemu_ip(agent_payload: dict[str, Any] | None) -> str | None:
"""Pull the first non-loopback IPv4 from a qemu guest-agent interfaces reply."""
if not agent_payload:
return None
result = agent_payload.get("result")
if not isinstance(result, list):
return None
for iface in result:
if not isinstance(iface, dict):
continue
for addr in iface.get("ip-addresses") or []:
if not isinstance(addr, dict):
continue
if addr.get("ip-address-type") != "ipv4":
continue
ip = addr.get("ip-address")
if isinstance(ip, str) and ip and not ip.startswith("127."):
return ip
return None
def _extract_lxc_ip(config_payload: dict[str, Any] | None) -> str | None:
"""Parse a static IPv4 from an LXC ``net0`` config string (skip dhcp)."""
if not config_payload:
return None
net0 = config_payload.get("net0")
if not isinstance(net0, str):
return None
match = _LXC_IP_RE.search(net0)
return match.group(1) if match else None
def _extract_net_mac(config_payload: dict[str, Any] | None) -> str | None:
"""Parse the NIC MAC from a qemu/lxc ``net0`` config string (normalized).
Works agent-free for both guest kinds and for stopped guests — the sole
identity we can reliably cross-match against an ARP-scanned device.
"""
if not config_payload:
return None
net0 = config_payload.get("net0")
if not isinstance(net0, str):
return None
match = _NET_MAC_RE.search(net0)
return normalize_mac(match.group(1)) if match else None
def _host_node(raw: dict[str, Any]) -> dict[str, Any] | None:
"""Build a homelable ``proxmox`` host node from a ``/nodes`` entry."""
name = raw.get("node")
if not name:
return None
ieee = f"pve-node-{name}"
return {
"id": ieee,
"label": name,
"type": "proxmox",
"ieee_address": ieee,
"hostname": name,
"ip": raw.get("ip") or None,
"status": "online" if raw.get("status") == "online" else "offline",
"cpu_count": _int_or_none(raw.get("maxcpu")),
"ram_gb": _gb(raw.get("maxmem")),
"disk_gb": _gb(raw.get("maxdisk")),
"vendor": "Proxmox VE",
"model": None,
"parent_ieee": None,
}
def _guest_node(
raw: dict[str, Any], host_name: str, kind: str, ip: str | None, mac: str | None = None
) -> dict[str, Any] | None:
"""Build a homelable vm/lxc node from a ``/qemu`` or ``/lxc`` list entry."""
vmid = raw.get("vmid")
if vmid is None:
return None
ieee = f"pve-{host_name}-{vmid}"
node_type = _guest_type_to_homelable(kind)
name = raw.get("name") or f"{node_type}-{vmid}"
return {
"id": ieee,
"label": name,
"type": node_type,
"ieee_address": ieee,
"hostname": name,
"ip": ip,
"mac": mac,
"status": "online" if raw.get("status") == "running" else "offline",
"cpu_count": _int_or_none(raw.get("maxcpu") or raw.get("cpus")),
"ram_gb": _gb(raw.get("maxmem")),
"disk_gb": _gb(raw.get("maxdisk")),
"vendor": "Proxmox VE",
"model": kind.upper(),
"vmid": vmid,
"parent_ieee": f"pve-node-{host_name}",
}
def build_proxmox_properties(node: dict[str, Any]) -> list[dict[str, Any]]:
"""Build a NodeProperty list for a Proxmox device (specs + identity).
Icons match the existing hardware-property convention (``Cpu`` /
``MemoryStick`` / ``HardDrive``). All rows default ``visible=False`` — the
user opts in from the right panel, same as the mesh importers."""
props: list[dict[str, Any]] = []
vmid = node.get("vmid")
if vmid is not None:
props.append({"key": "VMID", "value": str(vmid), "icon": None, "visible": False})
if node.get("model"):
props.append({"key": "Kind", "value": node["model"], "icon": None, "visible": False})
if node.get("cpu_count") is not None:
props.append({"key": "CPU Cores", "value": str(node["cpu_count"]), "icon": "Cpu", "visible": False})
if node.get("ram_gb") is not None:
props.append({"key": "RAM", "value": f"{node['ram_gb']} GB", "icon": "MemoryStick", "visible": False})
if node.get("disk_gb") is not None:
props.append({"key": "Disk", "value": f"{node['disk_gb']} GB", "icon": "HardDrive", "visible": False})
props.append({"key": "Source", "value": "Proxmox VE", "icon": None, "visible": False})
return props
def build_proxmox_cluster_links(nodes: list[dict[str, Any]]) -> list[tuple[str, str]]:
"""Chain host nodes (``type == 'proxmox'``) into cluster links.
Hosts from one import belong to the same cluster, so they are linked
host↔host (rendered as ``cluster`` edges via left/right handles, distinct
from the vertical host→guest ``virtual`` edges). Returns consecutive
``(source_ieee, target_ieee)`` pairs, or ``[]`` for a single host. Mirrors
the frontend ``buildProxmoxClusterEdges``.
"""
hosts = [n["ieee_address"] for n in nodes if n.get("type") == "proxmox" and n.get("ieee_address")]
if len(hosts) < 2:
return []
return [(hosts[i], hosts[i + 1]) for i in range(len(hosts) - 1)]
def _parse_inventory(
hosts_raw: list[dict[str, Any]],
guests_by_host: dict[str, list[dict[str, Any]]],
) -> tuple[list[dict[str, Any]], list[dict[str, Any]]]:
"""Assemble (nodes, edges) from fetched host + guest data.
``guests_by_host`` maps host name → list of already-normalized guest node
dicts. Edges are one host→guest link per guest (materialized as canvas
edges on approval, mirroring the zigbee/zwave link mechanism).
"""
nodes: list[dict[str, Any]] = []
edges: list[dict[str, Any]] = []
seen: set[str] = set()
for raw in hosts_raw:
host = _host_node(raw)
if host is None or host["id"] in seen:
continue
seen.add(host["id"])
nodes.append(host)
for host_name, guests in guests_by_host.items():
host_ieee = f"pve-node-{host_name}"
for guest in guests:
if guest["id"] in seen:
continue
seen.add(guest["id"])
nodes.append(guest)
edges.append({"source": host_ieee, "target": guest["id"]})
return nodes, edges
async def _get_json(client: httpx.AsyncClient, path: str) -> Any:
resp = await client.get(path)
resp.raise_for_status()
return resp.json().get("data")
async def _token_has_permissions(client: httpx.AsyncClient) -> bool:
"""True if the API token holds *any* ACL.
Proxmox list endpoints (``/qemu``, ``/lxc``) silently return an empty
``200`` when the token lacks ``VM.Audit`` — indistinguishable from a host
that genuinely has no guests. ``GET /access/permissions`` returns ``{}`` for
a token with no ACL at all, which is the common misconfiguration (a
privilege-separated token created without its own permission). Best-effort:
on any error assume permissions exist so we never block a valid import.
"""
try:
perms = await _get_json(client, "/access/permissions")
except httpx.HTTPError:
return True
return bool(perms) if isinstance(perms, dict) else True
async def fetch_proxmox_inventory(
host: str,
port: int,
token_id: str,
token_secret: str,
verify_tls: bool = True,
) -> tuple[list[dict[str, Any]], list[dict[str, Any]]]:
"""Fetch hosts + VMs + LXC from a Proxmox VE host, return (nodes, edges).
Raises:
ConnectionError: transport/DNS/TLS failures (sanitized message).
ValueError: malformed API response.
"""
base_url = f"https://{host}:{port}/api2/json"
timeout = httpx.Timeout(_READ_TIMEOUT, connect=_CONNECT_TIMEOUT)
guests_by_host: dict[str, list[dict[str, Any]]] = {}
try:
async with httpx.AsyncClient(
base_url=base_url,
headers=_auth_header(token_id, token_secret),
verify=verify_tls,
timeout=timeout,
) as client:
hosts_raw = await _get_json(client, "/nodes")
if not isinstance(hosts_raw, list):
raise ValueError("Malformed /nodes response")
for host_entry in hosts_raw:
name = host_entry.get("node")
if not name or host_entry.get("status") != "online":
# Offline nodes can't be queried for guests; still shown as host.
continue
guests_by_host[name] = await _fetch_host_guests(client, name)
except httpx.HTTPStatusError as exc:
raise ConnectionError(_sanitize_proxmox_error(exc)) from exc
except httpx.HTTPError as exc:
raise ConnectionError(_sanitize_proxmox_error(exc)) from exc
return _parse_inventory(hosts_raw, guests_by_host)
async def _fetch_host_guests(
client: httpx.AsyncClient, host_name: str
) -> list[dict[str, Any]]:
"""Fetch qemu + lxc guests for one host, resolving guest IPs best-effort."""
guests: list[dict[str, Any]] = []
for kind in ("qemu", "lxc"):
try:
entries = await _get_json(client, f"/nodes/{host_name}/{kind}")
except httpx.HTTPError as exc:
logger.warning("Proxmox %s list failed for %s: %s", kind, host_name, exc)
continue
if not isinstance(entries, list):
continue
for raw in entries:
ip, mac = await _resolve_guest_net(client, host_name, kind, raw)
node = _guest_node(raw, host_name, kind, ip, mac)
if node:
guests.append(node)
return guests
async def _resolve_guest_net(
client: httpx.AsyncClient, host_name: str, kind: str, raw: dict[str, Any]
) -> tuple[str | None, str | None]:
"""Best-effort (ip, mac) for a guest. Never raises.
- MAC comes from the guest ``/config`` net0 line for both kinds (agent-free,
works for stopped guests) — the cross-source dedup key.
- IP: qemu → guest agent (running only); lxc → static net0 config.
Partial results are returned even if a later call fails (e.g. config MAC is
kept when the qemu agent call errors).
"""
vmid = raw.get("vmid")
if vmid is None:
return None, None
ip: str | None = None
mac: str | None = None
try:
if kind == "qemu":
config = await _get_json(client, f"/nodes/{host_name}/qemu/{vmid}/config")
mac = _extract_net_mac(config)
if raw.get("status") == "running":
data = await _get_json(
client, f"/nodes/{host_name}/qemu/{vmid}/agent/network-get-interfaces"
)
ip = _extract_qemu_ip(data)
else:
config = await _get_json(client, f"/nodes/{host_name}/lxc/{vmid}/config")
ip = _extract_lxc_ip(config)
mac = _extract_net_mac(config)
except httpx.HTTPError:
# Guest agent not installed / container stopped / no perms. Keep whatever
# was resolved before the failure.
pass
return ip, mac
async def test_proxmox_connection(
host: str,
port: int,
token_id: str,
token_secret: str,
verify_tls: bool = True,
) -> tuple[bool, str]:
"""Quick reachability + auth check via ``GET /version``.
Returns (connected, message). Never raises credentials outward.
"""
base_url = f"https://{host}:{port}/api2/json"
timeout = httpx.Timeout(_READ_TIMEOUT, connect=_CONNECT_TIMEOUT)
try:
async with httpx.AsyncClient(
base_url=base_url,
headers=_auth_header(token_id, token_secret),
verify=verify_tls,
timeout=timeout,
) as client:
data = await _get_json(client, "/version")
has_perms = await _token_has_permissions(client)
version = (data or {}).get("version", "?") if isinstance(data, dict) else "?"
message = f"Connected to Proxmox VE {version}"
if not has_perms:
message += (
" — warning: this API token has no permissions, so VMs and LXC "
"will not be visible. Assign the PVEAuditor role at path '/' to the "
"token in Proxmox (Datacenter → Permissions → API Token Permission)."
)
return True, message
except httpx.HTTPError as exc:
return False, _sanitize_proxmox_error(exc)
except Exception as exc: # noqa: BLE001 — surface a safe message, log the rest
logger.exception("Unexpected error during Proxmox connection test")
return False, _sanitize_proxmox_error(exc)
File diff suppressed because it is too large Load Diff
+160 -2
View File
@@ -2,6 +2,7 @@
import asyncio
import logging
import socket
import sys
import time
from typing import Any
@@ -18,9 +19,16 @@ async def check_node(check_method: str, target: str | None, ip: str | None) -> d
if check_method == "none":
return {"status": "online", "response_time_ms": None}
host = target or ip
# Use only the first IP when the field contains comma-separated addresses
raw_ip = ip.split(",")[0].strip() if ip else None
host = target or raw_ip
if not host:
return {"status": "unknown", "response_time_ms": None}
# Reject hostnames that look like CLI flags — defends ping/tcp invocations
# against arg-injection if a malicious admin sets target like "-O".
if host.startswith("-"):
logger.warning("Rejecting check target that starts with '-': %r", host)
return {"status": "unknown", "response_time_ms": None}
start = time.monotonic()
try:
@@ -56,9 +64,37 @@ async def check_node(check_method: str, target: str | None, ip: str | None) -> d
return {"status": "offline", "response_time_ms": None}
def _is_ipv6(host: str) -> bool:
"""True if host is a literal IPv6 address (bracketed or bare)."""
try:
socket.inet_pton(socket.AF_INET6, host.strip("[]"))
return True
except OSError:
return False
async def _ping(host: str) -> bool:
# Send 2 probes with a ~2s timeout so a single dropped packet or a slow
# device (ESPHome, IoT) doesn't flap a node offline. Success = any reply.
#
# -W flag units differ by OS:
# Linux: seconds (-W 2 = 2s)
# macOS: milliseconds (-W 2000 = 2s)
# Windows: -w in ms (-w 2000 = 2s)
#
# IPv6-only hosts (e.g. Alexa) never answer IPv4 ping, so target the right
# stack: macOS ships a separate ping6; Linux/Windows take a -6 flag.
ipv6 = _is_ipv6(host)
if sys.platform == "win32":
family = ["-6"] if ipv6 else ["-4"]
args = ["ping", *family, "-n", "2", "-w", "2000", host]
elif sys.platform == "darwin":
args = ["ping6", "-c", "2", host] if ipv6 else ["ping", "-c", "2", "-W", "2000", host]
else:
family = ["-6"] if ipv6 else []
args = ["ping", *family, "-c", "2", "-W", "2", host]
proc = await asyncio.create_subprocess_exec(
"ping", "-c", "1", "-W", "1", host,
*args,
stdout=asyncio.subprocess.DEVNULL,
stderr=asyncio.subprocess.DEVNULL,
)
@@ -82,3 +118,125 @@ async def _tcp_connect(host: str, port: int) -> bool:
return True
except (TimeoutError, OSError, socket.gaierror):
return False
# --- Per-service status checks ---
# Ports that are not HTTP/web. These get NO status check — a service here stays
# grey (unknown) rather than going red. An open TCP socket doesn't prove the
# service is healthy, and a closed one flaps red misleadingly (e.g. SSH on a
# box that simply firewalls 22). Only HTTP(S)-reachable services are checked.
_NON_HTTP_PORTS = frozenset({
22, 21, 23, 25, 465, 587, 53, 110, 143, 993, 995, 389, 636, 445, 514,
1433, 3306, 5432, 5672, 6379, 9092, 11211, 27017, 27018,
})
_HTTPS_PORTS = frozenset({443, 8443})
def _service_host(svc: dict[str, Any], host: str) -> str:
"""Bracket bare IPv6 literals for use in a URL."""
return f"[{host}]" if _is_ipv6(host) else host
def _parse_override(raw: str) -> tuple[str, str | None, int | None]:
"""Split a service `host` override into (hostname, scheme, port).
Mirrors the frontend `parseHostParts`: a node can serve several domains, so
a service may carry its own host, optionally with a scheme and a port
(`blog.example.com`, `blog.example.com:8443`, `https://blog.example.com`).
"""
# Like a node ip, an override may list several hosts — the first one wins,
# same as the frontend's `splitFirstHost`.
rest = raw.split(",")[0].strip()
scheme: str | None = None
for prefix in ("https://", "http://"):
if rest.lower().startswith(prefix):
scheme = prefix[:-3]
rest = rest[len(prefix):]
break
rest = rest.split("/", 1)[0]
if rest.startswith("["):
closing = rest.find("]")
if closing == -1:
return rest, scheme, None
hostname = rest[1:closing]
remainder = rest[closing + 1:]
port = remainder[1:] if remainder.startswith(":") else ""
return hostname, scheme, int(port) if port.isdigit() else None
if rest.count(":") == 1:
hostname, _, port = rest.partition(":")
if hostname and port.isdigit():
return hostname, scheme, int(port)
return rest, scheme, None
async def check_service(svc: dict[str, Any], host: str | None) -> str:
"""Check a single service. Returns 'online' | 'offline' | 'unknown'.
Only HTTP(S)-reachable services get a real check (an HTTP GET). Everything
else — SSH, databases, mail, DNS, raw TCP, UDP, port-less — stays 'unknown'
so it keeps its category colour instead of flashing red. An open TCP socket
doesn't prove a non-web service is healthy, so we don't pretend it does.
"""
override = str(svc.get("host") or "").strip()
override_scheme: str | None = None
override_port: int | None = None
if override:
host, override_scheme, override_port = _parse_override(override)
if not host or host.startswith("-"):
return "unknown"
if str(svc.get("protocol", "")).lower() == "udp":
return "unknown"
port = svc.get("port")
port = int(port) if isinstance(port, int) or (isinstance(port, str) and port.isdigit()) else None
if port is None:
port = override_port
# Non-HTTP ports (SSH 22, DB, mail, …) are never checked — keep them grey.
if port is not None and port in _NON_HTTP_PORTS:
return "unknown"
name = str(svc.get("service_name", "")).lower()
is_web = port is not None or "http" in name or override_scheme is not None
if not is_web:
return "unknown"
try:
scheme = override_scheme or ("https" if (
port in _HTTPS_PORTS or "https" in name or "ssl" in name or "tls" in name
) else "http")
url_host = _service_host(svc, host)
url = f"{scheme}://{url_host}" + (f":{port}" if port is not None else "")
return "online" if await _http_get(url, verify=False) else "offline"
except Exception as exc:
logger.debug("Service check failed for %s:%s (%s)", host, port, exc)
return "offline"
async def check_services(
host: str | None, services: list[dict[str, Any]], concurrency: int = 10
) -> list[dict[str, Any]]:
"""Check every service against host concurrently (bounded).
Returns a list of {port, protocol, status} dicts, one per input service.
"""
sem = asyncio.Semaphore(concurrency)
async def _one(svc: dict[str, Any]) -> dict[str, Any]:
async with sem:
status = await check_service(svc, host)
# `host` rides along: several vhosts can share one port on one node, so
# port+protocol alone no longer identifies a service on the client side.
return {
"port": svc.get("port"),
"protocol": svc.get("protocol"),
"host": svc.get("host"),
"status": status,
}
return await asyncio.gather(*[_one(s) for s in services]) if services else []
+341
View File
@@ -0,0 +1,341 @@
"""Zigbee2MQTT service: connects to MQTT broker and fetches the network map."""
from __future__ import annotations
import asyncio
import json
import logging
from typing import Any
from app.services.mqtt_common import _build_tls_context, _sanitize_mqtt_error
logger = logging.getLogger(__name__)
try:
import aiomqtt
except ImportError: # pragma: no cover
aiomqtt = None # type: ignore[assignment]
_NETWORKMAP_REQUEST_TOPIC = "{base_topic}/bridge/request/networkmap"
_NETWORKMAP_RESPONSE_TOPIC = "{base_topic}/bridge/response/networkmap"
_CONNECTION_TIMEOUT = 5.0 # seconds to verify broker reachability
_NETWORKMAP_TIMEOUT = 300.0 # seconds to wait for the networkmap response (large meshes can be slow)
# Re-exported for backwards compatibility — these now live in mqtt_common.
__all__ = ["_build_tls_context", "_sanitize_mqtt_error"]
def build_zigbee_properties(
ieee: str | None,
vendor: str | None,
model: str | None,
lqi: int | None,
) -> list[dict[str, Any]]:
"""Build a NodeProperty list for a Zigbee device (IEEE, Vendor, Model, LQI).
Only includes a row when the value is non-empty. Shape matches the
frontend ``NodeProperty`` type: ``{key, value, icon, visible}``.
New props default to ``visible=False`` — users opt in to showing them on
the canvas card from the right panel.
"""
props: list[dict[str, Any]] = []
if ieee:
props.append({"key": "IEEE", "value": ieee, "icon": None, "visible": False})
if vendor:
props.append({"key": "Vendor", "value": vendor, "icon": None, "visible": False})
if model:
props.append({"key": "Model", "value": model, "icon": None, "visible": False})
if lqi is not None:
props.append({"key": "LQI", "value": str(lqi), "icon": None, "visible": False})
return props
def merge_zigbee_properties(
existing: list[dict[str, Any]] | None,
new_props: list[dict[str, Any]],
) -> list[dict[str, Any]]:
"""Merge fresh zigbee props into an existing property list.
For keys already present: update ``value`` but preserve the user's
``visible`` choice. New keys are appended with whatever visibility the
caller gave them (hidden by default per ``build_zigbee_properties``).
Non-zigbee custom properties are preserved untouched.
"""
out = [dict(p) for p in (existing or [])]
by_key = {p.get("key"): p for p in out}
for np in new_props:
key = np.get("key")
if key in by_key:
by_key[key]["value"] = np.get("value")
else:
out.append(dict(np))
return out
def _z2m_type_to_homelable(device_type: str) -> str:
"""Map a Z2M device type string to a homelable node type."""
mapping = {
"Coordinator": "zigbee_coordinator",
"Router": "zigbee_router",
"EndDevice": "zigbee_enddevice",
}
return mapping.get(device_type, "zigbee_enddevice")
def _node_from_z2m(raw: dict[str, Any]) -> dict[str, Any] | None:
"""Build a homelable node dict from a Z2M raw networkmap node entry."""
ieee: str = raw.get("ieeeAddr") or raw.get("ieee_address") or ""
if not ieee:
return None
device_type: str = raw.get("type") or "EndDevice"
friendly_name: str = (
raw.get("friendlyName") or raw.get("friendly_name") or ieee
)
definition: dict[str, Any] = raw.get("definition") or {}
model: str | None = (
raw.get("modelID")
or raw.get("model")
or definition.get("model")
or None
)
vendor: str | None = raw.get("vendor") or definition.get("vendor") or None
return {
"id": ieee,
"label": friendly_name,
"type": _z2m_type_to_homelable(device_type),
"ieee_address": ieee,
"friendly_name": friendly_name,
"device_type": device_type,
"model": model,
"vendor": vendor,
"lqi": None,
"parent_id": None,
}
def parse_networkmap(
payload: dict[str, Any],
) -> tuple[list[dict[str, Any]], list[dict[str, Any]]]:
"""Parse a Z2M ``bridge/response/networkmap`` payload into node + edge lists.
Z2M raw response shape::
{
"data": {
"type": "raw",
"routes": false,
"value": {
"nodes": [{"ieeeAddr": ..., "type": "Coordinator|Router|EndDevice",
"friendlyName": ..., "definition": {"model": ..., "vendor": ...}}],
"links": [{"source": {"ieeeAddr": ...}, "target": {"ieeeAddr": ...},
"lqi": 200, "depth": 1}]
}
},
"status": "ok"
}
Older or alternate shapes may put nodes/links directly under ``data``.
Both are accepted.
"""
data: dict[str, Any] = payload.get("data") or {}
value = data.get("value")
container: dict[str, Any] = value if isinstance(value, dict) else data
raw_nodes: list[dict[str, Any]] = container.get("nodes") or []
raw_links: list[dict[str, Any]] = container.get("links") or []
if not isinstance(raw_nodes, list):
raise ValueError("Malformed networkmap: 'nodes' is not a list")
if not isinstance(raw_links, list):
raise ValueError("Malformed networkmap: 'links' is not a list")
nodes_list: list[dict[str, Any]] = []
seen_ids: set[str] = set()
coordinator_id: str | None = None
for entry in raw_nodes:
if not isinstance(entry, dict):
continue
node = _node_from_z2m(entry)
if node is None or node["id"] in seen_ids:
continue
seen_ids.add(node["id"])
nodes_list.append(node)
if node["device_type"] == "Coordinator":
coordinator_id = node["id"]
# Z2M `links` is bidirectional/mesh: every pair appears twice and routers
# carry sibling-mesh paths. Walk it only to extract LQI per device and to
# resolve which router an end device hangs off; do NOT emit edges directly
# from links. The final edge set is the strict parent→child tree built
# from parent_id below — that avoids duplicate edges and keeps the visual
# flow consistent (parent bottom → child top).
raw_edges: list[dict[str, Any]] = []
lqi_by_id: dict[str, int] = {}
for link in raw_links:
if not isinstance(link, dict):
continue
src_obj = link.get("source") or {}
tgt_obj = link.get("target") or {}
src = src_obj.get("ieeeAddr") if isinstance(src_obj, dict) else None
tgt = tgt_obj.get("ieeeAddr") if isinstance(tgt_obj, dict) else None
if not src or not tgt:
continue
if src not in seen_ids or tgt not in seen_ids:
continue
raw_edges.append({"source": src, "target": tgt})
lqi = link.get("lqi") or link.get("linkquality")
if isinstance(lqi, int) and tgt not in lqi_by_id:
lqi_by_id[tgt] = lqi
for node in nodes_list:
if node["id"] in lqi_by_id:
node["lqi"] = lqi_by_id[node["id"]]
# Build parent_id hierarchy: coordinator → routers → end devices
if coordinator_id:
router_ids = {n["id"] for n in nodes_list if n["device_type"] == "Router"}
for node in nodes_list:
if node["device_type"] == "Router":
node["parent_id"] = coordinator_id
elif node["device_type"] == "EndDevice":
parent = _find_parent_router(node["id"], router_ids, raw_edges)
node["parent_id"] = parent or coordinator_id
# Final edges = strict parent → child tree (one edge per non-coordinator)
edges_list: list[dict[str, Any]] = [
{"source": node["parent_id"], "target": node["id"]}
for node in nodes_list
if node.get("parent_id")
]
return nodes_list, edges_list
def _find_parent_router(
device_id: str,
router_ids: set[str],
edges: list[dict[str, Any]],
) -> str | None:
"""Return the first router that has a direct edge to device_id."""
for edge in edges:
src: str = edge["source"]
tgt: str = edge["target"]
if tgt == device_id and src in router_ids:
return src
if src == device_id and tgt in router_ids:
return tgt
return None
async def fetch_networkmap(
mqtt_host: str,
mqtt_port: int,
base_topic: str,
username: str | None = None,
password: str | None = None,
tls: bool = False,
tls_insecure: bool = False,
) -> tuple[list[dict[str, Any]], list[dict[str, Any]]]:
"""Connect to the MQTT broker, request the Z2M networkmap, and return (nodes, edges).
Raises:
TimeoutError: if the broker does not respond in time.
ConnectionError: if the broker cannot be reached.
ValueError: if the response payload is malformed.
"""
if aiomqtt is None: # pragma: no cover
raise ImportError(
"aiomqtt is required for Zigbee import. "
"Install it with: pip install aiomqtt"
)
request_topic = _NETWORKMAP_REQUEST_TOPIC.format(base_topic=base_topic)
response_topic = _NETWORKMAP_RESPONSE_TOPIC.format(base_topic=base_topic)
response_payload: dict[str, Any] = {}
tls_context = _build_tls_context(tls_insecure) if tls else None
try:
async with aiomqtt.Client(
hostname=mqtt_host,
port=mqtt_port,
username=username,
password=password,
timeout=_CONNECTION_TIMEOUT,
tls_context=tls_context,
) as client:
await client.subscribe(response_topic)
# Give the broker a brief window to register the subscription
# before we publish the request. Without this, brokers that
# race SUBACK with our PUBLISH may deliver the response before
# the subscription is active and we'd hang until timeout.
await asyncio.sleep(0.1)
await client.publish(
request_topic,
json.dumps({"type": "raw", "routes": False}),
)
async def _wait_for_response() -> None:
async for message in client.messages:
if str(message.topic) != response_topic:
continue
raw = message.payload
try:
payload_str = (
raw.decode() if isinstance(raw, bytes | bytearray) else str(raw)
)
response_payload.update(json.loads(payload_str))
except (json.JSONDecodeError, TypeError) as exc:
raise ValueError(
f"Malformed networkmap response: {exc}"
) from exc
return
await asyncio.wait_for(_wait_for_response(), timeout=_NETWORKMAP_TIMEOUT)
except aiomqtt.MqttError as exc:
raise ConnectionError(_sanitize_mqtt_error(exc)) from exc
except asyncio.TimeoutError as exc:
raise TimeoutError("Timed out waiting for networkmap response") from exc
if not response_payload:
raise ValueError("Empty networkmap response received")
return parse_networkmap(response_payload)
async def test_mqtt_connection(
mqtt_host: str,
mqtt_port: int,
username: str | None = None,
password: str | None = None,
tls: bool = False,
tls_insecure: bool = False,
) -> bool:
"""Attempt a quick MQTT connection to verify broker reachability.
Returns True on success, raises ConnectionError on failure.
"""
if aiomqtt is None: # pragma: no cover
raise ImportError("aiomqtt is required")
tls_context = _build_tls_context(tls_insecure) if tls else None
try:
async with aiomqtt.Client(
hostname=mqtt_host,
port=mqtt_port,
username=username,
password=password,
timeout=_CONNECTION_TIMEOUT,
tls_context=tls_context,
):
return True
except aiomqtt.MqttError as exc:
raise ConnectionError(_sanitize_mqtt_error(exc)) from exc
except asyncio.TimeoutError as exc:
raise TimeoutError("Connection to broker timed out") from exc
+235
View File
@@ -0,0 +1,235 @@
"""Z-Wave JS UI (zwavejs2mqtt) service: fetch the node list via the MQTT gateway API.
Mirrors the Zigbee pipeline. Z-Wave JS UI exposes a request/response gateway over
MQTT: publish to ``<prefix>/_CLIENTS/ZWAVE_GATEWAY-<gateway>/api/getNodes/set`` and
read the answer from ``<prefix>/_CLIENTS/ZWAVE_GATEWAY-<gateway>/api/getNodes``.
"""
from __future__ import annotations
import logging
from typing import Any
from app.services.mqtt_common import request_response, test_connection
from app.services.zigbee_service import _find_parent_router, merge_zigbee_properties
logger = logging.getLogger(__name__)
# Reuse the zigbee merge logic verbatim — same NodeProperty shape + visibility rules.
merge_zwave_properties = merge_zigbee_properties
_REQUEST_TOPIC = "{prefix}/_CLIENTS/ZWAVE_GATEWAY-{gateway}/api/getNodes/set"
_RESPONSE_TOPIC = "{prefix}/_CLIENTS/ZWAVE_GATEWAY-{gateway}/api/getNodes"
def _zwave_type_to_homelable(raw: dict[str, Any]) -> str:
"""Map a Z-Wave node's role flags to a homelable node type.
Controller → coordinator. Mains-powered / routing nodes → router.
Everything else (battery sensors, etc.) → end device.
"""
if raw.get("isControllerNode"):
return "zwave_coordinator"
if raw.get("isRouting"):
return "zwave_router"
return "zwave_enddevice"
def _role_label(node_type: str) -> str:
"""Human role string stored as ``device_subtype`` / ``device_type``."""
return {
"zwave_coordinator": "Controller",
"zwave_router": "Router",
"zwave_enddevice": "EndDevice",
}.get(node_type, "EndDevice")
def _node_from_zwave(raw: dict[str, Any], home_id: str) -> dict[str, Any] | None:
"""Build a homelable node dict from a Z-Wave JS UI ``getNodes`` entry."""
node_id = raw.get("id")
if node_id is None:
return None
ieee = f"zwave-{home_id}-{node_id}"
node_type = _zwave_type_to_homelable(raw)
name = raw.get("name") or raw.get("loc") or f"Node {node_id}"
model = raw.get("productLabel") or raw.get("productDescription") or None
vendor = raw.get("manufacturer") or None
return {
"id": ieee,
"label": name,
"type": node_type,
"ieee_address": ieee,
"friendly_name": name,
"device_type": _role_label(node_type),
"node_id": node_id,
"model": model,
"vendor": vendor,
"lqi": None, # Z-Wave has no LQI; RSSI may be added later.
"parent_id": None,
"neighbors": raw.get("neighbors") or [],
}
def _resolve_home_id(raw_nodes: list[dict[str, Any]]) -> str:
"""Pick a home id for the network: prefer the controller's, else any node's."""
controller_home = None
for entry in raw_nodes:
if not isinstance(entry, dict):
continue
home = entry.get("homeId")
if home is None:
continue
if entry.get("isControllerNode"):
return str(home)
if controller_home is None:
controller_home = str(home)
return controller_home or "0"
def parse_zwave_nodes(
payload: dict[str, Any],
) -> tuple[list[dict[str, Any]], list[dict[str, Any]]]:
"""Parse a Z-Wave JS UI ``getNodes`` response into (nodes, edges).
Expected shape::
{"success": true, "result": [ {<node>}, ... ]}
Edges are a strict coordinator → router → end-device tree, derived from
each node's ``neighbors`` list (same approach as the Zigbee parser).
"""
if payload.get("success") is False:
raise ValueError("Z-Wave gateway reported failure")
result = payload.get("result")
if result is None:
result = []
if not isinstance(result, list):
raise ValueError("Malformed getNodes response: 'result' is not a list")
home_id = _resolve_home_id(result)
nodes_list: list[dict[str, Any]] = []
seen_ids: set[str] = set()
coordinator_id: str | None = None
# Map nodeId (int) → identity string, to translate neighbors → edges.
id_by_node_id: dict[Any, str] = {}
for entry in result:
if not isinstance(entry, dict):
continue
node = _node_from_zwave(entry, home_id)
if node is None or node["id"] in seen_ids:
continue
seen_ids.add(node["id"])
id_by_node_id[node["node_id"]] = node["id"]
nodes_list.append(node)
if node["type"] == "zwave_coordinator":
coordinator_id = node["id"]
# Translate neighbor lists into candidate edges (only between known nodes).
raw_edges: list[dict[str, Any]] = []
for node in nodes_list:
src = node["id"]
for neighbor in node.get("neighbors") or []:
tgt = id_by_node_id.get(neighbor)
if tgt and tgt != src:
raw_edges.append({"source": src, "target": tgt})
# Build parent_id hierarchy: coordinator → routers → end devices.
if coordinator_id:
router_ids = {n["id"] for n in nodes_list if n["type"] == "zwave_router"}
for node in nodes_list:
if node["type"] == "zwave_router":
node["parent_id"] = coordinator_id
elif node["type"] == "zwave_enddevice":
parent = _find_parent_router(node["id"], router_ids, raw_edges)
node["parent_id"] = parent or coordinator_id
# Final edges = strict parent → child tree (one edge per non-coordinator).
edges_list: list[dict[str, Any]] = [
{"source": node["parent_id"], "target": node["id"]}
for node in nodes_list
if node.get("parent_id")
]
# Drop transient helper keys before returning.
for node in nodes_list:
node.pop("neighbors", None)
node.pop("node_id", None)
return nodes_list, edges_list
def build_zwave_properties(
ieee: str | None,
vendor: str | None,
model: str | None,
) -> list[dict[str, Any]]:
"""Build a NodeProperty list for a Z-Wave device (Identity, Vendor, Model).
Z-Wave has no LQI, so that row is omitted. New props default to
``visible=False`` — users opt in from the right panel.
"""
props: list[dict[str, Any]] = []
if ieee:
props.append({"key": "Z-Wave ID", "value": ieee, "icon": None, "visible": False})
if vendor:
props.append({"key": "Vendor", "value": vendor, "icon": None, "visible": False})
if model:
props.append({"key": "Model", "value": model, "icon": None, "visible": False})
return props
async def fetch_zwave_network(
mqtt_host: str,
mqtt_port: int,
prefix: str = "zwave",
gateway_name: str = "zwavejs2mqtt",
username: str | None = None,
password: str | None = None,
tls: bool = False,
tls_insecure: bool = False,
) -> tuple[list[dict[str, Any]], list[dict[str, Any]]]:
"""Connect to the broker, request the Z-Wave node list, return (nodes, edges).
Raises:
TimeoutError: if the gateway does not respond in time.
ConnectionError: if the broker cannot be reached.
ValueError: if the response payload is malformed.
"""
request_topic = _REQUEST_TOPIC.format(prefix=prefix, gateway=gateway_name)
response_topic = _RESPONSE_TOPIC.format(prefix=prefix, gateway=gateway_name)
payload = await request_response(
mqtt_host=mqtt_host,
mqtt_port=mqtt_port,
request_topic=request_topic,
response_topic=response_topic,
request_payload={"args": []},
username=username,
password=password,
tls=tls,
tls_insecure=tls_insecure,
)
return parse_zwave_nodes(payload)
async def test_zwave_connection(
mqtt_host: str,
mqtt_port: int,
username: str | None = None,
password: str | None = None,
tls: bool = False,
tls_insecure: bool = False,
) -> bool:
"""Quick MQTT reachability check for the Z-Wave broker."""
return await test_connection(
mqtt_host=mqtt_host,
mqtt_port=mqtt_port,
username=username,
password=password,
tls=tls,
tls_insecure=tls_insecure,
)
+1
View File
@@ -2,3 +2,4 @@
*.db-shm
*.db-wal
scan_config.json
homelab.db.*
Binary file not shown.

After

Width:  |  Height:  |  Size: 31 KiB

+2
View File
@@ -25,6 +25,8 @@ addopts = "--tb=short -q"
[tool.coverage.run]
source = ["app"]
omit = ["*/migrations/*", "*/tests/*"]
concurrency = ["thread"]
core = "sysmon"
[tool.coverage.report]
skip_empty = true
+9 -6
View File
@@ -6,20 +6,23 @@ aiosqlite==0.20.0
alembic==1.13.3
pydantic==2.9.2
pydantic-settings==2.5.2
python-jose[cryptography]==3.5.0
passlib[bcrypt]==1.7.4
bcrypt==4.0.1
python-multipart==0.0.22
PyJWT==2.13.0
bcrypt==4.2.1
authlib==1.7.2
itsdangerous==2.2.0
python-multipart==0.0.31
apscheduler==3.10.4
python-nmap==0.7.1
pyyaml==6.0.2
types-PyYAML==6.0.12.20240917
websockets==13.1
httpx==0.27.2
zeroconf==0.149.16
aiomqtt==2.3.0
# Dev
ruff==0.6.9
mypy==1.11.2
pytest==8.3.3
pytest-asyncio==0.24.0
pytest==9.0.3
pytest-asyncio==1.3.0
pytest-cov==5.0.0
+3 -5
View File
@@ -1,13 +1,11 @@
"""Generate a bcrypt password hash for config.yml."""
"""Generate a bcrypt password hash for the AUTH_PASSWORD_HASH env var."""
import sys
from passlib.context import CryptContext
pwd_context = CryptContext(schemes=["bcrypt"], deprecated="auto")
import bcrypt
if len(sys.argv) < 2:
print("Usage: python scripts/hash_password.py <password>")
sys.exit(1)
password = sys.argv[1]
print(pwd_context.hash(password))
print(bcrypt.hashpw(password.encode("utf-8"), bcrypt.gensalt()).decode("utf-8"))
+6 -11
View File
@@ -5,23 +5,21 @@ os.environ.setdefault("SECRET_KEY", "test-only-secret-key-not-for-production")
import pytest
from httpx import ASGITransport, AsyncClient
from passlib.context import CryptContext
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker, create_async_engine
from app.core.security import hash_password
from app.db.database import Base, get_db
from app.main import app
TEST_DB_URL = "sqlite+aiosqlite:///:memory:"
_pwd_ctx = CryptContext(schemes=["bcrypt"], deprecated="auto")
@pytest.fixture(autouse=True, scope="session")
def test_credentials():
"""Configure test auth credentials directly on settings."""
from app.core.config import settings
settings.auth_username = "admin"
settings.auth_password_hash = _pwd_ctx.hash("admin")
settings.auth_password_hash = hash_password("admin")
@pytest.fixture
@@ -46,10 +44,7 @@ async def client(db_session: AsyncSession):
@pytest.fixture
def auth_headers(client):
"""Returns a coroutine that logs in and returns auth headers."""
async def _get():
res = await client.post("/api/v1/auth/login", json={"username": "admin", "password": "admin"})
token = res.json()["access_token"]
return {"Authorization": f"Bearer {token}"}
return _get
async def headers(client: AsyncClient):
"""Authenticated Bearer headers for the default admin test user."""
res = await client.post("/api/v1/auth/login", json={"username": "admin", "password": "admin"})
return {"Authorization": f"Bearer {res.json()['access_token']}"}
+57
View File
@@ -0,0 +1,57 @@
"""
Tests for automatic DB backup before migrations.
"""
import os
os.environ.setdefault("SECRET_KEY", "test-only-secret-key-not-for-production")
from pathlib import Path
from unittest.mock import patch
import pytest
from app.db.database import _backup_db
@pytest.fixture()
def tmp_db(tmp_path: Path):
db = tmp_path / "homelab.db"
db.write_bytes(b"SQLite placeholder")
return db
def test_backup_created_when_db_exists(tmp_db: Path):
with patch("app.db.database.settings") as mock_settings, \
patch("app.db.database.APP_VERSION", "1.9"):
mock_settings.sqlite_path = str(tmp_db)
_backup_db()
backup = tmp_db.parent / "homelab.db.back-1.9"
assert backup.exists()
assert backup.read_bytes() == b"SQLite placeholder"
def test_backup_skipped_when_db_missing(tmp_path: Path):
with patch("app.db.database.settings") as mock_settings, \
patch("app.db.database.APP_VERSION", "1.9"):
mock_settings.sqlite_path = str(tmp_path / "nonexistent.db")
_backup_db()
assert not any(tmp_path.glob("*.back-*"))
def test_backup_idempotent_second_call_no_overwrite(tmp_db: Path):
with patch("app.db.database.settings") as mock_settings, \
patch("app.db.database.APP_VERSION", "1.9"):
mock_settings.sqlite_path = str(tmp_db)
_backup_db()
backup = tmp_db.parent / "homelab.db.back-1.9"
backup.write_bytes(b"original backup")
_backup_db()
assert backup.read_bytes() == b"original backup"
def test_backup_version_in_filename(tmp_db: Path):
with patch("app.db.database.settings") as mock_settings, \
patch("app.db.database.APP_VERSION", "2.0"):
mock_settings.sqlite_path = str(tmp_db)
_backup_db()
assert (tmp_db.parent / "homelab.db.back-2.0").exists()
@@ -0,0 +1,156 @@
"""Backward-compatibility tests for the legacy → multi-design migration.
Simulates a database created by a pre-"designs" version of the app and asserts
that running init_db() adopts all existing nodes/edges/canvas into a single
default "Network Topology" design with no data loss. The rest of the test suite
builds the *current* schema via create_all and never exercises this upgrade
path, so this file guards real users upgrading in place.
"""
import os
os.environ.setdefault("SECRET_KEY", "test-only-secret-key-not-for-production")
import pytest
from sqlalchemy.ext.asyncio import create_async_engine
import app.db.database as database
@pytest.fixture
def legacy_engine(tmp_path, monkeypatch):
"""Point the module-global engine + sqlite_path at a throwaway legacy DB."""
db_path = tmp_path / "legacy.db"
monkeypatch.setattr(database.settings, "sqlite_path", str(db_path))
engine = create_async_engine(f"sqlite+aiosqlite:///{db_path}")
monkeypatch.setattr(database, "engine", engine)
return db_path, engine
async def _build_legacy_schema(engine) -> None:
"""Create the pre-designs schema (no design_id, integer canvas_state PK)."""
async with engine.begin() as conn:
await conn.exec_driver_sql(
"CREATE TABLE nodes (id VARCHAR PRIMARY KEY, type VARCHAR, label VARCHAR, "
"status VARCHAR, services JSON, pos_x FLOAT, pos_y FLOAT)"
)
await conn.exec_driver_sql(
"CREATE TABLE edges (id VARCHAR PRIMARY KEY, source VARCHAR, target VARCHAR, type VARCHAR)"
)
await conn.exec_driver_sql(
"CREATE TABLE canvas_state (id INTEGER PRIMARY KEY, viewport JSON, "
"custom_style JSON, saved_at DATETIME)"
)
await conn.exec_driver_sql(
"INSERT INTO nodes (id, type, label, status, services, pos_x, pos_y) "
"VALUES ('n1','server','Old Server','online','[]',10,20)"
)
await conn.exec_driver_sql(
"INSERT INTO nodes (id, type, label, status, services, pos_x, pos_y) "
"VALUES ('n2','router','Old Router','offline','[]',30,40)"
)
await conn.exec_driver_sql(
"INSERT INTO edges (id, source, target, type) VALUES ('e1','n1','n2','ethernet')"
)
await conn.exec_driver_sql(
"INSERT INTO canvas_state (id, viewport, custom_style, saved_at) "
"VALUES (1, '{\"x\":5,\"y\":6,\"zoom\":2}', NULL, '2024-01-01 00:00:00')"
)
async def test_legacy_canvas_migrates_into_default_design(legacy_engine):
db_path, engine = legacy_engine
await _build_legacy_schema(engine)
await database.init_db()
check = create_async_engine(f"sqlite+aiosqlite:///{db_path}")
try:
async with check.begin() as conn:
# Exactly one seeded default design.
designs = (await conn.exec_driver_sql(
"SELECT id, name, design_type, icon FROM designs"
)).fetchall()
assert len(designs) == 1
did, name, dtype, icon = designs[0]
assert name == "Network Topology"
assert dtype == "network"
assert icon == "dashboard"
# Every legacy node adopted into the default design, data preserved.
nodes = (await conn.exec_driver_sql(
"SELECT id, label, status, design_id FROM nodes ORDER BY id"
)).fetchall()
assert [(n[0], n[1], n[2]) for n in nodes] == [
("n1", "Old Server", "online"),
("n2", "Old Router", "offline"),
]
assert all(n[3] == did for n in nodes)
# Legacy edge adopted too.
edge = (await conn.exec_driver_sql(
"SELECT design_id FROM edges WHERE id='e1'"
)).fetchone()
assert edge[0] == did
# canvas_state rebuilt with design_id PK; the old id=1 row maps to the
# default design and the viewport survives.
cs = (await conn.exec_driver_sql(
"SELECT design_id, viewport FROM canvas_state"
)).fetchall()
assert len(cs) == 1
assert cs[0][0] == did
assert "zoom" in (cs[0][1] or "")
finally:
await check.dispose()
await engine.dispose()
async def test_legacy_nodes_gain_last_scan_column(legacy_engine):
"""A legacy nodes table (no last_scan) gains the column after init_db."""
db_path, engine = legacy_engine
await _build_legacy_schema(engine)
await database.init_db()
check = create_async_engine(f"sqlite+aiosqlite:///{db_path}")
try:
async with check.begin() as conn:
cols = (await conn.exec_driver_sql("PRAGMA table_info(nodes)")).fetchall()
assert "last_scan" in {c[1] for c in cols}
# Existing rows backfill to NULL (never scanned yet).
last_scan = (await conn.exec_driver_sql(
"SELECT last_scan FROM nodes WHERE id='n1'"
)).fetchone()
assert last_scan[0] is None
finally:
await check.dispose()
await engine.dispose()
async def test_migration_is_idempotent(legacy_engine):
"""Running init_db twice must not duplicate the design or drop any data."""
db_path, engine = legacy_engine
await _build_legacy_schema(engine)
await database.init_db()
await database.init_db() # second boot — should be a no-op
check = create_async_engine(f"sqlite+aiosqlite:///{db_path}")
try:
async with check.begin() as conn:
designs = (await conn.exec_driver_sql("SELECT id FROM designs")).fetchall()
assert len(designs) == 1
did = designs[0][0]
nodes = (await conn.exec_driver_sql(
"SELECT design_id FROM nodes"
)).fetchall()
assert len(nodes) == 2
assert all(n[0] == did for n in nodes)
cs = (await conn.exec_driver_sql("SELECT design_id FROM canvas_state")).fetchall()
assert len(cs) == 1
assert cs[0][0] == did
finally:
await check.dispose()
await engine.dispose()
@@ -0,0 +1,423 @@
"""The 3.2.0 → 3.3.0 upgrade of the `nodes` table.
3.3.0 moved the device facts off `nodes` onto `device_inventory` and dropped the
columns. A real 3.2.0 database declares several of them NOT NULL with no
server-side default (`status`, `services`, `properties`, `show_hardware`) — the
schema here is `create_all`'s output at tag v3.2.0, not a hand-simplified copy,
because that constraint is exactly what breaks: the 3.3.0 model no longer writes
those columns, so if they survive the upgrade every INSERT fails with
``NOT NULL constraint failed: nodes.status`` and approving a device is dead.
"""
import os
import shutil
os.environ.setdefault("SECRET_KEY", "test-only-secret-key-not-for-production")
import pytest
from sqlalchemy.exc import IntegrityError
from sqlalchemy.ext.asyncio import create_async_engine
import app.db.database as database
import app.services.inventory_sync as inventory_sync
# `create_all` under v3.2.0. NOT NULL is reproduced exactly where it was.
_NODES_320 = (
"CREATE TABLE nodes ("
"id VARCHAR NOT NULL PRIMARY KEY, type VARCHAR NOT NULL, label VARCHAR NOT NULL, "
"design_id VARCHAR, hostname VARCHAR, ip VARCHAR, mac VARCHAR, os VARCHAR, "
"status VARCHAR NOT NULL, check_method VARCHAR, check_target VARCHAR, "
"services JSON NOT NULL, notes TEXT, pos_x FLOAT NOT NULL, pos_y FLOAT NOT NULL, "
"parent_id VARCHAR, container_mode BOOLEAN NOT NULL, custom_colors JSON, "
"custom_icon VARCHAR, cpu_count INTEGER, cpu_model VARCHAR, ram_gb FLOAT, "
"disk_gb FLOAT, show_hardware BOOLEAN NOT NULL, show_port_numbers BOOLEAN NOT NULL, "
"properties JSON NOT NULL, width FLOAT, height FLOAT, "
"bottom_handles INTEGER NOT NULL, top_handles INTEGER NOT NULL, "
"left_handles INTEGER NOT NULL, right_handles INTEGER NOT NULL, "
"ieee_address VARCHAR, last_seen DATETIME, last_scan DATETIME, "
"response_time_ms INTEGER, created_at DATETIME NOT NULL, updated_at DATETIME NOT NULL)"
)
_DESIGNS_320 = (
"CREATE TABLE designs (id VARCHAR NOT NULL PRIMARY KEY, name VARCHAR NOT NULL, "
"design_type VARCHAR NOT NULL, icon VARCHAR, created_at DATETIME NOT NULL, "
"updated_at DATETIME NOT NULL)"
)
def _node_sql(node_id: str, label: str, node_type: str, ip: str | None) -> str:
ip_sql = f"'{ip}'" if ip else "NULL"
return (
"INSERT INTO nodes (id, type, label, design_id, ip, status, services, pos_x, pos_y, "
"container_mode, show_hardware, show_port_numbers, properties, bottom_handles, "
"top_handles, left_handles, right_handles, created_at, updated_at) VALUES "
f"('{node_id}', '{node_type}', '{label}', 'd1', {ip_sql}, 'online', '[]', 0, 0, "
"0, 0, 0, '[]', 1, 1, 0, 0, '2026-01-01 00:00:00', '2026-01-01 00:00:00')"
)
@pytest.fixture
def db_320(tmp_path, monkeypatch):
"""A v3.2.0 database, with the module-global engine pointed at it."""
db_path = tmp_path / "v320.db"
monkeypatch.setattr(database.settings, "sqlite_path", str(db_path))
engine = create_async_engine(f"sqlite+aiosqlite:///{db_path}")
monkeypatch.setattr(database, "engine", engine)
monkeypatch.setattr(
database, "AsyncSessionLocal", database.async_sessionmaker(engine, expire_on_commit=False)
)
return db_path, engine
async def _build_320(engine) -> None:
async with engine.begin() as conn:
await conn.exec_driver_sql(_DESIGNS_320)
await conn.exec_driver_sql(_NODES_320)
await conn.exec_driver_sql(
"INSERT INTO designs (id, name, design_type, icon, created_at, updated_at) "
"VALUES ('d1', 'Network Topology', 'network', 'dashboard', "
"'2026-01-01 00:00:00', '2026-01-01 00:00:00')"
)
await conn.exec_driver_sql(_node_sql("n1", "OpnSense", "firewall", "192.168.1.1"))
await conn.exec_driver_sql(_node_sql("n2", "NAS", "nas", "192.168.1.2"))
await conn.exec_driver_sql(_node_sql("n3", "a note", "text", None))
async def _insert_node(engine) -> None:
"""What approve does: insert a node the way the 3.3.0 model writes one."""
from app.db.models import Node
session_factory = database.async_sessionmaker(engine, expire_on_commit=False)
async with session_factory() as session:
session.add(Node(id="new", type="firewall", label="Approved", design_id="d1"))
await session.commit()
async def test_upgrade_drops_the_device_columns_and_leaves_nodes_insertable(db_320):
db_path, engine = db_320
await _build_320(engine)
await database.init_db()
check = create_async_engine(f"sqlite+aiosqlite:///{db_path}")
try:
async with check.begin() as conn:
cols = {c[1] for c in (await conn.exec_driver_sql("PRAGMA table_info(nodes)")).fetchall()}
assert "status" not in cols
assert "ip" not in cols
assert "device_id" in cols
# Every device node linked; the facts moved to the inventory row.
linked = (await conn.exec_driver_sql(
"SELECT label, device_id FROM nodes ORDER BY id"
)).fetchall()
assert [row[0] for row in linked] == ["OpnSense", "NAS", "a note"]
assert linked[0][1] and linked[1][1]
assert linked[2][1] is None # canvas furniture keeps no row
ips = (await conn.exec_driver_sql(
"SELECT ip FROM device_inventory ORDER BY ip"
)).fetchall()
assert [row[0] for row in ips] == ["192.168.1.1", "192.168.1.2"]
await _insert_node(engine)
finally:
await check.dispose()
await engine.dispose()
async def test_a_node_the_backfill_cannot_link_still_leaves_nodes_insertable(db_320, monkeypatch):
"""The regression: a failed backfill must not brick every later INSERT.
Before the fix, one unlinkable node kept `nodes.status` NOT NULL while the
3.3.0 model stopped writing it, so approving *any* device raised
``NOT NULL constraint failed: nodes.status`` (issue #351).
"""
db_path, engine = db_320
await _build_320(engine)
real_link_facts = inventory_sync.link_facts
async def link_facts(db, node, facts, **kwargs):
if node.id == "n2":
raise IntegrityError("boom", None, Exception("UNIQUE constraint failed"))
return await real_link_facts(db, node, facts, **kwargs)
monkeypatch.setattr(inventory_sync, "link_facts", link_facts)
await database.init_db()
check = create_async_engine(f"sqlite+aiosqlite:///{db_path}")
try:
async with check.begin() as conn:
info = (await conn.exec_driver_sql("PRAGMA table_info(nodes)")).fetchall()
cols = {c[1]: c for c in info}
# The columns stay — they are the only remaining copy of n2's facts.
assert "status" in cols
# But nothing is NOT NULL any more, so the model can insert again.
assert not any(c[3] for name, c in cols.items() if name in database._LEGACY_NODE_COLUMNS)
# Values preserved by the rebuild.
kept = (await conn.exec_driver_sql(
"SELECT status, ip FROM nodes WHERE id='n2'"
)).fetchone()
assert kept == ("online", "192.168.1.2")
# The node that did link is linked; the failed one is not.
linked = dict((await conn.exec_driver_sql(
"SELECT id, device_id FROM nodes"
)).fetchall())
assert linked["n1"] is not None
assert linked["n2"] is None
await _insert_node(engine)
# A boot where the failure persists changes nothing further: the relax is
# a one-off, not a rebuild on every start.
await database.init_db()
async with check.begin() as conn:
kept = (await conn.exec_driver_sql(
"SELECT status, ip FROM nodes WHERE id='n2'"
)).fetchone()
assert kept == ("online", "192.168.1.2")
assert (await conn.exec_driver_sql(
"SELECT COUNT(*) FROM nodes"
)).scalar() == 4 # the three seeded, plus the one just approved
# A later boot retries the backfill: with the failure gone, n2 links and
# the columns finally drop.
monkeypatch.setattr(inventory_sync, "link_facts", real_link_facts)
await database.init_db()
async with check.begin() as conn:
cols = {c[1] for c in (await conn.exec_driver_sql("PRAGMA table_info(nodes)")).fetchall()}
assert "status" not in cols
finally:
await check.dispose()
await engine.dispose()
async def test_upgrade_keeps_each_canvas_drawing_its_own_services(db_320):
"""3.3.3: order and visibility become the node's, and the upgrade freezes them.
Two canvases drew the same host with different service lists, and a scan had
already fingerprinted a third the user put on neither. All three converge on
one inventory row — so each node must come out of the upgrade still drawing
what it drew, with the scanner's guess hidden on both.
"""
db_path, engine = db_320
await _build_320(engine)
ssh = '[{"port": 22, "protocol": "tcp", "service_name": "ssh"}]'
both = '[{"port": 22, "protocol": "tcp", "service_name": "ssh"}, ' \
'{"port": 443, "protocol": "tcp", "service_name": "https"}]'
async with engine.begin() as conn:
await conn.exec_driver_sql(f"UPDATE nodes SET services = '{ssh}' WHERE id = 'n1'")
await conn.exec_driver_sql(f"UPDATE nodes SET services = '{both}' WHERE id = 'n2'")
await database.init_db()
from app.db.models import InventoryDevice, Node
session_factory = database.async_sessionmaker(engine, expire_on_commit=False)
async with session_factory() as session:
drawn = {}
for node_id in ("n1", "n2"):
node = await session.get(Node, node_id)
device = await session.get(InventoryDevice, node.device_id)
payload = inventory_sync.hydrated_node(node, device)
drawn[node_id] = [s["service_name"] for s in payload["services"] if s.get("visible", True)]
# Seeded, not left to "show the whole row".
assert node.display_view is not None
assert drawn == {"n1": ["ssh"], "n2": ["ssh", "https"]}
# What a scan finds next lands on the row, and on no canvas.
node = await session.get(Node, "n1")
device = await session.get(InventoryDevice, node.device_id)
device.services = [
*device.services,
{"port": 3001, "protocol": "tcp", "service_name": "Uptime Kuma"},
]
await session.commit()
payload = inventory_sync.hydrated_node(node, device)
assert [(s["service_name"], s.get("visible", True)) for s in payload["services"]] == [
("ssh", True), ("Uptime Kuma", False),
]
await engine.dispose()
async def _wind_back_to_3_3_2(engine, extra_service: dict) -> None:
"""A database that already took the 3.3.0 upgrade, leak included.
Both nodes point at one row holding the union of what each drew plus what a
scan found, and neither has a view — which is every 3.3.0-3.3.2 install.
"""
from app.db.models import InventoryDevice, Node
async with engine.begin() as conn:
await conn.exec_driver_sql("UPDATE nodes SET display_view = NULL")
session_factory = database.async_sessionmaker(engine, expire_on_commit=False)
async with session_factory() as session:
node = await session.get(Node, "n1")
one = await session.get(InventoryDevice, node.device_id)
other = await session.get(InventoryDevice, (await session.get(Node, "n2")).device_id)
for device in (one, other):
device.services = [
{"port": 22, "protocol": "tcp", "service_name": "ssh"},
{"port": 443, "protocol": "tcp", "service_name": "https"},
extra_service,
]
await session.commit()
async def test_a_database_already_on_3_3_recovers_its_layout_from_the_backup(db_320):
"""The second upgrade path: 3.3.0-3.3.2, where the legacy columns are gone.
3.3.0 unioned every canvas' services onto one row, so the row can no longer
say who drew what — but the backup taken before that migration still can.
Recovering from it is the difference between the user getting their canvases
back and getting each canvas showing every other canvas' services.
"""
_, engine = db_320
await _build_320(engine)
ssh = '[{"port": 22, "protocol": "tcp", "service_name": "ssh"}]'
both = '[{"port": 22, "protocol": "tcp", "service_name": "ssh"}, ' \
'{"port": 443, "protocol": "tcp", "service_name": "https"}]'
async with engine.begin() as conn:
await conn.exec_driver_sql(f"UPDATE nodes SET services = '{ssh}' WHERE id = 'n1'")
await conn.exec_driver_sql(f"UPDATE nodes SET services = '{both}' WHERE id = 'n2'")
await database.init_db() # 3.2.0 -> 3.3.x, and the backup that predates it.
kuma = {"port": 3001, "protocol": "tcp", "service_name": "Uptime Kuma"}
await _wind_back_to_3_3_2(engine, kuma)
await database.init_db()
from app.db.models import InventoryDevice, Node
session_factory = database.async_sessionmaker(engine, expire_on_commit=False)
async with session_factory() as session:
drawn = {}
for node_id in ("n1", "n2"):
node = await session.get(Node, node_id)
device = await session.get(InventoryDevice, node.device_id)
payload = inventory_sync.hydrated_node(node, device)
drawn[node_id] = [s["service_name"] for s in payload["services"] if s.get("visible", True)]
assert drawn == {"n1": ["ssh"], "n2": ["ssh", "https"]}
await engine.dispose()
async def test_a_property_added_while_on_3_3_survives_the_recovery(db_320):
"""The backup is 3.2.0-era and cannot know what the user added afterwards.
On 3.3.0-3.3.2 the row was the only place to add a property, and every
canvas drew it. Recovering the view strictly from the backup would hide it
on all of them at once — the "my properties disappeared" half of #347. The
scanner's service find is still held back: only properties are appended.
"""
_, engine = db_320
await _build_320(engine)
rack = '[{"key": "Rack", "value": "A", "icon": null, "visible": true}]'
ssh = '[{"port": 22, "protocol": "tcp", "service_name": "ssh"}]'
async with engine.begin() as conn:
await conn.exec_driver_sql(
f"UPDATE nodes SET properties = '{rack}', services = '{ssh}' WHERE id = 'n1'"
)
await database.init_db() # 3.2.0 -> 3.3.x, and the backup that predates it.
kuma = {"port": 3001, "protocol": "tcp", "service_name": "Uptime Kuma"}
await _wind_back_to_3_3_2(engine, kuma)
from app.db.models import InventoryDevice, Node
session_factory = database.async_sessionmaker(engine, expire_on_commit=False)
async with session_factory() as session:
node = await session.get(Node, "n1")
device = await session.get(InventoryDevice, node.device_id)
# What the user re-added by hand while running 3.3.x.
device.properties = [
*device.properties,
{"key": "Ports", "value": "8", "icon": None, "visible": True},
]
await session.commit()
await database.init_db()
async with session_factory() as session:
node = await session.get(Node, "n1")
device = await session.get(InventoryDevice, node.device_id)
payload = inventory_sync.hydrated_node(node, device)
assert [(p["key"], p.get("visible", True)) for p in payload["properties"]] == [
("Rack", True), ("Ports", True),
]
# And the scan's find is still off this canvas.
assert [s["service_name"] for s in payload["services"] if s.get("visible", True)] == ["ssh"]
await engine.dispose()
async def test_without_a_usable_backup_the_row_is_the_seed(db_320):
"""No backup to recover from: keep showing what the canvas shows today.
Nothing is taken away — the user simply keeps the merged list they have been
looking at since 3.3.0, and only what the row gains *after* this boot is
held back.
"""
db_path, engine = db_320
await _build_320(engine)
await database.init_db()
kuma = {"port": 3001, "protocol": "tcp", "service_name": "Uptime Kuma"}
await _wind_back_to_3_3_2(engine, kuma)
for backup in db_path.parent.glob(f"{db_path.name}.back-*"):
backup.unlink()
await database.init_db()
from app.db.models import InventoryDevice, Node
session_factory = database.async_sessionmaker(engine, expire_on_commit=False)
async with session_factory() as session:
node = await session.get(Node, "n1")
device = await session.get(InventoryDevice, node.device_id)
payload = inventory_sync.hydrated_node(node, device)
assert [s["service_name"] for s in payload["services"]] == ["ssh", "https", "Uptime Kuma"]
assert all(s.get("visible", True) for s in payload["services"])
# From here on the node is pinned: the next scan's find is held back.
device.services = [*device.services, {"port": 5001, "protocol": "tcp", "service_name": "Synology DSM HTTPS"}]
await session.commit()
payload = inventory_sync.hydrated_node(node, device)
assert [s["service_name"] for s in payload["services"] if not s.get("visible", True)] == [
"Synology DSM HTTPS"
]
await engine.dispose()
async def test_the_newest_backup_that_still_has_the_columns_is_the_one_read(db_320):
"""A 3.3.2 install has several backups; only some can answer.
`homelab.db.back-3.3.1` and `-3.3.2` were taken *after* the split and hold
nothing per node; `-3.3.0` was taken before it. Reading the wrong one would
either say nothing or resurrect a much older canvas.
"""
db_path, engine = db_320
await _build_320(engine)
async with engine.begin() as conn:
await conn.exec_driver_sql(
"UPDATE nodes SET services = "
"'[{\"port\": 22, \"protocol\": \"tcp\", \"service_name\": \"ssh\"}]' WHERE id = 'n1'"
)
# Ancient: same shape, but a canvas the user has long since moved on from.
old = db_path.parent / f"{db_path.name}.back-3.1.0"
shutil.copy2(db_path, old)
os.utime(old, (1, 1))
async with engine.begin() as conn:
await conn.exec_driver_sql(
"UPDATE nodes SET services = "
"'[{\"port\": 443, \"protocol\": \"tcp\", \"service_name\": \"https\"}]' WHERE id = 'n1'"
)
pre_split = db_path.parent / f"{db_path.name}.back-3.3.0"
shutil.copy2(db_path, pre_split)
await database.init_db() # drops the columns, and backs up under this version
# Post-split backups: newer, and unable to say who drew what.
for name in ("back-3.3.1", "back-3.3.2"):
shutil.copy2(db_path, db_path.parent / f"{db_path.name}.{name}")
assert database._pre_split_backup() == pre_split
assert database._views_from_backup()["n1"]["services"] == [
{"port": 443, "protocol": "tcp", "service_name": "https"}
]
await engine.dispose()
+176
View File
@@ -0,0 +1,176 @@
"""
Tests for the hardware → properties migration logic.
We test the migration function directly against an in-memory SQLite database
so we can set up legacy rows (with hardware columns, NULL properties) and
verify the migration produces the expected properties JSON.
"""
import json
import os
os.environ.setdefault("SECRET_KEY", "test-only-secret-key-not-for-production")
import pytest
from sqlalchemy.ext.asyncio import create_async_engine
TEST_DB_URL = "sqlite+aiosqlite:///:memory:"
async def _setup_legacy_table(conn):
"""Create a minimal nodes table that mimics the pre-migration schema."""
await conn.exec_driver_sql("""
CREATE TABLE IF NOT EXISTS nodes (
id TEXT PRIMARY KEY,
type TEXT NOT NULL DEFAULT 'generic',
label TEXT NOT NULL DEFAULT '',
cpu_model TEXT,
cpu_count INTEGER,
ram_gb REAL,
disk_gb REAL,
show_hardware BOOLEAN NOT NULL DEFAULT 0,
properties JSON
)
""")
async def _run_migration(conn):
"""Run only the properties migration portion (extracted from init_db)."""
rows = await conn.exec_driver_sql(
"SELECT id, cpu_model, cpu_count, ram_gb, disk_gb, show_hardware "
"FROM nodes WHERE properties IS NULL"
)
for row in rows.fetchall():
node_id, cpu_model, cpu_count, ram_gb, disk_gb, show_hardware = row
props = []
visible = bool(show_hardware)
if cpu_model:
props.append({"key": "CPU Model", "value": str(cpu_model), "icon": "Cpu", "visible": visible})
if cpu_count is not None:
props.append({"key": "CPU Cores", "value": str(cpu_count), "icon": "Cpu", "visible": visible})
if ram_gb is not None:
props.append({"key": "RAM", "value": f"{ram_gb} GB", "icon": "MemoryStick", "visible": visible})
if disk_gb is not None:
props.append({"key": "Disk", "value": f"{disk_gb} GB", "icon": "HardDrive", "visible": visible})
await conn.exec_driver_sql(
"UPDATE nodes SET properties = ? WHERE id = ?",
(json.dumps(props), node_id),
)
async def _get_properties(conn, node_id: str) -> list:
rows = await conn.exec_driver_sql("SELECT properties FROM nodes WHERE id = ?", (node_id,))
raw = rows.fetchone()[0]
return json.loads(raw) if raw else []
@pytest.mark.asyncio
async def test_migration_full_hardware():
"""Node with all 4 hardware fields → 4 property entries with correct icons."""
engine = create_async_engine(TEST_DB_URL)
async with engine.begin() as conn:
await _setup_legacy_table(conn)
await conn.exec_driver_sql(
"INSERT INTO nodes (id, cpu_model, cpu_count, ram_gb, disk_gb, show_hardware) "
"VALUES (?, ?, ?, ?, ?, ?)",
("node-1", "i7-12700K", 12, 32.0, 2000.0, 1),
)
await _run_migration(conn)
props = await _get_properties(conn, "node-1")
assert len(props) == 4
assert props[0] == {"key": "CPU Model", "value": "i7-12700K", "icon": "Cpu", "visible": True}
assert props[1] == {"key": "CPU Cores", "value": "12", "icon": "Cpu", "visible": True}
assert props[2] == {"key": "RAM", "value": "32.0 GB", "icon": "MemoryStick", "visible": True}
assert props[3] == {"key": "Disk", "value": "2000.0 GB", "icon": "HardDrive", "visible": True}
await engine.dispose()
@pytest.mark.asyncio
async def test_migration_partial_hardware():
"""Node with only cpu_model and ram_gb → 2 property entries."""
engine = create_async_engine(TEST_DB_URL)
async with engine.begin() as conn:
await _setup_legacy_table(conn)
await conn.exec_driver_sql(
"INSERT INTO nodes (id, cpu_model, ram_gb, show_hardware) VALUES (?, ?, ?, ?)",
("node-2", "Ryzen 5 5600", 16.0, 0),
)
await _run_migration(conn)
props = await _get_properties(conn, "node-2")
assert len(props) == 2
assert props[0]["key"] == "CPU Model"
assert props[0]["visible"] is False
assert props[1]["key"] == "RAM"
assert props[1]["icon"] == "MemoryStick"
await engine.dispose()
@pytest.mark.asyncio
async def test_migration_no_hardware():
"""Node with no hardware fields → empty properties array."""
engine = create_async_engine(TEST_DB_URL)
async with engine.begin() as conn:
await _setup_legacy_table(conn)
await conn.exec_driver_sql(
"INSERT INTO nodes (id) VALUES (?)",
("node-3",),
)
await _run_migration(conn)
props = await _get_properties(conn, "node-3")
assert props == []
await engine.dispose()
@pytest.mark.asyncio
async def test_migration_idempotent():
"""Running migration twice does not duplicate properties."""
engine = create_async_engine(TEST_DB_URL)
async with engine.begin() as conn:
await _setup_legacy_table(conn)
await conn.exec_driver_sql(
"INSERT INTO nodes (id, cpu_model, show_hardware) VALUES (?, ?, ?)",
("node-4", "Core i5", 1),
)
await _run_migration(conn)
await _run_migration(conn) # second pass — node already has properties, should be skipped
props = await _get_properties(conn, "node-4")
assert len(props) == 1
await engine.dispose()
@pytest.mark.asyncio
async def test_migration_show_hardware_false_sets_visible_false():
"""show_hardware=0 means all migrated properties have visible=False."""
engine = create_async_engine(TEST_DB_URL)
async with engine.begin() as conn:
await _setup_legacy_table(conn)
await conn.exec_driver_sql(
"INSERT INTO nodes (id, cpu_model, ram_gb, show_hardware) VALUES (?, ?, ?, ?)",
("node-5", "ARM Cortex-A72", 4.0, 0),
)
await _run_migration(conn)
props = await _get_properties(conn, "node-5")
assert all(p["visible"] is False for p in props)
await engine.dispose()
@pytest.mark.asyncio
async def test_migration_already_migrated_node_not_touched():
"""Node that already has properties is skipped — existing properties preserved."""
existing = [{"key": "GPU", "value": "RTX 4090", "icon": "Monitor", "visible": True}]
engine = create_async_engine(TEST_DB_URL)
async with engine.begin() as conn:
await _setup_legacy_table(conn)
await conn.exec_driver_sql(
"INSERT INTO nodes (id, cpu_model, ram_gb, show_hardware, properties) VALUES (?, ?, ?, ?, ?)",
("node-6", "i9-13900K", 64.0, 1, json.dumps(existing)),
)
await _run_migration(conn)
props = await _get_properties(conn, "node-6")
assert props == existing
await engine.dispose()
View File
+84
View File
@@ -0,0 +1,84 @@
"""Fixtures shared across the scan test modules."""
import uuid
import pytest
from app.db.models import InventoryDevice
@pytest.fixture
async def pending_device(db_session):
import uuid
device = InventoryDevice(
id=str(uuid.uuid4()),
ip="192.168.1.100",
mac="aa:bb:cc:dd:ee:ff",
hostname="my-server",
os="Linux",
services=[{"port": 22, "name": "ssh"}],
suggested_type="server",
status="pending",
)
db_session.add(device)
await db_session.commit()
await db_session.refresh(device)
return device
@pytest.fixture
async def mem_db():
from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine
from app.db.database import Base
engine = create_async_engine("sqlite+aiosqlite:///:memory:")
async with engine.begin() as conn:
await conn.run_sync(Base.metadata.create_all)
factory = async_sessionmaker(engine, expire_on_commit=False)
yield factory
await engine.dispose()
@pytest.fixture
async def two_device_inventory(db_session):
devices = []
for i in range(2):
d = InventoryDevice(
id=str(uuid.uuid4()),
ip=f"192.168.1.{10 + i}",
mac=None,
hostname=f"host-{i}",
os=None,
services=[],
suggested_type="generic",
status="pending",
)
db_session.add(d)
devices.append(d)
await db_session.commit()
for d in devices:
await db_session.refresh(d)
return devices
@pytest.fixture
async def zigbee_pending_device(db_session):
device = InventoryDevice(
id=str(uuid.uuid4()),
ip=None,
mac=None,
hostname=None,
friendly_name="bulb_1",
services=[],
suggested_type="zigbee_enddevice",
device_subtype="EndDevice",
ieee_address="0xABCDEF",
vendor="IKEA",
model="TRADFRI",
lqi=180,
status="pending",
discovery_source="zigbee",
)
db_session.add(device)
await db_session.commit()
await db_session.refresh(device)
return device
+83
View File
@@ -0,0 +1,83 @@
"""Shared builders for scan test suite (pure helpers, no fixtures)."""
import uuid
from app.db.models import Design, InventoryDevice, Node
async def _add_design(db_session, name: str) -> str:
design = Design(id=str(uuid.uuid4()), name=name)
db_session.add(design)
await db_session.commit()
return design.id
def _node(design_id: str, *, ip=None, ieee=None, mac=None, device_id=None) -> Node:
"""A canvas node. Addresses belong to the inventory row it draws, so pass
`device_id`, or let `_node_for` mint the row from the addresses."""
return Node(
id=str(uuid.uuid4()), label="n", type="server", pos_x=0.0, pos_y=0.0,
design_id=design_id, device_id=device_id,
)
async def _node_for(db_session, design_id: str, *, ip=None, ieee=None, mac=None, **device_kwargs) -> Node:
"""A node plus the inventory row carrying its addresses, linked.
Reuses a row that already describes these addresses — one device is one row,
so a test placing the same host on a second canvas gets a second node, not a
second device.
"""
from app.services.inventory_sync import find_device_for
device = await find_device_for(db_session, ip=ip, mac=mac, ieee=ieee)
if device is None:
device = InventoryDevice(
id=str(uuid.uuid4()), ip=ip, mac=mac, ieee_address=ieee,
status="approved", status_live="online", **device_kwargs,
)
db_session.add(device)
await db_session.flush()
node = _node(design_id, device_id=device.id)
db_session.add(node)
await db_session.commit()
return node
async def _seed_zigbee_pending_pair(db_session):
"""Create a coordinator Node + a pending device + a link between them."""
from app.db.models import InventoryDeviceLink, Node
coord_device = InventoryDevice(
ieee_address="0xCOORD",
friendly_name="Coordinator",
suggested_type="zigbee_coordinator",
status="approved",
)
db_session.add(coord_device)
await db_session.flush()
coord = Node(
label="Coordinator",
type="zigbee_coordinator",
device_id=coord_device.id,
)
db_session.add(coord)
pending = InventoryDevice(
ieee_address="0xR1",
friendly_name="router_1",
suggested_type="zigbee_router",
device_subtype="Router",
status="pending",
discovery_source="zigbee",
)
db_session.add(pending)
db_session.add(
InventoryDeviceLink(
source_ieee="0xCOORD",
target_ieee="0xR1",
discovery_source="zigbee",
)
)
await db_session.commit()
return coord, pending
+738
View File
@@ -0,0 +1,738 @@
"""Approve / hide / restore / ignore / bulk device flows and conflict handling."""
import uuid
import pytest
from httpx import AsyncClient
from sqlalchemy import select
from app.db.models import Design, InventoryDevice, Node
from tests.scan.helpers import _add_design, _node_for
@pytest.mark.asyncio
async def test_canvas_count_ignores_nodes_without_design(client, headers, db_session, pending_device):
# A node with no design_id is not "on a canvas".
await _node_for(db_session, None, ip="192.168.1.100")
await db_session.commit()
res = await client.get("/api/v1/scan/pending", headers=headers)
assert res.json()[0]["canvas_count"] == 0
@pytest.mark.asyncio
async def test_approve_device(client: AsyncClient, headers, pending_device):
node_payload = {
"label": "My Server",
"type": "server",
"ip": "192.168.1.100",
"hostname": "my-server",
"status": "unknown",
"services": [],
}
res = await client.post(
f"/api/v1/scan/pending/{pending_device.id}/approve",
json=node_payload,
headers=headers,
)
assert res.status_code == 200
data = res.json()
assert data["approved"] is True
assert "node_id" in data
# Approved devices stay in the inventory (status != "hidden") so they keep
# showing with an "In N canvas" badge — they are no longer dropped.
pending_res = await client.get("/api/v1/scan/pending", headers=headers)
inventory = pending_res.json()
assert len(inventory) == 1
assert inventory[0]["id"] == pending_device.id
assert inventory[0]["status"] == "approved"
@pytest.mark.asyncio
async def test_approve_device_conflicts_on_existing_ieee_same_design(
client: AsyncClient, headers, db_session
):
"""Approving a device whose IEEE is already on the target design prompts the
user (409) instead of silently merging/replacing — same UX as ip/mac."""
design = Design(name="d1")
db_session.add(design)
await db_session.flush()
device = InventoryDevice(
id=str(uuid.uuid4()), ieee_address="0xZZZ", suggested_type="zigbee_enddevice",
status="pending", discovery_source="zigbee",
)
db_session.add(device)
await db_session.flush()
# The node already on this design draws that very row.
existing = Node(
label="sensor", type="zigbee_enddevice", device_id=device.id, design_id=design.id,
)
db_session.add(existing)
await db_session.commit()
res = await client.post(
f"/api/v1/scan/pending/{device.id}/approve",
json={
"label": "sensor", "type": "zigbee_enddevice",
"status": "online", "services": [], "design_id": design.id,
},
headers=headers,
)
assert res.status_code == 409
detail = res.json()["detail"]
assert detail["duplicate"] is True
assert detail["existing_node_id"] == existing.id
assert detail["match"] == "ieee"
assert detail["value"] == "0xZZZ"
# No second node created; device stays pending until the user decides.
nodes = (
await db_session.execute(select(Node).join(InventoryDevice, Node.device_id == InventoryDevice.id).where(InventoryDevice.ieee_address == "0xZZZ"))
).scalars().all()
assert len(nodes) == 1
@pytest.mark.asyncio
async def test_approve_device_force_creates_duplicate_ieee(
client: AsyncClient, headers, db_session
):
"""force=True lets the user place a second card for the same IEEE."""
design = Design(name="d1")
db_session.add(design)
await db_session.flush()
device = InventoryDevice(
id=str(uuid.uuid4()), ieee_address="0xZZZ", suggested_type="zigbee_enddevice",
status="pending", discovery_source="zigbee",
)
db_session.add(device)
await db_session.flush()
db_session.add(Node(
label="sensor", type="zigbee_enddevice", device_id=device.id, design_id=design.id,
))
await db_session.commit()
res = await client.post(
f"/api/v1/scan/pending/{device.id}/approve",
json={
"label": "sensor", "type": "zigbee_enddevice", "status": "online",
"services": [], "design_id": design.id, "force": True,
},
headers=headers,
)
assert res.status_code == 200
nodes = (
await db_session.execute(select(Node).join(InventoryDevice, Node.device_id == InventoryDevice.id).where(InventoryDevice.ieee_address == "0xZZZ"))
).scalars().all()
assert len(nodes) == 2
@pytest.mark.asyncio
async def test_approve_device_conflicts_on_existing_ip(
client: AsyncClient, headers, db_session, pending_device
):
"""An ordinary host whose ip already sits on the target design is NOT
silently duplicated: the approve returns 409 with the existing node so the
UI can ask the user."""
design = await _add_design(db_session, "Home")
existing = await _node_for(db_session, design, ip="192.168.1.100")
await db_session.commit()
res = await client.post(
f"/api/v1/scan/pending/{pending_device.id}/approve",
json={"label": "dup", "type": "server", "ip": "192.168.1.100",
"status": "unknown", "services": [], "design_id": design},
headers=headers,
)
assert res.status_code == 409
detail = res.json()["detail"]
assert detail["duplicate"] is True
assert detail["existing_node_id"] == existing.id
assert detail["match"] == "ip"
assert detail["value"] == "192.168.1.100"
# No node created, device left pending (user hasn't decided yet).
nodes = (await db_session.execute(select(Node).where(Node.design_id == design))).scalars().all()
assert len(nodes) == 1
await db_session.refresh(pending_device)
assert pending_device.status == "pending"
@pytest.mark.asyncio
async def test_approve_device_conflicts_on_existing_mac(
client: AsyncClient, headers, db_session, pending_device
):
"""MAC match (device re-IP'd via DHCP) also triggers the duplicate guard."""
design = await _add_design(db_session, "Home")
# The node already on this design draws a row carrying that MAC.
await _node_for(db_session, design, ip="10.0.0.9", mac="aa:bb:cc:dd:ee:ff")
# pending_device carries mac aa:bb:cc:dd:ee:ff but a different ip.
res = await client.post(
f"/api/v1/scan/pending/{pending_device.id}/approve",
json={"label": "dup", "type": "server", "ip": "192.168.1.55",
"mac": "aa:bb:cc:dd:ee:ff", "status": "unknown", "services": [],
"design_id": design},
headers=headers,
)
assert res.status_code == 409
assert res.json()["detail"]["match"] == "mac"
@pytest.mark.asyncio
async def test_approve_device_conflicts_on_ip_in_comma_list(
client: AsyncClient, headers, db_session, pending_device
):
"""The existing node's ip holds an IPv6 before the IPv4 the device scanned
as. Exact-string matching missed it (issue #258); per-token matching catches
the duplicate."""
design = await _add_design(db_session, "Home")
existing = await _node_for(db_session, design, ip="fe80::1, 192.168.1.100")
await db_session.commit()
res = await client.post(
f"/api/v1/scan/pending/{pending_device.id}/approve",
json={"label": "dup", "type": "server", "ip": "192.168.1.100",
"status": "unknown", "services": [], "design_id": design},
headers=headers,
)
assert res.status_code == 409
detail = res.json()["detail"]
assert detail["existing_node_id"] == existing.id
assert detail["match"] == "ip"
assert detail["value"] == "192.168.1.100"
@pytest.mark.asyncio
async def test_approve_device_no_conflict_on_ip_substring(
client: AsyncClient, headers, db_session, pending_device
):
"""The ip guard must match whole addresses, not substrings: a node at
10.0.0.40 is not a duplicate of a device at 10.0.0.4."""
design = await _add_design(db_session, "Home")
await _node_for(db_session, design, ip="10.0.0.40")
await db_session.commit()
res = await client.post(
f"/api/v1/scan/pending/{pending_device.id}/approve",
json={"label": "new", "type": "server", "ip": "10.0.0.4",
"mac": None, "status": "unknown", "services": [], "design_id": design},
headers=headers,
)
assert res.status_code == 200
@pytest.mark.asyncio
async def test_approve_device_force_creates_duplicate(
client: AsyncClient, headers, db_session, pending_device
):
"""force=True (user confirmed) bypasses the guard and creates the node."""
design = await _add_design(db_session, "Home")
await _node_for(db_session, design, ip="192.168.1.100")
await db_session.commit()
res = await client.post(
f"/api/v1/scan/pending/{pending_device.id}/approve",
json={"label": "dup", "type": "server", "ip": "192.168.1.100",
"status": "unknown", "services": [], "design_id": design, "force": True},
headers=headers,
)
assert res.status_code == 200
nodes = (await db_session.execute(select(Node).where(Node.design_id == design))).scalars().all()
assert len(nodes) == 2 # duplicate deliberately created
@pytest.mark.asyncio
async def test_approve_device_allows_same_ip_on_other_design(
client: AsyncClient, headers, db_session, pending_device
):
"""The guard is per-design: the same host on a different canvas is fine."""
other = await _add_design(db_session, "Lab")
target = await _add_design(db_session, "Home")
await _node_for(db_session, other, ip="192.168.1.100") # exists on a DIFFERENT design
await db_session.commit()
res = await client.post(
f"/api/v1/scan/pending/{pending_device.id}/approve",
json={"label": "ok", "type": "server", "ip": "192.168.1.100",
"status": "unknown", "services": [], "design_id": target},
headers=headers,
)
assert res.status_code == 200
@pytest.mark.asyncio
async def test_approve_device_places_already_approved_on_another_design(
client: AsyncClient, headers, db_session
):
"""A device already approved on ANOTHER canvas (global status="approved")
must still be placeable on a new design — status is global, canvas
membership is per-design (mirrors bulk_approve)."""
other = await _add_design(db_session, "Other")
target = await _add_design(db_session, "Network Topology")
# Device is on `other` already (its global status is "approved").
device = InventoryDevice(
id=str(uuid.uuid4()), ieee_address="0x00158d0005292b83",
suggested_type="zigbee_enddevice", status="approved",
discovery_source="zigbee",
)
db_session.add(device)
await db_session.flush()
db_session.add(Node(
label="sensor", type="zigbee_enddevice", device_id=device.id, design_id=other,
))
await db_session.commit()
res = await client.post(
f"/api/v1/scan/pending/{device.id}/approve",
json={"label": "sensor", "type": "zigbee_enddevice", "status": "online",
"services": [], "design_id": target},
headers=headers,
)
assert res.status_code == 200
# A node now exists on the target design too (one per canvas).
nodes = (
await db_session.execute(
select(Node).join(InventoryDevice, Node.device_id == InventoryDevice.id).where(InventoryDevice.ieee_address == "0x00158d0005292b83")
)
).scalars().all()
assert {n.design_id for n in nodes} == {other, target}
@pytest.mark.asyncio
async def test_approve_device_rejects_hidden(client: AsyncClient, headers, db_session, pending_device):
"""A user-hidden device is not approvable via this endpoint."""
pending_device.status = "hidden"
db_session.add(pending_device)
await db_session.commit()
res = await client.post(
f"/api/v1/scan/pending/{pending_device.id}/approve",
json={"label": "x", "type": "server", "status": "unknown", "services": []},
headers=headers,
)
assert res.status_code == 409
@pytest.mark.asyncio
async def test_approve_nonexistent_device(client: AsyncClient, headers):
node_payload = {
"label": "Ghost",
"type": "generic",
"ip": "10.0.0.1",
"status": "unknown",
"services": [],
}
res = await client.post(
"/api/v1/scan/pending/nonexistent-id/approve",
json=node_payload,
headers=headers,
)
assert res.status_code == 404
@pytest.mark.asyncio
async def test_hide_device(client: AsyncClient, headers, pending_device):
res = await client.post(f"/api/v1/scan/pending/{pending_device.id}/hide", headers=headers)
assert res.status_code == 200
assert res.json()["hidden"] is True
# Should no longer appear in pending
pending_res = await client.get("/api/v1/scan/pending", headers=headers)
assert pending_res.json() == []
# Should appear in hidden
hidden_res = await client.get("/api/v1/scan/hidden", headers=headers)
assert len(hidden_res.json()) == 1
@pytest.mark.asyncio
async def test_restore_device(client: AsyncClient, headers, pending_device):
# Hide first
await client.post(f"/api/v1/scan/pending/{pending_device.id}/hide", headers=headers)
# Restore
res = await client.post(f"/api/v1/scan/pending/{pending_device.id}/restore", headers=headers)
assert res.status_code == 200
assert res.json()["restored"] is True
# Now back in pending, gone from hidden
pending_res = await client.get("/api/v1/scan/pending", headers=headers)
assert len(pending_res.json()) == 1
hidden_res = await client.get("/api/v1/scan/hidden", headers=headers)
assert hidden_res.json() == []
@pytest.mark.asyncio
async def test_restore_device_rejects_non_hidden(client: AsyncClient, headers, pending_device):
res = await client.post(f"/api/v1/scan/pending/{pending_device.id}/restore", headers=headers)
assert res.status_code == 409
@pytest.mark.asyncio
async def test_bulk_restore_devices(client: AsyncClient, headers, pending_device):
# Hide
await client.post(f"/api/v1/scan/pending/{pending_device.id}/hide", headers=headers)
res = await client.post(
"/api/v1/scan/pending/bulk-restore",
headers=headers,
json={"device_ids": [pending_device.id]},
)
assert res.status_code == 200
assert res.json()["restored"] == 1
assert res.json()["skipped"] == 0
pending_res = await client.get("/api/v1/scan/pending", headers=headers)
assert len(pending_res.json()) == 1
@pytest.mark.asyncio
async def test_ignore_device(client: AsyncClient, headers, pending_device):
res = await client.post(f"/api/v1/scan/pending/{pending_device.id}/ignore", headers=headers)
assert res.status_code == 200
assert res.json()["ignored"] is True
# Device should be gone from both pending and hidden
pending_res = await client.get("/api/v1/scan/pending", headers=headers)
assert pending_res.json() == []
hidden_res = await client.get("/api/v1/scan/hidden", headers=headers)
assert hidden_res.json() == []
@pytest.mark.asyncio
async def test_bulk_approve_approves_devices(client: AsyncClient, headers, two_device_inventory):
ids = [d.id for d in two_device_inventory]
res = await client.post("/api/v1/scan/pending/bulk-approve", json={"device_ids": ids}, headers=headers)
assert res.status_code == 200
data = res.json()
assert data["approved"] == 2
assert len(data["node_ids"]) == 2
assert all(nid is not None for nid in data["node_ids"]), "node_ids must be non-null UUIDs"
assert len(data["device_ids"]) == 2
assert data["skipped"] == 0
# Approved devices stay in the inventory, now marked "approved".
pending_res = await client.get("/api/v1/scan/pending", headers=headers)
inventory = pending_res.json()
assert len(inventory) == 2
assert all(d["status"] == "approved" for d in inventory)
@pytest.mark.asyncio
async def test_bulk_approve_places_already_approved_device_on_another_design(
client: AsyncClient, headers, db_session, two_device_inventory
):
"""Regression: a device already approved (status='approved', e.g. placed on
another canvas) must still get a node on the design being approved onto.
Previously bulk-approve filtered status=='pending', so selecting an
already-approved device created no node — the user saw fewer nodes than
they selected."""
ids = [d.id for d in two_device_inventory]
design_a = await _add_design(db_session, "Canvas A")
design_b = await _add_design(db_session, "Canvas B")
# Approve both onto design A.
res_a = await client.post(
"/api/v1/scan/pending/bulk-approve",
json={"device_ids": ids, "design_id": design_a},
headers=headers,
)
assert res_a.json()["approved"] == 2
# Re-approve the same (now status='approved') devices onto design B.
res_b = await client.post(
"/api/v1/scan/pending/bulk-approve",
json={"device_ids": ids, "design_id": design_b},
headers=headers,
)
data_b = res_b.json()
assert data_b["approved"] == 2, "already-approved devices must place onto the new canvas"
assert data_b["skipped"] == 0
# Two nodes now exist on each design.
from app.db.models import Node as NodeModel
nodes_b = (
await db_session.execute(select(NodeModel).where(NodeModel.design_id == design_b))
).scalars().all()
assert len(nodes_b) == 2
@pytest.mark.asyncio
async def test_bulk_approve_skips_device_already_on_target_design(
client: AsyncClient, headers, db_session, two_device_inventory
):
"""A device already on the target canvas (same ip) is not placed twice."""
ids = [d.id for d in two_device_inventory]
design = await _add_design(db_session, "Canvas")
# First device already sits on the canvas (matched by ip).
await _node_for(db_session, design, ip="192.168.1.10")
await db_session.commit()
res = await client.post(
"/api/v1/scan/pending/bulk-approve",
json={"device_ids": ids, "design_id": design},
headers=headers,
)
data = res.json()
assert data["approved"] == 1 # only the second device (192.168.1.11)
assert data["skipped"] == 1
from app.db.models import Node as NodeModel
nodes = (
await db_session.execute(select(NodeModel).where(NodeModel.design_id == design))
).scalars().all()
devices = [await db_session.get(InventoryDevice, n.device_id) for n in nodes]
# The pre-existing node plus the one newly approved — no duplicate for .10.
assert len(nodes) == 2
assert sorted(d.ip for d in devices) == ["192.168.1.10", "192.168.1.11"]
@pytest.mark.asyncio
async def test_bulk_approve_skips_device_matching_ip_in_comma_list(
client: AsyncClient, headers, db_session, two_device_inventory
):
"""The on-canvas node's ip holds an IPv6 before the IPv4; the device scanned
as the plain IPv4 is still recognised as already placed (issue #258)."""
ids = [d.id for d in two_device_inventory]
design = await _add_design(db_session, "Canvas")
await _node_for(db_session, design, ip="fe80::1, 192.168.1.10")
await db_session.commit()
res = await client.post(
"/api/v1/scan/pending/bulk-approve",
json={"device_ids": ids, "design_id": design},
headers=headers,
)
data = res.json()
assert data["approved"] == 1 # only the second device (192.168.1.11)
assert data["skipped"] == 1
assert data["skipped_devices"][0]["value"] == "192.168.1.10"
@pytest.mark.asyncio
async def test_bulk_approve_reports_skipped_devices(
client: AsyncClient, headers, db_session, two_device_inventory
):
"""Bulk can't prompt per-device, so it reports each duplicate it skipped
(with the existing node id) instead of silently dropping it."""
ids = [d.id for d in two_device_inventory]
design = await _add_design(db_session, "Canvas")
existing = await _node_for(db_session, design, ip="192.168.1.10")
await db_session.commit()
res = await client.post(
"/api/v1/scan/pending/bulk-approve",
json={"device_ids": ids, "design_id": design},
headers=headers,
)
data = res.json()
assert data["approved"] == 1
skipped = data["skipped_devices"]
assert len(skipped) == 1
entry = skipped[0]
assert entry["match"] == "ip"
assert entry["value"] == "192.168.1.10"
assert entry["existing_node_id"] == existing.id
assert entry["device_id"] in ids
@pytest.mark.asyncio
async def test_approve_device_copies_mac_to_node_and_properties(
client: AsyncClient, headers, pending_device, db_session
):
"""Approving a scanned device must carry its MAC onto the node + properties."""
from sqlalchemy import select
from app.db.models import Node as NodeModel
# Payload intentionally omits mac — it must come from the pending device.
res = await client.post(
f"/api/v1/scan/pending/{pending_device.id}/approve",
json={"label": "My Server", "type": "server", "ip": "192.168.1.100", "status": "unknown", "services": []},
headers=headers,
)
assert res.status_code == 200
node = (
await db_session.execute(select(NodeModel).where(NodeModel.device_id.is_not(None)))
).scalars().first()
assert node is not None
# The device facts belong to the inventory row the node draws.
device = await db_session.get(InventoryDevice, node.device_id)
assert device is not None
assert device.mac == "aa:bb:cc:dd:ee:ff"
mac_props = [p for p in device.properties if p["key"] == "MAC"]
assert mac_props == [
{"key": "MAC", "value": "aa:bb:cc:dd:ee:ff", "icon": None, "visible": False}
]
@pytest.mark.asyncio
async def test_bulk_approve_copies_mac_to_node_and_properties(
client: AsyncClient, headers, db_session
):
"""Bulk approve must also propagate the scanned MAC to node + properties."""
from sqlalchemy import select
from app.db.models import Node as NodeModel
device = InventoryDevice(
id=str(uuid.uuid4()),
ip="192.168.1.55",
mac="11:22:33:44:55:66",
hostname="host-mac",
services=[],
suggested_type="generic",
status="pending",
)
db_session.add(device)
await db_session.commit()
res = await client.post(
"/api/v1/scan/pending/bulk-approve",
json={"device_ids": [device.id]},
headers=headers,
)
assert res.status_code == 200
node = (
await db_session.execute(select(NodeModel).where(NodeModel.device_id.is_not(None)))
).scalars().first()
assert node is not None
# The device facts belong to the inventory row the node draws.
device = await db_session.get(InventoryDevice, node.device_id)
assert device is not None
assert device.mac == "11:22:33:44:55:66"
mac_props = [p for p in device.properties if p["key"] == "MAC"]
assert mac_props == [
{"key": "MAC", "value": "11:22:33:44:55:66", "icon": None, "visible": False}
]
@pytest.mark.asyncio
async def test_bulk_approve_sets_default_check_method(client: AsyncClient, headers, two_device_inventory, db_session):
"""Approved devices with an IP must default to ping; otherwise scheduler skips them."""
from sqlalchemy import select
from app.db.models import Node as NodeModel
ids = [d.id for d in two_device_inventory]
res = await client.post("/api/v1/scan/pending/bulk-approve", json={"device_ids": ids}, headers=headers)
assert res.status_code == 200
nodes = (await db_session.execute(select(NodeModel))).scalars().all()
for n in nodes:
device = await db_session.get(InventoryDevice, n.device_id)
assert device is not None
if device.ip:
assert device.check_method == "ping", f"device {device.id} has no check_method"
@pytest.mark.asyncio
async def test_approve_device_sets_default_check_method(client: AsyncClient, headers, pending_device, db_session):
from sqlalchemy import select
from app.db.models import Node as NodeModel
res = await client.post(
f"/api/v1/scan/pending/{pending_device.id}/approve",
json={"label": "h", "type": "generic", "ip": "192.168.1.10", "status": "unknown", "services": []},
headers=headers,
)
assert res.status_code == 200
node = (await db_session.execute(select(NodeModel))).scalars().first()
assert node is not None
device = await db_session.get(InventoryDevice, node.device_id)
assert device is not None
assert device.check_method == "ping"
@pytest.mark.asyncio
async def test_bulk_approve_skips_already_approved(client: AsyncClient, headers, two_device_inventory):
ids = [d.id for d in two_device_inventory]
# Approve first device individually first
await client.post(
f"/api/v1/scan/pending/{ids[0]}/approve",
json={"label": "h", "type": "generic", "ip": "192.168.1.10", "status": "unknown", "services": []},
headers=headers,
)
# Bulk approve both — first one is already approved (not pending), should be skipped
res = await client.post("/api/v1/scan/pending/bulk-approve", json={"device_ids": ids}, headers=headers)
assert res.status_code == 200
data = res.json()
assert data["approved"] == 1
assert data["skipped"] == 1
@pytest.mark.asyncio
async def test_bulk_approve_requires_auth(client: AsyncClient, two_device_inventory):
ids = [d.id for d in two_device_inventory]
res = await client.post("/api/v1/scan/pending/bulk-approve", json={"device_ids": ids})
assert res.status_code == 401
@pytest.mark.asyncio
async def test_bulk_hide_hides_devices(client: AsyncClient, headers, two_device_inventory):
ids = [d.id for d in two_device_inventory]
res = await client.post("/api/v1/scan/pending/bulk-hide", json={"device_ids": ids}, headers=headers)
assert res.status_code == 200
data = res.json()
assert data["hidden"] == 2
assert data["skipped"] == 0
# Should appear in hidden list
hidden_res = await client.get("/api/v1/scan/hidden", headers=headers)
assert len(hidden_res.json()) == 2
@pytest.mark.asyncio
async def test_bulk_hide_skips_non_pending(client: AsyncClient, headers, two_device_inventory):
ids = [d.id for d in two_device_inventory]
# Hide first device individually first
await client.post(f"/api/v1/scan/pending/{ids[0]}/hide", headers=headers)
# Bulk hide both — first is already hidden (not pending anymore)
res = await client.post("/api/v1/scan/pending/bulk-hide", json={"device_ids": ids}, headers=headers)
assert res.status_code == 200
data = res.json()
assert data["hidden"] == 1
assert data["skipped"] == 1
@pytest.mark.asyncio
async def test_bulk_hide_requires_auth(client: AsyncClient, two_device_inventory):
ids = [d.id for d in two_device_inventory]
res = await client.post("/api/v1/scan/pending/bulk-hide", json={"device_ids": ids})
assert res.status_code == 401
@pytest.mark.asyncio
async def test_bulk_approve_targets_requested_design(client, headers, db_session):
"""bulk-approve must place nodes on the design_id sent by the UI, not the
first design — otherwise approved devices land on the wrong canvas."""
first = await _add_design(db_session, "Default") # first design (fallback)
active = await _add_design(db_session, "zwave") # the design the user is on
dev = InventoryDevice(
id=str(uuid.uuid4()),
ieee_address="zwave-H-2",
friendly_name="Living Room Plug",
suggested_type="zwave_router",
device_subtype="Router",
vendor="Aeotec",
model="ZW096",
status="pending",
discovery_source="zwave",
)
db_session.add(dev)
await db_session.commit()
res = await client.post(
"/api/v1/scan/pending/bulk-approve",
json={"device_ids": [dev.id], "design_id": active},
headers=headers,
)
assert res.status_code == 200
assert res.json()["approved"] == 1
node = (
await db_session.execute(select(Node).join(InventoryDevice, Node.device_id == InventoryDevice.id).where(InventoryDevice.ieee_address == "zwave-H-2"))
).scalar_one()
assert node.design_id == active
assert node.design_id != first
# Z-Wave device → online + Z-Wave property rows, no ICMP check. All of that
# is the device's, so it is read off the row the node draws.
device = await db_session.get(InventoryDevice, node.device_id)
assert device is not None
assert device.status_live == "online"
assert device.check_method == "none"
assert {p["key"] for p in device.properties} == {"Z-Wave ID", "Vendor", "Model"}
@@ -0,0 +1,165 @@
"""Editing an inventory row: PATCH /scan/pending/{id} and the widened create.
The inventory row owns the device facts, so it has to be editable on its own —
not only through a canvas node.
"""
import pytest
from httpx import AsyncClient
from app.db.models import InventoryDevice
async def _seed(db_session, **kwargs) -> InventoryDevice:
device = InventoryDevice(
ip="192.168.1.10",
mac="aa:bb:cc:11:22:33",
hostname="nas",
services=[{"port": 22, "protocol": "tcp", "service_name": "ssh"}],
suggested_type="nas",
status="pending",
discovery_source="arp",
discovery_sources=["arp"],
properties=[{"key": "Rack", "value": "A1", "icon": None, "visible": True}],
**kwargs,
)
db_session.add(device)
await db_session.commit()
return device
@pytest.mark.asyncio
async def test_update_pending_requires_auth(client: AsyncClient, db_session):
device = await _seed(db_session)
res = await client.patch(f"/api/v1/scan/pending/{device.id}", json={"hostname": "x"})
assert res.status_code == 401
@pytest.mark.asyncio
async def test_update_pending_edits_curated_fields(client: AsyncClient, headers, db_session):
device = await _seed(db_session)
res = await client.patch(
f"/api/v1/scan/pending/{device.id}",
headers=headers,
json={
"label": "Big NAS",
"type": "nas",
"notes": "in the garage rack",
"cpu_count": 4,
"cpu_model": "N5105",
"ram_gb": 16.0,
"disk_gb": 4000.0,
"show_hardware": True,
"check_method": "http",
"check_target": "http://192.168.1.10:5000",
"properties": [
{"key": "Rack", "value": "B2", "icon": None, "visible": True},
{"key": "Owner", "value": "me", "icon": None, "visible": False},
],
"services": [
{"port": 22, "protocol": "tcp", "service_name": "ssh"},
{"port": 5000, "protocol": "tcp", "service_name": "http"},
],
},
)
assert res.status_code == 200
data = res.json()
assert data["label"] == "Big NAS"
assert data["notes"] == "in the garage rack"
assert data["cpu_count"] == 4
assert data["ram_gb"] == 16.0
assert data["show_hardware"] is True
assert data["check_method"] == "http"
assert len(data["properties"]) == 2
assert len(data["services"]) == 2
@pytest.mark.asyncio
async def test_update_pending_is_partial(client: AsyncClient, headers, db_session):
"""Sending one field must not clear the others."""
device = await _seed(db_session)
res = await client.patch(
f"/api/v1/scan/pending/{device.id}", headers=headers, json={"notes": "only this"}
)
assert res.status_code == 200
data = res.json()
assert data["notes"] == "only this"
assert data["hostname"] == "nas"
assert data["ip"] == "192.168.1.10"
assert data["mac"] == "aa:bb:cc:11:22:33"
assert data["suggested_type"] == "nas"
assert len(data["services"]) == 1
assert data["properties"][0]["key"] == "Rack"
@pytest.mark.asyncio
async def test_update_pending_normalizes_mac(client: AsyncClient, headers, db_session):
device = await _seed(db_session)
res = await client.patch(
f"/api/v1/scan/pending/{device.id}", headers=headers, json={"mac": "AA-BB-CC-99-88-77"}
)
assert res.status_code == 200
assert res.json()["mac"] == "aa:bb:cc:99:88:77"
@pytest.mark.asyncio
async def test_update_pending_leaves_lifecycle_alone(client: AsyncClient, headers, db_session):
"""`status` is owned by approve/hide, not by the edit modal."""
device = await _seed(db_session)
res = await client.patch(
f"/api/v1/scan/pending/{device.id}",
headers=headers,
json={"label": "x", "status": "approved", "discovery_source": "manual"},
)
assert res.status_code == 200
data = res.json()
assert data["label"] == "x"
assert data["status"] == "pending"
assert data["discovery_source"] == "arp"
@pytest.mark.asyncio
async def test_update_pending_unknown_id(client: AsyncClient, headers):
res = await client.patch("/api/v1/scan/pending/nope", headers=headers, json={"notes": "x"})
assert res.status_code == 404
@pytest.mark.asyncio
async def test_create_pending_accepts_curated_fields(client: AsyncClient, headers):
"""A hand-made entry carries everything the modal shows, no PATCH round trip."""
res = await client.post(
"/api/v1/scan/pending",
headers=headers,
json={
"hostname": "patch-panel",
"os": "n/a",
"label": "Patch panel 24p",
"type": "switch",
"notes": "passive",
"services": [{"port": 80, "protocol": "tcp", "service_name": "http"}],
"friendly_name": "PP-1",
"device_subtype": "passive",
"check_method": "none",
"discovery_source": "manual",
},
)
assert res.status_code == 201
data = res.json()
assert data["label"] == "Patch panel 24p"
assert data["type"] == "switch"
assert data["os"] == "n/a"
assert data["notes"] == "passive"
assert data["friendly_name"] == "PP-1"
assert data["services"][0]["port"] == 80
assert data["status"] == "pending"
@pytest.mark.asyncio
async def test_legacy_row_defaults_are_coerced(client: AsyncClient, headers, db_session):
"""Rows predating the 3.3.0 columns read back with defaults, not nulls."""
device = await _seed(db_session)
res = await client.get("/api/v1/scan/pending", headers=headers)
assert res.status_code == 200
row = next(d for d in res.json() if d["id"] == device.id)
assert row["status_live"] == "unknown"
assert row["show_hardware"] is False
assert row["label"] is None
+352
View File
@@ -0,0 +1,352 @@
"""MAC-property builders and Zigbee/Z-Wave property population on approve."""
import uuid
import pytest
from httpx import AsyncClient
from sqlalchemy import select
from app.db.models import InventoryDevice, Node
from tests.scan.helpers import _add_design, _seed_zigbee_pending_pair
@pytest.mark.asyncio
async def test_approve_zigbee_device_populates_properties(
client: AsyncClient, headers, zigbee_pending_device, db_session
):
"""Approving a zigbee device must populate IEEE/Vendor/Model/LQI in properties."""
from sqlalchemy import select
from app.db.models import Node as NodeModel
payload = {
"label": "bulb_1",
"type": "zigbee_enddevice",
"status": "online",
"services": [],
"check_method": "none",
}
res = await client.post(
f"/api/v1/scan/pending/{zigbee_pending_device.id}/approve",
json=payload,
headers=headers,
)
assert res.status_code == 200
node = (
await db_session.execute(select(NodeModel).where(NodeModel.device_id.is_not(None)))
).scalars().first()
assert node is not None
# The device facts belong to the inventory row the node draws.
device = await db_session.get(InventoryDevice, node.device_id)
assert device is not None
keys = {p["key"]: p["value"] for p in device.properties}
assert keys == {
"IEEE": "0xABCDEF",
"Vendor": "IKEA",
"Model": "TRADFRI",
"LQI": "180",
}
@pytest.mark.asyncio
async def test_bulk_approve_zigbee_populates_properties(
client: AsyncClient, headers, zigbee_pending_device, db_session
):
from sqlalchemy import select
from app.db.models import Node as NodeModel
res = await client.post(
"/api/v1/scan/pending/bulk-approve",
json={"device_ids": [zigbee_pending_device.id]},
headers=headers,
)
assert res.status_code == 200
node = (
await db_session.execute(select(NodeModel).where(NodeModel.device_id.is_not(None)))
).scalars().first()
assert node is not None
# The device facts belong to the inventory row the node draws.
device = await db_session.get(InventoryDevice, node.device_id)
assert device is not None
keys = {p["key"]: p["value"] for p in device.properties}
assert keys["IEEE"] == "0xABCDEF"
assert keys["Vendor"] == "IKEA"
assert keys["Model"] == "TRADFRI"
assert keys["LQI"] == "180"
assert device.check_method == "none"
def test_build_mac_property_returns_hidden_row():
from app.api.routes.scan import build_mac_property
assert build_mac_property("aa:bb:cc:dd:ee:ff") == [
{"key": "MAC", "value": "aa:bb:cc:dd:ee:ff", "icon": None, "visible": False}
]
def test_build_mac_property_empty_when_no_mac():
from app.api.routes.scan import build_mac_property
assert build_mac_property(None) == []
assert build_mac_property("") == []
def test_merge_mac_property_appends_when_absent():
from app.api.routes.scan import merge_mac_property
existing = [{"key": "Custom", "value": "x", "icon": None, "visible": True}]
merged = merge_mac_property(existing, "aa:bb:cc:dd:ee:ff")
assert {"key": "MAC", "value": "aa:bb:cc:dd:ee:ff", "icon": None, "visible": False} in merged
# Existing prop preserved untouched.
assert existing[0] in merged
def test_merge_mac_property_idempotent_and_preserves_visibility():
from app.api.routes.scan import merge_mac_property
existing = [{"key": "MAC", "value": "aa:bb:cc:dd:ee:ff", "icon": None, "visible": True}]
merged = merge_mac_property(existing, "aa:bb:cc:dd:ee:ff")
# No duplicate MAC row; user's visible=True choice kept.
macs = [p for p in merged if p["key"] == "MAC"]
assert len(macs) == 1
assert macs[0]["visible"] is True
def test_merge_mac_property_noop_without_mac():
from app.api.routes.scan import merge_mac_property
existing = [{"key": "Custom", "value": "x", "icon": None, "visible": True}]
assert merge_mac_property(existing, None) == existing
@pytest.mark.asyncio
async def test_approve_device_does_not_duplicate_mac_property(
client: AsyncClient, headers, pending_device, db_session
):
"""If the approve payload already carries a MAC prop, don't add a second one."""
from sqlalchemy import select
from app.db.models import Node as NodeModel
res = await client.post(
f"/api/v1/scan/pending/{pending_device.id}/approve",
json={
"label": "My Server",
"type": "server",
"ip": "192.168.1.100",
"status": "unknown",
"services": [],
"properties": [
{"key": "MAC", "value": "aa:bb:cc:dd:ee:ff", "icon": None, "visible": True}
],
},
headers=headers,
)
assert res.status_code == 200
node = (
await db_session.execute(select(NodeModel).where(NodeModel.device_id.is_not(None)))
).scalars().first()
assert node is not None
# The device facts belong to the inventory row the node draws.
device = await db_session.get(InventoryDevice, node.device_id)
assert device is not None
mac_props = [p for p in device.properties if p["key"] == "MAC"]
assert len(mac_props) == 1
# User's visibility choice is preserved.
assert mac_props[0]["visible"] is True
@pytest.mark.asyncio
async def test_approve_zigbee_creates_edge_when_other_endpoint_is_node(
client: AsyncClient, headers, db_session
):
from sqlalchemy import select
from app.db.models import Edge
coord, pending = await _seed_zigbee_pending_pair(db_session)
res = await client.post(
f"/api/v1/scan/pending/{pending.id}/approve",
json={
"label": "router_1",
"type": "zigbee_router",
"ip": None,
"status": "unknown",
"services": [],
},
headers=headers,
)
assert res.status_code == 200
data = res.json()
assert data["approved"] is True
assert data["edges_created"] == 1
edges = (await db_session.execute(select(Edge))).scalars().all()
assert len(edges) == 1
assert edges[0].source == coord.id
assert edges[0].target == data["node_id"]
assert edges[0].source_handle == "bottom"
# Bare side name (canonical stored form); renders at the top like before.
assert edges[0].target_handle == "top"
assert edges[0].type == "iot"
@pytest.mark.asyncio
async def test_approve_zigbee_skips_duplicate_edge(
client: AsyncClient, headers, db_session
):
"""Re-running the resolution does not create a second edge for the same pair."""
from sqlalchemy import select
from app.db.models import Edge, InventoryDevice, InventoryDeviceLink
coord, pending = await _seed_zigbee_pending_pair(db_session)
body = {"label": "router_1", "type": "zigbee_router", "ip": None, "status": "unknown", "services": []}
await client.post(f"/api/v1/scan/pending/{pending.id}/approve", json=body, headers=headers)
# Simulate a second pending row + link between same coord and a new device,
# but keep an existing edge in place to verify dedupe also handles
# the swapped-direction case.
new_pending = InventoryDevice(
ieee_address="0xR1B",
friendly_name="r1b",
suggested_type="zigbee_router",
status="pending",
discovery_source="zigbee",
)
db_session.add(new_pending)
db_session.add(
InventoryDeviceLink(source_ieee="0xCOORD", target_ieee="0xR1B", discovery_source="zigbee")
)
await db_session.commit()
res = await client.post(
f"/api/v1/scan/pending/{new_pending.id}/approve", json=body, headers=headers
)
assert res.json()["edges_created"] == 1 # only the new pair
edges = (await db_session.execute(select(Edge))).scalars().all()
assert len(edges) == 2 # original + new, no duplicate
@pytest.mark.asyncio
async def test_approve_zigbee_skips_when_other_endpoint_still_pending(
client: AsyncClient, headers, db_session
):
"""Both endpoints pending → no edge yet, link row preserved for later."""
from sqlalchemy import select
from app.db.models import Edge, InventoryDevice, InventoryDeviceLink
a = InventoryDevice(
ieee_address="0xA",
friendly_name="a",
suggested_type="zigbee_router",
status="pending",
discovery_source="zigbee",
)
b = InventoryDevice(
ieee_address="0xB",
friendly_name="b",
suggested_type="zigbee_enddevice",
status="pending",
discovery_source="zigbee",
)
db_session.add_all([a, b])
db_session.add(
InventoryDeviceLink(source_ieee="0xA", target_ieee="0xB", discovery_source="zigbee")
)
await db_session.commit()
res = await client.post(
f"/api/v1/scan/pending/{a.id}/approve",
json={
"label": "a",
"type": "zigbee_router",
"ip": None,
"status": "unknown",
"services": [],
},
headers=headers,
)
assert res.status_code == 200
assert res.json()["edges_created"] == 0
edges = (await db_session.execute(select(Edge))).scalars().all()
assert edges == []
links = (await db_session.execute(select(InventoryDeviceLink))).scalars().all()
assert len(links) == 1 # preserved for later resolution
@pytest.mark.asyncio
async def test_approve_zigbee_resolves_link_after_second_approval(
client: AsyncClient, headers, db_session
):
"""First approval keeps link (other endpoint pending); second approval
creates the edge. The link row is retained afterwards so the same pair can
be re-approved onto another canvas — it's topology, wiped only on reimport."""
from sqlalchemy import select
from app.db.models import Edge, InventoryDevice, InventoryDeviceLink
a = InventoryDevice(
ieee_address="0xA",
friendly_name="a",
suggested_type="zigbee_router",
status="pending",
discovery_source="zigbee",
)
b = InventoryDevice(
ieee_address="0xB",
friendly_name="b",
suggested_type="zigbee_enddevice",
status="pending",
discovery_source="zigbee",
)
db_session.add_all([a, b])
db_session.add(
InventoryDeviceLink(source_ieee="0xA", target_ieee="0xB", discovery_source="zigbee")
)
await db_session.commit()
body = {"label": "x", "type": "zigbee_router", "ip": None, "status": "unknown", "services": []}
await client.post(f"/api/v1/scan/pending/{a.id}/approve", json=body, headers=headers)
res = await client.post(f"/api/v1/scan/pending/{b.id}/approve", json=body, headers=headers)
assert res.json()["edges_created"] == 1
edges = (await db_session.execute(select(Edge))).scalars().all()
assert len(edges) == 1
links = (await db_session.execute(select(InventoryDeviceLink))).scalars().all()
assert len(links) == 1 # retained for re-approval onto other canvases
@pytest.mark.asyncio
async def test_single_approve_zwave_sets_wireless_fields(client, headers, db_session):
active = await _add_design(db_session, "zwave")
dev = InventoryDevice(
id=str(uuid.uuid4()),
ieee_address="zwave-H-9",
friendly_name="Door Sensor",
suggested_type="zwave_enddevice",
vendor="Aeotec",
model="ZW120",
status="pending",
discovery_source="zwave",
)
db_session.add(dev)
await db_session.commit()
res = await client.post(
f"/api/v1/scan/pending/{dev.id}/approve",
json={"label": "Door Sensor", "type": "zwave_enddevice", "design_id": active},
headers=headers,
)
assert res.status_code == 200
node = (
await db_session.execute(select(Node).join(InventoryDevice, Node.device_id == InventoryDevice.id).where(InventoryDevice.ieee_address == "zwave-H-9"))
).scalar_one()
assert node.design_id == active
node = (await db_session.execute(select(Node))).scalars().first()
assert node is not None
device = await db_session.get(InventoryDevice, node.device_id)
assert device is not None
assert device.status_live == "online"
assert device.check_method == "none"
assert any(p["key"] == "Z-Wave ID" for p in device.properties)
+567
View File
@@ -0,0 +1,567 @@
"""Scan API routes: trigger, pending list, canvas-count correlation, timestamps, config."""
import uuid
from datetime import datetime, timezone
from unittest.mock import AsyncMock, patch
import pytest
from httpx import AsyncClient
from app.db.models import InventoryDevice
from tests.scan.helpers import _add_design, _node_for
@pytest.mark.asyncio
async def test_trigger_scan_requires_auth(client: AsyncClient):
res = await client.post("/api/v1/scan/trigger")
assert res.status_code == 401
@pytest.mark.asyncio
async def test_trigger_scan_creates_run(client: AsyncClient, headers):
with (
patch("app.api.routes.scan._background_scan", new_callable=AsyncMock),
patch("app.api.routes.scan.settings") as mock_settings,
):
mock_settings.scanner_ranges = ["192.168.1.0/24"]
res = await client.post("/api/v1/scan/trigger", headers=headers)
assert res.status_code == 200
data = res.json()
assert data["status"] == "running"
assert data["ranges"] == ["192.168.1.0/24"]
assert "id" in data
@pytest.mark.asyncio
async def test_list_pending_empty(client: AsyncClient, headers):
res = await client.get("/api/v1/scan/pending", headers=headers)
assert res.status_code == 200
assert res.json() == []
@pytest.mark.asyncio
async def test_list_pending_returns_device(client: AsyncClient, headers, pending_device):
res = await client.get("/api/v1/scan/pending", headers=headers)
assert res.status_code == 200
data = res.json()
assert len(data) == 1
assert data[0]["ip"] == "192.168.1.100"
assert data[0]["hostname"] == "my-server"
# No matching node → not on any canvas.
assert data[0]["canvas_count"] == 0
@pytest.mark.asyncio
async def test_canvas_count_matches_ip_in_comma_list(client, headers, db_session, pending_device):
# Node.ip holds several comma-separated addresses (IPv6 added first). The
# device scanned as the plain IPv4 must still correlate (issue #258).
d1 = await _add_design(db_session, "Home")
await _node_for(db_session, d1, ip="fe80::1, 192.168.1.100")
await db_session.commit()
data = (await client.get("/api/v1/scan/pending", headers=headers)).json()
assert data[0]["canvas_count"] == 1
@pytest.mark.asyncio
async def test_canvas_count_correlates_by_mac(client, headers, db_session, pending_device):
# Node's ip differs entirely (user edited it) but the MAC still matches:
# the device is on the canvas (issue #258, MAC is the stable identifier).
d1 = await _add_design(db_session, "Home")
await _node_for(db_session, d1, ip="10.9.9.9", mac="aa:bb:cc:dd:ee:ff")
await db_session.commit()
data = (await client.get("/api/v1/scan/pending", headers=headers)).json()
assert data[0]["canvas_count"] == 1
@pytest.mark.asyncio
async def test_canvas_count_counts_distinct_designs_by_ip(client, headers, db_session, pending_device):
# Same IP placed on two different canvases → canvas_count == 2.
d1 = await _add_design(db_session, "Home")
d2 = await _add_design(db_session, "Lab")
await _node_for(db_session, d1, ip="192.168.1.100")
await _node_for(db_session, d2, ip="192.168.1.100")
await db_session.commit()
res = await client.get("/api/v1/scan/pending", headers=headers)
data = res.json()
assert len(data) == 1
assert data[0]["canvas_count"] == 2
@pytest.mark.asyncio
async def test_canvas_count_correlates_by_ieee(client, headers, db_session):
device = InventoryDevice(
id=str(uuid.uuid4()), ieee_address="0x00124b001", discovery_source="zigbee",
suggested_type="zigbee_enddevice", services=[], status="pending",
)
db_session.add(device)
d1 = await _add_design(db_session, "Zigbee")
await _node_for(db_session, d1, ieee="0x00124b001")
await db_session.commit()
res = await client.get("/api/v1/scan/pending", headers=headers)
by_id = {d["id"]: d for d in res.json()}
assert by_id[device.id]["canvas_count"] == 1
@pytest.mark.asyncio
async def test_pending_device_without_node_has_null_node_timestamps(client, headers, pending_device):
# No matching canvas node → node_* timestamps are all null; the device still
# carries its own discovered_at for the "Discovered" fallback on the tile.
data = (await client.get("/api/v1/scan/pending", headers=headers)).json()[0]
assert data["discovered_at"] is not None
assert data["node_created_at"] is None
assert data["node_last_scan"] is None
assert data["node_last_modified"] is None
assert data["node_last_seen"] is None
@pytest.mark.asyncio
async def test_pending_device_exposes_linked_node_timestamps(client, headers, db_session, pending_device):
d1 = await _add_design(db_session, "Home")
node = await _node_for(db_session, d1, ip="192.168.1.100")
# When the scanner last saw the device, and when it last answered, are facts
# about the device — they live on its row, not on any one drawing of it.
device = await db_session.get(InventoryDevice, node.device_id)
assert device is not None
device.last_scan = datetime(2026, 6, 1, 8, 30, tzinfo=timezone.utc)
device.last_seen = datetime(2026, 6, 25, 9, 15, tzinfo=timezone.utc)
await db_session.commit()
data = (await client.get("/api/v1/scan/pending", headers=headers)).json()[0]
assert data["node_created_at"] is not None # defaulted on insert
assert data["node_last_modified"] is not None # updated_at defaulted on insert
assert data["node_last_scan"].startswith("2026-06-01")
assert data["node_last_seen"].startswith("2026-06-25")
@pytest.mark.asyncio
async def test_node_timestamps_aggregate_across_matches(client, headers, db_session, pending_device):
# Two canvas nodes share the device IP: created_at takes the OLDEST,
# last_scan takes the NEWEST.
d1 = await _add_design(db_session, "Home")
d2 = await _add_design(db_session, "Lab")
older = await _node_for(db_session, d1, ip="192.168.1.100")
older.created_at = datetime(2026, 1, 1, 0, 0, tzinfo=timezone.utc)
newer = await _node_for(db_session, d2, ip="192.168.1.100")
newer.created_at = datetime(2026, 5, 1, 0, 0, tzinfo=timezone.utc)
# Both nodes draw the one row, which carries the scan observation.
assert older.device_id == newer.device_id
device = await db_session.get(InventoryDevice, older.device_id)
assert device is not None
device.last_scan = datetime(2026, 6, 1, 0, 0, tzinfo=timezone.utc)
db_session.add_all([older, newer])
await db_session.commit()
data = (await client.get("/api/v1/scan/pending", headers=headers)).json()[0]
assert data["node_created_at"].startswith("2026-01-01") # oldest node
assert data["node_last_scan"].startswith("2026-06-01") # off the device row
@pytest.mark.asyncio
async def test_resolve_deep_scan_falls_back_to_settings():
from app.api.routes.scan import TriggerScanRequest, _resolve_deep_scan
with patch("app.api.routes.scan.settings") as mock_settings:
mock_settings.scanner_http_ranges = ["7000-7100"]
mock_settings.scanner_http_probe_enabled = True
mock_settings.scanner_http_verify_tls = False
# Empty payload → all values come from settings defaults
ds = _resolve_deep_scan(TriggerScanRequest())
assert ds.http_ranges == ["7000-7100"]
assert ds.http_probe_enabled is True
assert ds.verify_tls is False
@pytest.mark.asyncio
async def test_resolve_deep_scan_override_wins():
from app.api.routes.scan import TriggerScanRequest, _resolve_deep_scan
with patch("app.api.routes.scan.settings") as mock_settings:
mock_settings.scanner_http_ranges = []
mock_settings.scanner_http_probe_enabled = False
mock_settings.scanner_http_verify_tls = False
ds = _resolve_deep_scan(
TriggerScanRequest(http_ranges=["9000"], http_probe_enabled=True, verify_tls=True)
)
assert ds.http_ranges == ["9000"]
assert ds.http_probe_enabled is True
assert ds.verify_tls is True
@pytest.mark.asyncio
async def test_trigger_scan_passes_deep_scan_options(client: AsyncClient, headers):
captured = {}
async def fake_bg(run_id, ranges, deep_scan):
captured["deep_scan"] = deep_scan
with (
patch("app.api.routes.scan._background_scan", new=fake_bg),
patch("app.api.routes.scan.settings") as mock_settings,
):
mock_settings.scanner_ranges = ["192.168.1.0/24"]
mock_settings.scanner_http_ranges = []
mock_settings.scanner_http_probe_enabled = False
mock_settings.scanner_http_verify_tls = False
res = await client.post(
"/api/v1/scan/trigger",
json={"http_probe_enabled": True, "http_ranges": ["8000-8100"]},
headers=headers,
)
assert res.status_code == 200
assert captured["deep_scan"].http_probe_enabled is True
assert captured["deep_scan"].http_ranges == ["8000-8100"]
@pytest.mark.asyncio
async def test_trigger_scan_rejects_invalid_port_range(client: AsyncClient, headers):
with patch("app.api.routes.scan.settings") as mock_settings:
mock_settings.scanner_ranges = ["192.168.1.0/24"]
res = await client.post(
"/api/v1/scan/trigger",
json={"http_ranges": ["70000-80000"]},
headers=headers,
)
assert res.status_code == 422
@pytest.mark.asyncio
async def test_get_scan_config_includes_deep_scan(client: AsyncClient, headers):
with patch("app.api.routes.scan.settings") as mock_settings:
mock_settings.scanner_ranges = ["192.168.1.0/24"]
mock_settings.scanner_http_ranges = ["8000-8100"]
mock_settings.scanner_http_probe_enabled = True
mock_settings.scanner_http_verify_tls = False
res = await client.get("/api/v1/scan/config", headers=headers)
assert res.status_code == 200
data = res.json()
assert data["http_ranges"] == ["8000-8100"]
assert data["http_probe_enabled"] is True
@pytest.mark.asyncio
async def test_update_scan_config_persists_deep_scan(client: AsyncClient, headers):
saved = {}
with patch("app.api.routes.scan.settings") as mock_settings:
mock_settings.scanner_ranges = ["192.168.1.0/24"]
mock_settings.scanner_http_ranges = []
mock_settings.scanner_http_probe_enabled = False
mock_settings.scanner_http_verify_tls = False
mock_settings.save_overrides = lambda: saved.update(
http_ranges=mock_settings.scanner_http_ranges,
probe=mock_settings.scanner_http_probe_enabled,
)
res = await client.post(
"/api/v1/scan/config",
json={
"ranges": ["192.168.1.0/24"],
"http_ranges": ["9000-9100"],
"http_probe_enabled": True,
"verify_tls": True,
},
headers=headers,
)
assert res.status_code == 200
assert saved == {"http_ranges": ["9000-9100"], "probe": True}
@pytest.mark.asyncio
async def test_create_pending_normalizes_the_mac(client: AsyncClient, headers):
# Dedup compares MACs by equality, so a hand-typed entry in any other
# notation would never match the scanned row and approve would build a
# duplicate node.
res = await client.post(
"/api/v1/scan/pending",
json={
"hostname": "printer",
"mac": "AA-BB-CC-11-22-33",
"discovery_source": "manual",
},
headers=headers,
)
assert res.status_code == 201, res.text
assert res.json()["mac"] == "aa:bb:cc:11:22:33"
@pytest.mark.asyncio
async def test_create_pending_keeps_a_missing_mac_null(client: AsyncClient, headers):
res = await client.post(
"/api/v1/scan/pending",
json={"hostname": "no-mac", "discovery_source": "manual"},
headers=headers,
)
assert res.status_code == 201, res.text
assert res.json()["mac"] is None
@pytest.mark.asyncio
async def test_create_pending_unhides_the_row_it_merges_into(client: AsyncClient, headers, db_session):
"""Adding a device by hand outranks an earlier hide of the same host.
One device is one row, hidden ones included — but merging into a hidden row
and leaving it hidden answers 201 while nothing appears in the inventory, so
the add reads as a no-op.
"""
db_session.add(InventoryDevice(id="d-1", ip="192.168.1.50", status="hidden"))
await db_session.commit()
res = await client.post(
"/api/v1/scan/pending",
json={"ip": "192.168.1.50", "hostname": "printer", "discovery_source": "manual"},
headers=headers,
)
assert res.status_code == 201, res.text
assert res.json()["id"] == "d-1"
assert res.json()["status"] == "pending"
listed = (await client.get("/api/v1/scan/pending", headers=headers)).json()
assert [d["id"] for d in listed] == ["d-1"]
@pytest.mark.asyncio
async def test_create_pending_leaves_an_approved_row_approved(client: AsyncClient, headers, db_session):
"""Only `hidden` is lifted — a merge must not walk a device back down the
lifecycle and re-queue something already on a canvas."""
db_session.add(InventoryDevice(id="d-1", ip="192.168.1.51", status="approved"))
await db_session.commit()
res = await client.post(
"/api/v1/scan/pending",
json={"ip": "192.168.1.51", "hostname": "nas", "discovery_source": "manual"},
headers=headers,
)
assert res.status_code == 201, res.text
assert res.json()["status"] == "approved"
@pytest.mark.asyncio
async def test_delete_pending_removes_one_entry(client: AsyncClient, headers, pending_device):
# The rack canvas drops the placeholder it created for a plate once that
# plate is pointed at a real inventory row.
res = await client.delete(f"/api/v1/scan/pending/{pending_device.id}", headers=headers)
assert res.status_code == 200, res.text
assert res.json() == {"deleted": True}
listed = (await client.get("/api/v1/scan/pending", headers=headers)).json()
assert listed == []
@pytest.mark.asyncio
async def test_delete_pending_unknown_is_404(client: AsyncClient, headers):
res = await client.delete(f"/api/v1/scan/pending/{uuid.uuid4()}", headers=headers)
assert res.status_code == 404
@pytest.mark.asyncio
async def test_delete_pending_requires_auth(client: AsyncClient, pending_device):
res = await client.delete(f"/api/v1/scan/pending/{pending_device.id}")
assert res.status_code == 401
@pytest.mark.asyncio
async def test_delete_pending_refuses_a_mounted_device(client: AsyncClient, headers, pending_device):
# `rack_devices.device_id` is ON DELETE SET NULL: without this guard the
# delete would strip the mount of its device instead of erroring.
design = (
await client.post(
"/api/v1/designs",
json={"name": "Rack Room", "icon": "server", "design_type": "rack"},
headers=headers,
)
).json()["id"]
saved = await client.post(
"/api/v1/racks/save",
json={
"design_id": design,
"racks": [
{
"id": "rack-1",
"name": "Main",
"u_height": 12,
"width_standard": "19",
"numbering": "bottom-up",
"style": {},
"pos_x": 0,
"pos_y": 0,
}
],
"devices": [
{
"id": "dev-1",
"rack_id": "rack-1",
"device_id": pending_device.id,
"label": "my-server",
"u_start": 1,
"u_height": 1,
"col_start": 0,
"col_span": 12,
"faceplate_id": "server-1u",
"status": "unknown",
"ports": [],
}
],
"cables": [],
"viewport": {"x": 0, "y": 0, "zoom": 1},
},
headers=headers,
)
assert saved.status_code == 200, saved.text
res = await client.delete(f"/api/v1/scan/pending/{pending_device.id}", headers=headers)
assert res.status_code == 409
listed = (await client.get("/api/v1/scan/pending", headers=headers)).json()
assert [d["id"] for d in listed] == [pending_device.id]
# ---------------------------------------------------------------------------
# Per-device deep rescan (issue #350)
# ---------------------------------------------------------------------------
@pytest.mark.asyncio
async def test_rescan_device_requires_auth(client: AsyncClient, pending_device):
res = await client.post(f"/api/v1/scan/pending/{pending_device.id}/rescan")
assert res.status_code == 401
@pytest.mark.asyncio
async def test_rescan_device_creates_run_for_that_ip(client: AsyncClient, headers, pending_device):
with patch("app.api.routes.scan._background_device_scan", new_callable=AsyncMock) as bg:
res = await client.post(
f"/api/v1/scan/pending/{pending_device.id}/rescan", headers=headers
)
assert res.status_code == 200, res.text
data = res.json()
assert data["status"] == "running"
assert data["kind"] == "device"
# One host, not the configured ranges — the scan targets this device only.
assert data["ranges"] == ["192.168.1.100/32"]
bg.assert_called_once()
assert bg.call_args.args[1] == pending_device.id
# Full range unless the caller says otherwise.
assert bg.call_args.args[3] is True
@pytest.mark.asyncio
async def test_rescan_device_passes_the_requested_port_range(
client: AsyncClient, headers, pending_device
):
"""The dialog's range reaches the scanner verbatim."""
with patch("app.api.routes.scan._background_device_scan", new_callable=AsyncMock) as bg:
res = await client.post(
f"/api/v1/scan/pending/{pending_device.id}/rescan",
headers=headers,
json={"ports": " 80,443,8000-9000 "},
)
assert res.status_code == 200, res.text
assert bg.call_args.args[4] == "80,443,8000-9000"
@pytest.mark.asyncio
async def test_rescan_device_rejects_an_unusable_port_range(
client: AsyncClient, headers, pending_device
):
"""A bad spec is refused here, not handed to nmap."""
for bad in ["0", "65536", "100-50", "80,", "http"]:
res = await client.post(
f"/api/v1/scan/pending/{pending_device.id}/rescan",
headers=headers,
json={"ports": bad},
)
assert res.status_code == 422, f"{bad}: {res.text}"
@pytest.mark.asyncio
async def test_rescan_device_treats_a_blank_range_as_the_full_sweep(
client: AsyncClient, headers, pending_device
):
with patch("app.api.routes.scan._background_device_scan", new_callable=AsyncMock) as bg:
res = await client.post(
f"/api/v1/scan/pending/{pending_device.id}/rescan",
headers=headers,
json={"ports": " "},
)
assert res.status_code == 200, res.text
assert bg.call_args.args[4] is None
assert bg.call_args.args[3] is True
@pytest.mark.asyncio
async def test_rescan_device_unknown_id_404(client: AsyncClient, headers):
res = await client.post(f"/api/v1/scan/pending/{uuid.uuid4()}/rescan", headers=headers)
assert res.status_code == 404
@pytest.mark.asyncio
async def test_rescan_device_without_ip_409(client: AsyncClient, headers, db_session):
device = InventoryDevice(id=str(uuid.uuid4()), ip=None, hostname="zigbee-lamp", status="pending")
db_session.add(device)
await db_session.commit()
res = await client.post(f"/api/v1/scan/pending/{device.id}/rescan", headers=headers)
assert res.status_code == 409
assert "IP" in res.json()["detail"]
@pytest.mark.asyncio
async def test_rescan_device_hidden_409(client: AsyncClient, headers, db_session):
device = InventoryDevice(id=str(uuid.uuid4()), ip="192.168.1.77", status="hidden")
db_session.add(device)
await db_session.commit()
res = await client.post(f"/api/v1/scan/pending/{device.id}/rescan", headers=headers)
assert res.status_code == 409
@pytest.mark.asyncio
async def test_rescan_device_serialized_per_device(client: AsyncClient, headers, pending_device):
"""A second rescan while the first still runs is refused, not duplicated."""
with patch("app.api.routes.scan._background_device_scan", new_callable=AsyncMock):
first = await client.post(
f"/api/v1/scan/pending/{pending_device.id}/rescan", headers=headers
)
second = await client.post(
f"/api/v1/scan/pending/{pending_device.id}/rescan", headers=headers
)
assert first.status_code == 200
assert second.status_code == 409
assert "already running" in second.json()["detail"]
@pytest.mark.asyncio
async def test_rescan_device_allows_a_new_run_once_the_first_finished(
client: AsyncClient, headers, pending_device
):
with patch("app.api.routes.scan._background_device_scan", new_callable=AsyncMock):
first = await client.post(
f"/api/v1/scan/pending/{pending_device.id}/rescan", headers=headers
)
assert first.status_code == 200
run_id = first.json()["id"]
stopped = await client.post(f"/api/v1/scan/{run_id}/stop", headers=headers)
assert stopped.status_code == 200
again = await client.post(
f"/api/v1/scan/pending/{pending_device.id}/rescan", headers=headers
)
assert again.status_code == 200
@pytest.mark.asyncio
async def test_get_run_returns_status(client: AsyncClient, headers, pending_device):
with patch("app.api.routes.scan._background_device_scan", new_callable=AsyncMock):
started = await client.post(
f"/api/v1/scan/pending/{pending_device.id}/rescan", headers=headers
)
run_id = started.json()["id"]
res = await client.get(f"/api/v1/scan/runs/{run_id}", headers=headers)
assert res.status_code == 200
assert res.json()["id"] == run_id
assert res.json()["status"] == "running"
@pytest.mark.asyncio
async def test_get_run_unknown_id_404(client: AsyncClient, headers):
res = await client.get(f"/api/v1/scan/runs/{uuid.uuid4()}", headers=headers)
assert res.status_code == 404
+470
View File
@@ -0,0 +1,470 @@
"""run_scan service persistence, _background_scan lifecycle, stop/cancel."""
import uuid
from unittest.mock import AsyncMock, patch
import pytest
from httpx import AsyncClient
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.db.models import InventoryDevice, Node, ScanRun
from app.services.scanner import _cancelled_runs, request_cancel, run_scan
@pytest.mark.asyncio
async def test_background_scan_marks_run_errored_on_exception(mem_db):
"""If run_scan() raises, the ScanRun must transition running → error and the
session rollback path must execute without a follow-on exception.
"error" and not "failed": one condition, one name — it is what run_scan
writes when it catches the failure itself, and the only one Scan History
filters and colours.
"""
from app.api.routes.scan import _background_scan
async with mem_db() as session:
run = ScanRun(status="running", ranges=["10.0.0.0/24"])
session.add(run)
await session.commit()
run_id = run.id
with (
patch("app.api.routes.scan.AsyncSessionLocal", mem_db),
patch(
"app.api.routes.scan.run_scan",
new_callable=AsyncMock,
side_effect=RuntimeError("boom"),
),
):
await _background_scan(run_id, ["10.0.0.0/24"])
async with mem_db() as session:
refreshed = await session.get(ScanRun, run_id)
assert refreshed is not None
assert refreshed.status == "error"
@pytest.mark.asyncio
async def test_background_scan_leaves_non_running_status_alone(mem_db):
"""If the run was already stopped/cancelled before run_scan failed, _background_scan
must NOT overwrite that terminal status with 'error'."""
from app.api.routes.scan import _background_scan
async with mem_db() as session:
run = ScanRun(status="cancelled", ranges=["10.0.0.0/24"])
session.add(run)
await session.commit()
run_id = run.id
with (
patch("app.api.routes.scan.AsyncSessionLocal", mem_db),
patch(
"app.api.routes.scan.run_scan",
new_callable=AsyncMock,
side_effect=RuntimeError("boom"),
),
):
await _background_scan(run_id, ["10.0.0.0/24"])
async with mem_db() as session:
refreshed = await session.get(ScanRun, run_id)
assert refreshed is not None
assert refreshed.status == "cancelled"
@pytest.mark.asyncio
async def test_background_scan_success_path_invokes_run_scan(mem_db):
from app.api.routes.scan import _background_scan
async with mem_db() as session:
run = ScanRun(status="running", ranges=["10.0.0.0/24"])
session.add(run)
await session.commit()
run_id = run.id
with (
patch("app.api.routes.scan.AsyncSessionLocal", mem_db),
patch("app.api.routes.scan.run_scan", new_callable=AsyncMock) as mock_run_scan,
):
from app.services.scanner import DeepScanOptions
await _background_scan(run_id, ["10.0.0.0/24"], DeepScanOptions())
mock_run_scan.assert_awaited_once()
@pytest.mark.asyncio
async def test_list_runs_empty(client: AsyncClient, headers):
res = await client.get("/api/v1/scan/runs", headers=headers)
assert res.status_code == 200
assert res.json() == []
# --- run_scan: re-scan updates existing pending devices ---
MOCK_HOST = {
"ip": "192.168.1.50",
"mac": "aa:bb:cc:dd:ee:ff",
"hostname": "myhost.lan",
"os": "Linux",
"open_ports": [{"port": 8096, "protocol": "tcp", "banner": "Jellyfin"}],
}
@pytest.mark.asyncio
async def test_run_scan_creates_new_pending_device(db_session: AsyncSession):
run_id = str(uuid.uuid4())
run = ScanRun(id=run_id, status="running", ranges=["192.168.1.0/24"])
db_session.add(run)
await db_session.commit()
with (
patch("app.services.scanner._nmap_scan", return_value=[MOCK_HOST]),
patch("app.api.routes.status.broadcast_scan_update", new_callable=AsyncMock),
):
await run_scan(["192.168.1.0/24"], db_session, run_id)
result = await db_session.execute(
select(InventoryDevice).where(InventoryDevice.ip == "192.168.1.50")
)
device = result.scalar_one_or_none()
assert device is not None
assert device.hostname == "myhost.lan"
assert any(s["port"] == 8096 for s in device.services)
assert device.suggested_type == "server"
@pytest.mark.asyncio
async def test_run_scan_keeps_stale_pending_for_canvas_nodes(db_session: AsyncSession):
"""Pending devices whose IP is already on a canvas are NOT purged — they stay
in the inventory and are surfaced with an "In N canvas" badge."""
stale = InventoryDevice(
id=str(uuid.uuid4()),
ip="192.168.1.50",
mac=None,
hostname=None,
os=None,
services=[],
suggested_type="generic",
status="pending",
)
db_session.add(stale)
await db_session.flush()
# A canvas already draws that device.
db_session.add(Node(
id=str(uuid.uuid4()),
label="Existing Server",
type="server",
device_id=stale.id,
pos_x=0.0,
pos_y=0.0,
))
await db_session.commit()
run_id = str(uuid.uuid4())
run = ScanRun(id=run_id, status="running", ranges=["192.168.1.0/24"])
db_session.add(run)
await db_session.commit()
with (
patch("app.services.scanner._nmap_scan", return_value=[]),
patch("app.api.routes.status.broadcast_scan_update", new_callable=AsyncMock),
):
await run_scan(["192.168.1.0/24"], db_session, run_id)
result = await db_session.execute(
select(InventoryDevice).where(InventoryDevice.ip == "192.168.1.50")
)
assert result.scalar_one_or_none() is not None
@pytest.mark.asyncio
async def test_run_scan_records_ip_already_in_canvas(db_session: AsyncSession):
"""A scanned IP already drawn on a canvas refreshes the one inventory row.
The device is never suppressed, and never duplicated: the node draws that
row, so a second row for the same host would split the device in two.
"""
existing = InventoryDevice(id=str(uuid.uuid4()), ip="192.168.1.50", status="approved")
db_session.add(existing)
await db_session.flush()
existing_id = existing.id
db_session.add(Node(
id=str(uuid.uuid4()),
label="Existing Server",
type="server",
device_id=existing.id,
pos_x=0.0,
pos_y=0.0,
))
await db_session.commit()
run_id = str(uuid.uuid4())
run = ScanRun(id=run_id, status="running", ranges=["192.168.1.0/24"])
db_session.add(run)
await db_session.commit()
with (
patch("app.services.scanner._nmap_scan", return_value=[MOCK_HOST]),
patch("app.api.routes.status.broadcast_scan_update", new_callable=AsyncMock),
):
await run_scan(["192.168.1.0/24"], db_session, run_id)
rows = (
await db_session.execute(
select(InventoryDevice).where(InventoryDevice.ip == "192.168.1.50")
)
).scalars().all()
assert len(rows) == 1
assert rows[0].id == existing_id
# The scan refreshed it; it did not reset the lifecycle.
assert rows[0].status == "approved"
assert rows[0].hostname == MOCK_HOST["hostname"]
@pytest.mark.asyncio
async def test_run_scan_refreshes_approved_device_without_duplicating(db_session: AsyncSession):
"""Re-scanning an already-approved device updates its row in place instead of
spawning a fresh pending duplicate, and keeps it approved."""
approved = InventoryDevice(
id=str(uuid.uuid4()), ip="192.168.1.50", mac=None, hostname="old",
os=None, services=[], suggested_type="server", status="approved",
)
db_session.add(approved)
run_id = str(uuid.uuid4())
db_session.add(ScanRun(id=run_id, status="running", ranges=["192.168.1.0/24"]))
await db_session.commit()
with (
patch("app.services.scanner._nmap_scan", return_value=[MOCK_HOST]),
patch("app.api.routes.status.broadcast_scan_update", new_callable=AsyncMock),
):
await run_scan(["192.168.1.0/24"], db_session, run_id)
rows = (await db_session.execute(
select(InventoryDevice).where(InventoryDevice.ip == "192.168.1.50")
)).scalars().all()
assert len(rows) == 1
assert rows[0].status == "approved"
assert rows[0].hostname == "myhost.lan" # refreshed from the scan
@pytest.mark.asyncio
async def test_run_scan_collapses_existing_duplicate_rows(db_session: AsyncSession):
"""Pre-existing duplicate inventory rows for one IP are collapsed to a single
row at scan start, even if the device is not re-discovered."""
for status in ("approved", "pending", "pending"):
db_session.add(InventoryDevice(
id=str(uuid.uuid4()), ip="192.168.1.77", mac=None, hostname=None,
os=None, services=[], suggested_type="server", status=status,
))
run_id = str(uuid.uuid4())
db_session.add(ScanRun(id=run_id, status="running", ranges=["192.168.1.0/24"]))
await db_session.commit()
with (
patch("app.services.scanner._nmap_scan", return_value=[]),
patch("app.api.routes.status.broadcast_scan_update", new_callable=AsyncMock),
):
await run_scan(["192.168.1.0/24"], db_session, run_id)
rows = (await db_session.execute(
select(InventoryDevice).where(InventoryDevice.ip == "192.168.1.77")
)).scalars().all()
assert len(rows) == 1
assert rows[0].status == "approved" # approved row is the one kept
@pytest.mark.asyncio
async def test_run_scan_skips_hidden_device(db_session: AsyncSession):
"""Devices previously hidden by the user must not re-appear in pending on re-scan."""
hidden = InventoryDevice(
id=str(uuid.uuid4()),
ip="192.168.1.50",
mac=None,
hostname=None,
os=None,
services=[],
suggested_type="generic",
status="hidden",
)
db_session.add(hidden)
await db_session.commit()
run_id = str(uuid.uuid4())
run = ScanRun(id=run_id, status="running", ranges=["192.168.1.0/24"])
db_session.add(run)
await db_session.commit()
with (
patch("app.services.scanner._nmap_scan", return_value=[MOCK_HOST]),
patch("app.api.routes.status.broadcast_scan_update", new_callable=AsyncMock),
):
await run_scan(["192.168.1.0/24"], db_session, run_id)
result = await db_session.execute(
select(InventoryDevice).where(
InventoryDevice.ip == "192.168.1.50",
InventoryDevice.status == "pending",
)
)
assert result.scalar_one_or_none() is None
@pytest.mark.asyncio
async def test_stop_scan_requires_auth(client: AsyncClient):
res = await client.post("/api/v1/scan/fake-id/stop")
assert res.status_code == 401
@pytest.mark.asyncio
async def test_stop_scan_not_found(client: AsyncClient, headers):
import uuid as _uuid
res = await client.post(f"/api/v1/scan/{_uuid.uuid4()}/stop", headers=headers)
assert res.status_code == 404
@pytest.mark.asyncio
async def test_stop_scan_not_running(client: AsyncClient, headers, db_session: AsyncSession):
run = ScanRun(id=str(uuid.uuid4()), status="done", ranges=["192.168.1.0/24"])
db_session.add(run)
await db_session.commit()
res = await client.post(f"/api/v1/scan/{run.id}/stop", headers=headers)
assert res.status_code == 409
@pytest.mark.asyncio
async def test_stop_scan_success(client: AsyncClient, headers, db_session: AsyncSession):
run = ScanRun(id=str(uuid.uuid4()), status="running", ranges=["192.168.1.0/24"])
db_session.add(run)
await db_session.commit()
res = await client.post(f"/api/v1/scan/{run.id}/stop", headers=headers)
assert res.status_code == 200
assert res.json() == {"stopping": True}
# run_id added to cancel set
assert run.id in _cancelled_runs
# status flipped eagerly so the UI reacts without waiting for a checkpoint
await db_session.refresh(run)
assert run.status == "cancelled"
assert run.finished_at is not None
# cleanup for other tests
_cancelled_runs.discard(run.id)
@pytest.mark.asyncio
async def test_run_scan_cancelled_marks_status(db_session: AsyncSession):
"""When cancel is requested before the scan starts, status becomes 'cancelled'."""
run_id = str(uuid.uuid4())
run = ScanRun(id=run_id, status="running", ranges=["192.168.1.0/24"])
db_session.add(run)
await db_session.commit()
request_cancel(run_id)
with (
patch("app.services.scanner._nmap_scan", return_value=[MOCK_HOST]) as mock_nmap,
patch("app.api.routes.status.broadcast_scan_update", new_callable=AsyncMock),
):
await run_scan(["192.168.1.0/24"], db_session, run_id)
# nmap should not have been called — cancelled before first range
mock_nmap.assert_not_called()
await db_session.refresh(run)
assert run.status == "cancelled"
assert run.finished_at is not None
@pytest.mark.asyncio
async def test_run_scan_cancelled_mid_scan_skips_remaining_cidrs(db_session: AsyncSession):
"""Cancel flag set after first CIDR is started prevents processing of the second CIDR."""
run_id = str(uuid.uuid4())
run = ScanRun(id=run_id, status="running", ranges=["10.0.0.0/24", "10.0.1.0/24"])
db_session.add(run)
await db_session.commit()
call_count = 0
def nmap_side_effect(target: str, port_spec: str | None = None, run_id: str | None = None):
nonlocal call_count
call_count += 1
# Signal cancellation after the first CIDR scan completes
if call_count == 1:
request_cancel(run_id)
return []
with (
patch("app.services.scanner._nmap_scan", side_effect=nmap_side_effect),
patch("app.api.routes.status.broadcast_scan_update", new_callable=AsyncMock),
):
await run_scan(["10.0.0.0/24", "10.0.1.0/24"], db_session, run_id)
assert call_count == 1 # second CIDR was skipped
await db_session.refresh(run)
assert run.status == "cancelled"
@pytest.mark.asyncio
async def test_run_scan_updates_existing_pending_device(db_session: AsyncSession):
"""Re-scanning the same IP updates services instead of creating a duplicate."""
# Pre-existing pending device with no services
existing = InventoryDevice(
id=str(uuid.uuid4()),
ip="192.168.1.50",
mac=None,
hostname=None,
os=None,
services=[],
suggested_type="generic",
status="pending",
)
db_session.add(existing)
await db_session.commit()
run_id = str(uuid.uuid4())
run = ScanRun(id=run_id, status="running", ranges=["192.168.1.0/24"])
db_session.add(run)
await db_session.commit()
with (
patch("app.services.scanner._nmap_scan", return_value=[MOCK_HOST]),
patch("app.api.routes.status.broadcast_scan_update", new_callable=AsyncMock),
):
await run_scan(["192.168.1.0/24"], db_session, run_id)
# Should still be only one device
result = await db_session.execute(
select(InventoryDevice).where(InventoryDevice.ip == "192.168.1.50")
)
devices = list(result.scalars().all())
assert len(devices) == 1
device = devices[0]
# Services and hostname should be updated
assert device.hostname == "myhost.lan"
assert any(s["port"] == 8096 for s in device.services)
@pytest.mark.asyncio
async def test_background_device_scan_marks_run_errored_on_exception(mem_db):
"""Same word as the network scan — a device run that blows up reads alike."""
from app.api.routes.scan import _background_device_scan
async with mem_db() as session:
run = ScanRun(status="running", kind="device", ranges=["10.0.0.5/32"])
session.add(run)
await session.commit()
run_id = run.id
with (
patch("app.api.routes.scan.AsyncSessionLocal", mem_db),
patch(
"app.api.routes.scan.run_device_scan",
new_callable=AsyncMock,
side_effect=RuntimeError("boom"),
),
):
await _background_device_scan(run_id, "d1")
async with mem_db() as session:
refreshed = await session.get(ScanRun, run_id)
assert refreshed is not None
assert refreshed.status == "error"
+427
View File
@@ -1,5 +1,56 @@
import time
from unittest.mock import AsyncMock, patch
from urllib.parse import parse_qs, urlparse
import pytest
from authlib.integrations.base_client.errors import OAuthError
from authlib.jose.errors import ExpiredTokenError, JoseError
from fastapi import WebSocketDisconnect
from httpx import AsyncClient
from starlette.responses import RedirectResponse
from starlette.testclient import TestClient
class FakeOIDCClient:
def __init__(self, token=None, error: OAuthError | JoseError | None = None):
self.token = token or {}
self.error = error
self.redirect_uri: str | None = None
async def authorize_redirect(self, _request, redirect_uri: str):
self.redirect_uri = redirect_uri
return RedirectResponse("https://idp.example/application/o/authorize/?state=test", status_code=302)
async def authorize_access_token(self, _request):
if self.error is not None:
raise self.error
return self.token
@pytest.fixture
def oidc_settings():
from app.core.config import settings
from app.core.security import _build_oidc_client
values = {
"auth_mode": "oidc",
"oidc_discovery_url": "https://idp.example/application/o/homelable/.well-known/openid-configuration",
"oidc_client_id": "homelable",
"oidc_client_secret": "test-client-secret",
"oidc_redirect_uri": "http://test/api/v1/auth/oidc/callback",
"oidc_scopes": "openid profile email",
"oidc_cookie_secure": False,
"oidc_session_expire_minutes": 60,
"cors_origins": ["http://test"],
}
original = {name: getattr(settings, name) for name in values}
for name, value in values.items():
setattr(settings, name, value)
_build_oidc_client.cache_clear()
yield settings
for name, value in original.items():
setattr(settings, name, value)
_build_oidc_client.cache_clear()
async def test_login_success(client: AsyncClient):
@@ -10,6 +61,12 @@ async def test_login_success(client: AsyncClient):
assert data["token_type"] == "bearer"
async def test_auth_config_defaults_to_local(client: AsyncClient):
res = await client.get("/api/v1/auth/config")
assert res.status_code == 200
assert res.json() == {"mode": "local", "oidc_login_url": None}
async def test_login_wrong_password(client: AsyncClient):
res = await client.post("/api/v1/auth/login", json={"username": "admin", "password": "wrong"})
assert res.status_code == 401
@@ -68,3 +125,373 @@ async def test_login_with_malformed_hash_returns_401_not_500(client: AsyncClient
assert res.status_code == 401
finally:
settings.auth_password_hash = original
# --- JWT-level cases ---
async def test_expired_token_rejected(client: AsyncClient):
"""A JWT whose `exp` is in the past must be refused."""
from datetime import datetime, timedelta, timezone
import jwt
from app.core.config import settings
payload = {
"sub": "admin",
"exp": datetime.now(timezone.utc) - timedelta(minutes=1),
}
token = jwt.encode(payload, settings.secret_key, algorithm=settings.algorithm)
res = await client.get("/api/v1/nodes", headers={"Authorization": f"Bearer {token}"})
assert res.status_code == 401
async def test_malformed_token_rejected(client: AsyncClient):
res = await client.get("/api/v1/nodes", headers={"Authorization": "Bearer not-a-jwt"})
assert res.status_code == 401
async def test_token_signed_with_wrong_secret_rejected(client: AsyncClient):
"""A token signed with a different key must not be accepted."""
from datetime import datetime, timedelta, timezone
import jwt
from app.core.config import settings
payload = {
"sub": "admin",
"exp": datetime.now(timezone.utc) + timedelta(minutes=5),
}
forged = jwt.encode(payload, "different-secret-that-is-at-least-32-bytes", algorithm=settings.algorithm)
res = await client.get("/api/v1/nodes", headers={"Authorization": f"Bearer {forged}"})
assert res.status_code == 401
async def test_missing_authorization_header_rejected(client: AsyncClient):
res = await client.get("/api/v1/nodes")
assert res.status_code == 401
async def test_empty_password_does_not_pass_when_hash_empty(client: AsyncClient):
"""No credentials configured server-side must not authenticate an empty password."""
from app.core.config import settings
original_hash = settings.auth_password_hash
settings.auth_password_hash = ""
try:
res = await client.post("/api/v1/auth/login", json={"username": "admin", "password": ""})
assert res.status_code == 401
finally:
settings.auth_password_hash = original_hash
# --- Password helper ---
def test_verify_password_handles_empty_inputs():
"""verify_password must be safe against empty plain / empty hash without raising."""
from app.core.security import hash_password, verify_password
h = hash_password("hunter2")
assert verify_password("hunter2", h) is True
assert verify_password("", h) is False
assert verify_password("hunter2", "") is False
assert verify_password("", "") is False
# --- OpenID Connect ---
async def test_oidc_config_exposes_login_url(client: AsyncClient, oidc_settings):
res = await client.get("/api/v1/auth/config")
assert res.status_code == 200
assert res.json() == {
"mode": "oidc",
"oidc_login_url": "/api/v1/auth/oidc/login",
}
async def test_local_login_is_disabled_in_oidc_mode(client: AsyncClient, oidc_settings):
res = await client.post("/api/v1/auth/login", json={"username": "admin", "password": "admin"})
assert res.status_code == 404
async def test_oidc_login_is_hidden_in_local_mode(client: AsyncClient):
res = await client.get("/api/v1/auth/oidc/login")
assert res.status_code == 404
async def test_oidc_login_uses_configured_callback(client: AsyncClient, oidc_settings):
fake_client = FakeOIDCClient()
with patch("app.api.routes.auth.get_oidc_client", return_value=fake_client):
res = await client.get("/api/v1/auth/oidc/login")
assert res.status_code == 302
assert res.headers["location"].startswith("https://idp.example/")
assert fake_client.redirect_uri == "http://test/api/v1/auth/oidc/callback"
async def test_oidc_client_enables_pkce_s256_and_nonce(oidc_settings):
from app.core.security import get_oidc_client
oidc_client = get_oidc_client()
metadata = {"authorization_endpoint": "https://idp.example/application/o/authorize/"}
with patch.object(oidc_client, "load_server_metadata", AsyncMock(return_value=metadata)):
authorization = await oidc_client.create_authorization_url(oidc_settings.oidc_redirect_uri)
query = parse_qs(urlparse(authorization["url"]).query)
assert query["response_type"] == ["code"]
assert query["code_challenge_method"] == ["S256"]
assert "code_challenge" in query
assert "code_verifier" not in query
assert query["nonce"] == [authorization["nonce"]]
assert authorization["code_verifier"]
async def test_oidc_callback_creates_session_for_api(client: AsyncClient, oidc_settings):
fake_client = FakeOIDCClient(token={
"userinfo": {
"iss": "https://idp.example/application/o/homelable/",
"sub": "user-123",
"preferred_username": "alice",
},
})
with patch("app.api.routes.auth.get_oidc_client", return_value=fake_client):
callback = await client.get("/api/v1/auth/oidc/callback")
assert callback.status_code == 303
assert callback.headers["location"] == "/"
cookie = callback.headers["set-cookie"]
assert "homelable-session=" in cookie
assert "HttpOnly" in cookie
assert "SameSite=lax" in cookie
me = await client.get("/api/v1/auth/me")
assert me.status_code == 200
assert me.json()["subject"] == "user-123"
assert me.json()["display_name"] == "alice"
assert me.json()["auth_method"] == "oidc"
assert me.json()["issuer"] == "https://idp.example/application/o/homelable/"
assert me.json()["csrf_token"]
protected = await client.get("/api/v1/nodes")
assert protected.status_code == 200
@pytest.mark.parametrize("userinfo", [None, {}, {"sub": "user-123"}, {"iss": "https://idp.example/"}])
async def test_oidc_callback_rejects_missing_identity_claims(client: AsyncClient, oidc_settings, userinfo):
token = {} if userinfo is None else {"userinfo": userinfo}
fake_client = FakeOIDCClient(token=token)
with patch("app.api.routes.auth.get_oidc_client", return_value=fake_client):
res = await client.get("/api/v1/auth/oidc/callback")
assert res.status_code == 401
assert res.json() == {"detail": "OIDC authentication failed"}
async def test_oidc_callback_hides_protocol_error_details(client: AsyncClient, oidc_settings):
fake_client = FakeOIDCClient(error=OAuthError(error="invalid_grant", description="sensitive detail"))
with patch("app.api.routes.auth.get_oidc_client", return_value=fake_client):
res = await client.get("/api/v1/auth/oidc/callback")
assert res.status_code == 401
assert res.json() == {"detail": "OIDC authentication failed"}
assert "sensitive detail" not in res.text
async def test_oidc_callback_hides_claim_validation_error_details(client: AsyncClient, oidc_settings):
fake_client = FakeOIDCClient(error=ExpiredTokenError())
with patch("app.api.routes.auth.get_oidc_client", return_value=fake_client):
res = await client.get("/api/v1/auth/oidc/callback")
assert res.status_code == 401
assert res.json() == {"detail": "OIDC authentication failed"}
assert "expired" not in res.text.lower()
async def test_oidc_session_token_cannot_be_used_as_bearer(client: AsyncClient, oidc_settings):
from app.core.security import create_oidc_session_token
token = create_oidc_session_token({"iss": "https://idp.example/", "sub": "user-123"})
res = await client.get("/api/v1/nodes", headers={"Authorization": f"Bearer {token}"})
assert res.status_code == 401
async def test_oidc_cookie_mutations_require_csrf_and_allowed_origin(client: AsyncClient, oidc_settings):
fake_client = FakeOIDCClient(token={
"userinfo": {
"iss": "https://idp.example/application/o/homelable/",
"sub": "user-123",
"preferred_username": "alice",
},
})
with patch("app.api.routes.auth.get_oidc_client", return_value=fake_client):
await client.get("/api/v1/auth/oidc/callback")
csrf_token = (await client.get("/api/v1/auth/me")).json()["csrf_token"]
missing = await client.post("/api/v1/auth/logout")
assert missing.status_code == 403
wrong_origin = await client.post(
"/api/v1/auth/logout",
headers={"Origin": "https://evil.example", "X-Homelable-CSRF": csrf_token},
)
assert wrong_origin.status_code == 403
valid = await client.post(
"/api/v1/auth/logout",
headers={"Origin": "http://test", "X-Homelable-CSRF": csrf_token},
)
assert valid.status_code == 204
assert "Max-Age=0" in valid.headers["set-cookie"]
assert (await client.get("/api/v1/auth/me")).status_code == 401
async def test_oidc_mutation_allows_app_origin_missing_from_cors_origins(
client: AsyncClient, oidc_settings
):
"""Regression for #356: a same-origin reverse-proxy deploy needs no CORS, so
CORS_ORIGINS is left at its localhost default while the browser still sends
Origin — every POST used to 403."""
oidc_settings.cors_origins = ["http://localhost:5173", "http://localhost:3000"]
fake_client = FakeOIDCClient(token={
"userinfo": {
"iss": "https://idp.example/application/o/homelable/",
"sub": "user-123",
"preferred_username": "alice",
},
})
with patch("app.api.routes.auth.get_oidc_client", return_value=fake_client):
await client.get("/api/v1/auth/oidc/callback")
csrf_token = (await client.get("/api/v1/auth/me")).json()["csrf_token"]
created = await client.post(
"/api/v1/designs",
json={"name": "From OIDC", "design_type": "network"},
headers={"Origin": "http://test", "X-Homelable-CSRF": csrf_token},
)
assert created.status_code == 201
# The fallback widens the allowlist by exactly one origin, nothing more.
evil = await client.post(
"/api/v1/designs",
json={"name": "Nope", "design_type": "network"},
headers={"Origin": "https://evil.example", "X-Homelable-CSRF": csrf_token},
)
assert evil.status_code == 403
def test_origin_is_allowed_falls_back_to_the_oidc_redirect_origin(oidc_settings):
from app.core.security import origin_is_allowed
oidc_settings.cors_origins = ["http://localhost:3000"]
oidc_settings.oidc_redirect_uri = "https://homelable.example.com/api/v1/auth/oidc/callback"
assert origin_is_allowed("https://homelable.example.com") is True
assert origin_is_allowed("https://homelable.example.com/") is True
assert origin_is_allowed("http://localhost:3000") is True
assert origin_is_allowed("https://evil.example") is False
assert origin_is_allowed(None) is False
# A redirect URI that is not an absolute URL contributes no origin.
oidc_settings.oidc_redirect_uri = "/api/v1/auth/oidc/callback"
assert origin_is_allowed("https://homelable.example.com") is False
def test_oidc_settings_warn_when_cors_origins_omits_the_app_origin(caplog):
from app.core.config import Settings
base = {
"secret_key": "x" * 32,
"auth_mode": "oidc",
"oidc_discovery_url": "https://idp.example/.well-known/openid-configuration",
"oidc_client_id": "homelable",
"oidc_client_secret": "secret",
"oidc_redirect_uri": "https://homelable.example.com/api/v1/auth/oidc/callback",
}
with caplog.at_level("WARNING"):
Settings(**base, cors_origins=["http://localhost:3000"])
assert "https://homelable.example.com" in caplog.text
assert "CORS_ORIGINS" in caplog.text
caplog.clear()
with caplog.at_level("WARNING"):
Settings(**base, cors_origins=["https://homelable.example.com/"])
assert caplog.text == ""
async def test_local_bearer_logout_does_not_require_csrf(client: AsyncClient, headers):
res = await client.post("/api/v1/auth/logout", headers=headers)
assert res.status_code == 204
# --- WebSocket authentication ---
def _wait_for_connection_count(expected: int) -> None:
from app.api.routes import status as status_routes
deadline = time.monotonic() + 1
while len(status_routes._connections) != expected and time.monotonic() < deadline:
time.sleep(0.01)
assert len(status_routes._connections) == expected
def test_local_websocket_keeps_first_message_bearer_flow():
from app.core.security import create_access_token
from app.main import app
test_client = TestClient(app)
try:
with test_client.websocket_connect(
"/api/v1/status/ws/status",
headers={"Origin": "http://localhost:3000"},
) as websocket:
websocket.send_json({"token": create_access_token("admin")})
_wait_for_connection_count(1)
_wait_for_connection_count(0)
finally:
test_client.close()
def test_websocket_rejects_untrusted_origin_before_auth():
from app.main import app
test_client = TestClient(app)
try:
with (
pytest.raises(WebSocketDisconnect) as caught,
test_client.websocket_connect(
"/api/v1/status/ws/status",
headers={"Origin": "https://evil.example"},
) as websocket,
):
websocket.receive_text()
assert caught.value.code == 1008
finally:
test_client.close()
def test_oidc_websocket_uses_session_cookie_without_javascript_token(oidc_settings):
from app.core.security import create_oidc_session_token, oidc_session_cookie_name
from app.main import app
session_token = create_oidc_session_token({"iss": "https://idp.example/", "sub": "user-123"})
test_client = TestClient(app)
test_client.cookies.set(oidc_session_cookie_name(), session_token)
try:
with test_client.websocket_connect(
"/api/v1/status/ws/status",
headers={"Origin": "http://test"},
):
_wait_for_connection_count(1)
_wait_for_connection_count(0)
finally:
test_client.close()
def test_oidc_websocket_rejects_missing_session_cookie(oidc_settings):
from app.main import app
test_client = TestClient(app)
try:
with (
pytest.raises(WebSocketDisconnect) as caught,
test_client.websocket_connect(
"/api/v1/status/ws/status",
headers={"Origin": "http://test"},
) as websocket,
):
websocket.receive_text()
assert caught.value.code == 1008
finally:
test_client.close()
+457 -7
View File
@@ -1,15 +1,8 @@
import uuid
import pytest
from httpx import AsyncClient
@pytest.fixture
async def headers(client: AsyncClient):
res = await client.post("/api/v1/auth/login", json={"username": "admin", "password": "admin"})
return {"Authorization": f"Bearer {res.json()['access_token']}"}
def node_payload(**kwargs):
return {"id": str(uuid.uuid4()), "type": "server", "label": "N", "status": "unknown", "pos_x": 0, "pos_y": 0, **kwargs}
@@ -29,6 +22,23 @@ async def test_load_canvas_empty(client: AsyncClient, headers: dict):
assert data["viewport"] == {"x": 0, "y": 0, "zoom": 1}
async def test_load_canvas_uninitialized_reports_initialized_false(client: AsyncClient, headers: dict):
# A never-saved canvas has no CanvasState row → initialized False. The frontend
# uses this to show the demo canvas only to genuinely new users.
data = (await client.get("/api/v1/canvas", headers=headers)).json()
assert data["initialized"] is False
async def test_load_canvas_initialized_true_after_save(client: AsyncClient, headers: dict):
# Saving an EMPTY canvas still creates a CanvasState row, so a subsequently
# loaded empty canvas is reported initialized — the user cleared it on purpose
# and must not get the demo re-seeded.
await client.post("/api/v1/canvas/save", json={"nodes": [], "edges": [], "viewport": {}}, headers=headers)
data = (await client.get("/api/v1/canvas", headers=headers)).json()
assert data["nodes"] == []
assert data["initialized"] is True
async def test_load_canvas_requires_auth(client: AsyncClient):
res = await client.get("/api/v1/canvas")
assert res.status_code == 401
@@ -51,6 +61,88 @@ async def test_save_canvas_creates_nodes_and_edges(client: AsyncClient, headers:
assert canvas["viewport"] == {"x": 1, "y": 2, "zoom": 1.5}
async def test_save_canvas_round_trips_marker_shapes(client: AsyncClient, headers: dict):
n1 = node_payload(label="Router", type="router")
n2 = node_payload(label="Switch", type="switch")
e1 = edge_payload(n1["id"], n2["id"], marker_start="diamond", marker_end="arrow")
await client.post("/api/v1/canvas/save", json={"nodes": [n1, n2], "edges": [e1], "viewport": {}}, headers=headers)
edge = (await client.get("/api/v1/canvas", headers=headers)).json()["edges"][0]
assert edge["marker_start"] == "diamond"
assert edge["marker_end"] == "arrow"
async def test_save_canvas_round_trips_line_style_and_width(client: AsyncClient, headers: dict):
n1 = node_payload(label="Router", type="router")
n2 = node_payload(label="Switch", type="switch")
e1 = edge_payload(n1["id"], n2["id"], type="wifi", line_style="dotted", width_mult=3)
await client.post("/api/v1/canvas/save", json={"nodes": [n1, n2], "edges": [e1], "viewport": {}}, headers=headers)
edge = (await client.get("/api/v1/canvas", headers=headers)).json()["edges"][0]
assert edge["line_style"] == "dotted"
assert edge["width_mult"] == 3
async def test_save_canvas_coerces_legacy_boolean_marker(client: AsyncClient, headers: dict):
n1 = node_payload(label="Router", type="router")
n2 = node_payload(label="Switch", type="switch")
e1 = edge_payload(n1["id"], n2["id"], marker_start=True, marker_end=False)
await client.post("/api/v1/canvas/save", json={"nodes": [n1, n2], "edges": [e1], "viewport": {}}, headers=headers)
edge = (await client.get("/api/v1/canvas", headers=headers)).json()["edges"][0]
assert edge["marker_start"] == "arrow"
assert edge["marker_end"] == "none"
async def test_save_canvas_defaults_markers_none(client: AsyncClient, headers: dict):
n1 = node_payload(label="Router", type="router")
n2 = node_payload(label="Switch", type="switch")
e1 = edge_payload(n1["id"], n2["id"])
await client.post("/api/v1/canvas/save", json={"nodes": [n1, n2], "edges": [e1], "viewport": {}}, headers=headers)
edge = (await client.get("/api/v1/canvas", headers=headers)).json()["edges"][0]
assert edge["marker_start"] == "none"
assert edge["marker_end"] == "none"
async def test_save_canvas_round_trips_per_side_handles(client: AsyncClient, headers: dict):
# Regression (#243): top/left/right_handles must persist across save+reload,
# not just bottom_handles.
n1 = node_payload(label="Cam", type="camera", top_handles=2, bottom_handles=3, left_handles=1, right_handles=4)
res = await client.post("/api/v1/canvas/save", json={"nodes": [n1], "edges": [], "viewport": {"x": 0, "y": 0, "zoom": 1}}, headers=headers)
assert res.status_code == 200
node = (await client.get("/api/v1/canvas", headers=headers)).json()["nodes"][0]
assert node["top_handles"] == 2
assert node["bottom_handles"] == 3
assert node["left_handles"] == 1
assert node["right_handles"] == 4
async def test_save_canvas_defaults_per_side_handles(client: AsyncClient, headers: dict):
# Nodes saved without the new fields fall back to top/bottom=1, left/right=0.
n1 = node_payload(label="Srv", type="server")
await client.post("/api/v1/canvas/save", json={"nodes": [n1], "edges": [], "viewport": {"x": 0, "y": 0, "zoom": 1}}, headers=headers)
node = (await client.get("/api/v1/canvas", headers=headers)).json()["nodes"][0]
assert node["top_handles"] == 1
assert node["bottom_handles"] == 1
assert node["left_handles"] == 0
assert node["right_handles"] == 0
async def test_load_canvas_exposes_inventory_timestamps(client: AsyncClient, headers: dict):
n1 = node_payload(label="Router", type="router")
await client.post("/api/v1/canvas/save", json={"nodes": [n1], "edges": [], "viewport": {}}, headers=headers)
node = (await client.get("/api/v1/canvas", headers=headers)).json()["nodes"][0]
# created_at / updated_at always set; last_seen / last_scan null until observed.
assert node["created_at"] is not None
assert node["updated_at"] is not None
assert "last_seen" in node
assert node["last_scan"] is None
async def test_save_canvas_updates_existing_node(client: AsyncClient, headers: dict):
n1 = node_payload(label="Old Label")
await client.post("/api/v1/canvas/save", json={"nodes": [n1], "edges": [], "viewport": {}}, headers=headers)
@@ -199,6 +291,24 @@ async def test_save_canvas_show_hardware_defaults_false(client: AsyncClient, hea
assert canvas["nodes"][0]["show_hardware"] is False
# Regression (#184): show_port_numbers was dropped by the save schema, so the
# toggle reset on every reload.
async def test_save_canvas_persists_show_port_numbers(client: AsyncClient, headers: dict):
n1 = node_payload(show_port_numbers=True)
await client.post("/api/v1/canvas/save", json={"nodes": [n1], "edges": [], "viewport": {}}, headers=headers)
canvas = (await client.get("/api/v1/canvas", headers=headers)).json()
assert canvas["nodes"][0]["show_port_numbers"] is True
async def test_save_canvas_show_port_numbers_defaults_false(client: AsyncClient, headers: dict):
n1 = node_payload()
await client.post("/api/v1/canvas/save", json={"nodes": [n1], "edges": [], "viewport": {}}, headers=headers)
canvas = (await client.get("/api/v1/canvas", headers=headers)).json()
assert canvas["nodes"][0]["show_port_numbers"] is False
async def test_save_canvas_hardware_fields_cleared_on_update(client: AsyncClient, headers: dict):
n1 = node_payload(cpu_count=8, ram_gb=32.0)
await client.post("/api/v1/canvas/save", json={"nodes": [n1], "edges": [], "viewport": {}}, headers=headers)
@@ -256,3 +366,343 @@ async def test_save_canvas_dimensions_cleared_when_null(client: AsyncClient, hea
canvas = (await client.get("/api/v1/canvas", headers=headers)).json()
assert canvas["nodes"][0]["width"] is None
assert canvas["nodes"][0]["height"] is None
# ── properties ────────────────────────────────────────────────────────────────
async def test_save_canvas_properties_default_empty(client: AsyncClient, headers: dict):
n1 = node_payload()
await client.post("/api/v1/canvas/save", json={"nodes": [n1], "edges": [], "viewport": {}}, headers=headers)
canvas = (await client.get("/api/v1/canvas", headers=headers)).json()
assert canvas["nodes"][0]["properties"] == []
async def test_save_canvas_persists_properties(client: AsyncClient, headers: dict):
props = [
{"key": "RAM", "value": "32 GB", "icon": "MemoryStick", "visible": True},
{"key": "CPU", "value": "Intel i9", "icon": "Cpu", "visible": False},
]
n1 = node_payload(properties=props)
await client.post("/api/v1/canvas/save", json={"nodes": [n1], "edges": [], "viewport": {}}, headers=headers)
canvas = (await client.get("/api/v1/canvas", headers=headers)).json()
returned = canvas["nodes"][0]["properties"]
assert len(returned) == 2
assert returned[0] == {"key": "RAM", "value": "32 GB", "icon": "MemoryStick", "visible": True}
assert returned[1] == {"key": "CPU", "value": "Intel i9", "icon": "Cpu", "visible": False}
async def test_save_canvas_properties_updated_on_second_save(client: AsyncClient, headers: dict):
n1 = node_payload(properties=[{"key": "RAM", "value": "16 GB", "icon": None, "visible": True}])
await client.post("/api/v1/canvas/save", json={"nodes": [n1], "edges": [], "viewport": {}}, headers=headers)
n1_updated = {**n1, "properties": [
{"key": "RAM", "value": "64 GB", "icon": "MemoryStick", "visible": True},
{"key": "Disk", "value": "2 TB", "icon": "HardDrive", "visible": True},
]}
await client.post("/api/v1/canvas/save", json={"nodes": [n1_updated], "edges": [], "viewport": {}}, headers=headers)
canvas = (await client.get("/api/v1/canvas", headers=headers)).json()
props = canvas["nodes"][0]["properties"]
assert len(props) == 2
assert props[0]["value"] == "64 GB"
assert props[1]["key"] == "Disk"
async def test_save_canvas_properties_with_null_icon(client: AsyncClient, headers: dict):
props = [{"key": "Note", "value": "custom rack", "icon": None, "visible": True}]
n1 = node_payload(properties=props)
await client.post("/api/v1/canvas/save", json={"nodes": [n1], "edges": [], "viewport": {}}, headers=headers)
canvas = (await client.get("/api/v1/canvas", headers=headers)).json()
assert canvas["nodes"][0]["properties"][0]["icon"] is None
async def test_save_canvas_properties_cleared_to_empty(client: AsyncClient, headers: dict):
n1 = node_payload(properties=[{"key": "RAM", "value": "32 GB", "icon": None, "visible": True}])
await client.post("/api/v1/canvas/save", json={"nodes": [n1], "edges": [], "viewport": {}}, headers=headers)
n1_cleared = {**n1, "properties": []}
await client.post("/api/v1/canvas/save", json={"nodes": [n1_cleared], "edges": [], "viewport": {}}, headers=headers)
canvas = (await client.get("/api/v1/canvas", headers=headers)).json()
assert canvas["nodes"][0]["properties"] == []
# ── edge waypoints & handles ──────────────────────────────────────────────────
async def test_save_canvas_edge_waypoints_default_null(client: AsyncClient, headers: dict):
n1 = node_payload()
n2 = node_payload()
e1 = edge_payload(n1["id"], n2["id"])
await client.post("/api/v1/canvas/save", json={"nodes": [n1, n2], "edges": [e1], "viewport": {}}, headers=headers)
canvas = (await client.get("/api/v1/canvas", headers=headers)).json()
assert canvas["edges"][0]["waypoints"] is None
async def test_save_canvas_persists_waypoints_on_edge(client: AsyncClient, headers: dict):
n1 = node_payload()
n2 = node_payload()
waypoints = [{"x": 100.0, "y": 200.0}, {"x": 300.0, "y": 150.0}]
e1 = edge_payload(n1["id"], n2["id"], waypoints=waypoints)
await client.post("/api/v1/canvas/save", json={"nodes": [n1, n2], "edges": [e1], "viewport": {}}, headers=headers)
canvas = (await client.get("/api/v1/canvas", headers=headers)).json()
returned = canvas["edges"][0]["waypoints"]
assert returned == [{"x": 100.0, "y": 200.0}, {"x": 300.0, "y": 150.0}]
async def test_save_canvas_waypoints_updated_on_second_save(client: AsyncClient, headers: dict):
n1 = node_payload()
n2 = node_payload()
e1 = edge_payload(n1["id"], n2["id"], waypoints=[{"x": 10.0, "y": 20.0}])
await client.post("/api/v1/canvas/save", json={"nodes": [n1, n2], "edges": [e1], "viewport": {}}, headers=headers)
e1_updated = {**e1, "waypoints": [{"x": 50.0, "y": 60.0}, {"x": 70.0, "y": 80.0}]}
await client.post("/api/v1/canvas/save", json={"nodes": [n1, n2], "edges": [e1_updated], "viewport": {}}, headers=headers)
canvas = (await client.get("/api/v1/canvas", headers=headers)).json()
assert canvas["edges"][0]["waypoints"] == [{"x": 50.0, "y": 60.0}, {"x": 70.0, "y": 80.0}]
async def test_save_canvas_persists_edge_handles(client: AsyncClient, headers: dict):
n1 = node_payload(bottom_handles=3)
n2 = node_payload()
e1 = edge_payload(n1["id"], n2["id"], source_handle="bottom-1", target_handle="top")
await client.post("/api/v1/canvas/save", json={"nodes": [n1, n2], "edges": [e1], "viewport": {}}, headers=headers)
canvas = (await client.get("/api/v1/canvas", headers=headers)).json()
edge = canvas["edges"][0]
assert edge["source_handle"] == "bottom-1"
assert edge["target_handle"] == "top"
async def test_save_canvas_persists_animated_edge(client: AsyncClient, headers: dict):
n1 = node_payload()
n2 = node_payload()
e1 = edge_payload(n1["id"], n2["id"], animated="snake")
await client.post("/api/v1/canvas/save", json={"nodes": [n1, n2], "edges": [e1], "viewport": {}}, headers=headers)
canvas = (await client.get("/api/v1/canvas", headers=headers)).json()
assert canvas["edges"][0]["animated"] == "snake"
async def test_save_canvas_persists_animated_basic(client: AsyncClient, headers: dict):
n1 = node_payload()
n2 = node_payload()
e1 = edge_payload(n1["id"], n2["id"], animated="basic")
await client.post("/api/v1/canvas/save", json={"nodes": [n1, n2], "edges": [e1], "viewport": {}}, headers=headers)
canvas = (await client.get("/api/v1/canvas", headers=headers)).json()
assert canvas["edges"][0]["animated"] == "basic"
# ── node fields ───────────────────────────────────────────────────────────────
async def test_save_canvas_persists_all_node_fields(client: AsyncClient, headers: dict):
n1 = node_payload(
type="server",
label="Main Server",
hostname="server.local",
ip="192.168.1.10",
mac="aa:bb:cc:dd:ee:ff",
os="Ubuntu 22.04",
status="online",
check_method="http",
check_target="http://192.168.1.10",
services=[{"name": "nginx", "port": 80}],
notes="Primary web server",
pos_x=150.0,
pos_y=250.0,
bottom_handles=2,
)
await client.post("/api/v1/canvas/save", json={"nodes": [n1], "edges": [], "viewport": {}}, headers=headers)
canvas = (await client.get("/api/v1/canvas", headers=headers)).json()
node = canvas["nodes"][0]
assert node["hostname"] == "server.local"
assert node["ip"] == "192.168.1.10"
assert node["mac"] == "aa:bb:cc:dd:ee:ff"
assert node["os"] == "Ubuntu 22.04"
assert node["status"] == "online"
assert node["check_method"] == "http"
assert node["check_target"] == "http://192.168.1.10"
assert node["services"] == [{"name": "nginx", "port": 80}]
assert node["notes"] == "Primary web server"
assert node["pos_x"] == 150.0
assert node["pos_y"] == 250.0
assert node["bottom_handles"] == 2
async def test_save_canvas_persists_bottom_handles(client: AsyncClient, headers: dict):
n1 = node_payload(bottom_handles=4)
await client.post("/api/v1/canvas/save", json={"nodes": [n1], "edges": [], "viewport": {}}, headers=headers)
canvas = (await client.get("/api/v1/canvas", headers=headers)).json()
assert canvas["nodes"][0]["bottom_handles"] == 4
async def test_save_canvas_bottom_handles_defaults_one(client: AsyncClient, headers: dict):
n1 = node_payload()
await client.post("/api/v1/canvas/save", json={"nodes": [n1], "edges": [], "viewport": {}}, headers=headers)
canvas = (await client.get("/api/v1/canvas", headers=headers)).json()
assert canvas["nodes"][0]["bottom_handles"] == 1
async def test_save_canvas_persists_services_and_notes(client: AsyncClient, headers: dict):
services = [{"name": "ssh", "port": 22}, {"name": "http", "port": 80}]
n1 = node_payload(services=services, notes="My NAS device")
await client.post("/api/v1/canvas/save", json={"nodes": [n1], "edges": [], "viewport": {}}, headers=headers)
canvas = (await client.get("/api/v1/canvas", headers=headers)).json()
node = canvas["nodes"][0]
assert node["services"] == services
assert node["notes"] == "My NAS device"
async def test_save_canvas_persists_service_paths(client: AsyncClient, headers: dict):
services = [{"service_name": "Grafana", "protocol": "tcp", "port": 3000, "path": "/login"}]
n1 = node_payload(ip="192.168.1.50:8080", services=services)
await client.post("/api/v1/canvas/save", json={"nodes": [n1], "edges": [], "viewport": {}}, headers=headers)
canvas = (await client.get("/api/v1/canvas", headers=headers)).json()
assert canvas["nodes"][0]["services"] == services
async def test_save_canvas_persists_check_fields(client: AsyncClient, headers: dict):
n1 = node_payload(check_method="ping", check_target="192.168.1.1")
await client.post("/api/v1/canvas/save", json={"nodes": [n1], "edges": [], "viewport": {}}, headers=headers)
canvas = (await client.get("/api/v1/canvas", headers=headers)).json()
node = canvas["nodes"][0]
assert node["check_method"] == "ping"
assert node["check_target"] == "192.168.1.1"
# ── parent/child nodes ────────────────────────────────────────────────────────
async def test_save_canvas_persists_parent_child_nodes(client: AsyncClient, headers: dict):
parent = node_payload(type="proxmox", label="PVE Host")
child = node_payload(type="vm", label="VM-100", parent_id=parent["id"])
await client.post("/api/v1/canvas/save", json={"nodes": [parent, child], "edges": [], "viewport": {}}, headers=headers)
canvas = (await client.get("/api/v1/canvas", headers=headers)).json()
node_map = {n["id"]: n for n in canvas["nodes"]}
assert node_map[child["id"]]["parent_id"] == parent["id"]
assert node_map[parent["id"]]["parent_id"] is None
async def test_save_canvas_child_removed_with_parent(client: AsyncClient, headers: dict):
parent = node_payload(type="proxmox", label="PVE Host")
child = node_payload(type="lxc", label="LXC-101", parent_id=parent["id"])
await client.post("/api/v1/canvas/save", json={"nodes": [parent, child], "edges": [], "viewport": {}}, headers=headers)
# Remove both parent and child
await client.post("/api/v1/canvas/save", json={"nodes": [], "edges": [], "viewport": {}}, headers=headers)
canvas = (await client.get("/api/v1/canvas", headers=headers)).json()
assert canvas["nodes"] == []
# ── groupRect / group node ────────────────────────────────────────────────────
async def test_save_canvas_persists_group_node(client: AsyncClient, headers: dict):
group = node_payload(type="group", label="Network Zone", width=400.0, height=300.0)
member = node_payload(type="server", label="Member", parent_id=group["id"])
await client.post("/api/v1/canvas/save", json={"nodes": [group, member], "edges": [], "viewport": {}}, headers=headers)
canvas = (await client.get("/api/v1/canvas", headers=headers)).json()
node_map = {n["id"]: n for n in canvas["nodes"]}
assert node_map[group["id"]]["type"] == "group"
assert node_map[group["id"]]["width"] == 400.0
assert node_map[group["id"]]["height"] == 300.0
assert node_map[member["id"]]["parent_id"] == group["id"]
# ── viewport ──────────────────────────────────────────────────────────────────
async def test_load_canvas_returns_default_viewport_when_no_state(client: AsyncClient, headers: dict):
canvas = (await client.get("/api/v1/canvas", headers=headers)).json()
assert canvas["viewport"] == {"x": 0, "y": 0, "zoom": 1}
async def test_save_canvas_updates_existing_canvas_state(client: AsyncClient, headers: dict):
"""Second save updates the existing CanvasState row (exercises the state.viewport branch)."""
await client.post("/api/v1/canvas/save", json={"nodes": [], "edges": [], "viewport": {"x": 1, "y": 2, "zoom": 1}}, headers=headers)
await client.post("/api/v1/canvas/save", json={"nodes": [], "edges": [], "viewport": {"x": 99, "y": 88, "zoom": 0.75}}, headers=headers)
canvas = (await client.get("/api/v1/canvas", headers=headers)).json()
assert canvas["viewport"] == {"x": 99, "y": 88, "zoom": 0.75}
# ── edge types ────────────────────────────────────────────────────────────────
async def test_save_canvas_persists_edge_type_vlan(client: AsyncClient, headers: dict):
n1 = node_payload()
n2 = node_payload()
e1 = edge_payload(n1["id"], n2["id"], type="vlan", vlan_id=10, label="VLAN 10")
await client.post("/api/v1/canvas/save", json={"nodes": [n1, n2], "edges": [e1], "viewport": {}}, headers=headers)
canvas = (await client.get("/api/v1/canvas", headers=headers)).json()
edge = canvas["edges"][0]
assert edge["type"] == "vlan"
assert edge["vlan_id"] == 10
assert edge["label"] == "VLAN 10"
async def test_save_canvas_edge_update_existing(client: AsyncClient, headers: dict):
"""Second save updates an existing edge (exercises the db_edge branch)."""
n1 = node_payload()
n2 = node_payload()
e1 = edge_payload(n1["id"], n2["id"], label="original")
await client.post("/api/v1/canvas/save", json={"nodes": [n1, n2], "edges": [e1], "viewport": {}}, headers=headers)
e1_updated = {**e1, "label": "updated", "custom_color": "#ff0000"}
await client.post("/api/v1/canvas/save", json={"nodes": [n1, n2], "edges": [e1_updated], "viewport": {}}, headers=headers)
canvas = (await client.get("/api/v1/canvas", headers=headers)).json()
edge = canvas["edges"][0]
assert edge["label"] == "updated"
assert edge["custom_color"] == "#ff0000"
# ── custom_style ──────────────────────────────────────────────────────────────
async def test_save_and_load_custom_style(client: AsyncClient, headers: dict):
custom_style = {
"nodes": {
"server": {"borderColor": "#ff0000", "borderOpacity": 0.8, "bgColor": "#000000", "bgOpacity": 1, "iconColor": "#ff0000", "iconOpacity": 1, "width": 200, "height": 80},
},
"edges": {
"ethernet": {"color": "#00ff00", "opacity": 1, "pathStyle": "bezier", "animated": "none"},
},
}
payload = {"nodes": [], "edges": [], "viewport": {"theme_id": "custom"}, "custom_style": custom_style}
res = await client.post("/api/v1/canvas/save", json=payload, headers=headers)
assert res.status_code == 200
canvas = (await client.get("/api/v1/canvas", headers=headers)).json()
assert canvas["custom_style"] is not None
assert canvas["custom_style"]["nodes"]["server"]["borderColor"] == "#ff0000"
assert canvas["custom_style"]["edges"]["ethernet"]["color"] == "#00ff00"
async def test_load_canvas_custom_style_null_by_default(client: AsyncClient, headers: dict):
res = await client.get("/api/v1/canvas", headers=headers)
assert res.status_code == 200
assert res.json()["custom_style"] is None
async def test_save_canvas_custom_style_overwrite(client: AsyncClient, headers: dict):
style_v1 = {"nodes": {"server": {"borderColor": "#aabbcc", "borderOpacity": 1, "bgColor": "#000000", "bgOpacity": 1, "iconColor": "#aabbcc", "iconOpacity": 1, "width": 0, "height": 0}}, "edges": {}}
style_v2 = {"nodes": {"proxmox": {"borderColor": "#ff6e00", "borderOpacity": 1, "bgColor": "#111111", "bgOpacity": 1, "iconColor": "#ff6e00", "iconOpacity": 1, "width": 0, "height": 0}}, "edges": {}}
await client.post("/api/v1/canvas/save", json={"nodes": [], "edges": [], "viewport": {}, "custom_style": style_v1}, headers=headers)
await client.post("/api/v1/canvas/save", json={"nodes": [], "edges": [], "viewport": {}, "custom_style": style_v2}, headers=headers)
canvas = (await client.get("/api/v1/canvas", headers=headers)).json()
assert "proxmox" in canvas["custom_style"]["nodes"]
assert "server" not in canvas["custom_style"]["nodes"]
+131
View File
@@ -0,0 +1,131 @@
"""The pending_devices -> device_inventory rename, run at startup.
It is the one migration that must happen *before* `create_all`, so it gets its
own suite: get the order wrong and every device a user ever scanned reads as
gone, because the app looks at a freshly created empty table.
"""
import pytest
from sqlalchemy.ext.asyncio import create_async_engine
from app.db.database import _rename_legacy_tables
pytestmark = pytest.mark.asyncio
def _legacy_ddl(table: str) -> str:
return f"CREATE TABLE {table} (id VARCHAR PRIMARY KEY, hostname VARCHAR)"
async def _engine(tmp_path, *, statements: list[str]):
engine = create_async_engine(f"sqlite+aiosqlite:///{tmp_path / 'test.db'}")
async with engine.begin() as conn:
for sql in statements:
await conn.exec_driver_sql(sql)
return engine
async def _tables(engine) -> set[str]:
async with engine.begin() as conn:
rows = (
await conn.exec_driver_sql("SELECT name FROM sqlite_master WHERE type='table'")
).fetchall()
return {r[0] for r in rows}
async def _rename(engine) -> None:
async with engine.begin() as conn:
await _rename_legacy_tables(conn)
async def test_renames_the_legacy_tables_keeping_their_rows(tmp_path):
engine = await _engine(
tmp_path,
statements=[
_legacy_ddl("pending_devices"),
_legacy_ddl("pending_device_links"),
"INSERT INTO pending_devices (id, hostname) VALUES ('d1', 'nas')",
],
)
await _rename(engine)
assert await _tables(engine) == {"device_inventory", "device_inventory_links"}
async with engine.begin() as conn:
rows = (await conn.exec_driver_sql("SELECT hostname FROM device_inventory")).fetchall()
assert [r[0] for r in rows] == ["nas"]
await engine.dispose()
async def test_rewrites_the_reference_from_rack_devices(tmp_path):
# `rack_devices.device_id` names the inventory table. Left pointing at the
# old name, the schema references a table that no longer exists.
engine = await _engine(
tmp_path,
statements=[
_legacy_ddl("pending_devices"),
"CREATE TABLE rack_devices (id VARCHAR PRIMARY KEY, device_id VARCHAR "
"REFERENCES pending_devices(id) ON DELETE SET NULL)",
],
)
await _rename(engine)
async with engine.begin() as conn:
ddl = (
await conn.exec_driver_sql(
"SELECT sql FROM sqlite_master WHERE name='rack_devices'"
)
).scalar()
assert "device_inventory" in ddl
assert "pending_devices" not in ddl
await engine.dispose()
async def test_replaces_an_empty_table_left_by_an_earlier_start(tmp_path):
# A start that ran `create_all` before this migration existed created the new
# table empty, and the app read that one instead of the user's devices.
engine = await _engine(
tmp_path,
statements=[
_legacy_ddl("pending_devices"),
_legacy_ddl("device_inventory"),
"INSERT INTO pending_devices (id, hostname) VALUES ('d1', 'nas')",
],
)
await _rename(engine)
assert await _tables(engine) == {"device_inventory"}
async with engine.begin() as conn:
count = (await conn.exec_driver_sql("SELECT COUNT(*) FROM device_inventory")).scalar()
assert count == 1
await engine.dispose()
async def test_leaves_two_populated_tables_alone(tmp_path):
# Rows on both sides is not a state this migration created, and dropping
# either one would lose data.
engine = await _engine(
tmp_path,
statements=[
_legacy_ddl("pending_devices"),
_legacy_ddl("device_inventory"),
"INSERT INTO pending_devices (id, hostname) VALUES ('d1', 'old')",
"INSERT INTO device_inventory (id, hostname) VALUES ('d2', 'new')",
],
)
await _rename(engine)
assert await _tables(engine) == {"pending_devices", "device_inventory"}
await engine.dispose()
async def test_is_a_no_op_on_a_fresh_database(tmp_path):
engine = await _engine(tmp_path, statements=[_legacy_ddl("device_inventory")])
await _rename(engine)
await _rename(engine) # And still a no-op the second time round.
assert await _tables(engine) == {"device_inventory"}
await engine.dispose()
+270
View File
@@ -0,0 +1,270 @@
import uuid
from httpx import AsyncClient
def node_payload(**kwargs):
return {"id": str(uuid.uuid4()), "type": "server", "label": "N", "status": "unknown", "pos_x": 0, "pos_y": 0, **kwargs}
def edge_payload(src, tgt, **kwargs):
return {"id": str(uuid.uuid4()), "source": src, "target": tgt, "type": "ethernet", **kwargs}
async def _create(client: AsyncClient, headers: dict, **body) -> dict:
res = await client.post("/api/v1/designs", json={"name": "D", **body}, headers=headers)
assert res.status_code == 201, res.text
return res.json()
# ── auth ──────────────────────────────────────────────────────────────────────
async def test_list_designs_requires_auth(client: AsyncClient):
res = await client.get("/api/v1/designs")
assert res.status_code == 401
async def test_create_design_requires_auth(client: AsyncClient):
res = await client.post("/api/v1/designs", json={"name": "X"})
assert res.status_code == 401
# ── list / create ─────────────────────────────────────────────────────────────
async def test_list_designs_empty(client: AsyncClient, headers: dict):
res = await client.get("/api/v1/designs", headers=headers)
assert res.status_code == 200
assert res.json() == []
async def test_create_design_defaults(client: AsyncClient, headers: dict):
design = await _create(client, headers, name="Workshop")
assert design["name"] == "Workshop"
assert design["design_type"] == "network"
assert design["icon"] == "dashboard"
assert "id" in design and design["id"]
async def test_create_design_explicit_type(client: AsyncClient, headers: dict):
design = await _create(client, headers, name="Net", design_type="network")
assert design["design_type"] == "network"
async def test_create_design_with_custom_icon(client: AsyncClient, headers: dict):
design = await _create(client, headers, name="Power", icon="zap")
assert design["icon"] == "zap"
async def test_update_design_changes_icon(client: AsyncClient, headers: dict):
design = await _create(client, headers, name="D", icon="dashboard")
res = await client.put(f"/api/v1/designs/{design['id']}", json={"icon": "server"}, headers=headers)
assert res.status_code == 200
assert res.json()["icon"] == "server"
# Name left untouched when only icon is sent.
assert res.json()["name"] == "D"
async def test_update_design_name_and_icon_together(client: AsyncClient, headers: dict):
design = await _create(client, headers, name="Old", icon="dashboard")
res = await client.put(
f"/api/v1/designs/{design['id']}", json={"name": "New", "icon": "network"}, headers=headers,
)
assert res.status_code == 200
body = res.json()
assert body["name"] == "New"
assert body["icon"] == "network"
async def test_create_design_creates_empty_canvas_state(client: AsyncClient, headers: dict):
design = await _create(client, headers, name="Has Canvas")
# Loading the new design returns an (empty) canvas without falling back to another design.
res = await client.get("/api/v1/canvas", params={"design_id": design["id"]}, headers=headers)
assert res.status_code == 200
body = res.json()
assert body["nodes"] == []
assert body["edges"] == []
async def test_list_returns_created_designs_ordered(client: AsyncClient, headers: dict):
a = await _create(client, headers, name="First")
b = await _create(client, headers, name="Second")
listed = (await client.get("/api/v1/designs", headers=headers)).json()
ids = [d["id"] for d in listed]
assert ids == [a["id"], b["id"]]
# ── counts in list ──────────────────────────────────────────────────────────
async def test_list_includes_node_group_text_counts(client: AsyncClient, headers: dict):
design = await _create(client, headers, name="Counted")
server = node_payload(label="S", type="server")
group = node_payload(label="G", type="groupRect")
text = node_payload(label="T", type="text")
save = await client.post(
"/api/v1/canvas/save",
json={"nodes": [server, group, text], "edges": [], "viewport": {}, "design_id": design["id"]},
headers=headers,
)
assert save.status_code == 200
listed = (await client.get("/api/v1/designs", headers=headers)).json()
d = next(x for x in listed if x["id"] == design["id"])
assert d["node_count"] == 1
assert d["group_count"] == 1
assert d["text_count"] == 1
async def test_list_counts_zero_for_empty_design(client: AsyncClient, headers: dict):
design = await _create(client, headers, name="Empty")
listed = (await client.get("/api/v1/designs", headers=headers)).json()
d = next(x for x in listed if x["id"] == design["id"])
assert d["node_count"] == 0
assert d["group_count"] == 0
assert d["text_count"] == 0
# ── copy ──────────────────────────────────────────────────────────────────────
async def test_copy_requires_auth(client: AsyncClient):
res = await client.post(f"/api/v1/designs/{uuid.uuid4()}/copy", json={"name": "X"})
assert res.status_code == 401
async def test_copy_missing_source_returns_404(client: AsyncClient, headers: dict):
res = await client.post(f"/api/v1/designs/{uuid.uuid4()}/copy", json={"name": "X"}, headers=headers)
assert res.status_code == 404
async def test_copy_duplicates_nodes_edges_and_remaps_ids(client: AsyncClient, headers: dict):
source = await _create(client, headers, name="Source", icon="server")
n1 = node_payload(label="A")
n2 = node_payload(label="B")
e1 = edge_payload(n1["id"], n2["id"], label="link")
save = await client.post(
"/api/v1/canvas/save",
json={"nodes": [n1, n2], "edges": [e1], "viewport": {"x": 5, "y": 6, "zoom": 2}, "design_id": source["id"]},
headers=headers,
)
assert save.status_code == 200
res = await client.post(
f"/api/v1/designs/{source['id']}/copy", json={"name": "Copy", "icon": "network"}, headers=headers,
)
assert res.status_code == 201, res.text
copy = res.json()
assert copy["name"] == "Copy"
assert copy["icon"] == "network"
assert copy["design_type"] == "network"
assert copy["id"] != source["id"]
# Copied canvas has the same shape but fresh node ids, and edge re-pointed.
canvas = (await client.get("/api/v1/canvas", params={"design_id": copy["id"]}, headers=headers)).json()
assert {n["label"] for n in canvas["nodes"]} == {"A", "B"}
copied_ids = {n["id"] for n in canvas["nodes"]}
assert copied_ids.isdisjoint({n1["id"], n2["id"]})
assert len(canvas["edges"]) == 1
edge = canvas["edges"][0]
assert edge["source"] in copied_ids
assert edge["target"] in copied_ids
assert edge["label"] == "link"
assert canvas["viewport"] == {"x": 5, "y": 6, "zoom": 2}
async def test_copy_remaps_parent_child_relationship(client: AsyncClient, headers: dict):
source = await _create(client, headers, name="Nested")
parent = node_payload(label="P", type="proxmox", container_mode=True)
child = node_payload(label="C", type="vm", parent_id=parent["id"])
save = await client.post(
"/api/v1/canvas/save",
json={"nodes": [parent, child], "edges": [], "viewport": {}, "design_id": source["id"]},
headers=headers,
)
assert save.status_code == 200
res = await client.post(f"/api/v1/designs/{source['id']}/copy", json={"name": "Copy"}, headers=headers)
assert res.status_code == 201
copy = res.json()
canvas = (await client.get("/api/v1/canvas", params={"design_id": copy["id"]}, headers=headers)).json()
by_label = {n["label"]: n for n in canvas["nodes"]}
# Child's parent_id points at the COPIED parent, not the original.
assert by_label["C"]["parent_id"] == by_label["P"]["id"]
assert by_label["C"]["parent_id"] != parent["id"]
async def test_copy_leaves_source_untouched(client: AsyncClient, headers: dict):
source = await _create(client, headers, name="Source")
n1 = node_payload(label="A")
await client.post(
"/api/v1/canvas/save",
json={"nodes": [n1], "edges": [], "viewport": {}, "design_id": source["id"]},
headers=headers,
)
await client.post(f"/api/v1/designs/{source['id']}/copy", json={"name": "Copy"}, headers=headers)
src_canvas = (await client.get("/api/v1/canvas", params={"design_id": source["id"]}, headers=headers)).json()
assert len(src_canvas["nodes"]) == 1
assert src_canvas["nodes"][0]["id"] == n1["id"]
# ── update ────────────────────────────────────────────────────────────────────
async def test_update_design_renames(client: AsyncClient, headers: dict):
design = await _create(client, headers, name="Old Name")
res = await client.put(f"/api/v1/designs/{design['id']}", json={"name": "New Name"}, headers=headers)
assert res.status_code == 200
assert res.json()["name"] == "New Name"
async def test_update_design_missing_returns_404(client: AsyncClient, headers: dict):
res = await client.put(f"/api/v1/designs/{uuid.uuid4()}", json={"name": "X"}, headers=headers)
assert res.status_code == 404
# ── delete ────────────────────────────────────────────────────────────────────
async def test_delete_last_design_blocked(client: AsyncClient, headers: dict):
design = await _create(client, headers, name="Only One")
res = await client.delete(f"/api/v1/designs/{design['id']}", headers=headers)
assert res.status_code == 400
async def test_delete_design_missing_returns_404(client: AsyncClient, headers: dict):
# Need >1 design so we get past nothing; 404 path is checked before the count guard.
await _create(client, headers, name="Keep")
res = await client.delete(f"/api/v1/designs/{uuid.uuid4()}", headers=headers)
assert res.status_code == 404
async def test_delete_design_removes_its_nodes_edges_and_canvas(client: AsyncClient, headers: dict):
keep = await _create(client, headers, name="Keep")
victim = await _create(client, headers, name="Victim")
# Populate the victim design with nodes + an edge via canvas save.
n1 = node_payload(label="A")
n2 = node_payload(label="B")
e1 = edge_payload(n1["id"], n2["id"])
save = await client.post(
"/api/v1/canvas/save",
json={"nodes": [n1, n2], "edges": [e1], "viewport": {}, "design_id": victim["id"]},
headers=headers,
)
assert save.status_code == 200
# Populate the kept design too, to prove scoping.
k1 = node_payload(label="K")
await client.post(
"/api/v1/canvas/save",
json={"nodes": [k1], "edges": [], "viewport": {}, "design_id": keep["id"]},
headers=headers,
)
res = await client.delete(f"/api/v1/designs/{victim['id']}", headers=headers)
assert res.status_code == 204
# Victim gone from list.
listed = (await client.get("/api/v1/designs", headers=headers)).json()
assert [d["id"] for d in listed] == [keep["id"]]
# Kept design's node survives untouched.
kept_canvas = (await client.get("/api/v1/canvas", params={"design_id": keep["id"]}, headers=headers)).json()
assert len(kept_canvas["nodes"]) == 1
assert kept_canvas["nodes"][0]["label"] == "K"
+205 -7
View File
@@ -2,13 +2,6 @@ import pytest
from httpx import AsyncClient
@pytest.fixture
async def headers(client: AsyncClient):
res = await client.post("/api/v1/auth/login", json={"username": "admin", "password": "admin"})
token = res.json()["access_token"]
return {"Authorization": f"Bearer {token}"}
@pytest.fixture
async def two_nodes(client: AsyncClient, headers: dict):
n1 = (await client.post("/api/v1/nodes", json={"type": "router", "label": "R1", "status": "online"}, headers=headers)).json()
@@ -91,12 +84,107 @@ async def test_update_edge_custom_color_and_path_style(client: AsyncClient, head
assert res.json()["path_style"] == "smooth"
async def test_create_edge_with_line_style_and_width(client: AsyncClient, headers: dict, two_nodes):
src, tgt = two_nodes
res = await client.post("/api/v1/edges", json={"source": src, "target": tgt, "type": "wifi", "line_style": "dotted", "width_mult": 3}, headers=headers)
assert res.status_code == 201
assert res.json()["line_style"] == "dotted"
assert res.json()["width_mult"] == 3
async def test_update_edge_line_style_and_width(client: AsyncClient, headers: dict, two_nodes):
src, tgt = two_nodes
edge_id = (await client.post("/api/v1/edges", json={"source": src, "target": tgt, "type": "ethernet"}, headers=headers)).json()["id"]
res = await client.patch(f"/api/v1/edges/{edge_id}", json={"line_style": "dashed", "width_mult": 2}, headers=headers)
assert res.status_code == 200
assert res.json()["line_style"] == "dashed"
assert res.json()["width_mult"] == 2
async def test_create_edge_defaults_line_style_none(client: AsyncClient, headers: dict, two_nodes):
src, tgt = two_nodes
res = await client.post("/api/v1/edges", json={"source": src, "target": tgt, "type": "ethernet"}, headers=headers)
assert res.status_code == 201
assert res.json()["line_style"] is None
assert res.json()["width_mult"] is None
async def test_create_edge_with_marker_shapes(client: AsyncClient, headers: dict, two_nodes):
src, tgt = two_nodes
res = await client.post("/api/v1/edges", json={"source": src, "target": tgt, "type": "ethernet", "marker_start": "diamond", "marker_end": "arrow"}, headers=headers)
assert res.status_code == 201
assert res.json()["marker_start"] == "diamond"
assert res.json()["marker_end"] == "arrow"
async def test_create_edge_defaults_markers_none(client: AsyncClient, headers: dict, two_nodes):
src, tgt = two_nodes
res = await client.post("/api/v1/edges", json={"source": src, "target": tgt, "type": "ethernet"}, headers=headers)
assert res.status_code == 201
assert res.json()["marker_start"] == "none"
assert res.json()["marker_end"] == "none"
async def test_create_edge_coerces_legacy_boolean_marker(client: AsyncClient, headers: dict, two_nodes):
src, tgt = two_nodes
res = await client.post("/api/v1/edges", json={"source": src, "target": tgt, "type": "ethernet", "marker_end": True}, headers=headers)
assert res.status_code == 201
assert res.json()["marker_end"] == "arrow"
async def test_create_edge_rejects_unknown_marker_shape(client: AsyncClient, headers: dict, two_nodes):
src, tgt = two_nodes
res = await client.post("/api/v1/edges", json={"source": src, "target": tgt, "type": "ethernet", "marker_end": "bogus"}, headers=headers)
assert res.status_code == 201
assert res.json()["marker_end"] == "none"
async def test_update_edge_marker_shape(client: AsyncClient, headers: dict, two_nodes):
src, tgt = two_nodes
edge_id = (await client.post("/api/v1/edges", json={"source": src, "target": tgt, "type": "ethernet"}, headers=headers)).json()["id"]
res = await client.patch(f"/api/v1/edges/{edge_id}", json={"marker_end": "circle"}, headers=headers)
assert res.status_code == 200
assert res.json()["marker_end"] == "circle"
assert res.json()["marker_start"] == "none"
async def test_create_edge_requires_auth(client: AsyncClient, two_nodes):
src, tgt = two_nodes
res = await client.post("/api/v1/edges", json={"source": src, "target": tgt, "type": "ethernet"})
assert res.status_code == 401
async def test_create_edge_without_design_id_falls_back_to_first_design(client: AsyncClient, headers: dict, two_nodes):
# Regression for #225: MCP create_edge sent no design_id, so edges were
# persisted with design_id=null and never rendered until a restart.
src, tgt = two_nodes
design = await client.post("/api/v1/designs", json={"name": "Primary"}, headers=headers)
design_id = design.json()["id"]
res = await client.post(
"/api/v1/edges",
json={"source": src, "target": tgt, "type": "ethernet"},
headers=headers,
)
assert res.status_code == 201
assert res.json()["design_id"] == design_id
async def test_create_edge_respects_explicit_design_id(client: AsyncClient, headers: dict, two_nodes):
src, tgt = two_nodes
await client.post("/api/v1/designs", json={"name": "First"}, headers=headers)
second = await client.post("/api/v1/designs", json={"name": "Second"}, headers=headers)
second_id = second.json()["id"]
res = await client.post(
"/api/v1/edges",
json={"source": src, "target": tgt, "type": "ethernet", "design_id": second_id},
headers=headers,
)
assert res.status_code == 201
assert res.json()["design_id"] == second_id
async def test_create_cluster_edge_with_handles(client: AsyncClient, headers: dict, two_nodes):
src, tgt = two_nodes
res = await client.post(
@@ -132,3 +220,113 @@ async def test_source_and_target_handle_persist_through_update(client: AsyncClie
assert data["source_handle"] == "cluster-right"
assert data["target_handle"] == "cluster-left"
assert data["label"] == "corosync"
# ---------------------------------------------------------------------------
# Auto edge handles (issue #265): omitting source_handle/target_handle lets
# the backend infer up/downstream sides from the nodes' absolute canvas Y.
# ---------------------------------------------------------------------------
@pytest.fixture
async def stacked_nodes(client: AsyncClient, headers: dict):
"""Two root nodes at known, distinct Y positions (top at y=0, bottom at y=300)."""
top = (
await client.post(
"/api/v1/nodes",
json={"type": "router", "label": "Top", "status": "online", "pos_x": 0, "pos_y": 0},
headers=headers,
)
).json()
bottom = (
await client.post(
"/api/v1/nodes",
json={"type": "switch", "label": "Bottom", "status": "online", "pos_x": 0, "pos_y": 300},
headers=headers,
)
).json()
return top["id"], bottom["id"]
async def test_create_edge_auto_handles_source_above_target(client: AsyncClient, headers: dict, stacked_nodes):
"""Source above target -> downstream flow: exit bottom, enter top-t."""
top, bottom = stacked_nodes
res = await client.post(
"/api/v1/edges", json={"source": top, "target": bottom, "type": "ethernet"}, headers=headers
)
assert res.status_code == 201
data = res.json()
assert data["source_handle"] == "bottom"
assert data["target_handle"] == "top-t"
async def test_create_edge_auto_handles_source_below_target(client: AsyncClient, headers: dict, stacked_nodes):
"""Source below target -> upstream flow: exit top, enter bottom-t."""
top, bottom = stacked_nodes
res = await client.post(
"/api/v1/edges", json={"source": bottom, "target": top, "type": "ethernet"}, headers=headers
)
assert res.status_code == 201
data = res.json()
assert data["source_handle"] == "top"
assert data["target_handle"] == "bottom-t"
async def test_create_edge_auto_handles_equal_y_defaults_downstream(client: AsyncClient, headers: dict, two_nodes):
"""Same Y (auto-placed side by side) falls back to the default bottom/top-t."""
src, tgt = two_nodes
res = await client.post(
"/api/v1/edges", json={"source": src, "target": tgt, "type": "ethernet"}, headers=headers
)
assert res.status_code == 201
data = res.json()
assert data["source_handle"] == "bottom"
assert data["target_handle"] == "top-t"
async def test_create_edge_partial_handle_auto_fills_the_other(client: AsyncClient, headers: dict, stacked_nodes):
"""An explicit source_handle is kept; the omitted target_handle is inferred."""
top, bottom = stacked_nodes
res = await client.post(
"/api/v1/edges",
json={"source": top, "target": bottom, "type": "ethernet", "source_handle": "cluster-right"},
headers=headers,
)
assert res.status_code == 201
data = res.json()
assert data["source_handle"] == "cluster-right"
assert data["target_handle"] == "top-t"
async def test_create_edge_child_node_abs_y_resolved_through_parent(client: AsyncClient, headers: dict):
"""A child's absolute Y = parent Y + its own pos_y, so a VM low inside a high
container still resolves as downstream of a node above the container."""
parent = (
await client.post(
"/api/v1/nodes",
json={"type": "proxmox", "label": "PVE", "status": "online", "container_mode": True, "pos_x": 0, "pos_y": 0},
headers=headers,
)
).json()
child = (
await client.post(
"/api/v1/nodes",
json={"type": "vm", "label": "VM", "status": "online", "parent_id": parent["id"], "pos_x": 0, "pos_y": 50},
headers=headers,
)
).json()
below = (
await client.post(
"/api/v1/nodes",
json={"type": "server", "label": "Below", "status": "online", "pos_x": 0, "pos_y": 500},
headers=headers,
)
).json()
# child abs Y = 0 + 50 = 50, below at 500 -> child is upstream.
res = await client.post(
"/api/v1/edges", json={"source": child["id"], "target": below["id"], "type": "virtual"}, headers=headers
)
assert res.status_code == 201
data = res.json()
assert data["source_handle"] == "bottom"
assert data["target_handle"] == "top-t"
+238 -1
View File
@@ -2,7 +2,13 @@ from unittest.mock import patch
import pytest
from app.services.fingerprint import fingerprint_ports, match_port, suggest_node_type
from app.services.fingerprint import (
fingerprint_ports,
match_port,
match_service,
suggest_node_type,
suggest_type_from_mac,
)
MOCK_SIGNATURES = [
{"port": 80, "protocol": "tcp", "banner_regex": None, "service_name": "HTTP", "icon": "🌐", "category": "web", "suggested_node_type": "server"},
@@ -118,6 +124,41 @@ def test_suggest_node_type_priority_proxmox_over_server():
assert result == "proxmox"
# A port-agnostic signature: only an HTTP probe can ever match it.
KVM_SIGNATURE = {
"port": None, "protocol": "tcp", "banner_regex": None, "http_regex": "PiKVM",
"service_name": "PiKVM", "icon": "monitorcog", "category": "kvm",
"suggested_node_type": "kvm",
}
def test_suggest_node_type_kvm_from_http_title():
# An IP-KVM answers on 443 like any web app; only the probed title identifies it.
with patch("app.services.fingerprint._load", return_value=[*MOCK_SIGNATURES, KVM_SIGNATURE]):
result = suggest_node_type([
{"port": 443, "protocol": "tcp", "http_signals": {"title": "PiKVM", "headers": {}}},
])
assert result == "kvm"
def test_suggest_node_type_priority_kvm_over_server():
# The same host also answers on 80 (→ server); the KVM signal outranks it.
with patch("app.services.fingerprint._load", return_value=[*MOCK_SIGNATURES, KVM_SIGNATURE]):
result = suggest_node_type([
{"port": 80, "protocol": "tcp"},
{"port": 443, "protocol": "tcp", "http_signals": {"title": "PiKVM", "headers": {}}},
])
assert result == "kvm"
def test_suggest_node_type_ignores_port_agnostic_sig_without_probe():
# No http_signals (probe disabled) → port-agnostic entries can't match and the
# port-only guess stands, unchanged from pre-probe behaviour.
with patch("app.services.fingerprint._load", return_value=[*MOCK_SIGNATURES, KVM_SIGNATURE]):
result = suggest_node_type([{"port": 443, "protocol": "tcp"}])
assert result == "server"
def test_suggest_node_type_camera_from_rtsp_port():
# RTSP port 554 → camera (via _PORT_TYPE_HINTS)
result = suggest_node_type([{"port": 554, "protocol": "tcp"}])
@@ -131,3 +172,199 @@ def test_suggest_node_type_camera_from_signature():
]):
result = suggest_node_type([{"port": 554, "protocol": "tcp"}])
assert result == "camera"
# ── IoT detection ─────────────────────────────────────────────────────────────
def test_suggest_node_type_iot_from_mqtt_port():
result = suggest_node_type([{"port": 1883, "protocol": "tcp"}])
assert result == "iot"
def test_suggest_node_type_iot_from_coap_port():
result = suggest_node_type([{"port": 5683, "protocol": "tcp"}])
assert result == "iot"
def test_suggest_node_type_iot_from_esphome_port():
result = suggest_node_type([{"port": 6052, "protocol": "tcp"}])
assert result == "iot"
def test_suggest_node_type_shelly_mac_overrides_http_port():
# Shelly exposes port 80 (would suggest "server") but MAC identifies it as IoT
result = suggest_node_type([{"port": 80, "protocol": "tcp"}], mac="34:94:54:aa:bb:cc")
assert result == "iot"
def test_suggest_node_type_espressif_mac_returns_iot():
result = suggest_node_type([], mac="a0:20:a6:11:22:33")
assert result == "iot"
def test_suggest_node_type_tuya_mac_returns_iot():
result = suggest_node_type([{"port": 80, "protocol": "tcp"}], mac="d8:f1:5b:aa:bb:cc")
assert result == "iot"
def test_suggest_node_type_iot_wins_over_server_when_mqtt_present():
# MQTT port + HTTP port → iot wins (iot is higher priority than server now)
result = suggest_node_type([
{"port": 80, "protocol": "tcp"},
{"port": 1883, "protocol": "tcp"},
])
assert result == "iot"
# ── OUI vendor detection ──────────────────────────────────────────────────────
def test_suggest_type_from_mac_mikrotik_returns_router():
# The motivating case: MikroTik MAC should be recognized as a router
assert suggest_type_from_mac("4c:5e:0c:11:22:33") == "router"
assert suggest_type_from_mac("b8:69:f4:aa:bb:cc") == "router"
def test_suggest_type_from_mac_ubiquiti_returns_ap():
# Ubiquiti makes routers, switches, APs, cameras — most homelab gear is UniFi APs,
# so OUI defaults to "ap". Port hints can still upgrade to "router" if BGP/VPN open.
assert suggest_type_from_mac("24:a4:3c:11:22:33") == "ap"
assert suggest_type_from_mac("fc:ec:da:aa:bb:cc") == "ap"
def test_suggest_type_from_mac_synology_returns_nas():
assert suggest_type_from_mac("00:11:32:11:22:33") == "nas"
def test_suggest_type_from_mac_qnap_returns_nas():
assert suggest_type_from_mac("24:5e:be:aa:bb:cc") == "nas"
def test_suggest_type_from_mac_hikvision_returns_camera():
assert suggest_type_from_mac("28:57:be:11:22:33") == "camera"
def test_suggest_type_from_mac_dahua_returns_camera():
assert suggest_type_from_mac("3c:ef:8c:aa:bb:cc") == "camera"
def test_suggest_type_from_mac_cisco_returns_switch():
assert suggest_type_from_mac("b8:38:61:11:22:33") == "switch"
def test_suggest_type_from_mac_raspberry_pi_returns_server():
assert suggest_type_from_mac("b8:27:eb:11:22:33") == "server"
def test_suggest_type_from_mac_handles_uppercase():
# MACs may arrive in any case; lookup must be case-insensitive
assert suggest_type_from_mac("4C:5E:0C:11:22:33") == "router"
def test_suggest_type_from_mac_unknown_oui_returns_none():
assert suggest_type_from_mac("00:00:01:11:22:33") is None
def test_suggest_node_type_mikrotik_mac_returns_router_no_ports():
# MikroTik device with no scanned ports should still be classified as router via MAC
assert suggest_node_type([], mac="4c:5e:0c:11:22:33") == "router"
def test_suggest_node_type_synology_mac_with_http_returns_nas():
# NAS priority beats server, so a Synology MAC + open HTTP → nas
result = suggest_node_type(
[{"port": 80, "protocol": "tcp"}],
mac="00:11:32:11:22:33",
)
assert result == "nas"
def test_suggest_node_type_ubiquiti_mac_with_bgp_upgrades_to_router():
# Ubiquiti OUI suggests "ap", but BGP port hint upgrades to "router" (higher priority)
result = suggest_node_type(
[{"port": 179, "protocol": "tcp"}],
mac="24:a4:3c:11:22:33",
)
assert result == "router"
# ── match_service: HTTP probe + port-agnostic ──────────────────────────────────
HTTP_SIGNATURES = [
# Generic web fallback on 8096 (port-only guess)
{"port": 8096, "protocol": "tcp", "banner_regex": None, "http_regex": None,
"service_name": "HTTP", "icon": "🌐", "category": "web", "suggested_node_type": "server"},
# Same port, but confirmed by HTML title → should win when probe confirms
{"port": 8096, "protocol": "tcp", "banner_regex": None, "http_regex": "Jellyfin",
"service_name": "Jellyfin", "icon": "🎬", "category": "media", "suggested_node_type": "server"},
# Port-agnostic: matches on HTTP content regardless of port
{"port": None, "protocol": "tcp", "banner_regex": None, "http_regex": "Portainer",
"service_name": "Portainer", "icon": "🐳", "category": "container", "suggested_node_type": "server"},
# Banner-based entry, no http
{"port": 9090, "protocol": "tcp", "banner_regex": "prometheus", "http_regex": None,
"service_name": "Prometheus", "icon": "🔥", "category": "monitoring", "suggested_node_type": "server"},
]
@pytest.fixture
def http_signatures():
with patch("app.services.fingerprint._load", return_value=HTTP_SIGNATURES):
yield
def test_http_regex_confirmed_beats_port_only(http_signatures):
# Probe ran and title matches → Jellyfin (tier 1) beats generic HTTP (tier 4)
sig = match_service(8096, "tcp", banner=None,
http_signals={"title": "Jellyfin", "headers": {}})
assert sig["service_name"] == "Jellyfin"
def test_http_regex_matches_on_header(http_signatures):
sig = match_service(8096, "tcp", banner=None,
http_signals={"title": None, "headers": {"Server": "Jellyfin"}})
assert sig["service_name"] == "Jellyfin"
def test_http_regex_miss_falls_back_to_port_only(http_signatures):
# Probe ran but nothing matched the http_regex → generic port-only entry wins
sig = match_service(8096, "tcp", banner=None,
http_signals={"title": "Some Other App", "headers": {}})
assert sig["service_name"] == "HTTP"
def test_probe_disabled_ignores_http_regex(http_signatures):
# http_signals=None (deep scan off) → http_regex entry degrades to port-only,
# generic entry (listed first) wins — identical to pre-probe behaviour.
sig = match_service(8096, "tcp", banner=None, http_signals=None)
assert sig["service_name"] == "HTTP"
def test_port_agnostic_match_on_custom_port(http_signatures):
# Portainer found on a non-standard port, recognised purely by HTTP content
sig = match_service(54321, "tcp", banner=None,
http_signals={"title": "Portainer", "headers": {}})
assert sig["service_name"] == "Portainer"
def test_port_agnostic_requires_probe(http_signatures):
# Same custom port, probe off → no signal → no match
assert match_service(54321, "tcp", banner=None, http_signals=None) is None
def test_banner_match_still_works_with_probe(http_signatures):
sig = match_service(9090, "tcp", banner="prometheus 2.x",
http_signals={"title": "x", "headers": {}})
assert sig["service_name"] == "Prometheus"
def test_match_port_alias_has_no_http(http_signatures):
# match_port() is the probe-less alias → http_regex entry degrades to port-only
sig = match_port(8096, "tcp")
assert sig["service_name"] == "HTTP"
def test_fingerprint_ports_uses_http_signals(http_signatures):
results = fingerprint_ports([
{"port": 8096, "protocol": "tcp", "banner": None,
"http_signals": {"title": "Jellyfin", "headers": {}}},
])
assert results[0]["service_name"] == "Jellyfin"
+118
View File
@@ -0,0 +1,118 @@
"""Tests for the HTTP probe used by deep-scan service identification."""
from unittest.mock import AsyncMock, patch
import httpx
import pytest
from app.services.http_probe import (
_extract_title,
probe_open_ports,
probe_port,
)
def _response(text: str = "", headers: dict | None = None, status: int = 200) -> httpx.Response:
return httpx.Response(status_code=status, text=text, headers=headers or {})
# ── _extract_title ──────────────────────────────────────────────────────────
def test_extract_title_basic():
assert _extract_title("<html><title>Jellyfin</title></html>") == "Jellyfin"
def test_extract_title_collapses_whitespace():
assert _extract_title("<title>\n My App\n</title>") == "My App"
def test_extract_title_missing():
assert _extract_title("<html><body>no title</body></html>") is None
def test_extract_title_case_insensitive():
assert _extract_title("<TITLE>Portainer</TITLE>") == "Portainer"
# ── probe_port ──────────────────────────────────────────────────────────────
@pytest.mark.asyncio
async def test_probe_port_reads_title():
with patch("httpx.AsyncClient.get", new=AsyncMock(return_value=_response("<title>Jellyfin</title>"))):
result = await probe_port("10.0.0.5", 8096)
assert result == {"title": "Jellyfin", "headers": {}}
@pytest.mark.asyncio
async def test_probe_port_reads_headers():
resp = _response("", headers={"Server": "nginx", "X-Powered-By": "Express"})
with patch("httpx.AsyncClient.get", new=AsyncMock(return_value=resp)):
result = await probe_port("10.0.0.5", 3000)
assert result["headers"] == {"Server": "nginx", "X-Powered-By": "Express"}
@pytest.mark.asyncio
async def test_probe_port_falls_back_to_http():
# https raises, http succeeds
calls = {"n": 0}
async def fake_get(self, url, **kw):
calls["n"] += 1
if url.startswith("https"):
raise httpx.ConnectError("tls fail")
return _response("<title>HTTP App</title>")
with patch("httpx.AsyncClient.get", new=fake_get):
result = await probe_port("10.0.0.5", 8080)
assert result["title"] == "HTTP App"
assert calls["n"] == 2 # tried https then http
@pytest.mark.asyncio
async def test_probe_port_no_signal_returns_none():
with patch("httpx.AsyncClient.get", new=AsyncMock(return_value=_response(""))):
result = await probe_port("10.0.0.5", 8080)
assert result is None
@pytest.mark.asyncio
async def test_probe_port_timeout_returns_none():
with patch("httpx.AsyncClient.get", new=AsyncMock(side_effect=httpx.TimeoutException("slow"))):
result = await probe_port("10.0.0.5", 8080)
assert result is None
@pytest.mark.asyncio
async def test_probe_port_skips_non_http_ports():
# SSH should never trigger an HTTP request
get = AsyncMock()
with patch("httpx.AsyncClient.get", new=get):
result = await probe_port("10.0.0.5", 22)
assert result is None
get.assert_not_called()
@pytest.mark.asyncio
async def test_probe_port_verify_tls_flag_passed():
with patch("app.services.http_probe.httpx.AsyncClient") as client_cls:
instance = client_cls.return_value.__aenter__.return_value
instance.get = AsyncMock(return_value=_response("<title>X</title>"))
await probe_port("10.0.0.5", 8443, verify_tls=True)
assert client_cls.call_args.kwargs["verify"] is True
# ── probe_open_ports ─────────────────────────────────────────────────────────
@pytest.mark.asyncio
async def test_probe_open_ports_enriches_each_port():
async def fake_get(self, url, **kw):
if ":8096" in url:
return _response("<title>Jellyfin</title>")
return _response("")
ports = [{"port": 8096, "protocol": "tcp"}, {"port": 9999, "protocol": "tcp"}]
with patch("httpx.AsyncClient.get", new=fake_get):
result = await probe_open_ports("10.0.0.5", ports)
by_port = {p["port"]: p for p in result}
assert by_port[8096]["http_signals"]["title"] == "Jellyfin"
assert by_port[9999]["http_signals"] is None
File diff suppressed because it is too large Load Diff
+99 -2
View File
@@ -82,10 +82,9 @@ async def test_liveview_does_not_require_jwt(client: AsyncClient):
@pytest.mark.asyncio
async def test_liveview_returns_saved_canvas(client: AsyncClient, auth_headers):
async def test_liveview_returns_saved_canvas(client: AsyncClient, headers):
"""Canvas saved via POST /canvas/save appears in liveview response."""
settings.liveview_key = "test-key"
headers = await auth_headers()
# Save a canvas with one node
payload = {
@@ -112,6 +111,27 @@ async def test_liveview_returns_saved_canvas(client: AsyncClient, auth_headers):
assert nodes[0]["label"] == "Live Node"
# ── custom_style + theme propagation ─────────────────────────────────────────
@pytest.mark.asyncio
async def test_liveview_returns_custom_style_and_theme(client: AsyncClient, headers):
"""custom_style and viewport.theme_id from a saved canvas surface in liveview."""
settings.liveview_key = "test-key"
payload = {
"nodes": [],
"edges": [],
"viewport": {"x": 0, "y": 0, "zoom": 1, "theme_id": "matrix"},
"custom_style": {"fontFamily": "Inter", "nodeRadius": 12},
}
await client.post("/api/v1/canvas/save", json=payload, headers=headers)
res = await client.get("/api/v1/liveview?key=test-key")
assert res.status_code == 200
body = res.json()
assert body["viewport"].get("theme_id") == "matrix"
assert body["custom_style"] == {"fontFamily": "Inter", "nodeRadius": 12}
# ── Re-disable after enabling ─────────────────────────────────────────────────
@pytest.mark.asyncio
@@ -124,3 +144,80 @@ async def test_liveview_disabled_after_key_cleared(client: AsyncClient):
res = await client.get("/api/v1/liveview?key=was-enabled")
assert res.status_code == 403
assert res.json()["detail"] == "Live view is disabled"
# ── /config (authenticated) — key used to build share links ──────────────────
@pytest.mark.asyncio
async def test_liveview_config_requires_auth(client: AsyncClient):
"""The config endpoint exposes the key, so it must reject unauthenticated calls."""
settings.liveview_key = "secret"
res = await client.get("/api/v1/liveview/config")
assert res.status_code == 401
@pytest.mark.asyncio
async def test_liveview_config_returns_key_when_enabled(client: AsyncClient, headers):
settings.liveview_key = "share-me"
res = await client.get("/api/v1/liveview/config", headers=headers)
assert res.status_code == 200
body = res.json()
assert body == {"enabled": True, "key": "share-me"}
@pytest.mark.asyncio
async def test_liveview_config_disabled_hides_key(client: AsyncClient, headers):
settings.liveview_key = None
res = await client.get("/api/v1/liveview/config", headers=headers)
assert res.status_code == 200
assert res.json() == {"enabled": False, "key": None}
@pytest.mark.asyncio
async def test_liveview_config_empty_key_disabled(client: AsyncClient, headers):
settings.liveview_key = ""
res = await client.get("/api/v1/liveview/config", headers=headers)
assert res.status_code == 200
assert res.json() == {"enabled": False, "key": None}
# ── design_id selects which canvas is rendered ───────────────────────────────
@pytest.mark.asyncio
async def test_liveview_design_id_selects_canvas(client: AsyncClient, headers):
"""?design_id=<id> renders that design's canvas, not the first one."""
settings.liveview_key = "test-key"
# Create two designs
d1 = (await client.post("/api/v1/designs", json={"name": "Network"}, headers=headers)).json()
d2 = (await client.post("/api/v1/designs", json={"name": "Electrical"}, headers=headers)).json()
# Save a distinct node into each design
for design, node_id, label in ((d1, "n-net", "Net Node"), (d2, "n-elec", "Elec Node")):
payload = {
"nodes": [{
"id": node_id,
"type": "server",
"label": label,
"status": "online",
"services": [],
"pos_x": 0,
"pos_y": 0,
}],
"edges": [],
"viewport": {"x": 0, "y": 0, "zoom": 1},
"design_id": design["id"],
}
await client.post("/api/v1/canvas/save", json=payload, headers=headers)
# Requesting d2 returns only the electrical node
res = await client.get(f"/api/v1/liveview?key=test-key&design_id={d2['id']}")
assert res.status_code == 200
nodes = res.json()["nodes"]
assert [n["id"] for n in nodes] == ["n-elec"]
# Requesting d1 returns only the network node
res = await client.get(f"/api/v1/liveview?key=test-key&design_id={d1['id']}")
assert res.status_code == 200
nodes = res.json()["nodes"]
assert [n["id"] for n in nodes] == ["n-net"]
+27
View File
@@ -0,0 +1,27 @@
"""Unit tests for MAC normalization (the cross-source dedup key)."""
from __future__ import annotations
from app.services.discovery_sources import add_source
from app.services.mac_utils import normalize_mac
def test_normalize_mac_lowercases_and_unifies_separators() -> None:
assert normalize_mac("BC:24:11:AA:BB:CC") == "bc:24:11:aa:bb:cc"
assert normalize_mac("bc-24-11-aa-bb-cc") == "bc:24:11:aa:bb:cc"
assert normalize_mac(" BC:24:11:AA:BB:CC ") == "bc:24:11:aa:bb:cc"
def test_normalize_mac_blank_is_none() -> None:
assert normalize_mac(None) is None
assert normalize_mac("") is None
assert normalize_mac(" ") is None
def test_add_source_unions_without_duplicates() -> None:
assert add_source(None, "arp") == ["arp"]
assert add_source(["arp"], "proxmox") == ["arp", "proxmox"]
assert add_source(["arp", "proxmox"], "proxmox") == ["arp", "proxmox"]
assert add_source(["arp"], None) == ["arp"]
# Drops falsy members already present.
assert add_source(["arp", ""], "proxmox") == ["arp", "proxmox"]
+94
View File
@@ -0,0 +1,94 @@
import re
import pytest
from app.api.routes import media
from app.core.config import settings
PNG_BYTES = b"\x89PNG\r\n\x1a\n" + b"0" * 32
@pytest.fixture
def media_dir(tmp_path, monkeypatch):
"""Point uploads at a temp folder for the duration of a test."""
monkeypatch.setattr(settings, "upload_dir", str(tmp_path))
return tmp_path
async def _upload(client, headers, name="plan.png", data=PNG_BYTES, content_type="image/png"):
return await client.post(
"/api/v1/media/upload",
files={"file": (name, data, content_type)},
headers=headers,
)
@pytest.mark.asyncio
async def test_upload_requires_auth(client, media_dir):
res = await _upload(client, headers={})
assert res.status_code == 401
@pytest.mark.asyncio
async def test_upload_stores_file_and_returns_url(client, headers, media_dir):
res = await _upload(client, headers)
assert res.status_code == 200
body = res.json()
assert re.fullmatch(r"/api/v1/media/[0-9a-f]{32}\.png", body["url"])
# File written to disk with the server-generated name (client name ignored).
assert (media_dir / body["filename"]).read_bytes() == PNG_BYTES
assert body["filename"] != "plan.png"
@pytest.mark.asyncio
async def test_upload_rejects_unsupported_type(client, headers, media_dir):
res = await _upload(client, headers, name="a.txt", data=b"hello", content_type="text/plain")
assert res.status_code == 415
@pytest.mark.asyncio
async def test_upload_rejects_content_type_magic_mismatch(client, headers, media_dir):
# Claims PNG but bytes are not a PNG.
res = await _upload(client, headers, data=b"not-a-real-png", content_type="image/png")
assert res.status_code == 415
@pytest.mark.asyncio
async def test_upload_rejects_oversize(client, headers, media_dir, monkeypatch):
monkeypatch.setattr(media, "MAX_BYTES", 8)
res = await _upload(client, headers, data=PNG_BYTES) # > 8 bytes
assert res.status_code == 413
@pytest.mark.asyncio
async def test_get_serves_uploaded_file(client, headers, media_dir):
up = await _upload(client, headers)
res = await client.get(up.json()["url"]) # public, no auth
assert res.status_code == 200
assert res.content == PNG_BYTES
@pytest.mark.asyncio
async def test_get_rejects_bad_filename(client, media_dir):
res = await client.get("/api/v1/media/..%2f..%2fetc%2fpasswd")
assert res.status_code == 404
@pytest.mark.asyncio
async def test_delete_rejects_bad_filename(client, headers, media_dir):
res = await client.delete("/api/v1/media/..%2f..%2fetc%2fpasswd", headers=headers)
assert res.status_code == 404
@pytest.mark.asyncio
async def test_delete_requires_auth_and_removes_file(client, headers, media_dir):
up = await _upload(client, headers)
filename = up.json()["filename"]
assert (await client.delete(f"/api/v1/media/{filename}")).status_code == 401
assert (media_dir / filename).exists()
res = await client.delete(f"/api/v1/media/{filename}", headers=headers)
assert res.status_code == 204
assert not (media_dir / filename).exists()
assert (await client.get(up.json()["url"])).status_code == 404
+183
View File
@@ -0,0 +1,183 @@
"""Unit tests for the shared MQTT helpers in mqtt_common."""
from __future__ import annotations
import json
import ssl
from unittest.mock import patch
import pytest
from app.services.mqtt_common import (
_build_tls_context,
_sanitize_mqtt_error,
request_response,
)
from app.services.mqtt_common import test_connection as _test_connection
# ---------------------------------------------------------------------------
# _sanitize_mqtt_error — never leak credentials
# ---------------------------------------------------------------------------
def test_sanitize_auth_error() -> None:
msg = _sanitize_mqtt_error(Exception("Not authorized: bad username for user=admin pwd=secret"))
assert msg == "Authentication failed"
assert "secret" not in msg
def test_sanitize_refused() -> None:
assert _sanitize_mqtt_error(Exception("Connection refused")) == "Connection refused by broker"
def test_sanitize_dns() -> None:
msg = _sanitize_mqtt_error(Exception("nodename nor servname provided: broker.lan"))
assert msg == "Broker hostname could not be resolved"
assert "broker.lan" not in msg
def test_sanitize_tls() -> None:
assert _sanitize_mqtt_error(
Exception("[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed")
) == "TLS handshake failed"
def test_sanitize_timeout() -> None:
assert _sanitize_mqtt_error(Exception("operation timed out")) == "Connection to broker timed out"
def test_sanitize_unknown_falls_back() -> None:
msg = _sanitize_mqtt_error(Exception("mqtt://admin:hunter2@broker weird"))
assert msg == "MQTT connection failed"
assert "hunter2" not in msg
# ---------------------------------------------------------------------------
# _build_tls_context
# ---------------------------------------------------------------------------
def test_tls_secure_verifies() -> None:
ctx = _build_tls_context(insecure=False)
assert ctx.check_hostname is True
assert ctx.verify_mode == ssl.CERT_REQUIRED
def test_tls_insecure_disables_verification() -> None:
ctx = _build_tls_context(insecure=True)
assert ctx.check_hostname is False
assert ctx.verify_mode == ssl.CERT_NONE
# ---------------------------------------------------------------------------
# request_response (mocked aiomqtt)
# ---------------------------------------------------------------------------
_SAMPLE = {"success": True, "result": []}
def _fake_client_factory(topic: str, payload: dict):
class _FakeMessage:
_yielded = False
def __init__(self) -> None:
self.topic = topic
self.payload = json.dumps(payload).encode()
def __aiter__(self):
return self
async def __anext__(self):
if self._yielded:
raise StopAsyncIteration
self._yielded = True
return self
class _FakeClient:
async def __aenter__(self):
return self
async def __aexit__(self, *_):
pass
async def subscribe(self, _topic: str) -> None:
pass
async def publish(self, _topic: str, _payload: str) -> None:
pass
@property
def messages(self):
return _FakeMessage()
return _FakeClient
@pytest.mark.asyncio
async def test_request_response_success() -> None:
topic = "zwave/_CLIENTS/ZWAVE_GATEWAY-zwavejs2mqtt/api/getNodes"
with patch("app.services.mqtt_common.aiomqtt") as mock_aiomqtt:
mock_aiomqtt.Client.return_value = _fake_client_factory(topic, _SAMPLE)()
mock_aiomqtt.MqttError = Exception
out = await request_response(
"localhost", 1883, "req/topic", topic, {"args": []}
)
assert out == _SAMPLE
@pytest.mark.asyncio
async def test_request_response_connection_error() -> None:
class _FakeClient:
async def __aenter__(self):
raise Exception("Connection refused")
async def __aexit__(self, *_):
pass
with patch("app.services.mqtt_common.aiomqtt") as mock_aiomqtt:
mock_aiomqtt.Client.return_value = _FakeClient()
mock_aiomqtt.MqttError = Exception
with pytest.raises(ConnectionError):
await request_response("bad", 1883, "req", "resp", {})
@pytest.mark.asyncio
async def test_request_response_passes_tls_context() -> None:
topic = "resp"
with patch("app.services.mqtt_common.aiomqtt") as mock_aiomqtt:
mock_aiomqtt.Client.return_value = _fake_client_factory(topic, _SAMPLE)()
mock_aiomqtt.MqttError = Exception
await request_response("h", 8883, "req", topic, {}, tls=True, tls_insecure=True)
ctx = mock_aiomqtt.Client.call_args.kwargs["tls_context"]
assert ctx.verify_mode == ssl.CERT_NONE
@pytest.mark.asyncio
async def test_test_connection_success() -> None:
class _FakeClient:
async def __aenter__(self):
return self
async def __aexit__(self, *_):
pass
with patch("app.services.mqtt_common.aiomqtt") as mock_aiomqtt:
mock_aiomqtt.Client.return_value = _FakeClient()
mock_aiomqtt.MqttError = Exception
assert await _test_connection("localhost", 1883) is True
@pytest.mark.asyncio
async def test_test_connection_failure() -> None:
class _FakeClient:
async def __aenter__(self):
raise Exception("refused")
async def __aexit__(self, *_):
pass
with patch("app.services.mqtt_common.aiomqtt") as mock_aiomqtt:
mock_aiomqtt.Client.return_value = _FakeClient()
mock_aiomqtt.MqttError = Exception
with pytest.raises(ConnectionError):
await _test_connection("bad", 1883)
+144
View File
@@ -0,0 +1,144 @@
"""Tests for the same-canvas node dedupe repair (app.services.node_dedupe).
Identity is the device link now: two nodes are duplicates when they draw the
same `device_inventory` row on the same design. The device facts live on that
row, so nothing has to be merged out of the extras before they go — only edges
and parent links are re-pointed.
"""
import pytest
from sqlalchemy import select
from app.db.models import Design, Edge, InventoryDevice, Node
from app.services.node_dedupe import dedupe_nodes_by_device
async def _design(db, name="d1"):
d = Design(name=name)
db.add(d)
await db.flush()
return d
async def _device(db, ieee="0xAAA", **kwargs):
device = InventoryDevice(ieee_address=ieee, status="approved", **kwargs)
db.add(device)
await db.flush()
return device
@pytest.mark.asyncio
async def test_collapses_same_device_same_design(db_session):
d = await _design(db_session)
device = await _device(db_session)
keep = Node(
label="Sensor", type="zigbee_enddevice", design_id=d.id,
device_id=device.id, pos_x=100, pos_y=200,
)
db_session.add(keep)
await db_session.flush()
dup = Node(label="Sensor", type="zigbee_enddevice", design_id=d.id, device_id=device.id)
db_session.add(dup)
await db_session.flush()
removed = await dedupe_nodes_by_device(db_session)
assert removed == 1
nodes = (
await db_session.execute(select(Node).where(Node.device_id == device.id))
).scalars().all()
assert len(nodes) == 1
survivor = nodes[0]
assert survivor.id == keep.id # oldest kept
assert survivor.pos_x == 100 # canvas position preserved
# The device facts were never on the node, so none can be lost here.
assert await db_session.get(InventoryDevice, device.id) is not None
@pytest.mark.asyncio
async def test_preserves_same_device_across_designs(db_session):
"""Same device on two canvases is valid — must NOT be merged."""
d1 = await _design(db_session, "d1")
d2 = await _design(db_session, "d2")
device = await _device(db_session, ieee="0xBBB")
for d in (d1, d2):
db_session.add(Node(label="S", type="zigbee_enddevice", design_id=d.id, device_id=device.id))
await db_session.flush()
removed = await dedupe_nodes_by_device(db_session)
assert removed == 0
nodes = (
await db_session.execute(select(Node).where(Node.device_id == device.id))
).scalars().all()
assert len(nodes) == 2
@pytest.mark.asyncio
async def test_leaves_furniture_alone(db_session):
"""Zones and text carry no device link, so they are never duplicates."""
d = await _design(db_session)
for _ in range(2):
db_session.add(Node(label="Zone", type="groupRect", design_id=d.id))
await db_session.flush()
assert await dedupe_nodes_by_device(db_session) == 0
@pytest.mark.asyncio
async def test_repoints_edges_and_drops_dupes(db_session):
d = await _design(db_session)
device = await _device(db_session, ieee="0xCCC")
keep = Node(label="A", type="server", design_id=d.id, device_id=device.id)
other = Node(label="B", type="server", design_id=d.id)
db_session.add_all([keep, other])
await db_session.flush()
dup = Node(label="A", type="server", design_id=d.id, device_id=device.id)
db_session.add(dup)
await db_session.flush()
# keep<->other and dup<->other (parallel after repoint), plus dup<->keep (self-loop).
db_session.add_all([
Edge(source=keep.id, target=other.id, type="ethernet", design_id=d.id),
Edge(source=dup.id, target=other.id, type="ethernet", design_id=d.id),
Edge(source=dup.id, target=keep.id, type="ethernet", design_id=d.id),
])
await db_session.flush()
removed = await dedupe_nodes_by_device(db_session)
assert removed == 1
edges = (await db_session.execute(select(Edge))).scalars().all()
# self-loop dropped, parallel edge collapsed -> a single keep<->other edge
assert len(edges) == 1
e = edges[0]
assert {e.source, e.target} == {keep.id, other.id}
@pytest.mark.asyncio
async def test_repoints_child_parent(db_session):
d = await _design(db_session)
device = await _device(db_session, ieee="0xDDD")
keep = Node(label="Host", type="proxmox", design_id=d.id, device_id=device.id)
db_session.add(keep)
await db_session.flush()
dup = Node(label="Host", type="proxmox", design_id=d.id, device_id=device.id)
db_session.add(dup)
await db_session.flush()
child = Node(label="VM", type="vm", design_id=d.id, parent_id=dup.id)
db_session.add(child)
await db_session.flush()
await dedupe_nodes_by_device(db_session)
await db_session.refresh(child)
assert child.parent_id == keep.id
@pytest.mark.asyncio
async def test_idempotent_noop_when_unique(db_session):
d = await _design(db_session)
device = await _device(db_session, ieee="0xEEE")
db_session.add(Node(label="X", type="server", design_id=d.id, device_id=device.id))
await db_session.flush()
assert await dedupe_nodes_by_device(db_session) == 0
assert await dedupe_nodes_by_device(db_session) == 0
+296 -8
View File
@@ -1,14 +1,6 @@
import pytest
from httpx import AsyncClient
@pytest.fixture
async def headers(client: AsyncClient):
res = await client.post("/api/v1/auth/login", json={"username": "admin", "password": "admin"})
token = res.json()["access_token"]
return {"Authorization": f"Bearer {token}"}
async def test_list_nodes_empty(client: AsyncClient, headers: dict):
res = await client.get("/api/v1/nodes", headers=headers)
assert res.status_code == 200
@@ -67,6 +59,91 @@ async def test_update_node_not_found(client: AsyncClient, headers: dict):
assert res.status_code == 404
async def test_create_node_without_design_id_falls_back_to_first_design(client: AsyncClient, headers: dict):
# Regression for #225: MCP create_node sent no design_id, so nodes were
# persisted with design_id=null and never rendered on the canvas until a
# container restart reconciled them. They must attach to a design on create.
design = await client.post("/api/v1/designs", json={"name": "Primary"}, headers=headers)
design_id = design.json()["id"]
res = await client.post(
"/api/v1/nodes",
json={"type": "generic", "label": "mcp-node", "ip": "192.168.18.99"},
headers=headers,
)
assert res.status_code == 201
assert res.json()["design_id"] == design_id
async def test_create_node_respects_explicit_design_id(client: AsyncClient, headers: dict):
# When a design_id is supplied it must win over the first-design fallback.
first = await client.post("/api/v1/designs", json={"name": "First"}, headers=headers)
second = await client.post("/api/v1/designs", json={"name": "Second"}, headers=headers)
second_id = second.json()["id"]
assert first.json()["id"] != second_id
res = await client.post(
"/api/v1/nodes",
json={"type": "generic", "label": "n", "design_id": second_id},
headers=headers,
)
assert res.status_code == 201
assert res.json()["design_id"] == second_id
async def test_create_node_rejects_duplicate_ip_on_same_design(client: AsyncClient, headers: dict):
# A second node with the same ip on the same design is a silent duplicate —
# scripts/MCP clients get 409 with the existing node id instead. (#260)
design = await client.post("/api/v1/designs", json={"name": "D"}, headers=headers)
design_id = design.json()["id"]
first = await client.post(
"/api/v1/nodes",
json={"type": "server", "label": "srv", "ip": "192.168.1.5", "design_id": design_id},
headers=headers,
)
assert first.status_code == 201
existing_id = first.json()["id"]
dup = await client.post(
"/api/v1/nodes",
json={"type": "server", "label": "srv-again", "ip": "192.168.1.5", "design_id": design_id},
headers=headers,
)
assert dup.status_code == 409
detail = dup.json()["detail"]
assert detail["duplicate"] is True
assert detail["existing_node_id"] == existing_id
assert detail["match"] == "ip"
async def test_create_node_force_bypasses_duplicate_guard(client: AsyncClient, headers: dict):
design = await client.post("/api/v1/designs", json={"name": "D"}, headers=headers)
design_id = design.json()["id"]
await client.post(
"/api/v1/nodes",
json={"type": "server", "label": "srv", "ip": "192.168.1.5", "design_id": design_id},
headers=headers,
)
forced = await client.post(
"/api/v1/nodes",
json={"type": "server", "label": "srv", "ip": "192.168.1.5", "design_id": design_id, "force": True},
headers=headers,
)
assert forced.status_code == 201
async def test_create_node_without_any_design_stays_null(client: AsyncClient, headers: dict):
# No designs exist yet: fallback can't invent one, so design_id stays null
# rather than erroring.
res = await client.post(
"/api/v1/nodes",
json={"type": "generic", "label": "orphan"},
headers=headers,
)
assert res.status_code == 201
assert res.json()["design_id"] is None
async def test_delete_node_not_found(client: AsyncClient, headers: dict):
res = await client.delete("/api/v1/nodes/nonexistent", headers=headers)
assert res.status_code == 404
@@ -115,3 +192,214 @@ async def test_update_node_parent_id(client: AsyncClient, headers: dict):
async def test_create_node_requires_auth(client: AsyncClient):
res = await client.post("/api/v1/nodes", json={"type": "server", "label": "N", "status": "unknown"})
assert res.status_code == 401
# --- Properties tests ---
async def test_create_node_default_properties_empty(client: AsyncClient, headers: dict):
"""New node has an empty properties list by default."""
res = await client.post("/api/v1/nodes", json={"type": "server", "label": "Srv", "status": "unknown"}, headers=headers)
assert res.status_code == 201
assert res.json()["properties"] == []
async def test_create_node_with_properties(client: AsyncClient, headers: dict):
"""Node created with properties round-trips correctly."""
props = [
{"key": "CPU Model", "value": "i7-12700K", "icon": "Cpu", "visible": True},
{"key": "RAM", "value": "32 GB", "icon": "MemoryStick", "visible": False},
]
res = await client.post(
"/api/v1/nodes",
json={"type": "server", "label": "Srv", "status": "unknown", "properties": props},
headers=headers,
)
assert res.status_code == 201
assert res.json()["properties"] == props
async def test_patch_node_properties(client: AsyncClient, headers: dict):
"""PATCH with properties replaces the full properties array."""
create = await client.post("/api/v1/nodes", json={"type": "server", "label": "Srv", "status": "unknown"}, headers=headers)
node_id = create.json()["id"]
props = [{"key": "Disk", "value": "2 TB", "icon": "HardDrive", "visible": True}]
res = await client.patch(f"/api/v1/nodes/{node_id}", json={"properties": props}, headers=headers)
assert res.status_code == 200
assert res.json()["properties"] == props
async def test_patch_node_without_properties_does_not_wipe(client: AsyncClient, headers: dict):
"""PATCH that omits properties leaves existing properties untouched."""
props = [{"key": "GPU", "value": "RTX 4090", "icon": "Monitor", "visible": True}]
create = await client.post(
"/api/v1/nodes",
json={"type": "server", "label": "Srv", "status": "unknown", "properties": props},
headers=headers,
)
node_id = create.json()["id"]
# PATCH only the label — properties must survive
res = await client.patch(f"/api/v1/nodes/{node_id}", json={"label": "Updated"}, headers=headers)
assert res.status_code == 200
assert res.json()["properties"] == props
assert res.json()["label"] == "Updated"
async def test_patch_node_clears_properties_with_empty_array(client: AsyncClient, headers: dict):
"""PATCH with properties=[] explicitly clears all properties."""
props = [{"key": "CPU Model", "value": "i5", "icon": "Cpu", "visible": True}]
create = await client.post(
"/api/v1/nodes",
json={"type": "server", "label": "Srv", "status": "unknown", "properties": props},
headers=headers,
)
node_id = create.json()["id"]
res = await client.patch(f"/api/v1/nodes/{node_id}", json={"properties": []}, headers=headers)
assert res.status_code == 200
assert res.json()["properties"] == []
async def test_get_node_returns_properties(client: AsyncClient, headers: dict):
"""GET /nodes/:id returns the properties field."""
props = [{"key": "OS", "value": "Debian 12", "icon": "Server", "visible": True}]
create = await client.post(
"/api/v1/nodes",
json={"type": "server", "label": "Srv", "status": "unknown", "properties": props},
headers=headers,
)
node_id = create.json()["id"]
res = await client.get(f"/api/v1/nodes/{node_id}", headers=headers)
assert res.status_code == 200
assert res.json()["properties"] == props
async def test_properties_icon_can_be_null(client: AsyncClient, headers: dict):
"""A property with icon=null is valid and round-trips correctly."""
props = [{"key": "Notes", "value": "custom value", "icon": None, "visible": False}]
create = await client.post(
"/api/v1/nodes",
json={"type": "generic", "label": "G", "status": "unknown", "properties": props},
headers=headers,
)
assert create.status_code == 201
assert create.json()["properties"] == props
# ---------------------------------------------------------------------------
# Auto-positioning (issue #265): omitting pos_x/pos_y snaps the node to the
# first free 200x100 grid slot instead of stacking everything at (0, 0).
# ---------------------------------------------------------------------------
async def test_create_node_auto_positions_first_at_origin(client: AsyncClient, headers: dict):
"""First root node with no coordinates lands at (0, 0)."""
res = await client.post(
"/api/v1/nodes", json={"type": "server", "label": "A", "status": "unknown"}, headers=headers
)
assert res.status_code == 201
data = res.json()
assert (data["pos_x"], data["pos_y"]) == (0.0, 0.0)
async def test_create_node_auto_position_avoids_collision(client: AsyncClient, headers: dict):
"""A second auto-placed root node takes the next free grid cell, not (0, 0)."""
first = await client.post(
"/api/v1/nodes", json={"type": "server", "label": "A", "status": "unknown"}, headers=headers
)
assert (first.json()["pos_x"], first.json()["pos_y"]) == (0.0, 0.0)
second = await client.post(
"/api/v1/nodes", json={"type": "server", "label": "B", "status": "unknown"}, headers=headers
)
assert second.status_code == 201
# Next free cell in row 0 is column 1 -> x = 1 * 200.
assert (second.json()["pos_x"], second.json()["pos_y"]) == (200.0, 0.0)
async def test_create_node_child_defaults_to_parent_origin(client: AsyncClient, headers: dict):
"""A child node (parent_id set) with no coordinates defaults to (0, 0) relative to its parent."""
parent = (
await client.post(
"/api/v1/nodes",
json={"type": "proxmox", "label": "PVE", "status": "online", "container_mode": True},
headers=headers,
)
).json()
res = await client.post(
"/api/v1/nodes",
json={"type": "vm", "label": "VM1", "status": "online", "parent_id": parent["id"]},
headers=headers,
)
assert res.status_code == 201
data = res.json()
assert (data["pos_x"], data["pos_y"]) == (0.0, 0.0)
async def test_create_node_explicit_position_preserved(client: AsyncClient, headers: dict):
"""Explicit coordinates are honored, never auto-placed."""
res = await client.post(
"/api/v1/nodes",
json={"type": "server", "label": "A", "status": "unknown", "pos_x": 512.0, "pos_y": 384.0},
headers=headers,
)
assert res.status_code == 201
data = res.json()
assert (data["pos_x"], data["pos_y"]) == (512.0, 384.0)
async def test_create_node_explicit_zero_position_preserved(client: AsyncClient, headers: dict):
"""pos=0 is an explicit value, not 'omitted' — auto-position must not treat 0 as None."""
await client.post(
"/api/v1/nodes", json={"type": "server", "label": "A", "status": "unknown"}, headers=headers
)
# Second node explicitly pinned to (0, 0) even though the cell is taken.
res = await client.post(
"/api/v1/nodes",
json={"type": "server", "label": "B", "status": "unknown", "pos_x": 0, "pos_y": 0},
headers=headers,
)
assert res.status_code == 201
assert (res.json()["pos_x"], res.json()["pos_y"]) == (0.0, 0.0)
async def test_list_nodes_filters_by_label_exact(client: AsyncClient, headers: dict):
await client.post("/api/v1/nodes", json={"type": "server", "label": "Proxmox", "status": "unknown"}, headers=headers)
await client.post("/api/v1/nodes", json={"type": "router", "label": "Router", "status": "unknown"}, headers=headers)
res = await client.get("/api/v1/nodes?label=Proxmox", headers=headers)
assert res.status_code == 200
assert [n["label"] for n in res.json()] == ["Proxmox"]
async def test_list_nodes_filters_by_label_substring(client: AsyncClient, headers: dict):
await client.post("/api/v1/nodes", json={"type": "server", "label": "pve1-node", "status": "unknown"}, headers=headers)
await client.post("/api/v1/nodes", json={"type": "server", "label": "pve2-node", "status": "unknown"}, headers=headers)
await client.post("/api/v1/nodes", json={"type": "router", "label": "Router", "status": "unknown"}, headers=headers)
res = await client.get("/api/v1/nodes?label=pve", headers=headers)
assert res.status_code == 200
assert sorted(n["label"] for n in res.json()) == ["pve1-node", "pve2-node"]
async def test_list_nodes_filters_by_label_case_insensitive(client: AsyncClient, headers: dict):
await client.post("/api/v1/nodes", json={"type": "server", "label": "Proxmox", "status": "unknown"}, headers=headers)
res = await client.get("/api/v1/nodes?label=PROXMOX", headers=headers)
assert res.status_code == 200
assert [n["label"] for n in res.json()] == ["Proxmox"]
async def test_list_nodes_filters_by_label_no_match(client: AsyncClient, headers: dict):
await client.post("/api/v1/nodes", json={"type": "server", "label": "Proxmox", "status": "unknown"}, headers=headers)
res = await client.get("/api/v1/nodes?label=nonexistent", headers=headers)
assert res.status_code == 200
assert res.json() == []
async def test_list_nodes_without_label_param_returns_all(client: AsyncClient, headers: dict):
"""Omitting label is fully backward compatible with the old unfiltered behavior."""
for label in ("A", "B", "C"):
await client.post("/api/v1/nodes", json={"type": "generic", "label": label, "status": "unknown"}, headers=headers)
res = await client.get("/api/v1/nodes", headers=headers)
assert res.status_code == 200
assert len(res.json()) == 3
+511
View File
@@ -0,0 +1,511 @@
"""API + persistence tests for /api/v1/proxmox/*."""
from __future__ import annotations
import uuid
from types import SimpleNamespace
from unittest.mock import AsyncMock, MagicMock, patch
import pytest
from httpx import AsyncClient
from sqlalchemy import select
from app.api.routes.proxmox import (
_background_proxmox_import,
_guest_visibility_advisory,
_persist_pending_import,
)
from app.api.routes.scan import _is_proxmox_cluster_member, _resolve_pending_links_for_ieee
from app.core.config import settings
from app.db.models import Design, Edge, InventoryDevice, InventoryDeviceLink, Node
@pytest.fixture(autouse=True)
def _clear_env_token():
"""Ensure a clean token state per test; restore afterwards."""
tid, sec = settings.proxmox_token_id, settings.proxmox_token_secret
settings.proxmox_token_id = ""
settings.proxmox_token_secret = ""
yield
settings.proxmox_token_id, settings.proxmox_token_secret = tid, sec
def _host_node() -> dict:
return {
"id": "pve-node-pve1", "label": "pve1", "type": "proxmox",
"ieee_address": "pve-node-pve1", "hostname": "pve1", "ip": None,
"status": "online", "cpu_count": 8, "ram_gb": 16.0, "disk_gb": 500.0,
"vendor": "Proxmox VE", "model": None, "parent_ieee": None,
}
def _guest_node(vmid: int, ip: str | None, status: str = "online", mac: str | None = None) -> dict:
return {
"id": f"pve-pve1-{vmid}", "label": f"vm{vmid}", "type": "vm",
"ieee_address": f"pve-pve1-{vmid}", "hostname": f"vm{vmid}", "ip": ip,
"mac": mac,
"status": status, "cpu_count": 2, "ram_gb": 4.0, "disk_gb": 32.0,
"vendor": "Proxmox VE", "model": "QEMU", "vmid": vmid,
"parent_ieee": "pve-node-pve1",
}
# --- endpoints -------------------------------------------------------------
@pytest.mark.asyncio
async def test_test_connection_uses_body_token(client: AsyncClient, headers: dict) -> None:
with patch("app.api.routes.proxmox.test_proxmox_connection", new=AsyncMock(return_value=(True, "ok"))):
res = await client.post(
"/api/v1/proxmox/test-connection",
json={"host": "pve", "port": 8006, "token_id": "u@pam!t", "token_secret": "s"},
headers=headers,
)
assert res.status_code == 200
assert res.json()["connected"] is True
@pytest.mark.asyncio
async def test_missing_token_is_rejected(client: AsyncClient, headers: dict) -> None:
res = await client.post(
"/api/v1/proxmox/test-connection",
json={"host": "pve", "port": 8006},
headers=headers,
)
assert res.status_code == 400
@pytest.mark.asyncio
async def test_import_pending_creates_scan_run(client: AsyncClient, headers: dict) -> None:
with patch("app.api.routes.proxmox._background_proxmox_import", new_callable=AsyncMock):
res = await client.post(
"/api/v1/proxmox/import-pending",
json={"host": "pve", "port": 8006, "token_id": "u@pam!t", "token_secret": "s"},
headers=headers,
)
assert res.status_code == 200
data = res.json()
assert data["kind"] == "proxmox"
assert data["status"] == "running"
@pytest.mark.asyncio
async def test_sync_now_creates_scan_run(client: AsyncClient, headers: dict) -> None:
settings.proxmox_host = "pve"
settings.proxmox_token_id = "u@pam!t"
settings.proxmox_token_secret = "s"
with patch("app.api.routes.proxmox._background_proxmox_import", new_callable=AsyncMock):
res = await client.post("/api/v1/proxmox/sync-now", headers=headers)
assert res.status_code == 200
data = res.json()
assert data["kind"] == "proxmox"
assert data["status"] == "running"
@pytest.mark.asyncio
async def test_sync_now_rejected_without_token(client: AsyncClient, headers: dict) -> None:
settings.proxmox_host = "pve"
# _clear_env_token leaves token empty
res = await client.post("/api/v1/proxmox/sync-now", headers=headers)
assert res.status_code == 400
@pytest.mark.asyncio
async def test_sync_now_requires_auth(client: AsyncClient) -> None:
res = await client.post("/api/v1/proxmox/sync-now")
assert res.status_code == 401
@pytest.mark.asyncio
async def test_requires_auth(client: AsyncClient) -> None:
res = await client.post("/api/v1/proxmox/import-pending", json={"host": "pve"})
assert res.status_code == 401
@pytest.mark.asyncio
async def test_config_omits_token(client: AsyncClient, headers: dict) -> None:
settings.proxmox_token_id = "u@pam!t"
settings.proxmox_token_secret = "supersecret"
res = await client.get("/api/v1/proxmox/config", headers=headers)
assert res.status_code == 200
body = res.text
assert "supersecret" not in body
assert res.json()["token_configured"] is True
@pytest.mark.asyncio
async def test_enable_sync_without_token_rejected(client: AsyncClient, headers: dict) -> None:
# _clear_env_token leaves token empty → enabling auto-sync is rejected.
res = await client.post(
"/api/v1/proxmox/config",
json={"sync_enabled": True, "sync_interval": 600},
headers=headers,
)
assert res.status_code == 400
@pytest.mark.asyncio
async def test_save_config_persists_only_sync_fields(client: AsyncClient, headers: dict) -> None:
"""Connection config is env-only: even if a client sends host/port/verify,
the endpoint ignores them and persists only the sync activation."""
settings.proxmox_host = "pve"
settings.proxmox_token_id = "u@pam!t"
settings.proxmox_token_secret = "s"
saved: dict = {}
with patch.object(type(settings), "save_overrides", lambda self: saved.update(
host=self.proxmox_host, enabled=self.proxmox_sync_enabled, interval=self.proxmox_sync_interval
)), patch("app.api.routes.proxmox.set_proxmox_sync_enabled"), \
patch("app.api.routes.proxmox.reschedule_proxmox_sync"):
res = await client.post(
"/api/v1/proxmox/config",
json={"host": "attacker", "port": 1, "verify_tls": False, "sync_enabled": True, "sync_interval": 900},
headers=headers,
)
assert res.status_code == 200
# host untouched by the request body; sync activation applied.
assert settings.proxmox_host == "pve"
assert saved == {"host": "pve", "enabled": True, "interval": 900}
@pytest.mark.asyncio
async def test_background_import_broadcasts_refresh() -> None:
"""After persisting, the import emits a scan update so an open inventory
reloads without a manual refresh (same signal the IP scan uses)."""
fake_db = AsyncMock()
fake_db.get = AsyncMock(return_value=None) # no ScanRun row → skip status update
cm = AsyncMock()
cm.__aenter__.return_value = fake_db
cm.__aexit__.return_value = False
with patch("app.api.routes.proxmox.AsyncSessionLocal", MagicMock(return_value=cm)), \
patch("app.api.routes.proxmox.fetch_proxmox_inventory", new=AsyncMock(return_value=([], []))), \
patch("app.api.routes.proxmox._persist_pending_import",
new=AsyncMock(return_value=SimpleNamespace(device_count=3))), \
patch("app.api.routes.status.broadcast_scan_update", new=AsyncMock()) as bcast:
await _background_proxmox_import("run1", "h", 8006, "u@pam!t", "s", True)
bcast.assert_awaited_once()
assert bcast.await_args.kwargs["devices_found"] == 3
# --- guest-visibility advisory ---------------------------------------------
def test_advisory_when_hosts_only() -> None:
msg = _guest_visibility_advisory([_host_node()])
assert msg is not None
assert "PVEAuditor" in msg
def test_no_advisory_when_guests_present() -> None:
assert _guest_visibility_advisory([_host_node(), _guest_node(101, "10.0.0.5")]) is None
def test_no_advisory_when_nothing_imported() -> None:
assert _guest_visibility_advisory([]) is None
# --- persistence / dedupe --------------------------------------------------
@pytest.mark.asyncio
async def test_persist_creates_pending(db_session) -> None:
nodes = [_host_node(), _guest_node(101, "10.0.0.5")]
edges = [{"source": "pve-node-pve1", "target": "pve-pve1-101"}]
result = await _persist_pending_import(db_session, nodes, edges)
assert result.pending_created == 2
assert result.links_recorded == 1
rows = (await db_session.execute(select(InventoryDevice))).scalars().all()
assert {r.suggested_type for r in rows} == {"proxmox", "vm"}
# Specs carried as properties.
vm = next(r for r in rows if r.suggested_type == "vm")
assert any(p["key"] == "CPU Cores" for p in vm.properties)
@pytest.mark.asyncio
async def test_persist_merges_existing_scanned_node_by_ip(db_session) -> None:
# A device previously found by an IP scan (no ieee, no specs).
# The scanned host: one inventory row, drawn by one node.
scanned_device = InventoryDevice(
id=str(uuid.uuid4()), ip="10.0.0.5", suggested_type="generic",
status="approved", discovery_source="arp", discovery_sources=["arp"],
)
db_session.add(scanned_device)
await db_session.flush()
scanned = Node(
id=str(uuid.uuid4()), type="generic", label="10.0.0.5",
device_id=scanned_device.id, pos_x=0, pos_y=0,
)
db_session.add(scanned)
await db_session.commit()
await _persist_pending_import(db_session, [_guest_node(101, "10.0.0.5")], [])
# No duplicate node, and no second row: the import merged into the row the
# node already draws.
nodes = (await db_session.execute(select(Node))).scalars().all()
assert len(nodes) == 1
merged = await db_session.get(InventoryDevice, nodes[0].device_id)
assert merged is not None
assert merged.ieee_address == "pve-pve1-101"
assert merged.cpu_count == 2
assert any(p["key"] == "CPU Cores" for p in (merged.properties or []))
# Still approved — it is on a canvas.
assert merged.status == "approved"
@pytest.mark.asyncio
async def test_persist_merges_pending_scan_row_by_mac(db_session) -> None:
# Device previously found by an IP scan: arp source, MAC known, no ieee.
db_session.add(InventoryDevice(
id=str(uuid.uuid4()), ip="10.0.0.5", mac="bc:24:11:aa:bb:cc",
suggested_type="generic", status="pending",
discovery_source="arp", discovery_sources=["arp"],
))
await db_session.commit()
# Proxmox import of the same box (stopped VM → no IP) but same NIC MAC in a
# different casing. Must merge, not duplicate.
await _persist_pending_import(db_session, [_guest_node(101, None, mac="BC:24:11:AA:BB:CC")], [])
rows = (await db_session.execute(select(InventoryDevice))).scalars().all()
assert len(rows) == 1
row = rows[0]
assert row.ieee_address == "pve-pve1-101" # adopted proxmox identity
assert row.suggested_type == "vm" # kept proxmox type
assert set(row.discovery_sources) == {"arp", "proxmox"} # shows in both filters
@pytest.mark.asyncio
async def test_persist_preserves_ip_tag_for_legacy_null_source_row(db_session) -> None:
# Legacy inventory row from an old IP scan, before discovery_source(s) were
# recorded: scalar NULL, sources empty — but it has an IP + MAC.
db_session.add(InventoryDevice(
id=str(uuid.uuid4()), ip="192.168.1.108", mac="bc:24:11:6c:96:52",
suggested_type="lxc", status="pending",
discovery_source=None, discovery_sources=[],
))
await db_session.commit()
await _persist_pending_import(
db_session, [_guest_node(108, "192.168.1.108", mac="BC:24:11:6C:96:52")], []
)
rows = (await db_session.execute(select(InventoryDevice))).scalars().all()
assert len(rows) == 1
# The IP tag must survive the proxmox merge even with no recorded origin.
assert set(rows[0].discovery_sources) == {"arp", "proxmox"}
@pytest.mark.asyncio
async def test_persist_preserves_ip_tag_on_canvas_merge_legacy_row(db_session) -> None:
# The immich case: an on-canvas node from an old scan, with a legacy
# inventory row (NULL source). Merge must keep the IP tag on the row.
inv = InventoryDevice(
id=str(uuid.uuid4()), ip="192.168.1.108", mac="bc:24:11:6c:96:52",
suggested_type="lxc", status="approved",
discovery_source=None, discovery_sources=[],
)
db_session.add(inv)
await db_session.flush()
db_session.add(Node(
id=str(uuid.uuid4()), type="lxc", label="immich",
device_id=inv.id, pos_x=0, pos_y=0,
))
await db_session.commit()
await _persist_pending_import(
db_session, [_guest_node(108, "192.168.1.108", mac="BC:24:11:6C:96:52")], []
)
inv_row = (await db_session.execute(
select(InventoryDevice).where(InventoryDevice.mac == "bc:24:11:6c:96:52")
)).scalar_one()
assert set(inv_row.discovery_sources) == {"arp", "proxmox"}
@pytest.mark.asyncio
async def test_persist_does_not_add_ip_tag_to_pure_proxmox_guest(db_session) -> None:
# A guest first seen via Proxmox (agent IP, pve ieee) must NOT gain a spurious
# IP tag on re-sync — it was never IP-scanned.
await _persist_pending_import(db_session, [_guest_node(101, "10.0.0.5")], [])
await _persist_pending_import(db_session, [_guest_node(101, "10.0.0.5")], []) # re-sync
row = (await db_session.execute(
select(InventoryDevice).where(InventoryDevice.ieee_address == "pve-pve1-101")
)).scalar_one()
assert set(row.discovery_sources) == {"proxmox"}
@pytest.mark.asyncio
async def test_persist_merges_canvas_node_by_mac(db_session) -> None:
# A scanned canvas node with a MAC but no IP recorded for the guest.
scanned_device = InventoryDevice(
id=str(uuid.uuid4()), mac="bc:24:11:aa:bb:cc", suggested_type="generic",
status="approved", discovery_source="arp", discovery_sources=["arp"],
)
db_session.add(scanned_device)
await db_session.flush()
scanned = Node(
id=str(uuid.uuid4()), type="generic", label="box",
device_id=scanned_device.id, pos_x=0, pos_y=0,
)
db_session.add(scanned)
await db_session.commit()
await _persist_pending_import(db_session, [_guest_node(101, None, mac="BC:24:11:AA:BB:CC")], [])
nodes = (await db_session.execute(select(Node))).scalars().all()
assert len(nodes) == 1 # no duplicate node
merged = await db_session.get(InventoryDevice, nodes[0].device_id)
assert merged is not None
assert merged.ieee_address == "pve-pve1-101"
assert merged.mac == "bc:24:11:aa:bb:cc"
assert merged.cpu_count == 2 # specs backfilled
@pytest.mark.asyncio
async def test_persist_resync_updates_in_place(db_session) -> None:
nodes = [_guest_node(101, "10.0.0.5")]
await _persist_pending_import(db_session, nodes, [])
# Second sync: same device, new IP. Should update, not duplicate.
await _persist_pending_import(db_session, [_guest_node(101, "10.0.0.9")], [])
rows = (await db_session.execute(select(InventoryDevice).where(InventoryDevice.ieee_address == "pve-pve1-101"))).scalars().all()
assert len(rows) == 1
assert rows[0].ip == "10.0.0.9"
@pytest.mark.asyncio
async def test_persist_keeps_hidden_hidden(db_session) -> None:
db_session.add(InventoryDevice(
id=str(uuid.uuid4()), ieee_address="pve-pve1-101", ip="10.0.0.5",
suggested_type="vm", status="hidden", discovery_source="proxmox",
))
await db_session.commit()
await _persist_pending_import(db_session, [_guest_node(101, "10.0.0.5")], [])
row = (await db_session.execute(select(InventoryDevice).where(InventoryDevice.ieee_address == "pve-pve1-101"))).scalar_one()
assert row.status == "hidden"
@pytest.mark.asyncio
async def test_pending_endpoint_tolerates_legacy_null_properties(client: AsyncClient, headers: dict, db_session) -> None:
# Legacy row: properties column NULL (added by migration on older DBs).
dev = InventoryDevice(
id=str(uuid.uuid4()), ip="192.168.1.9", suggested_type="server",
status="pending", discovery_source="arp",
)
dev.properties = None
db_session.add(dev)
await db_session.commit()
res = await client.get("/api/v1/scan/pending", headers=headers)
assert res.status_code == 200
assert res.json()[0]["properties"] == []
# --- cluster links ---------------------------------------------------------
@pytest.mark.asyncio
async def test_persist_records_cluster_links_between_hosts(db_session) -> None:
# Two hosts + a guest → one host↔host cluster link, one host→guest link.
nodes = [
{**_host_node(), "id": "pve-node-a", "ieee_address": "pve-node-a", "hostname": "a", "label": "a"},
{**_host_node(), "id": "pve-node-b", "ieee_address": "pve-node-b", "hostname": "b", "label": "b"},
_guest_node(101, "10.0.0.5"),
]
edges = [{"source": "pve-node-pve1", "target": "pve-pve1-101"}]
await _persist_pending_import(db_session, nodes, edges)
links = (await db_session.execute(select(InventoryDeviceLink))).scalars().all()
cluster = [ln for ln in links if ln.discovery_source == "proxmox_cluster"]
assert len(cluster) == 1
assert (cluster[0].source_ieee, cluster[0].target_ieee) == ("pve-node-a", "pve-node-b")
# Membership helper sees both endpoints.
assert await _is_proxmox_cluster_member(db_session, "pve-node-a") is True
assert await _is_proxmox_cluster_member(db_session, "pve-node-b") is True
assert await _is_proxmox_cluster_member(db_session, "pve-pve1-101") is False
@pytest.mark.asyncio
async def test_single_host_records_no_cluster_link(db_session) -> None:
await _persist_pending_import(db_session, [_host_node()], [])
links = (await db_session.execute(
select(InventoryDeviceLink).where(InventoryDeviceLink.discovery_source == "proxmox_cluster")
)).scalars().all()
assert links == []
@pytest.mark.asyncio
async def test_cluster_link_resolves_to_cluster_edge(db_session) -> None:
# Two host nodes already on a canvas + a pending cluster link between them.
design = Design(id=str(uuid.uuid4()), name="d")
db_session.add(design)
dev_a = InventoryDevice(id=str(uuid.uuid4()), ieee_address="pve-node-a", status="approved")
dev_b = InventoryDevice(id=str(uuid.uuid4()), ieee_address="pve-node-b", status="approved")
db_session.add_all([dev_a, dev_b])
await db_session.flush()
a = Node(id=str(uuid.uuid4()), type="proxmox", label="a", device_id=dev_a.id,
pos_x=0, pos_y=0, design_id=design.id, left_handles=1, right_handles=1)
b = Node(id=str(uuid.uuid4()), type="proxmox", label="b", device_id=dev_b.id,
pos_x=0, pos_y=0, design_id=design.id, left_handles=1, right_handles=1)
db_session.add_all([a, b])
db_session.add(InventoryDeviceLink(
id=str(uuid.uuid4()), source_ieee="pve-node-a", target_ieee="pve-node-b",
discovery_source="proxmox_cluster",
))
await db_session.commit()
await _resolve_pending_links_for_ieee(db_session, "pve-node-a", design.id)
edge = (await db_session.execute(select(Edge))).scalars().one()
assert edge.type == "cluster"
assert edge.source_handle == "right"
# Bare side name (canonical) so React Flow resolves it to the left side;
# a '-t' target would fall back to the top handle.
assert edge.target_handle == "left"
@pytest.mark.asyncio
async def test_link_survives_and_resolves_onto_second_design(db_session) -> None:
# Regression: approving the same mesh devices onto a second canvas must also
# get their edges. The link row is topology, not one-shot — resolving it on
# design A must not consume it, so design B resolves too.
da = Design(id=str(uuid.uuid4()), name="a")
db_ = Design(id=str(uuid.uuid4()), name="b")
db_session.add_all([da, db_])
# One row per device, whatever the canvas count.
dev_x = InventoryDevice(id=str(uuid.uuid4()), ieee_address="0xAAAA", status="approved")
dev_y = InventoryDevice(id=str(uuid.uuid4()), ieee_address="0xBBBB", status="approved")
db_session.add_all([dev_x, dev_y])
await db_session.flush()
# Same two devices placed on BOTH designs (one Node per canvas).
for d in (da, db_):
db_session.add_all([
Node(id=str(uuid.uuid4()), type="iot", label="x", device_id=dev_x.id,
pos_x=0, pos_y=0, design_id=d.id),
Node(id=str(uuid.uuid4()), type="iot", label="y", device_id=dev_y.id,
pos_x=0, pos_y=0, design_id=d.id),
])
db_session.add(InventoryDeviceLink(
id=str(uuid.uuid4()), source_ieee="0xAAAA", target_ieee="0xBBBB",
discovery_source="zigbee",
))
await db_session.commit()
first = await _resolve_pending_links_for_ieee(db_session, "0xAAAA", da.id)
second = await _resolve_pending_links_for_ieee(db_session, "0xAAAA", db_.id)
assert len(first) == 1
assert len(second) == 1 # would be 0 if the link were consumed on design A
# One iot edge per design, each between that design's own nodes.
edges = (await db_session.execute(select(Edge))).scalars().all()
assert len(edges) == 2
assert {e.design_id for e in edges} == {da.id, db_.id}
assert all(e.type == "iot" for e in edges)
# The link row is still present for any further design.
assert (await db_session.execute(select(InventoryDeviceLink))).scalars().one()
@pytest.mark.asyncio
async def test_persist_never_deletes(db_session) -> None:
await _persist_pending_import(db_session, [_guest_node(101, "10.0.0.5")], [])
# A later sync that no longer includes vm101 must not remove it.
await _persist_pending_import(db_session, [_guest_node(202, "10.0.0.6")], [])
rows = (await db_session.execute(select(InventoryDevice))).scalars().all()
ieees = {r.ieee_address for r in rows}
assert "pve-pve1-101" in ieees and "pve-pve1-202" in ieees
+220
View File
@@ -0,0 +1,220 @@
"""Unit tests for the Proxmox VE import service (parsing, props, sanitizer)."""
from __future__ import annotations
from unittest.mock import AsyncMock, patch
import httpx
import pytest
from app.services import proxmox_service as svc
def test_gb_conversion() -> None:
assert svc._gb(1024 ** 3) == 1.0
assert svc._gb(2 * 1024 ** 3) == 2.0
assert svc._gb(0) is None
assert svc._gb(None) is None
assert svc._gb("nope") is None
def test_extract_qemu_ip_skips_loopback() -> None:
payload = {
"result": [
{"name": "lo", "ip-addresses": [{"ip-address-type": "ipv4", "ip-address": "127.0.0.1"}]},
{"name": "eth0", "ip-addresses": [{"ip-address-type": "ipv4", "ip-address": "192.168.1.20"}]},
]
}
assert svc._extract_qemu_ip(payload) == "192.168.1.20"
assert svc._extract_qemu_ip(None) is None
assert svc._extract_qemu_ip({"result": []}) is None
def test_extract_lxc_ip_parses_net0_static() -> None:
cfg = {"net0": "name=eth0,bridge=vmbr0,ip=192.168.1.30/24,gw=192.168.1.1"}
assert svc._extract_lxc_ip(cfg) == "192.168.1.30"
# DHCP → no static IP
assert svc._extract_lxc_ip({"net0": "name=eth0,bridge=vmbr0,ip=dhcp"}) is None
assert svc._extract_lxc_ip(None) is None
def test_extract_net_mac_parses_qemu_and_lxc_forms() -> None:
# qemu: "virtio=<MAC>,bridge=.."
assert svc._extract_net_mac({"net0": "virtio=BC:24:11:AA:BB:CC,bridge=vmbr0"}) == "bc:24:11:aa:bb:cc"
# lxc: "..,hwaddr=<MAC>,.."
assert (
svc._extract_net_mac({"net0": "name=eth0,bridge=vmbr0,hwaddr=BC:24:11:11:22:33,ip=dhcp"})
== "bc:24:11:11:22:33"
)
# No MAC / missing net0 / None → None
assert svc._extract_net_mac({"net0": "name=eth0,ip=dhcp"}) is None
assert svc._extract_net_mac({}) is None
assert svc._extract_net_mac(None) is None
@pytest.mark.asyncio
async def test_fetch_inventory_captures_guest_mac_from_config() -> None:
"""Guests carry a normalized NIC MAC read agent-free from their config."""
async def fake_get_json(client, path: str):
if path == "/nodes":
return [{"node": "pve1", "status": "online", "maxcpu": 8}]
if path == "/nodes/pve1/qemu":
return [{"vmid": 101, "name": "web", "status": "stopped"}] # stopped: no agent IP
if path == "/nodes/pve1/lxc":
return [{"vmid": 200, "name": "db", "status": "running"}]
if path == "/nodes/pve1/qemu/101/config":
return {"net0": "virtio=AA:BB:CC:DD:EE:FF,bridge=vmbr0"}
if path == "/nodes/pve1/lxc/200/config":
return {"net0": "name=eth0,bridge=vmbr0,hwaddr=11:22:33:44:55:66,ip=10.0.0.6/24"}
return None
with patch.object(svc, "_get_json", new=AsyncMock(side_effect=fake_get_json)):
nodes, _ = await svc.fetch_proxmox_inventory("h", 8006, "u@pam!t", "sec")
vm = next(n for n in nodes if n["type"] == "vm")
assert vm["mac"] == "aa:bb:cc:dd:ee:ff" # captured even though VM is stopped (no IP)
assert vm["ip"] is None
ct = next(n for n in nodes if n["type"] == "lxc")
assert ct["mac"] == "11:22:33:44:55:66"
assert ct["ip"] == "10.0.0.6"
def test_host_and_guest_node_mapping() -> None:
host = svc._host_node({"node": "pve1", "status": "online", "maxcpu": 8, "maxmem": 16 * 1024 ** 3, "maxdisk": 500 * 1024 ** 3})
assert host["type"] == "proxmox"
assert host["ieee_address"] == "pve-node-pve1"
assert host["cpu_count"] == 8
assert host["ram_gb"] == 16.0
assert host["status"] == "online"
assert host["parent_ieee"] is None
vm = svc._guest_node({"vmid": 101, "name": "web", "status": "running", "maxcpu": 2, "maxmem": 2 * 1024 ** 3, "maxdisk": 32 * 1024 ** 3}, "pve1", "qemu", "10.0.0.5")
assert vm["type"] == "vm"
assert vm["ieee_address"] == "pve-pve1-101"
assert vm["ip"] == "10.0.0.5"
assert vm["status"] == "online"
assert vm["parent_ieee"] == "pve-node-pve1"
ct = svc._guest_node({"vmid": 200, "name": "db", "status": "stopped"}, "pve1", "lxc", None)
assert ct["type"] == "lxc"
assert ct["status"] == "offline"
def test_build_properties_includes_specs() -> None:
node = {"vmid": 101, "model": "QEMU", "cpu_count": 4, "ram_gb": 8.0, "disk_gb": 40.0}
props = svc.build_proxmox_properties(node)
keys = {p["key"] for p in props}
assert {"VMID", "CPU Cores", "RAM", "Disk", "Source"} <= keys
assert all(p["visible"] is False for p in props)
def test_parse_inventory_builds_host_guest_edges() -> None:
hosts = [{"node": "pve1", "status": "online", "maxcpu": 4}]
guests = {"pve1": [svc._guest_node({"vmid": 101, "name": "web", "status": "running"}, "pve1", "qemu", None)]}
nodes, edges = svc._parse_inventory(hosts, guests)
assert len(nodes) == 2
assert edges == [{"source": "pve-node-pve1", "target": "pve-pve1-101"}]
def test_build_cluster_links_chains_hosts() -> None:
nodes = [
svc._host_node({"node": "pve-a", "status": "online"}),
svc._guest_node({"vmid": 101, "status": "running"}, "pve-a", "qemu", None),
svc._host_node({"node": "pve-b", "status": "online"}),
svc._host_node({"node": "pve-c", "status": "online"}),
]
pairs = svc.build_proxmox_cluster_links(nodes)
assert pairs == [("pve-node-pve-a", "pve-node-pve-b"), ("pve-node-pve-b", "pve-node-pve-c")]
def test_build_cluster_links_single_host_is_not_a_cluster() -> None:
nodes = [svc._host_node({"node": "pve-a", "status": "online"})]
assert svc.build_proxmox_cluster_links(nodes) == []
# Guests alone never form a cluster.
guest = svc._guest_node({"vmid": 1, "status": "running"}, "pve-a", "qemu", None)
assert svc.build_proxmox_cluster_links([guest]) == []
def test_sanitize_error_hides_credentials() -> None:
exc = httpx.HTTPStatusError(
"boom", request=httpx.Request("GET", "https://h/api2/json"),
response=httpx.Response(401),
)
msg = svc._sanitize_proxmox_error(exc)
assert "token" not in msg.lower() or "check the api token" in msg.lower()
assert "Authentication failed" in msg
@pytest.mark.asyncio
async def test_fetch_inventory_happy_path() -> None:
async def fake_get_json(client, path: str):
if path == "/nodes":
return [{"node": "pve1", "status": "online", "maxcpu": 8, "maxmem": 16 * 1024 ** 3}]
if path == "/nodes/pve1/qemu":
return [{"vmid": 101, "name": "web", "status": "running", "maxmem": 2 * 1024 ** 3}]
if path == "/nodes/pve1/lxc":
return [{"vmid": 200, "name": "db", "status": "stopped"}]
if path.endswith("/agent/network-get-interfaces"):
return {"result": [{"name": "eth0", "ip-addresses": [{"ip-address-type": "ipv4", "ip-address": "10.0.0.5"}]}]}
if path.endswith("/config"):
return {"net0": "name=eth0,ip=10.0.0.6/24"}
return None
with patch.object(svc, "_get_json", new=AsyncMock(side_effect=fake_get_json)):
nodes, edges = await svc.fetch_proxmox_inventory("h", 8006, "u@pam!t", "sec")
by_type = {n["type"] for n in nodes}
assert by_type == {"proxmox", "vm", "lxc"}
vm = next(n for n in nodes if n["type"] == "vm")
assert vm["ip"] == "10.0.0.5"
ct = next(n for n in nodes if n["type"] == "lxc")
assert ct["ip"] == "10.0.0.6"
assert len(edges) == 2
@pytest.mark.asyncio
async def test_test_connection_returns_message() -> None:
async def fake_get_json(client, path: str):
if path == "/access/permissions":
return {"/": {"VM.Audit": 1}} # token has an ACL
return {"version": "8.2.2"}
with patch.object(svc, "_get_json", new=AsyncMock(side_effect=fake_get_json)):
ok, msg = await svc.test_proxmox_connection("h", 8006, "u@pam!t", "sec")
assert ok is True
assert "8.2.2" in msg
assert "warning" not in msg.lower()
@pytest.mark.asyncio
async def test_test_connection_warns_when_token_has_no_permissions() -> None:
async def fake_get_json(client, path: str):
if path == "/access/permissions":
return {} # privilege-separated token with no effective ACL
return {"version": "8.4.19"}
with patch.object(svc, "_get_json", new=AsyncMock(side_effect=fake_get_json)):
ok, msg = await svc.test_proxmox_connection("h", 8006, "u@pam!t", "sec")
# Auth still succeeds; the message flags the permission gap.
assert ok is True
assert "8.4.19" in msg
assert "PVEAuditor" in msg
@pytest.mark.asyncio
async def test_token_has_permissions_treats_empty_as_no_perms() -> None:
async def fake_get_json(client, path: str):
return {}
with patch.object(svc, "_get_json", new=AsyncMock(side_effect=fake_get_json)):
assert await svc._token_has_permissions(object()) is False
@pytest.mark.asyncio
async def test_token_has_permissions_assumes_ok_on_error() -> None:
async def boom(client, path: str):
raise httpx.ConnectError("nope")
with patch.object(svc, "_get_json", new=AsyncMock(side_effect=boom)):
# Never block a valid import on a permissions-probe failure.
assert await svc._token_has_permissions(object()) is True
+34
View File
@@ -0,0 +1,34 @@
"""The rackable exclusion list is duplicated in the frontend — keep them equal.
`GET /api/v1/racks/inventory` filters on `_UNRACKABLE_TYPES`, while the Device
Inventory modal's "Rackable" filter (used as the rack picker's default) filters
on `UNRACKABLE_TYPES` in `frontend/src/utils/rackable.ts`. If the two drift, the
picker offers devices the rack side then refuses.
"""
import re
from pathlib import Path
from app.api.routes.racks import _UNRACKABLE_TYPES
FRONTEND_FILE = (
Path(__file__).resolve().parents[2] / "frontend" / "src" / "utils" / "rackable.ts"
)
def _frontend_unrackable() -> set[str]:
src = FRONTEND_FILE.read_text()
match = re.search(r"export const UNRACKABLE_TYPES = new Set\(\[(.*?)\]\)", src, re.S)
assert match, (
"Could not locate UNRACKABLE_TYPES in frontend/src/utils/rackable.ts — "
"update this parser if the file's shape changed."
)
return set(re.findall(r"'([^']+)'", match.group(1)))
def test_unrackable_types_in_sync_with_frontend():
frontend = _frontend_unrackable()
assert frontend == _UNRACKABLE_TYPES, (
"racks.py _UNRACKABLE_TYPES is out of sync with frontend rackable.ts.\n"
f"Missing from the backend: {sorted(frontend - _UNRACKABLE_TYPES)}\n"
f"Missing from the frontend: {sorted(_UNRACKABLE_TYPES - frontend)}"
)
+804
View File
@@ -0,0 +1,804 @@
import pytest
from httpx import AsyncClient
from sqlalchemy import text
from app.db.models import InventoryDevice
pytestmark = pytest.mark.asyncio
async def _design(client: AsyncClient, headers: dict, *, name="Rack Room", design_type="rack") -> str:
res = await client.post(
"/api/v1/designs",
json={"name": name, "icon": "server", "design_type": design_type},
headers=headers,
)
assert res.status_code == 201, res.text
return res.json()["id"]
def _state(design_id: str, **overrides) -> dict:
"""A minimal but complete rack payload: one rack, two devices, one patch."""
payload = {
"design_id": design_id,
"racks": [
{
"id": "rack-1",
"name": "Main",
"u_height": 12,
"width_standard": "19",
"numbering": "bottom-up",
"location": "garage",
"style": {"frame": "#1c2129", "showNumbers": True},
"pos_x": 40,
"pos_y": 60,
}
],
"devices": [
{
"id": "dev-sw",
"rack_id": "rack-1",
"label": "sw-24",
"u_start": 10,
"u_height": 1,
"col_start": 0,
"col_span": 12,
"faceplate_id": "switch-24",
"status": "online",
"ports": [{"id": "p1", "label": "1", "type": "rj45", "x": 0.4, "y": 0.5}],
},
{
"id": "dev-nas",
"rack_id": "rack-1",
"label": "nas",
"u_start": 4,
"u_height": 2,
"col_start": 0,
"col_span": 12,
"faceplate_id": "nas-2u",
"status": "unknown",
"ports": [{"id": "p2", "label": "eth0", "type": "rj45", "x": 0.7, "y": 0.5}],
},
],
"cables": [
{
"id": "cbl-1",
"from_device_id": "dev-sw",
"from_port_id": "p1",
"to_device_id": "dev-nas",
"to_port_id": "p2",
"type": "ethernet",
"color": "#39d353",
}
],
"viewport": {"x": 10, "y": 20, "zoom": 1.5},
}
payload.update(overrides)
return payload
class TestDesignType:
async def test_creates_a_rack_design(self, client: AsyncClient, headers):
design_id = await _design(client, headers)
res = await client.get("/api/v1/designs", headers=headers)
design = next(d for d in res.json() if d["id"] == design_id)
assert design["design_type"] == "rack"
async def test_rejects_an_unknown_design_type(self, client: AsyncClient, headers):
res = await client.post(
"/api/v1/designs",
json={"name": "Nope", "design_type": "spaceship"},
headers=headers,
)
assert res.status_code == 422
class TestSaveAndLoad:
async def test_round_trips_the_full_state(self, client: AsyncClient, headers):
design_id = await _design(client, headers)
res = await client.post("/api/v1/racks/save", json=_state(design_id), headers=headers)
assert res.status_code == 200, res.text
loaded = (await client.get(f"/api/v1/racks?design_id={design_id}", headers=headers)).json()
assert [r["name"] for r in loaded["racks"]] == ["Main"]
assert loaded["racks"][0]["u_height"] == 12
assert loaded["racks"][0]["style"]["showNumbers"] is True
assert {d["id"] for d in loaded["devices"]} == {"dev-sw", "dev-nas"}
assert loaded["devices"][0]["ports"][0]["type"] == "rj45"
assert loaded["cables"][0]["from_port_id"] == "p1"
assert loaded["viewport"] == {"x": 10, "y": 20, "zoom": 1.5}
async def test_keeps_a_mount_that_follows_its_node_check(
self, client: AsyncClient, headers
):
# "Check device" is stored as `auto` on the mount: the rack has no
# checker, it reads the linked node's live status at render time. The
# status column is free-form, so this only guards it staying that way.
design_id = await _design(client, headers)
state = _state(design_id)
state["devices"][0]["status"] = "auto"
res = await client.post("/api/v1/racks/save", json=state, headers=headers)
assert res.status_code == 200, res.text
loaded = (await client.get(f"/api/v1/racks?design_id={design_id}", headers=headers)).json()
mount = next(d for d in loaded["devices"] if d["id"] == "dev-sw")
assert mount["status"] == "auto"
async def test_prunes_what_the_client_dropped(self, client: AsyncClient, headers):
design_id = await _design(client, headers)
await client.post("/api/v1/racks/save", json=_state(design_id), headers=headers)
trimmed = _state(design_id)
trimmed["devices"] = [d for d in trimmed["devices"] if d["id"] == "dev-sw"]
trimmed["cables"] = []
await client.post("/api/v1/racks/save", json=trimmed, headers=headers)
loaded = (await client.get(f"/api/v1/racks?design_id={design_id}", headers=headers)).json()
assert {d["id"] for d in loaded["devices"]} == {"dev-sw"}
assert loaded["cables"] == []
async def test_updates_an_existing_device_in_place(self, client: AsyncClient, headers):
design_id = await _design(client, headers)
await client.post("/api/v1/racks/save", json=_state(design_id), headers=headers)
moved = _state(design_id)
moved["devices"][1]["u_start"] = 1
moved["devices"][1]["label"] = "nas-renamed"
await client.post("/api/v1/racks/save", json=moved, headers=headers)
loaded = (await client.get(f"/api/v1/racks?design_id={design_id}", headers=headers)).json()
nas = next(d for d in loaded["devices"] if d["id"] == "dev-nas")
assert (nas["u_start"], nas["label"]) == (1, "nas-renamed")
async def test_keeps_designs_isolated(self, client: AsyncClient, headers):
first = await _design(client, headers, name="Rack A")
second = await _design(client, headers, name="Rack B")
await client.post("/api/v1/racks/save", json=_state(first), headers=headers)
loaded = (await client.get(f"/api/v1/racks?design_id={second}", headers=headers)).json()
assert loaded["racks"] == []
assert loaded["devices"] == []
async def test_refuses_to_steal_a_row_from_another_design(
self, client: AsyncClient, headers
):
"""Ids come from the client, so a stale tab or a copy can collide.
The upsert sets `design_id` from the payload: without a guard, saving
design B with an id owned by design A moved A's rack — devices and
cables with it — into B, and A silently lost it.
"""
first = await _design(client, headers, name="Rack A")
second = await _design(client, headers, name="Rack B")
await client.post("/api/v1/racks/save", json=_state(first), headers=headers)
res = await client.post("/api/v1/racks/save", json=_state(second), headers=headers)
assert res.status_code == 409
kept = (await client.get(f"/api/v1/racks?design_id={first}", headers=headers)).json()
assert [r["id"] for r in kept["racks"]] == ["rack-1"]
assert len(kept["devices"]) == 2
async def test_rejects_an_unknown_design(self, client: AsyncClient, headers):
res = await client.post("/api/v1/racks/save", json=_state("nope"), headers=headers)
assert res.status_code == 404
async def test_rejects_a_device_pointing_outside_the_payload(self, client: AsyncClient, headers):
design_id = await _design(client, headers)
broken = _state(design_id)
broken["devices"][0]["rack_id"] = "rack-ghost"
res = await client.post("/api/v1/racks/save", json=broken, headers=headers)
assert res.status_code == 400
async def test_rejects_a_cable_pointing_outside_the_payload(self, client: AsyncClient, headers):
design_id = await _design(client, headers)
broken = _state(design_id)
broken["cables"][0]["to_device_id"] = "dev-ghost"
res = await client.post("/api/v1/racks/save", json=broken, headers=headers)
assert res.status_code == 400
@pytest.mark.parametrize(
"patch",
[
{"u_height": 0},
{"width_standard": "23"},
{"numbering": "sideways"},
],
)
async def test_rejects_invalid_rack_geometry(self, client: AsyncClient, headers, patch):
design_id = await _design(client, headers)
payload = _state(design_id)
payload["racks"][0].update(patch)
res = await client.post("/api/v1/racks/save", json=payload, headers=headers)
assert res.status_code == 422
@pytest.mark.parametrize(
"patch",
[
{"u_start": 0},
{"col_start": 12},
{"col_span": 13},
],
)
async def test_rejects_invalid_device_geometry(self, client: AsyncClient, headers, patch):
design_id = await _design(client, headers)
payload = _state(design_id)
payload["devices"][0].update(patch)
res = await client.post("/api/v1/racks/save", json=payload, headers=headers)
assert res.status_code == 422
async def test_rejects_a_device_taller_than_its_rack(self, client: AsyncClient, headers):
# Each field is legal on its own; only the rack says otherwise. A mount
# above the top rail draws outside the chassis with no way to drag it back.
design_id = await _design(client, headers)
payload = _state(design_id)
payload["devices"][0]["u_start"] = 40 # rack is 12U
res = await client.post("/api/v1/racks/save", json=payload, headers=headers)
assert res.status_code == 422
async def test_rejects_a_device_whose_height_overruns_the_rack(
self, client: AsyncClient, headers
):
design_id = await _design(client, headers)
payload = _state(design_id)
payload["devices"][0].update({"u_start": 11, "u_height": 4}) # 11..14 of 12U
res = await client.post("/api/v1/racks/save", json=payload, headers=headers)
assert res.status_code == 422
async def test_accepts_a_device_that_ends_on_the_top_rail(
self, client: AsyncClient, headers
):
design_id = await _design(client, headers)
payload = _state(design_id)
payload["devices"][0].update({"u_start": 11, "u_height": 2}) # 11..12 of 12U
res = await client.post("/api/v1/racks/save", json=payload, headers=headers)
assert res.status_code == 200, res.text
async def test_rejects_a_span_that_overruns_the_column_grid(
self, client: AsyncClient, headers
):
design_id = await _design(client, headers)
payload = _state(design_id)
payload["devices"][0].update({"col_start": 11, "col_span": 12})
res = await client.post("/api/v1/racks/save", json=payload, headers=headers)
assert res.status_code == 422
async def test_accepts_a_half_width_pair_sharing_one_u(
self, client: AsyncClient, headers
):
design_id = await _design(client, headers)
payload = _state(design_id)
payload["devices"][0].update({"col_start": 0, "col_span": 6})
payload["devices"][1].update({"col_start": 6, "col_span": 6})
res = await client.post("/api/v1/racks/save", json=payload, headers=headers)
assert res.status_code == 200, res.text
async def test_rejects_an_unknown_cable_type(self, client: AsyncClient, headers):
design_id = await _design(client, headers)
payload = _state(design_id)
payload["cables"][0]["type"] = "power"
res = await client.post("/api/v1/racks/save", json=payload, headers=headers)
assert res.status_code == 422
async def test_round_trips_cable_annotations(self, client: AsyncClient, headers):
design_id = await _design(client, headers)
payload = _state(design_id)
payload["cables"][0].update(
{
"label": "Uplink to core",
"label_visible": True,
"properties": [
{"key": "Length", "value": "2 m", "icon": None, "visible": True},
{"key": "VLAN", "value": "20", "icon": None, "visible": False},
],
}
)
assert (
await client.post("/api/v1/racks/save", json=payload, headers=headers)
).status_code == 200
cable = (
await client.get(f"/api/v1/racks?design_id={design_id}", headers=headers)
).json()["cables"][0]
assert cable["label"] == "Uplink to core"
assert cable["label_visible"] is True
assert [p["key"] for p in cable["properties"]] == ["Length", "VLAN"]
assert cable["properties"][1]["visible"] is False
async def test_defaults_cable_annotations_when_absent(
self, client: AsyncClient, headers
):
# A client that predates the feature sends neither key; the response must
# still carry usable values rather than nulls.
design_id = await _design(client, headers)
assert (
await client.post("/api/v1/racks/save", json=_state(design_id), headers=headers)
).status_code == 200
cable = (
await client.get(f"/api/v1/racks?design_id={design_id}", headers=headers)
).json()["cables"][0]
assert cable["label_visible"] is False
assert cable["properties"] == []
async def test_drops_cable_properties_with_no_key(self, client: AsyncClient, headers):
design_id = await _design(client, headers)
payload = _state(design_id)
payload["cables"][0]["properties"] = [
{"key": " ", "value": "orphan", "icon": None, "visible": True},
{"key": "Length", "value": "2 m", "icon": None, "visible": True},
]
assert (
await client.post("/api/v1/racks/save", json=payload, headers=headers)
).status_code == 200
cable = (
await client.get(f"/api/v1/racks?design_id={design_id}", headers=headers)
).json()["cables"][0]
assert [p["key"] for p in cable["properties"]] == ["Length"]
async def test_updates_cable_annotations_on_a_second_save(
self, client: AsyncClient, headers
):
design_id = await _design(client, headers)
payload = _state(design_id)
payload["cables"][0]["properties"] = [
{"key": "Length", "value": "2 m", "icon": None, "visible": True}
]
await client.post("/api/v1/racks/save", json=payload, headers=headers)
payload["cables"][0]["properties"] = []
payload["cables"][0]["label_visible"] = True
await client.post("/api/v1/racks/save", json=payload, headers=headers)
cable = (
await client.get(f"/api/v1/racks?design_id={design_id}", headers=headers)
).json()["cables"][0]
assert cable["properties"] == []
assert cable["label_visible"] is True
async def test_requires_auth(self, client: AsyncClient):
assert (await client.get("/api/v1/racks?design_id=any")).status_code == 401
class TestInventory:
async def test_lists_rackable_devices_only(self, client: AsyncClient, headers):
design_id = await _design(client, headers)
await client.post(
"/api/v1/scan/pending",
json={"hostname": "pve-01", "suggested_type": "proxmox"},
headers=headers,
)
await client.post(
"/api/v1/scan/pending",
json={"hostname": "phone", "suggested_type": "mobile"},
headers=headers,
)
res = await client.get(f"/api/v1/racks/inventory?design_id={design_id}", headers=headers)
labels = [i["label"] for i in res.json()["items"]]
assert "pve-01" in labels
assert "phone" not in labels
async def test_names_a_device_the_way_the_inventory_does(
self, client: AsyncClient, headers, db_session
):
"""A scan find with no hostname is named after the app it runs, not its IP.
The rack picker used to show "192.168.1.63 · 192.168.1.63" where the
Device Inventory showed "jellyfin".
"""
design_id = await _design(client, headers)
db_session.add(
InventoryDevice(
id="pd-services",
ip="192.168.1.63",
services=[
{"port": 443, "category": "web", "service_name": "https"},
{"port": 8096, "category": "media", "service_name": "jellyfin"},
],
suggested_type="server",
)
)
db_session.add(InventoryDevice(id="pd-bare", ip="192.168.1.64", suggested_type="server"))
# A portless service is no fingerprint: the inventory skips it, so must
# this — `None not in _COMMON_PORTS` is True and used to let it through.
db_session.add(
InventoryDevice(
id="pd-portless",
ip="192.168.1.65",
services=[{"port": None, "category": "media", "service_name": "jellyfin"}],
suggested_type="server",
)
)
await db_session.commit()
items = (
await client.get(f"/api/v1/racks/inventory?design_id={design_id}", headers=headers)
).json()["items"]
assert next(i for i in items if i["id"] == "pd-services")["label"] == "jellyfin"
# Nothing better to say: the IP stays the label, and the UI stops doubling it.
assert next(i for i in items if i["id"] == "pd-bare")["label"] == "192.168.1.64"
assert next(i for i in items if i["id"] == "pd-portless")["label"] == "192.168.1.65"
async def test_flags_devices_already_mounted(self, client: AsyncClient, headers):
design_id = await _design(client, headers)
created = (
await client.post(
"/api/v1/scan/pending",
json={"hostname": "sw-24", "suggested_type": "switch"},
headers=headers,
)
).json()
payload = _state(design_id)
payload["devices"][0]["device_id"] = created["id"]
await client.post("/api/v1/racks/save", json=payload, headers=headers)
items = (
await client.get(f"/api/v1/racks/inventory?design_id={design_id}", headers=headers)
).json()["items"]
assert next(i for i in items if i["id"] == created["id"])["racked"] is True
async def test_resolves_the_canvas_node_for_live_status(self, client: AsyncClient, headers):
design_id = await _design(client, headers)
node = (
await client.post(
"/api/v1/nodes",
json={"type": "server", "label": "nas", "ip": "192.168.1.9", "status": "online"},
headers=headers,
)
).json()
await client.post(
"/api/v1/scan/pending",
json={"hostname": "nas", "ip": "192.168.1.9", "suggested_type": "nas"},
headers=headers,
)
items = (
await client.get(f"/api/v1/racks/inventory?design_id={design_id}", headers=headers)
).json()["items"]
entry = next(i for i in items if i["label"] == "nas")
assert entry["node_id"] == node["id"]
assert entry["node_status"] == "online"
async def test_reports_what_the_canvas_node_knows(
self, client: AsyncClient, headers, db_session
):
"""The rack prints the logical view's facts, so the endpoint must ship them."""
rack_design = await _design(client, headers)
network = await _design(client, headers, name="Network", design_type="network")
# Straight to the table: the manual-create schema carries no os/services,
# and a scan find does. Seeded before the node so the node links to this
# row rather than minting a second one for the same host.
db_session.add(
InventoryDevice(
id="pd-nas",
hostname="nas",
ip="192.168.1.9",
mac="aa:bb:cc:dd:ee:ff",
os="TrueNAS",
suggested_type="nas",
services=[{"port": 445, "service_name": "smb", "category": "storage"}],
)
)
await db_session.commit()
await client.post(
"/api/v1/nodes",
json={
"type": "nas",
"label": "nas-truenas",
"design_id": network,
"ip": "192.168.1.9",
"mac": "aa:bb:cc:dd:ee:ff",
"hostname": "nas.lan",
"os": "TrueNAS SCALE",
"status": "online",
"check_method": "http",
},
headers=headers,
)
items = (
await client.get(f"/api/v1/racks/inventory?design_id={rack_design}", headers=headers)
).json()["items"]
entry = next(i for i in items if i["ip"] == "192.168.1.9")
assert entry["mac"] == "aa:bb:cc:dd:ee:ff"
assert entry["os"] == "TrueNAS"
assert entry["services"] == [{"port": 445, "name": "smb"}]
assert entry["node_label"] == "nas-truenas"
assert entry["node_type"] == "nas"
# The node no longer keeps its own copy: both views read the row, which
# the node create filled in without overwriting what the scan found.
assert entry["node_hostname"] == "nas"
assert entry["node_os"] == "TrueNAS"
assert entry["node_check_method"] == "http"
assert entry["node_design_id"] == network
assert entry["node_design_name"] == "Network"
async def test_stamps_last_seen_with_an_offset(self, client: AsyncClient, headers, db_session):
"""SQLite reads `last_seen` back naive, and a naive ISO string is *local*
time to `new Date()` — the panel printed it shifted by the viewer's offset.
"""
design_id = await _design(client, headers)
node = (
await client.post(
"/api/v1/nodes",
json={"type": "server", "label": "seen", "ip": "192.168.1.77"},
headers=headers,
)
).json()
await db_session.execute(
text("UPDATE device_inventory SET last_seen = :ts WHERE id = :id"),
{"ts": "2026-08-08 10:00:00.000000", "id": node["device_id"]},
)
await db_session.commit()
items = (
await client.get(f"/api/v1/racks/inventory?design_id={design_id}", headers=headers)
).json()["items"]
entry = next(i for i in items if i["id"] == node["device_id"])
assert entry["node_last_seen"].endswith(("Z", "+00:00"))
async def test_prefers_the_node_the_mount_names(self, client: AsyncClient, headers):
"""A link the user picked in the rack beats the IEEE/IP guess.
The guess is what failed on gear with no MAC and no address, and it is
wrong — not merely absent — when two hosts have swapped IPs.
"""
design_id = await _design(client, headers)
picked = (
await client.post(
"/api/v1/nodes",
json={"type": "pdu", "label": "pdu-main", "status": "online"},
headers=headers,
)
).json()
guessed = (
await client.post(
"/api/v1/nodes",
json={"type": "server", "label": "someone-else", "ip": "192.168.1.90"},
headers=headers,
)
).json()
device = (
await client.post(
"/api/v1/scan/pending",
json={"hostname": "pdu", "ip": "192.168.1.90", "suggested_type": "generic"},
headers=headers,
)
).json()
payload = _state(design_id)
payload["devices"][0]["device_id"] = device["id"]
payload["devices"][0]["node_id"] = picked["id"]
await client.post("/api/v1/racks/save", json=payload, headers=headers)
items = (
await client.get(f"/api/v1/racks/inventory?design_id={design_id}", headers=headers)
).json()["items"]
entry = next(i for i in items if i["id"] == device["id"])
assert entry["node_id"] == picked["id"] != guessed["id"]
assert entry["node_label"] == "pdu-main"
assert entry["node_status"] == "online"
async def test_falls_back_when_the_named_node_is_gone(self, client: AsyncClient, headers):
"""A mount keeps its `node_id` in the payload the client last sent, so a
node deleted meanwhile must not report a node that no longer exists.
"""
design_id = await _design(client, headers)
guessed = (
await client.post(
"/api/v1/nodes",
json={"type": "server", "label": "nas", "ip": "192.168.1.91"},
headers=headers,
)
).json()
device = (
await client.post(
"/api/v1/scan/pending",
json={"hostname": "nas", "ip": "192.168.1.91", "suggested_type": "nas"},
headers=headers,
)
).json()
payload = _state(design_id)
payload["devices"][0]["device_id"] = device["id"]
payload["devices"][0]["node_id"] = "node-that-was-deleted"
await client.post("/api/v1/racks/save", json=payload, headers=headers)
items = (
await client.get(f"/api/v1/racks/inventory?design_id={design_id}", headers=headers)
).json()["items"]
entry = next(i for i in items if i["id"] == device["id"])
assert entry["node_id"] == guessed["id"]
async def test_leaves_the_node_fields_empty_without_a_match(
self, client: AsyncClient, headers
):
design_id = await _design(client, headers)
await client.post(
"/api/v1/scan/pending",
json={"hostname": "pdu-main", "suggested_type": "generic"},
headers=headers,
)
items = (
await client.get(f"/api/v1/racks/inventory?design_id={design_id}", headers=headers)
).json()["items"]
entry = next(i for i in items if i["label"] == "pdu-main")
assert entry["node_id"] is None
assert entry["node_label"] is None
assert entry["node_design_name"] is None
assert entry["services"] == []
async def test_reports_the_services_a_canvas_added(self, client: AsyncClient, headers, db_session):
"""Services documented on a canvas reach the rack — they are the device's.
The scan found the host but no services; the user typed one on the
logical canvas, and the rack prints it without the node being consulted.
"""
design_id = await _design(client, headers)
db_session.add(
InventoryDevice(id="pd-noservices", ip="192.168.1.63", suggested_type="server")
)
await db_session.commit()
await client.post(
"/api/v1/nodes",
json={
"type": "server",
"label": "media",
"ip": "192.168.1.63",
"services": [{"port": 8096, "service_name": "jellyfin", "category": "media"}],
},
headers=headers,
)
items = (
await client.get(f"/api/v1/racks/inventory?design_id={design_id}", headers=headers)
).json()["items"]
entry = next(i for i in items if i["id"] == "pd-noservices")
assert entry["services"] == [{"port": 8096, "name": "jellyfin"}]
async def test_manual_entry_is_tagged_as_such(self, client: AsyncClient, headers):
created = (
await client.post(
"/api/v1/scan/pending",
json={"hostname": "patch-panel", "suggested_type": "generic"},
headers=headers,
)
).json()
assert created["discovery_source"] == "manual"
assert created["status"] == "pending"
listed = (await client.get("/api/v1/scan/pending", headers=headers)).json()
assert any(d["id"] == created["id"] for d in listed)
class TestRackSourcedInventory:
"""Gear created from a rack canvas shares the Device Inventory, but never a
logical canvas: it documents a mount, not a host."""
async def _rack_device(self, client: AsyncClient, headers, hostname: str = "patch-house"):
return (
await client.post(
"/api/v1/scan/pending",
json={"hostname": hostname, "discovery_source": "rack"},
headers=headers,
)
).json()
async def test_records_the_rack_source(self, client: AsyncClient, headers):
created = await self._rack_device(client, headers)
assert created["discovery_source"] == "rack"
assert created["discovery_sources"] == ["rack"]
async def test_rejects_an_unknown_source(self, client: AsyncClient, headers):
res = await client.post(
"/api/v1/scan/pending",
json={"hostname": "spoof", "discovery_source": "zigbee"},
headers=headers,
)
assert res.status_code == 422
async def test_approve_refuses_rack_gear(self, client: AsyncClient, headers):
created = await self._rack_device(client, headers)
res = await client.post(
f"/api/v1/scan/pending/{created['id']}/approve",
json={"type": "generic", "label": "patch-house"},
headers=headers,
)
assert res.status_code == 409
assert "rack" in res.json()["detail"].lower()
async def test_bulk_approve_skips_rack_gear(self, client: AsyncClient, headers):
rack_device = await self._rack_device(client, headers)
scanned = (
await client.post(
"/api/v1/scan/pending",
json={"hostname": "nuc", "ip": "192.168.1.77", "suggested_type": "server"},
headers=headers,
)
).json()
res = await client.post(
"/api/v1/scan/pending/bulk-approve",
json={"device_ids": [rack_device["id"], scanned["id"]]},
headers=headers,
)
body = res.json()
assert body["approved"] == 1
assert body["device_ids"] == [scanned["id"]]
skipped = next(s for s in body["skipped_devices"] if s["device_id"] == rack_device["id"])
assert skipped["match"] == "rack"
async def test_rack_gear_is_offered_to_the_rack_inventory(self, client: AsyncClient, headers):
design_id = await _design(client, headers)
created = await self._rack_device(client, headers, hostname="blank-panel")
items = (
await client.get(f"/api/v1/racks/inventory?design_id={design_id}", headers=headers)
).json()["items"]
assert any(i["id"] == created["id"] for i in items)
class TestDesignLifecycle:
async def test_deleting_a_design_removes_its_rack_rows(self, client: AsyncClient, headers):
keeper = await _design(client, headers, name="Keeper", design_type="network")
design_id = await _design(client, headers)
await client.post("/api/v1/racks/save", json=_state(design_id), headers=headers)
assert (await client.delete(f"/api/v1/designs/{design_id}", headers=headers)).status_code == 204
assert (await client.get(f"/api/v1/racks?design_id={design_id}", headers=headers)).status_code == 404
# The other design is untouched.
assert (await client.get(f"/api/v1/racks?design_id={keeper}", headers=headers)).status_code == 200
async def test_unmounting_keeps_the_inventory_entry(self, client: AsyncClient, headers):
design_id = await _design(client, headers)
created = (
await client.post(
"/api/v1/scan/pending",
json={"hostname": "sw-24", "suggested_type": "switch"},
headers=headers,
)
).json()
payload = _state(design_id)
payload["devices"][0]["device_id"] = created["id"]
await client.post("/api/v1/racks/save", json=payload, headers=headers)
# Unmount: the device disappears from the rack payload entirely.
unmounted = _state(design_id)
unmounted["devices"] = [d for d in unmounted["devices"] if d["id"] != "dev-sw"]
unmounted["cables"] = []
await client.post("/api/v1/racks/save", json=unmounted, headers=headers)
items = (
await client.get(f"/api/v1/racks/inventory?design_id={design_id}", headers=headers)
).json()["items"]
entry = next(i for i in items if i["id"] == created["id"])
assert entry["racked"] is False
async def test_copying_a_rack_design_duplicates_its_racks(self, client: AsyncClient, headers):
design_id = await _design(client, headers)
await client.post("/api/v1/racks/save", json=_state(design_id), headers=headers)
copy = (
await client.post(
f"/api/v1/designs/{design_id}/copy",
json={"name": "Rack Room (copy)", "icon": "server"},
headers=headers,
)
).json()
assert copy["design_type"] == "rack"
loaded = (await client.get(f"/api/v1/racks?design_id={copy['id']}", headers=headers)).json()
assert [r["name"] for r in loaded["racks"]] == ["Main"]
assert len(loaded["devices"]) == 2
assert len(loaded["cables"]) == 1
# Fresh ids, so editing the copy cannot touch the original.
assert loaded["racks"][0]["id"] != "rack-1"
assert {d["id"] for d in loaded["devices"]}.isdisjoint({"dev-sw", "dev-nas"})
# Cables still join the copied devices, not the originals.
copied_ids = {d["id"] for d in loaded["devices"]}
assert loaded["cables"][0]["from_device_id"] in copied_ids
assert loaded["cables"][0]["to_device_id"] in copied_ids
-352
View File
@@ -1,352 +0,0 @@
"""Tests for scan routes: trigger, pending devices, approve/hide/ignore."""
import uuid
from unittest.mock import AsyncMock, patch
import pytest
from httpx import AsyncClient
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.db.models import Node, PendingDevice, ScanRun
from app.services.scanner import run_scan
@pytest.fixture
async def headers(client: AsyncClient):
res = await client.post("/api/v1/auth/login", json={"username": "admin", "password": "admin"})
token = res.json()["access_token"]
return {"Authorization": f"Bearer {token}"}
@pytest.fixture
async def pending_device(db_session):
import uuid
device = PendingDevice(
id=str(uuid.uuid4()),
ip="192.168.1.100",
mac="aa:bb:cc:dd:ee:ff",
hostname="my-server",
os="Linux",
services=[{"port": 22, "name": "ssh"}],
suggested_type="server",
status="pending",
)
db_session.add(device)
await db_session.commit()
await db_session.refresh(device)
return device
# --- Trigger scan ---
@pytest.mark.asyncio
async def test_trigger_scan_requires_auth(client: AsyncClient):
res = await client.post("/api/v1/scan/trigger")
assert res.status_code == 401
@pytest.mark.asyncio
async def test_trigger_scan_creates_run(client: AsyncClient, headers):
with (
patch("app.api.routes.scan._background_scan", new_callable=AsyncMock),
patch("app.api.routes.scan.settings") as mock_settings,
):
mock_settings.scanner_ranges = ["192.168.1.0/24"]
res = await client.post("/api/v1/scan/trigger", headers=headers)
assert res.status_code == 200
data = res.json()
assert data["status"] == "running"
assert data["ranges"] == ["192.168.1.0/24"]
assert "id" in data
# --- Pending devices ---
@pytest.mark.asyncio
async def test_list_pending_empty(client: AsyncClient, headers):
res = await client.get("/api/v1/scan/pending", headers=headers)
assert res.status_code == 200
assert res.json() == []
@pytest.mark.asyncio
async def test_list_pending_returns_device(client: AsyncClient, headers, pending_device):
res = await client.get("/api/v1/scan/pending", headers=headers)
assert res.status_code == 200
data = res.json()
assert len(data) == 1
assert data[0]["ip"] == "192.168.1.100"
assert data[0]["hostname"] == "my-server"
# --- Approve device ---
@pytest.mark.asyncio
async def test_approve_device(client: AsyncClient, headers, pending_device):
node_payload = {
"label": "My Server",
"type": "server",
"ip": "192.168.1.100",
"hostname": "my-server",
"status": "unknown",
"services": [],
}
res = await client.post(
f"/api/v1/scan/pending/{pending_device.id}/approve",
json=node_payload,
headers=headers,
)
assert res.status_code == 200
data = res.json()
assert data["approved"] is True
assert "node_id" in data
# Device should no longer appear in pending list
pending_res = await client.get("/api/v1/scan/pending", headers=headers)
assert pending_res.json() == []
@pytest.mark.asyncio
async def test_approve_nonexistent_device(client: AsyncClient, headers):
node_payload = {
"label": "Ghost",
"type": "generic",
"ip": "10.0.0.1",
"status": "unknown",
"services": [],
}
res = await client.post(
"/api/v1/scan/pending/nonexistent-id/approve",
json=node_payload,
headers=headers,
)
assert res.status_code == 200
assert res.json()["approved"] is False
# --- Hide device ---
@pytest.mark.asyncio
async def test_hide_device(client: AsyncClient, headers, pending_device):
res = await client.post(f"/api/v1/scan/pending/{pending_device.id}/hide", headers=headers)
assert res.status_code == 200
assert res.json()["hidden"] is True
# Should no longer appear in pending
pending_res = await client.get("/api/v1/scan/pending", headers=headers)
assert pending_res.json() == []
# Should appear in hidden
hidden_res = await client.get("/api/v1/scan/hidden", headers=headers)
assert len(hidden_res.json()) == 1
# --- Ignore device ---
@pytest.mark.asyncio
async def test_ignore_device(client: AsyncClient, headers, pending_device):
res = await client.post(f"/api/v1/scan/pending/{pending_device.id}/ignore", headers=headers)
assert res.status_code == 200
assert res.json()["ignored"] is True
# Device should be gone from both pending and hidden
pending_res = await client.get("/api/v1/scan/pending", headers=headers)
assert pending_res.json() == []
hidden_res = await client.get("/api/v1/scan/hidden", headers=headers)
assert hidden_res.json() == []
# --- Scan runs ---
@pytest.mark.asyncio
async def test_list_runs_empty(client: AsyncClient, headers):
res = await client.get("/api/v1/scan/runs", headers=headers)
assert res.status_code == 200
assert res.json() == []
# --- run_scan: re-scan updates existing pending devices ---
MOCK_HOST = {
"ip": "192.168.1.50",
"mac": "aa:bb:cc:dd:ee:ff",
"hostname": "myhost.lan",
"os": "Linux",
"open_ports": [{"port": 8096, "protocol": "tcp", "banner": "Jellyfin"}],
}
@pytest.mark.asyncio
async def test_run_scan_creates_new_pending_device(db_session: AsyncSession):
run_id = str(uuid.uuid4())
run = ScanRun(id=run_id, status="running", ranges=["192.168.1.0/24"])
db_session.add(run)
await db_session.commit()
with (
patch("app.services.scanner._nmap_scan", return_value=[MOCK_HOST]),
patch("app.api.routes.status.broadcast_scan_update", new_callable=AsyncMock),
):
await run_scan(["192.168.1.0/24"], db_session, run_id)
result = await db_session.execute(
select(PendingDevice).where(PendingDevice.ip == "192.168.1.50")
)
device = result.scalar_one_or_none()
assert device is not None
assert device.hostname == "myhost.lan"
assert any(s["port"] == 8096 for s in device.services)
assert device.suggested_type == "server"
@pytest.mark.asyncio
async def test_run_scan_purges_stale_pending_for_canvas_nodes(db_session: AsyncSession):
"""Pending devices that were already in canvas before scan starts must be removed."""
node = Node(
id=str(uuid.uuid4()),
label="Existing Server",
type="server",
ip="192.168.1.50",
status="online",
services=[],
pos_x=0.0,
pos_y=0.0,
)
stale = PendingDevice(
id=str(uuid.uuid4()),
ip="192.168.1.50",
mac=None,
hostname=None,
os=None,
services=[],
suggested_type="generic",
status="pending",
)
db_session.add(node)
db_session.add(stale)
await db_session.commit()
run_id = str(uuid.uuid4())
run = ScanRun(id=run_id, status="running", ranges=["192.168.1.0/24"])
db_session.add(run)
await db_session.commit()
with (
patch("app.services.scanner._nmap_scan", return_value=[]),
patch("app.api.routes.status.broadcast_scan_update", new_callable=AsyncMock),
):
await run_scan(["192.168.1.0/24"], db_session, run_id)
result = await db_session.execute(
select(PendingDevice).where(PendingDevice.ip == "192.168.1.50")
)
assert result.scalar_one_or_none() is None
@pytest.mark.asyncio
async def test_run_scan_skips_ip_already_in_canvas(db_session: AsyncSession):
"""Devices whose IP already exists as a canvas Node must not appear in pending."""
node = Node(
id=str(uuid.uuid4()),
label="Existing Server",
type="server",
ip="192.168.1.50",
status="online",
services=[],
pos_x=0.0,
pos_y=0.0,
)
db_session.add(node)
await db_session.commit()
run_id = str(uuid.uuid4())
run = ScanRun(id=run_id, status="running", ranges=["192.168.1.0/24"])
db_session.add(run)
await db_session.commit()
with (
patch("app.services.scanner._nmap_scan", return_value=[MOCK_HOST]),
patch("app.api.routes.status.broadcast_scan_update", new_callable=AsyncMock),
):
await run_scan(["192.168.1.0/24"], db_session, run_id)
result = await db_session.execute(
select(PendingDevice).where(PendingDevice.ip == "192.168.1.50")
)
assert result.scalar_one_or_none() is None
@pytest.mark.asyncio
async def test_run_scan_skips_hidden_device(db_session: AsyncSession):
"""Devices previously hidden by the user must not re-appear in pending on re-scan."""
hidden = PendingDevice(
id=str(uuid.uuid4()),
ip="192.168.1.50",
mac=None,
hostname=None,
os=None,
services=[],
suggested_type="generic",
status="hidden",
)
db_session.add(hidden)
await db_session.commit()
run_id = str(uuid.uuid4())
run = ScanRun(id=run_id, status="running", ranges=["192.168.1.0/24"])
db_session.add(run)
await db_session.commit()
with (
patch("app.services.scanner._nmap_scan", return_value=[MOCK_HOST]),
patch("app.api.routes.status.broadcast_scan_update", new_callable=AsyncMock),
):
await run_scan(["192.168.1.0/24"], db_session, run_id)
result = await db_session.execute(
select(PendingDevice).where(
PendingDevice.ip == "192.168.1.50",
PendingDevice.status == "pending",
)
)
assert result.scalar_one_or_none() is None
@pytest.mark.asyncio
async def test_run_scan_updates_existing_pending_device(db_session: AsyncSession):
"""Re-scanning the same IP updates services instead of creating a duplicate."""
# Pre-existing pending device with no services
existing = PendingDevice(
id=str(uuid.uuid4()),
ip="192.168.1.50",
mac=None,
hostname=None,
os=None,
services=[],
suggested_type="generic",
status="pending",
)
db_session.add(existing)
await db_session.commit()
run_id = str(uuid.uuid4())
run = ScanRun(id=run_id, status="running", ranges=["192.168.1.0/24"])
db_session.add(run)
await db_session.commit()
with (
patch("app.services.scanner._nmap_scan", return_value=[MOCK_HOST]),
patch("app.api.routes.status.broadcast_scan_update", new_callable=AsyncMock),
):
await run_scan(["192.168.1.0/24"], db_session, run_id)
# Should still be only one device
result = await db_session.execute(
select(PendingDevice).where(PendingDevice.ip == "192.168.1.50")
)
devices = list(result.scalars().all())
assert len(devices) == 1
device = devices[0]
# Services and hostname should be updated
assert device.hostname == "myhost.lan"
assert any(s["port"] == 8096 for s in device.services)

Some files were not shown because too many files have changed in this diff Show More