Compare commits

..
92 Commits
Author SHA1 Message Date
Pouzor 574b84d843 chore: bump version to 3.4.1
ha-relevant: no
2026-09-07 17:08:19 +02:00
Pouzor 0ffece7a0d fix(inventory): widen the merge dialog so an IEEE is readable
The addresses are what the choice is made on, and `pve-proxmox-ai-113` was
clipped at the edge of a max-w-lg dialog. Wider, and the address line wraps
instead of truncating; the list grows with the viewport so a long selection
scrolls less.

ha-relevant: no
2026-09-07 17:05:05 +02:00
Pouzor 46dac1a89f feat(inventory): merge duplicate device rows, by hand or after an import
The inventory matched identity only when a row was created and never looked
again, so two rows describing one host — minted before either carried the
address that would have matched them — stayed separate forever. Documents,
racks and canvases then multiplied them.

Two repairs:

* `POST /scan/pending/merge` and a Merge action in the inventory's select
  mode: the user picks the survivor. The only way to collapse rows sharing
  no address at all, since a name is not an identity.
* `reconcile_duplicates`, run at the end of the Proxmox import and of a
  scan: collapses what a shared MAC proves. Never a shared IP alone, and
  never across two distinct IEEEs.

Merging keeps everything. The survivor holds every fact it had and fills its
gaps from the others; addresses, services, properties and sources union.
Canvas nodes, rack mounts, documents and mesh links are re-pointed before the
extras go, and each node's `display_view` gains the facts it never saw as
visible — a whitelist left alone would have rendered them hidden and made the
merge look like a no-op on the canvas.

Two bugs fell out on the way:

* `dedupe_nodes_by_device` deleted duplicate nodes while `rack_devices.node_id`
  and `documents.node_id` still named them. SQLite runs with foreign keys off,
  so the declared `ON DELETE SET NULL` never fired.
* The scanner's duplicate collapse deleted inventory rows outright, losing
  their facts and orphaning whatever drew them. It merges now; its guard for
  a second approved row is unchanged.

ha-relevant: maybe
2026-09-07 17:05:05 +02:00
Pouzor 0adb2619dd chore: bump version to 3.4.0
Version bump to 3.4.0 plus the CHANGELOG entry for the release:
the Documentation space, the rack faceplate owned by the Device
Inventory, Proxmox guest nesting, subpath serving, the MCP rack and
inventory tools, and the fixes since 3.3.5.

ha-relevant: no
2026-09-07 16:11:49 +02:00
Pouzor 40f4a705dc fix(canvas): keep the pan and zoom when coming back from Documentation
Opening Documentation unmounts the canvas, and React Flow remounts at the
default 1:1 viewport at the origin — the graph came back zoomed in and pushed
under the panels. fitView only runs on a fresh load, so nothing corrected it.

The store now remembers the viewport: every pan/zoom the user ends is saved,
so is wherever fitView lands, and the canvas mounts at that value. Switching
design still fits, since loadCanvas clears it.

ha-relevant: yes
2026-09-07 15:52:58 +02:00
Pouzor b40e2cc1db fix(docs): keep a document's URL across a refresh
Reflecting the section into the URL and reading it back on boot were two
effects, and the writer was declared first. It ran with the defaults —
canvas, no document — and deleted `view=docs&doc=` before the reader
looked, so refreshing or opening a shared link always landed on the
canvas.

Both sides now live in `useDocsUrlSync`. The URL's request is captured in
a lazy initializer at the first render, before any effect can touch the
address bar, and the writer stays quiet until that request has been
honoured — or until there was none to honour.

The restore also waits for the session, instead of fetching the document
against the login page, and is awaited so `doc=` is not dropped while it
loads. It settles on failure too: a document that no longer exists must
not freeze the URL, or the section could never be left. Standalone keeps
ignoring the params, since documents need the backend.

ha-relevant: maybe
2026-09-07 15:22:23 +02:00
Pouzor da058001b8 docs: describe the link picker and documents in the canvas search
Both shipped in the two commits before this one. The Editing section gains the
`[[` picker and why it holds a bracket back; the Search section gains the canvas
modal that now queries the same index; and "Not built yet" drops the two entries
that are no longer true.

ha-relevant: no
2026-09-07 15:22:23 +02:00
Pouzor d7924d5ed0 feat(docs): write a wiki-link from a picker instead of from memory
`[[VLAN plan]]` only resolves if you remember the exact title or slug, and
nothing in the editor told you either. Typing `[[` now opens a picker at the
caret: filter, pick, and the finished link is written.

It offers documents only. A device with no document is not a link target — a
link at one renders red — and creating a document from inside an unsaved editor
is a different feature; a device that is documented is in the list through its
document, labelled as such.

A target is addressed by title, or as `[[doc:<id>]]` when another document
shares that title. A bare link resolves by title, so an ambiguous one would
silently point at whichever document came first.

The second bracket is swallowed on purpose. Opening the menu moves focus to its
filter field, and a character typed as focus moves is not reliably delivered to
either box, so the body holds exactly one `[` and that is what the insertion
replaces. Cancelling puts the second bracket back, so what was typed survives
whichever way the menu closes.

The popover itself is extracted as EditorMenu and the `/` insert menu now uses
it too — same filter field over a labelled list, placed at the caret and
measured before it is shown, and two copies of that would have drifted.

ha-relevant: no
2026-09-07 15:22:23 +02:00
Pouzor 5aa6dedce0 feat(docs): find documents from the canvas search
Ctrl/Cmd+K searched nodes and pending devices. Everything written about them
was invisible to it, so the only way to a document was to leave the canvas and
walk the tree — and the body, the part worth searching, was not reachable at all
from there.

Documents now appear under the two existing groups, with the snippet the index
matched on, and picking one switches to Documentation and opens it.

Unlike the other two halves this cannot filter a list in memory: the tree holds
document metadata and never bodies, on purpose. So it queries the server's
index, debounced and skipped under two characters, and an answer is kept with
the query it answered rather than in a bare list — a slow reply can no longer
appear under a query the user has moved on from. A failing index costs the
document rows and nothing else.

Standalone passes no handler, and without one nothing is requested and the
placeholder does not mention documents.

ha-relevant: no
2026-09-07 15:22:23 +02:00
Pouzor cf18521032 fix(docs): stop the template menu opening under the sidebar
The New document popover is 256px wide and right-aligned on a button at the
right edge of the document tree — a column that is about as wide, resizable,
and vertically scrolling. Its left edge therefore landed outside that column,
where two things went wrong at once: setting overflow-y makes the horizontal
axis clip as well, and whatever escaped was painted under the app sidebar. Half
the template names were unreadable.

The menu is portalled to the body and positioned in viewport coordinates taken
from the trigger, with its left edge clamped to the viewport, so no tree width
can push it off screen and no ancestor can clip it. Placement sets top or
bottom from the trigger's own edge.

Two consequences of leaving the component's subtree: the outside-click check
now tests the trigger and the menu, and the menu closes on scroll or resize
rather than floating on at coordinates that no longer mean anything.

ha-relevant: no
2026-09-07 15:22:23 +02:00
Pouzor 4bcba084db docs: say the documentation space exists
The README and FEATURES.md described nineteen features and not the one that
writes the homelab down. A user reading either had no way to learn that every
device gets a document, that there is a Library, or that any of it is there at
all.

README gets a Documentation section next to the Rack Canvas one and mentions it
where it lists what Homelable does. FEATURES.md gets section 13, after Device
Inventory, which is where it belongs — the rest shift up one and the table of
contents follows. The link docs/rack-canvas.md holds into that file points at
Device Inventory, which does not move.

docs/documentation.md picks up the two capabilities that just landed, a
backlinks row in the API table, and a "Not built yet" list that is honest about
what is still missing: export/import, a print view, an open-tasks view, images,
documents in the canvas search modal, [[ autocompletion, and the MCP tools.

ha-relevant: no
2026-09-07 15:22:23 +02:00
Pouzor 761a1a526c feat(docs): reach a document's history, and see what links to it
Two things the documentation section stored and never showed.

**History.** Up to 50 revisions per document have been written since the
section shipped — on every save, restore, regenerate, scaffold and the notes
migration — and nothing reached them. RegenerateDocModal went as far as
telling the user the old body "is in its history", which was true and useless:
there was no way there. The header now carries a history button; the rail lists
each version by what caused it, selecting one replaces the body in place, and
Changes turns it into a line diff against the current body. Restore snapshots
the body it replaces first, so a restore is itself undoable.

A revision's body is fetched only when it is opened — the list carries a size,
not the text, which is what RevisionSummary was shaped for.

The diff is a plain LCS over lines after the common head and tail are trimmed,
with unchanged runs collapsed to gaps: no library, and a pathological pair of
long unrelated bodies degrades to "replaced wholesale" rather than building a
quarter-million-cell matrix.

**Backlinks.** A wiki-link only says where it goes. Every document now lists
what points at it, with the line the link was written on and the label it was
written as; repeated links from one document collapse into a count.

The inversion is answered by the server, because the browser holds no body but
the open one — the list endpoint is metadata-only so the tree can badge without
downloading the space. doc_backlinks.py therefore mirrors the resolution rules
in wikilinks.ts, the way the generator's service_url mirrors serviceUrl.ts, and
its tests pin them. The inventory is loaded only when a body actually carries a
[[device:…]].

The client-side backlinkIndex goes with it. It was written, tested and never
rendered; keeping a second copy of the resolution rules on the side that cannot
see every body is only drift.

ha-relevant: no
2026-09-07 15:22:23 +02:00
Pouzor 080de69543 fix(proxmox): keep a migrated guest on its existing inventory row
Excluding claimed rows from both fallbacks was too broad. The synthetic ieee
embeds the host name, so a live migration or a node rename rewrites it for what
is the same guest (`pve-pve1-802` → `pve-pve2-802`). Its exact match then
missed, MAC and IP both skipped the existing row as claimed, and the import
filed a second row — leaving the first orphaned on a host that no longer runs
the guest.

A MAC is an identity; an IP is not, and only the IP caused #419. So the MAC
fallback now matches a claimed row too, and `_adopted_ieee` re-points it to the
new host. Re-pointing is the half that matters beyond the duplicate: links are
rebuilt keyed on the ieee, so a row left on the stale one stops resolving as a
host→guest endpoint. A mesh ieee is a real hardware address and is never
overwritten.

Accepted trade-off, pinned by a test: two guests sharing a NIC MAC now collapse
into one row. That is a misconfiguration which already breaks their networking,
where the alternative orphans a row on every migration.

ha-relevant: no
2026-09-07 11:32:18 +02:00
Pouzor 29525bca87 fix(proxmox): match a guest by its own identity, not a shared IP
`_find_pending` matched ieee OR ip OR mac in one flat `or_()` with `.first()`
and no ordering, so an import merged into whichever row the database happened
to yield first. Two guests on one address — a duplicated static lease, a
re-used DHCP address, two guests behind one NAT — let one guest's import
overwrite the other's hostname, VMID and specs, non-deterministically. Canvas
nodes hold their own label, so the map kept showing the old name over the wrong
machine's facts and the failure was invisible where you would look for it.

Match the synthetic `pve-{host}-{vmid}` ieee first (unique per guest), then
MAC, then IP, and exclude fallback rows already claimed by a different Proxmox
guest. Remaining ties go to the oldest row, so a re-import is stable. The
docstring promised IP as tier 1; the code never implemented any tier.

Upstream, `_extract_qemu_ip` returned the first non-loopback IPv4 the agent
listed, which on a guest running Docker or k8s is the CNI bridge — an address
several guests report identically. Prefer the interface whose hardware-address
is the NIC MAC already parsed from /config one call earlier.

Latent for anyone whose API token cannot reach the QEMU guest agent: guest IPs
are None, so no row ever matched on one. It surfaces the first time the token
gets VM.GuestAgent.Audit.

Already-corrupted rows cannot be repaired — the overwritten facts are gone.
Affected users delete the bad rows and re-import.

Fixes #419

ha-relevant: yes
2026-09-07 11:32:18 +02:00
Pouzor 94f42cf9c5 feat(docs): edit a document's tags, and say they exist
Group by Tag has always been there, and every document was Untagged: tags come
from the frontmatter `tags:` key, nothing but hand-typed YAML ever set one, and
no generated document mentioned the key. Both templates did pass `tags: []` —
`render_frontmatter` dropped every empty value on the way out, so the one place
a user could have learned tags exist never printed. An empty list prints now. An
empty string still does not: `owner:` teaches nothing the template already says.

The chips under the title become the editor. Each carries a cross, and a `+ Tag`
chip opens a field that takes a whole list — "web, prod" is two tags — and
refuses one the document already carries whatever its case. It writes straight
through like starring: a chip clicked off is not a draft of the document.

The body owns tags, the column caches them, so the store rewrites the block and
saves the body rather than patching a field the next body save would overwrite.
`withFrontmatter` would have done it in one line and round-tripped everything
else through the YAML dumper — `created: 2026-09-07` comes back a full timestamp,
because that is what the date parsed to. So `withTags` replaces that one entry,
inline or block list, and leaves the other lines byte for byte as written.

`test_frontmatter_omits_empty_values` asserted the dropping this changes; it is
renamed and now asserts the empty list survives.

ha-relevant: yes
2026-09-07 10:38:24 +02:00
Pouzor b910b03318 feat(inventory): open a device's document from its detail modal
The canvas' detail panel could reach a device's documentation; the inventory
detail — the sheet that owns what a device *is* — could not, though it is where
a user reads and curates the same facts. Its header gains the link beside Edit,
calling the entry point the canvas already uses: fetch the listing if the
section was never opened, open the document the device has, write one from its
facts when it has none.

The link only shows in view mode: while editing, the header belongs to the form,
and leaving a dirty draft for another view is not what that button should mean.

Both dialogs close on the way out. Documentation is a whole view, not something
to read underneath the list it was opened from, and reopening the inventory
later should not restore a detail the user has left.

Backend-only, like the rest of the document surface.

ha-relevant: yes
2026-09-07 10:38:24 +02:00
Pouzor f0b5175155 fix(docs): stop the frontmatter regex going quadratic on an open block
CodeQL, high: a polynomial regular expression on uncontrolled data. Both
frontmatter parsers fenced the block with `\s*`, and `\s` matches the newline
the fence line ends with — so the closing alternative was reachable two ways
and every added line multiplied the backtracking. The pathological input is
a body that opens `---` and never closes it, which is what a document looks
like for as long as the user is typing that block: 20k lines took 2.4s, and
the body is parsed on every save.

Fencing on `[ \t]*` instead leaves one way to reach the newline and the same
input takes no measurable time. A trailing space on the fence still parses;
what is no longer accepted is a fence followed by blank lines before the YAML,
which nothing writes.

Both copies, and a timing regression test each side.

ha-relevant: yes
2026-09-07 10:38:24 +02:00
Pouzor a4d333b3d7 test(auth): assert the CORS warning against the derived origin
CodeQL's only open alert, `py/incomplete-url-substring-sanitization`: a URL
literal on the left of an `in` reads as a host check, and a substring match is
not one that can be made safely. The check here is a log assertion rather than
sanitization, so the alert is a false positive — but the assertion was weaker
than it needed to be either way, pinning the warning to a URL retyped beside
it. It now compares against the origin the settings themselves derive, which
is what the warning is supposed to name.

ha-relevant: no
2026-09-07 10:38:24 +02:00
Pouzor 6ba3d1a38f fix(docs): give the document editor a working undo
Ctrl+Z in the editor undid the canvas. The global handler computed whether the
event came from a field and then used that only to gate `?`, so undo fired
wherever the caret was — the document body, a node label, the filter box — and
took the canvas back a step while swallowing the keystroke the field wanted.
It now yields to whatever is being typed in. Which exposed a branch that had
never run: both undo and redo compared a lowercase `z`, and a shifted `z`
arrives as `Z`, so Ctrl+Shift+Z did nothing at all. It redoes now.

Handing the keystroke back is not enough on its own. The textarea is
controlled and half its edits are programmatic — the toolbar and the slash
menu rewrite the body around the caret — and setting a value that way is
exactly what empties the browser's own undo stack. So the editor keeps its
own: snapshots of body and selection, typing folded into one entry while it
keeps coming inside 500ms, an insertion always a step of its own, the redo
branch dropped once editing carries on past an undo, and the stack bounded.

ha-relevant: yes
2026-09-07 10:38:24 +02:00
Pouzor 56d83668d4 fix(docs): follow the title the body gives, in the row and in the tree
Editing `title:` in the frontmatter left the tree showing the name the
document was created with. `_apply_body` refreshed the frontmatter and tags
caches from the body and stopped there, though that line is the only place a
document is ever named — `rename` exists in the store and nothing in the UI
calls it. So the title column follows the body too, on every save and on a
restore, which brings back the name that body carried. It reslugs with it,
ignores a blank or non-string value, and an explicit title in the same request
still wins.

That alone would not have moved the row the user was looking at:
`buildDeviceTree` labelled a device leaf from the inventory whatever the
document said. Once a device is documented the row is that document, so it
carries the document's title, and falls back to the device's label only while
there is nothing to open.

ha-relevant: yes
2026-09-07 10:38:24 +02:00
Pouzor 7c116571e3 feat(canvas): open a node's document from its detail panel
A selected node linked to a device could reach its inventory row and nothing
else, though the same device now has a document too. The panel gains a second
link beside the first, and the store gains the action behind it: the canvas
knows a device id and nothing about documents, so `openForDevice` fetches the
listing if the section was never opened, opens the document the device has,
and writes one from its facts when it has none — the same thing picking that
device in the tree does, rather than switching to an empty section.

Which exposed what the inventory link had always done: open the modal, scroll
the card into view, and stop. "Open in inventory" means show me that device,
so the highlight now opens its detail modal as well. The id it honoured is
remembered, so a later filter change cannot reopen a detail the user closed,
and picker mode is left alone — it wants a device back, not a dialog stacked
on the one it was opened from. The scroll moved after the open and is called
optionally: a browser without `scrollIntoView` should not cost the caller the
modal it asked for.

Both links are backend-only, like the rest of the scan and document surface.

ha-relevant: yes
2026-09-07 10:38:24 +02:00
Pouzor b9ffa996c6 refactor(sidebar): drop the Hidden Devices shortcut
Hidden is a filter of the Device Inventory, not a place of its own, and the
modal already carries the Pending / Hidden toggle that switches between them.
A second sidebar entry for it spent a permanent nav slot on a view most users
open rarely, and made the two look like separate lists.

The one remaining trigger no longer needs a table to loop over, so
`PENDING_TRIGGERS` goes with it. `onOpenInventory` keeps its `status`
argument — the modal still opens straight into either filter.

ha-relevant: yes
2026-09-07 10:38:24 +02:00
Pouzor ed49c53969 feat(docs): put the Library first and let a drag file a document
The Library was the section below a device list hundreds of rows long, and
its two create buttons sat in a footer far from it. It is also the only part
of the tree the user owns — Devices is a pivot recomputed from the inventory,
so nothing there is filed by hand. So the Library moves to the top of the
pane, its heading becomes the collapse toggle, and the New document and New
folder buttons move up beside it. The footer keeps the coverage counter.

Filing was then the missing half: `parent_id` has always been settable and
`move` has always been in the store, with no UI reaching either. A Library row
now drags — by its icon, not the row, because a row-wide drag fights the click
that opens a document and the double-click that opens a folder. Folders take
the drop at any depth; the space around the list is a drop zone of its own,
which is the only way back out of a folder.

Every drop asks first, naming both ends, and the target folder is expanded
afterwards so the document does not vanish into a closed one.

`canMoveInto` decides the four rules the server already enforces — tree kinds
only, folders only as parents, no move to the parent it already has, no folder
into its own subtree — so an invalid drag refuses rather than asking, moving,
and failing on the response. `isDescendant` moves to `tree.ts` with it and is
re-exported from the store.

ha-relevant: yes
2026-09-07 10:38:24 +02:00
Pouzor 185d9bcacb feat(docs): regenerate a document, and only warn about real drift
A document is generated once and then owned by the user, which leaves no way
back when the generated header is what you actually wanted. The viewer gains a
Regenerate button behind a confirmation that says plainly the written body is
erased, names what it will be rebuilt from — the device's current facts, or the
page's template — and points at the one consolation: the replaced body is
snapshotted into the history first, so a regenerate is undoable from there.

`POST /documents/{id}/regenerate` re-runs the scaffolder, clears `edited_at`
so the document counts as "only the generated header" again, and refreshes
`facts_snapshot`. A folder has no generated body and is refused. The store
drops the localStorage draft with the body, so a stale edit cannot be saved
back over the new one.

Which surfaced the drift banner being wrong: it read "The device has changed"
on every device document, including one regenerated a second earlier. The
comparison ran in the UI, field by field, against the inventory wire shape —
and `facts_snapshot` is the server's shape, not that one. `properties` is a
flat `{key: value}` map in the snapshot and a `NodeProperty[]` on the wire, so
`(current ?? null) !== (value ?? null)` compared two objects by reference and
was true even when both were empty; `label` and `type` are stored through the
`device_name` / `device_type` fallbacks, so a device with no curated label
never matched its own snapshot either.

So the rule moves to the server, which is the only side that knows the shape:
`drifted` on the document summary, `doc.facts_snapshot != facts_snapshot(row)`,
the same comparison the coverage count already made. The listing resolves it
for every document in one query, which finally lights the tree's `drifted`
badge — `buildDeviceTree` has always taken the set and nothing ever passed it.

ha-relevant: yes
2026-09-07 10:38:24 +02:00
Pouzor 079098cdda fix(docs): read the hardware table out of the device properties
The Hardware table printed three em dashes directly above a Properties table
listing CPU Cores, RAM and Disk for the same device.

It read `cpu_count` / `cpu_model` / `ram_gb` / `disk_gb`, and those columns are
barely written any more: the scanner, the Proxmox import and the inventory
modal all put hardware in `properties` instead. Four rows out of 176 in a real
inventory still carry the columns; every Proxmox guest carries the properties.

Properties win now, with the columns kept as the fallback for a row that still
has them. Keys are matched case-insensitively, and the common synonyms are
accepted alongside what the importers write — Processor/CPU, vCPU/Cores,
Memory/RAM, Storage/Disk.

A property value is printed verbatim. It arrives already formatted ("6.1 GB"),
so appending a unit would say "6.1 GB GB" — and guessing one onto a bare number
would be inventing a fact.

`facts_snapshot` gains the properties to match. The hardware table now depends
on them and the properties table already printed them, so an edit to one is a
real change to what the document says and should raise the drift banner. It is
stored as a plain dict: the snapshot is compared against its own JSON round
trip, and a tuple would come back a list and read as drifted forever.

A document generated before this keeps the empty table it was born with —
generation happens once. Re-inserting the block with `/hardware` fills it.

ha-relevant: yes
2026-09-07 10:38:24 +02:00
Pouzor d0f0fe0322 fix(docs): stop the tree clipping the last letter of every name
A device with no document yet is drawn in italic, and italic glyphs lean past
the advance width — the width a shrink-wrapped box gets. `truncate` puts
`overflow: hidden` on that box, so the overhang was cut off and every
undocumented name lost the tail of its last letter: "bazarr" and "glpi" ended
mid-stroke.

Padding on the label gives the overhang somewhere to live. Names that really
are too long still ellipsise, and now carry a `title` so they can be read on
hover instead of just being cut. Group headers get the same treatment.

The padding itself is invisible to a test; the title is not, and it is what
makes an over-long name readable at all, so that is what the new DocTree tests
pin — along with the italic rule that caused the clipping in the first place.

ha-relevant: yes
2026-09-07 10:38:24 +02:00
Pouzor bf0d8077bb fix(docs): open the insert menu at the caret, and insert where the slash is
The menu was `absolute bottom-4 left-4` — pinned to the bottom-left of the
source pane, not to the caret. Typing `/` halfway up a long document opened it
at the bottom of the screen, and as the query narrowed nothing the list still
grew downward off the viewport.

It is measured against the caret now. A textarea exposes no caret geometry, so
`caret.ts` uses the mirror technique: a hidden div carrying the same text
metrics and box, filled to the caret, with a marker span read back for its
position. `placeMenu` then puts the popup under that line, flips it above when
it would run past the bottom, and clamps it into the pane when it fits neither
way — so it can no longer end up off-screen whatever the list length or the
pane height. That maths is a pure function with its own tests; the measuring
itself is not testable, because jsdom lays nothing out and reports every offset
as 0.

The same trace turned up a worse bug behind it. `insertAtCursor` derived its
anchor from `el.selectionEnd` at the moment the command was clicked, then
walked back `slashQuery.length + 1` to swallow the "/query". But the query is
typed into the menu's own field and never enters the document, and opening the
menu moves focus out of the textarea — freezing its selection wherever it sat,
and widening the gap with every keystroke. Inserting `/task` on a line after
"Intro" produced "Intr- [ ] " with no query, and ate the line outright with
one: the user's text was deleted, silently.

The index of the `/` is recorded when the menu opens and passed to the insert,
which consumes exactly that one character. `runCommand` reads it before its
await, so the reset effect cannot clear it out from under the insert.

ha-relevant: yes
2026-09-07 10:38:24 +02:00
Pouzor d52f698de3 fix(docs): bring an existing documents table up to the current shape
Opening the Documentation section against a database that already had a
`documents` table failed with `no such column: documents.edited_at`.

`Base.metadata.create_all` creates a table that is missing; it never adds a
column to one that is already there. `edited_at` landed after the table had
been created on a machine running the branch, so that database kept the older
shape and every query naming the column failed — including the one behind the
section's first request.

The repo convention is that column additions go through `_try_migrate`, and the
documents tables were the one place leaning on `create_all` for more than the
initial CREATE. Every nullable or defaulted column is now restated in
DOCUMENT_DDL as an idempotent ALTER, the way the `device_inventory` migrations
are, so an existing table is brought up to the current shape at boot whatever
shape it starts from.

The test fixture applied the same DDL with a raw `exec_driver_sql`, which was
fine for pure CREATE statements but turns each of those ALTERs into a fatal
"duplicate column" on the fresh database create_all has just built. It goes
through `_try_migrate` too now — which is what its own comment already claimed
it did.

ha-relevant: no
2026-09-07 10:38:24 +02:00
Pouzor 4ef0cdc73f fix(canvas): keep zone and group descriptions on save
A group's Description textarea wrote to `data.notes`, and everything typed in
it was lost on the next save, with no error.

3.3.0 moved the device facts off `nodes` onto `device_inventory`, `notes`
included, so the save routes `notes` to the inventory row. But a zone, a group
and a text box are canvas furniture: they draw nothing physical and never get a
row, and `link_facts` returns early for them. The text had nowhere to land, and
on reload `data.notes` came back undefined.

Zones were worse off — `GroupRectModal` never had a description field, and the
serializer hardcoded `notes: null` for them, so there was no way to describe a
zone at all.

Only full mode was affected: standalone keeps furniture nodes whole, so the two
modes disagreed about whether a description survived.

Furniture gets its own `nodes.description` column. It is canvas content, not a
device fact — a zone describes no device — so it saves with the canvas and
costs nothing in standalone. `link_facts` clears it on any node that does draw
a device, so `description` and the row's `notes` never compete for the same
text.

The column is added by `_try_migrate` and, importantly, is also listed in
`_NODE_COLUMNS_SQL` and `_NODE_KEPT`: `_drop_legacy_node_columns` recreates the
table from those two strings, so a column missing from them is silently lost on
the boot that finally drops the pre-3.3.0 columns — which is exactly the boot a
user who wrote a description in the meantime would not expect to lose it.
A migration test covers that path.

Zones get the Description field they never had, in `GroupRectModal`.
`NodeModal`'s free-text field reads "Description" and binds to the new column
for furniture, and "Notes" on the device row for everything else. Reads fall
back to `notes` so a canvas loaded before this change still shows its text, and
standalone hoists the old value once on load.

Also folds the two duplicate `FURNITURE_TYPES` sets in `standaloneStorage.ts`
and `subnet.ts` into one export, rather than adding a third copy.

ha-relevant: yes
2026-09-07 10:38:24 +02:00
Pouzor e6d338be13 feat(docs): a real document space for the homelab
The only writing surface was `device_inventory.notes`, a single TEXT column
rendered as plain preformatted text. No structure, no links, no search, and
nowhere to write anything that is not about one device.

This adds markdown documents, a navigable tree and a Documentation section in
the left rail — the first app-level view that is not a canvas.

Model — three tables. `documents` (kind: device / node / design / page /
folder), `document_revisions` capped at 50 per document, and a `documents_fts`
FTS5 index. A document attaches to the *inventory row*, not to a node, so one
device reads the same on every canvas. Every link is SET NULL and the title is
denormalized: deleting a device or a canvas must never destroy what the user
wrote. Foreign keys are off in this SQLite setup, so `doc_links.unlink_documents`
clears those columns by hand in the nodes, scan and designs delete paths.

FTS5 is optional. The SQLite shipped in the LXC and Docker images may not carry
it — `doc_search` probes once at boot and falls back to LIKE, and the search
response names the engine it used so the UI can drop snippet highlighting.

A device document is scaffolded once, from the scan facts: identity table,
hardware, services (one section per service), network, operations,
troubleshooting, dependencies, changelog. Then it belongs to the user and
nothing rewrites it. The old `notes` are appended verbatim under `## Notes`
behind a provenance comment, and `device_inventory.notes` is left untouched —
HACS, the YAML export, MCP and canvas search still read it.

Drift is surfaced, never corrected: `facts_snapshot` powers a banner, and
`GET /documents/blocks` re-renders one generated block from current facts for
the editor's slash commands. `review_every` plus `reviewed_at` produce a
"needs attention" bucket, because homelab docs die of staleness.

The Devices tree is pivoted client-side from the already-loaded inventory —
zone, group, type, physical/virtual, subnet, rack, vendor, discovery source,
status, tag, A-Z — so re-pivoting costs no request. The Library beside it is
real folders the user makes. The Canvas pivot is scoped to the loaded canvas:
`NodeData` carries no `design_id`, so naming every canvas a device appears on
would be a guess.

Saving is explicit, per the product rule. The dirty body is mirrored to
localStorage and offered back on reopen; a draft whose base no longer matches
the document is discarded rather than replayed.

react-markdown is loaded behind React.lazy, so the canvas path does not pay for
it: the main bundle is unchanged and the section brings its own 199 kB chunk.
Raw HTML is not rendered, which removes the XSS surface and means no sanitizer.

Export/import, a print view, an aggregated open-tasks view and images inside a
document are deliberately not in this lot. MCP tools are lot 2.

ha-relevant: yes
2026-09-07 10:38:24 +02:00
Pouzor 8612f453d4 chore(deps): clear every Dependabot alert in the frontend lockfile
All eight advisories were transitive and dev-only, reached through
`eslint` and through `shadcn` -> `@modelcontextprotocol/sdk`. Nothing
here ships in the runtime bundle.

- fast-uri   3.1.5  -> 3.1.7   (#71 #70 #67 #66, high)
- @humanfs/node 0.16.7 -> 0.16.8 (#69, medium)
- qs         6.15.2 -> 6.16.0  (#72 #68, auto-dismissed)
- nanoid     3.3.16 -> 3.3.18  (#62, auto-dismissed)

The qs and nanoid alerts were auto-dismissed as dev dependencies, but
the bumps are free and keep `npm audit` at zero.

Lockfile only -- no `package.json` change, no direct dependency moved.
`@humanfs/types` 0.15.0 and `@eslint/config-array` 0.19.2 come along as
resolution side effects.

ha-relevant: no
2026-09-05 01:29:19 +02:00
Pouzor e1e7b5089b fix(scan): never collapse a second approved row into another
An approved inventory row is what canvas nodes and rack mounts point at,
under `ON DELETE SET NULL`. Both collapse paths kept the first approved
row of a duplicate group and deleted the rest, so a group holding two of
them lost one device's links silently — and grouping by shared address
rather than by the exact `ip` string made that group easy to form: an
approved `10.0.0.1` beside an approved `10.0.0.1, 10.0.0.2` is one.

`_collapse_targets` now picks the merge target and the rows that may go.
With one approved row nothing changes. With two, every approved row
stands and only the pending ones collapse: a background scan is not the
place to decide which canvas-linked device loses its links, and the pair
stays visible in the inventory for the user to resolve.

ha-relevant: yes
2026-09-05 01:14:47 +02:00
Pouzor df60664506 fix(scan): match a scanned host against every declared IP of a device
An inventory row may hold several comma-separated addresses — a router's
LAN and VLAN legs, a host behind a VIP. The user declares them by hand
precisely so a scan folds into the existing row, and every other matcher
already honours that: `find_device_for`, `node_dedupe`, the status
checker and the inventory modal's own placeholder all read the field as
a list.

The scanner did not. `process_host` matched on `InventoryDevice.ip == ip`,
an exact string compare a multi-address row can never satisfy, so it fell
back to the MAC — which differs across VLAN interfaces, bridges and
keepalived VIPs. Result: a fresh pending row for the same device on every
run, no matter how the user had annotated it.

- Match per token: narrow with `contains`, then confirm the token in
  Python. The confirmation is load-bearing — "10.0.0.4" is a substring of
  "10.0.0.40", and the rows a match does not keep are deleted, not skipped.
- Build `hidden_ips` from tokens, so hiding a multi-address device hides
  it at every one of its addresses.
- Group `_dedupe_pending_by_ip` by shared token, transitively, so
  declaring an extra address also collapses the duplicate row it already
  spawned.

`_ip_tokens` becomes public `ip_tokens` rather than growing a fourth copy.
A scan still never appends an address on its own: `keep.ip = keep.ip or ip`
is unchanged, and declaring the extras stays a user action.

Closes #373

ha-relevant: yes
2026-09-05 01:14:47 +02:00
Pouzor 87081ac926 fix(mcp): key node properties on key, not name
create_node and update_node advertised a property as {name, value}. The
backend keys properties on `key` — merge_properties unions on it and
apply_view orders the node's view by it — so every property an MCP client
sent arrived keyless, collapsed onto the empty key, and a node given
three of them drew one. The schema also *required* `name`, so a correct
{key, value} payload was rejected outright.

Only MCP clients were affected: the frontend has always sent `key`.

`icon` and `visible` are exposed too, the other two fields of a
NodeProperty, so a property can be created the way the UI creates one.

While here, document that `status` on an update is a live observation
rather than a setting: link_facts keeps the freshest one instead of the
last writer, so it only lands while the device's status is still unknown.
The tool silently no-opped and its schema said nothing about it.

ha-relevant: no
2026-09-04 15:10:05 +02:00
Pouzor 4c8831d5df feat(mcp): expose the rack canvas and the inventory write routes
The MCP server covered the logical canvas only. A client could create a
design whose design_type is `rack` and then had no way to put a rack, a
mount or a cable in it, and it could triage the Device Inventory but
never create, edit or delete an entry — though every route already
existed. 24 tools become 47.

Racks (mcp/app/racks.py), 13 tools: list_racks, get_rack,
list_rack_inventory, list_faceplates, create_rack, update_rack,
delete_rack, mount_device, mount_accessory, unmount_device, move_device,
set_device_faceplate, patch_cable, unpatch_cable.

`POST /api/v1/racks/save` is full-state upsert-and-prune, so every write
is load, mutate, save it all back. Two traps are encoded once, in the
helpers: the response rows carry `design_id` and the save schemas do not
(echoing one back verbatim is a 422), and the viewport has to be echoed
or the save resets the user's pan and zoom. The plate, its colour and its
ports need no special handling — /racks/save writes those through to the
inventory row itself.

Two frontend files are copied rather than imported, since the MCP server
is a separate service:

  - mcp/app/faceplates.py — the 21-plate catalog from
    frontend/src/rack/faceplates.ts, data only. `bank()` reproduces the
    port arithmetic exactly, so a plate seeded here lands its ports where
    the renderer draws them.
  - mcp/app/rack_layout.py — canPlace / findSlot / freeUnits from
    frontend/src/rack/layout.ts, to *choose* a slot. The backend still
    polices the one it is sent.

Both are guarded against drift: test_faceplates_sync.py parses
faceplates.ts and compares every plate down to each port's coordinates,
the way test_node_types_sync.py guards NODE_TYPES, and
test_layout_parity.py runs the frontend's own layout cases against the
Python port.

Devices (mcp/app/devices.py), 10 tools: create_device, update_device,
delete_device, bulk_approve_devices, bulk_hide_devices,
bulk_restore_devices, rescan_device, list_proxmox_children,
get_scan_config, update_scan_config.

Verified against a running backend: the rack flow end to end (create,
mount, patch, move, swap a plate, shrink, unmount) plus every refusal
path, and the device tools including a byte-identical scan-config
write-back.

ha-relevant: no
2026-09-04 15:10:05 +02:00
Pouzor 6b11e7ed34 fix(config): unwrap quoted secrets that podman-compose passes through
bcrypt hashes and most OIDC client secrets contain `$`, so `.env.example`
tells operators to single-quote them or Docker Compose eats the `$`. Docker
Compose then strips those quotes itself when reading `env_file`, and so does
python-dotenv on the `env_file=` path — but podman-compose (1.0.6) forwards
them verbatim. The container receives `'$2b$12$...'`, quotes included, every
bcrypt verify fails, and the logged error blames the operator's hash.

Unwrap one matched pair of surrounding quotes from SECRET_KEY,
AUTH_PASSWORD_HASH and OIDC_CLIENT_SECRET before validating them, so the same
.env works under either container runtime. Only a matched pair is removed: a
value with a single stray quote still reaches the existing validation instead
of being silently patched up, and SECRET_KEY is unwrapped before the 32-byte
OIDC floor is checked so quotes cannot pad a short key past it.

Fixes #410

ha-relevant: no
2026-09-04 13:40:25 +02:00
Pouzor 4e8d841807 feat(deploy): serve Homelable under a configurable base path
Homelable could only own the root of an origin. Behind an existing proxy
at `https://home.example/homelab/` the built `index.html` still asked for
`/assets/*`, which fell through to whatever owned the root — and when
that answered `text/html` for a `<script>` under `nosniff`, the browser
failed the load on an HTTP 200. The only workaround was patching the
checkout and rebuilding on every upgrade.

One build-time knob, `VITE_BASE_PATH`, becomes Vite's `base`. Vite
rewrites the asset URLs it emits; everything the app builds by hand goes
through the new `utils/basePath.ts` — the axios instances, the WebSocket
URL, the live-view route, the local brand icons, and the OIDC login href.
`resolveServerPath` covers what the *backend* hands back, which is always
root-absolute because it cannot know where the SPA is mounted: uploaded
floor-plan URLs already stored in a canvas are resolved at render time,
so plans predating the move keep loading. The OIDC callback used to
redirect to `/`, dropping subpath users at the origin root after login;
it now reads the prefix back out of `OIDC_REDIRECT_URI`.

The default is `/`, and stays a no-op there by construction: every helper
returns the string it returned before, the root build output is unchanged
and both nginx sites are byte-identical to what shipped — the Docker
image copies `docker/nginx.conf` verbatim and the installer keeps its
original heredoc. Only a non-default prefix takes the generated config.

Those generated configs use `root`, never `alias`, since `alias` plus
`try_files` mis-resolves `$uri` — the Docker build lands the bundle in
the matching subdirectory, and the installer symlinks it under
`/var/www/homelable`. Both accept either reverse-proxy style, prefix
forwarded intact or already stripped, with no redirect loop between them,
and `absolute_redirect off` stops the no-slash 301 from eating the port.

Closes #334.

ha-relevant: maybe
2026-09-04 11:23:16 +02:00
Pouzor 3cbda60277 feat(icons): add the Homelable logo to the brand icon picker
The brand catalog is the generated homarr-labs dashboard-icons manifest,
which carries no Homelable mark, so a user cannot label the app itself on
a canvas. Serve a small set of logos from `public/brand/` instead: the
picker prepends `LOCAL_BRAND_ICONS` to the manifest and `brandIconUrl`
routes those slugs at the local asset rather than the jsDelivr CDN.

`dashboardIcons.json` stays untouched — `scripts/fetch-dashboard-icons.mjs`
overwrites it wholesale.

Closes part of #397 (the Docker host polling request is untouched).

ha-relevant: yes
2026-09-04 10:21:32 +02:00
Pouzor 03e5ae939e ci(security): give pip-audit the same registry-outage retry as npm
pip-audit queries PyPI's advisory API and exits non-zero when it cannot
reach it, exactly as `npm audit` does — the same class of outage that
reddened #409 would have failed the Python half too.

Both now go through .github/scripts/audit-with-retry.sh, which retries
only when the output names a transport or availability failure and lets
a real finding fail on the first attempt, unretried. The npm loop added
in the previous commit is folded into it.

ha-relevant: no
2026-09-04 10:01:35 +02:00
Pouzor 37a3aee9cc ci(security): retry the npm audit when the registry endpoint is down
`npm audit` exits 1 both for a real advisory and for a registry that
refuses to answer, so a 503 from the audit endpoint reddened PRs that
changed no dependency at all — as it did on #409.

Retry up to three times, but only when the output names a transport or
availability failure. An advisory at or above high severity still fails
on the first attempt, unretried.

ha-relevant: no
2026-09-04 10:01:35 +02:00
Pouzor 37f893abe6 fix(rack): keep the saved pan/zoom when switching back to a rack canvas
Leaving a rack design for another canvas and coming back put the racks
at the wrong pan/zoom until a hard reload. Two causes, both fixed here.

The rack flow shared the App-level `ReactFlowProvider` with the logical
canvas, so that canvas' transform and pane size carried over into it.
`RackCanvas` now wraps its own provider.

The restore effect was also spent before it could work: a switch back
remounts the component before `App` calls `loadDesign`, so the first
pass applied the pre-load viewport and then locked itself out for that
design. The marker now clears whenever a load starts, and the apply
waits a frame so React Flow has measured its pane.

Closes #408

ha-relevant: no
2026-09-04 10:01:35 +02:00
Pouzor f51dd131c1 refactor(rack): tidy the device modal's last row and its actions
Status and the colour override shared a column each, one under the other,
while the pair fits on one row like the placement fields above them. The
footer sat flush against the last section, so Save read as part of it, and
a section rule carrying the faceplate button sat lower than the plain rule
beside it.

- Status and Colour override share a two-column grid.
- A rule separates the form from Unmount / Cancel / Save.
- SectionHeader is a fixed-height row, so an `aside` button no longer
  pushes its border down.

ha-relevant: yes
2026-09-04 01:40:34 +02:00
Pouzor 4b90c60cc6 refactor(rack): put the plate picker on the faceplate rule
The Faceplate field sat in the Information column, a two-line block whose only
job was to open the catalog — a long way from the plate it changes, and the
last thing left between the label and the placement fields.

It is now a small button on the right end of the Faceplate rule, above the
drawing it affects: the current plate names itself, and `Change` says the name
is a button rather than a read-out. The name truncates first, so a long one
("Switch 24 ports + 2 SFP") never pushes the verb off. The warning about a plate
swap dropping its ports moved with it, between the rule and the plate.

ha-relevant: yes
2026-09-04 01:40:34 +02:00
Pouzor b26e9d3137 refactor(rack): rebuild the device modal around a headed strip
The plate used to sit alone above the form at a fixed 660px, and what the
mount actually stands for was buried under the port list at the bottom of the
right column — the two things the user checks first, at opposite ends of the
dialog.

They now share a strip across the top: the linked device on the left, the plate
on the right. The plate fills the half it is given instead of a constant, which
is what made port placement so fiddly — a full-width switch drew at two thirds
of the space with sockets to match. `PortPositionEditor` measures its container
and derives the scale from that; `fullWidth` stays as an explicit override, and
both fall back to the old default where nothing can be measured.

The form is split by the same section rules the logical canvas' `NodeModal`
draws — Linked device, Faceplate, Information, Placement, Ports — so the two
editors read the same way. `SectionHeader` moves to its own component and takes
an `aside`, which is where the mount status now rides. An accessory has no
ports and no inventory row, so it gets neither heading.

Port rows go from two columns to three; a 24-port switch no longer runs past
the fold.

ha-relevant: yes
2026-09-04 01:40:34 +02:00
Pouzor 4ccad61e73 feat(rack): let a mount say when it shows its ports, and never end a cable on hidden metal
Two things about when a faceplate draws its sockets.

A mount now carries `portVisibility`: `auto` leaves the faceplate in charge —
switches and patch panels permanently, everything else on focus — while
`always` and `hover` override that either way. The selector sits in the port
section of the device modal. It is a canvas display choice, so it lives on the
rack mount and never reaches the inventory row the mount points at; `auto` is
the default, so nothing about existing racks changes.

The fix underneath: with the cabling overlay on `hover`, hovering a switch drew
its runs, but the far end landed on a plate that was not itself focused and so
drew no socket — a cable stopping on blank metal. A port a *drawn* cable ends
on is now always rendered, on both plates, whatever the plate rule or the mode
says. `cableVisibility.ts` owns the question of which cables are on screen and
`CableLayer` draws from the same answer, so the two can no longer disagree.

Fixes #403

ha-relevant: yes
2026-09-04 01:40:34 +02:00
Pouzor 6fa1b1a3d0 fix(scan): never send an HTTP GET to a raw printing port
A printer treats whatever bytes land on TCP/9100 as a print job, so our
probe's "GET / HTTP/1.1" came out of the tray as a page of plaintext.

Two paths reached it, and neither could be configured away — 9100 is
hard-coded in the scanner's port list, and user ranges only ever add to
that list:

- the deep-scan HTTP probe, once per scan;
- the status checker, every 60 s for as long as the node exists, which is
  the one that turns a scan annoyance into a ream of paper.

Both already had a non-HTTP port denylist; 515 and 9100-9107 were simply
missing from it. nmap ships `Exclude T:9100-9107` for this exact reason,
which is why the -sV pass never triggered it and only our own probes did.

Node Exporter also lives on 9100 and loses its status check as a result.
A grey dot is the cheaper mistake. Its fingerprint entry is port-match
only, so discovery and labelling are unaffected.

Fixes #404

ha-relevant: yes
2026-09-03 23:15:36 +02:00
Pouzor caaa408618 fix(rack): stop a clamped size from shrinking the device everywhere
The write-through was asymmetric. A load applies the inventory row's
height and width to a mount only where they still fit that rack, so a
mount can legitimately be drawn smaller than the row says. A save then
copied the mount's size back onto the row unconditionally — so opening a
rack where a 4U device sits too high, changing anything at all and
saving turned that device into a 1U everywhere, silently. The rack store
did the same in the live session.

Size now travels only when the save actually changed it; an unchanged
size is an echo of the clamp and is dropped. The plate, its colour and
its ports keep going through unconditionally: none of them can overflow
a rail.

The other half of the overlay had no collision check, so a model bigger
than the mount could be applied on top of the neighbour above it —
drawing two plates on the same U, a state the canvas itself refuses and
`RackSaveRequest` happily stored. The overlay now obeys the placement
rule too, and the save endpoint rejects overlapping mounts in one rack
(devices in different racks, and half-width plates side by side on one
U, still pass).

ha-relevant: no
2026-09-03 20:58:20 +02:00
Pouzor bffe1927dd refactor(rack): lead with the faceplate, and make the port list a chip grid
Three things the port editor got wrong on a real switch:

The plate sat under the form, so the thing being edited was the thing you
had to scroll to. It leads now, in both the rack device modal and the
inventory one.

The arrow keys only moved a port while its handle held focus. Selecting a
port in the list and reaching for them — the obvious gesture — did
nothing. Placement mode now listens for as long as it is on and moves the
selected port, stepping aside for a text field being typed in.

And the list drew one full-height row per port: a 24-port switch ran a
metre down the modal and buried the panels under it. Each port is now a
28px chip (borderless name, bare type select, remove) in a two-column
grid, so the same switch is a list rather than a page.

ha-relevant: no
2026-09-03 20:58:20 +02:00
Pouzor f933f774fc feat(inventory): draw and edit a device's rack faceplate from its inventory row
The row already owns the front panel; until now only a rack canvas could
show it. The device detail modal draws the plate full size for any device
some rack has modelled, and its edit mode carries the whole model —
faceplate, height, width, colour override and the port list, placement
mode included — writing back through `PATCH /scan/pending/{id}`. A device
therefore no longer has to be racked, or racked again, to have its ports
named and placed.

The port list is now one component (`PortListEditor`) instead of a copy
in each modal, since both edit the same list on the same row.

A device with no modelisation gets no section and no `rack_*` fields in
the PATCH: editing an ordinary host must not silently give it a plate.
Creating a model from the inventory is deliberately left out — a plate is
still born on a rack.

ha-relevant: no
2026-09-03 20:58:20 +02:00
Pouzor b658a68a09 feat(rack): let the inventory own the front panel, and place ports by hand
The Device Inventory row now owns the rack modelisation — faceplate, U
height, column span, colour override and the port list, positions
included. A device therefore wears the same front panel in every rack it
is mounted in; only its placement, pinned status and label stay per
canvas. `POST /racks/save` writes it through, `GET /racks` overlays it on
load, and size is applied only where it still fits the mount's own rack:
geometry is global, a rack is not.

Ports were also unplaceable. Adding one dropped it on the middle of the
plate, so three added ports drew as one socket, and the only preview was
a 160×18px thumbnail that lost its label and stacked its ports. The
modal is wider, the plate is drawn full size below the form, and a
Position mode turns it into the port editor: drag to place, arrow keys
to nudge, per-axis snap to the nearest peer so banks stay in line.

Accessories stand for no inventory row, so they carry no ports and get no
port UI. Standalone has nothing to hang a global panel on, so ports stay
per mount there.

ha-relevant: no
2026-09-03 20:58:20 +02:00
Pouzor 6e2ccf7b7f fix(canvas): make the basic edge animation direction independent of geometry
The basic dash flipped to `animation-direction: reverse` whenever the source
sat below the target on screen, so the same edge marched one way or the other
depending only on where its nodes had been dragged.

The flip was compensating for the keyframes: homelable-basic-dash decreased
stroke-dashoffset while homelable-snake and homelable-flow increase it, so the
basic dash ran against the other two modes. Flip the keyframes to match and
drop the geometry test.

Does not address the underlying handle bug: the invisible oversized target
handle in SideHandles wins the pointerdown, so a drag from A to B is stored as
source=B, target=A. That is why all three modes now march target to source.

Closes #395

ha-relevant: yes
2026-09-03 02:46:58 +02:00
Pouzor fb39663293 refactor(inventory): rename the Approve action to "Put in current Canvas"
"Approve" described a moderation step the inventory no longer really has —
a row is not judged, it is placed on the canvas the user is looking at. The
new label says what the button does and which canvas it targets.

Renames the button in the device detail modal and the bulk button in the
inventory grid, plus the success toasts and the walkthrough step that spoke
of "approved devices". The API paths, the `status` value and every handler
name keep the word: it is a published contract the MCP server also calls.

ha-relevant: no
2026-09-02 21:14:44 +02:00
Pouzor 54176983b6 fix(proxmox): no host link for a guest nested in container mode
A nested guest was still getting the host -> guest 'virtual' edge, which
loops out of the container box and back into it — the screenshot on #399.
Being inside the box already says the host runs it.

Both entry points drop the link only for guests they actually nest: the
import modal path skips the edge in App's handler, and the inventory path
filters the host <-> guest pairs out of the edges bulk-approve returns.
Linked mode, loose guests and cluster edges are untouched.

ha-relevant: maybe
2026-09-02 21:14:44 +02:00
Pouzor deff7a34d0 feat(proxmox): optionally nest guests inside their host on the canvas
Two entry points now offer container mode, where a Proxmox host is drawn as
a box holding its VMs and LXCs instead of sitting beside them.

Import modal: an opt-in "Nest guests inside their host" checkbox, shown only
in "Inventory + canvas" mode. `onAddToCanvas` carries the choice as a
`ProxmoxCanvasMode` ('linked' | 'container').

Device Inventory: approving a Proxmox host opens a new ProxmoxApproveModal
asking whether its guests come along, and nested or linked. The guests come
from a new `GET /scan/pending/{id}/proxmox-children`, which resolves the
host -> guest `device_inventory_links` the Proxmox import records (cluster
links excluded, hidden rows skipped). A host with no guests, or a failed
lookup, falls through to the plain single approve.

Placement lives in the new `utils/proxmoxContainerLayout`: `groupProxmoxGuests`
splits a selection into hosts, guests and loose nodes (one parent per guest),
and `layoutProxmoxContainers` grids the guests inside their host box, hosts
side by side, anything left over below. Positions are absolute — `addNode`
turns them relative when it nests. A host skipped as a same-canvas duplicate
leaves its guests loose rather than orphaning a parent_id.

Also extracts `inventoryNodeData()` so the single, bulk and Proxmox approve
paths build node data from one place.

ha-relevant: maybe
2026-09-02 21:14:44 +02:00
Pouzor e988427bb2 docs: explain why Docker bridge networking yields no MAC addresses
The default compose files put the backend on a bridge network, where ARP
only ever reaches the Docker gateway — so scans report IPs, ports and
services but never a MAC. NET_RAW does not help: it grants raw sockets,
not a place on the LAN. Rescan matching prefers MAC over IP, so this also
makes a DHCP device reappear as a new inventory entry when its lease
changes.

None of this was documented anywhere. Adds an INSTALLATION.md section
covering the network_mode: host fix, the Docker Desktop macOS/Windows
limitation and the macvlan alternative, a pointer from the README scanner
section, and a commented-out network_mode: host in both compose files so
it is a one-line change.

Reported in discussion #368.

ha-relevant: no
2026-09-02 15:58:07 +02:00
Pouzor a1e7eeb309 chore(deps): patch dependabot alerts in the frontend lockfile
Five open advisories, all transitive through devDependencies (shadcn CLI
and @vitejs/plugin-react), none reachable from the shipped bundle:

- browserslist  4.28.2 -> 4.28.8  (high, GHSA prototype write in normalizeStats)
- postcss-selector-parser 7.1.1 -> 7.1.5  (low, AST recursion DoS)
- hono 4.12.33 -> 4.13.5  (memo() SSR leak, Connection header, i18n DoS)

Lockfile only; no package.json range changed.

ha-relevant: no
2026-09-02 14:26:01 +02:00
Pouzor 97472b65ba fix(ci): run the install smoke test under bash, not dash
The runner hands container steps `sh -e {0}` unless told otherwise, so
every step died on `set: Illegal option -o pipefail` before the installer
ran. Pin the job's shell to bash; `[[ … ]]` in the assertions needs it too.

ha-relevant: no
2026-09-02 02:15:10 +02:00
Pouzor c5a49537ed test(ci): smoke-test the bare-metal installer in a debian:12 container
ShellCheck reads the installer but never runs it, so it cannot catch a
`set -euo pipefail` abort — the failure class that broke this script on a
fresh host. This job executes it for real in debian:12, which carries no
Node and gives the job no TTY, so both the nodesource path and the prompt
fallbacks are exercised on every run.

It asserts what the script promises: SECRET_KEY generated, the bcrypt hash
and the JSON values single-quoted (the systemd EnvironmentFile trap),
.env at mode 600, SQLITE_PATH under the install dir, the venv and the Vite
build present, the service user created, the unit's ExecStart and
EnvironmentFile correct. It then boots the backend by the unit's own
ExecStart and waits on /api/v1/health, which proves the generated .env
actually parses, and re-runs the installer to check the idempotency claim
leaves .env untouched.

systemd is out of reach in a container: systemctl is stubbed, so the unit
is written but never started, and the EnvironmentFile parse itself stays
untested. The job is gated on paths, since it costs roughly six minutes.

Also adds gnupg to the installer's apt list — the nodesource setup script
needs it and a minimal Debian does not have it.

Verified by running the same steps locally in debian:12: all assertions
pass, the backend answers /api/v1/health, and the second run keeps .env.

ha-relevant: no
2026-09-02 02:15:10 +02:00
Pouzor cefb96ddb4 fix(install): stop the bare-metal installer aborting on a fresh host
Two ways install-baremetal.sh failed on exactly the hosts it targets, both
caused by `set -euo pipefail` turning an expected non-zero into an exit.

Node detection ran `node --version` in a command substitution. On a host
with no Node — the normal case, since the base apt list does not install
it — that exits 127, pipefail propagates it, and the script died at that
line, before the block that installs Node. `|| true` plus a numeric
fallback, and the version is stripped to digits so `-lt` cannot error on
unexpected output. The same abort applied to the `ip` and `hostname -I`
substitutions, which now tolerate failure too.

The prompts had no TTY guard, so the documented `curl … | sudo bash` form
hit EOF on the first `read` and aborted. Both prompts are now skipped when
stdin is not a terminal, falling back to their defaults with a warning.
Two `[[ … ]] && …` one-liners were the same trap in miniature — a false
test is a non-zero list — and are now if/fi.

Docs follow the behaviour: the piped install is shown with ADMIN_PASSWORD
and SCANNER_RANGES set, and the fallback (password `admin`, guessed range)
is stated rather than implied.

Verified by running the patched blocks under `env -i` with no node on PATH
and stdin closed: both reach the end, exit 0. Shellcheck and `bash -n`
clean.

ha-relevant: no
2026-09-02 02:15:10 +02:00
Pouzor 9998b3784f feat(install): native bare-metal install path, no Docker
Adds scripts/install-baremetal.sh: installs Homelable natively on a
Debian/Ubuntu host — Python venv plus a homelable systemd unit for the
backend on 127.0.0.1:8000, the built frontend served by nginx on :3000.
Modeled on scripts/lxc-mcp-install.sh, with the install steps taken from
the community-scripts/ProxmoxVE recipe so the two stay recognisably the
same install.

The script clones into INSTALL_DIR when empty, creates the service user,
builds the venv and the frontend, generates backend/.env with a random
SECRET_KEY and a bcrypt hash, writes the systemd unit and the nginx site,
then waits on /api/v1/health. Re-running is safe and is the upgrade path:
an existing .env is kept, everything else is rebuilt. Every prompt has an
environment-variable override, so a non-interactive install is one line.

Two details worth calling out:

- JSON values in the generated .env are single-quoted. systemd's
  EnvironmentFile parser strips bare double quotes, which would hand
  pydantic [http://...] instead of ["http://..."] and fail startup.
- The admin password reaches Python through the environment rather than
  argv, which is world-readable in ps.

The unit runs unprivileged, so nmap falls back to a TCP connect scan;
AmbientCapabilities=CAP_NET_RAW is shipped commented out with the
trade-off spelled out in the unit and in the docs.

INSTALLATION.md gains a "Bare metal — no Docker" section covering the
quick start, the upgrade, the option table, the non-root scan trade-off
and the host-nginx blocks for anyone bringing their own reverse proxy.
README links it from the install line.

No test suite: shell installers have none in this repo, and CI's
lint-scripts job shellchecks scripts/. Shellcheck is clean.

Closes #333

ha-relevant: no
2026-09-02 02:15:10 +02:00
Pouzor 39cad9efc7 fix(canvas): keep a nested container ahead of its children
A container dropped into a zone left its own children draggable anywhere
on the canvas: they lost the box they were pinned inside (#366
follow-up).

The store reordered nodes for React Flow with a two-bucket split —
parentless nodes first, everything else after. That is only correct while
nesting is one level deep. A zone can hold a container, so the container
and its own children now sit in the same bucket, and a child whose row
predates its parent's lands ahead of it. React Flow then drops the
child's parent binding entirely: no parent-relative position, no
`extent: 'parent'` clamp, so it renders detached and drags free.

`orderParentsFirst` — a topological sort that already backs the zone drop
— now covers the other five reorder sites: loadCanvas, applyLayout,
updateNode's re-parent path, setProxmoxContainerMode and pasteNodes. The
child's stored state was never wrong; only the array order was.

ha-relevant: yes
2026-09-02 01:24:12 +02:00
Pouzor efcafee774 fix(mcp): keep the zone tools inside one design
/api/v1/nodes has no design filter, so list_zones reported every
design's zones and add_to_zone would happily parent a node from design B
under a zone on design A — which hides it on the canvas it belongs to.

list_zones takes an optional design_id and reports each zone's own, and
add_to_zone skips a node whose design differs from the zone's, alongside
the other skip rules. remove_from_zone needs neither: a node and the
zone it sits in are on the same design by construction.

ha-relevant: no
2026-09-01 08:52:31 +02:00
Pouzor 6c55cbfe58 fix(canvas): detach a selection from a zone in one undo step
Dragging a multi-selection out of a zone called removeFromGroup once per
node, and each call pushed its own history entry — so an add of three
nodes undid in one step but the matching detach took three, with the
intermediate states leaving part of the selection out.

removeNodesFromGroup detaches the batch in a single set, mirroring
addNodesToZone; removeFromGroup delegates to it.

ha-relevant: yes
2026-09-01 08:52:31 +02:00
Pouzor 1ec7ea7e8e feat(mcp): expose zones to the MCP server
An AI client could create nodes but had no way to create the areas that
organize them, or to move a node into one (#365).

Four tools, all on the existing /api/v1/nodes endpoints: create_zone
(a groupRect node, with size and border/text colours), list_zones (each
zone plus the ids it contains), add_to_zone and remove_from_zone, both
taking a list of node ids.

The canvas stores a child's position relative to its parent, so
add_to_zone rebases each node on the zone and remove_from_zone restores
absolute coordinates — otherwise a node jumps by the zone's offset the
next time the canvas loads. add_to_zone skips what it cannot move (the
zone itself, a node already inside it, an unknown id, and any node the
zone descends from, which would build a cycle) and reports them back
rather than failing the batch.

groupRect stays out of NODE_TYPES: create_node remains a device tool,
and the enum sync tests keep passing.

ha-relevant: no
2026-09-01 08:52:31 +02:00
Pouzor 3301f00e57 feat(canvas): add a whole selection to a zone in one drop
Dropping a multi-selection on a zone, a group or a container only moved
the node under the cursor: handleNodeDragStop read `dragNode` and ignored
`dragNodes`. Dragging a parent and its children in was especially
tedious (#365).

The drag handler now uses the whole dragged selection, with `dragNode`
only picking the destination, and the store grows batch counterparts
(addNodesToZone / addNodesToGroup / addNodesToContainer) so the move
lands in a single history entry and one undo takes it all back. The
singular actions delegate to them.

Ineligible children are skipped rather than failing the batch: the
destination itself, a node already parented to it, a node whose own
parent is in the same selection (it rides along with that parent), and
any node the destination descends from. Leaving a zone detaches every
dragged child too, not just one.

The confirmation modal takes labels instead of a label, and counts and
lists them ("Add 3 nodes (Router, Switch, NAS) to the zone DMZ?"),
summarizing the tail past five.

ha-relevant: yes
2026-09-01 08:52:31 +02:00
Pouzor 895d00a499 fix(canvas): stop a self-parented node from freezing the canvas
A node recorded as its own parent (`parent_id = id`) made the child walk in
`translateWaypointsForMovedNodes` recurse forever. The resulting stack overflow
was thrown inside the `onNodesChange` reducer, so the whole state update was
discarded: the node still selected but would never move. It also renders
unparented, sitting wherever its stored coordinates put it rather than inside
the container it appears to belong to.

Guard the walk, then close every path that could write the row:

- `propagate` keeps a `walked` set, matching the cycle guard `orderParentsFirst`
  already carries. Covers longer cycles (a -> b -> a) too, not just self-parent.
- `updateNode` / `addNode` drop a `parent_id` equal to the node's own id. The
  key is removed rather than nulled, so the rest of the edit still lands and an
  existing real parent is left alone.
- `importYaml` resolves parents by label, so a node naming its own label — or a
  duplicate label mapping back to it — imported as its own parent. Warn and skip.
- `node_dedupe` re-pointed every child of a duplicate at the canonical node,
  including the canonical node itself when it had been nested under one of its
  own duplicates. Detach instead, mirroring the self-loop edge deletion below it.
- `NodeSave` and the node PATCH route normalize it away. Dropped rather than
  rejected: a canvas that already carries the bad row must still be able to save,
  and a 422 would cost the user the whole save.

`_repair_self_parent_nodes` clears what is already persisted at startup. The
real parent is not recoverable from the row, so NULL returns the node to the top
level where the user can re-nest it. Idempotent, and never fatal to boot.

A longer cycle is deliberately left alone by the repair — there is no single
right link to cut, and the runtime guard keeps the canvas usable either way.

ha-relevant: yes
2026-09-01 01:53:03 +02:00
Pouzor 28f3ac3878 fix(services): stop appending the scanned port to a custom service host
A service `host` override usually names a reverse proxy, where the scanned
port is an internal detail: `https://proxy.example.com:8083/` is not a URL
that resolves to anything. The override now suppresses `svc.port` in both the
link the canvas opens and the URL the status checker probes — only a port
typed into the override itself (`host:port`) is printed.

The scanned port still gates the non-HTTP/SSH skip and the http/https guess,
so a MySQL or SSH service behind an override stays unlinked and unchecked.

Closes #382

ha-relevant: yes
2026-09-01 00:50:37 +02:00
Pouzor 2413ac3950 feat(imports): link a canvas-direct import to its Device Inventory row
A canvas-direct Zigbee / Z-Wave / Proxmox import placed nodes and left the
inventory alone. The first canvas save then minted a row for each of them
(`link_facts`), but the save wire shape carries no `ieee_address` — so the row
landed IEEE-less and tagged `canvas`, and the next mesh import, matching by
IEEE, created a second row for the same device.

The canvas import now upserts the inventory first, through the same
`_persist_pending_import` the pending path uses, and stamps each returned node
with the id of the row it draws. The canvas already round-trips `device_id`, so
the save links to that row instead of minting a rival. A row a node draws also
leaves the pending queue — the mirror of the existing "approved but no longer
drawn -> pending" revival. A hidden row stays hidden.

The mode selector is relabelled to match what it now does: both modes reach the
Device Inventory, so the choice is "Device inventory only" vs "Inventory +
canvas". The three modals also gain the width and spacing the two-line labels
needed.

Pre-existing split rows (a `canvas` row and an import row for one device) are
left alone: with no shared ieee/ip/mac they could only be matched on hostname,
and welding inventory rows together on a name match is worse than the drift.

ha-relevant: maybe
2026-08-31 13:26:37 +02:00
Pouzor f85130a27a test(backend): run background-task sessions against the test database
Background tasks open their own session through `AsyncSessionLocal`, which the
request-scoped `get_db` override never reached — so any test exercising one ran
against the real configured database, inserting rows and, through the imports'
wipe-and-reinsert of `device_inventory_links`, deleting real ones.

Point the factory at the test engine in `app.db.database` and in every route
module that imported it by name, and restore it when the fixture tears down.

ha-relevant: no
2026-08-31 13:26:37 +02:00
Pouzor c85bb46373 fix(mcp): make resources/read and resources/templates/list work
Two bugs made the Resources tab unusable in any MCP client.

read_resource returned mcp.types.TextContent, but the low-level
Server.read_resource decorator expects str, bytes or an iterable of
ReadResourceContents and reads .content / .mime_type off each item.
TextContent carries .text, so every resources/read failed at
serialisation with "'TextContent' object has no attribute 'content'".
The handler itself runs, so the error only ever reached the client and
never showed up in the server log.

No list_resource_templates handler was registered either, so the SDK
answered resources/templates/list with "Method not found". That hid the
homelable://nodes/{node_id} template read_resource already serves — it
is absent from RESOURCE_LIST because it is a template, not a concrete
URI, so nothing advertised it at all.

ha-relevant: no
2026-08-31 11:46:53 +02:00
Pouzor 0acfe3acc3 fix(zigbee): stop canvas imports dying on a proxy read timeout
A Zigbee2MQTT networkmap on a 200+ device mesh takes minutes to build.
Two separate failures fell out of that:

- POST /zigbee/import held the HTTP request open for the whole MQTT
  round-trip, so any reverse proxy in front of the API cut it first
  (Cloudflare returns a 524 at 120 s) and the browser never saw the map.
  It now registers a job, fetches in the background and answers 202; the
  client polls GET /zigbee/import/{job_id} until the payload is ready.
  Job results are transient and live in memory with a 15 min TTL — the
  same single-worker assumption the scheduler already makes. A failed
  fetch replays the status the synchronous route used to raise, so a bad
  broker is still a 502 and a slow mesh still a 504.

- The networkmap wait was hard-coded at 300 s with no way to raise it.
  It now reads ZIGBEE_NETWORKMAP_TIMEOUT, and the shared MQTT round-trip
  used by the Z-Wave import reads MQTT_RESPONSE_TIMEOUT. Both default to
  300 s, fall back to that if misconfigured to a non-positive value, and
  name themselves in the timeout message.

Also corrects the route and doc claims that the wait was 60 s.

The /import tests changed with the contract they cover, not to pass.

Fixes #380

ha-relevant: yes
2026-08-31 11:46:36 +02:00
Pouzor 9c2a1ba0eb docs(canvas): correct a stale comment on where a zone's height is stored
The zone size moved to the nodes.width/height columns; the comment still
described the custom_colors stash it replaced.

ha-relevant: yes
2026-08-27 01:45:23 +02:00
Pouzor 851141951d fix(canvas): store a zone's size in the width/height columns
Every node type persisted its size in `nodes.width` / `nodes.height` except
`groupRect`, which stashed it inside the `custom_colors` JSON next to its
colours. The columns already existed and were simply unused for zones, so
this was an inconsistency rather than a missing-column workaround, and it
put geometry in a blob that otherwise holds style.

The serializer now writes the columns for a zone too, and strips the legacy
`width`/`height` keys out of the blob so the two cannot drift apart and
leave an older canvas reading a stale size.

No data is lost on upgrade:

- `_backfill_zone_size` copies the blob geometry into the columns at
  startup. It only fills a column that is still NULL, so it cannot overwrite
  a size set since; it parses the JSON in Python rather than with
  `json_extract`, so it does not depend on the SQLite build carrying JSON1;
  and an unreadable row is skipped without costing the others their size.
  Re-running it is a no-op.
- the reader still falls back to the blob, covering a payload the backfill
  has not reached — an older server, or an import.

Standalone mode is unaffected: it stores React Flow nodes verbatim, so the
size was always on `node.width` / `node.height` there.

The four serializer tests that pinned the size to the blob now assert the
columns, since that is the behaviour being changed.

ha-relevant: yes
2026-08-27 01:45:23 +02:00
Pouzor f9f88c8fb0 refactor(canvas): drop a dead custom_colors.height write on zone growth
Growing a zone during a subnet import wrote the new height twice: to
`node.height`, the live field, and to `data.custom_colors.height`, which
nothing reads. The blob copy is produced by the serializer at save time
(rebuilt from `node.height`, so the value written here was overwritten
before it reached the API) and consumed at load time off the API payload.
Writing it from the store was invisible, and misleading in a blob that
otherwise holds colours and style.

The test asserting the dead field is replaced by a save/load round-trip
through the real serializer, which is what actually protects the height.

ha-relevant: yes
2026-08-27 01:45:23 +02:00
Pouzor 2bf9f1bca8 fix(canvas): keep a parent ahead of its children after a subnet import
The subnet import only guaranteed the zone preceded the nodes it pulled in.
An arrival that is itself a parent — a Proxmox host with nested VMs, whose
children stay put because they already have a parent — could end up listed
after those children, which React Flow renders detached with a
parent-not-found error. Nothing re-sorts on load, so the broken order
survived a save.

Reorder the whole array instead: every node now follows its parent, which
covers both directions at once. An already-valid list comes back untouched
and a parent cycle terminates rather than recursing.

ha-relevant: yes
2026-08-27 01:45:23 +02:00
Pouzor d9451f35dc feat(canvas): import devices into a zone by subnet
Closes #325.

A zone gains an "Import devices by subnet" action: type a CIDR, and every
free device whose IP falls in that range moves into the zone, laid out on a
grid in its free space.

The CIDR is an argument to a one-shot action, never a property of the zone —
it is not submitted with the form, not persisted, and cleared after each run.
So there is no schema change, no migration, and it works in standalone mode.
The trade-off is that a device scanned later does not join the zone by
itself; the user re-runs the import.

Deliberate rules:
- additive — running it twice with two subnets leaves both sets inside, and
  a device whose IP stops matching is never ejected
- only unparented devices move. A node nested in a group, a container host
  or another zone keeps the parent the user gave it
- canvas furniture (groupRect / group / text) is skipped, and a zone never
  swallows another zone
- one history entry per import, so a single Undo reverses the whole thing

Edit mode gets an Import button; add mode has none, since the zone does not
exist yet and a button would look broken — there the CIDR is applied right
after the zone is created.

IPv4 only: the modal rejects an IPv6 CIDR with a message rather than
silently matching nothing.

ha-relevant: yes
2026-08-27 01:45:23 +02:00
Pouzor 05c0da53e6 fix(scan): reconcile scan runs orphaned by a backend restart
A scan runs on a background thread inside the API process. If that process
dies mid-scan — an OOM kill, docker stop, a crash — the ScanRun row stays
"running" for ever, because nothing is left alive to finish it.

That row is not just cosmetic clutter in Scan History: the trigger endpoints
reject a new scan while one is "running" for the same target, so a single kill
locks that range out permanently.

Nothing can legitimately be "running" the moment we boot, so lifespan() now
marks every such row "error" — the same word run_scan and run_device_scan
write when they fail themselves — with finished_at and an explanatory message.

Reported alongside the OOM itself in #374, which is what produced the orphans.

Fixes #374

ha-relevant: maybe
2026-08-27 00:00:32 +02:00
Pouzor c29680a75f fix(status): stop an endless response body from OOM-killing the backend
Both HTTP paths buffered the whole response body before looking at it. An
endpoint that streams without end and sends no Content-Length — a Freebox
bandwidth-test port, an MJPEG camera, a log tail — grew the backend until the
cgroup limit killed it, every 5-10 minutes at a constant ~1.04 GB RSS.

httpx's timeout does not help: it applies per network operation, not to the
total time spent draining a socket that keeps delivering data.

- status_checker._http_get only needs the status line, so it now uses
  client.stream() and never reads the body at all.
- http_probe._probe_scheme needs at most _MAX_BODY_BYTES to hunt for <title>,
  so it streams and stops there. The cap existed already but was applied to
  resp.text, after the full body had been downloaded.

Regression tests serve an endless, Content-Length-less body through a
MockTransport and assert the read stays bounded. Both fail on the old code.

The existing mocks patched httpx.AsyncClient.get, which neither path calls
now; they are rebuilt on MockTransport — a real client over a fake network —
with every original assertion kept.

Fixes #375

ha-relevant: yes
2026-08-26 23:43:50 +02:00
Pouzor - Rémy JardientandGitHub eb7e74d36b Update README.md 2026-08-25 23:08:03 +02:00
Pouzor 390a712f14 chore: bump version to 3.3.5
ha-relevant: no
2026-08-21 12:56:17 +02:00
Pouzor 9daea4c4d0 fix(scan): keep an edit, a discovery source and one name for a failed run
Three findings from the PR review.

A finishing deep rescan threw away an edit in progress. The reset effect in
InventoryDeviceModal was keyed on the `device` object, and the parent hands
down a fresh one whenever the row is refreshed — including from the poll's
own onSaved. Same device, new object, so the effect reset the form and left
edit mode, minutes into a scan the user was waiting on. It is keyed on the
id now: a refreshed row is not a different device, and only a different
device is a reason to throw the form away. The fresh row still reaches the
canvas and the grid — gating that behind edit mode would discard the scan
result instead, and a save unions the services server-side anyway.

A rescan tagged every device it touched as "arp"-discovered, so a Proxmox
guest, a rack mount or a hand-added host started answering the network
source filter. It carries its own source through instead.

The two background wrappers wrote status "failed" where the scanners write
"error" for the same condition. Harmonized on "error" — the one Scan
History filters and colours; "failed" showed up unlabelled. The frontend
union keeps 'failed' for rows already recorded.

That last one meant updating an existing assertion in test_scan_run.py: it
encoded the old spelling.

ha-relevant: yes
2026-08-21 12:49:48 +02:00
Pouzor 2e642815f4 fix(scan): stop a scan from repainting hand-picked service icons
merge_services did {**existing, **incoming}, so the fingerprint's guess at
an icon overwrote the one the user chose — and on a port no signature
covers it wrote None, clearing it outright. Since 3.3.0 the inventory row
is the only copy of a device's services, so every "Scan network" repainted
the service on every canvas drawing that device at once.

The scanner now merges with discovered=True: it still adds services and
refreshes what it knows, but leaves an established icon and category alone.
A user edit from the modal or a canvas changes them as before.

Blank incoming values no longer clear established ones either, on both
paths — an absent field is silence, not a reset. Same rule merge_properties
already follows.

ha-relevant: yes
2026-08-21 12:49:48 +02:00
Pouzor c6d6b525ea feat(scan): choose the port range before a deep scan
The Deep scan link on a device now opens a small dialog instead of firing
straight away. It is prefilled with the full 1-65535 range — that is still
the point of the feature — but a user who knows where a service lives can
narrow it and get an answer in seconds instead of minutes.

The dialog takes an nmap-style spec: a port, a range, or a comma list
(80,443,8000-9000). It shows the port count live, refuses to start on
something nmap could not use, and carries three presets (all / 1-1024 /
1-10000).

Backend:
- _parse_port_spec / _port_chunks generalize the slicing that used to be
  full-range only. Ranges are merged before slicing, so an overlapping
  spec is never scanned twice, and small ranges are packed into one nmap
  call instead of one call each. _deep_port_chunks still yields the same
  eight slices as before.
- run_device_scan takes ports=; it wins over full_ports.
- The retry-free flags (--max-retries 0 --min-rate 2000) now key on the
  total port count rather than on full_ports. They pay for themselves over
  thousands of ports on a lossy host; over a handful they only cost
  accuracy.
- RescanDeviceRequest.ports validates the spec — 422 rather than handing
  nmap a bad -p. Blank means the full sweep.

Frontend utils/portSpec.ts mirrors the backend parser so a typo is caught
before the request; the backend validates again because it is the one
calling nmap.

The existing rescan tests now go through the dialog: the click path
changed, so the start is two steps.

ha-relevant: yes
2026-08-21 12:49:48 +02:00
Pouzor fbb660504e fix(scan): slice the deep rescan instead of timing out the host
A deep rescan of a slow host came back with nothing at all: the run took
its full 600s ceiling and the device's services were unchanged, so a
service deleted by hand was never rediscovered.

nmap answers --host-timeout with "Skipping host <ip> due to host timeout"
and discards every port it had already found — the ceiling turned a slow
scan into one that reports nothing. What costs the time is a host that
drops packets: 8188 of 8192 ports filtered, each waiting out its probe.

- No --host-timeout on the deep discovery pass, ever.
- The full range runs as 8 slices of 8192 ports, one nmap call each,
  unioning the open ports. A slice that overruns costs its own ports, not
  all of them, and the loop has somewhere to notice a stop request.
- `scanner_deep_host_timeout` is now a total budget checked between
  slices (default 2700s), not an nmap flag. The first slice always runs.
- A partial sweep is reported rather than passed off as complete: the run
  finishes `done` carrying "Scanned 3/8 port ranges …", and the modal
  toasts a warning instead of success.
- Deep slices use --max-retries 0 --min-rate 2000. Measured against a
  dropping host, 8192 ports took 329s at --max-retries 1 and 164s at 0,
  finding the same ports; capping the RTT changed nothing. The range scan
  keeps nmap's default retries on its curated port list.

ha-relevant: yes
2026-08-21 12:49:48 +02:00
Pouzor 1be96d1045 feat(scan): deep-rescan one device from the inventory detail
Devices added before the scanner knew a service showed an empty Services
section with no way to refresh it (#350). The detail modal now starts a
full-port scan of that single device.

- `process_host` lifted out of `run_scan` so the range scan and the new
  single-device scan share the same match / merge / dedupe rules — a
  rescan unions services, it never replaces what the user added by hand.
- `run_device_scan`: no ping sweep, no mDNS, straight to the phase-2 nmap
  pass on the device IP over all 65535 TCP ports.
- `POST /scan/pending/{id}/rescan` records a normal ScanRun
  (`kind=device`, `ranges=["<ip>/32"]`), so stop, progress and Scan
  History work unchanged. One run per device at a time — a second request
  while the first is scanning is a 409. 404 unknown, 409 no-IP or hidden.
- `GET /scan/runs/{id}` so a caller can poll the run it started.
- Deep scan button in the Services section of the device detail, hidden
  without an IP and for Zigbee. Swaps to a stop control while running,
  folds the fresh services back in on completion (never over an edit in
  progress).

ha-relevant: yes
2026-08-21 12:49:48 +02:00
Pouzor 9bc02d61c2 fix(canvas): give a valueless property label the full node width
The property line caps its label at max-w-15 so a long key cannot crowd
out the value drawn beside it. When the value is empty there is nothing
to protect, but the cap still applied, truncating the label against
empty space.

Drop the cap when the value is blank and let the label truncate against
the node width instead. Same fix in BaseNode and ProxmoxGroupNode.

Closes #361

ha-relevant: yes
2026-08-17 20:26:29 +02:00
Pouzor 21a64e52ab fix(auth): accept the app's own origin in the OIDC CSRF check
A same-origin deployment behind a reverse proxy needs no CORS, so
CORS_ORIGINS is left at its localhost default — but browsers still send
Origin on unsafe methods, and OIDCCSRFMiddleware validated it against
CORS_ORIGINS alone. Every POST/PUT/DELETE came back 403 while reads
worked, so creating a canvas failed with no usable error.

The OIDC callback URL is served by this app, so the origin of
OIDC_REDIRECT_URI is the app's own and is always a legitimate CSRF
origin. origin_is_allowed now accepts it in addition to CORS_ORIGINS,
and settings validation warns at startup when CORS_ORIGINS omits it so
the misconfiguration is visible rather than silent.

Fixes #356

ha-relevant: no
2026-08-17 20:13:04 +02:00
Pouzor ad0b1b427d chore: bump version to 3.3.4
ha-relevant: no
2026-08-17 19:19:25 +02:00
Pouzor 4c460cec1c fix(scan): stop a re-scan from deleting a device's hand-added services
Since 3.3.0 the inventory row is the only copy of a device's services and
every canvas drawing it reads that list, but the scanner still assigned
`keep.services = fingerprint_ports(...)` — the pre-split behaviour, when a
node held its own copy. One re-scan therefore deleted every service the user
added by hand, on every canvas at once, and brought back the ones they had
deleted. It unions now, like every other writer of that field.

Two more things #347 turned up:

* A node whose view matches nothing the row still holds drew nothing at all —
  the row had been replaced under it, so every key was gone and every key was
  new, and `apply_view` hid the lot. Such a view says nothing about the list
  that replaced it, so it is treated as no view: the row is drawn. An empty
  view is untouched, being a real answer ("this canvas draws none of them").

* The 3.3.3 view seed recovers a node's arrangement from the pre-3.3.0 backup,
  which is 3.2.0-era and cannot know about a property added afterwards. On
  3.3.0-3.3.2 the row was the only place to add one and every canvas drew it,
  so seeding strictly from the backup took it off all of them at once. Those
  are appended visible, keeping the recovered order and hidden flags for
  everything the backup does know. Services keep the strict recovery: holding
  back what a scan fingerprinted is the whole point of the view.

ha-relevant: maybe
2026-08-17 19:05:13 +02:00
Pouzor 1aa88de491 chore: bump version to 3.3.3
ha-relevant: no
2026-08-17 14:09:01 +02:00
Pouzor d0679ad72b fix(inventory): stop the view seed from re-reading the backup every boot
The seed selected any node with a `device_id` and no view. Deleting a device
leaves that column dangling — SQLite runs with foreign keys off, so the
`ON DELETE SET NULL` never fires on an existing table — and such a node can be
seeded from nothing, so it stayed selected. The query matched it again on every
later start, re-opening the pre-upgrade backup and logging a recovery that
recovered nothing.

Narrowing the query to nodes whose row actually exists restores the invariant
the seed is documented to have: it runs on one boot, and never opens a file
again.

ha-relevant: no
2026-08-17 14:06:09 +02:00
Pouzor 5df70ce128 feat(canvas): give each node its own view of a device's services and properties
Since 3.3.0 the inventory row owns a device's services and properties, and
every node drawing that device rendered the row wholesale. One row shared by
several canvases meant one rendering: a service the scanner fingerprinted
appeared on every canvas at once (users reported Uptime Kuma and Synology DSM
on hosts running neither — both are port-only signatures), and a property added
on one schematic showed up on all the others.

Order and visibility are presentation, so they move to the node. `display_view`
records, per node, which of the row's services and properties it draws and in
what order, keyed by `port|protocol|name` and by lowercased property key so the
view survives an edit to the fact itself. The facts stay on the row: hiding is
per node, deleting is still device-wide.

An item the view does not list is reported hidden rather than dropped, so what
a scan finds next is one toggle away instead of pushed onto every canvas.

The wire shape is unchanged. A client already sends its lists in display order
with their `visible` flags, so the view is read back out of them rather than
asking for a second field, and `visible` is only stamped when something is
hidden.

Upgrades keep what each canvas showed:

* from 3.2.0, the backfill seeds each node's view from its own legacy columns
  before they are dropped;
* from 3.3.0-3.3.2 those columns are gone and the row holds the union of every
  canvas, so the layout is recovered from the backup `_backup_db` took before
  the 3.3.0 migration — the newest one whose `nodes` table still has the
  columns, read read-only and matched by node id;
* with no usable backup the row is the seed, so every canvas keeps showing
  exactly what it shows today and only later additions are held back.

Refs #347

ha-relevant: maybe
2026-08-17 14:06:09 +02:00
262 changed files with 32196 additions and 1069 deletions
+14
View File
@@ -8,12 +8,19 @@ SQLITE_PATH=./data/homelab.db
# Set this to the URL(s) you use to access Homelable in your browser.
CORS_ORIGINS=["http://localhost:5173","http://localhost:3000"]
# Serve the UI under a subpath instead of the root of the origin, e.g. /homelab/.
# Read by docker-compose at *build* time (docker compose build frontend), since the
# prefix is baked into the frontend bundle. See INSTALLATION.md.
# VITE_BASE_PATH=/homelab/
# Auth — local mode is the backward-compatible default.
AUTH_MODE=local
# Local credentials: admin / admin by default.
# ⚠️ Change before exposing on a network.
# Generate a new hash: python3 -c "import bcrypt; print(bcrypt.hashpw(b'yourpassword', bcrypt.gensalt()).decode())"
# ⚠️ Keep the single quotes around the hash — bcrypt hashes contain \$ which Docker misinterprets without them.
# podman-compose does not strip those quotes the way Docker Compose does, so the
# backend unwraps one matched pair itself — quoted or not, both work everywhere.
AUTH_USERNAME=admin
AUTH_PASSWORD_HASH='$2b$12$RtMbyw17l4N5UGzeXMNAWuzCaVV.XFBY7ZetWheQhxcBDcxahapkG'
@@ -89,6 +96,10 @@ MCP_SERVICE_KEY=svc_changeme
# ZIGBEE_BASE_TOPIC=zigbee2mqtt
# ZIGBEE_MQTT_TLS=false # true for TLS brokers (typically port 8883)
# ZIGBEE_MQTT_TLS_INSECURE=false # skip cert verify (self-signed only; requires TLS)
# Seconds to wait for the Z2M bridge to answer a networkmap request. Applies to
# manual imports too. Raise it on a large mesh (200+ devices can take minutes)
# if an import fails with "Timed out waiting for networkmap response".
# ZIGBEE_NETWORKMAP_TIMEOUT=300
# Z-Wave JS UI (zwavejs2mqtt) auto-sync — same MQTT secret/env rules as Zigbee.
# ZWAVE_MQTT_HOST=192.168.1.20
@@ -99,3 +110,6 @@ MCP_SERVICE_KEY=svc_changeme
# ZWAVE_GATEWAY_NAME=zwavejs2mqtt
# ZWAVE_MQTT_TLS=false # true for TLS brokers (typically port 8883)
# ZWAVE_MQTT_TLS_INSECURE=false # skip cert verify (self-signed only; requires TLS)
# Seconds to wait for the gateway to answer an MQTT request. Same rationale as
# ZIGBEE_NETWORKMAP_TIMEOUT above.
# MQTT_RESPONSE_TIMEOUT=300
+38
View File
@@ -0,0 +1,38 @@
#!/usr/bin/env bash
#
# Run a dependency audit, retrying only when the registry could not be reached.
#
# Both `npm audit` and `pip-audit` exit non-zero for two very different reasons:
# a real advisory, and a registry that would not answer. Only the second is
# worth a retry — the audit endpoints 503 often enough to redden PRs that
# changed no dependency at all. An advisory fails on the first attempt, so a
# vulnerability can never be retried into a pass.
#
# Usage: audit-with-retry.sh <command> [args...]
set -uo pipefail
ATTEMPTS=${AUDIT_ATTEMPTS:-3}
DELAY=${AUDIT_RETRY_DELAY:-30}
# Transport and availability failures, from both npm and pip-audit/urllib3.
NETWORK_FAILURE='audit endpoint returned an error|service unavailable|max retries exceeded|connectionerror|readtimeout|read timed out|temporary failure in name resolution|50[234] server error|etimedout|econnreset|enotfound|socket hang up|connection reset|remotedisconnected|urlopen error|timeouterror'
for attempt in $(seq 1 "$ATTEMPTS"); do
if out=$("$@" 2>&1); then
echo "$out"
exit 0
fi
echo "$out"
if ! grep -qiE "$NETWORK_FAILURE" <<<"$out"; then
echo "::error::$1 reported findings — see the output above"
exit 1
fi
echo "::warning::$1 could not reach its registry (attempt $attempt/$ATTEMPTS)"
if [ "$attempt" -lt "$ATTEMPTS" ]; then sleep "$DELAY"; fi
done
echo "::error::$1 could not reach its registry after $ATTEMPTS attempts"
exit 1
+131
View File
@@ -0,0 +1,131 @@
name: Install smoke test
# Runs scripts/install-baremetal.sh for real, in a clean debian:12 container.
# ShellCheck reads the script but never executes it, so it cannot catch an
# unintended `set -euo pipefail` abort — the failure class this job exists for.
# debian:12 reproduces the two conditions that broke the installer once already:
# the image carries no Node at all (so the version probe and the nodesource
# install both run) and the job has no TTY (so the prompt fallbacks run).
#
# What it does not cover: systemd. A container has no PID 1 systemd, so
# `systemctl` is stubbed out and the unit is written but never started. The
# backend is booted by hand afterwards to prove the generated .env parses.
on:
pull_request:
branches: [main]
paths:
- 'scripts/install-baremetal.sh'
- 'backend/requirements.txt'
- 'frontend/package.json'
- 'frontend/package-lock.json'
- '.github/workflows/install-smoke.yml'
push:
branches: [main]
paths:
- 'scripts/install-baremetal.sh'
- 'backend/requirements.txt'
- 'frontend/package.json'
- 'frontend/package-lock.json'
- '.github/workflows/install-smoke.yml'
permissions:
contents: read
jobs:
baremetal:
runs-on: ubuntu-latest
container: debian:12
timeout-minutes: 25
# Without this the runner hands container steps `sh -e {0}` — dash, where
# `set -o pipefail` and `[[ … ]]` are syntax errors.
defaults:
run:
shell: bash
steps:
- name: Install checkout prerequisites
run: |
apt-get update -qq
DEBIAN_FRONTEND=noninteractive apt-get install -y -qq git ca-certificates >/dev/null
- uses: actions/checkout@v4
- name: Confirm the container reproduces a fresh host
run: |
set -euo pipefail
node --version || echo "no node — the nodesource path will run"
[ -t 0 ] && echo "TTY present (unexpected)" || echo "no TTY — prompt fallbacks will run"
- name: Stub systemctl
# No PID 1 systemd in a container. The stub logs the calls so the run
# shows which unit actions the script attempted.
run: |
printf '#!/bin/sh\necho "[systemctl stub] $*"\n' > /usr/local/bin/systemctl
chmod +x /usr/local/bin/systemctl
- name: Run the installer
# INSTALL_DIR is the checkout, so the clone is skipped and the script
# installs the tree under test rather than main. SKIP_NGINX because the
# front end is not what this job is asserting.
run: |
INSTALL_DIR="$GITHUB_WORKSPACE" \
SKIP_NGINX=1 \
ADMIN_PASSWORD=ci-smoke-password \
SCANNER_RANGES='["10.0.0.0/24"]' \
bash scripts/install-baremetal.sh
- name: The generated .env is well formed
run: |
set -euo pipefail
test -f backend/.env
grep -qE '^SECRET_KEY=[0-9a-f]{64}$' backend/.env
grep -qE '^AUTH_PASSWORD_HASH=.\$2b\$' backend/.env
# JSON values must stay single-quoted: systemd's EnvironmentFile
# parser strips bare double quotes, which would break the JSON.
grep -qE "^CORS_ORIGINS='\\[" backend/.env
grep -qE "^SCANNER_RANGES='\\[" backend/.env
grep -q "^SQLITE_PATH=$GITHUB_WORKSPACE/data/homelab.db$" backend/.env
test "$(stat -c '%a' backend/.env)" = "600"
- name: The build artefacts and service user exist
run: |
set -euo pipefail
test -x backend/.venv/bin/uvicorn
test -f frontend/dist/index.html
id homelable
- name: The systemd unit was written
run: |
set -euo pipefail
test -f /etc/systemd/system/homelable.service
grep -q 'ExecStart=.*/uvicorn app.main:app --host 127.0.0.1 --port 8000' \
/etc/systemd/system/homelable.service
grep -q "EnvironmentFile=$GITHUB_WORKSPACE/backend/.env" \
/etc/systemd/system/homelable.service
- name: The backend boots on the generated config
# The unit's own ExecStart, run by hand. Proves the .env pydantic reads
# is valid — the assertion that catches config drift.
run: |
set -euo pipefail
cd backend
.venv/bin/uvicorn app.main:app --host 127.0.0.1 --port 8000 &
for _ in $(seq 1 30); do
if curl -fsS http://127.0.0.1:8000/api/v1/health >/dev/null 2>&1; then
echo "backend healthy"
exit 0
fi
sleep 1
done
echo "backend did not answer /api/v1/health within 30s" >&2
exit 1
- name: Re-running is idempotent and keeps .env
run: |
set -euo pipefail
before="$(sha256sum backend/.env)"
INSTALL_DIR="$GITHUB_WORKSPACE" SKIP_NGINX=1 \
bash scripts/install-baremetal.sh | tee /tmp/run2.log
after="$(sha256sum backend/.env)"
test "$before" = "$after" || { echo "second run rewrote backend/.env" >&2; exit 1; }
grep -q 'keeping existing values' /tmp/run2.log
+7 -4
View File
@@ -61,10 +61,13 @@ jobs:
- run: mypy app/
- run: pytest
# Also guards the NODE_TYPES / EDGE_TYPES enums in mcp/app/tools.py against
# drift from frontend/src/types/index.ts (test_node_types_sync.py,
# test_edge_types_sync.py) — those tests need the frontend sources, so this
# job runs from a full checkout rather than the mcp Docker context.
# Also guards the copies the MCP server keeps of frontend data against drift:
# NODE_TYPES / EDGE_TYPES in mcp/app/tools.py against frontend/src/types/index.ts,
# and the faceplate catalog in mcp/app/faceplates.py against
# frontend/src/rack/faceplates.ts (test_node_types_sync.py,
# test_edge_types_sync.py, test_faceplates_sync.py) — those tests need the
# frontend sources, so this job runs from a full checkout rather than the mcp
# Docker context.
mcp:
runs-on: ubuntu-latest
defaults:
+10 -2
View File
@@ -40,9 +40,17 @@ jobs:
# --omit=dev: only ship-time deps gate the build. Dev-only tooling
# (shadcn CLI and its transitive tree: hono, fast-uri, …) is never
# bundled or served, so its advisories must not fail release CI.
run: cd frontend && npm audit --omit=dev --audit-level=high
#
# Wrapped so a registry outage retries instead of failing the PR; an
# advisory still fails on the first attempt. See the script's header.
working-directory: frontend
run: "$GITHUB_WORKSPACE/.github/scripts/audit-with-retry.sh npm audit --omit=dev --audit-level=high"
- uses: actions/setup-python@v5
with:
python-version: '3.11'
- name: Pip audit
run: pip install pip-audit && pip-audit -r backend/requirements.txt
# pip-audit queries PyPI's advisory API and fails the same way npm does
# when it cannot reach it, so it gets the same wrapper.
run: |
pip install pip-audit
"$GITHUB_WORKSPACE/.github/scripts/audit-with-retry.sh" pip-audit -r backend/requirements.txt
+76
View File
@@ -5,6 +5,82 @@ All notable changes to **Homelable** are documented here.
The format is loosely based on [Keep a Changelog](https://keepachangelog.com/),
and this project adheres to [Semantic Versioning](https://semver.org/).
## [3.4.1] - 2026-09-07
### Features
- Duplicate Device Inventory rows can be merged. A **Merge** action in the inventory's select mode lets you pick the survivor — the only way to collapse rows that share no address at all, since a name is not an identity — and `reconcile_duplicates` now runs at the end of a Proxmox import and of a scan to collapse what a shared MAC proves (never a shared IP alone, never across two distinct IEEEs). Merging keeps everything: the survivor fills its gaps from the others, addresses, services, properties and sources union, and canvas nodes, rack mounts, documents and mesh links are re-pointed before the extra rows go. (#424)
### Fixes
- `dedupe_nodes_by_device` deleted duplicate nodes while `rack_devices.node_id` and `documents.node_id` still named them; SQLite runs with foreign keys off, so the declared `ON DELETE SET NULL` never fired. (#424)
- The scanner's duplicate collapse merges instead of deleting inventory rows outright, which lost their facts and orphaned whatever drew them. Its guard against removing a second approved row is unchanged. (#424)
- The merge dialog is wide enough to read an IEEE. The addresses are what the choice is made on, and a long name was clipped at the edge of the dialog; the address line now wraps instead of truncating, and the list grows with the viewport. (#424)
## [3.4.0] - 2026-09-07
### Features
- **Documentation** — a real document space for the homelab, beside the canvases. Markdown documents attached to a device, a node, a canvas, or standing on their own; a Library of folders, a Devices tree pivoted by zone, group, type, subnet, rack, vendor or tag, wiki-links between documents, full-text search (FTS5, with a `LIKE` fallback where SQLite ships without it), up to 50 revisions per document with a line diff and restore, backlinks under every body, tags, and a "needs attention" bucket driven by a per-document review interval. A device document is scaffolded once from the scan facts and then belongs to you — nothing rewrites it — with an explicit "device data changed" banner and a per-block regenerate when you want the new facts. The old `device_inventory.notes` are migrated in verbatim and left in place. (#418, #421)
- Documents are reachable from where you already are: a node's detail panel, a device's inventory modal, and the canvas search modal, which now finds documents too. Wiki-links are written from a picker instead of from memory. (#418, #421)
- The rack faceplate is a real, editable object, and the **Device Inventory owns it**: plate, U height, column span, colour and the port list — with each port placed by hand — are carried by the device, so it wears the same panel in every rack it is mounted in. Only placement, pinned status and label stay per canvas. A device's front panel can also be drawn and edited straight from its inventory row. (#405)
- A rack mount chooses when it shows its ports — `auto`, `always` or `hover` — and a cable can no longer end on a plate drawing no socket. (#407)
- Proxmox guests can be nested inside their host on the canvas instead of sitting beside it on virtual edges. Both entry points ask: the import modal and adding a Proxmox host from the Device Inventory. (#399)
- Homelable can be served under a subpath. One build-time knob, `VITE_BASE_PATH`, becomes Vite's `base` and is threaded through every hand-built URL, the status WebSocket, the OIDC login and callback, and both shipped nginx configs. Defaults to `/`, where nothing changes. (#412)
- The MCP server covers the rack canvas and the Device Inventory writes — 24 tools to 47. Racks, mounts, accessories, moves, faceplates and cables, plus create/edit/delete on inventory entries. (#414)
- The Homelable logo is in the brand icon picker, first in the catalog, for both node and service icons. (#411)
### Fixes
- The scanner and the status checker no longer send an HTTP `GET` to TCP/9100. A printer speaking JetDirect printed that request as a page of plaintext — once a minute, forever, for an approved node. LPD (515) and JetDirect (9100-9107) are now on the non-HTTP denylist. Trade-off: Prometheus Node Exporter on 9100 loses its status check and shows grey. (#406)
- A device declaring several IPs is matched on every one of them instead of coming back as a new pending row on each scan. Hiding it hides it at all of its addresses, and the deduplication collapses the duplicates an extra address had already spawned — never deleting a second approved row. (#416)
- A Proxmox import no longer overwrites one guest's inventory row with another's facts. Identity is now the synthetic per-guest ieee first, then NIC MAC, then IP — and the IP fallback skips rows already claimed by a guest, since a shared VIP, a re-used lease or a CNI bridge address describes several machines. A migrated or renamed guest keeps its row. (#420)
- Coming back to the canvas from Documentation keeps the pan and zoom you left it at, instead of remounting zoomed in at the origin. (#422)
- Switching back to a rack canvas restores its saved pan and zoom instead of shrinking the racks into a corner until a hard reload. (#409)
- The basic edge animation runs the same way whichever direction the nodes were dragged in; the dash direction was derived from the source and target's on-screen geometry. (#402)
- Login works under `podman-compose`. Quoted `AUTH_PASSWORD_HASH`, `SECRET_KEY` and `OIDC_CLIENT_SECRET` values arrive with their quotes attached, which Docker Compose strips and podman-compose does not; a matched pair of surrounding quotes is now unwrapped in the backend, so one `.env` works under either runtime. (#413)
- A zone or group description is kept on save. (#418)
- MCP node properties are keyed on `key`, not `name`, so they no longer collapse silently. (#414)
- Eight Dependabot advisories cleared in the frontend lockfile (`fast-uri`, `@humanfs/node`, `qs`, `nanoid`); all dev-only, `npm audit` reports zero. (#417)
- The npm and pip audit CI jobs retry a registry outage instead of failing the run. (#409)
### Docs
- README and FEATURES.md cover the Documentation section, and `docs/documentation.md` documents the tree, wiki-links, search, history, backlinks and the API. (#421)
- `.env.example` notes that quoting a secret is optional and works either way. (#413)
## [3.3.5] - 2026-08-21
### Features
- Deep-scan a single device from its inventory detail: the Services section gets a **Deep scan** button that sweeps the device over a port range you pick, prefilled with the full 1-65535 range and narrowable when you know where the service lives. The run is a normal Scan Run, so stop, progress and Scan History work as usual, and the fresh services are merged in — never replacing what you added by hand. (#363)
### Fixes
- A deep rescan of a slow host no longer comes back empty. The run hit its timeout and nmap discarded every port it had already found; the full range is now scanned in slices, a partial sweep is reported as partial instead of passed off as complete, and the timeout is a total budget between slices rather than an nmap flag. (#363)
- A scan no longer repaints the service icons you picked by hand: the fingerprint's guess overwrote your choice, and cleared it outright on a port no signature covers. Since 3.3.0 the inventory row is the only copy, so one scan changed the service on every canvas at once. (#363)
- A rescan finishing while you edit a device no longer throws the edit away, and it no longer tags every device it touches as network-discovered — a Proxmox guest, a rack mount or a hand-added host keeps its own source. A failed run is also recorded as `error`, the status Scan History filters and colours. (#363)
- A property with no value gets the full node width for its label instead of being truncated against empty space. (#362)
- A same-origin OIDC deployment behind a reverse proxy no longer answers 403 to every write. The CSRF check validated the browser's `Origin` against `CORS_ORIGINS` alone; the app's own origin, taken from `OIDC_REDIRECT_URI`, is accepted too, and startup warns when `CORS_ORIGINS` omits it. (#360)
## [3.3.4] - 2026-08-17
### Fixes
- A network scan no longer deletes the services you added by hand. The scanner replaced a device's whole service list with what it fingerprinted, so one scan wiped manual entries on every canvas at once and brought back the ones you had deleted. It merges now. (#347)
- A node whose services were all replaced under it no longer renders empty: when nothing a node's view names is left on the device, the device's own list is drawn instead of hiding everything. (#347)
- Properties added while running 3.3.0–3.3.2 survive the 3.3.3 upgrade. The view recovered from the pre-3.3.0 backup could not know about them, so they were hidden on every canvas; they are kept visible, and the per-canvas arrangement the backup does know is still restored. (#347)
## [3.3.3] - 2026-08-17
### Features
- Each node now keeps its own view of a device's services and properties: order and visibility are per node, so a service the scanner fingerprinted no longer appears on every canvas drawing that device, and a property added on one schematic stays there. The facts still live on the inventory row — hiding is per node, deleting is device-wide. Upgrades keep what each canvas showed today. (#357)
### Fixes
- The one-off view seed no longer re-opens the pre-upgrade backup on every boot: a node pointing at a deleted device kept matching the seed query and logged a recovery that recovered nothing. (#357)
## [3.3.2] - 2026-08-17
### Fixes
+30 -8
View File
@@ -7,6 +7,12 @@ FROM --platform=$BUILDPLATFORM node:20-slim AS builder
ARG VITE_STANDALONE=false
ENV VITE_STANDALONE=$VITE_STANDALONE
# Subpath support: build with VITE_BASE_PATH=/homelab/ to serve Homelable under a
# prefix instead of the root of an origin. The default '/' produces the same build
# as before the option existed.
ARG VITE_BASE_PATH=/
ENV VITE_BASE_PATH=$VITE_BASE_PATH
WORKDIR /app
COPY frontend/package*.json ./
RUN npm ci
@@ -18,15 +24,31 @@ RUN npm run build
# Stage 2: serve
FROM nginx:alpine
COPY --from=builder /app/dist /usr/share/nginx/html
COPY --from=builder /app/dist /tmp/dist
# Nginx config: use standalone config (no backend proxy) or full config
# Nginx config: standalone (no backend proxy) or full, served at the root or under the
# VITE_BASE_PATH prefix. At the root the build lands in /usr/share/nginx/html and the
# shipped .conf is copied verbatim, so an existing deployment gets exactly what it had
# before this option existed. Under a prefix the build lands in the matching
# subdirectory, which keeps the generated config on plain `root` semantics.
ARG VITE_STANDALONE=false
COPY docker/nginx.conf /etc/nginx/conf.d/default.conf
COPY docker/nginx.standalone.conf /etc/nginx/conf.d/nginx.standalone.conf
RUN if [ "$VITE_STANDALONE" = "true" ]; then \
cp /etc/nginx/conf.d/nginx.standalone.conf /etc/nginx/conf.d/default.conf; \
fi && \
rm /etc/nginx/conf.d/nginx.standalone.conf
ARG VITE_BASE_PATH=/
COPY docker/nginx.conf docker/nginx.standalone.conf /tmp/nginx-conf/
COPY docker/nginx.subpath.conf.template docker/nginx.standalone.subpath.conf.template /tmp/nginx-conf/
# ash with pipefail: the base-path normalization below pipes printf into sed.
SHELL ["/bin/ash", "-eo", "pipefail", "-c"]
RUN set -eu; \
if [ "$VITE_STANDALONE" = "true" ]; then variant=nginx.standalone; else variant=nginx; fi; \
base=$(printf '%s' "${VITE_BASE_PATH:-/}" | sed -e 's#^/*#/#' -e 's#/*$#/#' -e 's#//*#/#g'); \
[ -n "$base" ] || base=/; \
mkdir -p "/usr/share/nginx/html${base}"; \
cp -a /tmp/dist/. "/usr/share/nginx/html${base}"; \
if [ "$base" = "/" ]; then \
cp "/tmp/nginx-conf/${variant}.conf" /etc/nginx/conf.d/default.conf; \
else \
sed -e "s#__BASE_NO_SLASH__#${base%/}#g" -e "s#__BASE__#${base}#g" \
"/tmp/nginx-conf/${variant}.subpath.conf.template" > /etc/nginx/conf.d/default.conf; \
fi; \
rm -rf /tmp/dist /tmp/nginx-conf
EXPOSE 80
+34 -15
View File
@@ -24,13 +24,14 @@ Here's what Homelable can do. One line on what each feature is, then how to swit
10. [Z-Wave Import](#10-z-wave-import-)
11. [Proxmox VE Import](#11-proxmox-ve-import-)
12. [Device Inventory](#12-device-inventory-)
13. [Live Status Monitoring](#13-live-status-monitoring-)
14. [Export (PNG / SVG / YAML / Markdown)](#14-export)
15. [Live View (read-only public canvas)](#15-live-view-)
16. [Gethomepage Widget](#16-gethomepage-widget-)
17. [MCP Server (AI integration)](#17-mcp-server-)
18. [Settings & Shortcuts](#18-settings--shortcuts)
19. [Authentication (Local / OpenID Connect)](#19-authentication-local--openid-connect-)
13. [Documentation](#13-documentation-)
14. [Live Status Monitoring](#14-live-status-monitoring-)
15. [Export (PNG / SVG / YAML / Markdown)](#15-export)
16. [Live View (read-only public canvas)](#16-live-view-)
17. [Gethomepage Widget](#17-gethomepage-widget-)
18. [MCP Server (AI integration)](#18-mcp-server-)
19. [Settings & Shortcuts](#19-settings--shortcuts)
20. [Authentication (Local / OpenID Connect)](#20-authentication-local--openid-connect-)
---
@@ -187,7 +188,25 @@ Nodes: `proxmox` (host) / `vm` / `lxc`, linked host→guest by a `virtual` edge.
---
## 13. Live Status Monitoring 🔒
## 13. Documentation 🔒
**What:** A markdown documentation space for the whole lab. Every device gets a document written once from what the scan actually found — identity, hardware, services, network, operations, troubleshooting — and it is yours from there; nothing rewrites it behind you. Next to it, a Library of pages you write yourself: runbooks, incidents, decisions, a network overview.
**Use:**
- Sidebar → **Documentation**. The **Devices** root re-pivots instantly by zone, group, type, physical/virtual, subnet, rack, vendor, discovery source, status, tag or A–Z; picking a device with no document writes one from its facts.
- The **Library** is folders you make. **New page** / **New folder** in the header, drag an item to file it. Templates: blank, runbook, service, network overview, incident, procedure, decision (ADR), zone.
- Editing is plain markdown and **saving is explicit**, like the canvas. An unsaved body is kept in your browser and offered back if you close the tab. `/` in the editor inserts a freshly generated block — services, hardware, network, rack — from the device's current data.
- **Link documents** with `[[VLAN plan]]`, `[[device:nas-01]]` or `[[doc:slug|label]]`. Typing `[[` opens a picker that writes the link for you. A link to a document that does not exist yet renders red and offers to create it, and every document lists **Linked from** at the bottom — who points here.
- **History**: the clock-arrow button in the header lists earlier versions, up to 50 per document. Read one, hit **Changes** for a line-by-line diff against the current body, **Restore** to bring it back — the body it replaces is saved to the history first.
- **Keeping it honest**: `review_every: 6m` in a document's frontmatter badges it "due for review" when it lapses; a **device data changed** banner appears when the device has moved on since the document was written; **Regenerate** rebuilds a device document from scratch (old body kept in the history). Tags are chips under the title.
- **Search** the whole space from the box above the tree — title, tags and body. The canvas' own search (Ctrl/Cmd+K) finds documents too, and picking one jumps here.
- Coming from the old per-device **Notes** field? A banner offers to migrate every device that has one into a document. It copies, it does not move: the `notes` field is left exactly as it was.
> **Full documentation:** [docs/documentation.md](./docs/documentation.md)
---
## 14. Live Status Monitoring 🔒
**What:** Keeps checking each node and shows its status (🟢 online / 🔴 offline / ⚫ unknown) right on the canvas.
@@ -208,7 +227,7 @@ Nodes: `proxmox` (host) / `vm` / `lxc`, linked host→guest by a `virtual` edge.
---
## 14. Export
## 15. Export
**What:** Get your canvas out as a picture or as structured data.
@@ -220,7 +239,7 @@ Nodes: `proxmox` (host) / `vm` / `lxc`, linked host→guest by a `virtual` edge.
---
## 15. Live View 🔒
## 16. Live View 🔒
**What:** A read-only, no-login snapshot of a canvas you can share on your LAN. Off by default.
@@ -232,7 +251,7 @@ Pan and zoom only, no editing. Click a node with an IP and it opens in a new tab
---
## 16. Gethomepage Widget 🔒
## 17. Gethomepage Widget 🔒
**What:** A tiny JSON stats endpoint for [gethomepage](https://gethomepage.dev)'s `customapi` widget. Off by default.
@@ -244,7 +263,7 @@ Widget snippet lives in the [README](./README.md#gethomepage-widget-read-only-st
---
## 17. MCP Server 🔒
## 18. MCP Server 🔒
**What:** A [Model Context Protocol](https://modelcontextprotocol.io) server so an MCP client (Claude Code, Claude Desktop, Open WebUI…) can read and change your topology. Optional, runs as its own service.
@@ -258,11 +277,11 @@ Widget snippet lives in the [README](./README.md#gethomepage-widget-read-only-st
2. `docker compose up -d mcp` (listens on `:8001`). No Docker? `sudo bash scripts/lxc-mcp-install.sh`.
3. Point your client at `http://<your-homelab-ip>:8001/mcp` with header `X-API-Key: <your key>`.
The AI can list nodes/edges/canvas/pending/scans, add/update/delete nodes and edges, kick off scans, and approve or hide devices. Keep port 8001 firewalled to your LAN. Full setup in the [README](./README.md#mcp-server-ai-integration-optional).
The AI can list nodes/edges/canvas/zones/designs/inventory/scans, add/update/delete nodes, edges and zones, kick off scans, triage and edit inventory entries, and work a rack canvas — create racks, mount and move gear, patch cables. Keep port 8001 firewalled to your LAN. Full setup in the [README](./README.md#mcp-server-ai-integration-optional).
---
## 18. Settings & Shortcuts
## 19. Settings & Shortcuts
**What:** App config and keyboard shortcuts.
@@ -273,7 +292,7 @@ The AI can list nodes/edges/canvas/pending/scans, add/update/delete nodes and ed
---
## 19. Authentication (Local / OpenID Connect) 🔒
## 20. Authentication (Local / OpenID Connect) 🔒
**What:** Homelable protects the app behind a login. Two exclusive modes, set once in `.env` with `AUTH_MODE`:
- **`local`** (default) — a single username + bcrypt-hashed password. Nothing changes for existing installs.
+272
View File
@@ -63,6 +63,48 @@ docker compose -f docker-compose.prebuilt.yml up -d
Update the same way: `docker compose -f docker-compose.prebuilt.yml pull && … up -d`.
## Scanning from Docker — MAC addresses
Out of the box the backend runs on a Docker bridge network, and **the scan will
never report a MAC address** from it. MACs come from ARP, which is layer 2: the
container's ARP cache holds only the Docker gateway, and nmap can read a target's
hardware address only when that target sits in the same broadcast domain. From a
bridge every LAN host is one hop away behind the gateway, so the field stays
empty. `cap_add: NET_RAW` does not change this — the capability grants raw
sockets, not a place on the LAN.
This also affects device identity. When a device is rescanned it is matched on
MAC first and IP second, so without MACs a DHCP lease change makes the device
come back as a new entry in the inventory instead of updating the old one.
To collect MACs, put the backend on the LAN itself. In `docker-compose.yml` (or
`docker-compose.prebuilt.yml`), comment out the backend's `networks:` key and
uncomment:
```yaml
network_mode: host
```
then `docker compose up -d`. Caveats:
- **Linux only.** On Docker Desktop for macOS and Windows the containers run
inside a VM, so host networking still does not reach your physical LAN. There
is no MAC-capable Docker setup on those platforms — run the
[bare-metal install](#bare-metal--no-docker) instead.
- The backend binds `8000` directly on the host, with no port mapping and no
network isolation from other host services.
- `frontend` and `mcp` reach the backend at `http://backend:8000` over the
`homelable` bridge; once the backend leaves that network they need
`http://127.0.0.1:8000` instead. Set `BACKEND_URL` on `mcp`, and for the front
end either give it `network_mode: host` too or point its nginx proxy at the
host address.
The alternative, if you would rather keep the backend isolated, is a **macvlan**
network, which gives the container its own MAC and IP on your physical LAN.
It needs a parent interface and a spare address range from your subnet, and on
most setups the Docker host itself cannot talk to a macvlan container without an
extra shim interface.
## Build from source
`docker-compose.yml` at the repo root builds the images locally instead of pulling them — use it for development, or to run a patched tree.
@@ -89,6 +131,236 @@ bash -c "$(curl -fsSL https://raw.githubusercontent.com/community-scripts/Proxmo
---
## Bare metal — no Docker
`scripts/install-baremetal.sh` installs Homelable natively on a Debian 12+ /
Ubuntu 22.04+ host (physical, VM or LXC): a Python venv and a `homelable`
systemd unit for the backend on `127.0.0.1:8000`, the built frontend served by
nginx on port 3000.
```bash
git clone https://github.com/Pouzor/homelable.git /opt/homelable
sudo bash /opt/homelable/scripts/install-baremetal.sh
```
It prompts for the admin password and the CIDR range to scan, then writes
`backend/.env` with a generated `SECRET_KEY` and bcrypt hash. Open
**http://\<host-ip\>:3000**.
The script can also clone for you — run it from anywhere and it fetches the repo
into `INSTALL_DIR` when that directory is empty. Piped into `bash` there is no
terminal to prompt on, so pass the two answers as environment variables:
```bash
curl -fsSL https://raw.githubusercontent.com/Pouzor/homelable/main/scripts/install-baremetal.sh \
| sudo ADMIN_PASSWORD=hunter2 SCANNER_RANGES='["192.168.1.0/24"]' bash
```
Without them the prompts are skipped and their defaults apply — the password
becomes `admin` and the range is guessed from the primary interface. The script
warns when that happens; change the password before exposing the host.
Re-running is safe and is how you upgrade — an existing `backend/.env` is kept
untouched, everything else is rebuilt:
```bash
cd /opt/homelable && git pull
sudo bash scripts/install-baremetal.sh
```
### Options
Every setting is an environment variable. Setting them all skips every prompt,
which is what makes an unattended install possible:
```bash
sudo HTTP_PORT=8080 ADMIN_PASSWORD=hunter2 SCANNER_RANGES='["10.0.0.0/24"]' \
bash scripts/install-baremetal.sh
```
| Variable | Default | What |
|---|---|---|
| `INSTALL_DIR` | `/opt/homelable` | Repo root |
| `REPO_URL` / `REPO_REF` | upstream / `main` | Used only when `INSTALL_DIR` is empty |
| `SERVICE_USER` | `homelable` | systemd `User=` |
| `BACKEND_PORT` | `8000` | uvicorn port, bound to loopback |
| `HTTP_PORT` | `3000` | nginx port |
| `SERVER_NAME` | `_` | nginx `server_name` |
| `ADMIN_PASSWORD` | prompt, else `admin` | Initial password for user `admin` |
| `SCANNER_RANGES` | prompt, else guessed | JSON array of CIDRs |
| `SKIP_NGINX=1` | off | Do not install or touch nginx |
| `BASE_PATH` | `/` | Serve under a subpath — see [Serving under a subpath](#serving-under-a-subpath) |
### Afterwards
```bash
systemctl status homelable
journalctl -u homelable -f
```
- Config: `/opt/homelable/backend/.env` — every other option (OIDC, MCP,
Proxmox, Zigbee, Z-Wave, live view) is documented in `.env.example`.
`systemctl restart homelable` after an edit.
- Data: `/opt/homelable/data` — SQLite DB and uploads. Back up this folder.
- nginx site: `/etc/nginx/sites-available/homelable`.
Change the password later:
```bash
/opt/homelable/backend/.venv/bin/python -c \
'import bcrypt; print(bcrypt.hashpw(b"newpassword", bcrypt.gensalt()).decode())'
# put it in backend/.env as AUTH_PASSWORD_HASH='$2b$12$...' (keep the single quotes)
systemctl restart homelable
```
⚠️ Keep JSON values in `backend/.env` single-quoted — `CORS_ORIGINS='["http://…"]'`.
systemd's `EnvironmentFile` parser strips bare double quotes, which breaks the
JSON before the backend parses it. This does not apply to the Docker install.
### Scanning as a non-root service
The unit runs as `homelable`, not root, so nmap has no raw sockets and silently
falls back to a TCP connect scan: hosts and open ports are still found, OS
detection (`-O`) and SYN scan (`-sS`) are not. To grant them, uncomment in
`/etc/systemd/system/homelable.service`:
```ini
AmbientCapabilities=CAP_NET_RAW CAP_NET_ADMIN
CapabilityBoundingSet=CAP_NET_RAW CAP_NET_ADMIN
```
then `systemctl daemon-reload && systemctl restart homelable`.
### Your own reverse proxy
With `SKIP_NGINX=1` the script leaves the front end to you: serve
`/opt/homelable/frontend/dist` as a static SPA and proxy the API to the backend.
The nginx translation of `docker/nginx.conf` — what the script writes, with
`backend:8000` replaced by `127.0.0.1:8000`:
```nginx
server {
listen 3000;
server_name _;
root /opt/homelable/frontend/dist;
index index.html;
client_max_body_size 20M;
# WebSocket — must come before /api/ to take priority
location /api/v1/status/ws/ {
proxy_pass http://127.0.0.1:8000;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
}
location /api/ {
proxy_pass http://127.0.0.1:8000;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
}
# Legacy /ws/ path
location /ws/ {
proxy_pass http://127.0.0.1:8000;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;
}
location / {
try_files $uri $uri/ /index.html;
}
}
```
Terminating TLS in front of it means adding your own hostname to
`CORS_ORIGINS` in `backend/.env` (`https://homelable.example`) and restarting
the service.
---
## Serving under a subpath
By default Homelable owns the root of its origin (`https://homelable.example/`).
To put it behind an existing reverse proxy on a shared hostname — one cert, one
dynamic-DNS name, one open port, every service on its own prefix — build it with
a base path.
The base path is **baked into the build**: the browser has no way to guess it, so
it cannot be a runtime setting. Changing it means rebuilding the frontend.
### Docker
```bash
# in .env, next to the backend settings
VITE_BASE_PATH=/homelab/
docker compose build frontend && docker compose up -d
```
Homelable then answers on `http://<host>:3000/homelab/`. The generated nginx
config accepts both reverse-proxy styles, so either of these works in front of
it:
```nginx
# prefix forwarded intact — no trailing slash on proxy_pass
location /homelab/ {
proxy_pass http://127.0.0.1:3000;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;
}
# or: prefix stripped — trailing slash on proxy_pass
location /homelab/ {
proxy_pass http://127.0.0.1:3000/;
...
}
```
The pre-built images (`docker-compose.prebuilt.yml`) are built for the root, so a
subpath needs a local build.
### Bare metal / LXC
```bash
sudo BASE_PATH=/homelab/ bash scripts/install-baremetal.sh
```
The script bakes the prefix into the build and writes the matching nginx site.
The build stays in `/opt/homelable/frontend/dist`; the site serves it through
`/var/www/homelable/homelab`, a symlink refreshed on every run. Re-running the
script with a different `BASE_PATH` (or none) rewrites both.
### Development
```bash
cd frontend && VITE_BASE_PATH=/homelab/ npm run dev # http://localhost:5173/homelab/
```
The Vite dev proxy follows the same prefix and strips it before forwarding to
uvicorn on `:8000`.
### What to expect
- WebSocket status updates, uploaded floor plans and the read-only live view
(`/homelab/view`) all follow the prefix.
- TLS in front still means adding your hostname to `CORS_ORIGINS` in
`backend/.env`, exactly as at the root.
- OIDC: `OIDC_REDIRECT_URI` must carry the prefix
(`https://home.example/homelab/api/v1/auth/oidc/callback`), and so must the
redirect URI registered with your provider.
- Floor plans uploaded before the move keep working — stored URLs are resolved
against the base path at render time.
---
## Configuration
All configuration is done via `.env` (copied from `.env.example`):
+40 -6
View File
@@ -35,6 +35,8 @@ Homelable is a self-hosted infrastructure visualization solution. It provides a
Homelable also offers a healthcheck system through multiple methods (ping/TCP, /health API, etc.) to get a global overview of online/offline services.
Every device also gets a **document** — written for you from what the scan found, and yours to maintain from there — next to a Library of pages you write yourself: runbooks, incidents, a network overview. Your homelab stops being documented in a wiki somewhere else.
You can also select some pre-built design styles, or personalize each device in your diagram.
If you just like the design, you can only run the frontend and export your design as PNG.
@@ -48,8 +50,8 @@ If you are running <img width="22" height="22" align="top" alt="New_Home_Assist
<p align="center">
<img src="docs/homelable1.png" alt="Homelable canvas overview" width="100%" />
<img alt="Homelable Device inventory" src="https://github.com/user-attachments/assets/f3903ac8-354d-4873-81ba-1914971890ed" />
<img width="49.5%" alt="Homelable Custom node" src="https://github.com/user-attachments/assets/813725b1-376b-4bad-bb1f-0985f3bc7546" />
<img width="49.5%" alt="Homelable Zigbee Network" src="https://github.com/user-attachments/assets/35e18d11-8363-498d-ae3d-642685cac76d" />
<img alt="Rack display" src="https://github.com/user-attachments/assets/43273605-4f46-4163-8aea-8bf8a76a3f76" />
</p>
@@ -57,7 +59,7 @@ If you are running <img width="22" height="22" align="top" alt="New_Home_Assist
## Features
From one-click **network scans** and **Proxmox / Zigbee / Z-Wave** imports to **live status monitoring**, floor plans, **rack canvases** with port-to-port patching, multi-canvas layouts and an **MCP server** for AI assistants — Homelable maps and watches your whole homelab.
From one-click **network scans** and **Proxmox / Zigbee / Z-Wave** imports to **live status monitoring**, floor plans, **rack canvases** with port-to-port patching, a **markdown documentation space** for the whole lab, multi-canvas layouts and an **MCP server** for AI assistants — Homelable maps, documents and watches your whole homelab.
Every feature, with how to turn it on and use it, is described in **[FEATURES.md](./FEATURES.md)**.
@@ -65,7 +67,7 @@ Every feature, with how to turn it on and use it, is described in **[FEATURES.md
## Installation
Docker (from the **[pre-built GHCR images](./INSTALLATION.md#pre-built-docker-images)**), Proxmox LXC, build from source, configuration, and development setup are all covered in **[INSTALLATION.md](./INSTALLATION.md)**.
Docker (from the **[pre-built GHCR images](./INSTALLATION.md#pre-built-docker-images)**), Proxmox LXC, **[bare metal without Docker](./INSTALLATION.md#bare-metal--no-docker)** (`sudo bash scripts/install-baremetal.sh` — systemd unit plus nginx on a Debian/Ubuntu host), build from source, configuration, and development setup are all covered in **[INSTALLATION.md](./INSTALLATION.md)**.
---
@@ -87,6 +89,28 @@ Gear sits in a U range and part of a 12-column width grid, so half- and third-wi
---
## Documentation
Homelable keeps its own markdown documentation space, so the lab is described where it is drawn. Sidebar → **Documentation**.
Every device has a document, generated once from what the scan actually found — identity, hardware, one section per service, network, operations, troubleshooting — and never rewritten behind you. Beside it, a **Library** of pages you write: runbooks, incidents, decisions, a network overview, from a template or blank.
### Usage
1. **Pick a device** in the tree — one with no document yet gets one written from its facts. Re-pivot the tree by zone, subnet, type, rack, vendor, tag… at no cost; it is grouped from data the app already holds
2. **Write markdown**, save explicitly — nothing is ever saved behind your back. `/` inserts a freshly generated block (services, hardware, network, rack) from the device's current data
3. **Link things**: `[[VLAN plan]]`, `[[device:nas-01]]`. Each document lists **Linked from** at the bottom, so you can see what points at it
4. **History**: up to 50 versions per document. Read one, diff it against the current body, restore it — what it replaces is kept too
5. **Search** the whole space, tag documents, and set `review_every: 6m` on the ones that rot; a document past its interval is badged as due
A device that changes after its document was written raises a **device data changed** banner, and **Regenerate** rebuilds the document from scratch when you want that. Old per-device **Notes** migrate into documents from a banner, non-destructively.
Full mode only — documents need the backend to store, index and search them.
> **Full documentation:** [docs/documentation.md](./docs/documentation.md)
---
## Network Scanner
The scanner runs `nmap -sV --open` on your configured CIDR ranges and populates a **Pending Devices** queue. From the sidebar you can then approve (adds a node to the canvas), hide, or ignore each discovered device.
@@ -110,6 +134,14 @@ SCANNER_HTTP_VERIFY_TLS=false # verify TLS certs on the HTTP probe
The listed ports are appended to nmap's `-p` spec. Invalid entries (out-of-range, malformed, or reversed ranges) are silently skipped.
### MAC addresses (Docker)
The scan reports no MAC address when the backend runs on a Docker bridge
network — ARP is layer 2, and from a bridge every LAN host sits behind the
Docker gateway. It also means a DHCP device that changes IP comes back as a new
inventory entry, since matching prefers the MAC. Fix and caveats:
[INSTALLATION.md](./INSTALLATION.md#scanning-from-docker--mac-addresses).
### macOS / root privileges
Some nmap scan types (SYN scan, OS detection) require root. If the scan fails with a permissions error, run it manually with sudo using the included script:
@@ -328,8 +360,8 @@ Homelable can exposes a [Model Context Protocol](https://modelcontextprotocol.io
| | Action |
|---|---|
| **Read** | List all nodes, edges, full canvas, pending devices, scan history |
| **Write** | Add / update / delete nodes and edges, trigger a network scan, approve or hide discovered devices |
| **Read** | List all nodes, edges, full canvas, zones, designs, the device inventory and scan history — and the rack canvases: racks, mounted gear, patched cables |
| **Write** | Add / update / delete nodes, edges and zones, trigger a network scan, approve / hide / restore discovered devices, create and edit inventory entries, build racks and mount, move and patch the gear in them |
### Setup
@@ -405,6 +437,8 @@ Or add it manually to `~/.claude.json`:
- *"Add a new LXC container named `pihole` at 192.168.1.5, connected to my switch."*
- *"Trigger a network scan on 192.168.1.0/24 and show me the pending devices."*
- *"Show me the full canvas topology."*
- *"How much free U is left in the garage rack?"*
- *"Mount the NAS in rack 1 and patch its first port to port 12 of the patch panel."*
### Security
+1 -1
View File
@@ -1 +1 @@
3.3.2
3.4.1
+5 -2
View File
@@ -10,7 +10,7 @@ from pydantic import BaseModel
from starlette.responses import RedirectResponse
from app.api.deps import AuthContext, get_auth_context
from app.core.config import settings
from app.core.config import app_base_path_of, settings
from app.core.security import (
clear_oidc_session_cookie,
create_access_token,
@@ -103,7 +103,10 @@ async def oidc_callback(request: Request) -> Response:
raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="OIDC authentication failed") from None
request.session.clear()
response = RedirectResponse(url="/", status_code=status.HTTP_303_SEE_OTHER)
# Back to the SPA — which is not necessarily at the root of the origin.
response = RedirectResponse(
url=app_base_path_of(settings.oidc_redirect_uri), status_code=status.HTTP_303_SEE_OTHER
)
set_oidc_session_cookie(response, session_token)
return response
+3
View File
@@ -8,6 +8,7 @@ from app.api.deps import get_current_user
from app.db.database import get_db
from app.db.models import CanvasState, Design, Edge, Node, Rack, RackCable, RackDevice
from app.schemas.designs import DesignCopy, DesignCreate, DesignResponse, DesignUpdate
from app.services.doc_links import unlink_documents
router = APIRouter()
@@ -233,6 +234,8 @@ async def delete_design(
for e in edges:
await db.delete(e)
nodes = (await db.execute(select(Node).where(Node.design_id == design_id))).scalars().all()
# Foreign keys are off, so orphan the documents by hand before the rows go.
await unlink_documents(db, node_ids=[n.id for n in nodes], design_ids=[design_id])
for n in nodes:
await db.delete(n)
await db.delete(design)
+719
View File
@@ -0,0 +1,719 @@
"""Documents — the markdown documentation space.
A document is either a page or a folder in the Library tree, or it describes one
thing elsewhere in the app: a Device Inventory row, a piece of canvas furniture
(a zone, a group), or a whole canvas. The tree the user navigates is *not* stored
here: device documents are pivoted client-side by zone, subnet, type and so on,
because every one of those groupings is derivable from data the frontend already
holds and re-pivoting must be instant.
The generated header is written once, at creation. Nothing in this module ever
rewrites a body the user owns; `GET /blocks` hands the editor a freshly generated
section on request, and `facts_snapshot` is what lets the UI say the device has
moved on since the document was written.
"""
import re
from datetime import datetime, timedelta, timezone
from typing import Any
from fastapi import APIRouter, Depends, HTTPException, Query
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.api.deps import get_current_user
from app.db.database import get_db
from app.db.models import Document, DocumentRevision, Edge, InventoryDevice, Node, Rack, RackDevice
from app.schemas.documents import (
BacklinkHit,
CoverageResponse,
DocumentCreate,
DocumentResponse,
DocumentSummary,
DocumentUpdate,
RevisionResponse,
RevisionSummary,
ScaffoldRequest,
ScaffoldResponse,
SearchHit,
SearchResponse,
)
from app.services import doc_backlinks, doc_search
from app.services.doc_template import (
BLOCKS,
TEMPLATE_DEVICE,
facts_snapshot,
render_block,
render_device_document,
render_library_document,
)
from app.services.doc_tree import (
REVISION_LIMIT,
TREE_KINDS,
is_ancestor,
parse_frontmatter,
subtree_ids,
tags_from,
unique_slug,
)
router = APIRouter()
# Node.type values that are canvas furniture. A zone or a group is documented
# through its node; everything else is documented through its device.
_FURNITURE_TYPES = {"group", "groupRect", "text"}
_INTERVAL = re.compile(r"^\s*(\d+)\s*([dwmy])\s*$", re.IGNORECASE)
_INTERVAL_DAYS = {"d": 1, "w": 7, "m": 30, "y": 365}
def _now() -> datetime:
return datetime.now(timezone.utc)
def _aware(value: datetime | None) -> datetime | None:
"""SQLite hands back naive datetimes; compare them as UTC."""
if value is None:
return None
return value if value.tzinfo else value.replace(tzinfo=timezone.utc)
def _parse_interval(raw: Any) -> timedelta | None:
"""`review_every: 6m` → 180 days. Anything unparseable means "never due"."""
match = _INTERVAL.match(str(raw or ""))
if not match:
return None
return timedelta(days=int(match.group(1)) * _INTERVAL_DAYS[match.group(2).lower()])
def _apply_body(doc: Document, body: str) -> None:
"""Store a body and refresh the frontmatter/tags cache from it.
The body is the source of truth — it is what exports to disk — so the two
JSON columns are only ever derived from it, never edited on their own.
"""
doc.body = body
frontmatter = parse_frontmatter(body)
doc.frontmatter = frontmatter
doc.tags = tags_from(frontmatter)
async def _adopt_frontmatter_title(db: AsyncSession, doc: Document) -> None:
"""Follow the name the body gives, since the body is what the user edits.
`title:` in the frontmatter is the only place a document is named — there is
no rename field anywhere in the UI — so a row that kept the title it was
created with went stale the moment someone edited that line, and the tree
went on showing the old name.
"""
front = doc.frontmatter if isinstance(doc.frontmatter, dict) else {}
title = front.get("title")
if not isinstance(title, str) or not title.strip() or title.strip() == doc.title:
return
doc.title = title.strip()
doc.slug = await unique_slug(db, doc.title, parent_id=doc.parent_id, exclude_id=doc.id)
async def _record_revision(db: AsyncSession, doc: Document, reason: str) -> None:
"""Snapshot the current body, then prune the document's oldest history."""
db.add(DocumentRevision(document_id=doc.id, title=doc.title, body=doc.body or "", reason=reason))
await db.flush()
stale = (
await db.execute(
select(DocumentRevision)
.where(DocumentRevision.document_id == doc.id)
.order_by(DocumentRevision.saved_at.desc(), DocumentRevision.id.desc())
.offset(REVISION_LIMIT)
)
).scalars().all()
for revision in stale:
await db.delete(revision)
async def _device_context(db: AsyncSession, device_id: str) -> dict[str, Any]:
"""Zone, rack placement and canvas neighbours for a device.
All three are optional: a device may be on no canvas, in no rack and in no
zone, and the template drops those sections rather than printing them empty.
"""
context: dict[str, Any] = {"zone_label": None, "rack": None, "connections": []}
node = (
await db.execute(select(Node).where(Node.device_id == device_id).order_by(Node.created_at))
).scalars().first()
if node is not None:
parent_id = node.parent_id
seen: set[str] = set()
while parent_id and parent_id not in seen:
seen.add(parent_id)
parent = await db.get(Node, parent_id)
if parent is None:
break
if parent.type in _FURNITURE_TYPES:
context["zone_label"] = parent.label
break
parent_id = parent.parent_id
edges = (
await db.execute(select(Edge).where((Edge.source == node.id) | (Edge.target == node.id)))
).scalars().all()
peer_ids = [e.target if e.source == node.id else e.source for e in edges]
if peer_ids:
peers = (await db.execute(select(Node.label).where(Node.id.in_(peer_ids)))).scalars().all()
context["connections"] = sorted({label for label in peers if label})
mount = (
await db.execute(select(RackDevice).where(RackDevice.device_id == device_id))
).scalars().first()
if mount is not None:
rack = await db.get(Rack, mount.rack_id)
context["rack"] = {
"name": rack.name if rack else None,
"u_start": mount.u_start,
"u_height": mount.u_height,
"col_span": mount.col_span,
}
return context
async def _scaffold_body(db: AsyncSession, doc: Document, template_id: str | None) -> None:
"""Generate this document's initial body. Called once, at creation."""
if doc.device_id:
device = await db.get(InventoryDevice, doc.device_id)
if device is not None:
context = await _device_context(db, doc.device_id)
_apply_body(doc, render_device_document(device, **context))
doc.facts_snapshot = facts_snapshot(device)
doc.facts_synced_at = _now()
doc.template_id = TEMPLATE_DEVICE
return
_apply_body(doc, render_library_document(template_id or "blank", doc.title))
doc.template_id = template_id or "blank"
def _has_drifted(doc: Document, device: InventoryDevice | None) -> bool:
"""Whether the device has moved on since this document was snapshotted.
The comparison lives on the server rather than in the UI because
`facts_snapshot` is the server's own shape — `label` and `type` are stored
through their fallbacks and `properties` as a flat map — so nothing else
can compare it to a device row correctly. Same rule as the coverage count.
"""
if not doc.device_id or not doc.facts_snapshot or device is None:
return False
return doc.facts_snapshot != facts_snapshot(device)
async def _devices_for(db: AsyncSession, docs: list[Document]) -> dict[str, InventoryDevice]:
"""The devices a batch of documents describes, in one query."""
wanted = {d.device_id for d in docs if d.device_id and d.facts_snapshot}
if not wanted:
return {}
rows = (
await db.execute(select(InventoryDevice).where(InventoryDevice.id.in_(wanted)))
).scalars().all()
return {device.id: device for device in rows}
def _summary(doc: Document, device: InventoryDevice | None) -> DocumentSummary:
payload = DocumentSummary.model_validate(doc)
payload.drifted = _has_drifted(doc, device)
return payload
async def _response(db: AsyncSession, doc: Document) -> DocumentResponse:
"""One document, with the drift flag resolved against the live device."""
payload = DocumentResponse.model_validate(doc)
payload.drifted = _has_drifted(doc, await db.get(InventoryDevice, doc.device_id) if doc.device_id else None)
return payload
# ── list / read ─────────────────────────────────────────────────────────────
@router.get("", response_model=list[DocumentSummary])
async def list_documents(
kind: str | None = Query(None, description="Filter to one document kind"),
parent_id: str | None = Query(None, description="Library children of this folder"),
device_id: str | None = Query(None, description="The document describing this device"),
tag: str | None = Query(None, description="Documents carrying this frontmatter tag"),
db: AsyncSession = Depends(get_db),
_: str = Depends(get_current_user),
) -> list[DocumentSummary]:
query = select(Document)
if kind:
query = query.where(Document.kind == kind)
if parent_id:
query = query.where(Document.parent_id == parent_id)
if device_id:
query = query.where(Document.device_id == device_id)
docs = (await db.execute(query.order_by(Document.sort_order, Document.title))).scalars().all()
if tag:
wanted = tag.lower()
docs = [d for d in docs if any(str(t).lower() == wanted for t in (d.tags or []))]
devices = await _devices_for(db, list(docs))
return [_summary(d, devices.get(d.device_id or "")) for d in docs]
@router.get("/coverage", response_model=CoverageResponse)
async def coverage(
db: AsyncSession = Depends(get_db),
_: str = Depends(get_current_user),
) -> CoverageResponse:
"""How much of the homelab is actually written down."""
devices = (
await db.execute(select(InventoryDevice).where(InventoryDevice.status != "hidden"))
).scalars().all()
docs = (await db.execute(select(Document))).scalars().all()
by_device = {d.device_id: d for d in docs if d.device_id}
header_only = drifted = overdue = 0
now = _now()
for device in devices:
doc = by_device.get(device.id)
if doc is None:
continue
# Never edited since it was generated: the template is all there is.
if doc.edited_at is None and doc.reviewed_at is None:
header_only += 1
if doc.facts_snapshot and doc.facts_snapshot != facts_snapshot(device):
drifted += 1
interval = _parse_interval((doc.frontmatter or {}).get("review_every"))
since = _aware(doc.reviewed_at) or _aware(doc.created_at)
if interval and since and now - since > interval:
overdue += 1
documented = sum(1 for device in devices if device.id in by_device)
return CoverageResponse(
devices=len(devices),
documented=documented,
header_only=header_only,
missing=len(devices) - documented,
drifted=drifted,
overdue=overdue,
notes_unmigrated=sum(
1 for device in devices if (device.notes or "").strip() and device.id not in by_device
),
library_pages=sum(1 for d in docs if d.kind in TREE_KINDS),
)
@router.get("/search", response_model=SearchResponse)
async def search_documents(
q: str = Query(..., description="Full-text query"),
limit: int = Query(25, ge=1, le=100),
db: AsyncSession = Depends(get_db),
_: str = Depends(get_current_user),
) -> SearchResponse:
engine, rows = await doc_search.search(db, q, limit)
if not rows:
return SearchResponse(engine=engine, hits=[])
docs = {
d.id: d
for d in (
await db.execute(select(Document).where(Document.id.in_([r["doc_id"] for r in rows])))
).scalars().all()
}
hits = [
SearchHit(
doc_id=row["doc_id"],
title=docs[row["doc_id"]].title,
kind=docs[row["doc_id"]].kind,
snippet=row["snippet"] or "",
device_id=docs[row["doc_id"]].device_id,
)
for row in rows
if row["doc_id"] in docs
]
return SearchResponse(engine=engine, hits=hits)
@router.get("/blocks", response_model=dict)
async def generated_block(
block: str = Query(..., description=f"One of {sorted(BLOCKS)}"),
device_id: str = Query(..., description="Device to read the facts from"),
db: AsyncSession = Depends(get_db),
_: str = Depends(get_current_user),
) -> dict[str, str]:
"""Render one generated section, for the editor's `/` insert menu.
This is the counterpart to generating the header only once: the user can
always pull a fresh block in, but nothing pushes one at them.
"""
if block not in BLOCKS:
raise HTTPException(400, f"block must be one of {sorted(BLOCKS)}")
device = await db.get(InventoryDevice, device_id)
if not device:
raise HTTPException(404, "Device not found")
context = await _device_context(db, device_id) if block in {"rack", "network"} else {}
return {"block": block, "markdown": render_block(block, device, **context)}
@router.get("/{document_id}", response_model=DocumentResponse)
async def get_document(
document_id: str,
db: AsyncSession = Depends(get_db),
_: str = Depends(get_current_user),
) -> DocumentResponse:
doc = await db.get(Document, document_id)
if not doc:
raise HTTPException(404, "Document not found")
return await _response(db, doc)
@router.get("/{document_id}/revisions", response_model=list[RevisionSummary])
async def list_revisions(
document_id: str,
db: AsyncSession = Depends(get_db),
_: str = Depends(get_current_user),
) -> list[RevisionSummary]:
doc = await db.get(Document, document_id)
if not doc:
raise HTTPException(404, "Document not found")
revisions = (
await db.execute(
select(DocumentRevision)
.where(DocumentRevision.document_id == document_id)
.order_by(DocumentRevision.saved_at.desc(), DocumentRevision.id.desc())
)
).scalars().all()
return [
RevisionSummary(
id=r.id,
document_id=r.document_id,
title=r.title,
reason=r.reason,
saved_at=r.saved_at,
size=len(r.body or ""),
)
for r in revisions
]
@router.get("/{document_id}/backlinks", response_model=list[BacklinkHit])
async def list_backlinks(
document_id: str,
db: AsyncSession = Depends(get_db),
_: str = Depends(get_current_user),
) -> list[BacklinkHit]:
"""The documents whose body links here.
Answered on the server because the browser only holds document *metadata* —
the list endpoint carries no bodies, and loading every body to invert the
links client-side would trade a small query for a large download on every
open.
"""
doc = await db.get(Document, document_id)
if not doc:
raise HTTPException(404, "Document not found")
# Every document is loaded because every one is a possible *target* of a
# link — a bare `[[VLAN plan]]` resolves by title. Only the bodies carrying
# `[[` are walked as sources, and the inventory is fetched only when a
# `[[device:…]]` is actually in play.
docs = list(
(
await db.execute(select(Document).order_by(Document.sort_order, Document.title))
).scalars().all()
)
devices = (
list((await db.execute(select(InventoryDevice))).scalars().all())
if doc_backlinks.has_device_link(docs)
else []
)
titles = {d.id: d for d in docs}
return [
BacklinkHit(
doc_id=hit.doc_id,
title=titles[hit.doc_id].title,
kind=titles[hit.doc_id].kind,
device_id=titles[hit.doc_id].device_id,
label=hit.label,
context=hit.context,
count=hit.count,
)
for hit in doc_backlinks.backlinks_for(document_id, docs, devices)
if hit.doc_id in titles
]
@router.get("/revisions/{revision_id}", response_model=RevisionResponse)
async def get_revision(
revision_id: str,
db: AsyncSession = Depends(get_db),
_: str = Depends(get_current_user),
) -> RevisionResponse:
revision = await db.get(DocumentRevision, revision_id)
if not revision:
raise HTTPException(404, "Revision not found")
return RevisionResponse(
id=revision.id,
document_id=revision.document_id,
title=revision.title,
reason=revision.reason,
saved_at=revision.saved_at,
size=len(revision.body or ""),
body=revision.body or "",
)
# ── write ───────────────────────────────────────────────────────────────────
@router.post("", response_model=DocumentResponse, status_code=201)
async def create_document(
body: DocumentCreate,
db: AsyncSession = Depends(get_db),
_: str = Depends(get_current_user),
) -> DocumentResponse:
links = [body.device_id, body.node_id, body.design_id]
if sum(1 for link in links if link) > 1:
raise HTTPException(400, "A document describes at most one device, node or design")
if body.kind in TREE_KINDS and any(links):
raise HTTPException(400, "A page or folder cannot also describe a device, node or design")
if body.kind not in TREE_KINDS and not any(links):
raise HTTPException(400, f"A '{body.kind}' document must name what it describes")
if body.parent_id and body.kind not in TREE_KINDS:
raise HTTPException(400, "Only pages and folders live in the Library tree")
for link, model, what in (
(body.device_id, InventoryDevice, "Device"),
(body.node_id, Node, "Node"),
):
if link and not await db.get(model, link):
raise HTTPException(404, f"{what} not found")
if body.parent_id and not await db.get(Document, body.parent_id):
raise HTTPException(404, "Parent folder not found")
for column, value in (
(Document.device_id, body.device_id),
(Document.node_id, body.node_id),
(Document.design_id, body.design_id),
):
if value and (await db.execute(select(Document).where(column == value))).scalars().first():
raise HTTPException(409, "That already has a document")
doc = Document(
kind=body.kind,
title=body.title,
slug=await unique_slug(db, body.title, parent_id=body.parent_id),
icon=body.icon,
parent_id=body.parent_id,
device_id=body.device_id,
node_id=body.node_id,
design_id=body.design_id,
)
if body.body is not None:
_apply_body(doc, body.body)
doc.template_id = body.template_id
else:
await _scaffold_body(db, doc, body.template_id)
db.add(doc)
await db.flush()
await doc_search.index_document(db, doc)
await db.commit()
await db.refresh(doc)
return await _response(db, doc)
@router.patch("/{document_id}", response_model=DocumentResponse)
async def update_document(
document_id: str,
body: DocumentUpdate,
db: AsyncSession = Depends(get_db),
_: str = Depends(get_current_user),
) -> DocumentResponse:
doc = await db.get(Document, document_id)
if not doc:
raise HTTPException(404, "Document not found")
sent = body.model_dump(exclude_unset=True)
if "parent_id" in sent:
parent_id = sent["parent_id"]
if doc.kind not in TREE_KINDS:
raise HTTPException(400, "Only pages and folders live in the Library tree")
if parent_id:
parent = await db.get(Document, parent_id)
if not parent:
raise HTTPException(404, "Parent folder not found")
if parent.kind != "folder":
raise HTTPException(400, "A document can only be filed under a folder")
if await is_ancestor(db, document_id, parent_id):
raise HTTPException(400, "A folder cannot be moved inside itself")
doc.parent_id = parent_id
doc.slug = await unique_slug(db, doc.title, parent_id=parent_id, exclude_id=doc.id)
if "body" in sent and sent["body"] is not None and sent["body"] != doc.body:
await _record_revision(db, doc, "edit")
_apply_body(doc, sent["body"])
doc.edited_at = _now()
# An explicit title in the same request still wins: it is applied below.
await _adopt_frontmatter_title(db, doc)
if "title" in sent and sent["title"]:
doc.title = sent["title"]
doc.slug = await unique_slug(db, doc.title, parent_id=doc.parent_id, exclude_id=doc.id)
if "icon" in sent:
doc.icon = sent["icon"]
if "sort_order" in sent and sent["sort_order"] is not None:
doc.sort_order = sent["sort_order"]
if "starred" in sent and sent["starred"] is not None:
doc.starred = sent["starred"]
if sent.get("reviewed"):
doc.reviewed_at = _now()
if sent.get("resync_facts") and doc.device_id:
device = await db.get(InventoryDevice, doc.device_id)
if device is not None:
doc.facts_snapshot = facts_snapshot(device)
doc.facts_synced_at = _now()
await db.flush()
await doc_search.index_document(db, doc)
await db.commit()
await db.refresh(doc)
return await _response(db, doc)
@router.post("/{document_id}/revisions/{revision_id}/restore", response_model=DocumentResponse)
async def restore_revision(
document_id: str,
revision_id: str,
db: AsyncSession = Depends(get_db),
_: str = Depends(get_current_user),
) -> DocumentResponse:
doc = await db.get(Document, document_id)
if not doc:
raise HTTPException(404, "Document not found")
revision = await db.get(DocumentRevision, revision_id)
if not revision or revision.document_id != document_id:
raise HTTPException(404, "Revision not found")
# The body being replaced becomes history too, so a restore is undoable.
await _record_revision(db, doc, "restore")
_apply_body(doc, revision.body or "")
# A restored body brings its own title back with it.
await _adopt_frontmatter_title(db, doc)
await db.flush()
await doc_search.index_document(db, doc)
await db.commit()
await db.refresh(doc)
return await _response(db, doc)
@router.post("/{document_id}/regenerate", response_model=DocumentResponse)
async def regenerate_document(
document_id: str,
db: AsyncSession = Depends(get_db),
_: str = Depends(get_current_user),
) -> DocumentResponse:
"""Throw the body away and scaffold it again from what the database holds.
The one place that overwrites a body the user owns, so it is only ever
reached from an explicit confirmation. The replaced body is snapshotted
first, which makes the whole thing undoable from the history list.
"""
doc = await db.get(Document, document_id)
if not doc:
raise HTTPException(404, "Document not found")
if doc.kind == "folder":
raise HTTPException(400, "A folder has no body to regenerate")
if doc.device_id and not await db.get(InventoryDevice, doc.device_id):
raise HTTPException(404, "Device not found")
await _record_revision(db, doc, "regenerate")
await _scaffold_body(db, doc, doc.template_id)
# Back to a freshly generated document: nothing of the user's is left in it.
doc.edited_at = None
await db.flush()
await doc_search.index_document(db, doc)
await db.commit()
await db.refresh(doc)
return await _response(db, doc)
@router.post("/scaffold", response_model=ScaffoldResponse)
async def scaffold_documents(
body: ScaffoldRequest,
db: AsyncSession = Depends(get_db),
_: str = Depends(get_current_user),
) -> ScaffoldResponse:
"""Create the device documents that do not exist yet.
This is the migration off `device_inventory.notes`: the old text is appended
verbatim under `## Notes` and the column is left untouched, so nothing is
lost and the move is repeatable.
"""
query = select(InventoryDevice).where(InventoryDevice.status != "hidden")
if body.device_ids:
query = query.where(InventoryDevice.id.in_(body.device_ids))
devices = (await db.execute(query.order_by(InventoryDevice.discovered_at))).scalars().all()
existing = {
doc_id
for doc_id in (
await db.execute(select(Document.device_id).where(Document.device_id.is_not(None)))
).scalars().all()
}
created: list[Document] = []
skipped = 0
for device in devices:
if device.id in existing:
skipped += 1
continue
if body.only_with_notes and not (device.notes or "").strip():
skipped += 1
continue
context = await _device_context(db, device.id)
doc = Document(
kind="device",
title=device.label or device.friendly_name or device.hostname or device.ip or "device",
slug="",
device_id=device.id,
template_id=TEMPLATE_DEVICE,
facts_snapshot=facts_snapshot(device),
facts_synced_at=_now(),
)
doc.slug = await unique_slug(db, doc.title, parent_id=None)
_apply_body(doc, render_device_document(device, **context))
db.add(doc)
await db.flush()
db.add(
DocumentRevision(
document_id=doc.id,
title=doc.title,
body=doc.body,
reason="migrate" if (device.notes or "").strip() else "scaffold",
)
)
await doc_search.index_document(db, doc)
created.append(doc)
await db.commit()
for doc in created:
await db.refresh(doc)
return ScaffoldResponse(created=[DocumentSummary.model_validate(d) for d in created], skipped=skipped)
@router.delete("/{document_id}", status_code=204)
async def delete_document(
document_id: str,
db: AsyncSession = Depends(get_db),
_: str = Depends(get_current_user),
) -> None:
doc = await db.get(Document, document_id)
if not doc:
raise HTTPException(404, "Document not found")
# SQLite does not always enforce the ON DELETE CASCADE, so walk the subtree
# by hand — a folder takes its children with it.
for doc_id in reversed(await subtree_ids(db, document_id)):
target = await db.get(Document, doc_id)
if target is None:
continue
for revision in (
await db.execute(select(DocumentRevision).where(DocumentRevision.document_id == doc_id))
).scalars().all():
await db.delete(revision)
await doc_search.unindex_document(db, doc_id)
await db.delete(target)
await db.commit()
+8
View File
@@ -8,6 +8,7 @@ from app.api.deps import get_current_user
from app.db.database import get_db
from app.db.models import Design, InventoryDevice, Node
from app.schemas.nodes import NodeCreate, NodeResponse, NodeUpdate
from app.services.doc_links import unlink_documents
from app.services.inventory_sync import (
facts_from_payload,
facts_from_update,
@@ -150,6 +151,11 @@ async def update_node(
if not node:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Node not found")
sent = body.model_dump(exclude_unset=True)
# A node can never be its own parent: the canvas' parent walks assume an
# acyclic tree and a self-parent row freezes the node on screen (#370).
# Dropped rather than rejected so the rest of the edit still lands.
if sent.get("parent_id") == node_id:
sent.pop("parent_id")
for field, value in node_columns(sent).items():
setattr(node, field, value)
# The user edited the device, not just its drawing: push the facts down.
@@ -173,5 +179,7 @@ async def delete_node(node_id: str, db: AsyncSession = Depends(get_db), _: str =
node = await db.get(Node, node_id)
if not node:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Node not found")
# Foreign keys are off, so the schema's ON DELETE SET NULL does not fire.
await unlink_documents(db, node_ids=[node_id])
await db.delete(node)
await db.commit()
+90 -14
View File
@@ -33,7 +33,9 @@ from app.schemas.proxmox import (
ProxmoxTestConnectionResponse,
)
from app.schemas.scan import ScanRunResponse
from app.services.device_merge import reconcile_duplicates
from app.services.discovery_sources import add_source
from app.services.inventory_sync import attach_device_ids
from app.services.mac_utils import normalize_mac
from app.services.node_dedupe import dedupe_nodes_by_device
from app.services.proxmox_service import (
@@ -51,6 +53,9 @@ router = APIRouter()
# 'virtual' edges; host↔host cluster links render as 'cluster' edges.
_PROXMOX_GUEST_SOURCE = "proxmox"
_PROXMOX_CLUSTER_SOURCE = "proxmox_cluster"
# Every imported guest/host carries a synthetic ieee under this prefix
# (``pve-node-{host}`` / ``pve-{host}-{vmid}``), built in proxmox_service.
_PVE_IEEE_PREFIX = "pve-"
def _resolve_credentials(payload: ProxmoxConnectionRequest) -> tuple[str, str]:
@@ -88,6 +93,7 @@ async def test_connection_endpoint(
@router.post("/import", response_model=ProxmoxImportResponse)
async def import_proxmox(
payload: ProxmoxConnectionRequest,
db: AsyncSession = Depends(get_db),
_: str = Depends(get_current_user),
) -> ProxmoxImportResponse:
"""Fetch the inventory and return nodes + edges ready for canvas drop."""
@@ -108,6 +114,12 @@ async def import_proxmox(
logger.exception("Unexpected error during Proxmox import")
raise HTTPException(status_code=500, detail="Unexpected error during Proxmox import") from exc
# A canvas import is an import: the guests land in the Device Inventory too,
# and each node carries the id of the row it draws so the canvas save links
# to it instead of minting a second row for the same guest.
await _persist_pending_import(db, nodes_raw, edges_raw)
nodes_raw = await attach_device_ids(db, nodes_raw)
nodes = [ProxmoxNodeOut(**n) for n in nodes_raw]
edges = [ProxmoxEdgeOut(**e) for e in edges_raw]
return ProxmoxImportResponse(nodes=nodes, edges=edges, device_count=len(nodes))
@@ -247,10 +259,12 @@ async def _persist_pending_import(
) -> ProxmoxImportPendingResponse:
"""Upsert Proxmox nodes/edges into device_inventory + device_inventory_links.
Two-tier identity (order matters):
1. Match an existing canvas Node or pending row by **IP** (merge into a
device previously found by a scan) — never duplicate.
2. Else match by synthetic ``ieee_address`` (``pve-...``).
Two-tier identity (order matters) — see ``_find_pending``:
1. Match by the synthetic ``ieee_address`` (``pve-{host}-{vmid}``), the
only key that identifies a guest.
2. Else match an existing pending row by MAC, then IP, to merge into a
device a scan found first — never duplicate. Rows already claimed by
another Proxmox guest are excluded.
Update-in-place only. Nothing is ever deleted; hidden rows stay hidden.
"""
@@ -308,6 +322,12 @@ async def _persist_pending_import(
en.left_handles = max(en.left_handles or 0, 1)
en.right_handles = max(en.right_handles or 0, 1)
# Fold in any row this import just proved to be the same device: a guest
# matched here by its synthetic ieee can share a MAC or an IP with a row an
# earlier scan (or an older canvas node) minted before either side carried
# the address that would have matched them.
await reconcile_duplicates(db)
links_recorded = await _replace_links(db, edges_raw, cluster_pairs)
await db.commit()
@@ -322,14 +342,70 @@ async def _persist_pending_import(
async def _find_pending(
db: AsyncSession, ieee: str, ip: str | None, mac: str | None
) -> InventoryDevice | None:
filters = [InventoryDevice.ieee_address == ieee]
if ip:
filters.append(InventoryDevice.ip == ip)
if mac:
filters.append(InventoryDevice.mac == mac)
return (
await db.execute(select(InventoryDevice).where(or_(*filters)))
"""The inventory row this guest *is*, in strict precedence order.
1. The synthetic ``ieee_address`` (``pve-{host}-{vmid}``). Unique per guest
and the only key that identifies one.
2. Else the NIC MAC, else the IP — merging into a row an IP/ARP scan found
first, or into this same guest under its previous host name.
An IP is not an identity: a duplicated static lease, a re-used DHCP address,
two guests behind one NAT, or a container-bridge address several guests
report to the agent all make one address describe several machines. So the
IP fallback skips any row already claimed by a Proxmox guest — matching one
used to overwrite that guest's hostname, VMID and specs, and
non-deterministically, since one flat OR with no ordering returned whichever
row the database yielded first. Oldest row wins, so a re-import is stable.
A MAC *is* an identity, so its fallback matches a claimed row too. It has to:
the synthetic ieee embeds the host name, and a live migration or a node
rename changes it for what is the same guest (``pve-pve1-802`` →
``pve-pve2-802``). Excluding claimed rows there would file the migrated
guest as a new device and orphan its old row. See ``_adopted_ieee``.
"""
exact = (
await db.execute(
select(InventoryDevice).where(InventoryDevice.ieee_address == ieee)
)
).scalars().first()
if exact is not None:
return exact
unclaimed = or_(
InventoryDevice.ieee_address.is_(None),
~InventoryDevice.ieee_address.startswith(_PVE_IEEE_PREFIX),
)
for column, value, claimed_ok in (
(InventoryDevice.mac, mac, True),
(InventoryDevice.ip, ip, False),
):
if not value:
continue
where = [column == value] if claimed_ok else [column == value, unclaimed]
row = (
await db.execute(
select(InventoryDevice)
.where(*where)
.order_by(InventoryDevice.discovered_at, InventoryDevice.id)
)
).scalars().first()
if row is not None:
return row
return None
def _adopted_ieee(current: str | None, ieee: str) -> str:
"""The ``ieee_address`` a row should carry once this guest merges into it.
Re-point a row that already belongs to a Proxmox guest: matched by NIC MAC,
it is this same guest under a stale host name (migrated, or the node was
renamed). Links are rebuilt keyed on the ieee, so a row left on the old one
stops resolving as a host→guest endpoint. A mesh ieee is a real hardware
address and is never overwritten.
"""
if not current or current.startswith(_PVE_IEEE_PREFIX):
return ieee
return current
def _new_pending(
@@ -372,7 +448,7 @@ def _sources_after_merge(row: InventoryDevice) -> list[str]:
device (no ``pve-`` ieee) was found by a scan; keep an IP-scan source.
"""
sources = add_source(row.discovery_sources, row.discovery_source)
was_scanned = not (row.ieee_address or "").startswith("pve-")
was_scanned = not (row.ieee_address or "").startswith(_PVE_IEEE_PREFIX)
if was_scanned and row.ip and not any(s in ("arp", "mdns") for s in sources):
sources = add_source(sources, "arp")
return add_source(sources, _PROXMOX_GUEST_SOURCE)
@@ -388,7 +464,7 @@ def _refresh_pending(
) -> None:
# Compute sources before adopting the pve ieee (needs the pre-merge origin).
pending.discovery_sources = _sources_after_merge(pending)
pending.ieee_address = pending.ieee_address or ieee
pending.ieee_address = _adopted_ieee(pending.ieee_address, ieee)
pending.ip = ip or pending.ip
pending.mac = pending.mac or mac
pending.hostname = n.get("hostname") or pending.hostname
@@ -420,7 +496,7 @@ async def _ensure_inventory_row(
else:
# Compute sources before adopting the pve ieee (needs the pre-merge origin).
inv.discovery_sources = _sources_after_merge(inv)
inv.ieee_address = inv.ieee_address or ieee
inv.ieee_address = _adopted_ieee(inv.ieee_address, ieee)
inv.ip = ip or inv.ip
inv.mac = inv.mac or mac
inv.hostname = n.get("hostname") or inv.hostname
+149 -1
View File
@@ -1,3 +1,4 @@
from collections.abc import Sequence
from datetime import datetime, timezone
from typing import Any, TypeVar
@@ -9,6 +10,7 @@ from app.api.deps import get_current_user
from app.db.database import get_db
from app.db.models import CanvasState, Design, InventoryDevice, Node, Rack, RackCable, RackDevice
from app.schemas.racks import (
RACK_COLUMNS,
RackCableResponse,
RackDeviceResponse,
RackInventoryItem,
@@ -124,6 +126,92 @@ def _ip_tokens(ip: str | None) -> list[str]:
return [t.strip() for t in ip.split(",") if t.strip()] if ip else []
def _rack_model(device: InventoryDevice) -> dict[str, Any] | None:
"""The rack modelisation the inventory row owns, or None when never modelled.
`rack_faceplate_id` is the flag: a device that has never been mounted (or was
mounted before this became inventory-owned) carries NULL, and the mount's own
denormalized copy then stands unchanged.
"""
if not device.rack_faceplate_id:
return None
return {
"faceplate_id": device.rack_faceplate_id,
"u_height": device.rack_u_height,
"col_span": device.rack_col_span,
"color": device.rack_color,
"ports": [p for p in (device.rack_ports or []) if isinstance(p, dict) and p.get("id")],
}
# A mount's footprint in the rack grid: (u_start, u_height, col_start, col_span).
_Box = tuple[int, int, int, int]
def _height_fits(u_start: int, u_height: int | None, rack_u_height: int | None) -> bool:
"""Whether a mount of that height, starting there, stays under the top rail."""
if not u_height:
return False
return rack_u_height is None or u_start + u_height - 1 <= rack_u_height
def _span_fits(col_start: int, col_span: int | None) -> bool:
"""Whether a mount of that width, starting there, stays inside the grid."""
if not col_span:
return False
return col_start + col_span <= RACK_COLUMNS
def _overlaps(box: _Box, others: Sequence[_Box]) -> bool:
"""Whether a footprint (u_start, u_height, col_start, col_span) hits any other."""
u_start, u_height, col_start, col_span = box
return any(
u_start < o_u + o_uh
and o_u < u_start + u_height
and col_start < o_c + o_cs
and o_c < col_start + col_span
for o_u, o_uh, o_c, o_cs in others
)
def _with_model(
device: RackDevice,
model: dict[str, Any] | None,
rack_u_height: int | None,
neighbours: Sequence[_Box] = (),
) -> RackDeviceResponse:
"""Overlay the inventory's rack modelisation onto a mount for the response.
Geometry is global, but a rack is not: a device grown to 4U in one rack may no
longer fit where it sits in another. Height and width are therefore applied
only when they still fit at the mount's own `u_start` / `col_start` *and* the
bigger footprint lands on no neighbour — the placement rule the canvas itself
enforces, and one a plain rail check misses: a 4U model overlaid on a mount
with a device one U above it would draw two plates on the same U, and save
back in that state. The plate, its colour and its ports always are, since
none of them can overflow anything.
"""
row = RackDeviceResponse.model_validate(device)
if model is None:
return row
patch: dict[str, Any] = {
"faceplate_id": model["faceplate_id"],
"ports": model["ports"],
"color": model["color"],
}
u_height = row.u_height
if _height_fits(row.u_start, model["u_height"], rack_u_height) and not _overlaps(
(row.u_start, model["u_height"], row.col_start, row.col_span), neighbours
):
u_height = model["u_height"]
patch["u_height"] = u_height
if _span_fits(row.col_start, model["col_span"]) and not _overlaps(
(row.u_start, u_height, row.col_start, model["col_span"]), neighbours
):
patch["col_span"] = model["col_span"]
return row.model_copy(update=patch)
_Row = TypeVar("_Row", Rack, RackDevice, RackCable)
@@ -165,9 +253,42 @@ async def load_racks(
cables = (await db.execute(select(RackCable).where(RackCable.design_id == design_id))).scalars().all()
state = await db.get(CanvasState, design_id)
# The inventory row owns the front panel, so it wins over the mount's copy.
inventory_ids = {d.device_id for d in devices if d.device_id}
models: dict[str, dict[str, Any]] = {}
if inventory_ids:
rows = (
await db.execute(
select(InventoryDevice).where(InventoryDevice.id.in_(inventory_ids))
)
).scalars().all()
for row in rows:
model = _rack_model(row)
if model is not None:
models[row.id] = model
heights = {r.id: r.u_height for r in racks}
# An overlaid size must not land on the mount above. Footprints start as
# persisted and are replaced by what the overlay actually applied, so a rack
# of devices that all grew elsewhere resolves in one deterministic pass
# instead of each one being measured against stale neighbours.
boxes: dict[str, _Box] = {
d.id: (d.u_start, d.u_height, d.col_start, d.col_span) for d in devices
}
mounted: list[RackDeviceResponse] = []
for d in devices:
placed = _with_model(
d,
models.get(d.device_id or ""),
heights.get(d.rack_id),
[boxes[o.id] for o in devices if o.id != d.id and o.rack_id == d.rack_id],
)
boxes[d.id] = (placed.u_start, placed.u_height, placed.col_start, placed.col_span)
mounted.append(placed)
return RackStateResponse(
racks=[RackResponse.model_validate(r) for r in racks],
devices=[RackDeviceResponse.model_validate(d) for d in devices],
devices=mounted,
cables=[RackCableResponse.model_validate(c) for c in cables],
viewport=state.viewport if state else {"x": 0, "y": 0, "zoom": 1},
)
@@ -228,11 +349,33 @@ async def save_racks(
payload = device_data.model_dump()
payload["design_id"] = body.design_id
db_device = await _owned(db, RackDevice, device_data.id, body.design_id)
# The size this mount held before the save, to tell a real resize from an
# echo of what the load handed out. See the write-through below.
was = (db_device.u_height, db_device.col_span) if db_device else None
if db_device:
for field, value in payload.items():
setattr(db_device, field, value)
else:
db.add(RackDevice(**payload))
# Write-through: the device's front panel belongs to the inventory row,
# not to this mount, so every rack showing the same device picks the
# change up on its next load. Accessories have no row and keep theirs.
if device_data.device_id:
entry = await db.get(InventoryDevice, device_data.device_id)
if entry is not None:
entry.rack_faceplate_id = device_data.faceplate_id
entry.rack_color = device_data.color
entry.rack_ports = [p for p in device_data.ports if isinstance(p, dict)]
# Size, unlike the plate, is not written back blind. The load
# overlays it only where it still fits this rack, so a mount can
# legitimately hold less than the row does — and saving that back
# would shrink the device in every other rack. Only a size this
# save actually changes travels; an unchanged one is an echo.
if was is None or device_data.u_height != was[0]:
entry.rack_u_height = device_data.u_height
if was is None or device_data.col_span != was[1]:
entry.rack_col_span = device_data.col_span
await db.flush() # devices must exist before cables point at them
for cable_data in body.cables:
@@ -349,6 +492,11 @@ async def rack_inventory(
),
node_last_seen=node_device.last_seen if node_device else None,
racked=device.id in mounted,
rack_faceplate_id=device.rack_faceplate_id,
rack_u_height=device.rack_u_height,
rack_col_span=device.rack_col_span,
rack_color=device.rack_color,
rack_ports=device.rack_ports or [],
)
)
return RackInventoryResponse(items=items)
+221 -2
View File
@@ -28,11 +28,20 @@ from app.schemas.scan import (
InventoryDeviceUpdate,
ScanRunResponse,
)
from app.services.device_merge import merge_devices
from app.services.discovery_sources import add_source
from app.services.doc_links import unlink_documents
from app.services.inventory_sync import find_device_for, merge_properties, merge_services
from app.services.mac_utils import normalize_mac
from app.services.node_dedupe import dedupe_nodes_by_device, find_duplicate_node
from app.services.scanner import DeepScanOptions, _valid_port_range, request_cancel, run_scan
from app.services.scanner import (
DeepScanOptions,
_valid_port_range,
_valid_port_spec,
request_cancel,
run_device_scan,
run_scan,
)
from app.services.zigbee_service import (
build_zigbee_properties,
merge_zigbee_properties,
@@ -109,6 +118,13 @@ def merge_mac_property(
return out
class MergeDevicesRequest(BaseModel):
"""Which inventory rows fold into which. ``winner_id`` is the survivor."""
winner_id: str
loser_ids: list[str]
class BulkActionRequest(BaseModel):
device_ids: list[str]
# Target design for approved nodes. Falls back to the first design when
@@ -176,7 +192,39 @@ async def _background_scan(
await db.rollback()
run = await db.get(ScanRun, run_id)
if run and run.status == "running":
run.status = "failed"
# "error", the same word run_scan / run_device_scan write when
# they fail themselves — one condition, one name. Scan History
# filters and colours that one; "failed" showed up unlabelled.
run.status = "error"
await db.commit()
async def _background_device_scan(
run_id: str,
device_id: str,
deep_scan: DeepScanOptions | None = None,
full_ports: bool = True,
ports: str | None = None,
) -> None:
async with AsyncSessionLocal() as db:
try:
await run_device_scan(
device_id,
db,
run_id,
deep_scan=deep_scan or DeepScanOptions(),
full_ports=full_ports,
ports=ports,
)
except Exception:
logger.exception("Device scan run %s failed unexpectedly", run_id)
await db.rollback()
run = await db.get(ScanRun, run_id)
if run and run.status == "running":
# "error", the same word run_scan / run_device_scan write when
# they fail themselves — one condition, one name. Scan History
# filters and colours that one; "failed" showed up unlabelled.
run.status = "error"
await db.commit()
@@ -215,6 +263,78 @@ async def trigger_scan(
return run
class RescanDeviceRequest(BaseModel):
"""Per-device deep rescan. Defaults to every TCP port — that is the point.
``ports`` narrows the sweep to what the user typed in the deep-scan dialog
(``80,443``, ``1-1024``); it wins over ``full_ports``.
"""
full_ports: bool = True
ports: str | None = None
http_probe_enabled: bool | None = None
verify_tls: bool | None = None
@field_validator("ports")
@classmethod
def _check_ports(cls, v: str | None) -> str | None:
if v is None or not v.strip():
return None
spec = v.strip()
if not _valid_port_spec(spec):
raise ValueError("Invalid port range")
return spec
@router.post("/pending/{device_id}/rescan", response_model=ScanRunResponse)
async def rescan_device(
device_id: str,
background_tasks: BackgroundTasks,
payload: RescanDeviceRequest | None = None,
db: AsyncSession = Depends(get_db),
_: str = Depends(get_current_user),
) -> ScanRun:
"""Deep-rescan one known device to refresh its services (issue #350).
Recorded as a ScanRun like any other scan, so progress, stop and history
work unchanged. One run per device at a time — a second request while the
first is still scanning is a 409, not a duplicate nmap over 65535 ports.
"""
device = await db.get(InventoryDevice, device_id)
if not device:
raise HTTPException(status_code=404, detail="Device not found")
if not device.ip:
raise HTTPException(
status_code=409, detail="Device has no IP address to scan"
)
if device.status == "hidden":
raise HTTPException(status_code=409, detail="Device is hidden")
target = f"{device.ip}/32"
running = (await db.execute(
select(ScanRun).where(ScanRun.status == "running", ScanRun.kind == "device")
)).scalars().all()
if any(target in (r.ranges or []) for r in running):
raise HTTPException(
status_code=409, detail="A scan is already running for this device"
)
p = payload or RescanDeviceRequest()
deep_scan = _resolve_deep_scan(
TriggerScanRequest(
http_probe_enabled=p.http_probe_enabled, verify_tls=p.verify_tls
)
)
run = ScanRun(status="running", kind="device", ranges=[target])
db.add(run)
await db.commit()
await db.refresh(run)
background_tasks.add_task(
_background_device_scan, run.id, device_id, deep_scan, p.full_ports, p.ports
)
return run
@router.post("/{run_id}/stop", response_model=dict)
async def stop_scan(
run_id: str,
@@ -421,6 +541,10 @@ async def clear_pending(
_: str = Depends(get_current_user),
) -> dict[str, int]:
from sqlalchemy import delete as sa_delete
pending_ids = (
await db.execute(select(InventoryDevice.id).where(InventoryDevice.status == "pending"))
).scalars().all()
await unlink_documents(db, device_ids=list(pending_ids))
result = await db.execute(sa_delete(InventoryDevice).where(InventoryDevice.status == "pending"))
await db.commit()
return {"deleted": result.rowcount}
@@ -452,6 +576,7 @@ async def delete_pending(
if mounted:
raise HTTPException(status_code=409, detail="Device is mounted in a rack")
await unlink_documents(db, device_ids=[device.id])
await db.delete(device)
await db.commit()
return {"deleted": True}
@@ -463,6 +588,86 @@ async def list_hidden(db: AsyncSession = Depends(get_db), _: str = Depends(get_c
return await _with_canvas_counts(db, list(result.scalars().all()))
@router.get("/pending/{device_id}/proxmox-children", response_model=list[InventoryDeviceResponse])
async def list_proxmox_children(
device_id: str,
db: AsyncSession = Depends(get_db),
_: str = Depends(get_current_user),
) -> list[InventoryDevice]:
"""Inventory rows for the guests a Proxmox host runs.
The Proxmox import records host -> guest as a ``device_inventory_links`` row
with ``discovery_source == "proxmox"``, keyed by IEEE. This resolves those
targets back to inventory rows so the UI can offer "add the children too"
when a host is placed on a canvas. Empty for anything that is not a host.
"""
device = await db.get(InventoryDevice, device_id)
if device is None:
raise HTTPException(status_code=404, detail="Device not found")
if not device.ieee_address:
return []
ieees = list(
(
await db.execute(
select(InventoryDeviceLink.target_ieee).where(
InventoryDeviceLink.source_ieee == device.ieee_address,
InventoryDeviceLink.discovery_source == "proxmox",
)
)
)
.scalars()
.all()
)
if not ieees:
return []
result = await db.execute(
select(InventoryDevice).where(
InventoryDevice.ieee_address.in_(ieees),
InventoryDevice.status != "hidden",
)
)
return await _with_canvas_counts(db, list(result.scalars().all()))
@router.post("/pending/merge", response_model=InventoryDeviceResponse)
async def merge_inventory_devices(
payload: MergeDevicesRequest,
db: AsyncSession = Depends(get_db),
_: str = Depends(get_current_user),
) -> InventoryDevice:
"""Fold several inventory rows into one, keeping ``winner_id``.
The manual half of the duplicate repair. Identity matching only runs when a
row is created, so two rows describing one host — minted before either
carried the address that would have matched them — stay separate forever.
Where they share no address at all, no automatic rule can join them safely
(a name is not an identity), and this is the only way to say "these are the
same machine".
Non-destructive: facts are unioned onto the survivor and every canvas node,
rack mount and document is re-pointed at it before the extras go.
"""
winner = await db.get(InventoryDevice, payload.winner_id)
if winner is None:
raise HTTPException(status_code=404, detail="Device not found")
loser_ids = [i for i in dict.fromkeys(payload.loser_ids) if i != winner.id]
if not loser_ids:
raise HTTPException(status_code=400, detail="Select at least two devices to merge")
losers = (
await db.execute(select(InventoryDevice).where(InventoryDevice.id.in_(loser_ids)))
).scalars().all()
missing = set(loser_ids) - {row.id for row in losers}
if missing:
raise HTTPException(status_code=404, detail="Device not found")
await merge_devices(db, winner, list(losers))
await db.commit()
await db.refresh(winner)
return (await _with_canvas_counts(db, [winner]))[0]
@router.post("/pending/bulk-approve", response_model=dict)
async def bulk_approve_devices(
payload: BulkActionRequest,
@@ -922,6 +1127,7 @@ async def ignore_device(
device = await db.get(InventoryDevice, device_id)
if not device:
raise HTTPException(status_code=404, detail="Device not found")
await unlink_documents(db, device_ids=[device.id])
await db.delete(device)
await db.commit()
return {"ignored": True}
@@ -933,6 +1139,19 @@ async def list_runs(db: AsyncSession = Depends(get_db), _: str = Depends(get_cur
return list(result.scalars().all())
@router.get("/runs/{run_id}", response_model=ScanRunResponse)
async def get_run(
run_id: str,
db: AsyncSession = Depends(get_db),
_: str = Depends(get_current_user),
) -> ScanRun:
"""One run, for a caller waiting on a scan it started (the device rescan)."""
run = await db.get(ScanRun, run_id)
if not run:
raise HTTPException(status_code=404, detail="Scan run not found")
return run
@router.get("/config", response_model=ScanConfig)
async def get_scan_config(_: str = Depends(get_current_user)) -> ScanConfig:
return ScanConfig(
+83 -17
View File
@@ -19,6 +19,8 @@ from app.schemas.zigbee import (
ZigbeeConfig,
ZigbeeCoordinatorOut,
ZigbeeEdgeOut,
ZigbeeImportJob,
ZigbeeImportJobResult,
ZigbeeImportPendingResponse,
ZigbeeImportRequest,
ZigbeeImportResponse,
@@ -27,6 +29,8 @@ from app.schemas.zigbee import (
ZigbeeTestConnectionRequest,
ZigbeeTestConnectionResponse,
)
from app.services.import_jobs import create_job, fail_job, finish_job, get_job
from app.services.inventory_sync import attach_device_ids
from app.services.node_dedupe import dedupe_nodes_by_device
from app.services.zigbee_service import (
build_zigbee_properties,
@@ -48,18 +52,70 @@ async def _is_drawn(db: AsyncSession, device_id: str) -> bool:
router = APIRouter()
@router.post("/import", response_model=ZigbeeImportResponse)
def _import_error(exc: BaseException) -> tuple[int, str]:
"""Map a fetch_networkmap failure to the (status, detail) the API reports."""
if isinstance(exc, ImportError):
return 500, str(exc)
if isinstance(exc, ConnectionError):
return 502, str(exc)
if isinstance(exc, TimeoutError):
return 504, str(exc)
if isinstance(exc, ValueError):
return 422, str(exc)
logger.exception("Unexpected error during Zigbee import", exc_info=exc)
return 500, "Unexpected error during Zigbee import"
@router.post("/import", response_model=ZigbeeImportJob, status_code=202)
async def import_zigbee_network(
payload: ZigbeeImportRequest,
background_tasks: BackgroundTasks,
_: str = Depends(get_current_user),
) -> ZigbeeImportResponse:
"""Fetch the Zigbee2MQTT network map and return nodes + edges ready for canvas drop.
) -> ZigbeeImportJob:
"""Start a canvas import and return a job id to poll.
Connects to the specified MQTT broker, publishes a networkmap request to
``<base_topic>/bridge/request/networkmap``, and waits up to 60 s for the
response (large meshes can take 30 s+). The devices are returned as typed homelable nodes with a
coordinator → router → end-device hierarchy.
``<base_topic>/bridge/request/networkmap`` and waits up to
``ZIGBEE_NETWORKMAP_TIMEOUT`` seconds (default 300) for the response.
The fetch runs in the background and the result is collected from
``GET /zigbee/import/{job_id}``: a 200+ device mesh takes minutes to answer,
which outlives the read timeout of any reverse proxy sitting in front of the
API. Polling keeps each request short.
"""
job = create_job()
background_tasks.add_task(_background_canvas_import, job.id, payload)
return ZigbeeImportJob(job_id=job.id, status=job.status)
@router.get("/import/{job_id}", response_model=ZigbeeImportJobResult)
async def get_zigbee_import_job(
job_id: str,
_: str = Depends(get_current_user),
) -> ZigbeeImportJobResult:
"""Poll a canvas import started by ``POST /zigbee/import``.
While running, returns ``status="running"`` with no payload. On success the
nodes and edges are carried in ``result``. A failed fetch is reported as the
status code the synchronous route used to raise, so the client keeps
distinguishing a bad broker (502) from a slow mesh (504).
"""
job = get_job(job_id)
if job is None:
raise HTTPException(status_code=404, detail="Import job not found or expired")
if job.status == "error":
raise HTTPException(
status_code=job.error_status or 500,
detail=job.error or "Zigbee import failed",
)
result = None
if job.status == "done" and job.result is not None:
result = ZigbeeImportResponse(**job.result)
return ZigbeeImportJobResult(job_id=job.id, status=job.status, result=result)
async def _background_canvas_import(job_id: str, payload: ZigbeeImportRequest) -> None:
"""Fetch the network map and park the canvas-ready payload on the job."""
try:
nodes_raw, edges_raw = await fetch_networkmap(
mqtt_host=payload.mqtt_host,
@@ -70,21 +126,31 @@ async def import_zigbee_network(
tls=payload.mqtt_tls,
tls_insecure=payload.mqtt_tls_insecure,
)
except ImportError as exc:
raise HTTPException(status_code=500, detail=str(exc)) from exc
except ConnectionError as exc:
raise HTTPException(status_code=502, detail=str(exc)) from exc
except TimeoutError as exc:
raise HTTPException(status_code=504, detail=str(exc)) from exc
except ValueError as exc:
raise HTTPException(status_code=422, detail=str(exc)) from exc
except Exception as exc:
logger.exception("Unexpected error during Zigbee import")
raise HTTPException(status_code=500, detail="Unexpected error during Zigbee import") from exc
status, detail = _import_error(exc)
fail_job(job_id, detail, status)
return
# A canvas import is an import: the devices land in the Device Inventory
# too, and each node carries the id of the row it draws so the canvas save
# links to it instead of minting a second, IEEE-less row.
try:
async with AsyncSessionLocal() as db:
await _persist_pending_import(db, nodes_raw, edges_raw)
nodes_raw = await attach_device_ids(db, nodes_raw)
except Exception as exc:
status, detail = _import_error(exc)
fail_job(job_id, detail, status)
return
nodes = [ZigbeeNodeOut(**n) for n in nodes_raw]
edges = [ZigbeeEdgeOut(**e) for e in edges_raw]
return ZigbeeImportResponse(nodes=nodes, edges=edges, device_count=len(nodes))
finish_job(
job_id,
ZigbeeImportResponse(
nodes=nodes, edges=edges, device_count=len(nodes)
).model_dump(),
)
@router.post("/import-pending", response_model=ScanRunResponse)
+8
View File
@@ -27,6 +27,7 @@ from app.schemas.zwave import (
ZwaveTestConnectionRequest,
ZwaveTestConnectionResponse,
)
from app.services.inventory_sync import attach_device_ids
from app.services.node_dedupe import dedupe_nodes_by_device
from app.services.zwave_service import (
build_zwave_properties,
@@ -51,6 +52,7 @@ router = APIRouter()
@router.post("/import", response_model=ZwaveImportResponse)
async def import_zwave_network(
payload: ZwaveImportRequest,
db: AsyncSession = Depends(get_db),
_: str = Depends(get_current_user),
) -> ZwaveImportResponse:
"""Fetch the Z-Wave node list and return nodes + edges ready for canvas drop.
@@ -82,6 +84,12 @@ async def import_zwave_network(
logger.exception("Unexpected error during Z-Wave import")
raise HTTPException(status_code=500, detail="Unexpected error during Z-Wave import") from exc
# A canvas import is an import: the devices land in the Device Inventory too,
# and each node carries the id of the row it draws so the canvas save links
# to it instead of minting a second, IEEE-less row.
await _persist_pending_import(db, nodes_raw, edges_raw)
nodes_raw = await attach_device_ids(db, nodes_raw)
nodes = [ZwaveNodeOut(**n) for n in nodes_raw]
edges = [ZwaveEdgeOut(**e) for e in edges_raw]
return ZwaveImportResponse(nodes=nodes, edges=edges, device_count=len(nodes))
+81
View File
@@ -2,12 +2,61 @@ import json
import logging
from pathlib import Path
from typing import Literal
from urllib.parse import urlsplit
from pydantic import Field, model_validator
from pydantic_settings import BaseSettings, SettingsConfigDict
logger = logging.getLogger(__name__)
def origin_of(url: str) -> str:
"""`scheme://host[:port]` of an absolute URL, or "" when it has none."""
parts = urlsplit(url)
if not parts.scheme or not parts.netloc:
return ""
return f"{parts.scheme}://{parts.netloc}"
OIDC_CALLBACK_PATH = "/api/v1/auth/oidc/callback"
def app_base_path_of(redirect_uri: str) -> str:
"""
Path the SPA is served under, read back from OIDC_REDIRECT_URI.
The frontend can be mounted under a prefix (`VITE_BASE_PATH=/homelab/`), and the
backend only ever learns about it through the callback URL the operator
configured — `https://home.example/homelab/api/v1/auth/oidc/callback` means the
app lives at `/homelab/`. Always returns a path with a trailing slash; `/` when
the URI is unset, relative, or not a callback URL.
"""
path = urlsplit(redirect_uri).path
if not path.endswith(OIDC_CALLBACK_PATH):
return "/"
base = path[: -len(OIDC_CALLBACK_PATH)]
return f"{base}/" if base else "/"
def unquote_env(value: str) -> str:
"""
Drop one matched pair of surrounding quotes from an env value.
Secrets that contain `$` (bcrypt hashes, most OIDC client secrets) have to be
single-quoted in `.env`, or Docker Compose eats the `$`. Compose then strips
those quotes itself when it reads `env_file`, and so does python-dotenv on the
`env_file=` path — but `podman-compose` (1.0.6) passes them through verbatim,
so the container receives `'$2b$12$…'`, quotes included, and every bcrypt
verify fails. Unwrap here rather than asking operators to quote differently
per container runtime.
Only a *matched* pair is removed, so a genuinely malformed value still reaches
the validation below instead of being silently patched up.
"""
if len(value) >= 2 and value[0] == value[-1] and value[0] in "\"'":
return value[1:-1]
return value
def _read_version() -> str:
for candidate in [
Path(__file__).parent.parent.parent.parent / "VERSION", # repo root (dev)
@@ -52,6 +101,12 @@ class Settings(BaseSettings):
@model_validator(mode="after")
def validate_auth_settings(self) -> "Settings":
# Runtime-dependent quoting: see unquote_env. Applied to the three values
# that routinely contain a `$` and so get quoted in .env.
self.secret_key = unquote_env(self.secret_key)
self.auth_password_hash = unquote_env(self.auth_password_hash)
self.oidc_client_secret = unquote_env(self.oidc_client_secret)
h = self.auth_password_hash
if h and not h.startswith("$2"):
logger.error(
@@ -88,6 +143,16 @@ class Settings(BaseSettings):
raise ValueError(f"{name} must use HTTPS when OIDC_COOKIE_SECURE=true")
if "*" in self.cors_origins:
raise ValueError("CORS_ORIGINS cannot contain '*' when AUTH_MODE=oidc")
app_origin = origin_of(self.oidc_redirect_uri)
allowed = {value.rstrip("/") for value in self.cors_origins}
if app_origin and app_origin not in allowed:
logger.warning(
"CORS_ORIGINS does not list %s (the origin of OIDC_REDIRECT_URI). "
"The CSRF check accepts that origin anyway, but add it to CORS_ORIGINS "
"so browser requests are not rejected: CORS_ORIGINS=[\"%s\"]",
app_origin,
app_origin,
)
return self
# Scanner
@@ -98,6 +163,14 @@ class Settings(BaseSettings):
# ports survive a timeout regardless; raise this on slow/overlay networks.
scanner_version_host_timeout: int = 60
# Per-device deep rescan: total time budget in seconds, checked between port
# slices. Unprivileged nmap falls back to a connect scan (-sT), and a host
# that drops packets makes every filtered port wait out its RTT — the whole
# range against such a host runs ~20-45 min. When the budget is spent the
# run keeps what it found and says how much it did not reach. This is NOT an
# nmap --host-timeout: that one discards a host's results wholesale.
scanner_deep_host_timeout: int = 2700
# Deep scan — persisted defaults (overridable per-scan from the scan dialog).
# http_ranges: extra nmap port ranges, opt-in, no default. Probe + TLS off by default.
scanner_http_ranges: list[str] = []
@@ -152,6 +225,10 @@ class Settings(BaseSettings):
zigbee_mqtt_tls_insecure: bool = False
zigbee_sync_enabled: bool = False
zigbee_sync_interval: int = 3600 # seconds (floor 300 enforced on write)
# Seconds to wait for the Z2M bridge to answer a networkmap request. A mesh
# of 200+ devices can take several minutes to build the map, so raise this
# if imports fail with "Timed out waiting for networkmap response".
zigbee_networkmap_timeout: int = 300
# Z-Wave JS UI (zwavejs2mqtt) auto-sync import. Same secret/env rules.
zwave_mqtt_host: str = ""
@@ -165,6 +242,10 @@ class Settings(BaseSettings):
zwave_sync_enabled: bool = False
zwave_sync_interval: int = 3600 # seconds (floor 300 enforced on write)
# Seconds to wait for any MQTT gateway request/response round-trip
# (the Z-Wave node dump today). Same rationale as the Zigbee twin above.
mqtt_response_timeout: int = 300
def _override_path(self) -> Path:
return Path(self.sqlite_path).parent / "scan_config.json"
+8 -1
View File
@@ -14,7 +14,7 @@ from jwt import InvalidTokenError
from starlette.middleware.base import BaseHTTPMiddleware, RequestResponseEndpoint
from starlette.responses import JSONResponse
from app.core.config import settings
from app.core.config import origin_of, settings
ACCESS_TOKEN_USE = "access"
OIDC_SESSION_TOKEN_USE = "oidc_session"
@@ -153,6 +153,13 @@ def origin_is_allowed(origin: str | None) -> bool:
if not origin:
return False
allowed_origins = {value.rstrip("/") for value in settings.cors_origins}
# A same-origin deployment behind a reverse proxy needs no CORS, so
# CORS_ORIGINS is often left at its localhost default — yet browsers still
# send Origin on unsafe methods. The OIDC callback URL is served by this app,
# so its origin is the app's own and is always a legitimate CSRF origin.
app_origin = origin_of(settings.oidc_redirect_uri)
if app_origin:
allowed_origins.add(app_origin)
return origin.rstrip("/") in allowed_origins
+346 -3
View File
@@ -1,9 +1,10 @@
import json as _json
import logging
import shutil
import sqlite3
import uuid as _uuid_mod
from collections.abc import AsyncGenerator
from contextlib import suppress
from contextlib import closing, suppress
from pathlib import Path
from typing import Any
@@ -16,6 +17,59 @@ from app.core.config import APP_VERSION, settings
logger = logging.getLogger(__name__)
# DDL for the documents feature that `Base.metadata.create_all` does not apply:
# the column additions it never makes to an existing table, plus a virtual table
# and partial indexes it cannot express at all. Shared with the test fixtures so
# the suite runs against the same schema production boots with.
DOCUMENT_DDL: tuple[tuple[str, str], ...] = (
# `create_all` creates a missing table; it never adds a column to one that
# already exists. A database that met an earlier shape of `documents` — an
# intermediate build, a branch checked out and run before a column landed —
# keeps that shape forever, and every query naming the new column fails with
# `no such column`. So every nullable / defaulted column is (re)stated here
# as an idempotent ALTER, the way the `device_inventory` migrations are.
("documents.icon", "ALTER TABLE documents ADD COLUMN icon TEXT"),
("documents.parent_id", "ALTER TABLE documents ADD COLUMN parent_id TEXT"),
("documents.sort_order", "ALTER TABLE documents ADD COLUMN sort_order INTEGER DEFAULT 0"),
("documents.device_id", "ALTER TABLE documents ADD COLUMN device_id TEXT"),
("documents.node_id", "ALTER TABLE documents ADD COLUMN node_id TEXT"),
("documents.design_id", "ALTER TABLE documents ADD COLUMN design_id TEXT"),
("documents.frontmatter", "ALTER TABLE documents ADD COLUMN frontmatter JSON"),
("documents.tags", "ALTER TABLE documents ADD COLUMN tags JSON"),
("documents.starred", "ALTER TABLE documents ADD COLUMN starred BOOLEAN DEFAULT 0"),
("documents.template_id", "ALTER TABLE documents ADD COLUMN template_id TEXT"),
("documents.facts_snapshot", "ALTER TABLE documents ADD COLUMN facts_snapshot JSON"),
("documents.facts_synced_at", "ALTER TABLE documents ADD COLUMN facts_synced_at DATETIME"),
("documents.reviewed_at", "ALTER TABLE documents ADD COLUMN reviewed_at DATETIME"),
# Set only when a body is edited, so "never touched since it was generated"
# is distinguishable from "edited" — `created_at` and `updated_at` are two
# separate clock reads on insert and are never equal.
("documents.edited_at", "ALTER TABLE documents ADD COLUMN edited_at DATETIME"),
("document_revisions.reason", "ALTER TABLE document_revisions ADD COLUMN reason TEXT"),
(
"documents_fts.table",
"CREATE VIRTUAL TABLE IF NOT EXISTS documents_fts USING fts5("
"doc_id UNINDEXED, title, tags, body, "
"tokenize = 'unicode61 remove_diacritics 2')",
),
(
"documents.device_id.unique",
"CREATE UNIQUE INDEX IF NOT EXISTS ux_documents_device "
"ON documents(device_id) WHERE device_id IS NOT NULL",
),
(
"documents.node_id.unique",
"CREATE UNIQUE INDEX IF NOT EXISTS ux_documents_node "
"ON documents(node_id) WHERE node_id IS NOT NULL",
),
(
"documents.design_id.unique",
"CREATE UNIQUE INDEX IF NOT EXISTS ux_documents_design "
"ON documents(design_id) WHERE design_id IS NOT NULL",
),
)
async def _try_migrate(conn: AsyncConnection, sql: str, *, label: str) -> None:
"""Run an idempotent migration statement, logging any error.
@@ -361,6 +415,18 @@ async def init_db() -> None:
"UPDATE rack_cables SET properties = '[]' WHERE properties IS NULL"
)
# A mount can override when its plate draws its sockets. Existing rows
# keep the old behaviour, which is exactly what "auto" means.
await _try_migrate(
conn,
"ALTER TABLE rack_devices ADD COLUMN port_visibility TEXT NOT NULL DEFAULT 'auto'",
label="rack_devices.port_visibility",
)
with suppress(OperationalError):
await conn.exec_driver_sql(
"UPDATE rack_devices SET port_visibility = 'auto' WHERE port_visibility IS NULL"
)
with suppress(OperationalError):
await conn.exec_driver_sql("ALTER TABLE edges ADD COLUMN waypoints JSON")
with suppress(OperationalError):
@@ -468,6 +534,34 @@ async def init_db() -> None:
"nodes.device_id.index",
"CREATE INDEX IF NOT EXISTS ix_nodes_device_id ON nodes(device_id)",
),
# Which of the row's services and properties this canvas shows, and
# in what order. Seeded from the row further down, once the backfill
# has had its say — see `_seed_node_views`.
("nodes.display_view", "ALTER TABLE nodes ADD COLUMN display_view JSON"),
# A zone or a group describes no device, so it has no inventory row to
# keep its text on. Furniture keeps it here instead — see `Node.description`.
("nodes.description", "ALTER TABLE nodes ADD COLUMN description TEXT"),
):
await _try_migrate(conn, sql, label=label)
# 3.3.x — the inventory row owns the device's rack modelisation: the
# faceplate it wears, its size and its ports (labels, types, positions).
# A mount reads them at load and writes them back on save, so a device
# keeps the same front panel in every rack it is mounted in.
for label, sql in (
(
"device_inventory.rack_faceplate_id",
"ALTER TABLE device_inventory ADD COLUMN rack_faceplate_id TEXT",
),
(
"device_inventory.rack_u_height",
"ALTER TABLE device_inventory ADD COLUMN rack_u_height INTEGER",
),
(
"device_inventory.rack_col_span",
"ALTER TABLE device_inventory ADD COLUMN rack_col_span INTEGER",
),
("device_inventory.rack_color", "ALTER TABLE device_inventory ADD COLUMN rack_color TEXT"),
("device_inventory.rack_ports", "ALTER TABLE device_inventory ADD COLUMN rack_ports JSON"),
):
await _try_migrate(conn, sql, label=label)
for label, sql in (
@@ -486,8 +580,19 @@ async def init_db() -> None:
):
await _try_migrate(conn, sql, label=label)
# Documents. create_all makes the two tables on a fresh database; this
# brings an existing one up to the current shape — the columns create_all
# would never add, the FTS5 index and the partial uniques that keep one
# document per device / node / design.
for label, sql in DOCUMENT_DDL:
await _try_migrate(conn, sql, label=label)
await _backfill_node_devices()
await _drop_legacy_node_columns()
await _seed_node_views()
await _backfill_zone_size()
await _repair_self_parent_nodes()
await _reindex_documents_fts()
@@ -508,12 +613,14 @@ _NODE_COLUMNS_SQL = (
"label VARCHAR NOT NULL,"
"design_id VARCHAR REFERENCES designs(id) ON DELETE SET NULL,"
"device_id VARCHAR REFERENCES device_inventory(id) ON DELETE SET NULL,"
"display_view JSON,"
"pos_x FLOAT,"
"pos_y FLOAT,"
"parent_id VARCHAR REFERENCES nodes(id) ON DELETE CASCADE,"
"container_mode BOOLEAN,"
"custom_colors JSON,"
"custom_icon VARCHAR,"
"description TEXT,"
"show_port_numbers BOOLEAN,"
"width FLOAT,"
"height FLOAT,"
@@ -526,8 +633,8 @@ _NODE_COLUMNS_SQL = (
)
_NODE_KEPT = (
"id, type, label, design_id, device_id, pos_x, pos_y, parent_id, container_mode, "
"custom_colors, custom_icon, show_port_numbers, width, height, bottom_handles, "
"id, type, label, design_id, device_id, display_view, pos_x, pos_y, parent_id, container_mode, "
"custom_colors, custom_icon, description, show_port_numbers, width, height, bottom_handles, "
"top_handles, left_handles, right_handles, created_at, updated_at"
)
@@ -651,6 +758,242 @@ async def _backfill_node_devices() -> None:
logger.warning("Inventory backfill failed: %s", exc)
def _pre_split_backup() -> Path | None:
"""The newest backup still holding the per-node device columns, if any.
`_backup_db` copies the database *before* the migrations of each new
version, so a user who upgraded to 3.3.0 has a `homelab.db.back-3.3.0`
carrying the last state in which `nodes` still owned its own services and
properties. That copy is the only record of which canvas showed what, since
3.3.0's backfill unioned them all onto one inventory row. Newest first: it is
the state closest to the upgrade, so it is what the user last saw.
"""
db_path = Path(settings.sqlite_path)
candidates = sorted(
db_path.parent.glob(f"{db_path.name}.back-*"),
key=lambda p: p.stat().st_mtime,
reverse=True,
)
for path in candidates:
try:
with closing(sqlite3.connect(f"file:{path}?mode=ro", uri=True)) as conn:
cols = {row[1] for row in conn.execute("PRAGMA table_info(nodes)")}
except sqlite3.Error:
continue
if {"services", "properties"} <= cols:
return path
return None
def _views_from_backup() -> dict[str, dict[str, Any]]:
"""What each node drew, read out of the pre-3.3.0 backup. Empty when there is none.
Keyed by node id, which is a uuid and stable across every version. A backup
that cannot be opened, or holds nodes this database no longer has, simply
contributes nothing — the caller falls back to the inventory row.
"""
path = _pre_split_backup()
if path is None:
return {}
try:
with closing(sqlite3.connect(f"file:{path}?mode=ro", uri=True)) as conn:
rows = conn.execute("SELECT id, services, properties FROM nodes").fetchall()
except sqlite3.Error as exc:
logger.warning("Could not read the pre-3.3.0 node lists from %s: %s", path.name, exc)
return {}
out: dict[str, dict[str, Any]] = {}
for node_id, services, properties in rows:
drawn: dict[str, Any] = {}
for kind, raw in (("services", services), ("properties", properties)):
if isinstance(raw, str | bytes):
with suppress(ValueError):
decoded = _json.loads(raw)
if isinstance(decoded, list):
drawn[kind] = decoded
if drawn:
out[node_id] = drawn
if out:
logger.info("Recovering the per-canvas service/property layout from %s", path.name)
return out
async def _seed_node_views() -> None:
"""Give pre-existing nodes an explicit view of their inventory row (3.3.3).
Order and visibility for services and properties moved to the node, so one
device drawn on two canvases can be rendered two ways. A node from before
that has no view, and where it comes from decides whether the user gets
their arrangement back:
* upgrading from 3.2.0, the backfill has already seeded it from the node's
own columns — nothing here to do;
* upgrading from 3.3.0-3.3.2, those columns are gone and the row holds the
union of every canvas, so the view is recovered from the backup taken
before the 3.3.0 migration. That is the difference between a canvas coming
back as the user left it and coming back showing every other canvas'
properties;
* with no usable backup, the row itself is the seed: what shows today keeps
showing, and only what the row gains *later* is held back.
Never fatal: without a view the node simply shows the whole row, which is
the behaviour it has now.
"""
# Imported here: app.services imports app.db.models, which imports this module.
from app.services.inventory_sync import seed_node_views
try:
async with AsyncSessionLocal() as session:
seeded = await seed_node_views(session, drawn=_views_from_backup)
if seeded:
await session.commit()
logger.info("Seeded the service/property view of %d node(s)", seeded)
except Exception as exc: # pragma: no cover - defensive, boot must not die
logger.warning("Seeding the node service/property views failed: %s", exc)
async def _backfill_zone_size() -> None:
"""Move a zone's size out of the custom_colors blob into the real columns.
Every node type stored its size in `nodes.width` / `nodes.height` except
`groupRect`, which stashed it inside the `custom_colors` JSON alongside its
colours. The serializer writes the columns for zones too now, so a canvas
saved before this upgrade would come back at the default 360x240 without
this backfill.
Only fills a column that is still NULL, so it cannot overwrite a size the
user has set since, and re-running it is a no-op. Parsed in Python rather
than with `json_extract`, so it does not depend on the SQLite build being
compiled with JSON1.
Never fatal: the reader falls back to the blob, so the worst case of a
failure here is that the geometry keeps coming from where it always did.
"""
try:
async with engine.begin() as conn:
rows = (
await conn.exec_driver_sql(
"SELECT id, custom_colors FROM nodes "
"WHERE type = 'groupRect' AND custom_colors IS NOT NULL "
"AND (width IS NULL OR height IS NULL)"
)
).fetchall()
moved = 0
for node_id, blob in rows:
if isinstance(blob, str):
try:
blob = _json.loads(blob)
except ValueError:
continue
if not isinstance(blob, dict):
continue
width, height = blob.get("width"), blob.get("height")
# A bool is an int in Python; a size that is not a real number
# is left alone rather than written as garbage.
if not isinstance(width, int | float) or isinstance(width, bool):
width = None
if not isinstance(height, int | float) or isinstance(height, bool):
height = None
if width is None and height is None:
continue
await conn.exec_driver_sql(
"UPDATE nodes SET width = COALESCE(width, ?), height = COALESCE(height, ?) "
"WHERE id = ?",
(width, height, node_id),
)
moved += 1
if moved:
logger.info("Moved the size of %d zone(s) to the width/height columns", moved)
except Exception as exc: # pragma: no cover - defensive, boot must not die
logger.warning("Backfilling zone width/height failed: %s", exc)
async def _repair_self_parent_nodes() -> None:
"""Detach any node that is recorded as its own parent.
Several write paths could persist ``parent_id = id`` before the guards
added alongside this repair: a YAML import resolving a parent by a label
that mapped back to the node, the node dedupe re-pointing a canonical node
that had been nested under one of its own duplicates, and any client PATCH
or canvas save, neither of which validated it (#370).
The row is fatal on the canvas: the parent walks assume an acyclic tree, so
dragging the node overflowed the stack inside the change reducer and the
move was silently dropped — the node selected but would not move. It also
renders unparented, so it sits wherever its stored coordinates put it
rather than inside the container it appears to belong to.
Clearing the column is the only safe repair: the real parent is not
recoverable from the row, and NULL simply returns the node to the top level
where the user can re-nest it. Idempotent — a second run matches nothing.
Never fatal: a failure here leaves the row as it was, and the runtime cycle
guards keep the canvas usable either way.
"""
try:
async with engine.begin() as conn:
rows = (
await conn.exec_driver_sql(
"SELECT id, label FROM nodes WHERE parent_id IS NOT NULL AND parent_id = id"
)
).fetchall()
if not rows:
return
await conn.exec_driver_sql(
"UPDATE nodes SET parent_id = NULL WHERE parent_id IS NOT NULL AND parent_id = id"
)
for node_id, label in rows:
logger.info("Detached self-parented node %s (%s)", node_id, label)
logger.info("Repaired %d node(s) recorded as their own parent", len(rows))
except Exception as exc: # pragma: no cover - defensive, boot must not die
logger.warning("Repairing self-parented nodes failed: %s", exc)
async def _reindex_documents_fts() -> None:
"""Rebuild the document search index from the documents table.
The index is maintained on every write, so this only has to catch up the
rows a write path could not have seen: the first boot after the feature
lands, a restored backup, or a hand-edited database. Cheap enough to run on
every boot — it does nothing once the counts already match.
Never fatal: SQLite may be built without FTS5, in which case the search
endpoint falls back to LIKE and this is a no-op.
"""
try:
async with engine.begin() as conn:
indexed = (await conn.exec_driver_sql("SELECT COUNT(*) FROM documents_fts")).scalar_one()
total = (await conn.exec_driver_sql("SELECT COUNT(*) FROM documents")).scalar_one()
if indexed == total:
return
await conn.exec_driver_sql("DELETE FROM documents_fts")
rows = (
await conn.exec_driver_sql("SELECT id, title, tags, body FROM documents")
).fetchall()
for doc_id, title, tags, body in rows:
await conn.exec_driver_sql(
"INSERT INTO documents_fts (doc_id, title, tags, body) VALUES (?, ?, ?, ?)",
(doc_id, title or "", _tags_text(tags), body or ""),
)
logger.info("Reindexed %d document(s) for search", len(rows))
except Exception as exc: # pragma: no cover - defensive, boot must not die
logger.debug("Document search index not rebuilt: %s", exc)
def _tags_text(raw: Any) -> str:
"""Flatten a document's JSON tag list into a single indexable string."""
if isinstance(raw, str):
with suppress(ValueError):
raw = _json.loads(raw)
if isinstance(raw, list):
return " ".join(str(tag) for tag in raw)
return ""
async def get_db() -> AsyncGenerator[AsyncSession, None]:
async with AsyncSessionLocal() as session:
yield session
+115
View File
@@ -49,12 +49,26 @@ class Node(Base):
device_id: Mapped[str | None] = mapped_column(
String, ForeignKey("device_inventory.id", ondelete="SET NULL"), index=True, nullable=True
)
# How this canvas renders the device's list facts: which services and which
# properties it shows, and in what order. The facts themselves stay on the
# inventory row, so the same device drawn on two canvases can show two
# different subsets — a scanner-guessed service on one, none on the other.
# {"services": [{"key": "443|tcp|https", "visible": true}, …],
# "properties": [{"key": "rack", "visible": false}, …]}
# NULL only for canvas furniture and for a node with no inventory row yet;
# `inventory_sync.link_facts` fills both lists as soon as there is one.
display_view: Mapped[dict[str, Any] | None] = mapped_column(JSON, nullable=True)
pos_x: Mapped[float] = mapped_column(Float, default=0)
pos_y: Mapped[float] = mapped_column(Float, default=0)
parent_id: Mapped[str | None] = mapped_column(String, ForeignKey("nodes.id", ondelete="CASCADE"))
container_mode: Mapped[bool] = mapped_column(Boolean, default=False)
custom_colors: Mapped[dict[str, Any] | None] = mapped_column(JSON, nullable=True)
custom_icon: Mapped[str | None] = mapped_column(String, nullable=True)
# What this piece of canvas furniture is for, in the user's words. Furniture
# (group / groupRect / text) draws no device, so it has no inventory row to
# carry a `notes` field — this column is that text's only home. NULL on every
# node that does draw a device: its notes belong to the inventory row.
description: Mapped[str | None] = mapped_column(Text, nullable=True)
show_port_numbers: Mapped[bool] = mapped_column(Boolean, default=False)
width: Mapped[float | None] = mapped_column(Float, nullable=True)
height: Mapped[float | None] = mapped_column(Float, nullable=True)
@@ -163,6 +177,9 @@ class RackDevice(Base):
faceplate_id: Mapped[str] = mapped_column(String, nullable=False, default="blank-1u")
color: Mapped[str | None] = mapped_column(String, nullable=True)
status: Mapped[str] = mapped_column(String, default="unknown")
# When the plate draws its sockets: "auto" (the faceplate decides), "always"
# or "hover". A canvas display choice about the mount, not about the device.
port_visibility: Mapped[str] = mapped_column(String, default="auto")
# [{id, label, type, x, y}] — positions are unit coordinates on the plate.
ports: Mapped[list[Any]] = mapped_column(JSON, default=list)
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=_now)
@@ -258,6 +275,19 @@ class InventoryDevice(Base):
DateTime(timezone=True), default=_now, onupdate=_now
)
# --- Rack modelisation (owned here, not by the mount) ------------------
# A device wears the same front panel in every rack it is mounted in, so the
# inventory row — not `rack_devices` — owns the faceplate, its size and its
# ports. The mount keeps a denormalized copy so a rack still renders after an
# inventory purge; `/api/v1/racks` overlays these on load and `save` writes
# them back. NULL means "never modelled" — the mount's own copy then stands.
rack_faceplate_id: Mapped[str | None] = mapped_column(String, nullable=True)
rack_u_height: Mapped[int | None] = mapped_column(Integer, nullable=True)
rack_col_span: Mapped[int | None] = mapped_column(Integer, nullable=True)
rack_color: Mapped[str | None] = mapped_column(String, nullable=True)
# [{id, label, type, x, y}] — x/y are unit coordinates on the plate.
rack_ports: Mapped[list[Any] | None] = mapped_column(JSON, nullable=True)
# Transient (not persisted): populated per-request by the scan routes to report
# how many canvases this device already appears on. Not a mapped column.
canvas_count: int = 0
@@ -299,3 +329,88 @@ class ScanRun(Base):
started_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=_now)
finished_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
error: Mapped[str | None] = mapped_column(Text)
class Document(Base):
"""One markdown document.
A document either stands alone in the Library tree (`kind` page/folder,
placed by `parent_id`) or describes exactly one thing on the other side of
an optional link: a Device Inventory row (`device_id`), a piece of canvas
furniture such as a zone or a group (`node_id`), or a whole canvas
(`design_id`). Only one of the three is ever set, and each is enforced
unique by a partial index created in `database.init_db`.
Every link is `SET NULL` and `title` is denormalized, so deleting a device
or a canvas never destroys what the user wrote — the document survives as
an orphan and can be re-linked or filed away.
`body` is the whole markdown file, YAML frontmatter included, so a document
exports to disk as-is. `frontmatter` and `tags` are a parsed cache of that
block, kept only so listings can filter without reading every body.
"""
__tablename__ = "documents"
id: Mapped[str] = mapped_column(String, primary_key=True, default=_uuid)
kind: Mapped[str] = mapped_column(String, nullable=False, default="page")
title: Mapped[str] = mapped_column(String, nullable=False)
slug: Mapped[str] = mapped_column(String, nullable=False, index=True)
icon: Mapped[str | None] = mapped_column(String, nullable=True)
# Library tree. Folders are documents too, so an empty folder is possible
# and a folder can carry an index body. Only page/folder set this.
parent_id: Mapped[str | None] = mapped_column(
String, ForeignKey("documents.id", ondelete="CASCADE"), nullable=True, index=True
)
sort_order: Mapped[int] = mapped_column(Integer, default=0)
device_id: Mapped[str | None] = mapped_column(
String, ForeignKey("device_inventory.id", ondelete="SET NULL"), nullable=True, index=True
)
node_id: Mapped[str | None] = mapped_column(
String, ForeignKey("nodes.id", ondelete="SET NULL"), nullable=True, index=True
)
design_id: Mapped[str | None] = mapped_column(
String, ForeignKey("designs.id", ondelete="SET NULL"), nullable=True, index=True
)
body: Mapped[str] = mapped_column(Text, nullable=False, default="")
frontmatter: Mapped[dict[str, Any]] = mapped_column(JSON, default=dict)
tags: Mapped[list[str]] = mapped_column(JSON, default=list)
starred: Mapped[bool] = mapped_column(Boolean, default=False)
template_id: Mapped[str | None] = mapped_column(String, nullable=True)
# The device facts as they read when this document was scaffolded or last
# reconciled. The header is generated once and then owned by the user, so
# this is what tells the UI the document has drifted from the device.
facts_snapshot: Mapped[dict[str, Any] | None] = mapped_column(JSON, nullable=True)
facts_synced_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
reviewed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
# Set the first time the body is edited by hand. NULL means the document is
# still only what the template generated — which is what the coverage view
# counts as "not really documented yet". A timestamp comparison cannot say
# this: created_at and updated_at are two separate clock reads on insert.
edited_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=_now)
updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=_now, onupdate=_now)
class DocumentRevision(Base):
"""A prior version of a document's body.
Written on every explicit save that actually changed the body, and pruned
to the most recent `REVISION_LIMIT` per document in the same transaction.
"""
__tablename__ = "document_revisions"
id: Mapped[str] = mapped_column(String, primary_key=True, default=_uuid)
document_id: Mapped[str] = mapped_column(
String, ForeignKey("documents.id", ondelete="CASCADE"), nullable=False, index=True
)
title: Mapped[str] = mapped_column(String, nullable=False)
body: Mapped[str] = mapped_column(Text, nullable=False, default="")
reason: Mapped[str] = mapped_column(String, nullable=False, default="edit")
saved_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=_now)
+8 -1
View File
@@ -12,6 +12,7 @@ from app.api.routes import (
auth,
canvas,
designs,
documents,
edges,
liveview,
media,
@@ -28,7 +29,8 @@ from app.api.routes import settings as settings_routes
from app.core.config import settings
from app.core.scheduler import start_scheduler, stop_scheduler
from app.core.security import OIDCCSRFMiddleware
from app.db.database import init_db
from app.db.database import AsyncSessionLocal, init_db
from app.services.scanner import reconcile_orphan_runs
@asynccontextmanager
@@ -45,6 +47,10 @@ async def lifespan(app: FastAPI) -> AsyncGenerator[None, None]:
logging.getLogger("app.services.scanner").setLevel(logging.INFO)
await init_db()
settings.load_overrides()
# A scan runs on a background thread in this process, so anything still
# flagged "running" now was orphaned by a previous one that died mid-scan.
async with AsyncSessionLocal() as db:
await reconcile_orphan_runs(db)
start_scheduler()
yield
stop_scheduler()
@@ -88,6 +94,7 @@ app.include_router(zwave.router, prefix="/api/v1/zwave", tags=["zwave"])
app.include_router(proxmox.router, prefix="/api/v1/proxmox", tags=["proxmox"])
app.include_router(stats.router, prefix="/api/v1/stats", tags=["stats"])
app.include_router(media.router, prefix="/api/v1/media", tags=["media"])
app.include_router(documents.router, prefix="/api/v1/documents", tags=["documents"])
@app.get("/api/v1/health")
+20 -1
View File
@@ -1,6 +1,6 @@
from typing import Any
from pydantic import BaseModel, field_validator
from pydantic import BaseModel, field_validator, model_validator
from app.schemas.edges import EdgeResponse
from app.schemas.nodes import NodeResponse
@@ -31,6 +31,9 @@ class NodeSave(BaseModel):
check_target: str | None = None
services: list[Any] = []
notes: str | None = None
# Canvas furniture only: the zone or group's own description. Furniture has no
# inventory row, so this is the one place the text can live.
description: str | None = None
parent_id: str | None = None
container_mode: bool = False
custom_colors: dict[str, Any] | None = None
@@ -51,6 +54,22 @@ class NodeSave(BaseModel):
pos_x: float = 0
pos_y: float = 0
@model_validator(mode="after")
def drop_self_parent(self) -> "NodeSave":
"""A node can never be its own parent.
Every parent walk on the canvas — waypoint propagation, ordering
parents before children, collapse — assumes an acyclic tree, and a
self-parent row makes the node undraggable once it is reloaded (#370).
Dropped rather than rejected: a canvas that already carries the bad row
must still be able to save, and a 422 here would cost the user the whole
save. The startup repair clears what is already persisted.
"""
if self.parent_id is not None and self.parent_id == self.id:
self.parent_id = None
return self
class EdgeSave(BaseModel):
id: str
+177
View File
@@ -0,0 +1,177 @@
from datetime import datetime
from typing import Any
from pydantic import BaseModel, field_validator
from app.services.doc_template import TEMPLATE_IDS
from app.services.doc_tree import DOCUMENT_KINDS
# Kept in sync with DocKind in frontend/src/documentation/types.ts.
# "page" and "folder" live in the Library tree and carry a parent; "device",
# "node" and "design" each point at exactly one thing elsewhere in the app and
# are placed by that link instead, so the frontend can pivot them however the
# user asks (by zone, subnet, type…) without the server storing a tree.
DOC_KINDS = set(DOCUMENT_KINDS)
class DocumentCreate(BaseModel):
kind: str = "page"
title: str
icon: str | None = None
parent_id: str | None = None
device_id: str | None = None
node_id: str | None = None
design_id: str | None = None
# Scaffolds the body when no body is supplied. "device" is implied for a
# device document and is the only template that reads the live facts.
template_id: str | None = None
body: str | None = None
@field_validator("kind")
@classmethod
def _known_kind(cls, v: str) -> str:
if v not in DOC_KINDS:
raise ValueError(f"kind must be one of {sorted(DOC_KINDS)}")
return v
@field_validator("template_id")
@classmethod
def _known_template(cls, v: str | None) -> str | None:
if v is not None and v not in TEMPLATE_IDS:
raise ValueError(f"template_id must be one of {sorted(TEMPLATE_IDS)}")
return v
class DocumentUpdate(BaseModel):
title: str | None = None
icon: str | None = None
body: str | None = None
parent_id: str | None = None
sort_order: int | None = None
starred: bool | None = None
# Explicit: "I have re-read this and it is still true." Sent as a bare true
# rather than a timestamp so the server owns the clock.
reviewed: bool | None = None
# Accept the device's current facts as documented, clearing the drift
# banner without touching the body.
resync_facts: bool | None = None
class DocumentSummary(BaseModel):
"""Everything the tree needs. Never carries a body — listings stay small."""
id: str
kind: str
title: str
slug: str
icon: str | None = None
parent_id: str | None = None
sort_order: int = 0
device_id: str | None = None
node_id: str | None = None
design_id: str | None = None
tags: list[str] = []
# The parsed frontmatter travels with the summary so the tree can badge a
# document as due for review without fetching every body.
frontmatter: dict[str, Any] = {}
starred: bool = False
template_id: str | None = None
# Whether the device has moved on since the snapshot was taken. Computed by
# the server because only the server knows the snapshot's shape: it holds
# `label` and `type` through their fallbacks and `properties` as a flat
# map, none of which the inventory wire shape can be compared against
# field by field.
drifted: bool = False
reviewed_at: datetime | None = None
edited_at: datetime | None = None
facts_synced_at: datetime | None = None
created_at: datetime
updated_at: datetime
model_config = {"from_attributes": True}
class DocumentResponse(DocumentSummary):
body: str = ""
facts_snapshot: dict[str, Any] | None = None
class RevisionSummary(BaseModel):
id: str
document_id: str
title: str
reason: str
saved_at: datetime
# Character count, so the history list can show how much changed without
# shipping every body.
size: int = 0
model_config = {"from_attributes": True}
class RevisionResponse(RevisionSummary):
body: str = ""
class SearchHit(BaseModel):
doc_id: str
title: str
kind: str
snippet: str
device_id: str | None = None
class SearchResponse(BaseModel):
# "fts5" or "like" — the UI drops snippet highlighting on the fallback.
engine: str
hits: list[SearchHit]
class BacklinkHit(BaseModel):
"""A document that links here, and the line it does it on."""
doc_id: str
title: str
kind: str
device_id: str | None = None
# What the link was written as, so a `[[…|label]]` reads back as the author
# meant it rather than as the target's own title.
label: str
context: str
count: int = 1
class ScaffoldRequest(BaseModel):
"""Create the missing device documents.
Omitting `device_ids` means every approved device that has none, which is
the one-click migration off the old notes field.
"""
device_ids: list[str] | None = None
template_id: str = "device"
# Only devices whose notes are not empty. What the migration banner sends.
only_with_notes: bool = False
class ScaffoldResponse(BaseModel):
created: list[DocumentSummary]
skipped: int
class DriftField(BaseModel):
field: str
documented: Any = None
current: Any = None
class CoverageResponse(BaseModel):
devices: int
documented: int
# A document whose body is still only what the template generated.
header_only: int
missing: int
drifted: int
overdue: int
notes_unmigrated: int
library_pages: int
+4
View File
@@ -16,6 +16,9 @@ class NodeBase(BaseModel):
check_target: str | None = None
services: list[Any] = []
notes: str | None = None
# Canvas furniture only (group / groupRect / text): what the zone or group is
# for. A node that draws a device keeps its text on the inventory row instead.
description: str | None = None
pos_x: float = 0
pos_y: float = 0
parent_id: str | None = None
@@ -61,6 +64,7 @@ class NodeUpdate(BaseModel):
check_target: str | None = None
services: list[Any] | None = None
notes: str | None = None
description: str | None = None
pos_x: float | None = None
pos_y: float | None = None
parent_id: str | None = None
+4
View File
@@ -41,6 +41,10 @@ class ProxmoxNodeOut(BaseModel):
vendor: str | None = None
model: str | None = None
parent_ieee: str | None = None
# The Device Inventory row this node draws, stamped by the import. The
# canvas carries it back on save so the node links to that row rather
# than minting a second one for the same device.
device_id: str | None = None
class ProxmoxEdgeOut(BaseModel):
+59
View File
@@ -7,6 +7,8 @@ from pydantic import BaseModel, field_validator, model_validator
WIDTH_STANDARDS = {"19", "10"}
NUMBERINGS = {"bottom-up", "top-down"}
CABLE_TYPES = {"ethernet", "fiber"}
# When a mount draws its ports; "auto" leaves the faceplate in charge.
PORT_VISIBILITIES = {"auto", "always", "hover"}
# 12-column horizontal grid: full = 12, half = 6, third = 4, quarter = 3.
RACK_COLUMNS = 12
@@ -57,8 +59,15 @@ class RackDeviceSave(BaseModel):
faceplate_id: str = "blank-1u"
color: str | None = None
status: str = "unknown"
port_visibility: str = "auto"
ports: list[Any] = []
@field_validator("port_visibility", mode="before")
@classmethod
def _known_port_visibility(cls, v: Any) -> str:
"""Anything unknown — including the NULL of a pre-column row — is `auto`."""
return str(v) if v in PORT_VISIBILITIES else "auto"
@field_validator("u_start", "u_height")
@classmethod
def _positive(cls, v: int) -> int:
@@ -161,6 +170,33 @@ class RackSaveRequest(BaseModel):
)
return self
@model_validator(mode="after")
def _devices_do_not_overlap(self) -> "RackSaveRequest":
"""Two mounts may not claim the same square of a rack.
The canvas already refuses such a drop (`canPlace`), so this is the same
rule stated where the state is persisted rather than where it is dragged
— a stale tab, a hand-built payload or a bug upstream would otherwise
store two plates drawn on top of each other, with no way to grab the one
underneath.
"""
by_rack: dict[str, list[RackDeviceSave]] = {}
for device in self.devices:
by_rack.setdefault(device.rack_id, []).append(device)
for mounts in by_rack.values():
for i, a in enumerate(mounts):
for b in mounts[i + 1 :]:
if (
a.u_start < b.u_start + b.u_height
and b.u_start < a.u_start + a.u_height
and a.col_start < b.col_start + b.col_span
and b.col_start < a.col_start + a.col_span
):
raise ValueError(
f"Devices {a.id} and {b.id} overlap in rack {a.rack_id}"
)
return self
class RackResponse(BaseModel):
id: str
@@ -196,8 +232,15 @@ class RackDeviceResponse(BaseModel):
faceplate_id: str
color: str | None = None
status: str
port_visibility: str = "auto"
ports: list[Any] = []
@field_validator("port_visibility", mode="before")
@classmethod
def _port_visibility_or_auto(cls, v: Any) -> str:
"""Rows written before the column existed read back as NULL."""
return str(v) if v in PORT_VISIBILITIES else "auto"
@field_validator("ports", mode="before")
@classmethod
def _coerce_ports(cls, v: Any) -> list[Any]:
@@ -282,6 +325,22 @@ class RackInventoryItem(BaseModel):
node_last_seen: datetime | None = None
# True when this device is already mounted somewhere in this design.
racked: bool = False
# Rack modelisation the inventory row owns — the same front panel wherever
# the device is mounted. All None/empty for a device never modelled; the
# tray then falls back to the type-suggested faceplate.
rack_faceplate_id: str | None = None
rack_u_height: int | None = None
rack_col_span: int | None = None
rack_color: str | None = None
rack_ports: list[dict[str, Any]] = []
@field_validator("rack_ports", mode="before")
@classmethod
def _clean_ports(cls, v: Any) -> list[dict[str, Any]]:
"""NULL for a row written before the column existed; junk stays out."""
if not isinstance(v, list):
return []
return [p for p in v if isinstance(p, dict) and p.get("id")]
@field_validator("node_last_seen")
@classmethod
+49
View File
@@ -3,6 +3,11 @@ from typing import Any
from pydantic import BaseModel, field_validator
from app.schemas.racks import RACK_COLUMNS
# Tallest plate the rack canvas offers; `MAX_RACK_U` on the frontend.
MAX_PLATE_U = 48
class InventoryDeviceResponse(BaseModel):
id: str
@@ -57,6 +62,20 @@ class InventoryDeviceResponse(BaseModel):
node_last_scan: datetime | None = None
node_last_modified: datetime | None = None
node_last_seen: datetime | None = None
# Rack modelisation: the front panel this device wears in every rack it is
# mounted in. The row owns it — see `RackDevice` and `/api/v1/racks`. All
# None/empty for a device that has never been racked or modelled.
rack_faceplate_id: str | None = None
rack_u_height: int | None = None
rack_col_span: int | None = None
rack_color: str | None = None
rack_ports: list[Any] = []
@field_validator("rack_ports", mode="before")
@classmethod
def _coerce_rack_ports(cls, v: Any) -> list[Any]:
"""NULL on every row written before the column existed."""
return v if isinstance(v, list) else []
@field_validator("properties", "discovery_sources", "services", mode="before")
@classmethod
@@ -159,6 +178,36 @@ class InventoryDeviceUpdate(BaseModel):
show_hardware: bool | None = None
check_method: str | None = None
check_target: str | None = None
# Rack modelisation, edited from the Device Inventory as well as from a rack
# canvas. Sending `rack_faceplate_id` is what models a device; the rest
# describes the plate it wears.
rack_faceplate_id: str | None = None
rack_u_height: int | None = None
rack_col_span: int | None = None
rack_color: str | None = None
rack_ports: list[Any] | None = None
@field_validator("rack_u_height")
@classmethod
def _plate_height(cls, v: int | None) -> int | None:
if v is not None and not 1 <= v <= MAX_PLATE_U:
raise ValueError(f"rack_u_height must be between 1 and {MAX_PLATE_U}")
return v
@field_validator("rack_col_span")
@classmethod
def _plate_span(cls, v: int | None) -> int | None:
if v is not None and not 1 <= v <= RACK_COLUMNS:
raise ValueError(f"rack_col_span must be between 1 and {RACK_COLUMNS}")
return v
@field_validator("rack_ports")
@classmethod
def _plate_ports(cls, v: list[Any] | None) -> list[Any] | None:
"""A port with no id is not a port: no cable could ever name it."""
if v is None:
return None
return [p for p in v if isinstance(p, dict) and p.get("id")]
class ScanRunResponse(BaseModel):
+19
View File
@@ -60,6 +60,10 @@ class ZigbeeNodeOut(BaseModel):
vendor: str | None = None
lqi: int | None = None
parent_id: str | None = None
# The Device Inventory row this node draws, stamped by the import. The
# canvas carries it back on save so the node links to that row rather
# than minting a second one for the same device.
device_id: str | None = None
class ZigbeeEdgeOut(BaseModel):
@@ -73,6 +77,21 @@ class ZigbeeImportResponse(BaseModel):
device_count: int
class ZigbeeImportJob(BaseModel):
"""Handle returned when a canvas import is queued."""
job_id: str
status: str # running | done | error
class ZigbeeImportJobResult(BaseModel):
"""Poll response for a canvas import. ``result`` is set once done."""
job_id: str
status: str # running | done
result: ZigbeeImportResponse | None = None
class ZigbeeTestConnectionResponse(BaseModel):
connected: bool
message: str
+4
View File
@@ -50,6 +50,10 @@ class ZwaveNodeOut(BaseModel):
vendor: str | None = None
lqi: int | None = None
parent_id: str | None = None
# The Device Inventory row this node draws, stamped by the import. The
# canvas carries it back on save so the node links to that row rather
# than minting a second one for the same device.
device_id: str | None = None
class ZwaveEdgeOut(BaseModel):
+447
View File
@@ -0,0 +1,447 @@
"""Fold several `device_inventory` rows describing one host into a single row.
The inventory matches on identity *when a row is created* (``find_device_for``:
ieee > ip > mac) and never looks again. A row minted while its facts were still
incomplete — a canvas node saved before anyone typed its IP, a legacy node the
3.3.0 backfill could only see by label — therefore stays a second row forever,
even once the missing address arrives and both rows plainly describe the same
machine. This module is the repair for that, in two shapes:
* :func:`merge_devices` — the user picks the survivor and what folds into it.
The only way to collapse rows that share nothing an automatic rule can trust:
a name is not an identity.
* :func:`reconcile_duplicates` — run after an import, collapses only what a
shared MAC proves. Never a shared IP alone: that is not an identity.
Merging is non-destructive by construction: facts are unioned, and every
reference (`nodes`, `rack_devices`, `documents`, mesh links) is re-pointed onto
the survivor *before* the extras go. That is what separates it from the
scanner's older collapse, which deleted duplicate rows outright and left canvas
nodes naming a row that no longer existed — SQLite runs with foreign keys off
here, so the declared ``ON DELETE SET NULL`` never fires.
"""
from __future__ import annotations
import logging
from datetime import datetime
from typing import Any
from sqlalchemy import or_, select, update
from sqlalchemy.ext.asyncio import AsyncSession
from app.db.models import Document, InventoryDevice, InventoryDeviceLink, Node, RackDevice
from app.services.discovery_sources import add_source
from app.services.inventory_sync import (
VIEW_LISTS,
ip_tokens,
merge_properties,
merge_services,
view_of_device,
)
from app.services.node_dedupe import dedupe_nodes_by_device
logger = logging.getLogger(__name__)
# Scalars the survivor fills from a loser when — and only when — it has none of
# its own. `ip`, `mac`, the lists and the timestamps are merged by rule below.
_FILL_SCALARS = (
"hostname",
"os",
"suggested_type",
"discovery_source",
"friendly_name",
"device_subtype",
"model",
"vendor",
"lqi",
"label",
"type",
"notes",
"cpu_count",
"cpu_model",
"ram_gb",
"disk_gb",
"check_method",
"check_target",
"response_time_ms",
"rack_faceplate_id",
"rack_u_height",
"rack_col_span",
"rack_color",
"rack_ports",
)
def _blank(value: Any) -> bool:
return value is None or value == ""
def _newest(left: datetime | None, right: datetime | None) -> datetime | None:
if left is None or right is None:
return left or right
return max(left, right)
def _merged_ip(winner: str | None, loser: str | None) -> str | None:
"""Union of both address lists, the survivor's own addresses first.
A device legitimately holds several addresses (a management VLAN, a second
NIC), and dropping the loser's would undo the very match that identified it
— the next import would mint the duplicate again.
"""
tokens = ip_tokens(winner)
for token in ip_tokens(loser):
if token not in tokens:
tokens.append(token)
return ", ".join(tokens) or None
def _merged_status(winner: str, losers: list[InventoryDevice]) -> str:
"""The lifecycle state the survivor carries.
``approved`` wins over everything: a canvas node or a rack mount is about to
point at this row, and a row that is pending or hidden would file a drawn
device back in the discovery queue or make it vanish from the inventory.
"""
if winner == "approved" or any(loser.status == "approved" for loser in losers):
return "approved"
return winner
async def _merge_documents(db: AsyncSession, winner: InventoryDevice, loser_ids: list[str]) -> int:
"""Re-point the losers' documents, or orphan them. Returns how many orphaned.
``documents.device_id`` is unique per device (a partial index built in
``init_db``), so only one document can describe the survivor. The oldest of
the losers' is adopted when the survivor has none; the rest are unlinked
rather than deleted — what the user wrote outlives the row it described, and
an orphan keeps its denormalized title and can be re-linked from the Library.
"""
docs = (
await db.execute(
select(Document)
.where(Document.device_id.in_(loser_ids))
.order_by(Document.created_at, Document.id)
)
).scalars().all()
if not docs:
return 0
has_own = (
await db.execute(select(Document.id).where(Document.device_id == winner.id).limit(1))
).scalar_one_or_none()
orphaned = 0
for i, doc in enumerate(docs):
if i == 0 and has_own is None:
doc.device_id = winner.id
continue
doc.device_id = None
orphaned += 1
return orphaned
async def _show_merged_facts(db: AsyncSession, winner: InventoryDevice) -> int:
"""Make the survivor's facts visible on every canvas that draws it.
A node's ``display_view`` is a whitelist: ``apply_view`` appends anything the
view does not list *hidden*, which is what keeps a service a later scan found
off canvases nobody has enabled it on. After a merge that rule works against
the user — a node that came from the poorer row lists only the poorer row's
keys, so every service and property it just gained would arrive invisible,
and the canvas would look exactly as bare as before the merge.
So each node drawing the survivor has the keys missing from its view appended
as shown. Only *missing* keys: a fact this canvas listed as hidden was hidden
on purpose and stays that way. Returns the number of nodes touched.
"""
nodes = (
await db.execute(select(Node).where(Node.device_id == winner.id))
).scalars().all()
seeds = view_of_device(winner)
touched = 0
for node in nodes:
view = dict(node.display_view or {})
changed = False
for kind in VIEW_LISTS:
entries = view.get(kind)
if not isinstance(entries, list):
# No view at all for this list — `apply_view` already shows
# everything the row holds, merged facts included.
continue
listed = {str(e.get("key")) for e in entries if isinstance(e, dict)}
missing = [entry for entry in seeds[kind] if entry["key"] not in listed]
if missing:
view[kind] = [*entries, *missing]
changed = True
if changed:
# Reassign rather than mutate: a JSON column tracks identity, not
# in-place edits, so a mutated dict would never be written back.
node.display_view = view
touched += 1
return touched
async def _refresh_rack_mounts(db: AsyncSession, winner: InventoryDevice, losers: list[InventoryDevice]) -> None:
"""Point the losers' rack mounts at the survivor and refresh what they copied.
A mount denormalizes the device's ``label`` so a rack still renders after an
inventory purge. Re-pointing alone would leave it printing the name of a row
that no longer exists, so a copy that still matches the loser is refreshed to
the survivor's name — a label the user typed on the mount itself does not
match and is left alone. The faceplate, its size and its ports are overlaid
from the inventory row on load, and the merge already filled any the survivor
lacked, so the plate keeps rendering as it did.
"""
loser_ids = [row.id for row in losers]
mounts = (
await db.execute(select(RackDevice).where(RackDevice.device_id.in_(loser_ids)))
).scalars().all()
stale = {
name
for row in losers
for name in (row.label, row.friendly_name, row.hostname)
if name
}
winner_name = winner.label or winner.friendly_name or winner.hostname
for mount in mounts:
mount.device_id = winner.id
if winner_name and mount.label in stale:
mount.label = winner_name
async def _rewrite_links(db: AsyncSession, winner_ieee: str, loser_ieees: list[str]) -> None:
"""Move mesh/Proxmox links off the losers' IEEEs onto the survivor's.
``device_inventory_links`` addresses endpoints by IEEE, not by row id, so a
link left on a merged-away address stops resolving to anything. Rewriting
can produce a self-link (both endpoints were the same device) or a duplicate
pair; both are dropped.
"""
if not loser_ieees:
return
links = (
await db.execute(
select(InventoryDeviceLink).where(
or_(
InventoryDeviceLink.source_ieee.in_(loser_ieees),
InventoryDeviceLink.target_ieee.in_(loser_ieees),
)
)
)
).scalars().all()
for link in links:
if link.source_ieee in loser_ieees:
link.source_ieee = winner_ieee
if link.target_ieee in loser_ieees:
link.target_ieee = winner_ieee
await db.flush()
seen: set[tuple[str, str, str]] = set()
all_links = (
await db.execute(
select(InventoryDeviceLink)
.where(
or_(
InventoryDeviceLink.source_ieee == winner_ieee,
InventoryDeviceLink.target_ieee == winner_ieee,
)
)
.order_by(InventoryDeviceLink.discovered_at, InventoryDeviceLink.id)
)
).scalars().all()
for link in all_links:
key = (link.source_ieee, link.target_ieee, link.discovery_source)
if link.source_ieee == link.target_ieee or key in seen:
await db.delete(link)
continue
seen.add(key)
async def merge_devices(
db: AsyncSession,
winner: InventoryDevice,
losers: list[InventoryDevice],
) -> dict[str, Any]:
"""Fold ``losers`` into ``winner``. Does not commit — the caller owns that.
The survivor keeps every fact it already has; a loser only fills a gap. That
rule is the whole safety of the operation: whichever row the user (or the
reconcile pass) picked, nothing it recorded is overwritten by an older or
less-curated copy, and nothing the extras recorded is lost — lists are
unioned, addresses are unioned, sources accumulate.
Returns counts for the caller to report.
"""
losers = [row for row in losers if row.id != winner.id]
if not losers:
return {
"merged": 0,
"device_id": winner.id,
"nodes_repointed": 0,
"documents_orphaned": 0,
"views_extended": 0,
}
loser_ids = [row.id for row in losers]
# Oldest first: where two losers both fill the same gap, the older sighting
# is the one that has survived longest without being contradicted.
for loser in sorted(losers, key=lambda d: (d.discovered_at, d.id)):
for field in _FILL_SCALARS:
if _blank(getattr(winner, field, None)):
value = getattr(loser, field, None)
if not _blank(value):
setattr(winner, field, value)
winner.ip = _merged_ip(winner.ip, loser.ip)
winner.mac = winner.mac or loser.mac
winner.services = merge_services(winner.services, loser.services)
winner.properties = merge_properties(winner.properties, loser.properties)
winner.show_hardware = winner.show_hardware or loser.show_hardware
for source in add_source(loser.discovery_sources, loser.discovery_source):
winner.discovery_sources = add_source(winner.discovery_sources, source)
if winner.status_live in (None, "", "unknown"):
winner.status_live = loser.status_live
winner.last_seen = _newest(winner.last_seen, loser.last_seen)
winner.last_scan = _newest(winner.last_scan, loser.last_scan)
# The device has been known since the earliest of the rows saw it.
if loser.discovered_at and winner.discovered_at:
winner.discovered_at = min(winner.discovered_at, loser.discovered_at)
winner.status = _merged_status(winner.status, losers)
nodes = await db.execute(
update(Node).where(Node.device_id.in_(loser_ids)).values(device_id=winner.id)
)
await _refresh_rack_mounts(db, winner, losers)
orphaned = await _merge_documents(db, winner, loser_ids)
await db.flush()
shown = await _show_merged_facts(db, winner)
loser_ieees = [row.ieee_address for row in losers if row.ieee_address]
adopted: str | None = None
if _blank(winner.ieee_address) and loser_ieees:
# `ieee_address` is UNIQUE, so the address can only move once the row
# holding it is gone — assigned after the delete below.
adopted = loser_ieees[0]
for loser in losers:
await db.delete(loser)
await db.flush()
if adopted is not None:
winner.ieee_address = adopted
await db.flush()
if winner.ieee_address:
await _rewrite_links(db, winner.ieee_address, loser_ieees)
# Two nodes on one canvas may now draw the survivor — that is exactly the
# same-design duplicate the node repair collapses.
await dedupe_nodes_by_device(db)
await db.flush()
logger.info(
"Inventory merge: %d row(s) folded into %s (%s)",
len(losers), winner.id, winner.label or winner.friendly_name or winner.ip,
)
return {
"merged": len(losers),
"device_id": winner.id,
"nodes_repointed": nodes.rowcount or 0,
"documents_orphaned": orphaned,
"views_extended": shown,
}
def _conflicting(left: InventoryDevice, right: InventoryDevice) -> bool:
"""True when two rows carry addresses that say they are different machines.
A distinct non-blank IEEE on each is decisive: two Proxmox guests (the
synthetic ``pve-{host}-{vmid}``) or two radios are never one device. A
distinct MAC says the same for hardware. Either one vetoes an automatic
merge — the user can still merge them by hand.
"""
if left.ieee_address and right.ieee_address and left.ieee_address.lower() != right.ieee_address.lower():
return True
return bool(left.mac and right.mac and left.mac != right.mac)
def _pick_winner(group: list[InventoryDevice]) -> InventoryDevice:
"""The row the others fold into.
An IEEE first: mesh links and the Proxmox host→guest graph are keyed on it,
and the import matches on it before anything else, so the row holding one is
the row every other writer will come back to. Then an approved row (it is
the one already drawn), then the oldest.
"""
return sorted(
group,
key=lambda d: (
0 if d.ieee_address else 1,
0 if d.status == "approved" else 1,
d.discovered_at,
d.id,
),
)[0]
def _auto_groups(rows: list[InventoryDevice]) -> list[list[InventoryDevice]]:
"""Group rows a shared MAC proves to be one device.
The MAC and nothing else. A shared IP is not enough on its own — a re-used
DHCP lease, two guests behind one NAT, or a container bridge address several
guests report all make one address describe several machines — and the
scanner already collapses what an IP does justify, under its own guard for
rows that are drawn on a canvas. Labels and hostnames are ignored outright:
two boxes are often called the same thing, and an automatic pass is not the
place to decide they are one. Whatever is left over is what the manual merge
is for.
A group is dropped whole when any pair inside it contradicts (see
:func:`_conflicting`) rather than guessing which member does not belong.
"""
by_mac: dict[str, list[InventoryDevice]] = {}
for row in rows:
if row.mac:
by_mac.setdefault(row.mac.lower(), []).append(row)
out = []
for group in by_mac.values():
if len(group) < 2:
continue
pairs = [(a, b) for i, a in enumerate(group) for b in group[i + 1:]]
if any(_conflicting(a, b) for a, b in pairs):
logger.info(
"Inventory reconcile: leaving %d row(s) sharing a MAC alone — "
"their IEEE says they are different devices (%s)",
len(group), ", ".join(row.id for row in group),
)
continue
out.append(group)
return out
async def reconcile_duplicates(db: AsyncSession) -> int:
"""Merge inventory rows a shared MAC proves to be one device. Does not commit.
Meant to run at the end of an import, where a row that was created blind may
have just gained the address that identifies it. Returns the number of rows
merged away.
Hidden rows are excluded on both sides: the user hid one deliberately, and
folding it into a visible row would put it back in front of them, while
folding a visible row into it would make a device they are using disappear.
"""
rows = (
await db.execute(
select(InventoryDevice)
.where(InventoryDevice.status != "hidden")
.order_by(InventoryDevice.discovered_at, InventoryDevice.id)
)
).scalars().all()
merged = 0
for group in _auto_groups(list(rows)):
winner = _pick_winner(group)
result = await merge_devices(db, winner, [row for row in group if row is not winner])
merged += int(result["merged"])
return merged
+158
View File
@@ -0,0 +1,158 @@
"""Backlinks — the documents pointing at a given document.
A wiki-link is one-way in the body: `[[device:nas-01]]` says where to go, not
where it came from. This walks every body once and inverts that.
The parsing and resolution rules **mirror `frontend/src/documentation/
wikilinks.ts` exactly** — same prefixes, same fallback order, same
case-insensitivity — the way `doc_template.service_url` mirrors
`utils/serviceUrl.ts`. They are duplicated rather than shared because the
forward direction has to resolve while the user types, with only what the
browser already holds, and the reverse direction needs every body, which the
browser does not hold: the list endpoint is metadata-only on purpose.
Change one side and change the other; `test_doc_backlinks.py` pins the rules.
"""
import re
from collections.abc import Iterator
from dataclasses import dataclass
from typing import Any
_LINK = re.compile(r"\[\[([^\]]+)\]\]")
_TARGETS = {"device", "doc", "node"}
# How much of the line the link sits on is worth showing next to it.
_CONTEXT_CHARS = 160
@dataclass(frozen=True)
class WikiLink:
"""`[[device:nas-01|the NAS]]` → target=device, key=nas-01, label=the NAS."""
target: str
key: str
label: str
@dataclass(frozen=True)
class Backlink:
"""One document pointing at another, and the line it does it on."""
doc_id: str
label: str
context: str
count: int
def parse_link(inner: str) -> WikiLink | None:
"""Parse the inside of a `[[…]]`. An unknown prefix is part of the key."""
address, _, label_part = inner.partition("|")
address = address.strip()
label = label_part.strip()
if not address:
return None
target = "doc"
key = address
prefix, sep, rest = address.partition(":")
if sep and prefix.lower() in _TARGETS:
target = prefix.lower()
key = rest.strip()
if not key:
return None
return WikiLink(target=target, key=key, label=label or key)
def iter_links(body: str) -> Iterator[tuple[str, WikiLink]]:
"""Every link in a body, with the line it was written on."""
for line in (body or "").splitlines():
for match in _LINK.finditer(line):
link = parse_link(match.group(1))
if link:
yield line, link
def device_label(device: Any) -> str:
"""Mirrors `deviceLabel` in `documentation/tree.ts`."""
return (
device.label
or device.friendly_name
or device.hostname
or device.ip
or "Unnamed device"
)
def resolve(link: WikiLink, docs: list[Any], devices: list[Any]) -> str | None:
"""The document a link points at, or None when nothing matches yet."""
key = link.key.lower()
if link.target == "device":
device = next(
(d for d in devices if d.id == link.key or device_label(d).lower() == key),
None,
)
if device is None:
return None
return next((doc.id for doc in docs if doc.device_id == device.id), None)
if link.target == "node":
return next((doc.id for doc in docs if doc.node_id == link.key), None)
# A bare or `doc:` link: id, then slug, then title.
return (
next((doc.id for doc in docs if doc.id == link.key), None)
or next((doc.id for doc in docs if (doc.slug or "").lower() == key), None)
or next((doc.id for doc in docs if (doc.title or "").lower() == key), None)
)
def _context(line: str, label: str) -> str:
"""The line the link is on, trimmed to a readable window around it."""
text = " ".join(line.split())
if len(text) <= _CONTEXT_CHARS:
return text
at = text.find(label)
if at < 0:
return text[:_CONTEXT_CHARS].rstrip() + "…"
start = max(0, at - _CONTEXT_CHARS // 2)
end = min(len(text), start + _CONTEXT_CHARS)
return ("…" if start else "") + text[start:end].strip() + ("…" if end < len(text) else "")
def backlinks_for(target_id: str, docs: list[Any], devices: list[Any]) -> list[Backlink]:
"""Which of `docs` link to `target_id`, in the order the tree lists them.
One entry per source document however many times it links, because the
reader wants the documents, not the occurrences; `count` keeps the rest.
A document linking to itself is not a backlink.
"""
found: dict[str, Backlink] = {}
for doc in docs:
# Every document is a resolution target, but only one carrying `[[`
# can be a source — skipping the rest early keeps this one cheap pass.
if doc.id == target_id or "[[" not in (doc.body or ""):
continue
for line, link in iter_links(doc.body or ""):
if resolve(link, docs, devices) != target_id:
continue
existing = found.get(doc.id)
if existing is None:
found[doc.id] = Backlink(
doc_id=doc.id,
label=link.label,
context=_context(line, link.label),
count=1,
)
else:
found[doc.id] = Backlink(
doc_id=existing.doc_id,
label=existing.label,
context=existing.context,
count=existing.count + 1,
)
return list(found.values())
def has_device_link(docs: list[Any]) -> bool:
"""Whether any body carries a `[[device:…]]`, so the devices load is skippable."""
return any(
link.target == "device" for doc in docs for _, link in iter_links(doc.body or "")
)
+40
View File
@@ -0,0 +1,40 @@
"""Clear document links when the thing a document describes is deleted.
The schema declares `ON DELETE SET NULL` on `documents.device_id`, `node_id` and
`design_id`, but SQLite runs here with foreign keys off (see
`api/routes/scan.py::delete_pending`), so nothing enforces it. Without this the
column would keep naming a row that no longer exists and the document would look
linked while resolving to nothing.
Clearing rather than deleting is deliberate: the inventory outlives the canvas,
and what the user wrote outlives both. The document survives as an orphan,
keeping its denormalized `title`, and can be re-linked or filed into the Library.
"""
from collections.abc import Sequence
from sqlalchemy import update
from sqlalchemy.ext.asyncio import AsyncSession
from app.db.models import Document
async def unlink_documents(
db: AsyncSession,
*,
device_ids: Sequence[str] | None = None,
node_ids: Sequence[str] | None = None,
design_ids: Sequence[str] | None = None,
) -> None:
"""Orphan every document pointing at any of the given rows.
Caller commits — this is meant to run inside the same transaction as the
delete it accompanies.
"""
for column, ids in (
(Document.device_id, device_ids),
(Document.node_id, node_ids),
(Document.design_id, design_ids),
):
if ids:
await db.execute(update(Document).where(column.in_(list(ids))).values({column: None}))
+132
View File
@@ -0,0 +1,132 @@
"""Full-text search over documents.
SQLite's FTS5 module is not guaranteed: the LXC and Docker images ship whatever
SQLite the base distribution built, and `database.py` already avoids depending
on JSON1 for the same reason. So the index is best-effort — every write keeps it
up to date when it exists, and `search` falls back to LIKE when it does not. The
endpoint reports which engine answered so the UI can drop snippet highlighting.
The index is maintained from Python rather than by triggers: the write paths are
few, they are already inside a transaction, and a trigger would have to be kept
in sync through `_try_migrate` forever.
"""
import logging
from typing import Any
from sqlalchemy import text
from sqlalchemy.exc import OperationalError
from sqlalchemy.ext.asyncio import AsyncSession
logger = logging.getLogger(__name__)
# Column positions inside documents_fts, for snippet().
_BODY_COLUMN = 3
_available: bool | None = None
def reset_availability_cache() -> None:
"""Forget the probed FTS5 state. Tests swap databases between cases."""
global _available
_available = None
async def fts_available(db: AsyncSession) -> bool:
"""Whether this SQLite build has FTS5 and the index table exists."""
global _available
if _available is None:
try:
await db.execute(text("SELECT doc_id FROM documents_fts LIMIT 1"))
_available = True
except OperationalError:
_available = False
logger.info("FTS5 unavailable — document search falls back to LIKE")
return _available
def tags_text(tags: Any) -> str:
"""Flatten a tag list into one indexable string."""
if isinstance(tags, list):
return " ".join(str(tag) for tag in tags)
return ""
async def index_document(db: AsyncSession, doc: Any) -> None:
"""Re-index one document. Safe to call when FTS5 is missing."""
if not await fts_available(db):
return
await unindex_document(db, doc.id)
await db.execute(
text("INSERT INTO documents_fts (doc_id, title, tags, body) VALUES (:i, :t, :g, :b)"),
{"i": doc.id, "t": doc.title or "", "g": tags_text(doc.tags), "b": doc.body or ""},
)
async def unindex_document(db: AsyncSession, doc_id: str) -> None:
if not await fts_available(db):
return
await db.execute(text("DELETE FROM documents_fts WHERE doc_id = :i"), {"i": doc_id})
def build_match_query(raw: str) -> str:
"""Turn user input into an FTS5 MATCH expression.
Every term is quoted, so punctuation a user types (`192.168.1.1`, `nas-01`,
an unbalanced quote) is data rather than FTS5 syntax, and gets a prefix `*`
so typing half a word still matches.
"""
terms = [term.replace('"', '""') for term in raw.split() if term.strip()]
return " AND ".join(f'"{term}"*' for term in terms)
async def search(db: AsyncSession, query: str, limit: int = 25) -> tuple[str, list[dict[str, Any]]]:
"""Return `(engine, hits)`; engine is "fts5" or "like"."""
query = query.strip()
if not query:
return ("fts5" if await fts_available(db) else "like", [])
if await fts_available(db):
match = build_match_query(query)
if not match:
return ("fts5", [])
try:
rows = (
await db.execute(
text(
"SELECT doc_id, "
f"snippet(documents_fts, {_BODY_COLUMN}, '<<', '>>', '…', 14) AS snip, "
"rank AS score "
"FROM documents_fts WHERE documents_fts MATCH :q "
"ORDER BY rank LIMIT :n"
),
{"q": match, "n": limit},
)
).fetchall()
return ("fts5", [{"doc_id": r[0], "snippet": r[1], "score": r[2]} for r in rows])
except OperationalError as exc:
# A malformed MATCH must degrade, never 500.
logger.debug("FTS query failed, falling back to LIKE: %s", exc)
like = f"%{query.lower()}%"
rows = (
await db.execute(
text(
"SELECT id, body FROM documents "
"WHERE lower(title) LIKE :q OR lower(body) LIKE :q "
"ORDER BY updated_at DESC LIMIT :n"
),
{"q": like, "n": limit},
)
).fetchall()
return ("like", [{"doc_id": r[0], "snippet": _excerpt(r[1] or "", query), "score": None} for r in rows])
def _excerpt(body: str, query: str, width: int = 120) -> str:
"""A LIKE-mode stand-in for snippet(): the text around the first match."""
at = body.lower().find(query.lower())
if at < 0:
return body[:width].strip()
start = max(0, at - width // 2)
end = min(len(body), at + len(query) + width // 2)
return ("…" if start else "") + body[start:end].strip() + ("…" if end < len(body) else "")
+559
View File
@@ -0,0 +1,559 @@
"""Markdown scaffolding for documents.
The header of a device document is generated **once**, when the document is
created or migrated from the old `notes` field, and from then on it belongs to
the user — nothing here ever rewrites a body that already exists. What keeps the
document honest instead is `facts_snapshot`, which records the facts as they read
at generation time so the UI can tell the user the device has moved on, and
`render_block`, which lets them drop a freshly generated section back in from the
editor's `/` menu whenever they want it.
Every section is emitted even when its data is empty, with an italic prompt in
place of the value. A blank section is the document telling the user what has not
been written yet; omitting it would hide the gap.
"""
from collections.abc import Callable
from datetime import date, datetime
from typing import Any
# Sections whose data is structural rather than descriptive: they are dropped
# entirely rather than printed empty, because an unracked device has no rack
# position to leave blank.
_EMPTY = "—"
_PROMPT = "_…_"
TEMPLATE_DEVICE = "device"
TEMPLATE_BLANK = "blank"
TEMPLATE_RUNBOOK = "runbook"
TEMPLATE_SERVICE = "service"
TEMPLATE_NETWORK = "network"
TEMPLATE_INCIDENT = "incident"
TEMPLATE_PROCEDURE = "procedure"
TEMPLATE_DECISION = "decision"
TEMPLATE_ZONE = "zone"
# Frontmatter carries a version from the first release: documents round-trip to
# disk through export/import, so the on-disk shape is a compatibility surface.
FRONTMATTER_VERSION = 1
def cell(value: Any) -> str:
"""Escape a value for a markdown table cell.
Mirrors `frontend/src/utils/exportMarkdown.ts`: backslash first, then pipe,
then collapse newlines so one field cannot break the row.
"""
if value is None or value == "":
return _EMPTY
text = str(value).replace("\\", "\\\\").replace("|", "\\|")
return " ".join(text.split())
def device_name(device: Any) -> str:
for attr in ("label", "friendly_name", "hostname", "ip"):
value = getattr(device, attr, None)
if value:
return str(value)
return "device"
def device_type(device: Any) -> str | None:
return getattr(device, "type", None) or getattr(device, "suggested_type", None)
def _fmt_date(value: Any) -> str:
if isinstance(value, datetime):
return value.date().isoformat()
if isinstance(value, date):
return value.isoformat()
return _EMPTY
def _sources(device: Any) -> str:
sources = getattr(device, "discovery_sources", None) or []
if not sources:
single = getattr(device, "discovery_source", None)
sources = [single] if single else []
return ", ".join(str(s) for s in sources) if sources else _EMPTY
def service_key(service: dict[str, Any]) -> str:
"""The identity used everywhere else for a service: port|protocol|name."""
port = service.get("port")
protocol = service.get("protocol") or "tcp"
name = (service.get("service_name") or "").lower()
return f"{port if port is not None else ''}|{protocol}|{name}"
# Ports that are definitely not HTTP. Mirrors NON_HTTP_PORTS in
# `frontend/src/utils/serviceUrl.ts` — a service link printed in a document has
# to be the same link the canvas offers, or the document is quietly wrong.
_NON_HTTP_PORTS = frozenset(
{
21, 23, 25, 53, 110, 143, 389, 445, 465, 514, 587, 636, 993, 995,
1433, 3306, 5432, 5672, 6379, 9092, 11211, 27017, 27018,
}
)
def service_url(service: dict[str, Any], host: str | None) -> str | None:
"""The browsable URL for a service, or None when there is not one.
Follows `getServiceUrl`: everything is HTTP unless the port says otherwise,
a host override wins over the device's address and hides the scanned port
(it usually names a reverse proxy), and UDP and SSH have no browser URL.
"""
override = (service.get("host") or "").strip()
effective_host = override or host
if not effective_host or service.get("protocol") == "udp":
return None
scheme: str | None = None
hostname = effective_host.split(",")[0].strip()
for prefix in ("https://", "http://"):
if hostname.startswith(prefix):
scheme = prefix[:-3]
hostname = hostname[len(prefix):].rstrip("/")
break
if not hostname:
return None
port = service.get("port")
if port == 22 or (port is not None and port in _NON_HTTP_PORTS):
return None
name = (service.get("service_name") or "").lower()
if scheme is None:
secure = any(token in name for token in ("https", "ssl", "tls")) or port in (443, 8443)
scheme = "https" if secure else "http"
# A host override points at the public name; the scanned port is internal.
printed_port = "" if override else (f":{port}" if port is not None else "")
path = (service.get("path") or "").strip()
if path and not path.startswith("/"):
path = f"/{path}"
return f"{scheme}://{hostname}{printed_port}{path}"
# ── generated blocks ────────────────────────────────────────────────────────
def block_device_info(device: Any, **_: Any) -> str:
check = getattr(device, "check_method", None)
target = getattr(device, "check_target", None)
check_cell = f"`{check}` → `{target}`" if check and target else (f"`{check}`" if check else _EMPTY)
vendor = getattr(device, "vendor", None)
model = getattr(device, "model", None)
vendor_model = " / ".join(part for part in (vendor, model) if part) or _EMPTY
rows = [
("Type", device_type(device)),
("Vendor / Model", vendor_model),
("Hostname", getattr(device, "hostname", None)),
("IP", getattr(device, "ip", None)),
("MAC", getattr(device, "mac", None)),
("IEEE", getattr(device, "ieee_address", None)),
("OS", getattr(device, "os", None)),
("First discovered", f"{_fmt_date(getattr(device, 'discovered_at', None))} ({_sources(device)})"),
("Status check", check_cell),
]
lines = ["| | |", "|---|---|"]
lines += [f"| {label} | {cell(value)} |" for label, value in rows]
return "\n".join(lines)
# What the hardware table looks for in `properties`, in order of preference.
# The scanner and the Proxmox import write these keys ("CPU Cores", "RAM",
# "Disk"); a user typing their own is likely to reach for one of the synonyms.
# Matched case-insensitively, so "ram" and "RAM" are the same key.
_HARDWARE_KEYS: dict[str, tuple[str, ...]] = {
"cpu_model": ("cpu model", "cpu", "processor"),
"cpu_count": ("cpu cores", "cores", "vcpu", "vcpus", "cpu count"),
"ram": ("ram", "memory", "mem"),
"disk": ("disk", "storage", "disk size", "capacity"),
}
def property_map(device: Any) -> dict[str, str]:
"""A device's properties as a lowercased key -> value lookup.
The first occurrence of a key wins, matching how the tables read: a device
carrying two "RAM" rows is showing the first one.
"""
found: dict[str, str] = {}
for entry in getattr(device, "properties", None) or []:
if not isinstance(entry, dict):
continue
key = entry.get("key")
value = entry.get("value")
if not key or value in (None, ""):
continue
found.setdefault(str(key).strip().lower(), str(value).strip())
return found
def _hardware_property(properties: dict[str, str], field: str) -> str | None:
for candidate in _HARDWARE_KEYS[field]:
if candidate in properties:
return properties[candidate]
return None
def block_hardware(device: Any, **_: Any) -> str:
"""CPU / RAM / Disk, read from `properties` first.
The `cpu_count` / `cpu_model` / `ram_gb` / `disk_gb` columns are barely
written any more — the scanner, the Proxmox import and the inventory modal
all put hardware in `properties` instead — so a document generated from the
columns alone printed an empty table beside a properties table listing the
very same facts. Properties win; the columns are the fallback for a row that
still carries them.
A property value is taken verbatim: it is already formatted the way the user
or the importer wrote it ("15.5 GB"), and guessing a unit onto a bare number
would be inventing a fact.
"""
properties = property_map(device)
cpu_model = _hardware_property(properties, "cpu_model") or getattr(device, "cpu_model", None)
cpu_count = _hardware_property(properties, "cpu_count") or getattr(device, "cpu_count", None)
if cpu_model and cpu_count:
cpu: Any = f"{cpu_model} ×{cpu_count}"
else:
cpu = cpu_model or (f"{cpu_count} cores" if cpu_count else None)
ram_gb = getattr(device, "ram_gb", None)
ram = _hardware_property(properties, "ram") or (f"{ram_gb:g} GB" if ram_gb else None)
disk_gb = getattr(device, "disk_gb", None)
disk = _hardware_property(properties, "disk") or (f"{disk_gb:g} GB" if disk_gb else None)
return "\n".join(
[
"| CPU | RAM | Disk |",
"|---|---|---|",
f"| {cell(cpu)} | {cell(ram)} | {cell(disk)} |",
]
)
def block_services(device: Any, **_: Any) -> str:
services = getattr(device, "services", None) or []
if not services:
return "_No service has been fingerprinted on this device yet._"
host = getattr(device, "ip", None) or getattr(device, "hostname", None)
sections: list[str] = []
for service in services:
if not isinstance(service, dict):
continue
name = service.get("service_name") or "Service"
port = service.get("port")
protocol = service.get("protocol") or "tcp"
heading = f"### {name}" + (f" — `{port}/{protocol}`" if port else "")
lines = [heading, ""]
url = service_url(service, host)
if url:
lines.append(f"- **URL** — <{url}>")
if service.get("category"):
lines.append(f"- **Category** — {service['category']}")
lines += [
f"- **Purpose** — {_PROMPT}",
f"- **Credentials** — {_PROMPT} _(where they are kept — never the secret itself)_",
f"- **Depends on** — {_PROMPT}",
]
sections.append("\n".join(lines))
return "\n\n".join(sections)
def block_properties(device: Any, **_: Any) -> str:
properties = getattr(device, "properties", None) or []
rows = [p for p in properties if isinstance(p, dict) and p.get("key")]
if not rows:
return ""
lines = ["| Key | Value |", "|---|---|"]
lines += [f"| {cell(p.get('key'))} | {cell(p.get('value'))} |" for p in rows]
return "\n".join(lines)
def block_rack(device: Any, *, rack: dict[str, Any] | None = None, zone_label: str | None = None, **_: Any) -> str:
parts: list[str] = []
if rack and rack.get("name"):
u_start = rack.get("u_start")
u_height = rack.get("u_height") or 1
span = rack.get("col_span")
where = f"Rack **{rack['name']}**"
if u_start:
top = u_start + u_height - 1
where += f", U{u_start}" + (f"–U{top}" if top != u_start else "")
if span:
where += ", full width" if span >= 12 else f", {span}/12 wide"
parts.append(where + ".")
if zone_label:
parts.append(f"Zone **{zone_label}**.")
return " ".join(parts)
def block_network(device: Any, *, connections: list[str] | None = None, **_: Any) -> str:
ip = getattr(device, "ip", None)
subnet = _guess_subnet(ip)
peers = ", ".join(f"`{peer}`" for peer in connections) if connections else _PROMPT
return "\n".join(
[
f"- **Subnet / VLAN** — {subnet or _PROMPT}",
f"- **Connected to** — {peers}",
f"- **DNS names** — {_PROMPT}",
f"- **Firewall / exposure** — {_PROMPT}",
f"- **Reverse proxy** — {_PROMPT}",
]
)
def _guess_subnet(ip: str | None) -> str | None:
"""The /24 an IPv4 address sits in. Multi-IP fields keep the first token."""
if not ip:
return None
first = ip.split(",")[0].strip()
octets = first.split(".")
if len(octets) != 4 or not all(o.isdigit() for o in octets):
return None
return f"{octets[0]}.{octets[1]}.{octets[2]}.0/24"
BLOCKS: dict[str, Callable[..., str]] = {
"device-info": block_device_info,
"hardware": block_hardware,
"services": block_services,
"properties": block_properties,
"rack": block_rack,
"network": block_network,
}
def render_block(name: str, device: Any, **context: Any) -> str:
render = BLOCKS.get(name)
if render is None:
raise ValueError(f"Unknown block: {name}")
return render(device, **context)
# ── the device document ─────────────────────────────────────────────────────
def facts_snapshot(device: Any) -> dict[str, Any]:
"""The device facts a generated document depends on.
Compared against the live row to tell the user their document has drifted,
so it holds the fields the template actually prints and nothing else.
"""
services = getattr(device, "services", None) or []
return {
"label": device_name(device),
"type": device_type(device),
"hostname": getattr(device, "hostname", None),
"ip": getattr(device, "ip", None),
"mac": getattr(device, "mac", None),
"ieee_address": getattr(device, "ieee_address", None),
"os": getattr(device, "os", None),
"vendor": getattr(device, "vendor", None),
"model": getattr(device, "model", None),
"cpu_count": getattr(device, "cpu_count", None),
"cpu_model": getattr(device, "cpu_model", None),
"ram_gb": getattr(device, "ram_gb", None),
"disk_gb": getattr(device, "disk_gb", None),
# The properties table prints these, and the hardware table now reads
# CPU / RAM / Disk out of them — so an edit to one is a real change to
# what the document says. A plain dict, because the snapshot is compared
# against its own JSON round-trip and a tuple would come back a list.
"properties": {
str(entry["key"]): str(entry.get("value") or "")
for entry in reversed(getattr(device, "properties", None) or [])
if isinstance(entry, dict) and entry.get("key")
},
"check_method": getattr(device, "check_method", None),
"check_target": getattr(device, "check_target", None),
"services": sorted(service_key(s) for s in services if isinstance(s, dict)),
}
def render_frontmatter(values: dict[str, Any]) -> str:
"""A minimal YAML block.
Hand-rolled rather than `yaml.safe_dump` so key order is the order written
here — the frontmatter is read by humans at the top of every document.
An empty list still prints. `tags: []` is the only advertisement tags have:
dropped, the key never appears in any document and nobody learns it exists.
An empty string is still dropped — `owner:` teaches nothing a reader of the
template does not already see.
"""
lines = ["---"]
for key, value in values.items():
if value is None or value == "":
continue
if isinstance(value, list):
lines.append(f"{key}: [{', '.join(str(v) for v in value)}]")
else:
lines.append(f"{key}: {value}")
lines.append("---")
return "\n".join(lines)
def render_device_document(
device: Any,
*,
zone_label: str | None = None,
rack: dict[str, Any] | None = None,
connections: list[str] | None = None,
today: date | None = None,
) -> str:
"""The full skeleton for a device, old notes appended verbatim."""
today = today or date.today()
name = device_name(device)
notes = (getattr(device, "notes", None) or "").strip()
front = render_frontmatter(
{
"homelable": FRONTMATTER_VERSION,
"title": name,
"device": getattr(device, "id", None),
"type": device_type(device),
"tags": [],
"criticality": "medium",
"owner": "",
"review_every": "6m",
"created": today.isoformat(),
}
)
parts = [
front,
"",
f"# {name}",
"",
"> _One line: what this device is for. Replace this quote._",
"",
"## Device Information",
"",
block_device_info(device),
"",
"### Hardware",
"",
block_hardware(device),
"",
]
location = block_rack(device, rack=rack, zone_label=zone_label)
if location:
parts += ["### Physical location", "", location, ""]
properties = block_properties(device)
if properties:
parts += ["### Properties", "", properties, ""]
parts += [
"## Services",
"",
block_services(device),
"",
"## Network",
"",
block_network(device, connections=connections),
"",
"## Configuration",
"",
"_How it is set up, where the config lives, what is not default, and what",
"would be needed to rebuild it from scratch._",
"",
"## Operations",
"",
"### Start / stop",
"",
_PROMPT,
"",
"### Backup",
"",
_PROMPT,
"",
"### Update procedure",
"",
_PROMPT,
"",
"### Monitoring and alerts",
"",
_PROMPT,
"",
"## Troubleshooting",
"",
"| Symptom | Likely cause | Fix |",
"|---|---|---|",
f"| {_PROMPT} | {_PROMPT} | {_PROMPT} |",
"",
"## Dependencies",
"",
f"- **Depends on** — {_PROMPT}",
f"- **Used by** — {_PROMPT}",
"",
"## Changelog",
"",
"| Date | Change |",
"|---|---|",
f"| {today.isoformat()} | Document created |",
"",
"## Notes",
"",
]
if notes:
parts += [
f"<!-- Migrated verbatim from this device's Notes field on {today.isoformat()}. -->",
"",
notes,
"",
]
else:
parts += ["_Anything worth remembering about this device._", ""]
return "\n".join(parts)
# ── library templates ───────────────────────────────────────────────────────
_LIBRARY_TEMPLATES: dict[str, tuple[str, list[str]]] = {
TEMPLATE_BLANK: ("", []),
TEMPLATE_RUNBOOK: (
"runbook",
["Trigger", "Preconditions", "Steps", "Verification", "Rollback", "Escalation"],
),
TEMPLATE_SERVICE: (
"service",
["What it does", "Endpoints", "Components", "Configuration", "Backup", "Failure modes"],
),
TEMPLATE_NETWORK: (
"network",
["Topology", "Subnets and VLANs", "DHCP and DNS", "Routing", "Firewall zones", "Remote access"],
),
TEMPLATE_INCIDENT: ("incident", ["Timeline", "Impact", "Root cause", "Fix", "What to change"]),
TEMPLATE_PROCEDURE: ("procedure", ["Goal", "Prerequisites", "Steps", "Verification"]),
TEMPLATE_DECISION: ("decision", ["Status", "Context", "Decision", "Consequences"]),
TEMPLATE_ZONE: ("zone", ["Purpose", "Members", "Addressing", "Access", "Notes"]),
}
TEMPLATE_IDS = frozenset({TEMPLATE_DEVICE, *_LIBRARY_TEMPLATES})
def render_library_document(template_id: str, title: str, *, today: date | None = None) -> str:
"""The skeleton for a page that is not about one device."""
today = today or date.today()
kind, headings = _LIBRARY_TEMPLATES.get(template_id, _LIBRARY_TEMPLATES[TEMPLATE_BLANK])
front = render_frontmatter(
{
"homelable": FRONTMATTER_VERSION,
"title": title,
"template": kind or None,
"tags": [],
"created": today.isoformat(),
}
)
parts = [front, "", f"# {title}", ""]
for heading in headings:
parts += [f"## {heading}", "", _PROMPT, ""]
return "\n".join(parts)
+137
View File
@@ -0,0 +1,137 @@
"""Placement and parsing helpers for documents.
The Library is a real tree — a folder is a document with `kind='folder'`, so an
empty folder exists and a folder can carry an index body. That makes three
things this module owns: slugs that stay unique among siblings, a re-parent
guard so the tree cannot be knotted into a cycle, and the subtree walk that
`DELETE` needs because SQLite does not always enforce `ON DELETE CASCADE`
(`api/routes/designs.py` unwinds its own deletes for the same reason).
It also parses the YAML frontmatter out of a body. The body is the source of
truth — it is what exports to disk — and `documents.frontmatter` / `documents.tags`
are only a cache so a listing can filter without reading every body.
"""
import re
from typing import Any
import yaml
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.db.models import Document
# A document is addressed by id; the slug is for readable URLs and export
# filenames, so it only has to be filesystem- and URL-safe.
_SLUG_STRIP = re.compile(r"[^a-z0-9]+")
# `[ \t]*`, not `\s*`: `\s` matches the newline the fence line ends with, and an
# alternative that can be reached two ways makes the match quadratic on a body
# that opens with `---` and never closes it — the shape a half-typed document
# has for as long as the user is typing it.
_FRONTMATTER = re.compile(r"\A---[ \t]*\n(.*?)\n---[ \t]*(?:\n|\Z)", re.DOTALL)
DOCUMENT_KINDS = frozenset({"device", "node", "design", "page", "folder"})
# Kinds that live in the Library tree and may therefore carry a parent.
TREE_KINDS = frozenset({"page", "folder"})
REVISION_LIMIT = 50
def slugify(title: str) -> str:
slug = _SLUG_STRIP.sub("-", (title or "").strip().lower()).strip("-")
return slug[:80] or "untitled"
async def unique_slug(db: AsyncSession, title: str, *, parent_id: str | None, exclude_id: str | None = None) -> str:
"""A slug no sibling is already using."""
base = slugify(title)
sibling = Document.parent_id.is_(None) if parent_id is None else Document.parent_id == parent_id
query = select(Document.slug).where(sibling)
if exclude_id is not None:
query = query.where(Document.id != exclude_id)
taken = {row for row in (await db.execute(query)).scalars().all()}
if base not in taken:
return base
suffix = 2
while f"{base}-{suffix}" in taken:
suffix += 1
return f"{base}-{suffix}"
def parse_frontmatter(body: str) -> dict[str, Any]:
"""The YAML block at the top of a body, or `{}`.
Malformed YAML is not an error the user should be blocked by — they are
mid-edit. The cache simply stays empty until the block parses again.
"""
match = _FRONTMATTER.match(body or "")
if not match:
return {}
try:
parsed = yaml.safe_load(match.group(1))
except yaml.YAMLError:
return {}
if not isinstance(parsed, dict):
return {}
return {str(key): _jsonable(value) for key, value in parsed.items()}
def _jsonable(value: Any) -> Any:
"""Coerce a parsed YAML value into something the JSON column can hold.
YAML resolves `created: 2026-09-05` to a `date` and `at: 10:30` to an int,
neither of which survives `json.dumps`. The cache is only ever read back for
filtering and display, so a string is the right shape — the body keeps the
literal the user typed either way.
"""
if isinstance(value, str | int | float | bool) or value is None:
return value
if isinstance(value, list):
return [_jsonable(item) for item in value]
if isinstance(value, dict):
return {str(key): _jsonable(item) for key, item in value.items()}
return str(value)
def tags_from(frontmatter: dict[str, Any]) -> list[str]:
raw = frontmatter.get("tags")
if isinstance(raw, str):
raw = [part.strip() for part in raw.split(",")]
if not isinstance(raw, list):
return []
return [str(tag).strip() for tag in raw if str(tag).strip()]
async def is_ancestor(db: AsyncSession, candidate_id: str, of_id: str) -> bool:
"""Whether `candidate_id` is `of_id` or sits above it.
Guards re-parenting: moving a folder inside its own subtree would orphan
everything under it. Mirrors the `_is_ancestor` guard the MCP zone tools use.
"""
if candidate_id == of_id:
return True
seen: set[str] = set()
current: str | None = of_id
while current and current not in seen:
seen.add(current)
current = (await db.execute(select(Document.parent_id).where(Document.id == current))).scalar_one_or_none()
if current == candidate_id:
return True
return False
async def subtree_ids(db: AsyncSession, root_id: str) -> list[str]:
"""`root_id` plus every document beneath it, deepest last."""
collected = [root_id]
frontier = [root_id]
while frontier:
children = (
await db.execute(select(Document.id).where(Document.parent_id.in_(frontier)))
).scalars().all()
children = [child for child in children if child not in collected]
if not children:
break
collected.extend(children)
frontier = children
return collected
+35 -5
View File
@@ -16,12 +16,42 @@ logger = logging.getLogger(__name__)
# Headers that commonly carry the application name.
_SIGNAL_HEADERS = ("Server", "X-Powered-By")
# Cap how much body we read when hunting for <title> — avoids large downloads.
# Cap how much body we read when hunting for <title>. This is a cap on the
# download itself, not just on what we scan: the body is streamed and the
# connection dropped once we hold this much. Some endpoints stream without end
# and send no Content-Length (bandwidth-test endpoints, MJPEG cameras), so
# buffering a full response would OOM the backend.
_MAX_BODY_BYTES = 64 * 1024
_TITLE_RE = re.compile(r"<title[^>]*>(.*?)</title>", re.IGNORECASE | re.DOTALL)
_PROBE_TIMEOUT = 3.0
# Ports we never bother probing over HTTP (not web services).
_NON_HTTP_PORTS = frozenset({22, 21, 23, 25, 53, 110, 143, 161, 162, 179, 445, 3306, 5432, 6379})
#
# 515 and 9100-9107 are raw printing (LPD / JetDirect). Those are not merely
# uninteresting: a printer takes whatever bytes land on 9100 as a print job, so
# a GET line makes it spit out a page of plaintext. Never send them anything.
# nmap already ships `Exclude T:9100-9107` for the same reason, which is why
# the -sV pass never triggered this — only our own probe did.
_PRINTER_PORTS = frozenset({515, *range(9100, 9108)})
_NON_HTTP_PORTS = frozenset({
22, 21, 23, 25, 53, 110, 143, 161, 162, 179, 445, 3306, 5432, 6379,
}) | _PRINTER_PORTS
async def _read_capped(resp: httpx.Response) -> str:
"""Read at most _MAX_BODY_BYTES of a streaming response, then stop."""
chunks: list[bytes] = []
size = 0
async for chunk in resp.aiter_bytes():
chunks.append(chunk)
size += len(chunk)
if size >= _MAX_BODY_BYTES:
break
raw = b"".join(chunks)[:_MAX_BODY_BYTES]
encoding = resp.charset_encoding or "utf-8"
try:
return raw.decode(encoding, errors="replace")
except LookupError:
return raw.decode("utf-8", errors="replace")
def _extract_title(body: str) -> str | None:
@@ -34,11 +64,11 @@ def _extract_title(body: str) -> str | None:
async def _probe_scheme(client: httpx.AsyncClient, url: str) -> dict[str, Any] | None:
try:
resp = await client.get(url, follow_redirects=True)
async with client.stream("GET", url, follow_redirects=True) as resp:
headers = {h: resp.headers[h] for h in _SIGNAL_HEADERS if h in resp.headers}
body = await _read_capped(resp)
except (httpx.HTTPError, OSError):
return None
headers = {h: resp.headers[h] for h in _SIGNAL_HEADERS if h in resp.headers}
body = resp.text[:_MAX_BODY_BYTES] if resp.text else ""
title = _extract_title(body)
if not title and not headers:
return None
+92
View File
@@ -0,0 +1,92 @@
"""In-process registry for long-running import jobs that return a payload.
Pending imports already run in the background and report through ``scan_runs``.
Canvas imports are different: the caller wants the fetched map *back* so it can
drop it on the canvas, which kept the request open for the whole MQTT
round-trip. On a large mesh that outlives any reverse proxy's read timeout
(Cloudflare cuts at 100 s and returns a 524), so the browser never sees the
result even though the fetch succeeded server-side.
The fix is to hand the client a job id immediately and let it poll. Results are
transient — a canvas import is meaningless once the user has closed the modal —
so they live in memory rather than the DB, and expire on a TTL. The backend runs
a single uvicorn worker, the same assumption the scheduler and ``BackgroundTasks``
already make.
"""
from __future__ import annotations
import time
import uuid
from dataclasses import dataclass, field
from typing import Any, Literal
JobStatus = Literal["running", "done", "error"]
# How long a finished job stays readable. Long enough for a client that polls
# slowly or reloads mid-import, short enough that a forgotten map is not held
# for the life of the process.
JOB_TTL_SECONDS = 900.0
@dataclass
class ImportJob:
id: str
status: JobStatus = "running"
result: dict[str, Any] | None = None
error: str | None = None
# HTTP status the synchronous route would have raised, so the client can
# keep reacting to failures the way it always has.
error_status: int | None = None
created_at: float = field(default_factory=time.monotonic)
finished_at: float | None = None
_jobs: dict[str, ImportJob] = {}
def _purge_expired(now: float) -> None:
for job_id, job in list(_jobs.items()):
if job.finished_at is not None and now - job.finished_at > JOB_TTL_SECONDS:
del _jobs[job_id]
def create_job() -> ImportJob:
"""Register a new running job and return it."""
now = time.monotonic()
_purge_expired(now)
job = ImportJob(id=str(uuid.uuid4()))
_jobs[job.id] = job
return job
def get_job(job_id: str) -> ImportJob | None:
"""Return a job, or None if it never existed or has expired."""
_purge_expired(time.monotonic())
return _jobs.get(job_id)
def finish_job(job_id: str, result: dict[str, Any]) -> None:
"""Mark a job done and attach its payload. No-op if it expired."""
job = _jobs.get(job_id)
if job is None:
return
job.status = "done"
job.result = result
job.finished_at = time.monotonic()
def fail_job(job_id: str, error: str, status: int) -> None:
"""Mark a job failed with a client-safe message. No-op if it expired."""
job = _jobs.get(job_id)
if job is None:
return
job.status = "error"
job.error = error
job.error_status = status
job.finished_at = time.monotonic()
def reset_jobs() -> None:
"""Drop every job. For tests."""
_jobs.clear()
+344 -13
View File
@@ -15,7 +15,7 @@ from __future__ import annotations
import json
import logging
from collections.abc import Mapping
from collections.abc import Callable, Mapping
from datetime import datetime
from typing import Any
@@ -54,11 +54,13 @@ def is_furniture(node_type: str | None) -> bool:
return (node_type or "") in FURNITURE_TYPES
def _ip_tokens(ip: str | None) -> list[str]:
def ip_tokens(ip: str | None) -> list[str]:
"""Split an ``ip`` field into individual addresses.
A node or device may carry several comma-separated addresses, so identity
matching compares per token — same rule as ``node_dedupe._ip_tokens``.
A node or device may carry several comma-separated addresses — a router with
a management VLAN, a host behind a VIP — so identity matching compares per
token rather than on the whole string. Every matcher that resolves a device
from addresses shares this rule, the scanner included.
"""
return [t.strip() for t in ip.split(",") if t.strip()] if ip else []
@@ -125,7 +127,7 @@ async def find_device_for(
Hidden rows are eligible: a hidden device is still that device, and silently
minting a second row for it would resurrect the duplicate the user hid.
"""
ip_toks = _ip_tokens(ip)
ip_toks = ip_tokens(ip)
conds = []
if ieee:
# Case-insensitive: `0x00124B00…` and `0x00124b00…` are the same radio,
@@ -151,7 +153,7 @@ async def find_device_for(
for device in candidates:
# "1.2.3.4" must not match "1.2.3.40" — confirm the token, don't trust
# the SQL `contains`.
if ip_toks and set(_ip_tokens(device.ip)) & set(ip_toks):
if ip_toks and set(ip_tokens(device.ip)) & set(ip_toks):
return device
for device in candidates:
if mac and device.mac == mac:
@@ -203,8 +205,181 @@ def _service_key(svc: Any) -> Any:
return (svc.get("port"), svc.get("protocol"), (svc.get("service_name") or "").lower())
def merge_services(base: list[Any] | None, incoming: list[Any] | None) -> list[Any]:
"""Union two service lists on (port, protocol, name); incoming wins."""
# --- Per-node view of the device's list facts -----------------------------
#
# The row owns the services and the properties; a node owns which of them it
# shows and in what order. Both are keyed by a stable string so the view
# survives an edit to a service's path or a property's value.
VIEW_LISTS = ("services", "properties")
def _service_view_key(svc: Any) -> str:
port, protocol, name = _service_key(svc) if isinstance(svc, dict) else (None, None, repr(svc))
return f"{port}|{protocol}|{name}"
def _property_view_key(prop: Any) -> str:
if not isinstance(prop, dict):
return repr(prop)
return str(prop.get("key") or "").lower()
_VIEW_KEY = {"services": _service_view_key, "properties": _property_view_key}
def view_entries(items: list[Any] | None, kind: str) -> list[dict[str, Any]]:
"""One list of device facts as a node's view of it: order plus visibility.
A service carries no ``visible`` of its own — one that reached a canvas was
always drawn — so it defaults to shown. A property carries an explicit flag
and keeps it. Duplicate keys collapse: the view addresses the row, and the
row holds one entry per key.
"""
key_of = _VIEW_KEY[kind]
out: list[dict[str, Any]] = []
seen: set[str] = set()
for item in items or []:
key = key_of(item)
if key in seen:
continue
seen.add(key)
visible = bool(item.get("visible", True)) if isinstance(item, dict) else True
out.append({"key": key, "visible": visible})
return out
def view_from_facts(facts: Mapping[str, Any]) -> dict[str, list[dict[str, Any]]]:
"""The view a node payload implies — only for the lists it actually sent.
The wire shape has no separate view: a client sends its services and its
properties in display order, each with its ``visible`` flag, exactly as it
draws them. That *is* the view, so it is read back out here rather than
asking clients for a second field.
"""
return {kind: view_entries(facts[kind], kind) for kind in VIEW_LISTS if kind in facts}
def view_of_device(device: InventoryDevice) -> dict[str, list[dict[str, Any]]]:
"""A view showing everything the row currently holds — the seed for a new node."""
return {
"services": view_entries(device.services, "services"),
"properties": view_entries(device.properties, "properties"),
}
def next_view(
current: Mapping[str, Any] | None,
incoming: Mapping[str, Any],
device: InventoryDevice | None,
*,
strict: bool = False,
) -> dict[str, Any] | None:
"""This node's view after a write: what it sent, then the row for the rest.
A list the write did not carry keeps the view it had. A node linked to a row
always ends up with both lists, so "not in the view" can mean one thing
only: this canvas does not show it. That is what keeps a service a later scan
discovers off every canvas until someone turns it on.
A node getting its *first* view is the exception: an empty list there means
the writer had nothing to say about it, not that the user hid everything —
creating a node for an already-scanned device sends no services and must
still draw the ones the row holds. ``strict`` turns that off for the one
caller whose empty list is a real answer: the legacy backfill, where the
node's own columns are the whole of what that canvas used to show.
"""
if device is None:
return dict(current) if current else None
out: dict[str, Any] = dict(current or {})
first_view = current is None
seed = view_of_device(device)
for kind in VIEW_LISTS:
entries = incoming.get(kind)
if entries or (entries is not None and (strict or not first_view)):
out[kind] = entries
elif kind not in out:
out[kind] = seed[kind]
return out
def apply_view(items: list[Any] | None, entries: Any, kind: str) -> list[Any]:
"""The row's facts as one node draws them: its order, its visibility.
Without a view — furniture, or a node whose row was linked by an older
version — everything shows, in the row's own order. With one, an item the
view does not list is appended hidden rather than dropped, so a service a
scan added is one toggle away instead of invisible.
A *non-empty* view that matches nothing the row still holds is treated as
having no view at all. It means the row was replaced wholesale under the
node — every key gone, every key new — and hiding the lot would leave a node
drawing nothing while its view claims otherwise. An empty view is different:
it is a real answer ("this canvas draws none of them") and keeps hiding
everything.
"""
key_of = _VIEW_KEY[kind]
facts = list(items or [])
if not isinstance(entries, list):
return facts
by_key: dict[str, Any] = {}
for item in facts:
by_key.setdefault(key_of(item), item)
out: list[Any] = []
taken: set[str] = set()
for entry in entries:
if not isinstance(entry, dict):
continue
key = str(entry.get("key"))
item = by_key.get(key)
if item is None or key in taken:
continue # Deleted from the row since — the view catches up on write.
taken.add(key)
out.append(_stamped(item, bool(entry.get("visible", True))))
if entries and not taken:
return facts
for item in facts:
if key_of(item) not in taken:
out.append(_stamped(item, False))
return out
def _stamped(item: Any, visible: bool) -> Any:
"""``item`` carrying this node's verdict on whether it is drawn.
A shown item that never had a ``visible`` key does not gain one: services
have always travelled without it, and readers treat its absence as shown.
Only hiding is news, and properties keep the explicit flag they arrived with.
"""
if not isinstance(item, dict):
return item
if visible and "visible" not in item:
return dict(item)
return {**item, "visible": visible}
# How a service looks is the user's call. A fingerprint only ever guesses it
# from a port number and a banner, so a scan that re-finds a known service must
# not repaint the icon someone picked by hand — that happened on every "Scan
# network", across every canvas drawing the device, at once.
_CURATED_SERVICE_FIELDS = ("icon", "category")
def merge_services(
base: list[Any] | None,
incoming: list[Any] | None,
*,
discovered: bool = False,
) -> list[Any]:
"""Union two service lists on (port, protocol, name); incoming wins.
``discovered`` marks ``incoming`` as scanner output rather than a user edit:
it still adds services and refreshes facts, but leaves an established icon
and category alone. Either way a blank incoming value never clears one that
is already set — an absent field is silence, not a reset.
"""
out: list[Any] = [dict(s) if isinstance(s, dict) else s for s in (base or [])]
index = {_service_key(s): i for i, s in enumerate(out)}
for svc in incoming or []:
@@ -214,7 +389,12 @@ def merge_services(base: list[Any] | None, incoming: list[Any] | None) -> list[A
out.append(dict(svc) if isinstance(svc, dict) else svc)
index[key] = len(out) - 1
elif isinstance(svc, dict) and isinstance(out[pos], dict):
out[pos] = {**out[pos], **svc}
merged = {**out[pos], **svc}
for field_name in _CURATED_SERVICE_FIELDS:
established = out[pos].get(field_name)
if established and (discovered or not svc.get(field_name)):
merged[field_name] = established
out[pos] = merged
else:
out[pos] = svc
return out
@@ -378,6 +558,7 @@ async def link_facts(
replace_lists: bool = False,
only_changed: bool = False,
changed_fields: list[str] | None = None,
strict_view: bool = False,
) -> InventoryDevice | None:
"""Point one node at its inventory row, creating or merging as needed.
@@ -400,6 +581,11 @@ async def link_facts(
node.device_id = None
return None
# A node that draws a device has no description of its own — what it is for
# is written on the inventory row, as `notes`. Clearing here keeps the two
# from becoming rival fields for the same text.
node.description = None
device = None
if node.device_id:
device = await db.get(InventoryDevice, node.device_id)
@@ -433,7 +619,20 @@ async def link_facts(
)
device.discovery_sources = add_source(device.discovery_sources, CANVAS_SOURCE)
# A row a node draws is past the pending queue — the mirror of the
# "approved but no longer drawn -> pending" revival the imports do. A hidden
# row stays hidden: the user hid it on purpose.
if device.status == "pending":
device.status = "approved"
node.device_id = device.id
# Order and visibility are this node's, not the device's, so they are taken
# from the full payload — never from the `changed_fields`/`only_changed`
# narrowing above, which exists to protect the *shared* row from a stale
# snapshot. A node's own view has no other writer to collide with.
node.display_view = next_view(
node.display_view, view_from_facts(facts), device, strict=strict_view
)
return device
@@ -444,10 +643,16 @@ def node_columns(payload: Mapping[str, Any]) -> dict[str, Any]:
inventory row now, so they are dropped here and applied through
:func:`link_facts` instead.
"""
allowed = {c.name for c in Node.__table__.columns}
allowed = {c.name for c in Node.__table__.columns} - _NODE_DERIVED_COLUMNS
return {k: v for k, v in payload.items() if k in allowed}
# Node columns the server derives rather than accepts: `display_view` is read
# back out of the services and properties a payload carries (see
# :func:`view_from_facts`), so a client sending one directly is ignored.
_NODE_DERIVED_COLUMNS = frozenset({"display_view"})
# Fields that are both a node column and a device fact: the node keeps a copy so
# a half-migrated database still renders, but the row is the truth.
_SHARED_FIELDS = ("label", "type")
@@ -502,6 +707,9 @@ def hydrated_node(node: Node, device: InventoryDevice | None) -> dict[str, Any]:
payload: dict[str, Any] = {
c.name: getattr(node, c.name) for c in node.__table__.columns
}
# An implementation detail of the split, not part of the wire shape: the
# view is reported *through* the services and properties it orders.
view = payload.pop("display_view", None) or {}
if device is None:
return payload
@@ -509,8 +717,8 @@ def hydrated_node(node: Node, device: InventoryDevice | None) -> dict[str, Any]:
payload[field] = getattr(device, field, None)
payload["label"] = device.label or node.label
payload["type"] = device.type or node.type
payload["services"] = device.services or []
payload["properties"] = device.properties or []
payload["services"] = apply_view(device.services, view.get("services"), "services")
payload["properties"] = apply_view(device.properties, view.get("properties"), "properties")
payload["show_hardware"] = bool(device.show_hardware)
payload["ieee_address"] = device.ieee_address
payload["status"] = device.status_live or "unknown"
@@ -591,6 +799,96 @@ async def _row_is_missing_facts(db: AsyncSession, device_id: str, facts: Mapping
)
def _with_later_properties(
view: dict[str, Any] | None, device: InventoryDevice
) -> dict[str, Any] | None:
"""``view``, plus the row's properties the pre-3.3.0 backup never saw.
They are what the user added while running 3.3.0-3.3.2 — the row was the
only place to add them then, and every canvas drew them. Recovering the old
view alone would take them off every canvas at once, which reads as data
loss. Appended in the row's order, after everything the backup placed.
"""
if view is None:
return None
entries = view.get("properties")
if not isinstance(entries, list):
return view
listed = {str(e.get("key")) for e in entries if isinstance(e, dict)}
later = [e for e in view_entries(device.properties, "properties") if e["key"] not in listed]
if not later:
return view
return {**view, "properties": [*entries, *later]}
async def seed_node_views(
db: AsyncSession, *, drawn: Callable[[], Mapping[str, Mapping[str, Any]]] | None = None
) -> int:
"""Give every linked node with no view one it can be held to.
Runs once, on the boot that adds `nodes.display_view`. Without it every
pre-existing node would fall through to "no view, show everything" and the
next scan would push a newly fingerprinted service onto all of them at once —
the leak this column exists to stop.
``drawn`` returns a map of node id to the services and properties that node
itself carried before 3.3.0 unioned them onto the row — the caller reads
them out of the pre-upgrade backup, and is only asked to when there is
actually a node to seed. Where it has an answer that answer wins, restoring
the arrangement the user made; where it does not, the row is the seed and
the canvas keeps showing exactly what it shows today. Does not commit.
The backup is 3.2.0-era, so it can only speak for what existed then. A
*property* the row has gained since — one the user added by hand while
running 3.3.0-3.3.2, when the row was all a canvas had — appears in no
backup entry, and seeding strictly from the backup would hide work the user
has been looking at for days. Those are appended visible, keeping the
recovered order and hidden flags for everything the backup does know.
Services are not treated that way: what a row gained since 3.3.0 is mostly a
scan's fingerprint, and holding it back is the whole point of the view.
"""
nodes = list(
(
await db.execute(
select(Node).where(
# A `device_id` naming a row that no longer exists cannot be
# seeded from anything, and leaving it in would match this
# query on every later boot — re-reading the backup file and
# logging a recovery that seeds nothing. Deleting a device
# leaves exactly that: SQLite runs with foreign keys off, so
# the `ON DELETE SET NULL` never fires on an existing table.
Node.device_id.in_(select(InventoryDevice.id)),
Node.display_view.is_(None),
)
)
)
.scalars()
.all()
)
if not nodes:
return 0
devices = await load_devices_for(db, nodes)
# Read the backup only now: on every later boot this function returns above
# and no file is opened at all.
was_drawn = drawn() if drawn is not None else {}
seeded = 0
for node in nodes:
device = devices.get(node.device_id or "")
if device is None: # pragma: no cover - the query already excluded these
continue
was = was_drawn.get(node.id)
# `strict`: an empty list in the backup is this canvas' real answer —
# it drew no service — and must not be read as "say nothing, show all".
if was:
view = next_view(None, view_from_facts(was), device, strict=True)
node.display_view = _with_later_properties(view, device)
else:
node.display_view = view_of_device(device)
seeded += 1
await db.flush()
return seeded
async def backfill_node_devices(db: AsyncSession) -> dict[str, int]:
"""Link every pre-3.3.0 canvas node to a Device Inventory row.
@@ -660,7 +958,16 @@ async def backfill_node_devices(db: AsyncSession) -> dict[str, int]:
# A node that is already linked only has its gaps filled: whatever
# is on the row was written after the migration and is newer.
device = await link_facts(
db, node, facts, overwrite_scalars=node.device_id is None
db,
node,
facts,
overwrite_scalars=node.device_id is None,
# The node's own columns are exactly what this canvas drew
# before the upgrade, empty lists included — so they define
# its view outright. Anything else the row carries (a service
# a scan fingerprinted, a property another canvas added)
# stays hidden here rather than appearing on every canvas.
strict_view=True,
)
if device is None:
continue
@@ -687,3 +994,27 @@ async def backfill_node_devices(db: AsyncSession) -> dict[str, int]:
await db.flush()
return {"linked": linked, "created": created, "merged": merged, "skipped": skipped}
async def attach_device_ids(
db: AsyncSession, nodes_raw: list[dict[str, Any]]
) -> list[dict[str, Any]]:
"""Stamp each imported node with the inventory row that already owns it.
A canvas-direct import upserts the inventory first, so the node dropped on
the canvas can point straight at that row instead of minting a second one
on the next save. Matching goes through :func:`find_device_for`, so a
Proxmox guest merged into a scanned row by ip/mac resolves too.
Returns new dicts — the caller's payload is left untouched.
"""
stamped: list[dict[str, Any]] = []
for node in nodes_raw:
device = await find_device_for(
db,
ip=node.get("ip"),
mac=node.get("mac"),
ieee=node.get("ieee_address"),
)
stamped.append({**node, "device_id": device.id} if device else dict(node))
return stamped
+23 -4
View File
@@ -13,6 +13,8 @@ import logging
import ssl
from typing import Any
from app.core.config import settings
logger = logging.getLogger(__name__)
try:
@@ -21,7 +23,10 @@ except ImportError: # pragma: no cover
aiomqtt = None # type: ignore[assignment]
_CONNECTION_TIMEOUT = 5.0 # seconds to verify broker reachability
_RESPONSE_TIMEOUT = 300.0 # seconds to wait for a gateway response (large meshes are slow)
# Fallback wait for a gateway response. The effective value comes from
# ``settings.mqtt_response_timeout`` (env MQTT_RESPONSE_TIMEOUT), read at call
# time so an operator with a large mesh can raise it without a code change.
_RESPONSE_TIMEOUT = 300.0
def _sanitize_mqtt_error(exc: BaseException) -> str:
@@ -71,11 +76,14 @@ async def request_response(
password: str | None = None,
tls: bool = False,
tls_insecure: bool = False,
response_timeout: float = _RESPONSE_TIMEOUT,
response_timeout: float | None = None,
) -> dict[str, Any]:
"""Publish ``request_payload`` to ``request_topic`` and return the first
JSON message received on ``response_topic`` as a dict.
``response_timeout`` defaults to ``settings.mqtt_response_timeout``
(env ``MQTT_RESPONSE_TIMEOUT``, 300 s) — a large mesh can take minutes.
Raises:
ImportError: if aiomqtt is not installed.
TimeoutError: if no response arrives in time.
@@ -88,6 +96,14 @@ async def request_response(
"Install it with: pip install aiomqtt"
)
timeout = (
float(settings.mqtt_response_timeout)
if response_timeout is None
else response_timeout
)
if timeout <= 0:
timeout = _RESPONSE_TIMEOUT
response_payload: dict[str, Any] = {}
tls_context = _build_tls_context(tls_insecure) if tls else None
@@ -122,12 +138,15 @@ async def request_response(
raise ValueError(f"Malformed MQTT response: {exc}") from exc
return
await asyncio.wait_for(_wait_for_response(), timeout=response_timeout)
await asyncio.wait_for(_wait_for_response(), timeout=timeout)
except aiomqtt.MqttError as exc:
raise ConnectionError(_sanitize_mqtt_error(exc)) from exc
except asyncio.TimeoutError as exc:
raise TimeoutError("Timed out waiting for MQTT response") from exc
raise TimeoutError(
f"Timed out waiting for MQTT response after {timeout:g}s — "
"raise MQTT_RESPONSE_TIMEOUT if your mesh is large"
) from exc
if not response_payload:
raise ValueError("Empty MQTT response received")
+30 -3
View File
@@ -15,10 +15,10 @@ from __future__ import annotations
import logging
from typing import Any
from sqlalchemy import select
from sqlalchemy import select, update
from sqlalchemy.ext.asyncio import AsyncSession
from app.db.models import Edge, Node
from app.db.models import Document, Edge, Node, RackDevice
from app.services.inventory_sync import find_device_for
logger = logging.getLogger(__name__)
@@ -121,7 +121,11 @@ async def dedupe_nodes_by_device(db: AsyncSession) -> int:
await db.execute(select(Node).where(Node.parent_id.in_(dup_ids)))
).scalars().all()
for child in children:
child.parent_id = canonical.id
# The canonical node may itself have been nested under one of its
# own duplicates. Re-pointing it here would make it its own parent,
# which freezes it on the canvas (#370) — detach it instead. Mirrors
# the self-loop edge deletion below.
child.parent_id = None if child.id == canonical.id else canonical.id
all_edges = (
await db.execute(
@@ -139,6 +143,29 @@ async def dedupe_nodes_by_device(db: AsyncSession) -> int:
continue
seen_pairs.add(key)
# A rack mount reads live status through `node_id`, and a document may
# describe the node itself. Both are `SET NULL` in the schema and SQLite
# runs with foreign keys off here, so a duplicate deleted from under them
# would leave the column naming a node that no longer exists.
await db.execute(
update(RackDevice).where(RackDevice.node_id.in_(dup_ids)).values(node_id=canonical.id)
)
own_doc = (
await db.execute(select(Document.id).where(Document.node_id == canonical.id).limit(1))
).scalar_one_or_none()
doc_rows = (
await db.execute(
select(Document)
.where(Document.node_id.in_(dup_ids))
.order_by(Document.created_at, Document.id)
)
).scalars().all()
for i, doc in enumerate(doc_rows):
# `documents.node_id` is unique (partial index): only one document
# can describe the survivor. The rest are orphaned, not deleted —
# what the user wrote outlives the node it was attached to.
doc.node_id = canonical.id if i == 0 and own_doc is None else None
await db.flush()
for dup in dups:
await db.delete(dup)
+39 -11
View File
@@ -93,24 +93,52 @@ def _guest_type_to_homelable(kind: str) -> str:
return "vm" if kind == "qemu" else "lxc"
def _extract_qemu_ip(agent_payload: dict[str, Any] | None) -> str | None:
"""Pull the first non-loopback IPv4 from a qemu guest-agent interfaces reply."""
def _iface_ipv4(iface: Any) -> str | None:
"""First non-loopback IPv4 of one guest-agent interface entry."""
if not isinstance(iface, dict):
return None
for addr in iface.get("ip-addresses") or []:
if not isinstance(addr, dict):
continue
if addr.get("ip-address-type") != "ipv4":
continue
ip = addr.get("ip-address")
if isinstance(ip, str) and ip and not ip.startswith("127."):
return ip
return None
def _extract_qemu_ip(
agent_payload: dict[str, Any] | None, mac: str | None = None
) -> str | None:
"""The guest's LAN IPv4 from a qemu guest-agent interfaces reply.
Prefer the interface whose ``hardware-address`` is the NIC MAC from the
guest ``/config`` — the agent lists every interface, and on a guest running
Docker/k8s the bridge (``cni0``, ``flannel``, ``docker0``) can come first.
Those addresses repeat across guests, and a shared address is what let one
guest's import land on another's inventory row. Fall back to the first
non-loopback IPv4 when no MAC is known or none matches.
"""
if not agent_payload:
return None
result = agent_payload.get("result")
if not isinstance(result, list):
return None
for iface in result:
if not isinstance(iface, dict):
continue
for addr in iface.get("ip-addresses") or []:
if not isinstance(addr, dict):
wanted = normalize_mac(mac)
if wanted:
for iface in result:
if not isinstance(iface, dict):
continue
if addr.get("ip-address-type") != "ipv4":
if normalize_mac(iface.get("hardware-address")) != wanted:
continue
ip = addr.get("ip-address")
if isinstance(ip, str) and ip and not ip.startswith("127."):
ip = _iface_ipv4(iface)
if ip:
return ip
for iface in result:
ip = _iface_ipv4(iface)
if ip:
return ip
return None
@@ -374,7 +402,7 @@ async def _resolve_guest_net(
data = await _get_json(
client, f"/nodes/{host_name}/qemu/{vmid}/agent/network-get-interfaces"
)
ip = _extract_qemu_ip(data)
ip = _extract_qemu_ip(data, mac)
else:
config = await _get_json(client, f"/nodes/{host_name}/lxc/{vmid}/config")
ip = _extract_lxc_ip(config)
+473 -90
View File
@@ -7,6 +7,7 @@ import re
import socket
import subprocess
import threading
import time
from dataclasses import dataclass, field
from datetime import datetime, timezone
from typing import Any
@@ -16,9 +17,11 @@ from sqlalchemy.ext.asyncio import AsyncSession
from app.core.config import settings
from app.db.models import InventoryDevice, ScanRun
from app.services.device_merge import merge_devices, reconcile_duplicates
from app.services.discovery_sources import add_source
from app.services.fingerprint import fingerprint_ports, suggest_node_type
from app.services.http_probe import probe_open_ports
from app.services.inventory_sync import ip_tokens, merge_services
from app.services.mac_utils import normalize_mac
logger = logging.getLogger(__name__)
@@ -61,8 +64,89 @@ def _valid_port_range(spec: str) -> bool:
return len(parts) == 1 or parts[0] <= parts[1]
def _build_port_spec(http_ranges: list[str] | None) -> str:
"""Combine the default port list with validated user ranges for nmap -p."""
# Every TCP port. Used by the per-device deep rescan, which trades minutes of
# nmap time for a service list that no curated port list can promise.
_FULL_PORTS = "1-65535"
# The deep rescan runs the full range in slices, one nmap call each, unioning
# what they find. A slice that runs long costs only its own ports — where a
# single 65535-port call that overruns costs everything (see _nmap_scan_single).
# It also gives the run somewhere to notice a stop request, and somewhere to
# give up when the budget is spent, without abandoning the ports already found.
_DEEP_CHUNK_SIZE = 8192
def _parse_port_spec(spec: str) -> list[tuple[int, int]]:
"""Parse an nmap ``-p`` spec into sorted, merged ``(start, end)`` ranges.
Accepts what the user can type in the deep-scan dialog: ``80``,
``8000-9000``, or a comma list of both. Returns ``[]`` for anything invalid
— the caller turns that into a 422 rather than handing nmap a bad ``-p``.
"""
ranges: list[tuple[int, int]] = []
for token in (t.strip() for t in spec.split(",")):
if not token or not _valid_port_range(token):
return []
parts = [int(p) for p in token.split("-")]
ranges.append((parts[0], parts[-1]))
if not ranges:
return []
ranges.sort()
merged = [ranges[0]]
for start, end in ranges[1:]:
last_start, last_end = merged[-1]
if start <= last_end + 1:
merged[-1] = (last_start, max(last_end, end))
else:
merged.append((start, end))
return merged
def _valid_port_spec(spec: str) -> bool:
"""True when ``spec`` is a usable comma list of ports/ranges."""
return bool(_parse_port_spec(spec))
def _port_chunks(spec: str, size: int = _DEEP_CHUNK_SIZE) -> list[str]:
"""Slice a port spec into nmap ``-p`` specs of at most ``size`` ports each.
Ranges are packed, not scanned one call per range: ``80,443`` is one chunk,
not two, while ``1-65535`` becomes eight. Each chunk is a scan boundary —
where a stop request lands and where the time budget is checked.
"""
chunks: list[str] = []
current: list[str] = []
budget = size
for start, end in _parse_port_spec(spec):
cursor = start
while cursor <= end:
take = min(budget, end - cursor + 1)
stop = cursor + take - 1
current.append(f"{cursor}-{stop}" if stop > cursor else str(cursor))
budget -= take
cursor = stop + 1
if budget == 0:
chunks.append(",".join(current))
current = []
budget = size
if current:
chunks.append(",".join(current))
return chunks
def _deep_port_chunks(size: int = _DEEP_CHUNK_SIZE) -> list[str]:
"""Slice the whole TCP range into nmap ``-p`` specs of ``size`` ports."""
return _port_chunks(_FULL_PORTS, size)
def _build_port_spec(http_ranges: list[str] | None, full: bool = False) -> str:
"""Combine the default port list with validated user ranges for nmap -p.
``full`` overrides everything with the whole TCP range — the deep rescan of
a single device, where completeness beats speed.
"""
if full:
return _FULL_PORTS
if not http_ranges:
return _EXTRA_PORTS
extra = [r.strip() for r in http_ranges if _valid_port_range(r.strip())]
@@ -237,7 +321,9 @@ async def _ping_sweep(target: str, run_id: str | None = None) -> dict[str, dict[
return alive
def _nmap_scan_single(host_dict: dict[str, Any], port_spec: str = _EXTRA_PORTS) -> dict[str, Any]:
def _nmap_scan_single(
host_dict: dict[str, Any], port_spec: str = _EXTRA_PORTS, bounded: bool = False
) -> dict[str, Any]:
"""
Phase 2 — single-IP port scan with service detection.
Runs in a thread (blocking). Returns the host dict enriched with open_ports.
@@ -261,8 +347,26 @@ def _nmap_scan_single(host_dict: dict[str, Any], port_spec: str = _EXTRA_PORTS)
# -sT without root but being explicit avoids edge cases.
scan_type = "-sS" if os.geteuid() == 0 else "-sT"
# --- Pass A: port discovery (no -sV, no host-timeout) ---
# --- Pass A: port discovery (no -sV) ---
# Default timing for the range scan: its ports are authoritative and a
# curated port list is fast either way.
#
# ``bounded`` is the deep rescan's timing, and carries NO --host-timeout on
# purpose: nmap answers a host timeout with "Skipping host <ip> due to host
# timeout" and discards *every* port it had already found, so a ceiling here
# turns a slow scan into one that reports nothing. The caller bounds total
# runtime by slicing the range instead.
#
# What costs the time is a host that drops packets: 8188 of 8192 ports
# filtered, each waiting out its probe. Measured against such a host, the
# retry pass is the whole cost — 8192 ports took 329s at --max-retries 1
# and 164s at 0, finding the same ports. Capping the RTT changed nothing
# (329s), so it is not set. Dropping retries risks missing a port that
# loses its one probe; that trade belongs to the deep scan alone, and the
# curated-port range scan keeps nmap's default retries.
discovery_args = f"{scan_type} --open -T4 -Pn -p {port_spec}"
if bounded:
discovery_args += " --max-retries 0 --min-rate 2000"
logger.debug("[Phase 2] %s discovery args: %s", ip, discovery_args)
nm_disc = nmap.PortScanner()
try:
@@ -469,11 +573,66 @@ def _mock_scan(target: str) -> list[dict[str, Any]]:
]
def _group_by_shared_ip(rows: list[InventoryDevice]) -> list[list[InventoryDevice]]:
"""Group rows that share at least one address, transitively.
A row may hold several comma-separated addresses, so grouping is per token:
``10.0.0.1`` and ``10.0.0.1, 10.0.1.1`` are the same device, and a row
bridging two others (``A,B`` next to ``A`` and ``B``) puts all three in one
group. Input order is preserved inside each group, so a caller ordering by
``discovered_at`` still gets the oldest row first.
"""
# Union-find over row indices, keyed by the addresses they carry.
parent = list(range(len(rows)))
def find(i: int) -> int:
while parent[i] != i:
parent[i] = parent[parent[i]]
i = parent[i]
return i
owner: dict[str, int] = {}
for i, row in enumerate(rows):
for tok in ip_tokens(row.ip):
first = owner.setdefault(tok, i)
root_a, root_b = find(first), find(i)
if root_a != root_b:
parent[max(root_a, root_b)] = min(root_a, root_b)
groups: dict[int, list[InventoryDevice]] = {}
for i, row in enumerate(rows):
groups.setdefault(find(i), []).append(row)
return list(groups.values())
def _collapse_targets(
group: list[InventoryDevice],
) -> tuple[InventoryDevice, list[InventoryDevice]]:
"""The row a duplicate group folds into, and the rows safe to delete for it.
An ``approved`` row owns the canvas-link semantics — nodes and rack mounts
point at it under ``ON DELETE SET NULL`` — so deleting one silently unlinks
whatever drew it. With one approved row that row is the merge target and the
rest collapse into it. With *two*, the group is two canvas-linked devices
laying claim to one address, and a background scan is not the place to
decide which of them loses its links: every approved row stands, and only
the pending ones are collapsed. The pair then stays visible in the inventory
for the user to sort out.
"""
approved = [row for row in group if row.status == "approved"]
keep = approved[0] if approved else group[0]
if len(approved) > 1:
return keep, [row for row in group if row.status != "approved"]
return keep, [row for row in group if row is not keep]
async def _dedupe_pending_by_ip(db: AsyncSession) -> int:
"""Collapse duplicate non-hidden inventory rows that share an IP into one.
Keeps an ``approved`` row when present (it carries canvas-link semantics),
otherwise the oldest row, and deletes the rest. Returns the number deleted.
Rows carrying several addresses count as duplicates of any row holding one
of them — that is what declaring the extra address is for.
"""
rows = (await db.execute(
select(InventoryDevice)
@@ -481,26 +640,140 @@ async def _dedupe_pending_by_ip(db: AsyncSession) -> int:
.order_by(InventoryDevice.discovered_at)
)).scalars().all()
by_ip: dict[str, list[InventoryDevice]] = {}
for row in rows:
if row.ip is None: # guarded by the query, but keeps the type checker happy
continue
by_ip.setdefault(row.ip, []).append(row)
deleted = 0
for group in by_ip.values():
for group in _group_by_shared_ip(list(rows)):
if len(group) < 2:
continue
keep = next((r for r in group if r.status == "approved"), group[0])
for dup in group:
if dup is not keep:
await db.delete(dup)
deleted += 1
keep, dups = _collapse_targets(group)
if dups:
# Non-destructive, for the same reason as `process_host`: whatever
# drew the duplicate follows it into `keep`.
await merge_devices(db, keep, dups)
deleted += len(dups)
if deleted:
await db.commit()
return deleted
async def process_host(
db: AsyncSession,
host: dict[str, Any],
*,
hidden_ips: set[str],
deep_scan: DeepScanOptions,
discovery_source: str = "arp",
) -> str:
"""Fold one scanned host into ``device_inventory`` and commit.
Shared by the range scan and the single-device deep rescan, so both apply
the same matching, merge and de-duplication rules.
Returns ``"skipped"`` (hidden by the user), ``"created"`` (a new inventory
row) or ``"updated"`` (an existing row refreshed).
"""
ip = host["ip"]
# Skip only user-hidden devices. On-canvas devices are kept so they
# surface in the inventory with a canvas-presence badge.
if ip in hidden_ips:
logger.debug("Skipping %s — hidden by user", ip)
return "skipped"
open_ports = host["open_ports"]
# Deep-scan HTTP probe: enrich open ports with title/header signals so
# fingerprint can confirm services on custom ports. No-op when disabled
# or when the host has no open ports (e.g. mDNS-only discovery).
if deep_scan.http_probe_enabled and open_ports:
open_ports = await probe_open_ports(
ip, open_ports, verify_tls=deep_scan.verify_tls
)
norm_mac = normalize_mac(host.get("mac"))
services = fingerprint_ports(open_ports)
suggested_type = suggest_node_type(open_ports, norm_mac)
# One inventory row per device. Match by IP OR MAC across pending AND
# approved so a re-scan refreshes the existing row instead of spawning
# a duplicate — and so a device previously imported from Proxmox (which
# may have no IP but a known NIC MAC) reconciles with this scan instead
# of doubling up. Hidden rows are already skipped above.
#
# The IP side matches per token: a row may hold several comma-separated
# addresses (a router's LAN and VLAN legs, a host and its VIP), declared by
# the user precisely so this scan folds into it rather than minting a second
# row every run. `contains` only narrows the query — the tokens are confirmed
# below, because "10.0.0.4" is a substring of "10.0.0.40" and these rows are
# deleted, not merely skipped.
match_cond = [InventoryDevice.ip.contains(ip)]
if norm_mac:
match_cond.append(InventoryDevice.mac == norm_mac)
candidates = (await db.execute(
select(InventoryDevice)
.where(or_(*match_cond), InventoryDevice.status != "hidden")
.order_by(InventoryDevice.discovered_at)
)).scalars().all()
existing_rows = [
row for row in candidates
if ip in ip_tokens(row.ip) or (norm_mac is not None and row.mac == norm_mac)
]
if existing_rows:
# Prefer an approved row (it owns the canvas link semantics),
# otherwise the oldest. Collapse any leftover duplicates created
# by earlier scans — except a second approved row, which keeps its
# own canvas links (see `_collapse_targets`).
keep, dups = _collapse_targets(existing_rows)
# Merge rather than delete: a duplicate row may be the one a canvas node
# or a rack mount points at, and SQLite runs with foreign keys off here
# so the declared `ON DELETE SET NULL` never fires. `merge_devices`
# re-points every reference onto `keep` and unions the facts first.
if dups:
await merge_devices(db, keep, dups)
keep.ip = keep.ip or ip # fill an IP a Proxmox import lacked
keep.mac = norm_mac or keep.mac
keep.hostname = host.get("hostname") or keep.hostname
keep.os = host.get("os") or keep.os
# Union, never replace. Since 3.3.0 the row is the only copy of
# a device's services — every canvas drawing it reads this list
# — so overwriting it with the fingerprint would delete services
# the user added by hand, on every canvas at once. What a scan
# finds is added; what it no longer sees stays. A service the
# user does not want on a canvas is hidden by that node's view,
# which is where "I deleted this" belongs.
# discovered=: the fingerprint may add services and refresh what it
# knows, but never repaints an icon or category the user chose.
keep.services = merge_services(keep.services, services, discovered=True)
# Don't downgrade a Proxmox-typed guest (vm/lxc) to the generic
# scan guess; the importer knows the true type.
if not (keep.ieee_address or "").startswith("pve-"):
keep.suggested_type = suggested_type
# Merged row carries both sources (e.g. ["proxmox", "arp"]).
keep.discovery_sources = add_source(keep.discovery_sources, discovery_source)
# status preserved — an approved device stays approved.
# Stamp last_scan on the row so every canvas drawing the device
# shows when the scanner last observed it. The row is the one
# place that fact belongs; a node only draws it.
keep.last_scan = datetime.now(timezone.utc)
outcome = "updated"
else:
db.add(InventoryDevice(
ip=ip,
mac=norm_mac,
hostname=host.get("hostname"),
os=host.get("os"),
services=services,
suggested_type=suggested_type,
status="pending",
discovery_source=discovery_source,
discovery_sources=[discovery_source],
last_scan=datetime.now(timezone.utc),
))
outcome = "created"
await db.commit()
return outcome
async def run_scan(
ranges: list[str],
db: AsyncSession,
@@ -529,7 +802,11 @@ async def run_scan(
hidden_ips_result = await db.execute(
select(InventoryDevice.ip).where(InventoryDevice.status == "hidden")
)
hidden_ips: set[str] = {row[0] for row in hidden_ips_result.fetchall()}
# Per address, not per row: a hidden device holding several addresses is
# hidden at every one of them.
hidden_ips: set[str] = {
tok for row in hidden_ips_result.fetchall() for tok in ip_tokens(row[0])
}
# Collapse any pre-existing duplicate inventory rows (same IP, non-hidden)
# left over from older scans, so the device shows up exactly once even if
@@ -544,81 +821,17 @@ async def run_scan(
async def _process_host(host: dict[str, Any], discovery_source: str = "arp") -> None:
nonlocal devices_found
ip = host["ip"]
# Skip only user-hidden devices. On-canvas devices are kept so they
# surface in the inventory with a canvas-presence badge.
if ip in hidden_ips:
logger.debug("Skipping %s — hidden by user", ip)
outcome = await process_host(
db,
host,
hidden_ips=hidden_ips,
deep_scan=deep_scan,
discovery_source=discovery_source,
)
if outcome == "skipped":
return
open_ports = host["open_ports"]
# Deep-scan HTTP probe: enrich open ports with title/header signals so
# fingerprint can confirm services on custom ports. No-op when disabled
# or when the host has no open ports (e.g. mDNS-only discovery).
if deep_scan.http_probe_enabled and open_ports:
open_ports = await probe_open_ports(
ip, open_ports, verify_tls=deep_scan.verify_tls
)
norm_mac = normalize_mac(host.get("mac"))
services = fingerprint_ports(open_ports)
suggested_type = suggest_node_type(open_ports, norm_mac)
# One inventory row per device. Match by IP OR MAC across pending AND
# approved so a re-scan refreshes the existing row instead of spawning
# a duplicate — and so a device previously imported from Proxmox (which
# may have no IP but a known NIC MAC) reconciles with this scan instead
# of doubling up. Hidden rows are already skipped above.
match_cond = [InventoryDevice.ip == ip]
if norm_mac:
match_cond.append(InventoryDevice.mac == norm_mac)
existing_rows = (await db.execute(
select(InventoryDevice)
.where(or_(*match_cond), InventoryDevice.status != "hidden")
.order_by(InventoryDevice.discovered_at)
)).scalars().all()
if existing_rows:
# Prefer an approved row (it owns the canvas link semantics),
# otherwise the oldest. Collapse any leftover duplicates created
# by earlier scans.
keep = next((r for r in existing_rows if r.status == "approved"), existing_rows[0])
for dup in existing_rows:
if dup is not keep:
await db.delete(dup)
keep.ip = keep.ip or ip # fill an IP a Proxmox import lacked
keep.mac = norm_mac or keep.mac
keep.hostname = host.get("hostname") or keep.hostname
keep.os = host.get("os") or keep.os
keep.services = services
# Don't downgrade a Proxmox-typed guest (vm/lxc) to the generic
# scan guess; the importer knows the true type.
if not (keep.ieee_address or "").startswith("pve-"):
keep.suggested_type = suggested_type
# Merged row carries both sources (e.g. ["proxmox", "arp"]).
keep.discovery_sources = add_source(keep.discovery_sources, discovery_source)
# status preserved — an approved device stays approved.
# Stamp last_scan on the row so every canvas drawing the device
# shows when the scanner last observed it. The row is the one
# place that fact belongs; a node only draws it.
keep.last_scan = datetime.now(timezone.utc)
else:
db.add(InventoryDevice(
ip=ip,
mac=norm_mac,
hostname=host.get("hostname"),
os=host.get("os"),
services=services,
suggested_type=suggested_type,
status="pending",
discovery_source=discovery_source,
discovery_sources=[discovery_source],
last_scan=datetime.now(timezone.utc),
))
if outcome == "created":
devices_found += 1
await db.commit()
await broadcast_scan_update(run_id=run_id, devices_found=devices_found)
# nmap scan per CIDR — results stream in progressively
@@ -651,6 +864,14 @@ async def run_scan(
else:
mdns_task.cancel()
# A scan is where a row first learns its MAC, so it is also where two
# rows can first be proven to be one device — fold those together before
# reporting the run done.
merged = await reconcile_duplicates(db)
if merged:
await db.commit()
logger.info("Scan %s: merged %d duplicate inventory row(s)", run_id, merged)
# Mark scan as done or cancelled
run = await db.get(ScanRun, run_id)
if run:
@@ -672,3 +893,165 @@ async def run_scan(
finally:
with _cancelled_lock:
_cancelled_runs.discard(run_id)
async def run_device_scan(
device_id: str,
db: AsyncSession,
run_id: str,
deep_scan: DeepScanOptions | None = None,
full_ports: bool = True,
ports: str | None = None,
) -> None:
"""Deep-rescan one known device and refresh its inventory row.
No ping sweep and no mDNS: the device is already known, so the IP goes
straight to the phase-2 port scan (``-Pn``). ``full_ports`` scans all 65535
TCP ports, which is the point of the feature — a device added before the
scanner knew a service, or listening on a port no curated list covers.
Minutes, not seconds; the run is cancellable like any other.
``ports`` narrows that to a user-chosen spec (``80,443``, ``1-1024``) and
wins over ``full_ports`` — the dialog prefills the full range, so a caller
that passes something else means it.
"""
from app.api.routes.status import broadcast_scan_update
deep_scan = deep_scan or DeepScanOptions()
port_spec = (
ports if ports else _build_port_spec(deep_scan.http_ranges, full=full_ports)
)
try:
device = await db.get(InventoryDevice, device_id)
if device is None or not device.ip:
raise ValueError("Device has no IP to scan")
host: dict[str, Any] = {
"ip": device.ip,
"mac": device.mac,
"hostname": device.hostname,
"os": device.os,
"open_ports": [],
}
# A rescan re-observes a device through the source it already has; it
# is not a network discovery. Hardcoding "arp" here would tell a
# Proxmox guest, a rack mount or a hand-added host that the network
# scanner found it, and it would then answer that source filter.
source = device.discovery_source or "arp"
# The full range goes out in slices so a stop request lands within one
# slice instead of at the end, and so a spent budget keeps the ports
# found so far. A curated port list is one call, as before.
if ports:
chunks = _port_chunks(ports)
elif full_ports:
chunks = _deep_port_chunks()
else:
chunks = [port_spec]
# Retry-free timing pays for itself over thousands of ports on a host
# that drops packets; over a handful it only costs accuracy.
total_ports = sum(end - start + 1 for start, end in _parse_port_spec(port_spec))
bounded = total_ports > _DEEP_CHUNK_SIZE
deadline = time.monotonic() + settings.scanner_deep_host_timeout
found: list[dict[str, Any]] = []
seen: set[tuple[str, int]] = set()
skipped = 0
for i, chunk in enumerate(chunks):
if _is_cancelled(run_id):
skipped = len(chunks) - i
break
if i and time.monotonic() > deadline:
skipped = len(chunks) - i
logger.warning(
"[Deep scan] %s — budget of %ds spent, %d port range(s) not scanned",
host["ip"], settings.scanner_deep_host_timeout, skipped,
)
break
# bounded: retry-free timing, for a host that drops packets.
scanned = await asyncio.to_thread(
_nmap_scan_single, dict(host), chunk, bounded
)
for port in scanned.get("open_ports") or []:
key = (port["protocol"], port["port"])
if key not in seen:
seen.add(key)
found.append(port)
host["mac"] = host["mac"] or scanned.get("mac")
host["os"] = scanned.get("os") or host["os"]
host["open_ports"] = found
# A partial sweep still says what it saw; the row unions it in.
partial = (
f"Scanned {len(chunks) - skipped}/{len(chunks)} port ranges "
f"({len(found)} open) — the rest was not reached"
if skipped
else None
)
devices_found = 0
if not _is_cancelled(run_id):
# The row is being rescanned on the user's request, so it is never
# "hidden" from itself — the route rejects hidden devices upfront.
outcome = await process_host(
db, host, hidden_ips=set(), deep_scan=deep_scan, discovery_source=source
)
if outcome == "created":
devices_found = 1
await broadcast_scan_update(run_id=run_id, devices_found=devices_found)
run = await db.get(ScanRun, run_id)
if run:
run.status = "cancelled" if _is_cancelled(run_id) else "done"
run.devices_found = devices_found
# Not a failure: a done run carrying an advisory, the way a Proxmox
# import reports what it could not see. Never let a partial sweep
# read as a complete one.
if partial and run.status == "done":
run.error = partial
run.finished_at = datetime.now(timezone.utc)
await db.commit()
except Exception as exc:
logger.error("Device scan failed: %s", exc)
await db.rollback()
run = await db.get(ScanRun, run_id)
if run:
run.status = "error"
run.error = str(exc)
run.finished_at = datetime.now(timezone.utc)
await db.commit()
finally:
with _cancelled_lock:
_cancelled_runs.discard(run_id)
async def reconcile_orphan_runs(db: AsyncSession) -> int:
"""
Mark every scan run still flagged `running` at startup as errored.
Scans live on a background thread inside this process, so nothing can
legitimately be `running` the moment we boot: any such row belongs to a
previous process that died mid-scan (an OOM kill, `docker stop`, a crash).
Left alone the row is immortal, and it also locks the target out — the
trigger endpoints reject a new scan while one is `running` for the same
range, so a single kill blocks re-scanning for good.
Returns the number of rows reconciled.
"""
orphans = (
await db.execute(select(ScanRun).where(ScanRun.status == "running"))
).scalars().all()
if not orphans:
return 0
now = datetime.now(timezone.utc)
for run in orphans:
# "error" is the word run_scan and run_device_scan already write when
# they fail; Scan History filters and colours that one.
run.status = "error"
run.error = "Interrupted: the backend restarted while this scan was running"
run.finished_at = now
await db.commit()
logger.warning("Reconciled %d scan run(s) left running by a previous process", len(orphans))
return len(orphans)
+23 -4
View File
@@ -103,8 +103,16 @@ async def _ping(host: str) -> bool:
async def _http_get(url: str, verify: bool = False) -> bool:
async with httpx.AsyncClient(verify=verify, timeout=5) as client:
resp = await client.get(url, follow_redirects=True)
# Only the status line matters here. A plain .get() buffers the whole body
# first, and some endpoints stream without end (bandwidth-test endpoints,
# MJPEG cameras, log tails) — enough to OOM the backend. timeout=5 does not
# save us: httpx applies it per network operation, not to the total time
# spent draining a socket that keeps delivering data. stream() closes the
# connection on exit without draining it.
async with (
httpx.AsyncClient(verify=verify, timeout=5) as client,
client.stream("GET", url, follow_redirects=True) as resp,
):
return resp.status_code < 500
@@ -126,10 +134,17 @@ async def _tcp_connect(host: str, port: int) -> bool:
# grey (unknown) rather than going red. An open TCP socket doesn't prove the
# service is healthy, and a closed one flaps red misleadingly (e.g. SSH on a
# box that simply firewalls 22). Only HTTP(S)-reachable services are checked.
#
# 515 and 9100-9107 are raw printing (LPD / JetDirect), and they are here for a
# stronger reason than the rest: a printer treats any bytes on 9100 as a print
# job, so a GET line makes it print a page — every 60 s, for as long as the node
# exists. Node Exporter also lives on 9100 and loses its status check because of
# this; a grey dot is the cheaper mistake.
_PRINTER_PORTS = frozenset({515, *range(9100, 9108)})
_NON_HTTP_PORTS = frozenset({
22, 21, 23, 25, 465, 587, 53, 110, 143, 993, 995, 389, 636, 445, 514,
1433, 3306, 5432, 5672, 6379, 9092, 11211, 27017, 27018,
})
}) | _PRINTER_PORTS
_HTTPS_PORTS = frozenset({443, 8443})
@@ -211,7 +226,11 @@ async def check_service(svc: dict[str, Any], host: str | None) -> str:
port in _HTTPS_PORTS or "https" in name or "ssl" in name or "tls" in name
) else "http")
url_host = _service_host(svc, host)
url = f"{scheme}://{url_host}" + (f":{port}" if port is not None else "")
# A host override usually points at a reverse proxy, where the scanned
# port is an internal detail that would break the public URL. Only a
# port typed into the override itself is used.
url_port = override_port if override else port
url = f"{scheme}://{url_host}" + (f":{url_port}" if url_port is not None else "")
return "online" if await _http_get(url, verify=False) else "offline"
except Exception as exc:
logger.debug("Service check failed for %s:%s (%s)", host, port, exc)
+23 -3
View File
@@ -7,6 +7,7 @@ import json
import logging
from typing import Any
from app.core.config import settings
from app.services.mqtt_common import _build_tls_context, _sanitize_mqtt_error
logger = logging.getLogger(__name__)
@@ -19,7 +20,10 @@ except ImportError: # pragma: no cover
_NETWORKMAP_REQUEST_TOPIC = "{base_topic}/bridge/request/networkmap"
_NETWORKMAP_RESPONSE_TOPIC = "{base_topic}/bridge/response/networkmap"
_CONNECTION_TIMEOUT = 5.0 # seconds to verify broker reachability
_NETWORKMAP_TIMEOUT = 300.0 # seconds to wait for the networkmap response (large meshes can be slow)
# Fallback wait for the networkmap response. The effective value comes from
# ``settings.zigbee_networkmap_timeout`` (env ZIGBEE_NETWORKMAP_TIMEOUT), read at
# call time so an operator with a large mesh can raise it without a code change.
_NETWORKMAP_TIMEOUT = 300.0
# Re-exported for backwards compatibility — these now live in mqtt_common.
__all__ = ["_build_tls_context", "_sanitize_mqtt_error"]
@@ -238,9 +242,14 @@ async def fetch_networkmap(
password: str | None = None,
tls: bool = False,
tls_insecure: bool = False,
response_timeout: float | None = None,
) -> tuple[list[dict[str, Any]], list[dict[str, Any]]]:
"""Connect to the MQTT broker, request the Z2M networkmap, and return (nodes, edges).
``response_timeout`` defaults to ``settings.zigbee_networkmap_timeout``
(env ``ZIGBEE_NETWORKMAP_TIMEOUT``, 300 s) — a 200+ device mesh can take
minutes to answer.
Raises:
TimeoutError: if the broker does not respond in time.
ConnectionError: if the broker cannot be reached.
@@ -252,6 +261,14 @@ async def fetch_networkmap(
"Install it with: pip install aiomqtt"
)
timeout = (
float(settings.zigbee_networkmap_timeout)
if response_timeout is None
else response_timeout
)
if timeout <= 0:
timeout = _NETWORKMAP_TIMEOUT
request_topic = _NETWORKMAP_REQUEST_TOPIC.format(base_topic=base_topic)
response_topic = _NETWORKMAP_RESPONSE_TOPIC.format(base_topic=base_topic)
@@ -295,12 +312,15 @@ async def fetch_networkmap(
) from exc
return
await asyncio.wait_for(_wait_for_response(), timeout=_NETWORKMAP_TIMEOUT)
await asyncio.wait_for(_wait_for_response(), timeout=timeout)
except aiomqtt.MqttError as exc:
raise ConnectionError(_sanitize_mqtt_error(exc)) from exc
except asyncio.TimeoutError as exc:
raise TimeoutError("Timed out waiting for networkmap response") from exc
raise TimeoutError(
f"Timed out waiting for networkmap response after {timeout:g}s — "
"raise ZIGBEE_NETWORKMAP_TIMEOUT if your mesh is large"
) from exc
if not response_payload:
raise ValueError("Empty networkmap response received")
+41 -1
View File
@@ -1,3 +1,4 @@
import importlib
import os
# Must be set before any app import so pydantic-settings can resolve the required field.
@@ -8,7 +9,7 @@ from httpx import ASGITransport, AsyncClient
from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker, create_async_engine
from app.core.security import hash_password
from app.db.database import Base, get_db
from app.db.database import DOCUMENT_DDL, Base, _try_migrate, get_db
from app.main import app
TEST_DB_URL = "sqlite+aiosqlite:///:memory:"
@@ -27,9 +28,38 @@ async def db_session():
engine = create_async_engine(TEST_DB_URL)
async with engine.begin() as conn:
await conn.run_sync(Base.metadata.create_all)
# Apply the same documents DDL init_db does, through the same helper, so
# the suite matches boot. `_try_migrate` matters here: on a database
# create_all has just built, every ALTER in that list is a no-op that
# raises "duplicate column" — swallowing it is the point, not a leniency.
for label, sql in DOCUMENT_DDL:
await _try_migrate(conn, sql, label=label)
session_factory = async_sessionmaker(engine, class_=AsyncSession, expire_on_commit=False)
# Background tasks open their own session through `AsyncSessionLocal` — the
# request-scoped `get_db` override does not reach them. Left alone they would
# run against the *real* configured database, so point the factory at the
# test engine everywhere it was imported by name.
patched = [
importlib.import_module(name)
for name in (
"app.db.database",
"app.api.routes.zigbee",
"app.api.routes.zwave",
"app.api.routes.proxmox",
"app.api.routes.scan",
)
]
originals = [
(module, getattr(module, "AsyncSessionLocal", None)) for module in patched
]
for module, original in originals:
if original is not None:
module.AsyncSessionLocal = session_factory
async with session_factory() as session:
yield session
for module, original in originals:
if original is not None:
module.AsyncSessionLocal = original
async with engine.begin() as conn:
await conn.run_sync(Base.metadata.drop_all)
await engine.dispose()
@@ -48,3 +78,13 @@ async def headers(client: AsyncClient):
"""Authenticated Bearer headers for the default admin test user."""
res = await client.post("/api/v1/auth/login", json={"username": "admin", "password": "admin"})
return {"Authorization": f"Bearer {res.json()['access_token']}"}
@pytest.fixture(autouse=True)
def _reset_doc_search_probe():
"""Forget whether FTS5 was available — each test gets a fresh database."""
from app.services import doc_search
doc_search.reset_availability_cache()
yield
doc_search.reset_availability_cache()
@@ -9,6 +9,7 @@ those columns, so if they survive the upgrade every INSERT fails with
``NOT NULL constraint failed: nodes.status`` and approving a device is dead.
"""
import os
import shutil
os.environ.setdefault("SECRET_KEY", "test-only-secret-key-not-for-production")
@@ -188,3 +189,303 @@ async def test_a_node_the_backfill_cannot_link_still_leaves_nodes_insertable(db_
finally:
await check.dispose()
await engine.dispose()
async def test_upgrade_keeps_each_canvas_drawing_its_own_services(db_320):
"""3.3.3: order and visibility become the node's, and the upgrade freezes them.
Two canvases drew the same host with different service lists, and a scan had
already fingerprinted a third the user put on neither. All three converge on
one inventory row — so each node must come out of the upgrade still drawing
what it drew, with the scanner's guess hidden on both.
"""
db_path, engine = db_320
await _build_320(engine)
ssh = '[{"port": 22, "protocol": "tcp", "service_name": "ssh"}]'
both = '[{"port": 22, "protocol": "tcp", "service_name": "ssh"}, ' \
'{"port": 443, "protocol": "tcp", "service_name": "https"}]'
async with engine.begin() as conn:
await conn.exec_driver_sql(f"UPDATE nodes SET services = '{ssh}' WHERE id = 'n1'")
await conn.exec_driver_sql(f"UPDATE nodes SET services = '{both}' WHERE id = 'n2'")
await database.init_db()
from app.db.models import InventoryDevice, Node
session_factory = database.async_sessionmaker(engine, expire_on_commit=False)
async with session_factory() as session:
drawn = {}
for node_id in ("n1", "n2"):
node = await session.get(Node, node_id)
device = await session.get(InventoryDevice, node.device_id)
payload = inventory_sync.hydrated_node(node, device)
drawn[node_id] = [s["service_name"] for s in payload["services"] if s.get("visible", True)]
# Seeded, not left to "show the whole row".
assert node.display_view is not None
assert drawn == {"n1": ["ssh"], "n2": ["ssh", "https"]}
# What a scan finds next lands on the row, and on no canvas.
node = await session.get(Node, "n1")
device = await session.get(InventoryDevice, node.device_id)
device.services = [
*device.services,
{"port": 3001, "protocol": "tcp", "service_name": "Uptime Kuma"},
]
await session.commit()
payload = inventory_sync.hydrated_node(node, device)
assert [(s["service_name"], s.get("visible", True)) for s in payload["services"]] == [
("ssh", True), ("Uptime Kuma", False),
]
await engine.dispose()
async def _wind_back_to_3_3_2(engine, extra_service: dict) -> None:
"""A database that already took the 3.3.0 upgrade, leak included.
Both nodes point at one row holding the union of what each drew plus what a
scan found, and neither has a view — which is every 3.3.0-3.3.2 install.
"""
from app.db.models import InventoryDevice, Node
async with engine.begin() as conn:
await conn.exec_driver_sql("UPDATE nodes SET display_view = NULL")
session_factory = database.async_sessionmaker(engine, expire_on_commit=False)
async with session_factory() as session:
node = await session.get(Node, "n1")
one = await session.get(InventoryDevice, node.device_id)
other = await session.get(InventoryDevice, (await session.get(Node, "n2")).device_id)
for device in (one, other):
device.services = [
{"port": 22, "protocol": "tcp", "service_name": "ssh"},
{"port": 443, "protocol": "tcp", "service_name": "https"},
extra_service,
]
await session.commit()
async def test_a_database_already_on_3_3_recovers_its_layout_from_the_backup(db_320):
"""The second upgrade path: 3.3.0-3.3.2, where the legacy columns are gone.
3.3.0 unioned every canvas' services onto one row, so the row can no longer
say who drew what — but the backup taken before that migration still can.
Recovering from it is the difference between the user getting their canvases
back and getting each canvas showing every other canvas' services.
"""
_, engine = db_320
await _build_320(engine)
ssh = '[{"port": 22, "protocol": "tcp", "service_name": "ssh"}]'
both = '[{"port": 22, "protocol": "tcp", "service_name": "ssh"}, ' \
'{"port": 443, "protocol": "tcp", "service_name": "https"}]'
async with engine.begin() as conn:
await conn.exec_driver_sql(f"UPDATE nodes SET services = '{ssh}' WHERE id = 'n1'")
await conn.exec_driver_sql(f"UPDATE nodes SET services = '{both}' WHERE id = 'n2'")
await database.init_db() # 3.2.0 -> 3.3.x, and the backup that predates it.
kuma = {"port": 3001, "protocol": "tcp", "service_name": "Uptime Kuma"}
await _wind_back_to_3_3_2(engine, kuma)
await database.init_db()
from app.db.models import InventoryDevice, Node
session_factory = database.async_sessionmaker(engine, expire_on_commit=False)
async with session_factory() as session:
drawn = {}
for node_id in ("n1", "n2"):
node = await session.get(Node, node_id)
device = await session.get(InventoryDevice, node.device_id)
payload = inventory_sync.hydrated_node(node, device)
drawn[node_id] = [s["service_name"] for s in payload["services"] if s.get("visible", True)]
assert drawn == {"n1": ["ssh"], "n2": ["ssh", "https"]}
await engine.dispose()
async def test_a_property_added_while_on_3_3_survives_the_recovery(db_320):
"""The backup is 3.2.0-era and cannot know what the user added afterwards.
On 3.3.0-3.3.2 the row was the only place to add a property, and every
canvas drew it. Recovering the view strictly from the backup would hide it
on all of them at once — the "my properties disappeared" half of #347. The
scanner's service find is still held back: only properties are appended.
"""
_, engine = db_320
await _build_320(engine)
rack = '[{"key": "Rack", "value": "A", "icon": null, "visible": true}]'
ssh = '[{"port": 22, "protocol": "tcp", "service_name": "ssh"}]'
async with engine.begin() as conn:
await conn.exec_driver_sql(
f"UPDATE nodes SET properties = '{rack}', services = '{ssh}' WHERE id = 'n1'"
)
await database.init_db() # 3.2.0 -> 3.3.x, and the backup that predates it.
kuma = {"port": 3001, "protocol": "tcp", "service_name": "Uptime Kuma"}
await _wind_back_to_3_3_2(engine, kuma)
from app.db.models import InventoryDevice, Node
session_factory = database.async_sessionmaker(engine, expire_on_commit=False)
async with session_factory() as session:
node = await session.get(Node, "n1")
device = await session.get(InventoryDevice, node.device_id)
# What the user re-added by hand while running 3.3.x.
device.properties = [
*device.properties,
{"key": "Ports", "value": "8", "icon": None, "visible": True},
]
await session.commit()
await database.init_db()
async with session_factory() as session:
node = await session.get(Node, "n1")
device = await session.get(InventoryDevice, node.device_id)
payload = inventory_sync.hydrated_node(node, device)
assert [(p["key"], p.get("visible", True)) for p in payload["properties"]] == [
("Rack", True), ("Ports", True),
]
# And the scan's find is still off this canvas.
assert [s["service_name"] for s in payload["services"] if s.get("visible", True)] == ["ssh"]
await engine.dispose()
async def test_without_a_usable_backup_the_row_is_the_seed(db_320):
"""No backup to recover from: keep showing what the canvas shows today.
Nothing is taken away — the user simply keeps the merged list they have been
looking at since 3.3.0, and only what the row gains *after* this boot is
held back.
"""
db_path, engine = db_320
await _build_320(engine)
await database.init_db()
kuma = {"port": 3001, "protocol": "tcp", "service_name": "Uptime Kuma"}
await _wind_back_to_3_3_2(engine, kuma)
for backup in db_path.parent.glob(f"{db_path.name}.back-*"):
backup.unlink()
await database.init_db()
from app.db.models import InventoryDevice, Node
session_factory = database.async_sessionmaker(engine, expire_on_commit=False)
async with session_factory() as session:
node = await session.get(Node, "n1")
device = await session.get(InventoryDevice, node.device_id)
payload = inventory_sync.hydrated_node(node, device)
assert [s["service_name"] for s in payload["services"]] == ["ssh", "https", "Uptime Kuma"]
assert all(s.get("visible", True) for s in payload["services"])
# From here on the node is pinned: the next scan's find is held back.
device.services = [*device.services, {"port": 5001, "protocol": "tcp", "service_name": "Synology DSM HTTPS"}]
await session.commit()
payload = inventory_sync.hydrated_node(node, device)
assert [s["service_name"] for s in payload["services"] if not s.get("visible", True)] == [
"Synology DSM HTTPS"
]
await engine.dispose()
async def test_the_newest_backup_that_still_has_the_columns_is_the_one_read(db_320):
"""A 3.3.2 install has several backups; only some can answer.
`homelab.db.back-3.3.1` and `-3.3.2` were taken *after* the split and hold
nothing per node; `-3.3.0` was taken before it. Reading the wrong one would
either say nothing or resurrect a much older canvas.
"""
db_path, engine = db_320
await _build_320(engine)
async with engine.begin() as conn:
await conn.exec_driver_sql(
"UPDATE nodes SET services = "
"'[{\"port\": 22, \"protocol\": \"tcp\", \"service_name\": \"ssh\"}]' WHERE id = 'n1'"
)
# Ancient: same shape, but a canvas the user has long since moved on from.
old = db_path.parent / f"{db_path.name}.back-3.1.0"
shutil.copy2(db_path, old)
os.utime(old, (1, 1))
async with engine.begin() as conn:
await conn.exec_driver_sql(
"UPDATE nodes SET services = "
"'[{\"port\": 443, \"protocol\": \"tcp\", \"service_name\": \"https\"}]' WHERE id = 'n1'"
)
pre_split = db_path.parent / f"{db_path.name}.back-3.3.0"
shutil.copy2(db_path, pre_split)
await database.init_db() # drops the columns, and backs up under this version
# Post-split backups: newer, and unable to say who drew what.
for name in ("back-3.3.1", "back-3.3.2"):
shutil.copy2(db_path, db_path.parent / f"{db_path.name}.{name}")
assert database._pre_split_backup() == pre_split
assert database._views_from_backup()["n1"]["services"] == [
{"port": 443, "protocol": "tcp", "service_name": "https"}
]
await engine.dispose()
async def test_a_furniture_description_survives_the_column_drop(db_320, monkeypatch):
"""The rebuild copies a fixed column list — `description` has to be on it.
`_drop_legacy_node_columns` recreates `nodes` from `_NODE_COLUMNS_SQL` /
`_NODE_KEPT`. A column missing from those two strings is silently lost on the
boot that finally drops the legacy ones, which is exactly the boot a user who
wrote a zone description in the meantime would not expect to lose it.
"""
db_path, engine = db_320
await _build_320(engine)
real_link_facts = inventory_sync.link_facts
async def link_facts(db, node, facts, **kwargs):
if node.id == "n2":
raise IntegrityError("boom", None, Exception("UNIQUE constraint failed"))
return await real_link_facts(db, node, facts, **kwargs)
monkeypatch.setattr(inventory_sync, "link_facts", link_facts)
# First boot: the backfill cannot finish, so the legacy columns stay.
await database.init_db()
check = create_async_engine(f"sqlite+aiosqlite:///{db_path}")
try:
async with check.begin() as conn:
cols = {c[1] for c in (await conn.exec_driver_sql("PRAGMA table_info(nodes)")).fetchall()}
assert "description" in cols
assert "status" in cols # legacy still there
# The user describes a piece of furniture.
await conn.exec_driver_sql(
"UPDATE nodes SET description = 'Behind the garage door.' WHERE id = 'n3'"
)
# Second boot: the backfill succeeds and the legacy columns finally go.
monkeypatch.setattr(inventory_sync, "link_facts", real_link_facts)
await database.init_db()
async with check.begin() as conn:
cols = {c[1] for c in (await conn.exec_driver_sql("PRAGMA table_info(nodes)")).fetchall()}
assert "status" not in cols
assert "description" in cols
kept = (await conn.exec_driver_sql(
"SELECT description FROM nodes WHERE id = 'n3'"
)).scalar()
assert kept == "Behind the garage door."
finally:
await check.dispose()
await engine.dispose()
async def test_the_description_column_is_added_once_and_is_idempotent(db_320):
db_path, engine = db_320
await _build_320(engine)
await database.init_db()
await database.init_db()
check = create_async_engine(f"sqlite+aiosqlite:///{db_path}")
try:
async with check.begin() as conn:
info = (await conn.exec_driver_sql("PRAGMA table_info(nodes)")).fetchall()
assert [c[1] for c in info].count("description") == 1
finally:
await check.dispose()
await engine.dispose()
@@ -163,3 +163,78 @@ async def test_legacy_row_defaults_are_coerced(client: AsyncClient, headers, db_
assert row["status_live"] == "unknown"
assert row["show_hardware"] is False
assert row["label"] is None
@pytest.mark.asyncio
async def test_update_pending_edits_the_rack_faceplate(
client: AsyncClient, headers, db_session
):
"""The front panel is a device fact, so the inventory can edit it too.
Same row the rack canvas writes on save — a plate edited here is the plate
every rack mounting this device draws on its next load.
"""
device = await _seed(db_session, rack_faceplate_id="server-1u", rack_u_height=1)
res = await client.patch(
f"/api/v1/scan/pending/{device.id}",
headers=headers,
json={
"rack_faceplate_id": "server-2u-bays",
"rack_u_height": 2,
"rack_col_span": 12,
"rack_color": "#ff6e00",
"rack_ports": [
{"id": "p1", "label": "eth0", "type": "rj45", "x": 0.2, "y": 0.6},
# No id: nothing could ever cable it, so it never lands.
{"label": "ghost", "type": "rj45", "x": 0.5, "y": 0.5},
],
},
)
assert res.status_code == 200, res.text
body = res.json()
assert body["rack_faceplate_id"] == "server-2u-bays"
assert body["rack_u_height"] == 2
assert body["rack_color"] == "#ff6e00"
assert [p["id"] for p in body["rack_ports"]] == ["p1"]
@pytest.mark.asyncio
async def test_update_pending_leaves_an_unsent_faceplate_alone(
client: AsyncClient, headers, db_session
):
device = await _seed(db_session, rack_faceplate_id="switch-24", rack_u_height=1)
res = await client.patch(
f"/api/v1/scan/pending/{device.id}", headers=headers, json={"notes": "x"}
)
assert res.status_code == 200
assert res.json()["rack_faceplate_id"] == "switch-24"
@pytest.mark.asyncio
@pytest.mark.parametrize(
"patch",
[
{"rack_u_height": 0},
{"rack_u_height": 49},
{"rack_col_span": 0},
{"rack_col_span": 13},
],
)
async def test_update_pending_rejects_an_impossible_plate(
client: AsyncClient, headers, db_session, patch
):
device = await _seed(db_session)
res = await client.patch(
f"/api/v1/scan/pending/{device.id}", headers=headers, json=patch
)
assert res.status_code == 422
@pytest.mark.asyncio
async def test_inventory_row_reports_no_plate_before_it_is_modelled(
client: AsyncClient, headers, db_session
):
await _seed(db_session)
rows = (await client.get("/api/v1/scan/pending", headers=headers)).json()
assert rows[0]["rack_faceplate_id"] is None
assert rows[0]["rack_ports"] == []
+151
View File
@@ -414,3 +414,154 @@ async def test_delete_pending_refuses_a_mounted_device(client: AsyncClient, head
assert res.status_code == 409
listed = (await client.get("/api/v1/scan/pending", headers=headers)).json()
assert [d["id"] for d in listed] == [pending_device.id]
# ---------------------------------------------------------------------------
# Per-device deep rescan (issue #350)
# ---------------------------------------------------------------------------
@pytest.mark.asyncio
async def test_rescan_device_requires_auth(client: AsyncClient, pending_device):
res = await client.post(f"/api/v1/scan/pending/{pending_device.id}/rescan")
assert res.status_code == 401
@pytest.mark.asyncio
async def test_rescan_device_creates_run_for_that_ip(client: AsyncClient, headers, pending_device):
with patch("app.api.routes.scan._background_device_scan", new_callable=AsyncMock) as bg:
res = await client.post(
f"/api/v1/scan/pending/{pending_device.id}/rescan", headers=headers
)
assert res.status_code == 200, res.text
data = res.json()
assert data["status"] == "running"
assert data["kind"] == "device"
# One host, not the configured ranges — the scan targets this device only.
assert data["ranges"] == ["192.168.1.100/32"]
bg.assert_called_once()
assert bg.call_args.args[1] == pending_device.id
# Full range unless the caller says otherwise.
assert bg.call_args.args[3] is True
@pytest.mark.asyncio
async def test_rescan_device_passes_the_requested_port_range(
client: AsyncClient, headers, pending_device
):
"""The dialog's range reaches the scanner verbatim."""
with patch("app.api.routes.scan._background_device_scan", new_callable=AsyncMock) as bg:
res = await client.post(
f"/api/v1/scan/pending/{pending_device.id}/rescan",
headers=headers,
json={"ports": " 80,443,8000-9000 "},
)
assert res.status_code == 200, res.text
assert bg.call_args.args[4] == "80,443,8000-9000"
@pytest.mark.asyncio
async def test_rescan_device_rejects_an_unusable_port_range(
client: AsyncClient, headers, pending_device
):
"""A bad spec is refused here, not handed to nmap."""
for bad in ["0", "65536", "100-50", "80,", "http"]:
res = await client.post(
f"/api/v1/scan/pending/{pending_device.id}/rescan",
headers=headers,
json={"ports": bad},
)
assert res.status_code == 422, f"{bad}: {res.text}"
@pytest.mark.asyncio
async def test_rescan_device_treats_a_blank_range_as_the_full_sweep(
client: AsyncClient, headers, pending_device
):
with patch("app.api.routes.scan._background_device_scan", new_callable=AsyncMock) as bg:
res = await client.post(
f"/api/v1/scan/pending/{pending_device.id}/rescan",
headers=headers,
json={"ports": " "},
)
assert res.status_code == 200, res.text
assert bg.call_args.args[4] is None
assert bg.call_args.args[3] is True
@pytest.mark.asyncio
async def test_rescan_device_unknown_id_404(client: AsyncClient, headers):
res = await client.post(f"/api/v1/scan/pending/{uuid.uuid4()}/rescan", headers=headers)
assert res.status_code == 404
@pytest.mark.asyncio
async def test_rescan_device_without_ip_409(client: AsyncClient, headers, db_session):
device = InventoryDevice(id=str(uuid.uuid4()), ip=None, hostname="zigbee-lamp", status="pending")
db_session.add(device)
await db_session.commit()
res = await client.post(f"/api/v1/scan/pending/{device.id}/rescan", headers=headers)
assert res.status_code == 409
assert "IP" in res.json()["detail"]
@pytest.mark.asyncio
async def test_rescan_device_hidden_409(client: AsyncClient, headers, db_session):
device = InventoryDevice(id=str(uuid.uuid4()), ip="192.168.1.77", status="hidden")
db_session.add(device)
await db_session.commit()
res = await client.post(f"/api/v1/scan/pending/{device.id}/rescan", headers=headers)
assert res.status_code == 409
@pytest.mark.asyncio
async def test_rescan_device_serialized_per_device(client: AsyncClient, headers, pending_device):
"""A second rescan while the first still runs is refused, not duplicated."""
with patch("app.api.routes.scan._background_device_scan", new_callable=AsyncMock):
first = await client.post(
f"/api/v1/scan/pending/{pending_device.id}/rescan", headers=headers
)
second = await client.post(
f"/api/v1/scan/pending/{pending_device.id}/rescan", headers=headers
)
assert first.status_code == 200
assert second.status_code == 409
assert "already running" in second.json()["detail"]
@pytest.mark.asyncio
async def test_rescan_device_allows_a_new_run_once_the_first_finished(
client: AsyncClient, headers, pending_device
):
with patch("app.api.routes.scan._background_device_scan", new_callable=AsyncMock):
first = await client.post(
f"/api/v1/scan/pending/{pending_device.id}/rescan", headers=headers
)
assert first.status_code == 200
run_id = first.json()["id"]
stopped = await client.post(f"/api/v1/scan/{run_id}/stop", headers=headers)
assert stopped.status_code == 200
again = await client.post(
f"/api/v1/scan/pending/{pending_device.id}/rescan", headers=headers
)
assert again.status_code == 200
@pytest.mark.asyncio
async def test_get_run_returns_status(client: AsyncClient, headers, pending_device):
with patch("app.api.routes.scan._background_device_scan", new_callable=AsyncMock):
started = await client.post(
f"/api/v1/scan/pending/{pending_device.id}/rescan", headers=headers
)
run_id = started.json()["id"]
res = await client.get(f"/api/v1/scan/runs/{run_id}", headers=headers)
assert res.status_code == 200
assert res.json()["id"] == run_id
assert res.json()["status"] == "running"
@pytest.mark.asyncio
async def test_get_run_unknown_id_404(client: AsyncClient, headers):
res = await client.get(f"/api/v1/scan/runs/{uuid.uuid4()}", headers=headers)
assert res.status_code == 404
+141 -6
View File
@@ -8,13 +8,23 @@ from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.db.models import InventoryDevice, Node, ScanRun
from app.services.scanner import _cancelled_runs, request_cancel, run_scan
from app.services.scanner import (
_cancelled_runs,
reconcile_orphan_runs,
request_cancel,
run_scan,
)
@pytest.mark.asyncio
async def test_background_scan_marks_run_failed_on_exception(mem_db):
"""If run_scan() raises, the ScanRun must transition running → failed and the
session rollback path must execute without a follow-on exception."""
async def test_background_scan_marks_run_errored_on_exception(mem_db):
"""If run_scan() raises, the ScanRun must transition running → error and the
session rollback path must execute without a follow-on exception.
"error" and not "failed": one condition, one name — it is what run_scan
writes when it catches the failure itself, and the only one Scan History
filters and colours.
"""
from app.api.routes.scan import _background_scan
async with mem_db() as session:
@@ -36,13 +46,13 @@ async def test_background_scan_marks_run_failed_on_exception(mem_db):
async with mem_db() as session:
refreshed = await session.get(ScanRun, run_id)
assert refreshed is not None
assert refreshed.status == "failed"
assert refreshed.status == "error"
@pytest.mark.asyncio
async def test_background_scan_leaves_non_running_status_alone(mem_db):
"""If the run was already stopped/cancelled before run_scan failed, _background_scan
must NOT overwrite that terminal status with 'failed'."""
must NOT overwrite that terminal status with 'error'."""
from app.api.routes.scan import _background_scan
async with mem_db() as session:
@@ -436,3 +446,128 @@ async def test_run_scan_updates_existing_pending_device(db_session: AsyncSession
# Services and hostname should be updated
assert device.hostname == "myhost.lan"
assert any(s["port"] == 8096 for s in device.services)
@pytest.mark.asyncio
async def test_background_device_scan_marks_run_errored_on_exception(mem_db):
"""Same word as the network scan — a device run that blows up reads alike."""
from app.api.routes.scan import _background_device_scan
async with mem_db() as session:
run = ScanRun(status="running", kind="device", ranges=["10.0.0.5/32"])
session.add(run)
await session.commit()
run_id = run.id
with (
patch("app.api.routes.scan.AsyncSessionLocal", mem_db),
patch(
"app.api.routes.scan.run_device_scan",
new_callable=AsyncMock,
side_effect=RuntimeError("boom"),
),
):
await _background_device_scan(run_id, "d1")
async with mem_db() as session:
refreshed = await session.get(ScanRun, run_id)
assert refreshed is not None
assert refreshed.status == "error"
# --- reconcile_orphan_runs (issue #374) --------------------------------------
@pytest.mark.asyncio
async def test_reconcile_orphan_runs_marks_running_as_error(mem_db):
"""A run left "running" by a process that died mid-scan is immortal, and it
also locks its range out of future scans. Startup must reconcile it."""
async with mem_db() as session:
orphan = ScanRun(status="running", ranges=["10.0.0.0/24"])
session.add(orphan)
await session.commit()
orphan_id = orphan.id
assert await reconcile_orphan_runs(session) == 1
reconciled = await session.get(ScanRun, orphan_id)
# "error", not "failed" — one condition, one name, and the only status
# Scan History filters and colours.
assert reconciled.status == "error"
assert reconciled.finished_at is not None
assert "restarted" in reconciled.error
@pytest.mark.asyncio
async def test_reconcile_orphan_runs_leaves_finished_runs_alone(mem_db):
"""Only "running" is orphaned. Terminal rows must not be rewritten."""
async with mem_db() as session:
done = ScanRun(status="done", ranges=["10.0.0.0/24"], devices_found=7)
cancelled = ScanRun(status="cancelled", ranges=["10.0.1.0/24"])
errored = ScanRun(status="error", ranges=["10.0.2.0/24"], error="nmap missing")
session.add_all([done, cancelled, errored])
await session.commit()
ids = (done.id, cancelled.id, errored.id)
assert await reconcile_orphan_runs(session) == 0
for run_id, expected in zip(ids, ("done", "cancelled", "error"), strict=True):
assert (await session.get(ScanRun, run_id)).status == expected
# The pre-existing error message must survive untouched.
assert (await session.get(ScanRun, ids[2])).error == "nmap missing"
@pytest.mark.asyncio
async def test_reconcile_orphan_runs_handles_several_and_an_empty_table(mem_db):
async with mem_db() as session:
# Nothing to do on a fresh database.
assert await reconcile_orphan_runs(session) == 0
session.add_all([
ScanRun(status="running", ranges=["10.0.0.0/24"]),
ScanRun(status="running", kind="device", ranges=["10.0.0.5"]),
ScanRun(status="done", ranges=["10.0.1.0/24"]),
])
await session.commit()
assert await reconcile_orphan_runs(session) == 2
remaining = (
await session.execute(select(ScanRun).where(ScanRun.status == "running"))
).scalars().all()
assert remaining == []
@pytest.mark.asyncio
async def test_reconcile_orphan_runs_unblocks_a_new_scan(mem_db):
"""The point of the reconcile: the trigger endpoints reject a scan while one
is "running" for the same target, so an orphan blocks that range for good."""
async with mem_db() as session:
session.add(ScanRun(status="running", kind="device", ranges=["10.0.0.5"]))
await session.commit()
await reconcile_orphan_runs(session)
blocking = (
await session.execute(
select(ScanRun).where(
ScanRun.status == "running", ScanRun.kind == "device"
)
)
).scalars().all()
assert not any("10.0.0.5" in (r.ranges or []) for r in blocking)
@pytest.mark.asyncio
async def test_lifespan_reconciles_orphan_runs_at_startup():
"""The reconcile is worthless unless startup actually calls it."""
from app.main import app as fastapi_app
from app.main import lifespan
with patch("app.main.init_db", new=AsyncMock()), \
patch("app.main.start_scheduler"), \
patch("app.main.stop_scheduler"), \
patch("app.main.reconcile_orphan_runs", new=AsyncMock()) as reconcile:
async with lifespan(fastapi_app):
pass
reconcile.assert_awaited_once()
+112
View File
@@ -272,6 +272,40 @@ async def test_oidc_callback_creates_session_for_api(client: AsyncClient, oidc_s
assert protected.status_code == 200
async def test_oidc_callback_returns_to_the_app_base_path(client: AsyncClient, oidc_settings):
"""The SPA can be mounted under a prefix — the post-login redirect follows it."""
oidc_settings.oidc_redirect_uri = "http://test/homelab/api/v1/auth/oidc/callback"
fake_client = FakeOIDCClient(token={
"userinfo": {
"iss": "https://idp.example/application/o/homelable/",
"sub": "user-123",
"preferred_username": "alice",
},
})
with patch("app.api.routes.auth.get_oidc_client", return_value=fake_client):
callback = await client.get("/api/v1/auth/oidc/callback")
assert callback.status_code == 303
assert callback.headers["location"] == "/homelab/"
@pytest.mark.parametrize(
("redirect_uri", "expected"),
[
("", "/"),
("https://home.example/api/v1/auth/oidc/callback", "/"),
("https://home.example/homelab/api/v1/auth/oidc/callback", "/homelab/"),
("https://home.example/apps/lab/api/v1/auth/oidc/callback", "/apps/lab/"),
("/api/v1/auth/oidc/callback", "/"),
("https://home.example/somewhere/else", "/"),
],
)
def test_app_base_path_of(redirect_uri: str, expected: str):
from app.core.config import app_base_path_of
assert app_base_path_of(redirect_uri) == expected
@pytest.mark.parametrize("userinfo", [None, {}, {"sub": "user-123"}, {"iss": "https://idp.example/"}])
async def test_oidc_callback_rejects_missing_identity_claims(client: AsyncClient, oidc_settings, userinfo):
token = {} if userinfo is None else {"userinfo": userinfo}
@@ -336,6 +370,84 @@ async def test_oidc_cookie_mutations_require_csrf_and_allowed_origin(client: Asy
assert (await client.get("/api/v1/auth/me")).status_code == 401
async def test_oidc_mutation_allows_app_origin_missing_from_cors_origins(
client: AsyncClient, oidc_settings
):
"""Regression for #356: a same-origin reverse-proxy deploy needs no CORS, so
CORS_ORIGINS is left at its localhost default while the browser still sends
Origin — every POST used to 403."""
oidc_settings.cors_origins = ["http://localhost:5173", "http://localhost:3000"]
fake_client = FakeOIDCClient(token={
"userinfo": {
"iss": "https://idp.example/application/o/homelable/",
"sub": "user-123",
"preferred_username": "alice",
},
})
with patch("app.api.routes.auth.get_oidc_client", return_value=fake_client):
await client.get("/api/v1/auth/oidc/callback")
csrf_token = (await client.get("/api/v1/auth/me")).json()["csrf_token"]
created = await client.post(
"/api/v1/designs",
json={"name": "From OIDC", "design_type": "network"},
headers={"Origin": "http://test", "X-Homelable-CSRF": csrf_token},
)
assert created.status_code == 201
# The fallback widens the allowlist by exactly one origin, nothing more.
evil = await client.post(
"/api/v1/designs",
json={"name": "Nope", "design_type": "network"},
headers={"Origin": "https://evil.example", "X-Homelable-CSRF": csrf_token},
)
assert evil.status_code == 403
def test_origin_is_allowed_falls_back_to_the_oidc_redirect_origin(oidc_settings):
from app.core.security import origin_is_allowed
oidc_settings.cors_origins = ["http://localhost:3000"]
oidc_settings.oidc_redirect_uri = "https://homelable.example.com/api/v1/auth/oidc/callback"
assert origin_is_allowed("https://homelable.example.com") is True
assert origin_is_allowed("https://homelable.example.com/") is True
assert origin_is_allowed("http://localhost:3000") is True
assert origin_is_allowed("https://evil.example") is False
assert origin_is_allowed(None) is False
# A redirect URI that is not an absolute URL contributes no origin.
oidc_settings.oidc_redirect_uri = "/api/v1/auth/oidc/callback"
assert origin_is_allowed("https://homelable.example.com") is False
def test_oidc_settings_warn_when_cors_origins_omits_the_app_origin(caplog):
from app.core.config import Settings, origin_of
base = {
"secret_key": "x" * 32,
"auth_mode": "oidc",
"oidc_discovery_url": "https://idp.example/.well-known/openid-configuration",
"oidc_client_id": "homelable",
"oidc_client_secret": "secret",
"oidc_redirect_uri": "https://homelable.example.com/api/v1/auth/oidc/callback",
}
with caplog.at_level("WARNING"):
Settings(**base, cors_origins=["http://localhost:3000"])
# Asserted against the origin the settings derive rather than one spelled
# out again here: this checks the warning names that exact origin, and a
# URL literal on the left of an `in` reads to a scanner as a host check
# that a substring match would not make safely.
assert origin_of(base["oidc_redirect_uri"]) in caplog.text
assert "CORS_ORIGINS" in caplog.text
caplog.clear()
with caplog.at_level("WARNING"):
Settings(**base, cors_origins=["https://homelable.example.com/"])
assert caplog.text == ""
async def test_local_bearer_logout_does_not_require_csrf(client: AsyncClient, headers):
res = await client.post("/api/v1/auth/logout", headers=headers)
assert res.status_code == 204
+87
View File
@@ -706,3 +706,90 @@ async def test_save_canvas_custom_style_overwrite(client: AsyncClient, headers:
canvas = (await client.get("/api/v1/canvas", headers=headers)).json()
assert "proxmox" in canvas["custom_style"]["nodes"]
assert "server" not in canvas["custom_style"]["nodes"]
# ── self-parent guard (#370) ─────────────────────────────────────────────────
async def test_save_canvas_drops_a_node_parented_to_itself(client: AsyncClient, headers: dict):
# A self-parent row freezes the node on the canvas and survives a reload, so
# the save normalizes it away. Dropped rather than rejected: a canvas that
# already carries the bad row must still be able to save.
n = node_payload(type="lxc", label="pihole")
n["parent_id"] = n["id"]
res = await client.post(
"/api/v1/canvas/save", json={"nodes": [n], "edges": [], "viewport": {}}, headers=headers
)
assert res.status_code == 200
saved = (await client.get("/api/v1/canvas", headers=headers)).json()["nodes"]
assert len(saved) == 1
assert saved[0]["parent_id"] is None
async def test_save_canvas_keeps_a_real_parent(client: AsyncClient, headers: dict):
host = node_payload(type="proxmox", label="pve", container_mode=True)
child = node_payload(type="lxc", label="pihole", parent_id=host["id"])
res = await client.post(
"/api/v1/canvas/save",
json={"nodes": [host, child], "edges": [], "viewport": {}},
headers=headers,
)
assert res.status_code == 200
saved = {n["label"]: n for n in (await client.get("/api/v1/canvas", headers=headers)).json()["nodes"]}
assert saved["pihole"]["parent_id"] == host["id"]
# ── furniture descriptions ───────────────────────────────────────────────────
async def test_save_canvas_keeps_zone_and_group_descriptions(client: AsyncClient, headers: dict):
# Regression: a zone's and a group's description used to ride on `notes`,
# which the save routes to the inventory row. Furniture has no such row, so
# the text was dropped without an error. It lives on `nodes.description` now.
zone = node_payload(type="groupRect", label="Garage", description="Everything behind the garage door.")
group = node_payload(type="group", label="Cluster", description="The three Proxmox boxes.")
res = await client.post(
"/api/v1/canvas/save", json={"nodes": [zone, group], "edges": [], "viewport": {}}, headers=headers
)
assert res.status_code == 200
saved = {n["label"]: n for n in (await client.get("/api/v1/canvas", headers=headers)).json()["nodes"]}
assert saved["Garage"]["description"] == "Everything behind the garage door."
assert saved["Cluster"]["description"] == "The three Proxmox boxes."
async def test_save_canvas_edits_a_furniture_description(client: AsyncClient, headers: dict):
zone = node_payload(type="groupRect", label="Garage", description="First take.")
await client.post("/api/v1/canvas/save", json={"nodes": [zone], "edges": [], "viewport": {}}, headers=headers)
zone["description"] = "Second take."
await client.post("/api/v1/canvas/save", json={"nodes": [zone], "edges": [], "viewport": {}}, headers=headers)
saved = (await client.get("/api/v1/canvas", headers=headers)).json()["nodes"]
assert saved[0]["description"] == "Second take."
async def test_save_canvas_clears_a_furniture_description(client: AsyncClient, headers: dict):
zone = node_payload(type="groupRect", label="Garage", description="Written once.")
await client.post("/api/v1/canvas/save", json={"nodes": [zone], "edges": [], "viewport": {}}, headers=headers)
zone["description"] = ""
await client.post("/api/v1/canvas/save", json={"nodes": [zone], "edges": [], "viewport": {}}, headers=headers)
saved = (await client.get("/api/v1/canvas", headers=headers)).json()["nodes"]
assert saved[0]["description"] == ""
async def test_save_canvas_drops_a_description_on_a_device_node(client: AsyncClient, headers: dict):
# A node that draws a device keeps its text on the inventory row, as `notes`.
# `description` is furniture-only, so it is cleared rather than shadowing it.
n = node_payload(type="nas", label="nas-01", ip="192.168.1.20", notes="Backs up nightly.", description="Not here.")
await client.post("/api/v1/canvas/save", json={"nodes": [n], "edges": [], "viewport": {}}, headers=headers)
saved = (await client.get("/api/v1/canvas", headers=headers)).json()["nodes"][0]
assert saved["description"] is None
assert saved["notes"] == "Backs up nightly."
+405
View File
@@ -0,0 +1,405 @@
"""Tests for the inventory duplicate repair (app.services.device_merge).
Two shapes, one merge: the user picks the survivor by hand, or an import proves
two rows are one device by their shared MAC. What both must guarantee is that
nothing is lost — not a fact, not a canvas node, not a rack mount, not a
document, and not the *visibility* of the facts that just arrived on a canvas
that never saw them.
"""
from datetime import datetime, timezone
import pytest
from sqlalchemy import select
from app.db.models import (
Design,
Document,
InventoryDevice,
InventoryDeviceLink,
Node,
Rack,
RackDevice,
)
from app.services.device_merge import merge_devices, reconcile_duplicates
def _at(day: int) -> datetime:
return datetime(2026, 1, day, tzinfo=timezone.utc)
async def _design(db, name="d1"):
design = Design(name=name)
db.add(design)
await db.flush()
return design
async def _device(db, **kwargs):
kwargs.setdefault("discovered_at", _at(1))
device = InventoryDevice(**kwargs)
db.add(device)
await db.flush()
return device
async def _node(db, design, device, **kwargs):
node = Node(design_id=design.id, device_id=device.id, label=kwargs.pop("label", "n8n"),
type=kwargs.pop("type", "lxc"), **kwargs)
db.add(node)
await db.flush()
return node
async def _rack(db, design):
rack = Rack(design_id=design.id, name="R1", u_height=42)
db.add(rack)
await db.flush()
return rack
@pytest.mark.asyncio
async def test_merge_unions_facts_without_overwriting_the_survivor(db_session):
winner = await _device(
db_session, id="w", ip="192.168.1.62", mac="bc:24:11:95:af:8c",
hostname="n8n", label="n8n", vendor="Proxmox VE", status="pending",
services=[{"service_name": "ssh", "port": 22, "protocol": "tcp"}],
properties=[{"key": "VMID", "value": "130", "visible": True}],
discovery_source="arp", discovery_sources=["arp", "proxmox"],
discovered_at=_at(5),
)
loser = await _device(
db_session, id="l", ip="10.0.0.9", mac=None, hostname="ignored",
notes="runs the automations", cpu_count=4, status="approved",
services=[{"service_name": "http", "port": 5678, "protocol": "tcp"}],
properties=[{"key": "Owner", "value": "me", "visible": True}],
discovery_source="canvas", discovery_sources=["canvas"],
discovered_at=_at(2),
)
result = await merge_devices(db_session, winner, [loser])
await db_session.flush()
assert result["merged"] == 1
assert await db_session.get(InventoryDevice, "l") is None
# Established facts are never overwritten by the folded-in row...
assert winner.hostname == "n8n"
# ...but every gap it can fill, it fills.
assert winner.notes == "runs the automations"
assert winner.cpu_count == 4
# Addresses and lists union; sources accumulate.
assert winner.ip == "192.168.1.62, 10.0.0.9"
assert {s["service_name"] for s in winner.services} == {"ssh", "http"}
assert {p["key"] for p in winner.properties} == {"VMID", "Owner"}
assert set(winner.discovery_sources) == {"arp", "proxmox", "canvas"}
# Drawn on a canvas, so the survivor cannot go back to the pending queue.
assert winner.status == "approved"
# Known since the earliest of the two rows saw it.
assert winner.discovered_at == _at(2)
@pytest.mark.asyncio
async def test_merge_repoints_canvas_nodes_and_rack_mounts(db_session):
design = await _design(db_session)
other = await _design(db_session, name="d2")
winner = await _device(db_session, id="w", ip="192.168.1.62", status="approved")
loser = await _device(db_session, id="l", ip="192.168.1.62", label="n8n", status="approved")
kept = await _node(db_session, design, winner, pos_x=10, pos_y=20)
moved = await _node(db_session, other, loser, pos_x=300, pos_y=400)
rack = await _rack(db_session, design)
mount = RackDevice(design_id=design.id, rack_id=rack.id, device_id=loser.id, label="n8n")
db_session.add(mount)
await db_session.flush()
await merge_devices(db_session, winner, [loser])
await db_session.flush()
# The node on the other canvas keeps its position and now draws the survivor.
await db_session.refresh(moved)
assert moved.device_id == "w"
assert (moved.pos_x, moved.pos_y) == (300, 400)
assert (await db_session.get(Node, kept.id)).device_id == "w"
# The mount follows too — a rack that rendered before still renders.
await db_session.refresh(mount)
assert mount.device_id == "w"
@pytest.mark.asyncio
async def test_merge_shows_the_new_facts_on_a_canvas_that_never_saw_them(db_session):
"""The point of the merge: a poorer node must gain the richer row's facts.
`display_view` is a whitelist — without extending it the folded-in services
and properties would render hidden and the canvas would look unchanged.
"""
design = await _design(db_session)
winner = await _device(
db_session, id="w", ip="192.168.1.62",
services=[{"service_name": "ssh", "port": 22, "protocol": "tcp"}],
properties=[{"key": "VMID", "value": "130", "visible": True}],
)
loser = await _device(
db_session, id="l", ip="192.168.1.62",
services=[{"service_name": "http", "port": 5678, "protocol": "tcp"}],
properties=[],
)
poor = await _node(db_session, design, loser)
poor.display_view = {
"services": [{"key": "5678|tcp|http", "visible": True}],
"properties": [],
}
await db_session.flush()
await merge_devices(db_session, winner, [loser])
await db_session.flush()
await db_session.refresh(poor)
services = poor.display_view["services"]
assert [e["visible"] for e in services] == [True, True]
assert {e["key"] for e in services} == {"5678|tcp|http", "22|tcp|ssh"}
# A property list this canvas had never seen picks up the survivor's.
assert [e["key"] for e in poor.display_view["properties"]] == ["vmid"]
@pytest.mark.asyncio
async def test_merge_keeps_a_deliberately_hidden_fact_hidden(db_session):
design = await _design(db_session)
winner = await _device(
db_session, id="w", ip="192.168.1.62",
services=[{"service_name": "ssh", "port": 22, "protocol": "tcp"}],
)
loser = await _device(db_session, id="l", ip="192.168.1.62", services=[])
node = await _node(db_session, design, loser)
node.display_view = {"services": [{"key": "22|tcp|ssh", "visible": False}], "properties": []}
await db_session.flush()
await merge_devices(db_session, winner, [loser])
await db_session.flush()
await db_session.refresh(node)
assert node.display_view["services"] == [{"key": "22|tcp|ssh", "visible": False}]
@pytest.mark.asyncio
async def test_merge_collapses_two_nodes_that_now_draw_one_device(db_session):
"""Both rows drawn on the *same* canvas: one node survives, the oldest."""
design = await _design(db_session)
winner = await _device(db_session, id="w", ip="192.168.1.62", status="approved")
loser = await _device(db_session, id="l", ip="192.168.1.62", status="approved")
keep = await _node(db_session, design, winner, pos_x=10, pos_y=20)
await _node(db_session, design, loser, pos_x=99, pos_y=99)
await merge_devices(db_session, winner, [loser])
await db_session.flush()
nodes = (
await db_session.execute(select(Node).where(Node.design_id == design.id))
).scalars().all()
assert [n.id for n in nodes] == [keep.id]
assert (nodes[0].pos_x, nodes[0].pos_y) == (10, 20)
@pytest.mark.asyncio
async def test_a_rack_mount_survives_the_node_collapse(db_session):
"""`rack_devices.node_id` must not be left naming a deleted duplicate node."""
design = await _design(db_session)
winner = await _device(db_session, id="w", ip="192.168.1.62", status="approved")
loser = await _device(db_session, id="l", ip="192.168.1.62", status="approved")
keep = await _node(db_session, design, winner)
dup = await _node(db_session, design, loser)
rack = await _rack(db_session, design)
mount = RackDevice(
design_id=design.id, rack_id=rack.id, device_id=loser.id, node_id=dup.id, label="n8n"
)
db_session.add(mount)
await db_session.flush()
await merge_devices(db_session, winner, [loser])
await db_session.flush()
await db_session.refresh(mount)
assert mount.device_id == "w"
assert mount.node_id == keep.id
assert await db_session.get(Node, dup.id) is None
@pytest.mark.asyncio
async def test_merge_adopts_a_document_or_orphans_it(db_session):
winner = await _device(db_session, id="w", ip="192.168.1.62")
loser = await _device(db_session, id="l", ip="192.168.1.62")
doc = Document(title="n8n", slug="n8n", device_id=loser.id, body="# n8n")
db_session.add(doc)
await db_session.flush()
result = await merge_devices(db_session, winner, [loser])
await db_session.flush()
await db_session.refresh(doc)
# No document on the survivor, so the loser's is adopted rather than orphaned.
assert doc.device_id == "w"
assert result["documents_orphaned"] == 0
@pytest.mark.asyncio
async def test_a_second_document_is_orphaned_not_deleted(db_session):
winner = await _device(db_session, id="w", ip="192.168.1.62")
loser = await _device(db_session, id="l", ip="192.168.1.62")
mine = Document(title="n8n", slug="n8n", device_id=winner.id, body="kept")
theirs = Document(title="n8n (dup)", slug="n8n-dup", device_id=loser.id, body="written by hand")
db_session.add_all([mine, theirs])
await db_session.flush()
result = await merge_devices(db_session, winner, [loser])
await db_session.flush()
await db_session.refresh(theirs)
assert result["documents_orphaned"] == 1
assert theirs.device_id is None
assert theirs.body == "written by hand" # survives as an orphan
@pytest.mark.asyncio
async def test_merge_moves_mesh_links_onto_the_survivor_ieee(db_session):
winner = await _device(db_session, id="w", ieee_address="pve-pve1-130", ip="192.168.1.62")
loser = await _device(db_session, id="l", ieee_address="0xAABB", ip="192.168.1.62")
db_session.add(InventoryDeviceLink(source_ieee="pve-host", target_ieee="0xAABB"))
db_session.add(InventoryDeviceLink(source_ieee="0xAABB", target_ieee="0xCCDD"))
await db_session.flush()
await merge_devices(db_session, winner, [loser])
await db_session.flush()
links = (await db_session.execute(select(InventoryDeviceLink))).scalars().all()
pairs = {(link.source_ieee, link.target_ieee) for link in links}
assert pairs == {("pve-host", "pve-pve1-130"), ("pve-pve1-130", "0xCCDD")}
@pytest.mark.asyncio
async def test_the_survivor_adopts_an_ieee_it_lacks(db_session):
winner = await _device(db_session, id="w", ip="192.168.1.62", ieee_address=None)
loser = await _device(db_session, id="l", ip="192.168.1.62", ieee_address="pve-pve1-130")
await merge_devices(db_session, winner, [loser])
await db_session.flush()
assert winner.ieee_address == "pve-pve1-130"
@pytest.mark.asyncio
async def test_reconcile_merges_rows_sharing_a_mac(db_session):
"""The reported case: a Proxmox row and a canvas row, same NIC."""
scanned = await _device(
db_session, id="scanned", ip="192.168.1.62", mac="bc:24:11:95:af:8c",
ieee_address="pve-proxmox-ai-130", status="pending", discovered_at=_at(1),
)
from_canvas = await _device(
db_session, id="canvas", ip="192.168.1.62", mac="bc:24:11:95:af:8c",
label="n8n", status="approved", discovered_at=_at(20),
)
merged = await reconcile_duplicates(db_session)
await db_session.flush()
assert merged == 1
# The IEEE-bearing row wins: the Proxmox import and the link graph key on it.
assert await db_session.get(InventoryDevice, from_canvas.id) is None
assert (await db_session.get(InventoryDevice, scanned.id)).label == "n8n"
@pytest.mark.asyncio
async def test_reconcile_leaves_rows_that_only_share_an_ip(db_session):
"""An IP is not an identity — a re-used lease must not merge two machines."""
await _device(db_session, id="a", ip="192.168.1.62", mac="aa:aa:aa:aa:aa:aa")
await _device(db_session, id="b", ip="192.168.1.62", mac="bb:bb:bb:bb:bb:bb")
assert await reconcile_duplicates(db_session) == 0
assert len((await db_session.execute(select(InventoryDevice))).scalars().all()) == 2
@pytest.mark.asyncio
async def test_reconcile_leaves_two_proxmox_guests_sharing_a_mac(db_session):
"""Distinct synthetic IEEEs are decisive: two guests are never one device."""
await _device(db_session, id="a", mac="bc:24:11:95:af:8c", ieee_address="pve-pve1-130")
await _device(db_session, id="b", mac="bc:24:11:95:af:8c", ieee_address="pve-pve1-131")
assert await reconcile_duplicates(db_session) == 0
assert len((await db_session.execute(select(InventoryDevice))).scalars().all()) == 2
@pytest.mark.asyncio
async def test_reconcile_never_touches_a_hidden_row(db_session):
"""A hidden row was hidden on purpose — merging it would resurrect it."""
await _device(db_session, id="visible", mac="bc:24:11:95:af:8c", status="pending")
await _device(db_session, id="hidden", mac="bc:24:11:95:af:8c", status="hidden")
assert await reconcile_duplicates(db_session) == 0
hidden = await db_session.get(InventoryDevice, "hidden")
assert hidden is not None and hidden.status == "hidden"
@pytest.mark.asyncio
async def test_reconcile_is_idempotent(db_session):
await _device(db_session, id="a", mac="bc:24:11:95:af:8c", ip="192.168.1.62")
await _device(db_session, id="b", mac="bc:24:11:95:af:8c", ip="192.168.1.62")
assert await reconcile_duplicates(db_session) == 1
await db_session.flush()
assert await reconcile_duplicates(db_session) == 0
# --- The route -----------------------------------------------------------
@pytest.mark.asyncio
async def test_merge_route_returns_the_survivor(client, headers, db_session):
await _device(db_session, id="w", ip="192.168.1.62", label="n8n", status="approved")
await _device(db_session, id="l", hostname="n8n", notes="the good notes")
await db_session.commit()
res = await client.post(
"/api/v1/scan/pending/merge",
json={"winner_id": "w", "loser_ids": ["l"]},
headers=headers,
)
assert res.status_code == 200
body = res.json()
assert body["id"] == "w"
assert body["notes"] == "the good notes"
assert await db_session.get(InventoryDevice, "l") is None
@pytest.mark.asyncio
async def test_merge_route_rejects_a_single_device(client, headers, db_session):
await _device(db_session, id="w", ip="192.168.1.62")
await db_session.commit()
res = await client.post(
"/api/v1/scan/pending/merge",
json={"winner_id": "w", "loser_ids": ["w"]},
headers=headers,
)
assert res.status_code == 400
@pytest.mark.asyncio
async def test_merge_route_404s_on_an_unknown_row(client, headers, db_session):
await _device(db_session, id="w", ip="192.168.1.62")
await db_session.commit()
res = await client.post(
"/api/v1/scan/pending/merge",
json={"winner_id": "w", "loser_ids": ["nope"]},
headers=headers,
)
assert res.status_code == 404
@pytest.mark.asyncio
async def test_merge_route_requires_auth(client, db_session):
res = await client.post(
"/api/v1/scan/pending/merge", json={"winner_id": "w", "loser_ids": ["l"]}
)
assert res.status_code in (401, 403)
+254
View File
@@ -0,0 +1,254 @@
"""Backlinks — the inverse of a wiki-link.
Two halves. The pure resolution rules, which must stay identical to
`frontend/src/documentation/wikilinks.ts`, and the route, which has to find a
target document that carries no links of its own and must not pay for the
inventory when nothing links to a device.
"""
from dataclasses import dataclass
from httpx import AsyncClient
from app.services import doc_backlinks
@dataclass
class FakeDoc:
id: str
slug: str = ""
title: str = ""
body: str = ""
device_id: str | None = None
node_id: str | None = None
@dataclass
class FakeDevice:
id: str
label: str | None = None
friendly_name: str | None = None
hostname: str | None = None
ip: str | None = None
# ── parsing ─────────────────────────────────────────────────────────────────
def test_a_bare_link_is_a_document_link():
link = doc_backlinks.parse_link("VLAN plan")
assert link is not None
assert (link.target, link.key, link.label) == ("doc", "VLAN plan", "VLAN plan")
def test_a_prefix_picks_the_target():
assert doc_backlinks.parse_link("device:nas-01").target == "device"
assert doc_backlinks.parse_link("NODE:abc").target == "node"
assert doc_backlinks.parse_link("doc:vlan-plan").target == "doc"
def test_an_unknown_prefix_stays_part_of_the_key():
link = doc_backlinks.parse_link("http://nas.lan")
assert link is not None
assert (link.target, link.key) == ("doc", "http://nas.lan")
def test_a_pipe_sets_the_label():
link = doc_backlinks.parse_link("device:nas-01|the big NAS")
assert link is not None
assert (link.key, link.label) == ("nas-01", "the big NAS")
def test_an_empty_link_is_not_a_link():
assert doc_backlinks.parse_link("") is None
assert doc_backlinks.parse_link("device:") is None
def test_links_are_collected_with_their_line():
body = "intro\nsee [[a]] and [[b]]\n"
found = list(doc_backlinks.iter_links(body))
assert [link.key for _, link in found] == ["a", "b"]
assert {line for line, _ in found} == {"see [[a]] and [[b]]"}
# ── resolution ──────────────────────────────────────────────────────────────
def _resolve(inner: str, docs: list[FakeDoc], devices: list[FakeDevice] | None = None):
link = doc_backlinks.parse_link(inner)
assert link is not None
return doc_backlinks.resolve(link, docs, devices or [])
def test_a_document_resolves_by_id_then_slug_then_title():
docs = [
FakeDoc(id="d1", slug="vlan-plan", title="VLAN plan"),
FakeDoc(id="d2", slug="other", title="Other"),
]
assert _resolve("d1", docs) == "d1"
assert _resolve("vlan-plan", docs) == "d1"
assert _resolve("VLAN PLAN", docs) == "d1"
def test_an_unmatched_document_link_resolves_to_nothing():
assert _resolve("nowhere", [FakeDoc(id="d1", slug="s", title="t")]) is None
def test_a_device_link_resolves_by_id_or_label():
docs = [FakeDoc(id="d1", device_id="dev1")]
devices = [FakeDevice(id="dev1", label="nas-01")]
assert _resolve("device:dev1", docs, devices) == "d1"
assert _resolve("device:NAS-01", docs, devices) == "d1"
def test_a_device_label_falls_back_the_way_the_tree_does():
assert doc_backlinks.device_label(FakeDevice(id="x", hostname="h")) == "h"
assert doc_backlinks.device_label(FakeDevice(id="x", ip="10.0.0.1")) == "10.0.0.1"
assert doc_backlinks.device_label(FakeDevice(id="x")) == "Unnamed device"
def test_a_device_with_no_document_resolves_to_nothing():
assert _resolve("device:dev1", [], [FakeDevice(id="dev1", label="nas")]) is None
def test_a_node_link_resolves_through_node_id():
docs = [FakeDoc(id="d1", node_id="n1")]
assert _resolve("node:n1", docs) == "d1"
assert _resolve("node:n2", docs) is None
# ── inversion ───────────────────────────────────────────────────────────────
def test_a_document_that_links_here_is_a_backlink():
docs = [
FakeDoc(id="d1", slug="a", title="A", body="see [[B]]"),
FakeDoc(id="d2", slug="b", title="B", body="no links"),
]
hits = doc_backlinks.backlinks_for("d2", docs, [])
assert [h.doc_id for h in hits] == ["d1"]
assert hits[0].context == "see [[B]]"
def test_a_document_does_not_link_to_itself():
docs = [FakeDoc(id="d1", slug="a", title="A", body="[[A]] again")]
assert doc_backlinks.backlinks_for("d1", docs, []) == []
def test_two_links_from_one_document_are_one_backlink_with_a_count():
docs = [
FakeDoc(id="d1", slug="a", title="A", body="[[B]] and later [[b]]"),
FakeDoc(id="d2", slug="b", title="B"),
]
hits = doc_backlinks.backlinks_for("d2", docs, [])
assert len(hits) == 1
assert hits[0].count == 2
def test_the_written_label_is_kept():
docs = [
FakeDoc(id="d1", slug="a", title="A", body="[[B|the other one]]"),
FakeDoc(id="d2", slug="b", title="B"),
]
assert doc_backlinks.backlinks_for("d2", docs, [])[0].label == "the other one"
def test_a_long_line_is_windowed_around_the_link():
filler = "word " * 60
docs = [
FakeDoc(id="d1", slug="a", title="A", body=f"{filler}[[B]]{filler}"),
FakeDoc(id="d2", slug="b", title="B"),
]
context = doc_backlinks.backlinks_for("d2", docs, [])[0].context
assert "[[B]]" in context
assert len(context) < 200
def test_has_device_link_only_fires_on_a_device_link():
assert doc_backlinks.has_device_link([FakeDoc(id="d", body="[[device:x]]")])
assert not doc_backlinks.has_device_link([FakeDoc(id="d", body="[[plain]]")])
# ── route ───────────────────────────────────────────────────────────────────
async def test_backlinks_requires_auth(client: AsyncClient):
assert (await client.get("/api/v1/documents/x/backlinks")).status_code == 401
async def test_backlinks_404_on_an_unknown_document(client: AsyncClient, headers: dict):
res = await client.get("/api/v1/documents/nope/backlinks", headers=headers)
assert res.status_code == 404
async def test_the_route_finds_a_target_that_carries_no_links(client: AsyncClient, headers: dict):
"""The target's own body has no `[[`, so it is only ever a resolution target."""
target = (
await client.post("/api/v1/documents", json={"title": "VLAN plan"}, headers=headers)
).json()
source = (
await client.post("/api/v1/documents", json={"title": "Runbook"}, headers=headers)
).json()
await client.patch(
f"/api/v1/documents/{source['id']}",
json={"body": "Read the [[VLAN plan]] first."},
headers=headers,
)
res = await client.get(f"/api/v1/documents/{target['id']}/backlinks", headers=headers)
assert res.status_code == 200
hits = res.json()
assert [h["doc_id"] for h in hits] == [source["id"]]
assert hits[0]["title"] == "Runbook"
assert hits[0]["label"] == "VLAN plan"
assert hits[0]["context"] == "Read the [[VLAN plan]] first."
async def test_a_device_document_is_reachable_by_its_device_link(client: AsyncClient, headers: dict):
res = await client.post(
"/api/v1/scan/pending",
json={
"label": "nas-01",
"hostname": "nas-01.lan",
"ip": "192.168.1.20",
"discovery_source": "manual",
},
headers=headers,
)
assert res.status_code in (200, 201), res.text
device = res.json()
doc = (
await client.post(
"/api/v1/documents",
json={"title": "nas-01", "kind": "device", "device_id": device["id"]},
headers=headers,
)
).json()
source = (
await client.post("/api/v1/documents", json={"title": "Backups"}, headers=headers)
).json()
await client.patch(
f"/api/v1/documents/{source['id']}",
json={"body": "Runs on [[device:nas-01]]."},
headers=headers,
)
hits = (await client.get(f"/api/v1/documents/{doc['id']}/backlinks", headers=headers)).json()
assert [h["doc_id"] for h in hits] == [source["id"]]
assert hits[0]["device_id"] is None # the *source* is a page, not a device
async def test_an_unresolved_link_is_not_a_backlink(client: AsyncClient, headers: dict):
target = (
await client.post("/api/v1/documents", json={"title": "Target"}, headers=headers)
).json()
source = (
await client.post("/api/v1/documents", json={"title": "Source"}, headers=headers)
).json()
await client.patch(
f"/api/v1/documents/{source['id']}",
json={"body": "Points at [[Something else]]."},
headers=headers,
)
hits = (await client.get(f"/api/v1/documents/{target['id']}/backlinks", headers=headers)).json()
assert hits == []
+172
View File
@@ -0,0 +1,172 @@
"""Search over documents, on both engines.
FTS5 is not guaranteed by the SQLite build the LXC and Docker images ship, so
the LIKE fallback is a supported path, not a safety net — it gets the same
coverage as the index.
"""
from sqlalchemy import text
from sqlalchemy.ext.asyncio import AsyncSession
from app.db.models import Document
from app.services import doc_search
def _doc(**kwargs) -> Document:
base = dict(kind="page", title="T", slug="t", body="", tags=[])
base.update(kwargs)
return Document(**base)
async def _seed(db: AsyncSession) -> list[Document]:
docs = [
_doc(title="NAS", slug="nas", body="The Synology holds every backup.", tags=["storage"]),
_doc(title="Router", slug="router", body="Runs OPNsense on 192.168.1.1.", tags=["network"]),
_doc(title="VLAN plan", slug="vlan-plan", body="Guest traffic is isolated.", tags=["network"]),
]
for doc in docs:
db.add(doc)
await db.flush()
await doc_search.index_document(db, doc)
await db.flush()
return docs
# ── the MATCH expression ────────────────────────────────────────────────────
def test_match_query_quotes_every_term_and_prefixes_it():
assert doc_search.build_match_query("nas backup") == '"nas"* AND "backup"*'
def test_match_query_neutralises_punctuation_a_user_types():
# An IP and a lone quote are data, not FTS5 syntax.
assert doc_search.build_match_query("192.168.1.1") == '"192.168.1.1"*'
assert doc_search.build_match_query('a"b') == '"a""b"*'
def test_match_query_of_whitespace_is_empty():
assert doc_search.build_match_query(" ") == ""
def test_tags_text_flattens_a_list():
assert doc_search.tags_text(["a", "b"]) == "a b"
assert doc_search.tags_text(None) == ""
# ── FTS5 ────────────────────────────────────────────────────────────────────
async def test_fts_is_available_when_the_index_table_exists(db_session: AsyncSession):
assert await doc_search.fts_available(db_session) is True
async def test_search_finds_a_word_that_is_only_in_a_body(db_session: AsyncSession):
await _seed(db_session)
engine, hits = await doc_search.search(db_session, "Synology")
assert engine == "fts5"
assert len(hits) == 1
assert "Synology" in hits[0]["snippet"]
async def test_search_matches_a_prefix(db_session: AsyncSession):
await _seed(db_session)
_, hits = await doc_search.search(db_session, "OPNsen")
assert len(hits) == 1
async def test_search_matches_a_title(db_session: AsyncSession):
await _seed(db_session)
_, hits = await doc_search.search(db_session, "VLAN")
assert len(hits) == 1
async def test_search_matches_a_tag(db_session: AsyncSession):
await _seed(db_session)
_, hits = await doc_search.search(db_session, "network")
assert len(hits) == 2
async def test_search_ands_its_terms(db_session: AsyncSession):
await _seed(db_session)
_, hits = await doc_search.search(db_session, "guest traffic")
assert len(hits) == 1
_, none = await doc_search.search(db_session, "guest Synology")
assert none == []
async def test_search_of_an_empty_query_returns_nothing(db_session: AsyncSession):
await _seed(db_session)
_, hits = await doc_search.search(db_session, " ")
assert hits == []
async def test_reindexing_replaces_the_previous_body(db_session: AsyncSession):
docs = await _seed(db_session)
docs[0].body = "Replaced entirely."
await doc_search.index_document(db_session, docs[0])
_, gone = await doc_search.search(db_session, "Synology")
assert gone == []
_, found = await doc_search.search(db_session, "Replaced")
assert len(found) == 1
async def test_unindexing_removes_the_document(db_session: AsyncSession):
docs = await _seed(db_session)
await doc_search.unindex_document(db_session, docs[0].id)
_, hits = await doc_search.search(db_session, "Synology")
assert hits == []
async def test_the_limit_is_honoured(db_session: AsyncSession):
for i in range(8):
doc = _doc(title=f"D{i}", slug=f"d{i}", body="shared word here")
db_session.add(doc)
await db_session.flush()
await doc_search.index_document(db_session, doc)
_, hits = await doc_search.search(db_session, "shared", limit=3)
assert len(hits) == 3
# ── the LIKE fallback ───────────────────────────────────────────────────────
async def test_search_falls_back_to_like_without_fts5(db_session: AsyncSession):
await _seed(db_session)
await db_session.execute(text("DROP TABLE documents_fts"))
doc_search.reset_availability_cache()
engine, hits = await doc_search.search(db_session, "Synology")
assert engine == "like"
assert len(hits) == 1
assert "Synology" in hits[0]["snippet"]
async def test_indexing_is_a_no_op_without_fts5(db_session: AsyncSession):
await db_session.execute(text("DROP TABLE documents_fts"))
doc_search.reset_availability_cache()
doc = _doc(title="NAS", slug="nas", body="body")
db_session.add(doc)
await db_session.flush()
# Must not raise — writes carry on, only search degrades.
await doc_search.index_document(db_session, doc)
await doc_search.unindex_document(db_session, doc.id)
async def test_like_fallback_is_case_insensitive(db_session: AsyncSession):
await _seed(db_session)
await db_session.execute(text("DROP TABLE documents_fts"))
doc_search.reset_availability_cache()
_, hits = await doc_search.search(db_session, "synology")
assert len(hits) == 1
def test_excerpt_centres_on_the_match():
body = "x" * 200 + "needle" + "y" * 200
excerpt = doc_search._excerpt(body, "needle")
assert "needle" in excerpt
assert excerpt.startswith("…") and excerpt.endswith("…")
def test_excerpt_falls_back_to_the_head_when_there_is_no_match():
assert doc_search._excerpt("short body", "absent") == "short body"
+392
View File
@@ -0,0 +1,392 @@
"""The generated skeleton. Written once, then owned by the user — so what it
emits on day one is the whole contract."""
from datetime import date, datetime, timezone
from types import SimpleNamespace
from app.services import doc_template as t
def _device(**overrides):
base = dict(
id="dev-1",
label="nas-01",
friendly_name=None,
hostname="nas-01.lan",
ip="192.168.1.20",
mac="00:11:32:aa:bb:cc",
ieee_address=None,
os="DSM 7.2",
vendor="Synology",
model="DS920+",
type="nas",
suggested_type=None,
services=[
{"port": 5000, "protocol": "tcp", "service_name": "Synology DSM", "category": "management"},
{"port": 445, "protocol": "tcp", "service_name": "SMB"},
],
properties=[{"key": "serial", "value": "20C0PDN", "icon": None, "visible": True}],
cpu_count=4,
cpu_model="Intel Celeron J4125",
ram_gb=20.0,
disk_gb=32000.0,
check_method="http",
check_target="http://192.168.1.20:5000",
discovery_source="arp",
discovery_sources=["arp", "mdns"],
discovered_at=datetime(2026, 3, 4, tzinfo=timezone.utc),
notes="",
)
base.update(overrides)
return SimpleNamespace(**base)
# ── cells and names ─────────────────────────────────────────────────────────
def test_cell_escapes_pipes_and_collapses_newlines():
assert t.cell("a|b") == r"a\|b"
assert t.cell("one\ntwo") == "one two"
assert t.cell(None) == "—"
assert t.cell("") == "—"
def test_cell_escapes_backslash_before_pipe():
# Backslash first, or the escape of the pipe would itself be escaped.
assert t.cell(r"a\|b") == r"a\\\|b"
def test_device_name_falls_back_through_label_hostname_ip():
assert t.device_name(_device()) == "nas-01"
assert t.device_name(_device(label=None)) == "nas-01.lan"
assert t.device_name(_device(label=None, hostname=None)) == "192.168.1.20"
assert t.device_name(_device(label=None, hostname=None, ip=None)) == "device"
def test_device_type_prefers_curated_over_guessed():
assert t.device_type(_device(type="nas", suggested_type="server")) == "nas"
assert t.device_type(_device(type=None, suggested_type="server")) == "server"
# ── blocks ──────────────────────────────────────────────────────────────────
def test_device_info_block_reads_every_address():
md = t.block_device_info(_device())
assert "| Hostname | nas-01.lan |" in md
assert "| IP | 192.168.1.20 |" in md
assert "| Vendor / Model | Synology / DS920+ |" in md
assert "`http` → `http://192.168.1.20:5000`" in md
assert "(arp, mdns)" in md
def test_device_info_block_prints_dashes_for_missing_facts():
md = t.block_device_info(_device(hostname=None, mac=None, os=None, vendor=None, model=None))
assert "| Hostname | — |" in md
assert "| Vendor / Model | — |" in md
def test_hardware_block_joins_cpu_model_and_count():
assert "Intel Celeron J4125 ×4" in t.block_hardware(_device())
assert "20 GB" in t.block_hardware(_device())
def test_hardware_block_handles_a_count_without_a_model():
assert "4 cores" in t.block_hardware(_device(cpu_model=None))
# ── hardware comes out of `properties` first ────────────────────────────────
# What the Proxmox import and the scanner actually write.
_PROXMOX_PROPERTIES = [
{"key": "VMID", "value": "108"},
{"key": "Kind", "value": "LXC"},
{"key": "CPU Cores", "value": "4"},
{"key": "RAM", "value": "6.1 GB"},
{"key": "Disk", "value": "50.0 GB"},
]
def _from_properties(**overrides):
"""A row as the importers leave it: hardware in properties, columns empty."""
return _device(
**{
"properties": _PROXMOX_PROPERTIES,
"cpu_count": None,
"cpu_model": None,
"ram_gb": None,
"disk_gb": None,
**overrides,
}
)
def test_hardware_block_reads_the_properties_when_the_columns_are_empty():
"""The regression: an empty hardware table beside a full properties one.
`cpu_count` / `ram_gb` / `disk_gb` are barely written any more — the same
facts arrive as properties — so the table printed three em dashes next to a
properties table listing all three.
"""
md = t.block_hardware(_from_properties())
assert "| 4 cores | 6.1 GB | 50.0 GB |" in md
assert "—" not in md
def test_hardware_block_takes_a_property_value_verbatim():
# "6.1 GB" is already formatted; appending a unit would say "6.1 GB GB", and
# guessing one onto a bare number would be inventing a fact.
md = t.block_hardware(_from_properties(properties=[{"key": "RAM", "value": "12288 MB"}]))
assert "| 12288 MB |" in md
def test_hardware_block_matches_property_keys_whatever_their_case():
md = t.block_hardware(_from_properties(properties=[{"key": "ram", "value": "8 GB"}]))
assert "8 GB" in md
def test_hardware_block_accepts_the_common_synonyms():
md = t.block_hardware(
_from_properties(
properties=[
{"key": "Processor", "value": "Xeon E3-1220"},
{"key": "vCPU", "value": "8"},
{"key": "Memory", "value": "64 GB"},
{"key": "Storage", "value": "4 TB"},
]
)
)
assert "| Xeon E3-1220 ×8 | 64 GB | 4 TB |" in md
def test_hardware_block_prefers_a_property_over_the_legacy_column():
md = t.block_hardware(_device(properties=[{"key": "RAM", "value": "64 GB"}]))
assert "64 GB" in md
assert "20 GB" not in md
def test_hardware_block_still_falls_back_to_the_columns():
# A row that predates the move, or one typed into the inventory modal.
md = t.block_hardware(_device(properties=[]))
assert "Intel Celeron J4125 ×4" in md
assert "20 GB" in md
def test_hardware_block_is_empty_when_neither_source_has_anything():
md = t.block_hardware(
_device(properties=[], cpu_count=None, cpu_model=None, ram_gb=None, disk_gb=None)
)
assert "| — | — | — |" in md
def test_hardware_block_ignores_a_property_with_no_value():
md = t.block_hardware(_from_properties(properties=[{"key": "RAM", "value": ""}]))
assert "| — | — | — |" in md
def test_property_map_keeps_the_first_of_a_repeated_key():
device = _device(properties=[{"key": "RAM", "value": "first"}, {"key": "RAM", "value": "second"}])
assert t.property_map(device)["ram"] == "first"
# ── the snapshot follows what the tables print ──────────────────────────────
def test_the_snapshot_carries_the_properties():
snapshot = t.facts_snapshot(_from_properties())
assert snapshot["properties"]["RAM"] == "6.1 GB"
def test_editing_a_property_is_drift():
before = t.facts_snapshot(_from_properties())
after = t.facts_snapshot(
_from_properties(properties=[*_PROXMOX_PROPERTIES[:3], {"key": "RAM", "value": "12 GB"}])
)
assert before != after
def test_the_snapshot_survives_its_json_round_trip():
"""It is stored as JSON and compared against a freshly computed one.
A tuple or a set would come back a list and every document would read as
drifted forever, so the shape has to be JSON-native on both sides.
"""
import json
snapshot = t.facts_snapshot(_from_properties())
assert json.loads(json.dumps(snapshot)) == snapshot
def test_services_block_writes_one_section_per_service():
md = t.block_services(_device())
assert "### Synology DSM — `5000/tcp`" in md
assert "### SMB — `445/tcp`" in md
assert "**Category** — management" in md
# Prompts, not values — the user fills these in.
assert md.count("**Purpose**") == 2
def test_services_block_says_so_when_nothing_was_fingerprinted():
assert "No service" in t.block_services(_device(services=[]))
def test_services_block_never_invents_a_url_for_a_non_http_port():
md = t.block_services(_device(services=[{"port": 445, "protocol": "tcp", "service_name": "SMB"}]))
assert "**URL**" not in md
def test_services_block_builds_an_http_url_from_the_device_ip():
assert "<http://192.168.1.20:5000>" in t.block_services(_device())
def test_properties_block_is_empty_when_the_device_has_none():
assert t.block_properties(_device(properties=[])) == ""
assert "| serial | 20C0PDN |" in t.block_properties(_device())
def test_rack_block_prints_the_span_of_units():
md = t.block_rack(_device(), rack={"name": "Rack-A", "u_start": 12, "u_height": 3, "col_span": 12})
assert md == "Rack **Rack-A**, U12–U14, full width."
def test_rack_block_collapses_a_single_unit():
md = t.block_rack(_device(), rack={"name": "Rack-A", "u_start": 7, "u_height": 1, "col_span": 6})
assert "U7," in md and "–" not in md
assert "6/12 wide" in md
def test_rack_block_is_empty_when_the_device_is_neither_racked_nor_zoned():
assert t.block_rack(_device()) == ""
assert t.block_rack(_device(), zone_label="Garage") == "Zone **Garage**."
def test_network_block_guesses_the_slash_24_and_lists_peers():
md = t.block_network(_device(), connections=["switch-core", "router"])
assert "192.168.1.0/24" in md
assert "`switch-core`, `router`" in md
def test_network_block_takes_the_first_ip_of_a_multi_ip_device():
assert "10.0.0.0/24" in t.block_network(_device(ip="10.0.0.5, 192.168.1.20"))
def test_network_block_survives_a_device_with_no_ip():
assert "**Subnet / VLAN** — _…_" in t.block_network(_device(ip=None))
def test_render_block_rejects_an_unknown_name():
try:
t.render_block("nope", _device())
except ValueError as exc:
assert "Unknown block" in str(exc)
else:
raise AssertionError("expected ValueError")
# ── the whole document ──────────────────────────────────────────────────────
def test_device_document_carries_every_section():
md = t.render_device_document(_device(), today=date(2026, 9, 5))
for heading in (
"## Device Information",
"### Hardware",
"## Services",
"## Network",
"## Configuration",
"## Operations",
"## Troubleshooting",
"## Dependencies",
"## Changelog",
"## Notes",
):
assert heading in md, heading
def test_device_document_opens_with_versioned_frontmatter():
md = t.render_device_document(_device(), today=date(2026, 9, 5))
assert md.startswith("---\n")
assert f"homelable: {t.FRONTMATTER_VERSION}" in md
assert "device: dev-1" in md
assert "created: 2026-09-05" in md
assert "review_every: 6m" in md
assert "tags: []" in md
def test_library_document_advertises_tags_too():
assert "tags: []" in t.render_library_document(t.TEMPLATE_RUNBOOK, "Reboot the NAS")
def test_device_document_appends_old_notes_verbatim_with_provenance():
notes = "Runs the backups.\n\n Indented line kept as typed | with a pipe."
md = t.render_device_document(_device(notes=notes), today=date(2026, 9, 5))
assert "<!-- Migrated verbatim from this device's Notes field on 2026-09-05. -->" in md
assert notes.strip() in md
def test_device_document_prompts_when_there_were_no_notes():
md = t.render_device_document(_device(notes=""))
assert "Migrated verbatim" not in md
assert "_Anything worth remembering about this device._" in md
def test_device_document_drops_the_location_section_when_there_is_no_place():
assert "### Physical location" not in t.render_device_document(_device())
md = t.render_device_document(_device(), zone_label="Garage")
assert "### Physical location" in md
def test_device_document_drops_the_properties_section_when_there_are_none():
assert "### Properties" not in t.render_device_document(_device(properties=[]))
def test_device_document_of_a_bare_device_still_renders():
bare = SimpleNamespace(id="d", label=None, notes=None)
md = t.render_device_document(bare)
assert "# device" in md
assert "## Services" in md
# ── library templates ───────────────────────────────────────────────────────
def test_runbook_template_has_its_headings():
md = t.render_library_document(t.TEMPLATE_RUNBOOK, "Reboot the NAS", today=date(2026, 9, 5))
assert "# Reboot the NAS" in md
assert "## Rollback" in md
assert "template: runbook" in md
def test_blank_template_is_just_a_title():
md = t.render_library_document(t.TEMPLATE_BLANK, "Scratch")
assert "# Scratch" in md
assert "##" not in md
def test_unknown_template_falls_back_to_blank_rather_than_failing():
md = t.render_library_document("does-not-exist", "Scratch")
assert "# Scratch" in md
def test_frontmatter_omits_empty_scalars_but_prints_an_empty_list():
# `tags: []` is how a user learns tags exist at all, so an empty list stays.
rendered = t.render_frontmatter({"a": 1, "b": None, "c": "", "d": [], "e": ["x", "y"]})
assert rendered == "---\na: 1\nd: []\ne: [x, y]\n---"
# ── the drift snapshot ──────────────────────────────────────────────────────
def test_facts_snapshot_keys_services_by_port_protocol_name():
snapshot = t.facts_snapshot(_device())
assert snapshot["services"] == ["445|tcp|smb", "5000|tcp|synology dsm"]
def test_facts_snapshot_is_stable_when_services_are_reordered():
a = t.facts_snapshot(_device())
b = t.facts_snapshot(_device(services=list(reversed(_device().services))))
assert a == b
def test_facts_snapshot_changes_when_the_ip_moves():
assert t.facts_snapshot(_device()) != t.facts_snapshot(_device(ip="192.168.1.21"))
+153
View File
@@ -0,0 +1,153 @@
"""Slugs, frontmatter parsing, and the two tree guards."""
from sqlalchemy.ext.asyncio import AsyncSession
from app.db.models import Document
from app.services import doc_tree
def _doc(**kwargs) -> Document:
base = dict(kind="folder", title="T", slug="t", body="")
base.update(kwargs)
return Document(**base)
# ── slugs ───────────────────────────────────────────────────────────────────
def test_slugify_lowercases_and_collapses_punctuation():
assert doc_tree.slugify("NAS-01 / main storage!") == "nas-01-main-storage"
def test_slugify_of_nothing_is_untitled():
assert doc_tree.slugify("") == "untitled"
assert doc_tree.slugify(" ") == "untitled"
assert doc_tree.slugify("!!!") == "untitled"
def test_slugify_truncates_a_long_title():
assert len(doc_tree.slugify("x" * 200)) == 80
async def test_unique_slug_suffixes_a_taken_sibling(db_session: AsyncSession):
db_session.add(_doc(title="Network", slug="network"))
await db_session.flush()
assert await doc_tree.unique_slug(db_session, "Network", parent_id=None) == "network-2"
async def test_unique_slug_keeps_counting_past_the_second(db_session: AsyncSession):
db_session.add(_doc(slug="network"))
db_session.add(_doc(slug="network-2"))
await db_session.flush()
assert await doc_tree.unique_slug(db_session, "Network", parent_id=None) == "network-3"
async def test_unique_slug_only_looks_at_siblings(db_session: AsyncSession):
parent = _doc(title="Folder", slug="folder")
db_session.add(parent)
await db_session.flush()
db_session.add(_doc(slug="network", parent_id=parent.id))
await db_session.flush()
# Same slug, different parent — no clash.
assert await doc_tree.unique_slug(db_session, "Network", parent_id=None) == "network"
async def test_unique_slug_ignores_the_document_being_renamed(db_session: AsyncSession):
doc = _doc(title="Network", slug="network")
db_session.add(doc)
await db_session.flush()
assert await doc_tree.unique_slug(db_session, "Network", parent_id=None, exclude_id=doc.id) == "network"
# ── frontmatter ─────────────────────────────────────────────────────────────
def test_parse_frontmatter_reads_the_leading_block():
body = "---\ntitle: NAS\ntags: [a, b]\n---\n\n# NAS\n"
assert doc_tree.parse_frontmatter(body) == {"title": "NAS", "tags": ["a", "b"]}
def test_parse_frontmatter_ignores_a_block_that_is_not_first():
assert doc_tree.parse_frontmatter("# Title\n\n---\ntitle: NAS\n---\n") == {}
def test_parse_frontmatter_of_malformed_yaml_is_empty_not_an_error():
# The user is mid-edit; the cache goes stale, the save does not fail.
assert doc_tree.parse_frontmatter("---\ntitle: [unclosed\n---\nbody") == {}
def test_parse_frontmatter_of_a_non_mapping_is_empty():
assert doc_tree.parse_frontmatter("---\n- a\n- b\n---\n") == {}
def test_parse_frontmatter_of_an_empty_body_is_empty():
assert doc_tree.parse_frontmatter("") == {}
def test_parse_frontmatter_keeps_a_trailing_space_on_the_fence():
assert doc_tree.parse_frontmatter("--- \ntitle: NAS\n--- \n") == {"title": "NAS"}
def test_parse_frontmatter_of_an_unclosed_block_does_not_go_quadratic():
"""A body that opens a block and never closes it must still be linear.
The shape a document has while the block is being typed, and the input the
scanner named: `---\n` then many repetitions of `\n `. With `\s*` around the
fences the closing alternative was reachable two ways and each added line
multiplied the backtracking.
"""
import time
body = "---\n" + "\n " * 40_000
started = time.perf_counter()
assert doc_tree.parse_frontmatter(body) == {}
assert time.perf_counter() - started < 1.0
def test_tags_accepts_a_list_or_a_comma_string():
assert doc_tree.tags_from({"tags": ["a", " b "]}) == ["a", "b"]
assert doc_tree.tags_from({"tags": "a, b"}) == ["a", "b"]
assert doc_tree.tags_from({"tags": 7}) == []
assert doc_tree.tags_from({}) == []
# ── tree guards ─────────────────────────────────────────────────────────────
async def _chain(db_session: AsyncSession, depth: int) -> list[Document]:
docs: list[Document] = []
parent_id = None
for i in range(depth):
doc = _doc(title=f"L{i}", slug=f"l{i}", parent_id=parent_id)
db_session.add(doc)
await db_session.flush()
docs.append(doc)
parent_id = doc.id
return docs
async def test_is_ancestor_walks_the_whole_chain(db_session: AsyncSession):
a, b, c = await _chain(db_session, 3)
assert await doc_tree.is_ancestor(db_session, a.id, c.id)
assert not await doc_tree.is_ancestor(db_session, c.id, a.id)
async def test_a_document_is_its_own_ancestor(db_session: AsyncSession):
(a,) = await _chain(db_session, 1)
assert await doc_tree.is_ancestor(db_session, a.id, a.id)
async def test_subtree_ids_collects_every_descendant(db_session: AsyncSession):
a, b, c = await _chain(db_session, 3)
sibling = _doc(title="S", slug="s", parent_id=a.id)
db_session.add(sibling)
await db_session.flush()
ids = await doc_tree.subtree_ids(db_session, a.id)
assert set(ids) == {a.id, b.id, c.id, sibling.id}
# Root first, so a caller deleting in reverse removes leaves first.
assert ids[0] == a.id
async def test_subtree_ids_of_a_leaf_is_just_itself(db_session: AsyncSession):
(a,) = await _chain(db_session, 1)
assert await doc_tree.subtree_ids(db_session, a.id) == [a.id]
+774
View File
@@ -0,0 +1,774 @@
"""The documents API.
Covers what the plan promises the user: a document per device generated once,
a Library tree that cannot be knotted, history that survives a restore, and a
document that outlives the device it describes.
"""
import uuid
from httpx import AsyncClient
async def _device(client: AsyncClient, headers: dict, **body) -> dict:
payload = {"label": "nas-01", "hostname": "nas-01.lan", "ip": "192.168.1.20", "discovery_source": "manual", **body}
res = await client.post("/api/v1/scan/pending", json=payload, headers=headers)
assert res.status_code in (200, 201), res.text
return res.json()
async def _create(client: AsyncClient, headers: dict, **body) -> dict:
res = await client.post("/api/v1/documents", json={"title": "Page", **body}, headers=headers)
assert res.status_code == 201, res.text
return res.json()
async def _design(client: AsyncClient, headers: dict) -> str:
res = await client.post("/api/v1/designs", json={"name": "D"}, headers=headers)
assert res.status_code == 201, res.text
return res.json()["id"]
# ── auth ────────────────────────────────────────────────────────────────────
async def test_list_requires_auth(client: AsyncClient):
assert (await client.get("/api/v1/documents")).status_code == 401
async def test_create_requires_auth(client: AsyncClient):
assert (await client.post("/api/v1/documents", json={"title": "X"})).status_code == 401
async def test_search_requires_auth(client: AsyncClient):
assert (await client.get("/api/v1/documents/search?q=x")).status_code == 401
async def test_coverage_requires_auth(client: AsyncClient):
assert (await client.get("/api/v1/documents/coverage")).status_code == 401
async def test_delete_requires_auth(client: AsyncClient):
assert (await client.delete("/api/v1/documents/x")).status_code == 401
# ── create ──────────────────────────────────────────────────────────────────
async def test_create_a_blank_page(client: AsyncClient, headers: dict):
doc = await _create(client, headers, title="VLAN plan")
assert doc["kind"] == "page"
assert doc["slug"] == "vlan-plan"
assert "# VLAN plan" in doc["body"]
async def test_create_a_page_from_a_template(client: AsyncClient, headers: dict):
doc = await _create(client, headers, title="Reboot the NAS", template_id="runbook")
assert "## Rollback" in doc["body"]
assert doc["template_id"] == "runbook"
async def test_create_a_page_with_an_explicit_body(client: AsyncClient, headers: dict):
doc = await _create(client, headers, title="Raw", body="---\ntags: [a]\n---\n\nhello")
assert doc["body"].endswith("hello")
# The frontmatter cache is derived from the body, never sent on its own.
assert doc["frontmatter"] == {"tags": ["a"]}
assert doc["tags"] == ["a"]
async def test_create_a_folder(client: AsyncClient, headers: dict):
folder = await _create(client, headers, title="Runbooks", kind="folder")
child = await _create(client, headers, title="Reboot", parent_id=folder["id"])
assert child["parent_id"] == folder["id"]
async def test_create_rejects_an_unknown_kind(client: AsyncClient, headers: dict):
res = await client.post("/api/v1/documents", json={"title": "X", "kind": "wat"}, headers=headers)
assert res.status_code == 422
async def test_create_rejects_an_unknown_template(client: AsyncClient, headers: dict):
res = await client.post("/api/v1/documents", json={"title": "X", "template_id": "wat"}, headers=headers)
assert res.status_code == 422
async def test_create_rejects_a_missing_parent(client: AsyncClient, headers: dict):
res = await client.post(
"/api/v1/documents", json={"title": "X", "parent_id": str(uuid.uuid4())}, headers=headers
)
assert res.status_code == 404
async def test_create_rejects_a_page_that_also_names_a_device(client: AsyncClient, headers: dict):
device = await _device(client, headers)
res = await client.post(
"/api/v1/documents", json={"title": "X", "kind": "page", "device_id": device["id"]}, headers=headers
)
assert res.status_code == 400
async def test_create_rejects_a_device_document_naming_nothing(client: AsyncClient, headers: dict):
res = await client.post("/api/v1/documents", json={"title": "X", "kind": "device"}, headers=headers)
assert res.status_code == 400
async def test_create_rejects_two_links_at_once(client: AsyncClient, headers: dict):
device = await _device(client, headers)
res = await client.post(
"/api/v1/documents",
json={"title": "X", "kind": "device", "device_id": device["id"], "design_id": await _design(client, headers)},
headers=headers,
)
assert res.status_code == 400
async def test_create_rejects_a_device_that_does_not_exist(client: AsyncClient, headers: dict):
res = await client.post(
"/api/v1/documents",
json={"title": "X", "kind": "device", "device_id": str(uuid.uuid4())},
headers=headers,
)
assert res.status_code == 404
async def test_a_device_gets_at_most_one_document(client: AsyncClient, headers: dict):
device = await _device(client, headers)
body = {"title": "nas", "kind": "device", "device_id": device["id"]}
assert (await client.post("/api/v1/documents", json=body, headers=headers)).status_code == 201
second = await client.post("/api/v1/documents", json=body, headers=headers)
assert second.status_code == 409
async def test_a_sibling_title_gets_a_distinct_slug(client: AsyncClient, headers: dict):
first = await _create(client, headers, title="Network")
second = await _create(client, headers, title="Network")
assert first["slug"] == "network"
assert second["slug"] == "network-2"
# ── the generated device document ───────────────────────────────────────────
async def test_a_device_document_is_scaffolded_from_the_live_facts(client: AsyncClient, headers: dict):
device = await _device(
client, headers, hostname="nas-01.lan", vendor="Synology", notes="Holds the backups."
)
doc = await _create(client, headers, title="nas-01", kind="device", device_id=device["id"])
assert "| Hostname | nas-01.lan |" in doc["body"]
assert "| IP | 192.168.1.20 |" in doc["body"]
assert "Holds the backups." in doc["body"]
assert doc["facts_snapshot"]["ip"] == "192.168.1.20"
assert doc["facts_synced_at"] is not None
async def test_the_old_notes_column_is_left_untouched_by_scaffolding(client: AsyncClient, headers: dict):
device = await _device(client, headers, notes="Holds the backups.")
await _create(client, headers, title="nas-01", kind="device", device_id=device["id"])
res = await client.get("/api/v1/scan/pending", headers=headers)
row = next(d for d in res.json() if d["id"] == device["id"])
assert row["notes"] == "Holds the backups."
async def test_a_device_document_records_its_zone_and_neighbours(client: AsyncClient, headers: dict):
design_id = await _design(client, headers)
device = await _device(client, headers)
zone = await client.post(
"/api/v1/nodes",
json={"type": "groupRect", "label": "Garage", "design_id": design_id, "pos_x": 0, "pos_y": 0},
headers=headers,
)
zone_id = zone.json()["id"]
# A node binds to its inventory row by its addresses, not by an id on the
# payload — the canvas is one more way to document hardware, not a store.
node = await client.post(
"/api/v1/nodes",
json={
"type": "nas",
"label": "nas-01",
"design_id": design_id,
"ip": "192.168.1.20",
"hostname": "nas-01.lan",
"parent_id": zone_id,
"pos_x": 0,
"pos_y": 0,
},
headers=headers,
)
assert node.json()["device_id"] == device["id"], node.text
peer = await client.post(
"/api/v1/nodes",
json={"type": "switch", "label": "switch-core", "design_id": design_id, "pos_x": 0, "pos_y": 0},
headers=headers,
)
await client.post(
"/api/v1/edges",
json={"source": node.json()["id"], "target": peer.json()["id"], "type": "ethernet"},
headers=headers,
)
doc = await _create(client, headers, title="nas-01", kind="device", device_id=device["id"])
assert "Zone **Garage**." in doc["body"]
assert "`switch-core`" in doc["body"]
# ── blocks ──────────────────────────────────────────────────────────────────
async def test_a_block_can_be_regenerated_on_demand(client: AsyncClient, headers: dict):
device = await _device(client, headers, hostname="nas-01.lan")
res = await client.get(
f"/api/v1/documents/blocks?block=device-info&device_id={device['id']}", headers=headers
)
assert res.status_code == 200, res.text
assert "| Hostname | nas-01.lan |" in res.json()["markdown"]
async def test_an_unknown_block_is_rejected(client: AsyncClient, headers: dict):
device = await _device(client, headers)
res = await client.get(f"/api/v1/documents/blocks?block=nope&device_id={device['id']}", headers=headers)
assert res.status_code == 400
async def test_a_block_for_a_missing_device_is_404(client: AsyncClient, headers: dict):
res = await client.get(
f"/api/v1/documents/blocks?block=device-info&device_id={uuid.uuid4()}", headers=headers
)
assert res.status_code == 404
# ── read / list ─────────────────────────────────────────────────────────────
async def test_get_a_missing_document_is_404(client: AsyncClient, headers: dict):
assert (await client.get(f"/api/v1/documents/{uuid.uuid4()}", headers=headers)).status_code == 404
async def test_listing_carries_no_body(client: AsyncClient, headers: dict):
await _create(client, headers, title="VLAN plan")
row = (await client.get("/api/v1/documents", headers=headers)).json()[0]
assert "body" not in row
async def test_listing_filters_by_kind_and_parent(client: AsyncClient, headers: dict):
folder = await _create(client, headers, title="Runbooks", kind="folder")
await _create(client, headers, title="Reboot", parent_id=folder["id"])
await _create(client, headers, title="Loose")
folders = (await client.get("/api/v1/documents?kind=folder", headers=headers)).json()
assert [d["title"] for d in folders] == ["Runbooks"]
children = (await client.get(f"/api/v1/documents?parent_id={folder['id']}", headers=headers)).json()
assert [d["title"] for d in children] == ["Reboot"]
async def test_listing_filters_by_tag(client: AsyncClient, headers: dict):
await _create(client, headers, title="A", body="---\ntags: [Network]\n---\n")
await _create(client, headers, title="B", body="---\ntags: [storage]\n---\n")
hits = (await client.get("/api/v1/documents?tag=network", headers=headers)).json()
assert [d["title"] for d in hits] == ["A"]
# ── update ──────────────────────────────────────────────────────────────────
async def test_editing_the_body_refreshes_the_frontmatter_cache(client: AsyncClient, headers: dict):
doc = await _create(client, headers, title="Page")
res = await client.patch(
f"/api/v1/documents/{doc['id']}", json={"body": "---\ntags: [x]\n---\n\nnew"}, headers=headers
)
assert res.status_code == 200, res.text
assert res.json()["tags"] == ["x"]
async def test_the_body_renames_the_document(client: AsyncClient, headers: dict):
"""`title:` in the frontmatter is the only place a document is named."""
doc = await _create(client, headers, title="Page")
res = await client.patch(
f"/api/v1/documents/{doc['id']}",
json={"body": "---\ntitle: SMB / CIFS\n---\n\n# SMB / CIFS\n"},
headers=headers,
)
assert res.status_code == 200, res.text
assert res.json()["title"] == "SMB / CIFS"
assert res.json()["slug"] == "smb-cifs"
async def test_a_body_without_a_title_keeps_the_one_it_has(client: AsyncClient, headers: dict):
doc = await _create(client, headers, title="Page")
res = await client.patch(
f"/api/v1/documents/{doc['id']}", json={"body": "---\ntags: [x]\n---\n\nnew"}, headers=headers
)
assert res.json()["title"] == "Page"
async def test_a_blank_title_in_the_body_is_ignored(client: AsyncClient, headers: dict):
doc = await _create(client, headers, title="Page")
res = await client.patch(
f"/api/v1/documents/{doc['id']}", json={"body": "---\ntitle: ' '\n---\n\nnew"}, headers=headers
)
assert res.json()["title"] == "Page"
async def test_an_explicit_title_wins_over_the_body_in_the_same_request(client: AsyncClient, headers: dict):
doc = await _create(client, headers, title="Page")
res = await client.patch(
f"/api/v1/documents/{doc['id']}",
json={"title": "Chosen", "body": "---\ntitle: From the body\n---\n\nnew"},
headers=headers,
)
assert res.json()["title"] == "Chosen"
async def test_a_device_document_renamed_in_its_body_keeps_its_device(client: AsyncClient, headers: dict):
device = await _device(client, headers)
doc = await _create(client, headers, title="nas-01", kind="device", device_id=device["id"])
res = await client.patch(
f"/api/v1/documents/{doc['id']}",
json={"body": "---\ntitle: The big NAS\n---\n\n# The big NAS\n"},
headers=headers,
)
assert res.json()["title"] == "The big NAS"
assert res.json()["device_id"] == device["id"]
async def test_renaming_reslugs(client: AsyncClient, headers: dict):
doc = await _create(client, headers, title="Page")
res = await client.patch(f"/api/v1/documents/{doc['id']}", json={"title": "New name"}, headers=headers)
assert res.json()["slug"] == "new-name"
async def test_starring_and_reviewing(client: AsyncClient, headers: dict):
doc = await _create(client, headers, title="Page")
res = await client.patch(
f"/api/v1/documents/{doc['id']}", json={"starred": True, "reviewed": True}, headers=headers
)
assert res.json()["starred"] is True
assert res.json()["reviewed_at"] is not None
async def test_patching_a_missing_document_is_404(client: AsyncClient, headers: dict):
res = await client.patch(f"/api/v1/documents/{uuid.uuid4()}", json={"title": "X"}, headers=headers)
assert res.status_code == 404
async def test_a_folder_cannot_be_moved_inside_itself(client: AsyncClient, headers: dict):
outer = await _create(client, headers, title="Outer", kind="folder")
inner = await _create(client, headers, title="Inner", kind="folder", parent_id=outer["id"])
res = await client.patch(f"/api/v1/documents/{outer['id']}", json={"parent_id": inner["id"]}, headers=headers)
assert res.status_code == 400
async def test_a_document_can_only_be_filed_under_a_folder(client: AsyncClient, headers: dict):
page = await _create(client, headers, title="Page")
other = await _create(client, headers, title="Other")
res = await client.patch(f"/api/v1/documents/{other['id']}", json={"parent_id": page["id"]}, headers=headers)
assert res.status_code == 400
async def test_a_device_document_does_not_live_in_the_library_tree(client: AsyncClient, headers: dict):
device = await _device(client, headers)
doc = await _create(client, headers, title="nas", kind="device", device_id=device["id"])
folder = await _create(client, headers, title="F", kind="folder")
res = await client.patch(f"/api/v1/documents/{doc['id']}", json={"parent_id": folder["id"]}, headers=headers)
assert res.status_code == 400
async def test_resyncing_facts_clears_the_drift_without_touching_the_body(client: AsyncClient, headers: dict):
device = await _device(client, headers)
doc = await _create(client, headers, title="nas", kind="device", device_id=device["id"])
await client.patch(f"/api/v1/scan/pending/{device['id']}", json={"ip": "192.168.1.99"}, headers=headers)
res = await client.patch(f"/api/v1/documents/{doc['id']}", json={"resync_facts": True}, headers=headers)
assert res.json()["facts_snapshot"]["ip"] == "192.168.1.99"
# The header the user owns still says what it always said.
assert "192.168.1.20" in res.json()["body"]
# ── drift ───────────────────────────────────────────────────────────────────
async def test_a_fresh_device_document_has_not_drifted(client: AsyncClient, headers: dict):
device = await _device(client, headers)
doc = await _create(client, headers, title="nas-01", kind="device", device_id=device["id"])
assert doc["drifted"] is False
read = (await client.get(f"/api/v1/documents/{doc['id']}", headers=headers)).json()
assert read["drifted"] is False
async def test_changing_the_device_marks_the_document_drifted(client: AsyncClient, headers: dict):
device = await _device(client, headers)
doc = await _create(client, headers, title="nas-01", kind="device", device_id=device["id"])
await client.patch(
f"/api/v1/scan/pending/{device['id']}", json={"ip": "192.168.1.99"}, headers=headers
)
read = (await client.get(f"/api/v1/documents/{doc['id']}", headers=headers)).json()
assert read["drifted"] is True
async def test_a_property_edit_alone_marks_the_document_drifted(client: AsyncClient, headers: dict):
device = await _device(client, headers)
doc = await _create(client, headers, title="nas-01", kind="device", device_id=device["id"])
await client.patch(
f"/api/v1/scan/pending/{device['id']}",
json={"properties": [{"key": "Rack", "value": "A1"}]},
headers=headers,
)
read = (await client.get(f"/api/v1/documents/{doc['id']}", headers=headers)).json()
assert read["drifted"] is True
async def test_editing_the_body_does_not_make_a_document_drift(client: AsyncClient, headers: dict):
device = await _device(client, headers)
doc = await _create(client, headers, title="nas-01", kind="device", device_id=device["id"])
res = await client.patch(f"/api/v1/documents/{doc['id']}", json={"body": "my words"}, headers=headers)
assert res.json()["drifted"] is False
async def test_accepting_the_facts_clears_the_drift(client: AsyncClient, headers: dict):
device = await _device(client, headers)
doc = await _create(client, headers, title="nas-01", kind="device", device_id=device["id"])
await client.patch(
f"/api/v1/scan/pending/{device['id']}", json={"ip": "192.168.1.99"}, headers=headers
)
res = await client.patch(
f"/api/v1/documents/{doc['id']}", json={"resync_facts": True}, headers=headers
)
assert res.json()["drifted"] is False
async def test_regenerating_clears_the_drift(client: AsyncClient, headers: dict):
device = await _device(client, headers)
doc = await _create(client, headers, title="nas-01", kind="device", device_id=device["id"])
await client.patch(
f"/api/v1/scan/pending/{device['id']}", json={"ip": "192.168.1.99"}, headers=headers
)
res = await client.post(f"/api/v1/documents/{doc['id']}/regenerate", headers=headers)
assert res.json()["drifted"] is False
async def test_the_listing_carries_the_drift_flag_for_the_tree_badge(
client: AsyncClient, headers: dict
):
fresh = await _device(client, headers, label="switch-01", ip="192.168.1.2")
stale = await _device(client, headers, label="nas-01", ip="192.168.1.20")
in_sync = await _create(client, headers, title="switch-01", kind="device", device_id=fresh["id"])
moved = await _create(client, headers, title="nas-01", kind="device", device_id=stale["id"])
await client.patch(
f"/api/v1/scan/pending/{stale['id']}", json={"ip": "192.168.1.99"}, headers=headers
)
listing = (await client.get("/api/v1/documents", headers=headers)).json()
by_id = {d["id"]: d for d in listing}
assert by_id[moved["id"]]["drifted"] is True
assert by_id[in_sync["id"]]["drifted"] is False
async def test_a_library_page_never_drifts(client: AsyncClient, headers: dict):
doc = await _create(client, headers, title="VLAN plan")
assert doc["drifted"] is False
# ── revisions ───────────────────────────────────────────────────────────────
async def test_an_edit_records_the_previous_body(client: AsyncClient, headers: dict):
doc = await _create(client, headers, title="Page", body="first")
await client.patch(f"/api/v1/documents/{doc['id']}", json={"body": "second"}, headers=headers)
revisions = (await client.get(f"/api/v1/documents/{doc['id']}/revisions", headers=headers)).json()
assert len(revisions) == 1
assert revisions[0]["reason"] == "edit"
stored = (await client.get(f"/api/v1/documents/revisions/{revisions[0]['id']}", headers=headers)).json()
assert stored["body"] == "first"
async def test_saving_an_unchanged_body_records_nothing(client: AsyncClient, headers: dict):
doc = await _create(client, headers, title="Page", body="same")
await client.patch(f"/api/v1/documents/{doc['id']}", json={"body": "same"}, headers=headers)
assert (await client.get(f"/api/v1/documents/{doc['id']}/revisions", headers=headers)).json() == []
async def test_restoring_brings_back_an_old_body_and_is_itself_undoable(client: AsyncClient, headers: dict):
doc = await _create(client, headers, title="Page", body="first")
await client.patch(f"/api/v1/documents/{doc['id']}", json={"body": "second"}, headers=headers)
revision = (await client.get(f"/api/v1/documents/{doc['id']}/revisions", headers=headers)).json()[0]
res = await client.post(
f"/api/v1/documents/{doc['id']}/revisions/{revision['id']}/restore", headers=headers
)
assert res.status_code == 200, res.text
assert res.json()["body"] == "first"
history = (await client.get(f"/api/v1/documents/{doc['id']}/revisions", headers=headers)).json()
assert [r["reason"] for r in history][0] == "restore"
async def test_restoring_brings_back_the_title_that_body_carried(client: AsyncClient, headers: dict):
doc = await _create(client, headers, title="Page", body="---\ntitle: First name\n---\n\nfirst")
await client.patch(
f"/api/v1/documents/{doc['id']}",
json={"body": "---\ntitle: Second name\n---\n\nsecond"},
headers=headers,
)
revision = (await client.get(f"/api/v1/documents/{doc['id']}/revisions", headers=headers)).json()[0]
res = await client.post(
f"/api/v1/documents/{doc['id']}/revisions/{revision['id']}/restore", headers=headers
)
assert res.json()["title"] == "First name"
async def test_restoring_a_revision_of_another_document_is_404(client: AsyncClient, headers: dict):
a = await _create(client, headers, title="A", body="one")
b = await _create(client, headers, title="B", body="one")
await client.patch(f"/api/v1/documents/{a['id']}", json={"body": "two"}, headers=headers)
revision = (await client.get(f"/api/v1/documents/{a['id']}/revisions", headers=headers)).json()[0]
res = await client.post(f"/api/v1/documents/{b['id']}/revisions/{revision['id']}/restore", headers=headers)
assert res.status_code == 404
async def test_history_is_pruned_to_the_limit(client: AsyncClient, headers: dict):
from app.services.doc_tree import REVISION_LIMIT
doc = await _create(client, headers, title="Page", body="v0")
for i in range(1, REVISION_LIMIT + 6):
await client.patch(f"/api/v1/documents/{doc['id']}", json={"body": f"v{i}"}, headers=headers)
revisions = (await client.get(f"/api/v1/documents/{doc['id']}/revisions", headers=headers)).json()
assert len(revisions) == REVISION_LIMIT
# ── regenerate ──────────────────────────────────────────────────────────────
async def test_regenerate_requires_auth(client: AsyncClient):
assert (await client.post("/api/v1/documents/x/regenerate")).status_code == 401
async def test_regenerating_a_device_document_rebuilds_it_from_the_facts(
client: AsyncClient, headers: dict
):
device = await _device(client, headers)
doc = await _create(client, headers, title="nas-01", kind="device", device_id=device["id"])
await client.patch(f"/api/v1/documents/{doc['id']}", json={"body": "everything I wrote"}, headers=headers)
res = await client.post(f"/api/v1/documents/{doc['id']}/regenerate", headers=headers)
assert res.status_code == 200, res.text
body = res.json()
assert "everything I wrote" not in body["body"]
assert "192.168.1.20" in body["body"]
# A regenerated document is only the template again, as it was on day one.
assert body["edited_at"] is None
assert body["template_id"] == "device"
async def test_regenerating_reads_the_devices_current_facts(client: AsyncClient, headers: dict):
device = await _device(client, headers)
doc = await _create(client, headers, title="nas-01", kind="device", device_id=device["id"])
await client.patch(
f"/api/v1/scan/pending/{device['id']}", json={"ip": "192.168.1.99"}, headers=headers
)
res = await client.post(f"/api/v1/documents/{doc['id']}/regenerate", headers=headers)
assert "192.168.1.99" in res.json()["body"]
# Regenerating documents the device as it is now, so the drift is gone.
assert res.json()["facts_snapshot"]["ip"] == "192.168.1.99"
async def test_regenerating_keeps_the_replaced_body_in_the_history(client: AsyncClient, headers: dict):
doc = await _create(client, headers, title="VLAN plan", body="my own words")
await client.post(f"/api/v1/documents/{doc['id']}/regenerate", headers=headers)
revisions = (await client.get(f"/api/v1/documents/{doc['id']}/revisions", headers=headers)).json()
assert revisions[0]["reason"] == "regenerate"
stored = (await client.get(f"/api/v1/documents/revisions/{revisions[0]['id']}", headers=headers)).json()
assert stored["body"] == "my own words"
async def test_regenerating_a_library_page_uses_its_template(client: AsyncClient, headers: dict):
doc = await _create(client, headers, title="Restart the NAS", template_id="runbook")
await client.patch(f"/api/v1/documents/{doc['id']}", json={"body": "gone"}, headers=headers)
res = await client.post(f"/api/v1/documents/{doc['id']}/regenerate", headers=headers)
assert res.status_code == 200, res.text
assert "gone" not in res.json()["body"]
assert "# Restart the NAS" in res.json()["body"]
assert res.json()["template_id"] == "runbook"
async def test_regenerating_a_folder_is_rejected(client: AsyncClient, headers: dict):
folder = await _create(client, headers, title="Runbooks", kind="folder")
res = await client.post(f"/api/v1/documents/{folder['id']}/regenerate", headers=headers)
assert res.status_code == 400
async def test_regenerating_an_unknown_document_is_404(client: AsyncClient, headers: dict):
res = await client.post(f"/api/v1/documents/{uuid.uuid4()}/regenerate", headers=headers)
assert res.status_code == 404
# ── delete ──────────────────────────────────────────────────────────────────
async def test_deleting_a_folder_takes_its_subtree(client: AsyncClient, headers: dict):
outer = await _create(client, headers, title="Outer", kind="folder")
inner = await _create(client, headers, title="Inner", kind="folder", parent_id=outer["id"])
leaf = await _create(client, headers, title="Leaf", parent_id=inner["id"])
assert (await client.delete(f"/api/v1/documents/{outer['id']}", headers=headers)).status_code == 204
for doc_id in (outer["id"], inner["id"], leaf["id"]):
assert (await client.get(f"/api/v1/documents/{doc_id}", headers=headers)).status_code == 404
async def test_deleting_a_missing_document_is_404(client: AsyncClient, headers: dict):
assert (await client.delete(f"/api/v1/documents/{uuid.uuid4()}", headers=headers)).status_code == 404
async def test_a_deleted_document_leaves_the_search_index(client: AsyncClient, headers: dict):
doc = await _create(client, headers, title="Page", body="unmistakable")
await client.delete(f"/api/v1/documents/{doc['id']}", headers=headers)
hits = (await client.get("/api/v1/documents/search?q=unmistakable", headers=headers)).json()["hits"]
assert hits == []
# ── the document outlives what it describes ─────────────────────────────────
async def test_deleting_a_device_orphans_its_document_but_keeps_the_body(client: AsyncClient, headers: dict):
device = await _device(client, headers, notes="Holds the backups.")
doc = await _create(client, headers, title="nas-01", kind="device", device_id=device["id"])
assert (await client.delete(f"/api/v1/scan/pending/{device['id']}", headers=headers)).status_code == 200
after = (await client.get(f"/api/v1/documents/{doc['id']}", headers=headers)).json()
assert after["device_id"] is None
assert after["title"] == "nas-01"
assert "Holds the backups." in after["body"]
async def test_deleting_a_node_orphans_the_document_describing_it(client: AsyncClient, headers: dict):
design_id = await _design(client, headers)
zone = await client.post(
"/api/v1/nodes",
json={"type": "groupRect", "label": "Garage", "design_id": design_id, "pos_x": 0, "pos_y": 0},
headers=headers,
)
doc = await _create(client, headers, title="Garage", kind="node", node_id=zone.json()["id"])
await client.delete(f"/api/v1/nodes/{zone.json()['id']}", headers=headers)
after = (await client.get(f"/api/v1/documents/{doc['id']}", headers=headers)).json()
assert after["node_id"] is None
assert after["title"] == "Garage"
# ── search ──────────────────────────────────────────────────────────────────
async def test_search_names_its_engine_and_returns_hits(client: AsyncClient, headers: dict):
await _create(client, headers, title="VLAN plan", body="Guest traffic is isolated.")
res = await client.get("/api/v1/documents/search?q=isolated", headers=headers)
assert res.status_code == 200, res.text
payload = res.json()
assert payload["engine"] in ("fts5", "like")
assert [h["title"] for h in payload["hits"]] == ["VLAN plan"]
async def test_search_carries_the_device_link_so_a_hit_can_be_opened(client: AsyncClient, headers: dict):
device = await _device(client, headers)
await _create(client, headers, title="nas-01", kind="device", device_id=device["id"])
hits = (await client.get("/api/v1/documents/search?q=nas", headers=headers)).json()["hits"]
assert hits[0]["device_id"] == device["id"]
async def test_search_rejects_a_silly_limit(client: AsyncClient, headers: dict):
assert (await client.get("/api/v1/documents/search?q=x&limit=0", headers=headers)).status_code == 422
# ── scaffold and coverage ───────────────────────────────────────────────────
async def test_scaffold_documents_every_device_that_has_none(client: AsyncClient, headers: dict):
await _device(client, headers, label="a", ip="10.0.0.1")
await _device(client, headers, label="b", ip="10.0.0.2")
res = await client.post("/api/v1/documents/scaffold", json={}, headers=headers)
assert res.status_code == 200, res.text
assert len(res.json()["created"]) == 2
async def test_scaffold_is_repeatable_and_skips_what_exists(client: AsyncClient, headers: dict):
await _device(client, headers, label="a", ip="10.0.0.1")
await client.post("/api/v1/documents/scaffold", json={}, headers=headers)
again = (await client.post("/api/v1/documents/scaffold", json={}, headers=headers)).json()
assert again["created"] == []
assert again["skipped"] == 1
async def test_scaffold_can_be_limited_to_devices_that_have_notes(client: AsyncClient, headers: dict):
await _device(client, headers, label="a", ip="10.0.0.1", notes="worth keeping")
await _device(client, headers, label="b", ip="10.0.0.2")
res = (
await client.post("/api/v1/documents/scaffold", json={"only_with_notes": True}, headers=headers)
).json()
assert [d["title"] for d in res["created"]] == ["a"]
assert res["skipped"] == 1
async def test_scaffold_can_be_limited_to_named_devices(client: AsyncClient, headers: dict):
first = await _device(client, headers, label="a", ip="10.0.0.1")
await _device(client, headers, label="b", ip="10.0.0.2")
res = (
await client.post("/api/v1/documents/scaffold", json={"device_ids": [first["id"]]}, headers=headers)
).json()
assert [d["title"] for d in res["created"]] == ["a"]
async def test_a_migrated_document_records_why_it_exists(client: AsyncClient, headers: dict):
device = await _device(client, headers, notes="worth keeping")
created = (
await client.post("/api/v1/documents/scaffold", json={}, headers=headers)
).json()["created"][0]
revisions = (await client.get(f"/api/v1/documents/{created['id']}/revisions", headers=headers)).json()
assert revisions[0]["reason"] == "migrate"
assert device["id"]
async def test_coverage_counts_what_is_missing_and_unmigrated(client: AsyncClient, headers: dict):
await _device(client, headers, label="a", ip="10.0.0.1", notes="notes here")
await _device(client, headers, label="b", ip="10.0.0.2")
before = (await client.get("/api/v1/documents/coverage", headers=headers)).json()
assert before["devices"] == 2
assert before["documented"] == 0
assert before["missing"] == 2
assert before["notes_unmigrated"] == 1
await client.post("/api/v1/documents/scaffold", json={}, headers=headers)
after = (await client.get("/api/v1/documents/coverage", headers=headers)).json()
assert after["documented"] == 2
assert after["missing"] == 0
assert after["notes_unmigrated"] == 0
# Generated and not yet touched.
assert after["header_only"] == 2
async def test_coverage_reports_a_document_that_has_drifted(client: AsyncClient, headers: dict):
device = await _device(client, headers)
await _create(client, headers, title="nas", kind="device", device_id=device["id"])
assert (await client.get("/api/v1/documents/coverage", headers=headers)).json()["drifted"] == 0
await client.patch(f"/api/v1/scan/pending/{device['id']}", json={"ip": "192.168.1.99"}, headers=headers)
assert (await client.get("/api/v1/documents/coverage", headers=headers)).json()["drifted"] == 1
async def test_coverage_stops_calling_a_document_header_only_once_it_is_edited(
client: AsyncClient, headers: dict
):
device = await _device(client, headers)
doc = await _create(client, headers, title="nas", kind="device", device_id=device["id"])
await client.patch(f"/api/v1/documents/{doc['id']}", json={"body": "written by hand"}, headers=headers)
assert (await client.get("/api/v1/documents/coverage", headers=headers)).json()["header_only"] == 0
async def test_coverage_counts_library_pages_separately(client: AsyncClient, headers: dict):
await _create(client, headers, title="VLAN plan")
await _create(client, headers, title="Runbooks", kind="folder")
assert (await client.get("/api/v1/documents/coverage", headers=headers)).json()["library_pages"] == 2
+158
View File
@@ -0,0 +1,158 @@
"""The documents DDL `create_all` does not apply.
Column additions to an existing table (which `create_all` never makes), plus a
virtual table and three partial indexes (which it cannot express at all), all
applied through `_try_migrate` at boot. Every one has to survive a second boot
untouched, and the partial uniques are what stop two documents claiming the same
device behind the route's own check.
"""
import pytest
from sqlalchemy.exc import IntegrityError
from sqlalchemy.ext.asyncio import create_async_engine
from app.db.database import DOCUMENT_DDL, Base, _try_migrate
pytestmark = pytest.mark.asyncio
async def _boot(tmp_path, times: int = 1):
engine = create_async_engine(f"sqlite+aiosqlite:///{tmp_path / 'test.db'}")
for _ in range(times):
async with engine.begin() as conn:
await conn.run_sync(Base.metadata.create_all)
for label, sql in DOCUMENT_DDL:
await _try_migrate(conn, sql, label=label)
return engine
async def _names(engine, kind: str) -> set[str]:
async with engine.begin() as conn:
rows = (
await conn.exec_driver_sql(f"SELECT name FROM sqlite_master WHERE type = '{kind}'")
).fetchall()
return {row[0] for row in rows}
async def test_the_tables_and_the_search_index_are_created(tmp_path):
engine = await _boot(tmp_path)
tables = await _names(engine, "table")
assert {"documents", "document_revisions", "documents_fts"} <= tables
await engine.dispose()
async def test_the_partial_uniques_are_created(tmp_path):
engine = await _boot(tmp_path)
indexes = await _names(engine, "index")
assert {"ux_documents_device", "ux_documents_node", "ux_documents_design"} <= indexes
await engine.dispose()
async def test_a_second_boot_changes_nothing(tmp_path):
engine = await _boot(tmp_path, times=3)
tables = await _names(engine, "table")
assert {"documents", "documents_fts"} <= tables
await engine.dispose()
async def test_two_documents_cannot_claim_the_same_device(tmp_path):
engine = await _boot(tmp_path)
async with engine.begin() as conn:
insert = (
"INSERT INTO documents (id, kind, title, slug, body, device_id, sort_order, "
"frontmatter, tags, starred, created_at, updated_at) "
"VALUES (?, 'device', 'nas', ?, '', 'dev-1', 0, '{}', '[]', 0, "
"'2026-09-05', '2026-09-05')"
)
await conn.exec_driver_sql(insert, ("a", "nas"))
with pytest.raises(IntegrityError):
await conn.exec_driver_sql(insert, ("b", "nas-2"))
await engine.dispose()
async def test_many_documents_may_have_no_device(tmp_path):
engine = await _boot(tmp_path)
async with engine.begin() as conn:
insert = (
"INSERT INTO documents (id, kind, title, slug, body, device_id, sort_order, "
"frontmatter, tags, starred, created_at, updated_at) "
"VALUES (?, 'page', 'p', ?, '', NULL, 0, '{}', '[]', 0, "
"'2026-09-05', '2026-09-05')"
)
await conn.exec_driver_sql(insert, ("a", "p1"))
await conn.exec_driver_sql(insert, ("b", "p2"))
count = (await conn.exec_driver_sql("SELECT COUNT(*) FROM documents")).scalar_one()
assert count == 2
await engine.dispose()
# ── columns added to a table that already exists ─────────────────────────────
# `documents` as an earlier build of the feature created it: no `edited_at`.
_DOCUMENTS_WITHOUT_EDITED_AT = (
"CREATE TABLE documents ("
"id VARCHAR NOT NULL PRIMARY KEY, kind VARCHAR NOT NULL, title VARCHAR NOT NULL, "
"slug VARCHAR NOT NULL, icon VARCHAR, parent_id VARCHAR, sort_order INTEGER, "
"device_id VARCHAR, node_id VARCHAR, design_id VARCHAR, body TEXT NOT NULL, "
"frontmatter JSON, tags JSON, starred BOOLEAN, template_id VARCHAR, "
"facts_snapshot JSON, facts_synced_at DATETIME, reviewed_at DATETIME, "
"created_at DATETIME, updated_at DATETIME)"
)
async def _columns(engine, table: str) -> list[str]:
async with engine.begin() as conn:
rows = (await conn.exec_driver_sql(f"PRAGMA table_info({table})")).fetchall()
return [row[1] for row in rows]
async def test_an_existing_table_gains_the_columns_create_all_would_not_add(tmp_path):
"""The regression: `create_all` makes a missing table, never a missing column.
A database that met an earlier shape of `documents` kept it, and every query
naming `edited_at` failed with ``no such column: documents.edited_at`` —
including the one behind opening the Documentation section.
"""
engine = create_async_engine(f"sqlite+aiosqlite:///{tmp_path / 'test.db'}")
async with engine.begin() as conn:
await conn.exec_driver_sql(_DOCUMENTS_WITHOUT_EDITED_AT)
assert "edited_at" not in await _columns(engine, "documents")
async with engine.begin() as conn:
await conn.run_sync(Base.metadata.create_all) # sees the table, leaves it be
for label, sql in DOCUMENT_DDL:
await _try_migrate(conn, sql, label=label)
assert "edited_at" in await _columns(engine, "documents")
await engine.dispose()
async def test_the_old_rows_survive_the_column_being_added(tmp_path):
engine = create_async_engine(f"sqlite+aiosqlite:///{tmp_path / 'test.db'}")
async with engine.begin() as conn:
await conn.exec_driver_sql(_DOCUMENTS_WITHOUT_EDITED_AT)
await conn.exec_driver_sql(
"INSERT INTO documents (id, kind, title, slug, body) "
"VALUES ('d1', 'page', 'Rebuild plan', 'rebuild-plan', '# Rebuild plan')"
)
async with engine.begin() as conn:
await conn.run_sync(Base.metadata.create_all)
for label, sql in DOCUMENT_DDL:
await _try_migrate(conn, sql, label=label)
async with engine.begin() as conn:
row = (
await conn.exec_driver_sql("SELECT title, body, edited_at FROM documents WHERE id = 'd1'")
).fetchone()
assert row == ("Rebuild plan", "# Rebuild plan", None)
await engine.dispose()
async def test_a_fresh_database_has_every_column(tmp_path):
engine = await _boot(tmp_path, times=2)
columns = await _columns(engine, "documents")
assert "edited_at" in columns
# The re-stated ALTERs are idempotent: no column is added twice.
assert len(columns) == len(set(columns))
await engine.dispose()
+156 -23
View File
@@ -1,10 +1,11 @@
"""Tests for the HTTP probe used by deep-scan service identification."""
from unittest.mock import AsyncMock, patch
from unittest.mock import patch
import httpx
import pytest
from app.services.http_probe import (
_MAX_BODY_BYTES,
_extract_title,
probe_open_ports,
probe_port,
@@ -15,6 +16,39 @@ def _response(text: str = "", headers: dict | None = None, status: int = 200) ->
return httpx.Response(status_code=status, text=text, headers=headers or {})
class _TransportClient:
"""
Patch target for httpx.AsyncClient that routes through a MockTransport.
Real client, fake network: the probe exercises the genuine httpx request
path (including .stream() and aiter_bytes()) instead of a mock that would
happily accept any call shape. `requests` records what was actually sent.
"""
def __init__(self, handler):
self._handler = handler
# Bound before patching, so building the real client here does not
# recurse back into this stand-in.
self._real = httpx.AsyncClient
self.kwargs: list[dict] = []
self.requests: list[httpx.Request] = []
def _record(self, request: httpx.Request) -> httpx.Response:
self.requests.append(request)
return self._handler(request)
def __call__(self, **kwargs):
self.kwargs.append(dict(kwargs))
kwargs.pop("verify", None)
return self._real(transport=httpx.MockTransport(self._record), **kwargs)
def _patch_client(handler) -> tuple:
"""Return (context manager, factory) patching http_probe's AsyncClient."""
factory = _TransportClient(handler)
return patch("app.services.http_probe.httpx.AsyncClient", factory), factory
# ── _extract_title ──────────────────────────────────────────────────────────
def test_extract_title_basic():
@@ -37,15 +71,18 @@ def test_extract_title_case_insensitive():
@pytest.mark.asyncio
async def test_probe_port_reads_title():
with patch("httpx.AsyncClient.get", new=AsyncMock(return_value=_response("<title>Jellyfin</title>"))):
ctx, _ = _patch_client(lambda req: _response("<title>Jellyfin</title>"))
with ctx:
result = await probe_port("10.0.0.5", 8096)
assert result == {"title": "Jellyfin", "headers": {}}
@pytest.mark.asyncio
async def test_probe_port_reads_headers():
resp = _response("", headers={"Server": "nginx", "X-Powered-By": "Express"})
with patch("httpx.AsyncClient.get", new=AsyncMock(return_value=resp)):
ctx, _ = _patch_client(
lambda req: _response("", headers={"Server": "nginx", "X-Powered-By": "Express"})
)
with ctx:
result = await probe_port("10.0.0.5", 3000)
assert result["headers"] == {"Server": "nginx", "X-Powered-By": "Express"}
@@ -53,30 +90,33 @@ async def test_probe_port_reads_headers():
@pytest.mark.asyncio
async def test_probe_port_falls_back_to_http():
# https raises, http succeeds
calls = {"n": 0}
async def fake_get(self, url, **kw):
calls["n"] += 1
if url.startswith("https"):
def handler(request):
if str(request.url).startswith("https"):
raise httpx.ConnectError("tls fail")
return _response("<title>HTTP App</title>")
with patch("httpx.AsyncClient.get", new=fake_get):
ctx, factory = _patch_client(handler)
with ctx:
result = await probe_port("10.0.0.5", 8080)
assert result["title"] == "HTTP App"
assert calls["n"] == 2 # tried https then http
assert len(factory.requests) == 2 # tried https then http
@pytest.mark.asyncio
async def test_probe_port_no_signal_returns_none():
with patch("httpx.AsyncClient.get", new=AsyncMock(return_value=_response(""))):
ctx, _ = _patch_client(lambda req: _response(""))
with ctx:
result = await probe_port("10.0.0.5", 8080)
assert result is None
@pytest.mark.asyncio
async def test_probe_port_timeout_returns_none():
with patch("httpx.AsyncClient.get", new=AsyncMock(side_effect=httpx.TimeoutException("slow"))):
def handler(request):
raise httpx.TimeoutException("slow")
ctx, _ = _patch_client(handler)
with ctx:
result = await probe_port("10.0.0.5", 8080)
assert result is None
@@ -84,33 +124,126 @@ async def test_probe_port_timeout_returns_none():
@pytest.mark.asyncio
async def test_probe_port_skips_non_http_ports():
# SSH should never trigger an HTTP request
get = AsyncMock()
with patch("httpx.AsyncClient.get", new=get):
ctx, factory = _patch_client(lambda req: _response("<title>nope</title>"))
with ctx:
result = await probe_port("10.0.0.5", 22)
assert result is None
get.assert_not_called()
assert factory.requests == []
@pytest.mark.parametrize("port", [515, 9100, 9101, 9107])
@pytest.mark.asyncio
async def test_probe_port_never_touches_a_raw_printing_port(port):
"""Issue #404: a printer prints whatever bytes reach 9100, GET line included."""
ctx, factory = _patch_client(lambda req: _response("<title>nope</title>"))
with ctx:
result = await probe_port("10.0.0.5", port)
assert result is None
assert factory.requests == []
@pytest.mark.asyncio
async def test_probe_open_ports_leaves_a_printer_port_alone():
"""The batch path must skip it too, without dropping the port from the result."""
ports = [{"port": 8096, "protocol": "tcp"}, {"port": 9100, "protocol": "tcp"}]
ctx, factory = _patch_client(lambda req: _response("<title>Jellyfin</title>"))
with ctx:
result = await probe_open_ports("10.0.0.5", ports)
by_port = {p["port"]: p for p in result}
assert by_port[9100]["http_signals"] is None
assert by_port[8096]["http_signals"]["title"] == "Jellyfin"
assert all(":9100" not in str(r.url) for r in factory.requests)
@pytest.mark.asyncio
async def test_probe_port_verify_tls_flag_passed():
with patch("app.services.http_probe.httpx.AsyncClient") as client_cls:
instance = client_cls.return_value.__aenter__.return_value
instance.get = AsyncMock(return_value=_response("<title>X</title>"))
ctx, factory = _patch_client(lambda req: _response("<title>X</title>"))
with ctx:
await probe_port("10.0.0.5", 8443, verify_tls=True)
assert client_cls.call_args.kwargs["verify"] is True
assert factory.kwargs[0]["verify"] is True
# ── endless bodies (issue #375) ─────────────────────────────────────────────
_CHUNK_SIZE = 16 * 1024
def _endless_body(counter: dict, head: bytes = b""):
"""
A body that never ends and declares no Content-Length. Bounded at 512
chunks so a regression fails the test instead of hanging the suite.
"""
async def gen():
if head:
counter["bytes"] += len(head)
yield head
for _ in range(512):
counter["bytes"] += _CHUNK_SIZE
yield b"\0" * _CHUNK_SIZE
return gen()
@pytest.mark.asyncio
async def test_probe_caps_the_download_of_an_endless_body():
# _MAX_BODY_BYTES caps the download, not just the <title> scan: the body is
# streamed and the connection dropped once we hold enough. Allow one chunk
# of overshoot — the read stops on a chunk boundary.
counter = {"bytes": 0}
def handler(request):
# Plain HTTP only, like the reproducer, so the counter covers one probe.
if str(request.url).startswith("https"):
raise httpx.ConnectError("no tls")
return httpx.Response(
200,
headers={"Content-Type": "application/octet-stream"},
content=_endless_body(counter),
)
ctx, _ = _patch_client(handler)
with ctx:
result = await probe_port("10.0.0.5", 8095)
assert result is None # null bytes carry no title and no signal header
assert counter["bytes"] <= _MAX_BODY_BYTES + _CHUNK_SIZE
@pytest.mark.asyncio
async def test_probe_still_reads_a_title_from_an_endless_body():
# Stopping early must not cost us the signal: a <title> in the first chunk
# is still found even though the rest of the stream is abandoned.
counter = {"bytes": 0}
def handler(request):
if str(request.url).startswith("https"):
raise httpx.ConnectError("no tls")
return httpx.Response(
200,
headers={"Content-Type": "text/html"},
content=_endless_body(counter, head=b"<html><title>Jellyfin</title>"),
)
ctx, _ = _patch_client(handler)
with ctx:
result = await probe_port("10.0.0.5", 8096)
assert result["title"] == "Jellyfin"
assert counter["bytes"] <= _MAX_BODY_BYTES + _CHUNK_SIZE
# ── probe_open_ports ─────────────────────────────────────────────────────────
@pytest.mark.asyncio
async def test_probe_open_ports_enriches_each_port():
async def fake_get(self, url, **kw):
if ":8096" in url:
def handler(request):
if ":8096" in str(request.url):
return _response("<title>Jellyfin</title>")
return _response("")
ports = [{"port": 8096, "protocol": "tcp"}, {"port": 9999, "protocol": "tcp"}]
with patch("httpx.AsyncClient.get", new=fake_get):
ctx, _ = _patch_client(handler)
with ctx:
result = await probe_open_ports("10.0.0.5", ports)
by_port = {p["port"]: p for p in result}
+90
View File
@@ -0,0 +1,90 @@
"""Unit tests for the in-process canvas-import job registry."""
from __future__ import annotations
import pytest
from app.services import import_jobs
from app.services.import_jobs import (
create_job,
fail_job,
finish_job,
get_job,
reset_jobs,
)
@pytest.fixture(autouse=True)
def _clean() -> None:
reset_jobs()
def test_create_job_starts_running_with_unique_id() -> None:
a = create_job()
b = create_job()
assert a.id != b.id
assert a.status == "running"
assert a.result is None
assert a.error is None
def test_get_job_returns_the_registered_job() -> None:
job = create_job()
assert get_job(job.id) is job
def test_get_unknown_job_returns_none() -> None:
assert get_job("nope") is None
def test_finish_job_records_the_payload() -> None:
job = create_job()
finish_job(job.id, {"device_count": 3})
stored = get_job(job.id)
assert stored is not None
assert stored.status == "done"
assert stored.result == {"device_count": 3}
assert stored.finished_at is not None
def test_fail_job_records_message_and_status() -> None:
job = create_job()
fail_job(job.id, "broker unreachable", 502)
stored = get_job(job.id)
assert stored is not None
assert stored.status == "error"
assert stored.error == "broker unreachable"
assert stored.error_status == 502
def test_finish_and_fail_are_noops_for_unknown_ids() -> None:
finish_job("gone", {"device_count": 0})
fail_job("gone", "boom", 500)
assert get_job("gone") is None
def test_finished_jobs_expire_after_the_ttl(monkeypatch) -> None:
clock = {"now": 0.0}
monkeypatch.setattr(import_jobs.time, "monotonic", lambda: clock["now"])
job = create_job()
finish_job(job.id, {"device_count": 1})
clock["now"] = import_jobs.JOB_TTL_SECONDS + 1
assert get_job(job.id) is None
def test_running_jobs_never_expire(monkeypatch) -> None:
"""A slow mesh must not have its job purged out from under it."""
clock = {"now": 0.0}
monkeypatch.setattr(import_jobs.time, "monotonic", lambda: clock["now"])
job = create_job()
clock["now"] = import_jobs.JOB_TTL_SECONDS * 10
assert get_job(job.id) is not None
def test_reset_jobs_clears_everything() -> None:
job = create_job()
reset_jobs()
assert get_job(job.id) is None
+503
View File
@@ -17,9 +17,11 @@ from app.services.inventory_sync import (
backfill_node_devices,
changed_facts,
find_device_for,
hydrated_node,
link_facts,
merge_properties,
merge_services,
seed_node_views,
)
@@ -83,6 +85,49 @@ class TestMergeRules:
assert out[1]["port"] == 80
def test_a_scan_never_repaints_an_icon_the_user_picked(self):
"""Every "Scan network" used to overwrite a hand-picked brand icon."""
base = [
{"port": 80, "protocol": "tcp", "service_name": "http",
"icon": "brand:pihole", "category": "network"},
]
# What fingerprint_ports returns for the same port: its own guess.
incoming = [
{"port": 80, "protocol": "tcp", "service_name": "http",
"icon": "Globe", "category": "web"},
]
out = merge_services(base, incoming, discovered=True)
assert len(out) == 1
assert out[0]["icon"] == "brand:pihole"
assert out[0]["category"] == "network"
def test_a_scan_still_fills_an_icon_the_service_never_had(self):
out = merge_services(
[{"port": 80, "protocol": "tcp", "service_name": "http"}],
[{"port": 80, "protocol": "tcp", "service_name": "http",
"icon": "Globe", "category": "web"}],
discovered=True,
)
assert out[0]["icon"] == "Globe"
assert out[0]["category"] == "web"
def test_a_user_edit_still_changes_the_icon(self):
"""The guard is for scanner output only — an edit is an edit."""
out = merge_services(
[{"port": 80, "protocol": "tcp", "service_name": "http", "icon": "Globe"}],
[{"port": 80, "protocol": "tcp", "service_name": "http", "icon": "brand:pihole"}],
)
assert out[0]["icon"] == "brand:pihole"
def test_services_keep_an_established_icon_when_incoming_has_none(self):
"""A blank field is silence, not a reset — mirrors merge_properties."""
out = merge_services(
[{"port": 80, "protocol": "tcp", "service_name": "http", "icon": "brand:pihole"}],
[{"port": 80, "protocol": "tcp", "service_name": "http", "icon": None}],
)
assert out[0]["icon"] == "brand:pihole"
class TestChangedFacts:
"""What a save is allowed to write: its edit, not its whole snapshot."""
@@ -234,6 +279,38 @@ class TestIeeeCollisions:
assert device is not None and device.ieee_address == "0xAAA"
class TestStatusOnLink:
"""A row a node draws is past the pending queue."""
@pytest.mark.asyncio
async def test_pending_row_is_approved_once_a_node_draws_it(self, db_session):
design = await _design(db_session)
db_session.add(InventoryDevice(id="d-1", ip="10.0.0.5", status="pending"))
node = _node(design)
db_session.add(node)
await db_session.commit()
device = await link_facts(db_session, node, {"ip": "10.0.0.5"})
await db_session.commit()
assert device is not None and device.status == "approved"
@pytest.mark.asyncio
async def test_hidden_row_stays_hidden(self, db_session):
"""The user hid it on purpose — drawing it does not undo that."""
design = await _design(db_session)
db_session.add(InventoryDevice(id="d-1", ip="10.0.0.5", status="hidden"))
node = _node(design)
db_session.add(node)
await db_session.commit()
device = await link_facts(db_session, node, {"ip": "10.0.0.5"})
await db_session.commit()
assert device is not None and device.status == "hidden"
# --- the backfill -----------------------------------------------------------
@@ -528,6 +605,61 @@ class TestBackfill:
rows = (await db_session.execute(select(InventoryDevice))).scalars().all()
assert len(rows) == 1
@pytest.mark.asyncio
async def test_each_canvas_keeps_showing_what_it_showed(self, db_session):
"""The upgrade must not redraw a canvas.
Two canvases drew the same host with different service lists, and the
scanner's row for it holds a third the user never put on either. They
converge on one row — so each node keeps a view of the subset it drew,
and the scanner's guess appears on neither.
"""
await self._legacy_nodes_table(db_session)
ssh = {"port": 22, "protocol": "tcp", "service_name": "ssh"}
https = {"port": 443, "protocol": "tcp", "service_name": "https"}
kuma = {"port": 3001, "protocol": "tcp", "service_name": "Uptime Kuma"}
db_session.add(InventoryDevice(id="d-1", ip="10.0.0.5", services=[kuma]))
await db_session.commit()
one, two = await _design(db_session, "Net"), await _design(db_session, "Rack")
a = await self._legacy_node(db_session, one, ip="10.0.0.5", services=[ssh])
b = await self._legacy_node(db_session, two, ip="10.0.0.5", services=[ssh, https])
await backfill_node_devices(db_session)
await db_session.commit()
device = await db_session.get(InventoryDevice, "d-1")
assert {s["service_name"] for s in device.services} == {"Uptime Kuma", "ssh", "https"}
drawn = {}
for node_id in (a, b):
node = await db_session.get(Node, node_id)
payload = hydrated_node(node, device)
drawn[node_id] = [s["service_name"] for s in payload["services"] if s.get("visible", True)]
assert drawn[a] == ["ssh"]
assert drawn[b] == ["ssh", "https"]
@pytest.mark.asyncio
async def test_a_node_that_drew_no_service_keeps_drawing_none(self, db_session):
"""An empty legacy list is an answer: that canvas showed no services."""
await self._legacy_nodes_table(db_session)
db_session.add(
InventoryDevice(
id="d-1", ip="10.0.0.5",
services=[{"port": 3001, "protocol": "tcp", "service_name": "Uptime Kuma"}],
)
)
await db_session.commit()
design = await _design(db_session)
node_id = await self._legacy_node(db_session, design, ip="10.0.0.5", services=[])
await backfill_node_devices(db_session)
await db_session.commit()
node = await db_session.get(Node, node_id)
device = await db_session.get(InventoryDevice, "d-1")
payload = hydrated_node(node, device)
assert [s.get("visible", True) for s in payload["services"]] == [False]
# --- routes -----------------------------------------------------------------
@@ -980,3 +1112,374 @@ class TestRoutesKeepTheLinkInStep:
assert res.status_code == 200
node = await db_session.get(Node, res.json()["node_id"])
assert node is not None and node.device_id == "d-1"
class TestPerNodeView:
"""Order and visibility belong to the node, the facts to the row.
The same device drawn on two canvases is one inventory row, so without a
per-node view every canvas showing it inherits every service a scan ever
fingerprinted and every property any other canvas added.
"""
async def _node_on(self, client, headers, design_id: str, **payload) -> dict:
body = {"type": "nas", "label": "NAS", "ip": "10.0.0.5", "design_id": design_id, "force": True}
body.update(payload)
res = await client.post("/api/v1/nodes", json=body, headers=body.pop("headers", None) or headers)
assert res.status_code == 201
return res.json()
@pytest.mark.asyncio
async def test_a_new_node_shows_what_the_row_already_holds(
self, client: AsyncClient, headers, db_session
):
"""An empty payload list means "I have nothing to say", not "hide it all"."""
db_session.add(
InventoryDevice(
id="d-1",
ip="10.0.0.5",
services=[{"port": 22, "protocol": "tcp", "service_name": "ssh"}],
properties=[{"key": "Rack", "value": "A1", "icon": None, "visible": True}],
)
)
await db_session.commit()
design = await _design(db_session)
node = await self._node_on(client, headers, design)
assert [s["service_name"] for s in node["services"]] == ["ssh"]
assert all(s.get("visible", True) for s in node["services"])
assert node["properties"][0]["visible"] is True
@pytest.mark.asyncio
async def test_hiding_a_service_on_one_node_leaves_the_other_alone(
self, client: AsyncClient, headers, db_session
):
db_session.add(
InventoryDevice(
id="d-1",
ip="10.0.0.5",
services=[
{"port": 22, "protocol": "tcp", "service_name": "ssh"},
{"port": 3001, "protocol": "tcp", "service_name": "Uptime Kuma"},
],
)
)
await db_session.commit()
one, two = await _design(db_session, "Net"), await _design(db_session, "Rack")
node_a = await self._node_on(client, headers, one)
node_b = await self._node_on(client, headers, two)
hidden = [
{"port": 22, "protocol": "tcp", "service_name": "ssh"},
{"port": 3001, "protocol": "tcp", "service_name": "Uptime Kuma", "visible": False},
]
res = await client.patch(
f"/api/v1/nodes/{node_a['id']}", json={"services": hidden}, headers=headers
)
assert res.status_code == 200
assert [(s["service_name"], s.get("visible", True)) for s in res.json()["services"]] == [
("ssh", True), ("Uptime Kuma", False),
]
other = (await client.get(f"/api/v1/nodes/{node_b['id']}", headers=headers)).json()
assert [s.get("visible", True) for s in other["services"]] == [True, True]
# The service itself is untouched — hiding is not deleting.
device = await db_session.get(InventoryDevice, "d-1")
await db_session.refresh(device)
assert len(device.services) == 2
@pytest.mark.asyncio
async def test_the_order_is_per_node_too(self, client: AsyncClient, headers, db_session):
db_session.add(
InventoryDevice(
id="d-1",
ip="10.0.0.5",
properties=[
{"key": "Rack", "value": "A1", "icon": None, "visible": True},
{"key": "Owner", "value": "me", "icon": None, "visible": True},
],
)
)
await db_session.commit()
one, two = await _design(db_session, "Net"), await _design(db_session, "Rack")
node_a = await self._node_on(client, headers, one)
node_b = await self._node_on(client, headers, two)
flipped = [
{"key": "Owner", "value": "me", "icon": None, "visible": True},
{"key": "Rack", "value": "A1", "icon": None, "visible": True},
]
res = await client.patch(
f"/api/v1/nodes/{node_a['id']}", json={"properties": flipped}, headers=headers
)
assert [p["key"] for p in res.json()["properties"]] == ["Owner", "Rack"]
other = (await client.get(f"/api/v1/nodes/{node_b['id']}", headers=headers)).json()
assert [p["key"] for p in other["properties"]] == ["Rack", "Owner"]
@pytest.mark.asyncio
async def test_a_service_the_row_gains_later_stays_off_the_canvas(
self, client: AsyncClient, headers, db_session
):
"""The leak this column exists to stop: a scan must not redraw every canvas."""
db_session.add(
InventoryDevice(
id="d-1", ip="10.0.0.5",
services=[{"port": 22, "protocol": "tcp", "service_name": "ssh"}],
)
)
await db_session.commit()
design = await _design(db_session)
node = await self._node_on(client, headers, design)
device = await db_session.get(InventoryDevice, "d-1")
device.services = [
*device.services,
{"port": 3001, "protocol": "tcp", "service_name": "Uptime Kuma"},
]
await db_session.commit()
after = (await client.get(f"/api/v1/nodes/{node['id']}", headers=headers)).json()
assert [(s["service_name"], s.get("visible", True)) for s in after["services"]] == [
("ssh", True), ("Uptime Kuma", False),
]
@pytest.mark.asyncio
async def test_removing_a_service_removes_it_from_the_device(
self, client: AsyncClient, headers, db_session
):
"""Delete is device-wide — hiding is what a single canvas does."""
db_session.add(
InventoryDevice(
id="d-1", ip="10.0.0.5",
services=[
{"port": 22, "protocol": "tcp", "service_name": "ssh"},
{"port": 3001, "protocol": "tcp", "service_name": "Uptime Kuma"},
],
)
)
await db_session.commit()
one, two = await _design(db_session, "Net"), await _design(db_session, "Rack")
node_a = await self._node_on(client, headers, one)
node_b = await self._node_on(client, headers, two)
await client.patch(
f"/api/v1/nodes/{node_a['id']}",
json={"services": [{"port": 22, "protocol": "tcp", "service_name": "ssh"}]},
headers=headers,
)
other = (await client.get(f"/api/v1/nodes/{node_b['id']}", headers=headers)).json()
assert [s["service_name"] for s in other["services"]] == ["ssh"]
@pytest.mark.asyncio
async def test_a_canvas_save_records_the_view_even_when_no_fact_changed(
self, client: AsyncClient, headers, db_session
):
"""`changed_facts` guards the shared row, not the node's own view.
A canvas save reporting no edited fact is exactly what hiding a service
looks like from the row's side — nothing about the device changed. The
view still has to land, or the toggle would not survive the save.
"""
db_session.add(
InventoryDevice(
id="d-1", ip="10.0.0.5",
services=[
{"port": 22, "protocol": "tcp", "service_name": "ssh"},
{"port": 3001, "protocol": "tcp", "service_name": "Uptime Kuma"},
],
)
)
await db_session.commit()
design = await _design(db_session)
node = await self._node_on(client, headers, design)
res = await client.post(
"/api/v1/canvas/save",
json={
"design_id": design,
"nodes": [{
"id": node["id"], "type": "nas", "label": "NAS", "ip": "10.0.0.5",
"device_id": "d-1", "changed_facts": [], "pos_x": 10.0, "pos_y": 20.0,
"services": [
{"port": 3001, "protocol": "tcp", "service_name": "Uptime Kuma", "visible": False},
{"port": 22, "protocol": "tcp", "service_name": "ssh"},
],
}],
"edges": [],
"viewport": {"x": 0, "y": 0, "zoom": 1},
},
headers=headers,
)
assert res.status_code == 200
loaded = (await client.get(f"/api/v1/canvas?design_id={design}", headers=headers)).json()
assert [(s["service_name"], s.get("visible", True)) for s in loaded["nodes"][0]["services"]] == [
("Uptime Kuma", False), ("ssh", True),
]
device = await db_session.get(InventoryDevice, "d-1")
await db_session.refresh(device)
assert [s["service_name"] for s in device.services] == ["ssh", "Uptime Kuma"]
@pytest.mark.asyncio
async def test_seeding_freezes_what_a_pre_upgrade_node_showed(self, db_session):
"""`display_view` arrives NULL on every existing node; the seed fills it."""
design = await _design(db_session)
db_session.add(
InventoryDevice(
id="d-1", ip="10.0.0.5",
services=[{"port": 22, "protocol": "tcp", "service_name": "ssh"}],
)
)
node = _node(design, device_id="d-1")
db_session.add(node)
await db_session.commit()
assert await seed_node_views(db_session) == 1
await db_session.commit()
assert node.display_view == {
"services": [{"key": "22|tcp|ssh", "visible": True}],
"properties": [],
}
# Idempotent: a second boot finds nothing without a view.
assert await seed_node_views(db_session) == 0
@pytest.mark.asyncio
async def test_a_node_whose_device_was_deleted_does_not_keep_the_seed_alive(self, db_session):
"""Deleting a device leaves `nodes.device_id` dangling — foreign keys are off.
Such a node can be seeded from nothing, so it must not keep matching the
query: it would re-open the pre-upgrade backup on every single boot and
log a recovery that recovers nothing.
"""
design = await _design(db_session)
db_session.add(_node(design, device_id="d-gone"))
await db_session.commit()
reads = []
def drawn():
reads.append(1)
return {}
assert await seed_node_views(db_session, drawn=drawn) == 0
assert reads == [] # the backup was never opened
@pytest.mark.asyncio
async def test_seeding_leaves_furniture_alone(self, db_session):
design = await _design(db_session)
node = _node(design, type="groupRect")
db_session.add(node)
await db_session.commit()
assert await seed_node_views(db_session) == 0
assert node.display_view is None
@pytest.mark.asyncio
async def test_a_view_matching_nothing_left_on_the_row_shows_the_row(self, db_session):
"""The row was replaced wholesale under the node — don't draw a blank node.
A view whose every key is gone tells us nothing about the list that
replaced it, and hiding all of it leaves a node showing no service while
"Show services" is on (#347). An *empty* view is a real answer and still
hides everything — that case is covered below.
"""
device = InventoryDevice(
id="d-1",
services=[
{"port": 8080, "protocol": "tcp", "service_name": "HTTP Alt"},
{"port": 5001, "protocol": "tcp", "service_name": "Synology DSM HTTPS"},
],
)
node = _node(
await _design(db_session),
device_id="d-1",
display_view={"services": [{"key": "9000|tcp|portainer", "visible": True}]},
)
assert [s["service_name"] for s in hydrated_node(node, device)["services"]] == [
"HTTP Alt", "Synology DSM HTTPS",
]
assert all(s.get("visible", True) for s in hydrated_node(node, device)["services"])
@pytest.mark.asyncio
async def test_an_empty_view_still_hides_everything(self, db_session):
""""This canvas draws none of them" is an answer, not a missing view."""
device = InventoryDevice(
id="d-1", services=[{"port": 22, "protocol": "tcp", "service_name": "ssh"}]
)
node = _node(await _design(db_session), device_id="d-1", display_view={"services": []})
assert hydrated_node(node, device)["services"] == [
{"port": 22, "protocol": "tcp", "service_name": "ssh", "visible": False}
]
@pytest.mark.asyncio
async def test_seeding_keeps_a_property_added_after_the_backup(self, db_session):
"""The backup is 3.2.0-era: it cannot know a property added since.
Adding one while on 3.3.0-3.3.2 wrote it to the row, and every canvas
drew it. Seeding strictly from the backup would take it off all of them
at once (#347), so it is appended visible after what the backup places.
"""
design = await _design(db_session)
db_session.add(
InventoryDevice(
id="d-1", ip="10.0.0.5",
properties=[
{"key": "Rack", "value": "A", "icon": None, "visible": True},
{"key": "Ports", "value": "8", "icon": None, "visible": True},
],
)
)
node = _node(design, device_id="d-1")
db_session.add(node)
await db_session.commit()
# The backup only knew "Rack", and this canvas had it hidden.
drawn = {node.id: {
"services": [],
"properties": [{"key": "Rack", "value": "A", "icon": None, "visible": False}],
}}
assert await seed_node_views(db_session, drawn=lambda: drawn) == 1
await db_session.commit()
assert node.display_view["properties"] == [
{"key": "rack", "visible": False}, # recovered, still hidden here
{"key": "ports", "visible": True}, # added since the backup, kept
]
@pytest.mark.asyncio
async def test_seeding_still_holds_back_a_service_found_since_the_backup(self, db_session):
"""Services are not treated that way: a scan's find stays off the canvas."""
design = await _design(db_session)
db_session.add(
InventoryDevice(
id="d-1", ip="10.0.0.5",
services=[
{"port": 22, "protocol": "tcp", "service_name": "ssh"},
{"port": 5001, "protocol": "tcp", "service_name": "Synology DSM HTTPS"},
],
)
)
node = _node(design, device_id="d-1")
db_session.add(node)
await db_session.commit()
drawn = {node.id: {
"services": [{"port": 22, "protocol": "tcp", "service_name": "ssh"}],
"properties": [],
}}
assert await seed_node_views(db_session, drawn=lambda: drawn) == 1
await db_session.commit()
assert node.display_view["services"] == [{"key": "22|tcp|ssh", "visible": True}]
@pytest.mark.asyncio
async def test_a_node_without_a_view_still_shows_everything(self, db_session):
"""No view — furniture, or a node an older version linked — is not "hide all"."""
device = InventoryDevice(
id="d-1", services=[{"port": 22, "protocol": "tcp", "service_name": "ssh"}]
)
node = _node(await _design(db_session), device_id="d-1")
assert hydrated_node(node, device)["services"] == [
{"port": 22, "protocol": "tcp", "service_name": "ssh"}
]
+100
View File
@@ -2,13 +2,16 @@
from __future__ import annotations
import asyncio
import json
import ssl
from unittest.mock import patch
import pytest
from app.core.config import settings
from app.services.mqtt_common import (
_RESPONSE_TIMEOUT,
_build_tls_context,
_sanitize_mqtt_error,
request_response,
@@ -181,3 +184,100 @@ async def test_test_connection_failure() -> None:
mock_aiomqtt.MqttError = Exception
with pytest.raises(ConnectionError):
await _test_connection("bad", 1883)
# ---------------------------------------------------------------------------
# Response timeout — configurable via MQTT_RESPONSE_TIMEOUT (issue #380)
# ---------------------------------------------------------------------------
class _SilentClient:
"""An MQTT client that connects but never delivers the response message."""
async def __aenter__(self):
return self
async def __aexit__(self, *_):
pass
async def subscribe(self, *_a, **_kw) -> None:
pass
async def publish(self, *_a, **_kw) -> None:
pass
@property
def messages(self):
async def _never():
await asyncio.Event().wait()
yield # pragma: no cover
return _never()
class _NeverMqttError(Exception):
"""Stand-in for aiomqtt.MqttError that no test path actually raises."""
async def _round_trip(**kwargs):
return await request_response(
mqtt_host="host",
mqtt_port=1883,
request_topic="req",
response_topic="res",
request_payload={},
**kwargs,
)
@pytest.mark.asyncio
async def test_request_response_uses_settings_timeout(monkeypatch) -> None:
monkeypatch.setattr(settings, "mqtt_response_timeout", 0.01)
with patch("app.services.mqtt_common.aiomqtt") as mock_aiomqtt:
mock_aiomqtt.Client.return_value = _SilentClient()
mock_aiomqtt.MqttError = _NeverMqttError
with pytest.raises(TimeoutError) as ei:
await _round_trip()
msg = str(ei.value)
assert "0.01s" in msg
assert "MQTT_RESPONSE_TIMEOUT" in msg
@pytest.mark.asyncio
async def test_request_response_explicit_timeout_overrides_settings(monkeypatch) -> None:
monkeypatch.setattr(settings, "mqtt_response_timeout", 999)
with patch("app.services.mqtt_common.aiomqtt") as mock_aiomqtt:
mock_aiomqtt.Client.return_value = _SilentClient()
mock_aiomqtt.MqttError = _NeverMqttError
with pytest.raises(TimeoutError) as ei:
await _round_trip(response_timeout=0.01)
assert "0.01s" in str(ei.value)
@pytest.mark.asyncio
async def test_request_response_non_positive_setting_falls_back(monkeypatch) -> None:
"""A misconfigured 0 must not mean 'give up immediately'."""
monkeypatch.setattr(settings, "mqtt_response_timeout", 0)
captured: dict[str, float] = {}
async def _fake_wait_for(awaitable, timeout):
captured["timeout"] = timeout
awaitable.close()
raise asyncio.TimeoutError
with patch("app.services.mqtt_common.aiomqtt") as mock_aiomqtt:
mock_aiomqtt.Client.return_value = _SilentClient()
mock_aiomqtt.MqttError = _NeverMqttError
with (
patch("app.services.mqtt_common.asyncio.wait_for", _fake_wait_for),
pytest.raises(TimeoutError),
):
await _round_trip()
assert captured["timeout"] == _RESPONSE_TIMEOUT
+24
View File
@@ -134,6 +134,30 @@ async def test_repoints_child_parent(db_session):
assert child.parent_id == keep.id
@pytest.mark.asyncio
async def test_canonical_nested_under_its_own_duplicate_is_detached(db_session):
"""#370 — re-pointing the canonical node made it its own parent.
The canonical node had been nested under one of its duplicates, so the
blanket re-point wrote `parent_id = id` and froze it on the canvas.
"""
d = await _design(db_session)
device = await _device(db_session, ieee="0xFFF")
keep = Node(label="Host", type="proxmox", design_id=d.id, device_id=device.id)
db_session.add(keep)
await db_session.flush()
dup = Node(label="Host", type="proxmox", design_id=d.id, device_id=device.id)
db_session.add(dup)
await db_session.flush()
# The oldest node is the one that survives, and it points at the newer twin.
keep.parent_id = dup.id
await db_session.flush()
await dedupe_nodes_by_device(db_session)
await db_session.refresh(keep)
assert keep.parent_id is None
@pytest.mark.asyncio
async def test_idempotent_noop_when_unique(db_session):
d = await _design(db_session)
+84
View File
@@ -403,3 +403,87 @@ async def test_list_nodes_without_label_param_returns_all(client: AsyncClient, h
res = await client.get("/api/v1/nodes", headers=headers)
assert res.status_code == 200
assert len(res.json()) == 3
# ── self-parent guard (#370) ─────────────────────────────────────────────────
async def test_update_node_ignores_a_parent_id_pointing_at_itself(client: AsyncClient, headers: dict):
create = await client.post(
"/api/v1/nodes", json={"type": "lxc", "label": "pihole", "status": "unknown"}, headers=headers
)
node_id = create.json()["id"]
res = await client.patch(
f"/api/v1/nodes/{node_id}", json={"parent_id": node_id, "label": "renamed"}, headers=headers
)
assert res.status_code == 200
assert res.json()["parent_id"] is None
# The key is dropped, not the whole edit.
assert res.json()["label"] == "renamed"
async def test_update_node_self_parent_leaves_a_real_parent_alone(client: AsyncClient, headers: dict):
host = await client.post(
"/api/v1/nodes",
json={"type": "proxmox", "label": "pve", "status": "unknown", "container_mode": True},
headers=headers,
)
host_id = host.json()["id"]
child = await client.post(
"/api/v1/nodes",
json={"type": "lxc", "label": "pihole", "status": "unknown", "parent_id": host_id},
headers=headers,
)
child_id = child.json()["id"]
res = await client.patch(f"/api/v1/nodes/{child_id}", json={"parent_id": child_id}, headers=headers)
assert res.status_code == 200
assert res.json()["parent_id"] == host_id
# ── furniture descriptions ───────────────────────────────────────────────────
async def test_create_and_update_a_group_description(client: AsyncClient, headers: dict):
create = await client.post(
"/api/v1/nodes",
json={"type": "group", "label": "Cluster", "description": "The three Proxmox boxes."},
headers=headers,
)
assert create.status_code == 201
assert create.json()["description"] == "The three Proxmox boxes."
node_id = create.json()["id"]
res = await client.patch(f"/api/v1/nodes/{node_id}", json={"description": "Now four."}, headers=headers)
assert res.status_code == 200
assert res.json()["description"] == "Now four."
assert (await client.get(f"/api/v1/nodes/{node_id}", headers=headers)).json()["description"] == "Now four."
async def test_update_a_zone_description_leaves_the_label_alone(client: AsyncClient, headers: dict):
create = await client.post("/api/v1/nodes", json={"type": "groupRect", "label": "Garage"}, headers=headers)
node_id = create.json()["id"]
res = await client.patch(f"/api/v1/nodes/{node_id}", json={"description": "Behind the door."}, headers=headers)
assert res.json()["label"] == "Garage"
assert res.json()["description"] == "Behind the door."
async def test_a_device_node_never_keeps_a_description(client: AsyncClient, headers: dict):
# `description` is furniture-only: a node that draws a device writes what it
# is for to the inventory row, as `notes`.
create = await client.post(
"/api/v1/nodes",
json={"type": "nas", "label": "nas-01", "ip": "192.168.1.20", "description": "Not here."},
headers=headers,
)
assert create.json()["description"] is None
res = await client.patch(
f"/api/v1/nodes/{create.json()['id']}",
json={"description": "Still not here.", "notes": "Backs up nightly."},
headers=headers,
)
assert res.json()["description"] is None
assert res.json()["notes"] == "Backs up nightly."
+195 -4
View File
@@ -3,6 +3,7 @@
from __future__ import annotations
import uuid
from datetime import datetime, timezone
from types import SimpleNamespace
from unittest.mock import AsyncMock, MagicMock, patch
@@ -39,14 +40,17 @@ def _host_node() -> dict:
}
def _guest_node(vmid: int, ip: str | None, status: str = "online", mac: str | None = None) -> dict:
def _guest_node(
vmid: int, ip: str | None, status: str = "online", mac: str | None = None,
host: str = "pve1",
) -> dict:
return {
"id": f"pve-pve1-{vmid}", "label": f"vm{vmid}", "type": "vm",
"ieee_address": f"pve-pve1-{vmid}", "hostname": f"vm{vmid}", "ip": ip,
"id": f"pve-{host}-{vmid}", "label": f"vm{vmid}", "type": "vm",
"ieee_address": f"pve-{host}-{vmid}", "hostname": f"vm{vmid}", "ip": ip,
"mac": mac,
"status": status, "cpu_count": 2, "ram_gb": 4.0, "disk_gb": 32.0,
"vendor": "Proxmox VE", "model": "QEMU", "vmid": vmid,
"parent_ieee": "pve-node-pve1",
"parent_ieee": f"pve-node-{host}",
}
@@ -273,6 +277,134 @@ async def test_persist_merges_pending_scan_row_by_mac(db_session) -> None:
assert set(row.discovery_sources) == {"arp", "proxmox"} # shows in both filters
@pytest.mark.asyncio
async def test_persist_never_overwrites_another_guest_row_on_a_shared_ip(db_session) -> None:
# Issue #419: two guests report the same address (duplicate lease, re-used
# DHCP, or a container bridge address the agent lists first). Importing 812
# used to match 802's row on that IP and overwrite its hostname and specs,
# leaving no row for the original guest.
db_session.add(InventoryDevice(
id=str(uuid.uuid4()), ieee_address="pve-pve1-802", ip="10.0.0.5",
hostname="vm802", suggested_type="vm", status="pending",
cpu_count=8, ram_gb=32.0, disk_gb=500.0,
discovery_source="proxmox", discovery_sources=["proxmox"],
))
await db_session.commit()
await _persist_pending_import(db_session, [_guest_node(812, "10.0.0.5")], [])
rows = (await db_session.execute(select(InventoryDevice))).scalars().all()
assert len(rows) == 2 # 802 still its own row
kept = next(r for r in rows if r.ieee_address == "pve-pve1-802")
assert kept.hostname == "vm802" # not stomped by 812
assert (kept.cpu_count, kept.ram_gb, kept.disk_gb) == (8, 32.0, 500.0)
added = next(r for r in rows if r.ieee_address == "pve-pve1-812")
assert added.hostname == "vm812"
@pytest.mark.asyncio
async def test_persist_matches_ieee_before_a_shared_ip(db_session) -> None:
# Both rows exist and share an IP. The guest's own ieee wins over the other
# row's matching address, whichever order the database yields them in.
for vmid, hostname in ((802, "vm802"), (812, "stale")):
db_session.add(InventoryDevice(
id=str(uuid.uuid4()), ieee_address=f"pve-pve1-{vmid}", ip="10.0.0.5",
hostname=hostname, suggested_type="vm", status="pending",
discovery_source="proxmox", discovery_sources=["proxmox"],
))
await db_session.commit()
await _persist_pending_import(db_session, [_guest_node(812, "10.0.0.5")], [])
rows = (await db_session.execute(select(InventoryDevice))).scalars().all()
assert len(rows) == 2
assert next(r for r in rows if r.ieee_address == "pve-pve1-802").hostname == "vm802"
assert next(r for r in rows if r.ieee_address == "pve-pve1-812").hostname == "vm812"
@pytest.mark.asyncio
async def test_persist_merges_the_oldest_scan_row_when_two_share_an_ip(db_session) -> None:
# Unclaimed scan rows on one address: pick deterministically (oldest), so a
# re-import lands on the same row instead of whichever comes back first.
for offset, mac in ((2, "aa:bb:cc:00:00:01"), (1, "aa:bb:cc:00:00:02")):
db_session.add(InventoryDevice(
id=str(uuid.uuid4()), ip="10.0.0.5", mac=mac,
suggested_type="generic", status="pending",
discovered_at=datetime(2026, 1, offset, tzinfo=timezone.utc),
discovery_source="arp", discovery_sources=["arp"],
))
await db_session.commit()
for _ in range(2):
await _persist_pending_import(db_session, [_guest_node(101, "10.0.0.5")], [])
rows = (await db_session.execute(select(InventoryDevice))).scalars().all()
assert len(rows) == 2 # merged, never a third
merged = next(r for r in rows if r.ieee_address == "pve-pve1-101")
assert merged.mac == "aa:bb:cc:00:00:02" # the older row
@pytest.mark.asyncio
async def test_persist_merges_a_guest_that_moved_to_another_host(db_session) -> None:
# The synthetic ieee embeds the host name, so a live migration or a node
# rename changes it for what is the same guest. The NIC MAC survives both:
# merge and re-point the row, rather than filing a new device and orphaning
# the old row (which would then never resolve as a host→guest link end).
await _persist_pending_import(
db_session, [_guest_node(802, "10.0.0.5", mac="bc:24:11:aa:bb:cc")], []
)
row = (await db_session.execute(select(InventoryDevice))).scalars().one()
row.status = "approved"
await db_session.commit()
await _persist_pending_import(
db_session,
[_guest_node(802, "10.0.0.5", mac="bc:24:11:aa:bb:cc", host="pve2")],
[],
)
rows = (await db_session.execute(select(InventoryDevice))).scalars().all()
assert len(rows) == 1 # no duplicate, no orphan
assert rows[0].ieee_address == "pve-pve2-802" # re-pointed to the new host
@pytest.mark.asyncio
async def test_persist_never_repoints_a_mesh_ieee(db_session) -> None:
# A zigbee/zwave ieee is a real hardware address. A Proxmox import that
# merges into such a row by MAC must not overwrite it.
db_session.add(InventoryDevice(
id=str(uuid.uuid4()), ieee_address="0x00124b0022a1b2c3",
mac="bc:24:11:aa:bb:cc", suggested_type="zigbee_device", status="pending",
discovery_source="zigbee", discovery_sources=["zigbee"],
))
await db_session.commit()
await _persist_pending_import(
db_session, [_guest_node(101, None, mac="bc:24:11:aa:bb:cc")], []
)
rows = (await db_session.execute(select(InventoryDevice))).scalars().all()
assert len(rows) == 1
assert rows[0].ieee_address == "0x00124b0022a1b2c3"
@pytest.mark.asyncio
async def test_persist_merges_two_guests_that_share_a_nic_mac(db_session) -> None:
# Accepted trade-off of matching on MAC: two guests configured with the same
# NIC MAC collapse into one row. That is a misconfiguration which already
# breaks their networking, and the alternative — excluding claimed rows from
# the MAC fallback — orphans every migrated guest. Shared *IPs* stay split;
# see test_persist_never_overwrites_another_guest_row_on_a_shared_ip.
nodes = [
_guest_node(802, "10.0.0.5", mac="bc:24:11:aa:bb:cc"),
_guest_node(812, "10.0.0.6", mac="bc:24:11:aa:bb:cc"),
]
await _persist_pending_import(db_session, nodes, [])
rows = (await db_session.execute(select(InventoryDevice))).scalars().all()
assert len(rows) == 1
@pytest.mark.asyncio
async def test_persist_preserves_ip_tag_for_legacy_null_source_row(db_session) -> None:
# Legacy inventory row from an old IP scan, before discovery_source(s) were
@@ -509,3 +641,62 @@ async def test_persist_never_deletes(db_session) -> None:
rows = (await db_session.execute(select(InventoryDevice))).scalars().all()
ieees = {r.ieee_address for r in rows}
assert "pve-pve1-101" in ieees and "pve-pve1-202" in ieees
@pytest.mark.asyncio
async def test_canvas_import_also_lands_in_device_inventory(
client: AsyncClient, headers: dict, db_session
) -> None:
"""A canvas import is an import: the guests reach the Device Inventory too."""
nodes = [_host_node(), _guest_node(101, "10.0.0.5")]
with patch("app.api.routes.proxmox.fetch_proxmox_inventory") as mock_fetch:
mock_fetch.return_value = (nodes, [])
res = await client.post(
"/api/v1/proxmox/import",
json={
"host": "pve1",
"token_id": "root@pam!t",
"token_secret": "s",
},
headers=headers,
)
assert res.status_code == 200, res.text
rows = (
await db_session.execute(
select(InventoryDevice).where(InventoryDevice.discovery_source == "proxmox")
)
).scalars().all()
by_ieee = {r.ieee_address: r for r in rows}
assert set(by_ieee) == {"pve-node-pve1", "pve-pve1-101"}
for node in res.json()["nodes"]:
assert node["device_id"] == by_ieee[node["ieee_address"]].id
@pytest.mark.asyncio
async def test_canvas_import_points_at_the_row_matched_by_ip(
client: AsyncClient, headers: dict, db_session
) -> None:
"""A guest merged into a scanned row by IP draws that row, not a new one."""
scanned = InventoryDevice(
id=str(uuid.uuid4()), ip="10.0.0.5", suggested_type="generic",
status="pending", discovery_source="arp", discovery_sources=["arp"],
)
db_session.add(scanned)
await db_session.commit()
with patch("app.api.routes.proxmox.fetch_proxmox_inventory") as mock_fetch:
mock_fetch.return_value = ([_guest_node(101, "10.0.0.5")], [])
res = await client.post(
"/api/v1/proxmox/import",
json={"host": "pve1", "token_id": "root@pam!t", "token_secret": "s"},
headers=headers,
)
assert res.status_code == 200, res.text
assert res.json()["nodes"][0]["device_id"] == scanned.id
rows = (
await db_session.execute(
select(InventoryDevice).where(InventoryDevice.ip == "10.0.0.5")
)
).scalars().all()
assert len(rows) == 1
+22
View File
@@ -30,6 +30,28 @@ def test_extract_qemu_ip_skips_loopback() -> None:
assert svc._extract_qemu_ip({"result": []}) is None
def test_extract_qemu_ip_prefers_the_configured_nic() -> None:
# A guest running k8s/Docker: the agent lists the CNI bridge first, and that
# address repeats across guests. Pick the interface holding the NIC MAC from
# /config instead (issue #419).
payload = {
"result": [
{
"name": "cni0", "hardware-address": "0a:58:0a:2a:00:01",
"ip-addresses": [{"ip-address-type": "ipv4", "ip-address": "10.42.0.1"}],
},
{
"name": "eth0", "hardware-address": "BC:24:11:AA:BB:CC",
"ip-addresses": [{"ip-address-type": "ipv4", "ip-address": "192.168.1.20"}],
},
]
}
assert svc._extract_qemu_ip(payload, "bc:24:11:aa:bb:cc") == "192.168.1.20"
# No MAC known, or none matching: first non-loopback IPv4, as before.
assert svc._extract_qemu_ip(payload) == "10.42.0.1"
assert svc._extract_qemu_ip(payload, "de:ad:be:ef:00:00") == "10.42.0.1"
def test_extract_lxc_ip_parses_net0_static() -> None:
cfg = {"net0": "name=eth0,bridge=vmbr0,ip=192.168.1.30/24,gw=192.168.1.1"}
assert svc._extract_lxc_ip(cfg) == "192.168.1.30"
+265
View File
@@ -93,6 +93,210 @@ class TestDesignType:
assert res.status_code == 422
class TestDeviceRackModel:
"""The inventory row owns the front panel; the mount only places it.
A device wears the same faceplate, size, colour and ports in every rack, so a
save writes them onto `device_inventory` and a load reads them back out.
"""
async def _mounted(self, client: AsyncClient, headers: dict, db_session, **device) -> str:
design_id = await _design(client, headers)
db_session.add(InventoryDevice(id="inv-sw", ip="192.168.1.10", suggested_type="switch"))
await db_session.commit()
state = _state(design_id, cables=[])
state["devices"] = [{**state["devices"][0], "device_id": "inv-sw", **device}]
res = await client.post("/api/v1/racks/save", json=state, headers=headers)
assert res.status_code == 200, res.text
return design_id
async def test_writes_the_mount_panel_onto_the_inventory_row(
self, client: AsyncClient, headers, db_session
):
await self._mounted(client, headers, db_session, color="#ff6e00")
entry = await db_session.get(InventoryDevice, "inv-sw")
await db_session.refresh(entry)
assert entry.rack_faceplate_id == "switch-24"
assert entry.rack_u_height == 1
assert entry.rack_col_span == 12
assert entry.rack_color == "#ff6e00"
assert entry.rack_ports == [{"id": "p1", "label": "1", "type": "rj45", "x": 0.4, "y": 0.5}]
async def test_leaves_the_inventory_alone_for_an_accessory(
self, client: AsyncClient, headers, db_session
):
design_id = await _design(client, headers)
db_session.add(InventoryDevice(id="inv-untouched", ip="192.168.1.11"))
await db_session.commit()
state = _state(design_id, cables=[])
state["devices"] = [{**state["devices"][0], "faceplate_id": "shelf-1u", "ports": []}]
await client.post("/api/v1/racks/save", json=state, headers=headers)
entry = await db_session.get(InventoryDevice, "inv-untouched")
await db_session.refresh(entry)
assert entry.rack_faceplate_id is None
async def test_load_serves_the_inventory_panel_over_the_mount_copy(
self, client: AsyncClient, headers, db_session
):
design_id = await self._mounted(client, headers, db_session)
# Another rack modelled the same device differently since this mount was
# written: the row wins, ports included.
entry = await db_session.get(InventoryDevice, "inv-sw")
entry.rack_faceplate_id = "switch-48"
entry.rack_color = "#a855f7"
entry.rack_ports = [{"id": "px", "label": "uplink", "type": "sfp+", "x": 0.9, "y": 0.4}]
await db_session.commit()
device = (
await client.get(f"/api/v1/racks?design_id={design_id}", headers=headers)
).json()["devices"][0]
assert device["faceplate_id"] == "switch-48"
assert device["color"] == "#a855f7"
assert [p["id"] for p in device["ports"]] == ["px"]
async def test_load_keeps_a_size_that_no_longer_fits_this_rack(
self, client: AsyncClient, headers, db_session
):
# The mount sits at U 10 of a 12U rack. Grown to 4U elsewhere, it would
# now run past the top rail — geometry is global, a rack is not.
design_id = await self._mounted(client, headers, db_session)
entry = await db_session.get(InventoryDevice, "inv-sw")
entry.rack_u_height = 4
entry.rack_col_span = 12
await db_session.commit()
device = (
await client.get(f"/api/v1/racks?design_id={design_id}", headers=headers)
).json()["devices"][0]
assert device["u_height"] == 1
# The plate itself cannot overflow a rail, so it still comes from the row.
assert device["faceplate_id"] == "switch-24"
async def test_load_keeps_a_width_that_overruns_the_column_grid(
self, client: AsyncClient, headers, db_session
):
design_id = await self._mounted(client, headers, db_session, col_start=6, col_span=6)
entry = await db_session.get(InventoryDevice, "inv-sw")
entry.rack_col_span = 12
await db_session.commit()
device = (
await client.get(f"/api/v1/racks?design_id={design_id}", headers=headers)
).json()["devices"][0]
assert device["col_span"] == 6
async def test_the_tray_offers_the_saved_panel(
self, client: AsyncClient, headers, db_session
):
design_id = await self._mounted(client, headers, db_session, color="#ff6e00")
items = (
await client.get(f"/api/v1/racks/inventory?design_id={design_id}", headers=headers)
).json()["items"]
item = next(i for i in items if i["id"] == "inv-sw")
assert item["rack_faceplate_id"] == "switch-24"
assert item["rack_color"] == "#ff6e00"
assert [p["id"] for p in item["rack_ports"]] == ["p1"]
async def test_a_never_racked_device_reports_no_panel(
self, client: AsyncClient, headers, db_session
):
design_id = await _design(client, headers)
db_session.add(InventoryDevice(id="inv-fresh", ip="192.168.1.12", suggested_type="nas"))
await db_session.commit()
items = (
await client.get(f"/api/v1/racks/inventory?design_id={design_id}", headers=headers)
).json()["items"]
item = next(i for i in items if i["id"] == "inv-fresh")
assert item["rack_faceplate_id"] is None
assert item["rack_ports"] == []
async def test_a_clamped_size_is_not_written_back_over_the_model(
self, client: AsyncClient, headers, db_session
):
# The mount is served 1U because 4U no longer fits above U 10. Saving the
# canvas then echoes that 1U back — and must not shrink the device in the
# rack where it really is 4U.
design_id = await self._mounted(client, headers, db_session)
entry = await db_session.get(InventoryDevice, "inv-sw")
entry.rack_u_height = 4
entry.rack_col_span = 12
await db_session.commit()
served = (
await client.get(f"/api/v1/racks?design_id={design_id}", headers=headers)
).json()["devices"][0]
assert served["u_height"] == 1
state = _state(design_id, cables=[])
state["devices"] = [{**state["devices"][0], "device_id": "inv-sw", "u_height": 1}]
res = await client.post("/api/v1/racks/save", json=state, headers=headers)
assert res.status_code == 200, res.text
await db_session.refresh(entry)
assert entry.rack_u_height == 4
async def test_a_real_resize_still_reaches_the_model(
self, client: AsyncClient, headers, db_session
):
design_id = await self._mounted(client, headers, db_session)
entry = await db_session.get(InventoryDevice, "inv-sw")
entry.rack_u_height = 1
await db_session.commit()
state = _state(design_id, cables=[])
state["devices"] = [
{**state["devices"][0], "device_id": "inv-sw", "u_start": 8, "u_height": 3}
]
res = await client.post("/api/v1/racks/save", json=state, headers=headers)
assert res.status_code == 200, res.text
await db_session.refresh(entry)
assert entry.rack_u_height == 3
async def test_load_does_not_inflate_a_mount_onto_its_neighbour(
self, client: AsyncClient, headers, db_session
):
# A 4U model fits the rails at U 1 of a 12U rack, but another device sits
# at U 2. Two plates drawn on the same U is not a state the canvas can
# produce, so the load must not invent it.
design_id = await _design(client, headers)
db_session.add(InventoryDevice(id="inv-sw", ip="192.168.1.10", suggested_type="switch"))
await db_session.commit()
state = _state(design_id, cables=[])
state["devices"] = [
{**state["devices"][0], "device_id": "inv-sw", "u_start": 1, "u_height": 1},
{**state["devices"][1], "u_start": 2, "u_height": 1},
]
assert (
await client.post("/api/v1/racks/save", json=state, headers=headers)
).status_code == 200
entry = await db_session.get(InventoryDevice, "inv-sw")
entry.rack_u_height = 4
await db_session.commit()
devices = (
await client.get(f"/api/v1/racks?design_id={design_id}", headers=headers)
).json()["devices"]
assert next(d for d in devices if d["id"] == "dev-sw")["u_height"] == 1
async def test_load_inflates_a_mount_that_has_the_room(
self, client: AsyncClient, headers, db_session
):
design_id = await self._mounted(client, headers, db_session, u_start=1, u_height=1)
entry = await db_session.get(InventoryDevice, "inv-sw")
entry.rack_u_height = 4
await db_session.commit()
device = (
await client.get(f"/api/v1/racks?design_id={design_id}", headers=headers)
).json()["devices"][0]
assert device["u_height"] == 4
class TestSaveAndLoad:
async def test_round_trips_the_full_state(self, client: AsyncClient, headers):
design_id = await _design(client, headers)
@@ -108,6 +312,35 @@ class TestSaveAndLoad:
assert loaded["cables"][0]["from_port_id"] == "p1"
assert loaded["viewport"] == {"x": 10, "y": 20, "zoom": 1.5}
async def test_round_trips_the_port_visibility_override(
self, client: AsyncClient, headers
):
# When a plate draws its sockets is a per-mount display choice, so it
# rides on the mount and not on the inventory row behind it.
design_id = await _design(client, headers)
state = _state(design_id, cables=[])
state["devices"][0]["port_visibility"] = "hover"
state["devices"][1]["port_visibility"] = "always"
res = await client.post("/api/v1/racks/save", json=state, headers=headers)
assert res.status_code == 200, res.text
loaded = (await client.get(f"/api/v1/racks?design_id={design_id}", headers=headers)).json()
by_id = {d["id"]: d for d in loaded["devices"]}
assert by_id["dev-sw"]["port_visibility"] == "hover"
assert by_id["dev-nas"]["port_visibility"] == "always"
async def test_defaults_port_visibility_to_auto(self, client: AsyncClient, headers):
# A payload from before the field existed — and any unknown value —
# keeps the old behaviour: the faceplate decides.
design_id = await _design(client, headers)
state = _state(design_id, cables=[])
state["devices"][1]["port_visibility"] = "sometimes"
res = await client.post("/api/v1/racks/save", json=state, headers=headers)
assert res.status_code == 200, res.text
loaded = (await client.get(f"/api/v1/racks?design_id={design_id}", headers=headers)).json()
assert {d["port_visibility"] for d in loaded["devices"]} == {"auto"}
async def test_keeps_a_mount_that_follows_its_node_check(
self, client: AsyncClient, headers
):
@@ -802,3 +1035,35 @@ class TestDesignLifecycle:
copied_ids = {d["id"] for d in loaded["devices"]}
assert loaded["cables"][0]["from_device_id"] in copied_ids
assert loaded["cables"][0]["to_device_id"] in copied_ids
class TestOverlappingMounts:
async def test_rejects_two_mounts_claiming_the_same_u(self, client: AsyncClient, headers):
design_id = await _design(client, headers)
state = _state(design_id, cables=[])
state["devices"][1]["u_start"] = 10 # straight onto dev-sw
res = await client.post("/api/v1/racks/save", json=state, headers=headers)
assert res.status_code == 422
assert "overlap" in res.text
async def test_accepts_two_mounts_sharing_a_u_side_by_side(
self, client: AsyncClient, headers
):
# Half-width plates on the same U are the whole point of the column grid.
design_id = await _design(client, headers)
state = _state(design_id, cables=[])
state["devices"][0].update(col_start=0, col_span=6)
state["devices"][1].update(u_start=10, u_height=1, col_start=6, col_span=6)
res = await client.post("/api/v1/racks/save", json=state, headers=headers)
assert res.status_code == 200, res.text
async def test_lets_two_racks_use_the_same_u(self, client: AsyncClient, headers):
design_id = await _design(client, headers)
state = _state(design_id, cables=[])
state["racks"].append({**state["racks"][0], "id": "rack-2", "name": "Second"})
state["devices"][1].update(rack_id="rack-2", u_start=10, u_height=1)
res = await client.post("/api/v1/racks/save", json=state, headers=headers)
assert res.status_code == 200, res.text
+103
View File
@@ -0,0 +1,103 @@
"""GET /api/v1/scan/pending/{id}/proxmox-children.
Resolves the host -> guest links the Proxmox import records into the inventory
rows the UI needs to offer "add the guests too" when a host reaches a canvas.
"""
import pytest
from httpx import AsyncClient
from sqlalchemy.ext.asyncio import AsyncSession
from app.db.models import InventoryDevice, InventoryDeviceLink
async def _seed(db: AsyncSession) -> dict[str, str]:
host = InventoryDevice(
ieee_address="pve-node-pve1", hostname="pve1", suggested_type="proxmox", status="pending"
)
vm = InventoryDevice(
ieee_address="pve-pve1-101", hostname="web", suggested_type="vm", status="pending"
)
lxc = InventoryDevice(
ieee_address="pve-pve1-102", hostname="dns", suggested_type="lxc", status="pending"
)
other = InventoryDevice(ip="192.168.1.9", hostname="nas", status="pending")
db.add_all([host, vm, lxc, other])
await db.flush()
db.add_all([
InventoryDeviceLink(
source_ieee="pve-node-pve1", target_ieee="pve-pve1-101", discovery_source="proxmox"
),
InventoryDeviceLink(
source_ieee="pve-node-pve1", target_ieee="pve-pve1-102", discovery_source="proxmox"
),
])
await db.commit()
return {"host": host.id, "vm": vm.id, "lxc": lxc.id, "other": other.id}
@pytest.mark.asyncio
async def test_returns_the_guests_of_a_host(client: AsyncClient, headers, db_session):
ids = await _seed(db_session)
res = await client.get(f"/api/v1/scan/pending/{ids['host']}/proxmox-children", headers=headers)
assert res.status_code == 200
assert {d["id"] for d in res.json()} == {ids["vm"], ids["lxc"]}
@pytest.mark.asyncio
async def test_empty_for_a_guest(client: AsyncClient, headers, db_session):
ids = await _seed(db_session)
res = await client.get(f"/api/v1/scan/pending/{ids['vm']}/proxmox-children", headers=headers)
assert res.status_code == 200
assert res.json() == []
@pytest.mark.asyncio
async def test_empty_for_a_device_with_no_ieee(client: AsyncClient, headers, db_session):
ids = await _seed(db_session)
res = await client.get(f"/api/v1/scan/pending/{ids['other']}/proxmox-children", headers=headers)
assert res.status_code == 200
assert res.json() == []
@pytest.mark.asyncio
async def test_skips_hidden_guests(client: AsyncClient, headers, db_session):
ids = await _seed(db_session)
lxc = await db_session.get(InventoryDevice, ids["lxc"])
lxc.status = "hidden"
await db_session.commit()
res = await client.get(f"/api/v1/scan/pending/{ids['host']}/proxmox-children", headers=headers)
assert [d["id"] for d in res.json()] == [ids["vm"]]
@pytest.mark.asyncio
async def test_ignores_cluster_links(client: AsyncClient, headers, db_session):
"""A host↔host cluster link is not a parent/child relation."""
ids = await _seed(db_session)
peer = InventoryDevice(
ieee_address="pve-node-pve2", hostname="pve2", suggested_type="proxmox", status="pending"
)
db_session.add(peer)
db_session.add(
InventoryDeviceLink(
source_ieee="pve-node-pve1",
target_ieee="pve-node-pve2",
discovery_source="proxmox_cluster",
)
)
await db_session.commit()
res = await client.get(f"/api/v1/scan/pending/{ids['host']}/proxmox-children", headers=headers)
assert {d["id"] for d in res.json()} == {ids["vm"], ids["lxc"]}
@pytest.mark.asyncio
async def test_404_for_an_unknown_device(client: AsyncClient, headers):
res = await client.get("/api/v1/scan/pending/nope/proxmox-children", headers=headers)
assert res.status_code == 404
@pytest.mark.asyncio
async def test_requires_auth(client: AsyncClient, db_session):
ids = await _seed(db_session)
res = await client.get(f"/api/v1/scan/pending/{ids['host']}/proxmox-children")
assert res.status_code == 401
+693
View File
@@ -1,5 +1,6 @@
"""Tests for scanner: two-phase nmap, mDNS discovery, run_scan integration."""
import uuid
from datetime import datetime, timezone
from unittest.mock import AsyncMock, MagicMock, patch
import pytest
@@ -375,6 +376,79 @@ def test_nmap_scan_single_non_root_uses_connect_scan():
assert result["open_ports"][0]["banner"] == "nginx 1.24"
def test_nmap_scan_single_discovery_is_unbounded_by_default():
"""The range scan's discovery pass keeps its authoritative, untimed run."""
from app.services.scanner import _nmap_scan_single
host = {"ip": "192.168.1.14", "hostname": None, "mac": None, "os": None, "open_ports": []}
disc = _fake_scanner("192.168.1.14", {})
with patch("app.services.scanner.nmap.PortScanner", side_effect=[disc]), \
patch("app.services.scanner.os.geteuid", return_value=1000):
_nmap_scan_single(host)
args = disc.scan.call_args.kwargs["arguments"]
assert "--host-timeout" not in args
assert "--min-rate" not in args
def test_nmap_scan_single_bounded_never_sets_a_host_timeout():
"""A deep slice drops retries — never --host-timeout.
nmap answers a host timeout with "Skipping host <ip> due to host timeout"
and throws away every port it had already found, so a ceiling here turns a
slow scan into one that reports nothing. The time on a dropping host goes to
the retry pass — measured 2x — so that is what the deep scan gives up.
"""
from app.services.scanner import _nmap_scan_single
host = {"ip": "192.168.1.15", "hostname": None, "mac": None, "os": None, "open_ports": []}
disc = _fake_scanner("192.168.1.15", {})
with patch("app.services.scanner.nmap.PortScanner", side_effect=[disc]), \
patch("app.services.scanner.os.geteuid", return_value=1000):
_nmap_scan_single(host, "1-8192", True)
args = disc.scan.call_args.kwargs["arguments"]
assert "-p 1-8192" in args
assert "--host-timeout" not in args
assert "--max-retries 0" in args
def test_deep_port_chunks_cover_every_port_once():
from app.services.scanner import _deep_port_chunks
chunks = _deep_port_chunks(8192)
assert chunks[0] == "1-8192"
assert chunks[-1].endswith("-65535")
covered = []
for c in chunks:
lo, hi = (int(x) for x in c.split("-"))
covered.extend(range(lo, hi + 1))
assert covered == list(range(1, 65536))
def test_port_chunks_pack_ranges_and_honour_the_slice_size():
from app.services.scanner import _port_chunks
# Small ranges share one call instead of one call each.
assert _port_chunks("80,443,8000-9000") == ["80,443,8000-9000"]
# A range wider than the slice is cut at the slice boundary.
assert _port_chunks("1-100", 40) == ["1-40", "41-80", "81-100"]
# Overlapping input is merged before slicing, so no port is scanned twice.
assert _port_chunks("1-100,50-200", 1000) == ["1-200"]
assert _port_chunks("nonsense") == []
def test_parse_port_spec_rejects_what_nmap_could_not_use():
from app.services.scanner import _parse_port_spec, _valid_port_spec
for bad in ["", " ", "0", "65536", "100-50", "80,", "http", "-80"]:
assert _parse_port_spec(bad) == [], bad
assert _valid_port_spec(bad) is False, bad
assert _valid_port_spec("80,443,8000-9000") is True
# ---------------------------------------------------------------------------
# _nmap_scan
# ---------------------------------------------------------------------------
@@ -750,6 +824,48 @@ async def test_run_scan_merges_proxmox_row_by_mac(mem_db):
assert set(row.discovery_sources) == {"proxmox", "arp"} # both filters
@pytest.mark.asyncio
async def test_run_scan_keeps_services_the_fingerprint_cannot_see(mem_db):
"""A re-scan unions its fingerprint onto the row — it never replaces it.
Since 3.3.0 the row is the only copy of a device's services and every canvas
drawing it reads that list, so overwriting it with what nmap happened to
match would delete hand-added services everywhere at once (#347).
"""
from app.services.scanner import run_scan
run_id = _make_run_id()
async with mem_db() as session:
session.add(_make_scan_run(run_id))
session.add(InventoryDevice(
id="row-1", ip="192.168.1.60", status="approved",
discovery_source="arp", discovery_sources=["arp"],
services=[
{"port": 9000, "protocol": "tcp", "service_name": "Portainer", "path": "/#!/home"},
{"port": 22, "protocol": "tcp", "service_name": "ssh"},
],
))
await session.commit()
nmap_hosts = [{"ip": "192.168.1.60", "hostname": "docker.lan", "mac": None, "os": None,
"open_ports": [{"port": 22, "protocol": "tcp", "banner": "OpenSSH 9.2"}]}]
async with mem_db() as session:
with patch("app.services.scanner._nmap_scan", return_value=nmap_hosts), \
patch("app.services.scanner._mdns_discover", new_callable=AsyncMock, return_value=[]), \
patch("app.api.routes.status.broadcast_scan_update", new_callable=AsyncMock):
await run_scan(["192.168.1.0/24"], session, run_id)
async with mem_db() as session:
row = await session.get(InventoryDevice, "row-1")
by_port = {s["port"]: s for s in row.services}
assert by_port[9000]["service_name"] == "Portainer" # hand-added, untouched
assert by_port[9000]["path"] == "/#!/home"
assert 22 in by_port # what the scan saw is still there
assert row.status == "approved"
@pytest.mark.asyncio
async def test_run_scan_mdns_skipped_if_already_in_nmap(mem_db):
"""If nmap and mDNS both find the same IP, it should not be double-counted."""
@@ -988,3 +1104,580 @@ async def test_run_scan_no_probe_when_disabled(mem_db):
await run_scan(["192.168.1.0/24"], session, run_id)
probe.assert_not_called()
# ---------------------------------------------------------------------------
# run_device_scan — per-device deep rescan (issue #350)
# ---------------------------------------------------------------------------
def test_build_port_spec_full_covers_every_tcp_port():
from app.services.scanner import _build_port_spec
# full wins over the curated list *and* over user ranges — the deep rescan
# is only worth its minutes if it really scans everything.
assert _build_port_spec(None, full=True) == "1-65535"
assert _build_port_spec(["8000-8100"], full=True) == "1-65535"
@pytest.mark.asyncio
async def test_run_device_scan_refreshes_services_and_marks_run_done(mem_db):
from app.services.scanner import run_device_scan
run_id = _make_run_id()
async with mem_db() as session:
session.add(ScanRun(id=run_id, status="running", kind="device", ranges=["192.168.1.9/32"]))
session.add(InventoryDevice(id="d1", ip="192.168.1.9", status="approved", services=[]))
await session.commit()
seen_specs = []
def _scanned(host_dict, port_spec, bounded=False):
seen_specs.append(port_spec)
# A full-range rescan is always bounded, or it never ends.
assert bounded is True
# Only the slice holding 22 reports it — the union is the caller's job.
if port_spec == "1-8192":
host_dict["open_ports"] = [{"port": 22, "protocol": "tcp", "banner": "OpenSSH 9.2"}]
return host_dict
async with mem_db() as session:
with patch("app.services.scanner._nmap_scan_single", side_effect=_scanned), \
patch("app.api.routes.status.broadcast_scan_update", new_callable=AsyncMock):
await run_device_scan("d1", session, run_id)
async with mem_db() as session:
device = await session.get(InventoryDevice, "d1")
run = await session.get(ScanRun, run_id)
assert device is not None and run is not None
assert run.status == "done"
assert device.last_scan is not None
assert any(s.get("port") == 22 for s in device.services)
# A rescan refreshes an existing row; it never spawns a second one.
assert device.status == "approved"
@pytest.mark.asyncio
async def test_run_device_scan_keeps_hand_added_services(mem_db):
"""Regression: the rescan unions, it does not replace.
Services the user typed in by hand are the only copy that exists — every
canvas drawing the device reads this list.
"""
from app.services.scanner import run_device_scan
run_id = _make_run_id()
hand_added = {"port": 9000, "protocol": "tcp", "service_name": "My App"}
async with mem_db() as session:
session.add(ScanRun(id=run_id, status="running", kind="device", ranges=["192.168.1.9/32"]))
session.add(InventoryDevice(id="d1", ip="192.168.1.9", status="pending", services=[hand_added]))
await session.commit()
def _scanned(host_dict, port_spec, bounded=False):
host_dict["open_ports"] = [{"port": 22, "protocol": "tcp", "banner": "OpenSSH 9.2"}]
return host_dict
async with mem_db() as session:
with patch("app.services.scanner._nmap_scan_single", side_effect=_scanned), \
patch("app.api.routes.status.broadcast_scan_update", new_callable=AsyncMock):
await run_device_scan("d1", session, run_id)
async with mem_db() as session:
device = await session.get(InventoryDevice, "d1")
assert device is not None
ports = {s.get("port") for s in device.services}
assert ports == {22, 9000}
@pytest.mark.asyncio
async def test_scan_keeps_the_icon_the_user_picked_for_a_service(mem_db):
"""Regression: every scan used to repaint hand-picked service icons.
The fingerprint guesses an icon from the port; the user's choice is the
only one that means anything, so a rescan leaves it where it is.
"""
from app.services.scanner import run_device_scan
run_id = _make_run_id()
curated = {
"port": 22,
"protocol": "tcp",
"service_name": "ssh",
"icon": "brand:openssh",
"category": "remote",
}
async with mem_db() as session:
session.add(ScanRun(id=run_id, status="running", kind="device", ranges=["192.168.1.9/32"]))
session.add(InventoryDevice(id="d1", ip="192.168.1.9", status="approved", services=[curated]))
await session.commit()
def _scanned(host_dict, port_spec, bounded=False):
host_dict["open_ports"] = [{"port": 22, "protocol": "tcp", "banner": "OpenSSH 9.2"}]
return host_dict
async with mem_db() as session:
with patch("app.services.scanner._nmap_scan_single", side_effect=_scanned), \
patch("app.api.routes.status.broadcast_scan_update", new_callable=AsyncMock):
await run_device_scan("d1", session, run_id)
async with mem_db() as session:
device = await session.get(InventoryDevice, "d1")
assert device is not None
matches = [s for s in device.services if s.get("port") == 22]
# Merged in place — the name key is case-insensitive, so "SSH" from the
# signature does not append a second row next to the user's "ssh".
assert len(matches) == 1
ssh = matches[0]
assert ssh["icon"] == "brand:openssh"
assert ssh["category"] == "remote"
@pytest.mark.asyncio
async def test_run_device_scan_keeps_the_device_own_discovery_source(mem_db):
"""A rescan re-observes a device; it does not discover it on the network.
Tagging every rescanned device "arp" told a Proxmox guest or a hand-added
host that the network scanner found it, and it then answered that filter.
"""
from app.services.scanner import run_device_scan
run_id = _make_run_id()
async with mem_db() as session:
session.add(ScanRun(id=run_id, status="running", kind="device", ranges=["192.168.1.9/32"]))
session.add(InventoryDevice(
id="d1",
ip="192.168.1.9",
status="approved",
services=[],
discovery_source="proxmox",
discovery_sources=["proxmox"],
))
await session.commit()
def _scanned(host_dict, port_spec, bounded=False):
host_dict["open_ports"] = [{"port": 8006, "protocol": "tcp", "banner": ""}]
return host_dict
async with mem_db() as session:
with patch("app.services.scanner._nmap_scan_single", side_effect=_scanned), \
patch("app.api.routes.status.broadcast_scan_update", new_callable=AsyncMock):
await run_device_scan("d1", session, run_id, ports="8006")
async with mem_db() as session:
device = await session.get(InventoryDevice, "d1")
assert device is not None
assert device.discovery_sources == ["proxmox"]
assert device.discovery_source == "proxmox"
@pytest.mark.asyncio
async def test_run_device_scan_marks_run_error_when_device_has_no_ip(mem_db):
from app.services.scanner import run_device_scan
run_id = _make_run_id()
async with mem_db() as session:
session.add(ScanRun(id=run_id, status="running", kind="device", ranges=["/32"]))
session.add(InventoryDevice(id="d1", ip=None, status="pending", services=[]))
await session.commit()
async with mem_db() as session:
with patch("app.api.routes.status.broadcast_scan_update", new_callable=AsyncMock):
await run_device_scan("d1", session, run_id)
async with mem_db() as session:
run = await session.get(ScanRun, run_id)
assert run is not None
assert run.status == "error"
assert run.error is not None
@pytest.mark.asyncio
async def test_run_device_scan_skips_nmap_when_cancelled(mem_db):
from app.services.scanner import _cancelled_runs, request_cancel, run_device_scan
run_id = _make_run_id()
async with mem_db() as session:
session.add(ScanRun(id=run_id, status="running", kind="device", ranges=["192.168.1.9/32"]))
session.add(InventoryDevice(id="d1", ip="192.168.1.9", status="pending", services=[]))
await session.commit()
request_cancel(run_id)
single = MagicMock()
async with mem_db() as session:
with patch("app.services.scanner._nmap_scan_single", new=single), \
patch("app.api.routes.status.broadcast_scan_update", new_callable=AsyncMock):
await run_device_scan("d1", session, run_id)
async with mem_db() as session:
run = await session.get(ScanRun, run_id)
single.assert_not_called()
assert run is not None
assert run.status == "cancelled"
# The run cleans up its cancellation flag on the way out.
assert run_id not in _cancelled_runs
@pytest.mark.asyncio
async def test_run_device_scan_unions_ports_across_slices(mem_db):
"""Every slice contributes; a slow one costs only its own ports."""
from app.services.scanner import _deep_port_chunks, run_device_scan
run_id = _make_run_id()
async with mem_db() as session:
session.add(ScanRun(id=run_id, status="running", kind="device", ranges=["192.168.1.9/32"]))
session.add(InventoryDevice(id="d1", ip="192.168.1.9", status="pending", services=[]))
await session.commit()
def _scanned(host_dict, port_spec, bounded=False):
lo = int(port_spec.split("-")[0])
if lo == 1:
host_dict["open_ports"] = [{"port": 22, "protocol": "tcp", "banner": ""}]
elif lo == 8193:
host_dict["open_ports"] = [{"port": 8096, "protocol": "tcp", "banner": ""}]
return host_dict
async with mem_db() as session:
with patch("app.services.scanner._nmap_scan_single", side_effect=_scanned), \
patch("app.api.routes.status.broadcast_scan_update", new_callable=AsyncMock):
await run_device_scan("d1", session, run_id)
async with mem_db() as session:
device = await session.get(InventoryDevice, "d1")
run = await session.get(ScanRun, run_id)
assert device is not None and run is not None
assert {s.get("port") for s in device.services} == {22, 8096}
# A complete sweep carries no advisory.
assert run.error is None
assert len(_deep_port_chunks()) == 8
@pytest.mark.asyncio
async def test_run_device_scan_honours_a_requested_port_range(mem_db):
"""A user-chosen range replaces the full sweep — and skips retry-free timing.
A handful of ports is cheap enough to scan properly; the bounded flags only
pay off over thousands of them.
"""
from app.services.scanner import run_device_scan
run_id = _make_run_id()
async with mem_db() as session:
session.add(ScanRun(id=run_id, status="running", kind="device", ranges=["192.168.1.9/32"]))
session.add(InventoryDevice(id="d1", ip="192.168.1.9", status="pending", services=[]))
await session.commit()
calls = []
def _scanned(host_dict, port_spec, bounded=False):
calls.append((port_spec, bounded))
host_dict["open_ports"] = [{"port": 8096, "protocol": "tcp", "banner": ""}]
return host_dict
async with mem_db() as session:
with patch("app.services.scanner._nmap_scan_single", side_effect=_scanned), \
patch("app.api.routes.status.broadcast_scan_update", new_callable=AsyncMock):
await run_device_scan("d1", session, run_id, ports="8000-9000")
async with mem_db() as session:
device = await session.get(InventoryDevice, "d1")
run = await session.get(ScanRun, run_id)
assert calls == [("8000-9000", False)]
assert device is not None and run is not None
assert {s.get("port") for s in device.services} == {8096}
assert run.status == "done"
assert run.error is None
@pytest.mark.asyncio
async def test_run_device_scan_keeps_what_it_found_when_the_budget_runs_out(mem_db):
"""A spent budget stops the sweep — it never discards the ports found.
The earlier --host-timeout did exactly that (nmap skips the host wholesale),
which is why a deep scan could come back empty on a slow host.
"""
from app.services.scanner import run_device_scan
run_id = _make_run_id()
async with mem_db() as session:
session.add(ScanRun(id=run_id, status="running", kind="device", ranges=["192.168.1.9/32"]))
session.add(InventoryDevice(id="d1", ip="192.168.1.9", status="pending", services=[]))
await session.commit()
calls = []
def _scanned(host_dict, port_spec, bounded=False):
calls.append(port_spec)
host_dict["open_ports"] = [{"port": 22, "protocol": "tcp", "banner": ""}]
return host_dict
async with mem_db() as session:
# Budget already spent when the first slice returns.
with patch("app.services.scanner._nmap_scan_single", side_effect=_scanned), \
patch("app.services.scanner.settings") as mock_settings, \
patch("app.api.routes.status.broadcast_scan_update", new_callable=AsyncMock):
mock_settings.scanner_deep_host_timeout = -1
await run_device_scan("d1", session, run_id)
async with mem_db() as session:
device = await session.get(InventoryDevice, "d1")
run = await session.get(ScanRun, run_id)
assert calls == ["1-8192"]
assert device is not None and run is not None
assert {s.get("port") for s in device.services} == {22}
assert run.status == "done"
# Partial coverage is reported, never passed off as a full sweep.
assert run.error is not None
assert "1/8" in run.error
# ---------------------------------------------------------------------------
# Multi-IP devices — one row per device, not one per address (issue #373)
# ---------------------------------------------------------------------------
def _host(ip: str, *, mac: str | None = None, hostname: str | None = None):
return {"ip": ip, "hostname": hostname, "mac": mac, "os": None, "open_ports": []}
@pytest.mark.asyncio
async def test_process_host_folds_into_declared_secondary_ip(mem_db):
"""A scan of a device's second address refreshes its row, never mints one.
The user declares the extra addresses by hand — a router's LAN and VLAN
legs, a host behind a VIP — precisely so the scanner stops reporting them
as separate discoveries.
"""
from app.services.scanner import DeepScanOptions, process_host
async with mem_db() as session:
session.add(InventoryDevice(
id="d1", ip="192.168.1.1, 192.168.10.1", status="approved", label="router",
))
await session.commit()
async with mem_db() as session:
outcome = await process_host(
session, _host("192.168.10.1", hostname="router-vlan.lan"),
hidden_ips=set(), deep_scan=DeepScanOptions(),
)
async with mem_db() as session:
devices = (await session.execute(sa_select(InventoryDevice))).scalars().all()
assert outcome == "updated"
assert len(devices) == 1
# The declared list is the user's, not the scanner's — it survives the merge.
assert devices[0].ip == "192.168.1.1, 192.168.10.1"
assert devices[0].hostname == "router-vlan.lan"
@pytest.mark.asyncio
async def test_process_host_does_not_match_an_address_prefix(mem_db):
"""`192.168.1.4` is a substring of `192.168.1.40` — and a different host.
The query narrows with a substring match, so the per-token confirmation is
what stops a false match. It matters more than usual here: the losing rows
of a match are deleted, not skipped.
"""
from app.services.scanner import DeepScanOptions, process_host
async with mem_db() as session:
session.add(InventoryDevice(id="d1", ip="192.168.1.40", status="approved"))
await session.commit()
async with mem_db() as session:
outcome = await process_host(
session, _host("192.168.1.4"), hidden_ips=set(), deep_scan=DeepScanOptions(),
)
async with mem_db() as session:
devices = (await session.execute(sa_select(InventoryDevice))).scalars().all()
assert outcome == "created"
assert {d.ip for d in devices} == {"192.168.1.40", "192.168.1.4"}
@pytest.mark.asyncio
async def test_process_host_matches_a_single_token_row_exactly_as_before(mem_db):
"""The ordinary one-address case is untouched by the token matching."""
from app.services.scanner import DeepScanOptions, process_host
async with mem_db() as session:
session.add(InventoryDevice(id="d1", ip="192.168.1.5", status="pending"))
await session.commit()
async with mem_db() as session:
outcome = await process_host(
session, _host("192.168.1.5", hostname="nas.lan"),
hidden_ips=set(), deep_scan=DeepScanOptions(),
)
async with mem_db() as session:
devices = (await session.execute(sa_select(InventoryDevice))).scalars().all()
assert outcome == "updated"
assert len(devices) == 1 and devices[0].hostname == "nas.lan"
@pytest.mark.asyncio
async def test_run_scan_skips_the_secondary_ip_of_a_hidden_device(mem_db):
"""Hiding a multi-address device hides it at every one of its addresses."""
from app.services.scanner import run_scan
run_id = _make_run_id()
async with mem_db() as session:
session.add(_make_scan_run(run_id))
session.add(InventoryDevice(id="d1", ip="192.168.1.1, 192.168.10.1", status="hidden"))
await session.commit()
async with mem_db() as session:
with patch("app.services.scanner._nmap_scan", return_value=[_host("192.168.10.1")]), \
patch("app.services.scanner._mdns_discover", new_callable=AsyncMock, return_value=[]), \
patch("app.api.routes.status.broadcast_scan_update", new_callable=AsyncMock):
await run_scan(["192.168.1.0/24"], session, run_id)
async with mem_db() as session:
devices = (await session.execute(sa_select(InventoryDevice))).scalars().all()
assert len(devices) == 1
assert devices[0].status == "hidden"
@pytest.mark.asyncio
async def test_dedupe_collapses_a_single_ip_row_into_a_multi_ip_one(mem_db):
"""Declaring an extra address folds in the duplicate row it already spawned.
The approved row wins even though it is the younger of the two: it carries
the canvas links.
"""
from app.services.scanner import _dedupe_pending_by_ip
async with mem_db() as session:
session.add(InventoryDevice(
id="stale", ip="192.168.10.1", status="pending",
discovered_at=datetime(2026, 1, 1, tzinfo=timezone.utc),
))
session.add(InventoryDevice(
id="keep", ip="192.168.1.1, 192.168.10.1", status="approved",
discovered_at=datetime(2026, 2, 1, tzinfo=timezone.utc),
))
await session.commit()
async with mem_db() as session:
deleted = await _dedupe_pending_by_ip(session)
async with mem_db() as session:
devices = (await session.execute(sa_select(InventoryDevice))).scalars().all()
assert deleted == 1
assert [d.id for d in devices] == ["keep"]
@pytest.mark.asyncio
async def test_dedupe_leaves_rows_that_share_no_address(mem_db):
from app.services.scanner import _dedupe_pending_by_ip
async with mem_db() as session:
session.add(InventoryDevice(id="a", ip="192.168.1.1, 192.168.10.1", status="pending"))
session.add(InventoryDevice(id="b", ip="192.168.1.2, 192.168.10.2", status="pending"))
await session.commit()
async with mem_db() as session:
deleted = await _dedupe_pending_by_ip(session)
async with mem_db() as session:
devices = (await session.execute(sa_select(InventoryDevice))).scalars().all()
assert deleted == 0
assert len(devices) == 2
def test_group_by_shared_ip_is_transitive():
"""`A,B` next to `A` and `B` is one device, not two pairs."""
from app.services.scanner import _group_by_shared_ip
rows = [
InventoryDevice(id="a", ip="10.0.0.1"),
InventoryDevice(id="b", ip="10.0.1.1"),
InventoryDevice(id="bridge", ip="10.0.0.1, 10.0.1.1"),
InventoryDevice(id="other", ip="10.0.2.1"),
]
groups = {frozenset(r.id for r in group) for group in _group_by_shared_ip(rows)}
assert groups == {frozenset({"a", "b", "bridge"}), frozenset({"other"})}
@pytest.mark.asyncio
async def test_dedupe_never_deletes_a_second_approved_row(mem_db):
"""Two approved rows sharing an address both keep their canvas links.
An approved row is what nodes and rack mounts point at, so collapsing one
into the other would silently unlink whatever drew it. The scan collapses
the pending row and leaves the pair for the user to resolve.
"""
from app.services.scanner import _dedupe_pending_by_ip
async with mem_db() as session:
session.add(InventoryDevice(
id="a", ip="10.0.0.1", status="approved",
discovered_at=datetime(2026, 1, 1, tzinfo=timezone.utc),
))
session.add(InventoryDevice(
id="b", ip="10.0.0.1, 10.0.0.2", status="approved",
discovered_at=datetime(2026, 2, 1, tzinfo=timezone.utc),
))
session.add(InventoryDevice(
id="pending", ip="10.0.0.2", status="pending",
discovered_at=datetime(2026, 3, 1, tzinfo=timezone.utc),
))
await session.commit()
async with mem_db() as session:
deleted = await _dedupe_pending_by_ip(session)
async with mem_db() as session:
devices = (await session.execute(sa_select(InventoryDevice))).scalars().all()
assert deleted == 1
assert sorted(d.id for d in devices) == ["a", "b"]
@pytest.mark.asyncio
async def test_process_host_never_deletes_a_second_approved_row(mem_db):
"""The same guard on the scan path: the merge target refreshes, no row goes."""
from app.services.scanner import DeepScanOptions, process_host
async with mem_db() as session:
session.add(InventoryDevice(
id="a", ip="10.0.0.1", status="approved",
discovered_at=datetime(2026, 1, 1, tzinfo=timezone.utc),
))
session.add(InventoryDevice(
id="b", ip="10.0.0.1, 10.0.0.2", status="approved",
discovered_at=datetime(2026, 2, 1, tzinfo=timezone.utc),
))
await session.commit()
async with mem_db() as session:
outcome = await process_host(
session, _host("10.0.0.1", hostname="claimed.lan"),
hidden_ips=set(), deep_scan=DeepScanOptions(),
)
async with mem_db() as session:
devices = (await session.execute(sa_select(InventoryDevice))).scalars().all()
oldest = await session.get(InventoryDevice, "a")
assert outcome == "updated"
assert sorted(d.id for d in devices) == ["a", "b"]
# The oldest approved row is the merge target.
assert oldest is not None and oldest.hostname == "claimed.lan"
+92
View File
@@ -0,0 +1,92 @@
"""Detaching nodes recorded as their own parent (#370).
A ``parent_id = id`` row is fatal on the canvas: every parent walk assumes an
acyclic tree, so dragging the node overflowed the stack inside the change
reducer and the move was silently dropped — the node selected but would not
move. It also renders unparented, sitting wherever its stored coordinates put
it rather than inside the container it appears to belong to.
The repair runs at startup because the bad rows are already in users' databases;
the write-path guards shipped alongside it only stop new ones.
"""
import pytest
from sqlalchemy.ext.asyncio import create_async_engine
import app.db.database as database
pytestmark = pytest.mark.asyncio
_DDL = "CREATE TABLE nodes (id VARCHAR PRIMARY KEY, label VARCHAR, parent_id VARCHAR)"
async def _engine(tmp_path, rows):
engine = create_async_engine(f"sqlite+aiosqlite:///{tmp_path / 'selfparent.db'}")
async with engine.begin() as conn:
await conn.exec_driver_sql(_DDL)
for node_id, parent_id in rows:
await conn.exec_driver_sql(
"INSERT INTO nodes (id, label, parent_id) VALUES (?, ?, ?)",
(node_id, node_id, parent_id),
)
return engine
async def _parents(engine) -> dict[str, str | None]:
async with engine.begin() as conn:
rows = (await conn.exec_driver_sql("SELECT id, parent_id FROM nodes")).fetchall()
return {r[0]: r[1] for r in rows}
async def _run(engine, monkeypatch) -> None:
monkeypatch.setattr(database, "engine", engine)
await database._repair_self_parent_nodes()
async def test_detaches_a_self_parented_node(tmp_path, monkeypatch):
engine = await _engine(tmp_path, [("pihole", "pihole")])
await _run(engine, monkeypatch)
assert (await _parents(engine))["pihole"] is None
async def test_leaves_real_parents_alone(tmp_path, monkeypatch):
engine = await _engine(
tmp_path,
[("pve", None), ("vm", "pve"), ("broken", "broken")],
)
await _run(engine, monkeypatch)
parents = await _parents(engine)
assert parents == {"pve": None, "vm": "pve", "broken": None}
async def test_is_idempotent(tmp_path, monkeypatch):
engine = await _engine(tmp_path, [("a", "a"), ("b", None)])
await _run(engine, monkeypatch)
await _run(engine, monkeypatch)
assert await _parents(engine) == {"a": None, "b": None}
async def test_does_not_touch_a_two_node_cycle(tmp_path, monkeypatch):
"""Only the exact self-parent case is repairable here.
A longer cycle has no single right answer for which link to cut, and the
runtime cycle guards keep the canvas usable, so the rows are left as they
are rather than guessed at.
"""
engine = await _engine(tmp_path, [("a", "b"), ("b", "a")])
await _run(engine, monkeypatch)
assert await _parents(engine) == {"a": "b", "b": "a"}
async def test_survives_a_missing_table(tmp_path, monkeypatch):
"""Boot must not die on a repair; the canvas works either way."""
engine = create_async_engine(f"sqlite+aiosqlite:///{tmp_path / 'empty.db'}")
await _run(engine, monkeypatch)
+54
View File
@@ -255,3 +255,57 @@ def test_valid_oidc_configuration_is_accepted():
configured = _valid_oidc_settings()
assert configured.auth_mode == "oidc"
assert configured.oidc_cookie_secure is True
# --- Env values arriving still quoted (podman-compose, issue #410) ---
def test_quoted_password_hash_is_unwrapped():
"""podman-compose passes .env quotes through verbatim; Docker Compose strips them."""
configured = Settings(
_env_file=None,
secret_key="test-secret",
auth_password_hash="'$2b$12$RtMbyw17l4N5UGzeXMNAWuzCaVV.XFBY7ZetWheQhxcBDcxahapkG'",
)
assert configured.auth_password_hash == "$2b$12$RtMbyw17l4N5UGzeXMNAWuzCaVV.XFBY7ZetWheQhxcBDcxahapkG"
def test_double_quoted_password_hash_is_unwrapped():
configured = Settings(
_env_file=None,
secret_key="test-secret",
auth_password_hash='"$2b$12$RtMbyw17l4N5UGzeXMNAWuzCaVV.XFBY7ZetWheQhxcBDcxahapkG"',
)
assert configured.auth_password_hash.startswith("$2b$12$")
def test_unquoted_password_hash_is_left_alone():
raw = "$2b$12$RtMbyw17l4N5UGzeXMNAWuzCaVV.XFBY7ZetWheQhxcBDcxahapkG"
configured = Settings(_env_file=None, secret_key="test-secret", auth_password_hash=raw)
assert configured.auth_password_hash == raw
def test_unmatched_quote_is_not_stripped():
"""Only a matched pair is unwrapped, so a truly broken value still fails validation."""
configured = Settings(
_env_file=None,
secret_key="test-secret",
auth_password_hash="'$2b$12$RtMbyw17l4N5UGzeXMNAWu",
)
assert configured.auth_password_hash.startswith("'")
def test_quoted_secret_key_is_unwrapped():
configured = _valid_oidc_settings(secret_key="'test-secret-key-that-is-at-least-32-bytes'")
assert configured.secret_key == "test-secret-key-that-is-at-least-32-bytes"
def test_secret_key_length_is_checked_after_unwrapping():
"""The quotes must not pad a too-short key past the 32-byte OIDC floor."""
with pytest.raises(ValueError, match="SECRET_KEY must be at least 32 bytes"):
_valid_oidc_settings(secret_key="'" + "a" * 31 + "'")
def test_quoted_oidc_client_secret_is_unwrapped():
configured = _valid_oidc_settings(oidc_client_secret="'sup3r$ecret'")
assert configured.oidc_client_secret == "sup3r$ecret"
+104 -13
View File
@@ -1,6 +1,7 @@
"""Tests for status_checker service: each check method."""
from unittest.mock import AsyncMock, MagicMock, patch
import httpx
import pytest
from app.services.status_checker import (
@@ -13,16 +14,31 @@ from app.services.status_checker import (
)
class _TransportClient:
"""
Patch target for httpx.AsyncClient that routes through a MockTransport.
Real client, fake network: _http_get exercises the genuine httpx request
path (including .stream()) instead of a mock that would happily accept any
call shape.
"""
def __init__(self, handler):
self._handler = handler
# Bound before patching, so building the real client here does not
# recurse back into this stand-in.
self._real = httpx.AsyncClient
self.kwargs = []
def __call__(self, **kwargs):
self.kwargs.append(dict(kwargs))
kwargs.pop("verify", None)
return self._real(transport=httpx.MockTransport(self._handler), **kwargs)
def _mock_httpx_client(status_code):
"""Build a stand-in for httpx.AsyncClient whose GET returns status_code."""
resp = MagicMock()
resp.status_code = status_code
client = MagicMock()
client.get = AsyncMock(return_value=resp)
ctx = MagicMock()
ctx.__aenter__ = AsyncMock(return_value=client)
ctx.__aexit__ = AsyncMock(return_value=False)
return MagicMock(return_value=ctx)
return _TransportClient(lambda request: httpx.Response(status_code))
# --- check_node dispatcher ---
@@ -434,6 +450,37 @@ async def test_check_service_non_http_port_is_unknown():
mock_http.assert_not_called()
@pytest.mark.parametrize("port", [515, 9100, 9101, 9107])
@pytest.mark.asyncio
async def test_check_service_never_touches_a_raw_printing_port(port):
"""Issue #404: a GET to JetDirect/LPD makes the printer print it.
The status checker runs every 60 s, so a printer approved as a node used to
print one page a minute, forever. These ports get no request at all.
"""
svc = {"port": port, "protocol": "tcp", "service_name": "jetdirect"}
with patch("app.services.status_checker._tcp_connect", new_callable=AsyncMock) as mock_tcp, \
patch("app.services.status_checker._http_get", new_callable=AsyncMock) as mock_http:
result = await check_service(svc, "10.0.0.1")
assert result == "unknown"
mock_tcp.assert_not_called()
mock_http.assert_not_called()
@pytest.mark.asyncio
async def test_check_service_printer_port_named_http_is_still_skipped():
"""A service_name saying "http" must not talk the checker onto 9100.
Fingerprinting labels 9100 "Node Exporter", and an override can put any
name there — the port decides, not the label.
"""
svc = {"port": 9100, "protocol": "tcp", "service_name": "http-node-exporter"}
with patch("app.services.status_checker._http_get", new_callable=AsyncMock) as mock_http:
result = await check_service(svc, "10.0.0.1")
assert result == "unknown"
mock_http.assert_not_called()
@pytest.mark.asyncio
async def test_check_service_ssh_port_22_is_unknown():
"""SSH (port 22) is never checked — keep it grey, not red/green."""
@@ -500,7 +547,14 @@ async def _captured_url(svc, host):
@pytest.mark.asyncio
async def test_check_service_uses_the_host_override():
svc = {"port": 8080, "protocol": "tcp", "service_name": "blog", "host": "blog.example.com"}
assert await _captured_url(svc, "10.0.0.1") == "http://blog.example.com:8080"
assert await _captured_url(svc, "10.0.0.1") == "http://blog.example.com"
@pytest.mark.asyncio
async def test_check_service_drops_the_scanned_port_behind_a_reverse_proxy():
"""Issue #382 — an override points at a proxy; the internal port breaks it."""
svc = {"port": 8083, "protocol": "tcp", "service_name": "blog", "host": "https://mywebserver.mydomain.com"}
assert await _captured_url(svc, "192.168.100.100") == "https://mywebserver.mydomain.com"
@pytest.mark.asyncio
@@ -522,21 +576,21 @@ async def test_check_service_takes_the_port_from_the_override():
@pytest.mark.asyncio
async def test_check_service_lets_the_service_port_beat_the_override_port():
async def test_check_service_keeps_the_port_typed_into_the_override():
svc = {"port": 3000, "protocol": "tcp", "service_name": "blog", "host": "blog.example.com:8443"}
assert await _captured_url(svc, "10.0.0.1") == "http://blog.example.com:3000"
assert await _captured_url(svc, "10.0.0.1") == "http://blog.example.com:8443"
@pytest.mark.asyncio
async def test_check_service_uses_the_first_host_of_a_comma_list_override():
svc = {"port": 8080, "protocol": "tcp", "service_name": "blog", "host": "blog.example.com, alt.example.com"}
assert await _captured_url(svc, "10.0.0.1") == "http://blog.example.com:8080"
assert await _captured_url(svc, "10.0.0.1") == "http://blog.example.com"
@pytest.mark.asyncio
async def test_check_service_brackets_an_ipv6_override():
svc = {"port": 80, "protocol": "tcp", "service_name": "http", "host": "[2001:db8::1]:8080"}
assert await _captured_url(svc, "10.0.0.1") == "http://[2001:db8::1]:80"
assert await _captured_url(svc, "10.0.0.1") == "http://[2001:db8::1]:8080"
@pytest.mark.asyncio
@@ -554,6 +608,43 @@ async def test_check_services_empty_list():
# --- _http_get status-code interpretation (real primitive, mocked transport) ---
# --- Regression: an endless response body must not be buffered (issue #375) ---
_CHUNK = b"\0" * 65536
def _endless_body(counter: dict):
"""
A body that never ends and declares no Content-Length — the Freebox
bandwidth-test endpoint from issue #375. Bounded at 512 chunks (32 MiB) so
a regression fails the test instead of hanging the suite forever.
"""
async def gen():
for _ in range(512):
counter["chunks"] += 1
yield _CHUNK
return gen()
@pytest.mark.asyncio
async def test_http_get_does_not_read_the_body():
# _http_get only needs the status line. Buffering the body of an endless
# stream is what OOM-killed the backend, so assert not one byte is pulled.
counter = {"chunks": 0}
factory = _TransportClient(
lambda request: httpx.Response(
200,
headers={"Content-Type": "application/octet-stream"},
content=_endless_body(counter),
)
)
with patch("app.services.status_checker.httpx.AsyncClient", factory):
assert await _http_get("http://192.168.1.254:8095/") is True
assert counter["chunks"] == 0
@pytest.mark.asyncio
async def test_http_get_true_on_2xx():
with patch("app.services.status_checker.httpx.AsyncClient", _mock_httpx_client(200)):
+179 -40
View File
@@ -7,10 +7,34 @@ from unittest.mock import AsyncMock, patch
import pytest
from httpx import AsyncClient
from app.services.import_jobs import reset_jobs
# ---------------------------------------------------------------------------
# Fixtures
# ---------------------------------------------------------------------------
@pytest.fixture(autouse=True)
def _clear_import_jobs() -> None:
"""The canvas-import job registry is process-global; isolate each test."""
reset_jobs()
async def _start_canvas_import(
client: AsyncClient, headers: dict, body: dict
) -> str:
"""POST /zigbee/import and return the job id.
The route is fire-and-poll: it answers 202 immediately and the fetch runs as
a Starlette background task, which the ASGI transport completes before this
await returns — so the job is already settled by the time we poll.
"""
res = await client.post("/api/v1/zigbee/import", json=body, headers=headers)
assert res.status_code == 202, res.text
job_id: str = res.json()["job_id"]
assert res.json()["status"] == "running"
return job_id
# ---------------------------------------------------------------------------
# /api/v1/zigbee/test-connection
# ---------------------------------------------------------------------------
@@ -104,18 +128,18 @@ _SAMPLE_EDGES = [
async def test_import_success(client: AsyncClient, headers: dict) -> None:
with patch("app.api.routes.zigbee.fetch_networkmap") as mock_fetch:
mock_fetch.return_value = (_SAMPLE_NODES, _SAMPLE_EDGES)
res = await client.post(
"/api/v1/zigbee/import",
json={
"mqtt_host": "localhost",
"mqtt_port": 1883,
"base_topic": "zigbee2mqtt",
},
headers=headers,
job_id = await _start_canvas_import(
client,
headers,
{"mqtt_host": "localhost", "mqtt_port": 1883, "base_topic": "zigbee2mqtt"},
)
res = await client.get(f"/api/v1/zigbee/import/{job_id}", headers=headers)
assert res.status_code == 200
data = res.json()
body = res.json()
assert body["status"] == "done"
assert body["job_id"] == job_id
data = body["result"]
assert data["device_count"] == 2
assert len(data["nodes"]) == 2
assert len(data["edges"]) == 1
@@ -123,22 +147,57 @@ async def test_import_success(client: AsyncClient, headers: dict) -> None:
assert coordinator["ieee_address"] == "0x00000000"
@pytest.mark.asyncio
async def test_import_post_does_not_block_on_the_fetch(
client: AsyncClient, headers: dict
) -> None:
"""The POST answers 202 without a payload — that is what keeps a slow mesh
from outliving a reverse proxy's read timeout (issue #380)."""
with patch("app.api.routes.zigbee.fetch_networkmap") as mock_fetch:
mock_fetch.return_value = (_SAMPLE_NODES, _SAMPLE_EDGES)
res = await client.post(
"/api/v1/zigbee/import",
json={"mqtt_host": "localhost", "mqtt_port": 1883},
headers=headers,
)
assert res.status_code == 202
assert set(res.json()) == {"job_id", "status"}
@pytest.mark.asyncio
async def test_import_job_unknown_id_returns_404(
client: AsyncClient, headers: dict
) -> None:
res = await client.get("/api/v1/zigbee/import/no-such-job", headers=headers)
assert res.status_code == 404
@pytest.mark.asyncio
async def test_import_job_requires_auth(client: AsyncClient, headers: dict) -> None:
with patch("app.api.routes.zigbee.fetch_networkmap") as mock_fetch:
mock_fetch.return_value = ([], [])
job_id = await _start_canvas_import(
client, headers, {"mqtt_host": "localhost", "mqtt_port": 1883}
)
res = await client.get(f"/api/v1/zigbee/import/{job_id}")
assert res.status_code == 401
@pytest.mark.asyncio
async def test_import_with_credentials(client: AsyncClient, headers: dict) -> None:
with patch("app.api.routes.zigbee.fetch_networkmap") as mock_fetch:
mock_fetch.return_value = ([], [])
res = await client.post(
"/api/v1/zigbee/import",
json={
await _start_canvas_import(
client,
headers,
{
"mqtt_host": "localhost",
"mqtt_port": 1883,
"mqtt_username": "admin",
"mqtt_password": "secret",
"base_topic": "z2m",
},
headers=headers,
)
assert res.status_code == 200
mock_fetch.assert_called_once_with(
mqtt_host="localhost",
mqtt_port=1883,
@@ -154,11 +213,10 @@ async def test_import_with_credentials(client: AsyncClient, headers: dict) -> No
async def test_import_connection_error_returns_502(client: AsyncClient, headers: dict) -> None:
with patch("app.api.routes.zigbee.fetch_networkmap") as mock_fetch:
mock_fetch.side_effect = ConnectionError("broker unreachable")
res = await client.post(
"/api/v1/zigbee/import",
json={"mqtt_host": "bad-host", "mqtt_port": 1883},
headers=headers,
job_id = await _start_canvas_import(
client, headers, {"mqtt_host": "bad-host", "mqtt_port": 1883}
)
res = await client.get(f"/api/v1/zigbee/import/{job_id}", headers=headers)
assert res.status_code == 502
assert "broker unreachable" in res.json()["detail"]
@@ -167,23 +225,22 @@ async def test_import_connection_error_returns_502(client: AsyncClient, headers:
async def test_import_timeout_returns_504(client: AsyncClient, headers: dict) -> None:
with patch("app.api.routes.zigbee.fetch_networkmap") as mock_fetch:
mock_fetch.side_effect = TimeoutError("timed out")
res = await client.post(
"/api/v1/zigbee/import",
json={"mqtt_host": "localhost", "mqtt_port": 1883},
headers=headers,
job_id = await _start_canvas_import(
client, headers, {"mqtt_host": "localhost", "mqtt_port": 1883}
)
res = await client.get(f"/api/v1/zigbee/import/{job_id}", headers=headers)
assert res.status_code == 504
assert "timed out" in res.json()["detail"]
@pytest.mark.asyncio
async def test_import_malformed_payload_returns_422(client: AsyncClient, headers: dict) -> None:
with patch("app.api.routes.zigbee.fetch_networkmap") as mock_fetch:
mock_fetch.side_effect = ValueError("malformed response")
res = await client.post(
"/api/v1/zigbee/import",
json={"mqtt_host": "localhost", "mqtt_port": 1883},
headers=headers,
job_id = await _start_canvas_import(
client, headers, {"mqtt_host": "localhost", "mqtt_port": 1883}
)
res = await client.get(f"/api/v1/zigbee/import/{job_id}", headers=headers)
assert res.status_code == 422
@@ -201,13 +258,12 @@ async def test_import_empty_network(client: AsyncClient, headers: dict) -> None:
"""An empty Zigbee network (coordinator only) is a valid response."""
with patch("app.api.routes.zigbee.fetch_networkmap") as mock_fetch:
mock_fetch.return_value = ([], [])
res = await client.post(
"/api/v1/zigbee/import",
json={"mqtt_host": "localhost", "mqtt_port": 1883},
headers=headers,
job_id = await _start_canvas_import(
client, headers, {"mqtt_host": "localhost", "mqtt_port": 1883}
)
res = await client.get(f"/api/v1/zigbee/import/{job_id}", headers=headers)
assert res.status_code == 200
data = res.json()
data = res.json()["result"]
assert data["device_count"] == 0
assert data["nodes"] == []
assert data["edges"] == []
@@ -227,16 +283,11 @@ async def test_import_missing_mqtt_host(client: AsyncClient, headers: dict) -> N
async def test_import_with_tls_passes_flags(client: AsyncClient, headers: dict) -> None:
with patch("app.api.routes.zigbee.fetch_networkmap") as mock_fetch:
mock_fetch.return_value = ([], [])
res = await client.post(
"/api/v1/zigbee/import",
json={
"mqtt_host": "broker.example.com",
"mqtt_port": 8883,
"mqtt_tls": True,
},
headers=headers,
await _start_canvas_import(
client,
headers,
{"mqtt_host": "broker.example.com", "mqtt_port": 8883, "mqtt_tls": True},
)
assert res.status_code == 200
kwargs = mock_fetch.call_args.kwargs
assert kwargs["tls"] is True
assert kwargs["tls_insecure"] is False
@@ -864,3 +915,91 @@ async def test_sync_now_rejected_without_host(
async def test_sync_now_requires_auth(client: AsyncClient) -> None:
res = await client.post("/api/v1/zigbee/sync-now")
assert res.status_code == 401
@pytest.mark.asyncio
async def test_canvas_import_also_lands_in_device_inventory(
client: AsyncClient, headers: dict, db_session
) -> None:
"""A canvas import is an import: the devices reach the Device Inventory too."""
from sqlalchemy import select
from app.db.models import InventoryDevice
with patch("app.api.routes.zigbee.fetch_networkmap") as mock_fetch:
mock_fetch.return_value = (_SAMPLE_NODES, _SAMPLE_EDGES)
job_id = await _start_canvas_import(
client, headers, {"mqtt_host": "localhost", "mqtt_port": 1883}
)
res = await client.get(f"/api/v1/zigbee/import/{job_id}", headers=headers)
nodes = res.json()["result"]["nodes"]
rows = (
await db_session.execute(
select(InventoryDevice).where(InventoryDevice.discovery_source == "zigbee")
)
).scalars().all()
by_ieee = {r.ieee_address: r for r in rows}
assert set(by_ieee) == {"0x00000000", "0x00000001"}
# Each node names the row it draws, so the canvas save links to it rather
# than minting a second, IEEE-less row.
for node in nodes:
assert node["device_id"] == by_ieee[node["ieee_address"]].id
@pytest.mark.asyncio
async def test_canvas_import_reuses_the_existing_inventory_row(
client: AsyncClient, headers: dict, db_session
) -> None:
from sqlalchemy import select
from app.db.models import InventoryDevice
existing = InventoryDevice(
ieee_address="0x00000001",
friendly_name="router_1",
status="pending",
discovery_source="zigbee",
)
db_session.add(existing)
await db_session.commit()
existing_id = existing.id
with patch("app.api.routes.zigbee.fetch_networkmap") as mock_fetch:
mock_fetch.return_value = (_SAMPLE_NODES, _SAMPLE_EDGES)
job_id = await _start_canvas_import(
client, headers, {"mqtt_host": "localhost", "mqtt_port": 1883}
)
res = await client.get(f"/api/v1/zigbee/import/{job_id}", headers=headers)
router = next(
n for n in res.json()["result"]["nodes"] if n["ieee_address"] == "0x00000001"
)
assert router["device_id"] == existing_id
rows = (
await db_session.execute(
select(InventoryDevice).where(InventoryDevice.ieee_address == "0x00000001")
)
).scalars().all()
assert len(rows) == 1
@pytest.mark.asyncio
async def test_canvas_import_failure_is_reported_on_the_job(
client: AsyncClient, headers: dict
) -> None:
"""A failed inventory write fails the job rather than returning half a map."""
with patch("app.api.routes.zigbee.fetch_networkmap") as mock_fetch, patch(
"app.api.routes.zigbee._persist_pending_import",
new=AsyncMock(side_effect=ValueError("inventory write failed")),
):
mock_fetch.return_value = (_SAMPLE_NODES, _SAMPLE_EDGES)
job_id = await _start_canvas_import(
client, headers, {"mqtt_host": "localhost", "mqtt_port": 1883}
)
res = await client.get(f"/api/v1/zigbee/import/{job_id}", headers=headers)
assert res.status_code == 422
assert "inventory write failed" in res.json()["detail"]
+99
View File
@@ -2,6 +2,7 @@
from __future__ import annotations
import asyncio
import json
from typing import Any
from unittest.mock import patch
@@ -9,7 +10,9 @@ from unittest.mock import patch
import aiomqtt # noqa: F401
import pytest
from app.core.config import settings
from app.services.zigbee_service import (
_NETWORKMAP_TIMEOUT,
_find_parent_router,
_z2m_type_to_homelable,
fetch_networkmap,
@@ -433,6 +436,10 @@ async def test_test_mqtt_connection_failure() -> None:
await _test_mqtt_connection("bad-host", 1883)
class _NeverMqttError(Exception):
"""Stand-in for aiomqtt.MqttError that no test path actually raises."""
# ---------------------------------------------------------------------------
# TLS context
# ---------------------------------------------------------------------------
@@ -571,3 +578,95 @@ async def test_fetch_networkmap_does_not_leak_creds_in_connection_error() -> Non
assert "hunter2" not in msg
assert "admin" not in msg
assert msg == "Authentication failed"
# ---------------------------------------------------------------------------
# Networkmap response timeout — configurable via ZIGBEE_NETWORKMAP_TIMEOUT
# (issue #380: a 200+ device mesh needs longer than the hard-coded 300 s)
# ---------------------------------------------------------------------------
class _SilentClient:
"""An MQTT client that connects but never delivers the response message."""
async def __aenter__(self):
return self
async def __aexit__(self, *_):
pass
async def subscribe(self, *_a, **_kw) -> None:
pass
async def publish(self, *_a, **_kw) -> None:
pass
@property
def messages(self):
async def _never():
await asyncio.Event().wait()
yield # pragma: no cover
return _never()
@pytest.mark.asyncio
async def test_fetch_networkmap_uses_settings_timeout(monkeypatch) -> None:
monkeypatch.setattr(settings, "zigbee_networkmap_timeout", 0.01)
with patch("app.services.zigbee_service.aiomqtt") as mock_aiomqtt:
mock_aiomqtt.Client.return_value = _SilentClient()
mock_aiomqtt.MqttError = _NeverMqttError
with pytest.raises(TimeoutError) as ei:
await fetch_networkmap(
mqtt_host="host", mqtt_port=1883, base_topic="zigbee2mqtt"
)
msg = str(ei.value)
assert "0.01s" in msg
assert "ZIGBEE_NETWORKMAP_TIMEOUT" in msg
@pytest.mark.asyncio
async def test_fetch_networkmap_explicit_timeout_overrides_settings(monkeypatch) -> None:
monkeypatch.setattr(settings, "zigbee_networkmap_timeout", 999)
with patch("app.services.zigbee_service.aiomqtt") as mock_aiomqtt:
mock_aiomqtt.Client.return_value = _SilentClient()
mock_aiomqtt.MqttError = _NeverMqttError
with pytest.raises(TimeoutError) as ei:
await fetch_networkmap(
mqtt_host="host",
mqtt_port=1883,
base_topic="zigbee2mqtt",
response_timeout=0.01,
)
assert "0.01s" in str(ei.value)
@pytest.mark.asyncio
async def test_fetch_networkmap_non_positive_setting_falls_back(monkeypatch) -> None:
"""A misconfigured 0 must not mean 'give up immediately'."""
monkeypatch.setattr(settings, "zigbee_networkmap_timeout", 0)
captured: dict[str, float] = {}
async def _fake_wait_for(awaitable, timeout):
captured["timeout"] = timeout
awaitable.close()
raise asyncio.TimeoutError
with patch("app.services.zigbee_service.aiomqtt") as mock_aiomqtt:
mock_aiomqtt.Client.return_value = _SilentClient()
mock_aiomqtt.MqttError = _NeverMqttError
with (
patch("app.services.zigbee_service.asyncio.wait_for", _fake_wait_for),
pytest.raises(TimeoutError),
):
await fetch_networkmap(
mqtt_host="host", mqtt_port=1883, base_topic="zigbee2mqtt"
)
assert captured["timeout"] == _NETWORKMAP_TIMEOUT
+139
View File
@@ -0,0 +1,139 @@
"""Moving a zone's size out of the custom_colors blob into the real columns.
Every node type stored its size in `nodes.width` / `nodes.height` except
`groupRect`, which kept it inside the style JSON. Get this backfill wrong and
every zone a user ever drew comes back at the default 360x240, losing a layout
they arranged by hand.
"""
import json
import pytest
from sqlalchemy.ext.asyncio import create_async_engine
import app.db.database as database
pytestmark = pytest.mark.asyncio
_DDL = (
"CREATE TABLE nodes ("
"id VARCHAR PRIMARY KEY, type VARCHAR, label VARCHAR, "
"custom_colors JSON, width FLOAT, height FLOAT)"
)
async def _engine(tmp_path, rows):
engine = create_async_engine(f"sqlite+aiosqlite:///{tmp_path / 'zones.db'}")
async with engine.begin() as conn:
await conn.exec_driver_sql(_DDL)
for node_id, node_type, colors, width, height in rows:
await conn.exec_driver_sql(
"INSERT INTO nodes (id, type, label, custom_colors, width, height) "
"VALUES (?, ?, ?, ?, ?, ?)",
(node_id, node_type, node_id, json.dumps(colors) if colors else None, width, height),
)
return engine
async def _sizes(engine) -> dict[str, tuple]:
async with engine.begin() as conn:
rows = (await conn.exec_driver_sql("SELECT id, width, height FROM nodes")).fetchall()
return {r[0]: (r[1], r[2]) for r in rows}
async def _run(engine, monkeypatch) -> None:
monkeypatch.setattr(database, "engine", engine)
await database._backfill_zone_size()
async def test_moves_the_size_from_the_blob_to_the_columns(tmp_path, monkeypatch):
engine = await _engine(
tmp_path,
[("z1", "groupRect", {"width": 640, "height": 480, "border": "#0ff"}, None, None)],
)
await _run(engine, monkeypatch)
assert (await _sizes(engine))["z1"] == (640, 480)
# The style the blob actually owns is untouched.
async with engine.begin() as conn:
blob = (await conn.exec_driver_sql("SELECT custom_colors FROM nodes")).scalar()
assert json.loads(blob)["border"] == "#0ff"
await engine.dispose()
async def test_never_overwrites_a_size_already_in_the_columns(tmp_path, monkeypatch):
# A zone resized since the upgrade: the column is the truth, and a stale
# blob left over from before must not win.
engine = await _engine(
tmp_path,
[("z1", "groupRect", {"width": 111, "height": 222}, 800, None)],
)
await _run(engine, monkeypatch)
assert (await _sizes(engine))["z1"] == (800, 222)
await engine.dispose()
async def test_leaves_other_node_types_alone(tmp_path, monkeypatch):
# Only zones ever stashed their size in the blob. A width key on any other
# type is not geometry we should be moving.
engine = await _engine(tmp_path, [("s1", "server", {"width": 999}, None, None)])
await _run(engine, monkeypatch)
assert (await _sizes(engine))["s1"] == (None, None)
await engine.dispose()
async def test_skips_a_blob_with_no_usable_size(tmp_path, monkeypatch):
engine = await _engine(
tmp_path,
[
("colors_only", "groupRect", {"border": "#0ff"}, None, None),
("not_a_number", "groupRect", {"width": "wide", "height": True}, None, None),
("no_blob", "groupRect", None, None, None),
],
)
await _run(engine, monkeypatch)
sizes = await _sizes(engine)
assert sizes["colors_only"] == (None, None)
assert sizes["not_a_number"] == (None, None)
assert sizes["no_blob"] == (None, None)
await engine.dispose()
async def test_survives_a_corrupt_blob_without_killing_boot(tmp_path, monkeypatch):
engine = create_async_engine(f"sqlite+aiosqlite:///{tmp_path / 'zones.db'}")
async with engine.begin() as conn:
await conn.exec_driver_sql(_DDL)
await conn.exec_driver_sql(
"INSERT INTO nodes (id, type, label, custom_colors) VALUES ('bad', 'groupRect', 'b', '{oops')"
)
await conn.exec_driver_sql(
"INSERT INTO nodes (id, type, label, custom_colors) "
"VALUES ('good', 'groupRect', 'g', '{\"width\": 500, \"height\": 400}')"
)
await _run(engine, monkeypatch)
sizes = await _sizes(engine)
assert sizes["bad"] == (None, None)
# One unreadable row must not cost the others their size.
assert sizes["good"] == (500, 400)
await engine.dispose()
async def test_is_idempotent(tmp_path, monkeypatch):
engine = await _engine(
tmp_path,
[("z1", "groupRect", {"width": 640, "height": 480}, None, None)],
)
await _run(engine, monkeypatch)
await _run(engine, monkeypatch)
assert (await _sizes(engine))["z1"] == (640, 480)
await engine.dispose()
+76
View File
@@ -601,3 +601,79 @@ async def test_sync_now_rejected_without_host(
async def test_sync_now_requires_auth(client: AsyncClient) -> None:
res = await client.post("/api/v1/zwave/sync-now")
assert res.status_code == 401
@pytest.mark.asyncio
async def test_import_also_lands_in_device_inventory(
client: AsyncClient, headers: dict, db_session
) -> None:
"""A canvas import is an import: the devices reach the Device Inventory too."""
from sqlalchemy import select
from app.db.models import InventoryDevice
with patch("app.api.routes.zwave.fetch_zwave_network") as mock_fetch:
mock_fetch.return_value = (_SAMPLE_NODES, _SAMPLE_EDGES)
res = await client.post(
"/api/v1/zwave/import",
json={"mqtt_host": "localhost", "mqtt_port": 1883},
headers=headers,
)
assert res.status_code == 200
rows = (
await db_session.execute(
select(InventoryDevice).where(InventoryDevice.discovery_source == "zwave")
)
).scalars().all()
by_ieee = {r.ieee_address: r for r in rows}
assert set(by_ieee) == {"zwave-0xh-1", "zwave-0xh-2"}
# Every returned node names the row it draws, so the canvas save links to it
# rather than minting a second one.
for node in res.json()["nodes"]:
assert node["device_id"] == by_ieee[node["ieee_address"]].id
@pytest.mark.asyncio
async def test_import_reuses_the_existing_inventory_row(
client: AsyncClient, headers: dict, db_session
) -> None:
"""A device already in the inventory is pointed at, not duplicated."""
from sqlalchemy import select
from app.db.models import InventoryDevice
existing = InventoryDevice(
ieee_address="zwave-0xh-1",
friendly_name="Controller",
status="pending",
discovery_source="zwave",
)
db_session.add(existing)
await db_session.commit()
existing_id = existing.id
with patch("app.api.routes.zwave.fetch_zwave_network") as mock_fetch:
mock_fetch.return_value = (_SAMPLE_NODES, _SAMPLE_EDGES)
res = await client.post(
"/api/v1/zwave/import",
json={"mqtt_host": "localhost", "mqtt_port": 1883},
headers=headers,
)
assert res.status_code == 200
count = len(
(
await db_session.execute(
select(InventoryDevice).where(
InventoryDevice.ieee_address == "zwave-0xh-1"
)
)
).scalars().all()
)
assert count == 1
coordinator = next(
n for n in res.json()["nodes"] if n["ieee_address"] == "zwave-0xh-1"
)
assert coordinator["device_id"] == existing_id
+6
View File
@@ -13,6 +13,12 @@ services:
- homelable
cap_add:
- NET_RAW
# MAC addresses: on this bridge network the scanner can never see them —
# ARP is layer 2 and every LAN host is one hop away behind the Docker
# gateway. To collect MACs (and so keep DHCP devices matched across an IP
# change), put the backend on the LAN: comment out the `networks:` key
# above and uncomment the line below (Linux hosts only).
# network_mode: host
frontend:
image: ghcr.io/pouzor/homelable-frontend:latest
+10
View File
@@ -16,6 +16,12 @@ services:
# Required for ping-based status checks
cap_add:
- NET_RAW
# MAC addresses: on this bridge network the scanner can never see them —
# ARP is layer 2 and every LAN host is one hop away behind the Docker
# gateway. To collect MACs (and so keep DHCP devices matched across an IP
# change), put the backend on the LAN: comment out the `networks:` key
# above and uncomment the line below (Linux hosts only).
# network_mode: host
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:8000/api/v1/health"]
interval: 10s
@@ -43,6 +49,10 @@ services:
build:
context: .
dockerfile: Dockerfile.frontend
args:
# Serve Homelable under a subpath, e.g. VITE_BASE_PATH=/homelab/ in .env.
# Defaults to the root — see docs/INSTALLATION.md.
VITE_BASE_PATH: ${VITE_BASE_PATH:-/}
restart: unless-stopped
ports:
- "3000:80"
@@ -0,0 +1,26 @@
# Standalone (no backend) Homelable served under a subpath (__BASE__).
# Generated at image build time — see docker/nginx.subpath.conf.template.
server {
listen 80;
server_name _;
root /usr/share/nginx/html;
index index.html;
# Relative Location headers — the port and scheme belong to whatever proxy is
# in front, not to this server block.
absolute_redirect off;
location = __BASE_NO_SLASH__ {
return 301 __BASE__;
}
# SPA fallback under the prefix — no backend proxy
location __BASE__ {
try_files $uri $uri/ __BASE__index.html;
}
# Same, for a front proxy that strips the prefix before forwarding
location / {
root /usr/share/nginx/html__BASE_NO_SLASH__;
try_files $uri $uri/ /index.html;
}
}
+86
View File
@@ -0,0 +1,86 @@
# Homelable served under a subpath (__BASE__) instead of the root of the origin.
#
# Generated at image build time by Dockerfile.frontend when VITE_BASE_PATH is set:
# __BASE__ is the prefix with its trailing slash, __BASE_NO_SLASH__ without it. A
# default build keeps docker/nginx.conf and this file is never read.
#
# The static build is copied to /usr/share/nginx/html__BASE__, so every block below
# uses plain `root` semantics — `alias` plus `try_files` mis-resolves $uri.
#
# Both reverse-proxy styles work against this config:
# - prefix forwarded intact (proxy_pass http://homelable;) -> the __BASE__ blocks
# - prefix stripped (proxy_pass http://homelable/;) -> the trailing blocks
server {
listen 80;
server_name _;
root /usr/share/nginx/html;
index index.html;
# Relative Location headers — the port and scheme belong to whatever proxy is
# in front, not to this server block.
absolute_redirect off;
# --- prefix forwarded intact ------------------------------------------------
location = __BASE_NO_SLASH__ {
return 301 __BASE__;
}
# WebSocket (must come before the API block to take priority)
location __BASE__api/v1/status/ws/ {
proxy_pass http://backend:8000/api/v1/status/ws/;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
}
location __BASE__api/ {
proxy_pass http://backend:8000/api/;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
}
# Legacy /ws/ path
location __BASE__ws/ {
proxy_pass http://backend:8000/ws/;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;
}
# SPA fallback under the prefix
location __BASE__ {
try_files $uri $uri/ __BASE__index.html;
}
# --- prefix already stripped by the front proxy -----------------------------
# No redirect to __BASE__ here: the front proxy would strip it again and loop.
location /api/v1/status/ws/ {
proxy_pass http://backend:8000;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
}
location /api/ {
proxy_pass http://backend:8000;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
}
location /ws/ {
proxy_pass http://backend:8000;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;
}
location / {
root /usr/share/nginx/html__BASE_NO_SLASH__;
try_files $uri $uri/ /index.html;
}
}
+249
View File
@@ -0,0 +1,249 @@
# Documentation
A markdown document per device, plus a free-form Library of pages and folders,
behind the **Documentation** entry in the left sidebar.
Before this, the only writing surface in Homelable was `device_inventory.notes`
— one `TEXT` column rendered as plain text. This replaces it with something that
can be structured, linked, searched and kept honest, without ever throwing the
old notes away.
---
## The model
Three tables. See `docs/database-model.md` for the columns.
| Table | What |
|---|---|
| `documents` | One markdown document. Either a Library `page`/`folder`, or a `device` / `node` / `design` document describing exactly one thing. |
| `document_revisions` | Prior bodies, pruned to the most recent 50 per document. |
| `documents_fts` | The FTS5 search index. Optional — see *Search* below. |
**A document attaches to the device, not to the node.** Since 3.3.0 a node draws
a device and the facts live on `device_inventory`, so one device reads the same
on every canvas it appears on.
**Every link is `SET NULL` and the title is denormalized.** Deleting a device or
a canvas never destroys what was written: the document survives as an orphan and
can be re-linked or filed into the Library. Foreign keys are off in this SQLite
setup, so the delete paths clear the columns by hand through
`app/services/doc_links.py::unlink_documents` — `nodes`, `scan` and `designs`
each call it before they delete.
**A folder is a document** (`kind='folder'`). That is what allows an empty
folder, and lets a folder carry an index body.
**One document per target.** Enforced twice: a 409 in the route, and partial
unique indexes created in `init_db` (`DOCUMENT_DDL` in `db/database.py` —
`create_all` cannot express a partial index or a virtual table).
---
## The generated header
`app/services/doc_template.py` scaffolds a device document from the live facts:
addresses, hardware, one section per fingerprinted service, rack and zone
placement, canvas neighbours, custom properties. Old `notes` are appended
verbatim under `## Notes` behind an HTML-comment provenance marker.
**It is generated once and then belongs to the user.** Nothing rewrites a body
that already exists. Two things keep it from going stale silently:
- `facts_snapshot` records the facts as they read at generation time, so the UI
can say *the device has changed* and list what.
- `GET /api/v1/documents/blocks?block=…&device_id=…` renders any single block on
demand — that is what the editor's `/device`, `/services`, `/rack`… commands
insert. Regeneration is always one keystroke away, and never automatic.
A section whose data is empty is still emitted, with an italic prompt: the
document is a checklist of what has not been written yet. Sections that are
structural rather than descriptive — the rack position of an unracked device —
are dropped instead.
Service URLs in a document follow `frontend/src/utils/serviceUrl.ts`, including
its non-HTTP port denylist, so a link in a document is the link the canvas
offers.
---
## The tree
Two roots, with different natures.
**Devices** is a pivot, rebuilt client-side in
`frontend/src/documentation/tree.ts` from data the app already holds. Grouping
by zone, group, type, physical/virtual, subnet, canvas, rack, vendor, discovery
source, status, tag, or flat A–Z is instant and needs no server round trip.
- Zone and group come from walking `parent_id` up to the nearest `groupRect` /
`group` node.
- Subnet prefers a configured scanner range, else the address's own /24. A
device with several IPs appears under each subnet it reaches.
- Physical / Virtual / Host is derived: `vm`/`lxc`/`docker_container` are
virtual, `proxmox`/`docker_host` and anything drawn as a container are hosts.
- Canvas answers "on the loaded canvas or not" — the app only holds one at a
time.
**Library** is the real folder tree, ordered by `sort_order` then title.
Badges: `·` no document, `○` only the generated header, `●` written, `⚠` the
device changed since, `⏰` past its `review_every`.
---
## Editing
Markdown source on the left, live preview on the right. Source rather than a
rich editor on purpose: the body is exported verbatim, so what is typed is what
the file holds.
**Saving is explicit** — Ctrl/Cmd+S or the Save button — matching the canvas
rule that nothing persists on a timer. An unsaved body is mirrored to
`localStorage` under `homelable_docdraft:<id>` on every keystroke and cleared on
save. On reopening, a draft taken against the version being opened is *offered*;
a draft taken against an older version is discarded, because replaying it would
revert a change made elsewhere.
`/` at the start of a line opens the insert menu: the generated blocks above,
plus table, checklist, callout, wiki-link and date.
`[[` opens the link picker, anywhere in the line: filter by title, pick, and the
finished link is written for you. It offers documents only — a device with no
document is not a link target yet — and it addresses a document by title, or by
`[[doc:<id>]]` when another document shares that title, since a bare link
resolves by title and an ambiguous one would silently pick the first. The second
bracket is held back while the menu is open (opening it moves focus, and a
keystroke landing mid-move belongs to neither box) and given back if you cancel,
so what you typed survives either way.
---
## Links between documents
`[[device:nas-01]]`, `[[doc:vlan-plan]]`, `[[node:<id>]]`, or a bare
`[[VLAN plan]]`; `[[…|label]]` sets the text. A device resolves by id or label,
a document by id, slug, then title, all case-insensitively. An unresolved link
renders red and offers to create the document.
Parsed as a text pass over the rendered children (`wikilinks.ts` +
`markdown/WikiText.tsx`) rather than as a remark plugin, so the file stays
ordinary markdown for anything else that reads it.
**Raw HTML is deliberately not rendered** — `rehype-raw` is absent — so a
document cannot inject markup and no sanitiser is needed.
### Backlinks
Every document lists what points at it, under **Linked from**, with the line the
link was written on and the label it was written as.
The inversion is done **server-side** (`services/doc_backlinks.py`,
`GET /documents/{id}/backlinks`), because the browser holds no bodies but the
open one: `GET /documents` is metadata-only so the tree can badge without
downloading the space. That service mirrors the resolution rules in
`wikilinks.ts` — same prefixes, same id → slug → title fallback, same
case-insensitivity — the way the generator's `service_url` mirrors
`utils/serviceUrl.ts`. Change one and change the other;
`test_doc_backlinks.py` pins the rules.
Repeated links from the same document collapse into one entry with a count, and
a document never backlinks itself. The inventory is only loaded when some body
actually carries a `[[device:…]]`.
---
## History
Every explicit save that changes the body snapshots the previous one, capped at
`REVISION_LIMIT` (50) per document; so do restore, regenerate, scaffold and the
notes migration, each recording why.
The clock-arrow button in the header opens the history rail: what each version
was (Saved, Regenerated, Restored, Migrated from notes…), when, and how big.
Selecting one replaces the body with that version — same title, same chips, same
rail — and **Changes** turns it into a line diff against the current body, with
long unchanged runs collapsed. **Restore** brings it back, snapshotting the body
it replaces first, so a restore is itself undoable.
A revision's body is fetched only when it is opened: the list carries a size, not
the text, so fifty versions cost one small request.
The diff is a plain LCS over lines (`documentation/diff.ts`), after the common
head and tail are trimmed — no diff library, and a pathological pair of long,
wholly different bodies degrades to "replaced wholesale" instead of building a
250k-cell matrix.
---
## Search
`GET /api/v1/documents/search?q=` over title, tags and body.
The same index answers the canvas' own search modal (Ctrl/Cmd+K), which lists
matching documents under the nodes and pending devices; picking one switches to
Documentation and opens it. The request is debounced and skipped below two
characters, and a failure there costs only the document rows — the node and
device halves filter lists already in memory.
FTS5 is **not** assumed: the SQLite shipped in the LXC and Docker images may be
built without it, the same reason `database.py` avoids JSON1. `doc_search.py`
probes once and falls back to `LIKE`, and the response says which engine
answered (`fts5` or `like`) so the UI can drop snippet highlighting. The index is
maintained from Python on every write — no triggers — and `_reindex_documents_fts`
catches up on boot after an upgrade or a restored backup.
User input is never FTS5 syntax: `build_match_query` quotes every term, so an
IP or a stray quote is data.
---
## Migrating the old notes
Non-destructive and repeatable.
1. `GET /api/v1/documents/coverage` reports `notes_unmigrated`.
2. A banner offers to migrate.
3. `POST /api/v1/documents/scaffold {"only_with_notes": true}` creates one
document per device, old notes appended verbatim, with a `migrate` revision.
4. **`device_inventory.notes` is left exactly as it was.** It is still the
column the HACS integration, the YAML export, the MCP tools and the canvas
search read. Deprecating it is a separate decision.
---
## API
| Method | Path |
|---|---|
| `GET` | `/api/v1/documents` — metadata only, filterable by `kind`, `parent_id`, `device_id`, `tag` |
| `GET` | `/api/v1/documents/{id}` |
| `POST` | `/api/v1/documents` |
| `PATCH` | `/api/v1/documents/{id}` |
| `DELETE` | `/api/v1/documents/{id}` — a folder takes its subtree |
| `GET` | `/api/v1/documents/{id}/revisions`, `/revisions/{rev_id}` |
| `GET` | `/api/v1/documents/{id}/backlinks` — the documents linking here |
| `POST` | `/api/v1/documents/{id}/revisions/{rev_id}/restore` |
| `POST` | `/api/v1/documents/{id}/regenerate` — erase the body and scaffold it again |
| `GET` | `/api/v1/documents/search?q=&limit=` |
| `GET` | `/api/v1/documents/blocks?block=&device_id=` |
| `GET` | `/api/v1/documents/coverage` |
| `POST` | `/api/v1/documents/scaffold` |
---
## Standalone mode
Documents are **full-mode only**. Standalone has no backend to store or search
them, and the same reasoning as ADR-001 applies: the section is hidden in the
sidebar and explains itself if reached directly, rather than being polyfilled
into `localStorage` with no search and no history.
---
## Not built yet
Export/import of the tree as `.md` files, a print/handbook view, an aggregated
open-tasks view, image upload inside a document (would reuse
`api/routes/media.py`, full-mode only), and the MCP tools — those are the
planned second lot.
+2 -1
View File
@@ -236,7 +236,8 @@ diagram's online/offline tally. **PNG export** works here too.
- No power draw or outlet budgeting.
- No undo/redo on a rack canvas (the diagram canvas keeps its own).
- Rack canvases are not shown by the read-only [Live View](../README.md#live-view-read-only-public-canvas).
- The MCP server exposes diagram nodes and links only, not racks.
- The MCP server can read and edit a rack canvas, but its save is last-writer-wins:
an AI write while you have the canvas open with unsaved changes loses one side.
---
+17 -1
View File
@@ -55,9 +55,25 @@ Click **Fetch Devices**. Homelable will:
1. Connect to the broker
2. Subscribe to the response topic
3. Publish `{"type": "raw", "routes": false}` to the request topic
4. Wait up to 60 seconds for the network map response (large meshes can take 30 s+)
4. Wait up to `ZIGBEE_NETWORKMAP_TIMEOUT` seconds (default 300) for the network map response
5. Parse and group devices by type
The fetch runs server-side and the browser polls for the result, so a mesh that
takes minutes to answer cannot be cut short by a reverse proxy's read timeout.
### Large meshes and timeouts
A network of 200+ devices can take several minutes to build its map. Two knobs:
| Variable | Default | What it bounds |
|---|---|---|
| `ZIGBEE_NETWORKMAP_TIMEOUT` | `300` | Seconds to wait for the Z2M bridge to answer a networkmap request. Raise it if an import fails with *Timed out waiting for networkmap response*. |
| `MQTT_RESPONSE_TIMEOUT` | `300` | The same bound for the Z-Wave MQTT round-trip. |
Both apply to manual imports and to auto-sync. If you reverse-proxy the API,
these are the only timeouts that matter — the import request itself returns
immediately.
### 5. Select and add to canvas
Devices are grouped by type (Coordinator / Router / End Device).
+1577 -66
View File
File diff suppressed because it is too large Load Diff
+4 -1
View File
@@ -1,7 +1,7 @@
{
"name": "frontend",
"private": true,
"version": "3.3.2",
"version": "3.4.1",
"type": "module",
"scripts": {
"dev": "vite",
@@ -31,6 +31,9 @@
"next-themes": "^0.4.6",
"react": "^19.2.0",
"react-dom": "^19.2.0",
"react-markdown": "^10.1.0",
"rehype-slug": "^6.0.0",
"remark-gfm": "^4.0.1",
"sonner": "^2.0.7",
"tailwind-merge": "^3.5.0",
"tw-animate-css": "^1.4.0",
+46
View File
@@ -0,0 +1,46 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 64 64" fill="none">
<defs>
<radialGradient id="bg-glow" cx="50%" cy="35%" r="55%">
<stop offset="0%" stop-color="#00d4ff" stop-opacity="0.08"/>
<stop offset="100%" stop-color="#0d1117" stop-opacity="0"/>
</radialGradient>
<filter id="node-glow">
<feGaussianBlur stdDeviation="1.5" result="blur"/>
<feMerge><feMergeNode in="blur"/><feMergeNode in="SourceGraphic"/></feMerge>
</filter>
</defs>
<!-- Background -->
<circle cx="32" cy="32" r="32" fill="#0d1117"/>
<circle cx="32" cy="32" r="32" fill="url(#bg-glow)"/>
<!-- House body -->
<path d="M32 11 L53 30 L48 30 L48 53 L16 53 L16 30 L11 30 Z"
fill="#161b22" stroke="#00d4ff" stroke-width="1.5" stroke-linejoin="round"/>
<!-- Floor line (subtle) -->
<line x1="16" y1="30" x2="48" y2="30" stroke="#00d4ff" stroke-width="0.5" opacity="0.25"/>
<!-- Door -->
<rect x="27" y="40" width="10" height="13" rx="1.5"
fill="#0d1117" stroke="#00d4ff" stroke-width="1" opacity="0.9"/>
<!-- Network lines (drawn under nodes) -->
<line x1="32" y1="23" x2="32" y2="30" stroke="#a855f7" stroke-width="1.2" opacity="0.7" stroke-linecap="round"/>
<line x1="21" y1="38" x2="29" y2="33" stroke="#39d353" stroke-width="1.2" opacity="0.7" stroke-linecap="round"/>
<line x1="43" y1="38" x2="35" y2="33" stroke="#39d353" stroke-width="1.2" opacity="0.7" stroke-linecap="round"/>
<!-- Center hub glow -->
<circle cx="32" cy="33" r="5" fill="#00d4ff" opacity="0.12"/>
<!-- Center hub -->
<circle cx="32" cy="33" r="3" fill="#00d4ff" filter="url(#node-glow)"/>
<!-- Left node -->
<circle cx="21" cy="38" r="2.5" fill="#39d353" filter="url(#node-glow)"/>
<!-- Right node -->
<circle cx="43" cy="38" r="2.5" fill="#39d353" filter="url(#node-glow)"/>
<!-- Top node -->
<circle cx="32" cy="23" r="2" fill="#a855f7" filter="url(#node-glow)"/>
</svg>

After

Width:  |  Height:  |  Size: 1.9 KiB

+201 -27
View File
@@ -1,4 +1,4 @@
import { useEffect, useCallback, useMemo, useRef, useState } from 'react'
import { useEffect, useCallback, useMemo, useRef, useState, lazy, Suspense } from 'react'
import { ReactFlowProvider, type Connection, type Edge } from '@xyflow/react'
import { type Node } from '@xyflow/react'
import { applyDagreLayout } from '@/utils/layout'
@@ -9,12 +9,15 @@ import { resolveVirtualEdgeParent } from '@/utils/virtualEdgeParent'
import { generateMarkdownTable } from '@/utils/exportMarkdown'
import { copyToClipboard } from '@/utils/clipboard'
import { getDesignIdFromUrl, setDesignIdInUrl } from '@/utils/designUrl'
import { useDocsUrlSync } from '@/hooks/useDocsUrlSync'
import { withBase } from '@/utils/basePath'
import { ExportModal } from '@/components/modals/ExportModal'
import { exportCanvasToYaml, downloadYaml } from '@/utils/exportYaml'
import { parseYamlToCanvas } from '@/utils/importYaml'
import { TooltipProvider } from '@/components/ui/tooltip'
import { Toaster } from '@/components/ui/sonner'
import { toast } from 'sonner'
import { isZoneSubnetCandidate } from '@/utils/subnet'
import { CanvasContainer } from '@/components/canvas/CanvasContainer'
import { Sidebar } from '@/components/panels/Sidebar'
import { Toolbar } from '@/components/panels/Toolbar'
@@ -52,24 +55,35 @@ import { WalkthroughOverlay } from '@/walkthrough/WalkthroughOverlay'
import { DEMO_SCAN_RUNS, DEMO_INVENTORY_DEVICES } from '@/walkthrough/demoTourData'
import { useStatusPolling } from '@/hooks/useStatusPolling'
import { bootstrapAuth } from '@/auth/bootstrap'
// Code-split: the Documentation section pulls in the markdown renderer, which a
// user who never leaves the canvas should not pay for on first load.
const DocumentationView = lazy(async () => ({
default: (await import('@/documentation/components/DocumentationView')).DocumentationView,
}))
import { useDocsStore } from '@/documentation/store'
import { useUiStore } from '@/stores/uiStore'
import { RackCanvas } from '@/rack/components/RackCanvas'
import { RackCablePanel } from '@/rack/components/RackCablePanel'
import { useRackStore } from '@/rack/store'
import type { NodeData, EdgeData, CustomStyleDef, DesignType, FloorMapConfig, NodeType } from '@/types'
import type { ZigbeeNode, ZigbeeEdge } from '@/components/zigbee/types'
import type { ZwaveNode, ZwaveEdge } from '@/components/zwave/types'
import type { ProxmoxNode, ProxmoxEdge } from '@/components/proxmox/types'
import type { ProxmoxNode, ProxmoxEdge, ProxmoxCanvasMode } from '@/components/proxmox/types'
import { buildProxmoxClusterEdges } from '@/components/proxmox/clusterEdges'
import { groupProxmoxGuests, layoutProxmoxContainers, measureProxmoxContainers } from '@/utils/proxmoxContainerLayout'
const STANDALONE = import.meta.env.VITE_STANDALONE === 'true'
export default function App() {
const { loadCanvas, applyLayout, markSaved, markUnsaved, hasUnsavedChanges, editSeq, selectedNodeId, selectedNodeIds, addNode, updateNode, deleteNode, onConnect, updateEdge, deleteEdge, setProxmoxContainerMode, setNodeZIndex, editingGroupRectId, setEditingGroupRectId, editingTextId, setEditingTextId, nodes, edges, snapshotHistory, undo, redo, addToGroup, addToContainer, addToZone, floorMap, setFloorMap } = useCanvasStore()
const { loadCanvas, applyLayout, markSaved, markUnsaved, hasUnsavedChanges, editSeq, selectedNodeId, selectedNodeIds, addNode, updateNode, deleteNode, onConnect, updateEdge, deleteEdge, setProxmoxContainerMode, setNodeZIndex, editingGroupRectId, setEditingGroupRectId, editingTextId, setEditingTextId, nodes, edges, snapshotHistory, undo, redo, addNodesToGroup, addNodesToContainer, addNodesToZone, importZoneSubnet, floorMap, setFloorMap } = useCanvasStore()
const canvasRef = useRef<HTMLDivElement>(null)
const { isAuthenticated, isInitialized } = useAuthStore()
const authBootstrapStarted = useRef(false)
const { activeTheme, setTheme, customStyle, setCustomStyle } = useThemeStore()
const { activeDesignId, activeDesignType, setDesigns, setActiveDesign } = useDesignStore()
const appView = useUiStore((s) => s.view)
const setAppView = useUiStore((s) => s.setView)
const openDocId = useDocsStore((s) => s.openDoc?.id ?? null)
const isRackDesign = activeDesignType === 'rack'
const rackDirty = useRackStore((s) => s.hasUnsavedChanges)
const rackEditSeq = useRackStore((s) => s.editSeq)
@@ -131,9 +145,14 @@ export default function App() {
const [addTextOpen, setAddTextOpen] = useState(false)
const [editNodeId, setEditNodeId] = useState<string | null>(null)
const [pendingConnection, setPendingConnection] = useState<Connection | null>(null)
const [pendingGroupAdd, setPendingGroupAdd] = useState<{ nodeId: string; groupId: string } | null>(null)
const [pendingContainerAdd, setPendingContainerAdd] = useState<{ nodeId: string; containerId: string } | null>(null)
const [pendingZoneAdd, setPendingZoneAdd] = useState<{ nodeId: string; zoneId: string } | null>(null)
const [pendingGroupAdd, setPendingGroupAdd] = useState<{ nodeIds: string[]; groupId: string } | null>(null)
const [pendingContainerAdd, setPendingContainerAdd] = useState<{ nodeIds: string[]; containerId: string } | null>(null)
const [pendingZoneAdd, setPendingZoneAdd] = useState<{ nodeIds: string[]; zoneId: string } | null>(null)
// Labels for the add-to-group/container/zone confirmation, in the order dropped.
const labelsOf = useCallback(
(ids: string[]) => ids.map((id) => nodes.find((n) => n.id === id)?.data.label ?? ''),
[nodes],
)
const [editEdgeId, setEditEdgeId] = useState<string | null>(null)
const [scanConfigOpen, setScanConfigOpen] = useState(false)
const [settingsOpen, setSettingsOpen] = useState(false)
@@ -470,6 +489,35 @@ export default function App() {
if (activeDesignId) setDesignIdInUrl(activeDesignId)
}, [activeDesignId])
// A document is linkable in the same way: `?view=docs&doc=<id>` — written on
// every change, and read back on boot so a refresh reopens it.
const restoreDocsUrl = useCallback(
async (docId: string | null) => {
setAppView('documentation')
if (docId) await useDocsStore.getState().open(docId)
},
[setAppView],
)
useDocsUrlSync({
view: appView,
openDocId,
ready: isAuthenticated,
enabled: !STANDALONE,
onRestore: restoreDocsUrl,
})
// The canvas' way into Documentation: it knows a device id, and the section
// resolves that to a document — writing one from the device's facts when
// there is none, exactly as picking the device in its tree would.
const openDocumentationFor = useCallback(
async (deviceId: string, label: string) => {
setAppView('documentation')
const ok = await useDocsStore.getState().openForDevice(deviceId, label)
if (!ok) toast.error('Could not open the documentation for that device')
},
[setAppView],
)
// Keep refs for store actions so keydown handler is always up-to-date without re-registering
const undoRef = useRef(undo)
const redoRef = useRef(redo)
@@ -485,8 +533,19 @@ export default function App() {
const isInput = tag === 'INPUT' || tag === 'TEXTAREA' || (e.target as HTMLElement).isContentEditable
if (ctrl && e.key === 's') { e.preventDefault(); handleSaveRef.current(); return }
if (ctrl && e.key === 'z') { e.preventDefault(); undoRef.current(); return }
if (ctrl && (e.key === 'y' || (e.shiftKey && e.key === 'z'))) { e.preventDefault(); redoRef.current(); return }
// Undo belongs to whatever is being typed in — the document editor keeps
// its own history, a plain field has the browser's — and the canvas only
// gets it when nothing has the caret.
if (ctrl && e.key.toLowerCase() === 'z') {
if (isInput) return
e.preventDefault()
// Shift+Z is the other redo, and matched neither branch while both
// compared a lowercase key — a shifted `z` arrives as `Z`.
if (e.shiftKey) redoRef.current()
else undoRef.current()
return
}
if (ctrl && e.key.toLowerCase() === 'y') { if (isInput) return; e.preventDefault(); redoRef.current(); return }
if (ctrl && e.key === 'k') { e.preventDefault(); setSearchOpen(true); return }
// Copy/paste (Ctrl/Cmd+C/V) handled in CanvasContainer so paste can place
// nodes under the cursor / viewport center.
@@ -526,6 +585,26 @@ export default function App() {
toast.success(`Added "${data.label}"`)
}, [addNode, nodes, snapshotHistory])
// Subnet import is a one-shot action on an existing zone, so on the Add modal
// there is no zone to run it against yet: the CIDR is held here and applied
// once, right after the zone is created. Cleared whenever that modal closes.
const pendingZoneSubnet = useRef<string | null>(null)
const countSubnetMatches = useCallback(
(cidr: string, zoneId?: string) =>
nodes.filter((n) => isZoneSubnetCandidate(n, cidr, zoneId)).length,
[nodes],
)
const reportSubnetImport = useCallback((moved: number, cidr: string) => {
if (moved === 0) toast.info(`No unparented device in ${cidr}`)
else toast.success(`Moved ${moved} device${moved > 1 ? 's' : ''} from ${cidr} into the zone`)
}, [])
const handleImportSubnetIntoZone = useCallback((zoneId: string, cidr: string) => {
reportSubnetImport(importZoneSubnet(zoneId, cidr), cidr)
}, [importZoneSubnet, reportSubnetImport])
const handleAddGroupRect = useCallback((data: GroupRectFormData) => {
snapshotHistory()
const id = generateUUID()
@@ -535,6 +614,7 @@ export default function App() {
position: getCenteredPosition(360, 240),
data: {
label: data.label,
description: data.description,
type: 'groupRect',
status: 'unknown',
services: [],
@@ -556,7 +636,12 @@ export default function App() {
zIndex: data.z_order - 10,
}
addNode(newNode)
}, [addNode, snapshotHistory])
const cidr = pendingZoneSubnet.current
pendingZoneSubnet.current = null
// addNode has already committed, so the zone is in the store by now.
if (cidr) reportSubnetImport(importZoneSubnet(id, cidr), cidr)
}, [addNode, snapshotHistory, importZoneSubnet, reportSubnetImport])
const handleUpdateGroupRect = useCallback((data: GroupRectFormData) => {
if (!editingGroupRectId) return
@@ -564,6 +649,7 @@ export default function App() {
const existing = nodes.find((n) => n.id === editingGroupRectId)
updateNode(editingGroupRectId, {
label: data.label,
description: data.description,
custom_colors: {
...existing?.data.custom_colors,
border: data.border_color,
@@ -740,7 +826,9 @@ export default function App() {
if (STANDALONE) {
// Standalone reads canvas from localStorage; pass the active design id so
// the read-only tab renders the same canvas the user is viewing.
const url = activeDesignId ? `/view?design=${encodeURIComponent(activeDesignId)}` : '/view'
const url = activeDesignId
? withBase(`view?design=${encodeURIComponent(activeDesignId)}`)
: withBase('view')
window.open(url, '_blank', 'noopener,noreferrer')
return
}
@@ -752,7 +840,7 @@ export default function App() {
}
const params = new URLSearchParams({ key: res.data.key })
if (activeDesignId) params.set('design', activeDesignId)
window.open(`/view?${params.toString()}`, '_blank', 'noopener,noreferrer')
window.open(withBase(`view?${params.toString()}`), '_blank', 'noopener,noreferrer')
} catch {
toast.error('Failed to open live view')
}
@@ -787,6 +875,9 @@ export default function App() {
type: zn.type as NodeData['type'],
status: 'unknown' as const,
services: [],
// The import already upserted the Device Inventory row; carry its id so
// the canvas save links to that row instead of minting a second one.
...(zn.device_id ? { device_id: zn.device_id } : {}),
...(zn.lqi != null ? { properties: [{ key: 'LQI', value: String(zn.lqi), icon: 'signal', visible: true }] } : {}),
...(zn.model ? { os: zn.model } : {}),
...(zn.parent_id ? { parent_id: zn.parent_id } : {}),
@@ -837,6 +928,8 @@ export default function App() {
type: zn.type as NodeData['type'],
status: 'unknown' as const,
services: [],
// Same as the Zigbee import: the row exists already, point at it.
...(zn.device_id ? { device_id: zn.device_id } : {}),
...(zn.model ? { os: zn.model } : {}),
...(zn.parent_id ? { parent_id: zn.parent_id } : {}),
},
@@ -862,42 +955,84 @@ export default function App() {
markUnsaved()
}, [addNode, onConnect, snapshotHistory, markUnsaved])
const handleProxmoxAddToCanvas = useCallback((pmNodes: ProxmoxNode[], pmEdges: ProxmoxEdge[]) => {
const handleProxmoxAddToCanvas = useCallback((
pmNodes: ProxmoxNode[],
pmEdges: ProxmoxEdge[],
mode: ProxmoxCanvasMode = 'linked',
) => {
snapshotHistory()
const COLS = 4
const SPACING_X = 190
const SPACING_Y = 110
const cols = Math.min(COLS, pmNodes.length)
const rows = Math.ceil(pmNodes.length / COLS)
const origin = getCenteredPosition(cols * SPACING_X, rows * SPACING_Y)
// Multiple hosts from one import = a cluster → chain them via left/right
// 'cluster' edges. Those endpoints need one left + one right handle each
// (both default to 0), so grant them to the host nodes up front.
const clusterEdges = buildProxmoxClusterEdges(pmNodes)
const cluster = clusterEdges.length > 0
pmNodes.forEach((pn, i) => {
const col = i % COLS
const row = Math.floor(i / COLS)
const position = { x: origin.x + col * SPACING_X, y: origin.y + row * SPACING_Y }
// Container mode: each host becomes a box and its guests are nested inside
// it. Anything the import gives no host for stays a loose node below.
const { hostIds, childrenByHost, hostOfChild, looseIds } = groupProxmoxGuests(pmNodes, pmEdges)
const hostIdSet = new Set(hostIds)
let placement: Record<string, { x: number; y: number }>
let hostSizes: Record<string, { width: number; height: number }> = {}
if (mode === 'container') {
const measured = measureProxmoxContainers(hostIds, childrenByHost, looseIds.length)
const origin = getCenteredPosition(measured.width, measured.height)
const laid = layoutProxmoxContainers(hostIds, childrenByHost, looseIds, origin)
placement = laid.positions
hostSizes = laid.hostSizes
} else {
const cols = Math.min(COLS, pmNodes.length)
const rows = Math.ceil(pmNodes.length / COLS)
const origin = getCenteredPosition(cols * SPACING_X, rows * SPACING_Y)
placement = {}
pmNodes.forEach((pn, i) => {
placement[pn.id] = {
x: origin.x + (i % COLS) * SPACING_X,
y: origin.y + Math.floor(i / COLS) * SPACING_Y,
}
})
}
// Hosts first: addNode resolves `data.parent_id` against the nodes already
// in the store, so a guest added before its container would not nest.
const ordered = mode === 'container'
? [...pmNodes.filter((n) => hostIdSet.has(n.id)), ...pmNodes.filter((n) => !hostIdSet.has(n.id))]
: pmNodes
ordered.forEach((pn) => {
const position = placement[pn.id] ?? getCenteredPosition()
const isClusterHost = cluster && pn.type === 'proxmox'
const isContainerHost = mode === 'container' && hostIdSet.has(pn.id)
const parentId = mode === 'container' ? hostOfChild[pn.id] : undefined
const size = hostSizes[pn.id]
const newNode: import('@xyflow/react').Node<NodeData> = {
id: pn.id,
type: pn.type,
position,
...(isContainerHost && size ? { width: size.width, height: size.height } : {}),
data: {
label: pn.label,
type: pn.type as NodeData['type'],
status: (pn.status === 'online' ? 'online' : 'unknown') as NodeData['status'],
services: [],
// Same as the Zigbee import: the row exists already, point at it.
...(pn.device_id ? { device_id: pn.device_id } : {}),
...(pn.ip ? { ip: pn.ip } : {}),
...(pn.hostname ? { hostname: pn.hostname } : {}),
...(isClusterHost ? { left_handles: 1, right_handles: 1 } : {}),
...(isContainerHost ? { container_mode: true } : {}),
...(parentId ? { parent_id: parentId } : {}),
},
}
addNode(newNode)
})
// Host → guest links render as 'virtual' edges (VM/LXC ↔ host).
// Host → guest links render as 'virtual' edges (VM/LXC ↔ host). A nested
// guest gets none: sitting inside the host already says it runs there, and
// the edge just loops out of the box and back (#399 follow-up).
pmEdges.forEach((pe) => {
if (hostOfChild[pe.target] === pe.source && mode === 'container') return
onConnect({
source: pe.source,
sourceHandle: 'bottom',
@@ -1032,6 +1167,14 @@ export default function App() {
</button>
</div>
)}
{appView === 'documentation' ? (
<Suspense
fallback={<div className="flex flex-1 items-center justify-center text-xs text-muted-foreground">Loading documentation…</div>}
>
<DocumentationView />
</Suspense>
) : (
<>
<Toolbar
onSave={handleSave}
onAutoLayout={handleAutoLayout}
@@ -1073,9 +1216,16 @@ export default function App() {
// Standalone has no Device Inventory to open (ADR-001 style
// gate: the whole scan surface is backend-only).
onOpenInventory={STANDALONE ? undefined : (deviceId) => openInventoryModal(deviceId)}
onOpenDocumentation={
STANDALONE
? undefined
: (deviceId, label) => void openDocumentationFor(deviceId, label)
}
/>
)}
</div>
</>
)}
</div>
</div>
@@ -1190,8 +1340,11 @@ export default function App() {
<GroupRectModal
open={addGroupRectOpen}
onClose={() => setAddGroupRectOpen(false)}
onClose={() => { pendingZoneSubnet.current = null; setAddGroupRectOpen(false) }}
onSubmit={handleAddGroupRect}
onImportSubnet={(cidr) => { pendingZoneSubnet.current = cidr }}
countSubnetMatches={(cidr) => countSubnetMatches(cidr)}
importOnSubmit
title="Add Zone"
/>
@@ -1202,12 +1355,17 @@ export default function App() {
onClose={() => setEditingGroupRectId(null)}
onSubmit={handleUpdateGroupRect}
onDelete={handleDeleteGroupRect}
onImportSubnet={(cidr) => { if (editingGroupRectId) handleImportSubnetIntoZone(editingGroupRectId, cidr) }}
countSubnetMatches={(cidr) => countSubnetMatches(cidr, editingGroupRectId ?? undefined)}
initial={(() => {
const n = editingGroupRectId ? nodes.find((nd) => nd.id === editingGroupRectId) : null
if (!n) return undefined
const rc = n.data.custom_colors ?? {}
return {
label: n.data.label,
// `notes` is where a zone description used to try to live; read it
// as a fallback so a canvas loaded before the fix still shows it.
description: n.data.description ?? n.data.notes ?? '',
font: rc.font ?? 'inter',
text_color: rc.text_color ?? '#e6edf3',
text_position: rc.text_position ?? 'top-left',
@@ -1274,15 +1432,23 @@ export default function App() {
open={searchOpen}
onClose={() => setSearchOpen(false)}
onOpenInventory={(deviceId) => openInventoryModal(deviceId)}
onOpenDocument={
STANDALONE
? undefined
: (docId) => {
setAppView('documentation')
void useDocsStore.getState().open(docId)
}
}
/>
<ShortcutsModal open={shortcutsOpen} onClose={() => setShortcutsOpen(false)} />
<ConfirmAddToGroupModal
open={!!pendingGroupAdd}
nodeLabel={pendingGroupAdd ? (nodes.find((n) => n.id === pendingGroupAdd.nodeId)?.data.label ?? '') : ''}
nodeLabels={pendingGroupAdd ? labelsOf(pendingGroupAdd.nodeIds) : []}
targetLabel={pendingGroupAdd ? (nodes.find((n) => n.id === pendingGroupAdd.groupId)?.data.label ?? '') : ''}
onConfirm={() => {
if (pendingGroupAdd) addToGroup(pendingGroupAdd.groupId, pendingGroupAdd.nodeId)
if (pendingGroupAdd) addNodesToGroup(pendingGroupAdd.groupId, pendingGroupAdd.nodeIds)
setPendingGroupAdd(null)
}}
onCancel={() => setPendingGroupAdd(null)}
@@ -1291,10 +1457,10 @@ export default function App() {
<ConfirmAddToGroupModal
open={!!pendingContainerAdd}
variant="container"
nodeLabel={pendingContainerAdd ? (nodes.find((n) => n.id === pendingContainerAdd.nodeId)?.data.label ?? '') : ''}
nodeLabels={pendingContainerAdd ? labelsOf(pendingContainerAdd.nodeIds) : []}
targetLabel={pendingContainerAdd ? (nodes.find((n) => n.id === pendingContainerAdd.containerId)?.data.label ?? '') : ''}
onConfirm={() => {
if (pendingContainerAdd) addToContainer(pendingContainerAdd.containerId, pendingContainerAdd.nodeId)
if (pendingContainerAdd) addNodesToContainer(pendingContainerAdd.containerId, pendingContainerAdd.nodeIds)
setPendingContainerAdd(null)
}}
onCancel={() => setPendingContainerAdd(null)}
@@ -1303,10 +1469,10 @@ export default function App() {
<ConfirmAddToGroupModal
open={!!pendingZoneAdd}
variant="zone"
nodeLabel={pendingZoneAdd ? (nodes.find((n) => n.id === pendingZoneAdd.nodeId)?.data.label ?? '') : ''}
nodeLabels={pendingZoneAdd ? labelsOf(pendingZoneAdd.nodeIds) : []}
targetLabel={pendingZoneAdd ? (nodes.find((n) => n.id === pendingZoneAdd.zoneId)?.data.label ?? '') : ''}
onConfirm={() => {
if (pendingZoneAdd) addToZone(pendingZoneAdd.zoneId, pendingZoneAdd.nodeId)
if (pendingZoneAdd) addNodesToZone(pendingZoneAdd.zoneId, pendingZoneAdd.nodeIds)
setPendingZoneAdd(null)
}}
onCancel={() => setPendingZoneAdd(null)}
@@ -1319,6 +1485,14 @@ export default function App() {
<DeviceInventoryModal
open={inventoryModalOpen}
onClose={() => setInventoryModalOpen(false)}
onOpenDocumentation={
STANDALONE
? undefined
: (deviceId, label) => {
setInventoryModalOpen(false)
void openDocumentationFor(deviceId, label)
}
}
highlightId={inventoryHighlightId}
initialStatus={inventoryModalStatus}
demoDevices={tourInventoryDemo ? DEMO_INVENTORY_DEVICES : undefined}
+18
View File
@@ -70,6 +70,19 @@ describe('api/client', () => {
expect(publicApi.defaults.baseURL).toBe('/api/v1')
})
it('mounts both instances under a configured base path', async () => {
const before = instances.length
vi.stubEnv('BASE_URL', '/homelab/')
vi.resetModules()
await import('../client')
const [subApi, subPublicApi] = instances.slice(before)
expect(subApi.defaults.baseURL).toBe('/homelab/api/v1')
expect(subApi.defaults.withCredentials).toBe(true)
expect(subPublicApi.defaults.baseURL).toBe('/homelab/api/v1')
vi.unstubAllEnvs()
vi.resetModules()
})
it('exports `api` matching the first created instance', () => {
expect(mod.api).toBe(api)
})
@@ -242,6 +255,11 @@ describe('api/client', () => {
expect(api.post).toHaveBeenCalledWith('/zigbee/import-pending', cfg)
})
it('zigbeeApi.getImportJob polls the job by id', () => {
mod.zigbeeApi.getImportJob('job-1')
expect(api.get).toHaveBeenCalledWith('/zigbee/import/job-1')
})
it('zwaveApi.testConnection/importNetwork/importToPending', () => {
const cfg = { mqtt_host: 'h', mqtt_port: 1883, prefix: 'zwave', gateway_name: 'zwavejs2mqtt' }
mod.zwaveApi.testConnection(cfg)
+112 -7
View File
@@ -1,15 +1,16 @@
import axios from 'axios'
import { API_BASE_URL } from '@/utils/basePath'
import { useAuthStore } from '@/stores/authStore'
import type { AuthMode, AuthUser } from '@/stores/authStore'
import type { InventoryEntry } from '@/types'
export const api = axios.create({
baseURL: '/api/v1',
baseURL: API_BASE_URL,
withCredentials: true,
})
// Unauthenticated axios instance — no JWT, no 401 redirect (used for public endpoints)
const publicApi = axios.create({ baseURL: '/api/v1' })
const publicApi = axios.create({ baseURL: API_BASE_URL })
api.interceptors.request.use((config) => {
const { token, csrfToken } = useAuthStore.getState()
@@ -109,6 +110,20 @@ export interface DeepScanConfig {
export type ScanConfigData = { ranges: string[] } & DeepScanConfig
/** A row of `scan_runs` — what `/scan/runs` and the device rescan return. */
export interface ScanRunSummary {
id: string
// 'failed' is legacy: runs recorded before the backend settled on 'error'
// for the same condition. Still read, never written.
status: 'running' | 'done' | 'cancelled' | 'error' | 'failed'
kind: string
ranges: string[]
devices_found: number
started_at: string
finished_at: string | null
error: string | null
}
// A device the backend refused to place because an equivalent node already
// exists on the target design (same ip/mac/ieee). `existing_node_id` points at
// the node already there so the UI can link to it.
@@ -158,8 +173,23 @@ export const scanApi = {
*/
updatePending: (id: string, data: Partial<Omit<InventoryEntry, 'id' | 'status' | 'discovered_at'>>) =>
api.patch<InventoryEntry>(`/scan/pending/${id}`, data),
/**
* Deep-rescan one known device: every TCP port, then re-fingerprint. Answers
* "this device predates the scanner knowing that service" (issue #350).
* Returns the ScanRun, so the caller polls `run` and can `stop` it.
* 409 when the device has no IP, is hidden, or is already being rescanned.
*/
rescanDevice: (id: string, opts?: { full_ports?: boolean; ports?: string; http_probe_enabled?: boolean; verify_tls?: boolean }) =>
api.post<ScanRunSummary>(`/scan/pending/${id}/rescan`, opts ?? {}),
hidden: () => api.get('/scan/hidden'),
/**
* Inventory rows for the guests a Proxmox host runs, resolved from the
* host→guest links the Proxmox import records. Empty for a non-host device.
*/
proxmoxChildren: (id: string) =>
api.get<InventoryEntry[]>(`/scan/pending/${id}/proxmox-children`),
runs: () => api.get('/scan/runs'),
run: (runId: string) => api.get<ScanRunSummary>(`/scan/runs/${runId}`),
clearPending: () => api.delete('/scan/pending'),
/** Remove one inventory entry. 409 when a rack still mounts it. */
deletePending: (id: string) => api.delete<{ deleted: boolean }>(`/scan/pending/${id}`),
@@ -182,6 +212,14 @@ export const scanApi = {
skipped: number
skipped_devices: SkippedDevice[]
}>('/scan/pending/bulk-approve', { device_ids: ids, design_id: designId ?? undefined }),
/**
* Fold several inventory rows into one, keeping `winnerId`. Returns the
* survivor. The losers' facts, canvas nodes, rack mounts and documents move
* onto it first — the identity match only runs when a row is created, so two
* rows for one host never converge on their own.
*/
merge: (winnerId: string, loserIds: string[]) =>
api.post<InventoryEntry>('/scan/pending/merge', { winner_id: winnerId, loser_ids: loserIds }),
bulkHide: (ids: string[]) => api.post<{ hidden: number; skipped: number }>('/scan/pending/bulk-hide', { device_ids: ids }),
restore: (id: string) => api.post<{ restored: boolean; device_id: string }>(`/scan/pending/${id}/restore`),
bulkRestore: (ids: string[]) => api.post<{ restored: number; skipped: number }>('/scan/pending/bulk-restore', { device_ids: ids }),
@@ -271,6 +309,62 @@ export const designsApi = {
delete: (id: string) => api.delete(`/designs/${id}`),
}
export const documentsApi = {
list: (params?: { kind?: string; parent_id?: string; device_id?: string; tag?: string }) =>
api.get<import('@/documentation/types').DocumentSummary[]>('/documents', { params }),
get: (id: string) => api.get<import('@/documentation/types').Doc>(`/documents/${id}`),
create: (data: {
title: string
kind?: string
parent_id?: string | null
device_id?: string | null
node_id?: string | null
design_id?: string | null
template_id?: string | null
body?: string
}) => api.post<import('@/documentation/types').Doc>('/documents', data),
update: (
id: string,
data: {
title?: string
icon?: string | null
body?: string
parent_id?: string | null
sort_order?: number
starred?: boolean
reviewed?: boolean
resync_facts?: boolean
},
) => api.patch<import('@/documentation/types').Doc>(`/documents/${id}`, data),
delete: (id: string) => api.delete(`/documents/${id}`),
revisions: (id: string) =>
api.get<import('@/documentation/types').DocRevision[]>(`/documents/${id}/revisions`),
revision: (revisionId: string) =>
api.get<import('@/documentation/types').DocRevision & { body: string }>(
`/documents/revisions/${revisionId}`,
),
restore: (id: string, revisionId: string) =>
api.post<import('@/documentation/types').Doc>(`/documents/${id}/revisions/${revisionId}/restore`),
backlinks: (id: string) =>
api.get<import('@/documentation/types').DocBacklink[]>(`/documents/${id}/backlinks`),
regenerate: (id: string) =>
api.post<import('@/documentation/types').Doc>(`/documents/${id}/regenerate`),
search: (q: string, limit = 25) =>
api.get<import('@/documentation/types').DocSearchResult>('/documents/search', {
params: { q, limit },
}),
block: (block: string, deviceId: string) =>
api.get<{ block: string; markdown: string }>('/documents/blocks', {
params: { block, device_id: deviceId },
}),
coverage: () => api.get<import('@/documentation/types').DocCoverage>('/documents/coverage'),
scaffold: (data: { device_ids?: string[]; only_with_notes?: boolean }) =>
api.post<{ created: import('@/documentation/types').DocumentSummary[]; skipped: number }>(
'/documents/scaffold',
data,
),
}
export const racksApi = {
load: (designId: string) =>
api.get<import('@/utils/rackSerializer').ApiRackState>('/racks', { params: { design_id: designId } }),
@@ -282,6 +376,8 @@ export const racksApi = {
}),
}
export type ZigbeeImportJobStatus = 'running' | 'done' | 'error'
export interface ZigbeeConfigData {
mqtt_host: string
mqtt_port: number
@@ -335,11 +431,20 @@ export const zigbeeApi = {
mqtt_tls?: boolean
mqtt_tls_insecure?: boolean
}) =>
api.post<{
nodes: import('@/components/zigbee/types').ZigbeeNode[]
edges: import('@/components/zigbee/types').ZigbeeEdge[]
device_count: number
}>('/zigbee/import', data),
api.post<{ job_id: string; status: ZigbeeImportJobStatus }>('/zigbee/import', data),
// Poll a canvas import started by importNetwork. The fetch runs server-side
// so a slow mesh cannot outlive a reverse proxy's read timeout.
getImportJob: (jobId: string) =>
api.get<{
job_id: string
status: ZigbeeImportJobStatus
result: {
nodes: import('@/components/zigbee/types').ZigbeeNode[]
edges: import('@/components/zigbee/types').ZigbeeEdge[]
device_count: number
} | null
}>(`/zigbee/import/${jobId}`),
importToPending: (data: {
mqtt_host: string
+2 -1
View File
@@ -7,6 +7,7 @@ import { Logo } from '@/components/ui/Logo'
import { authApi } from '@/api/client'
import { useAuthStore } from '@/stores/authStore'
import { cn } from '@/lib/utils'
import { API_BASE_URL, resolveServerPath } from '@/utils/basePath'
export function LoginPage() {
const [username, setUsername] = useState('')
@@ -111,7 +112,7 @@ export function LoginPage() {
Continue through the configured OpenID Connect provider.
</p>
<a
href={oidcLoginUrl ?? '/api/v1/auth/oidc/login'}
href={resolveServerPath(oidcLoginUrl ?? `${API_BASE_URL}/auth/oidc/login`)}
className={cn(
buttonVariants(),
'bg-[#00d4ff] text-[#0d1117] hover:bg-[#00d4ff]/90 font-medium',
@@ -11,6 +11,7 @@ import {
type Node,
type Edge,
type Connection,
type Viewport,
} from '@xyflow/react'
import { MousePointer2, Hand } from 'lucide-react'
import '@xyflow/react/dist/style.css'
@@ -32,9 +33,9 @@ interface CanvasContainerProps {
onEdgeDoubleClick?: (edge: Edge<EdgeData>) => void
onNodeDoubleClick?: (node: Node<NodeData>) => void
onNodeDragStart?: () => void
onRequestAddToGroup?: (payload: { nodeId: string; groupId: string }) => void
onRequestAddToContainer?: (payload: { nodeId: string; containerId: string }) => void
onRequestAddToZone?: (payload: { nodeId: string; zoneId: string }) => void
onRequestAddToGroup?: (payload: { nodeIds: string[]; groupId: string }) => void
onRequestAddToContainer?: (payload: { nodeIds: string[]; containerId: string }) => void
onRequestAddToZone?: (payload: { nodeIds: string[]; zoneId: string }) => void
onOpenInventory?: (deviceId: string) => void
}
@@ -45,10 +46,16 @@ export function CanvasContainer({ onConnect: onConnectProp, onEdgeDoubleClick, o
onNodesChange, onEdgesChange,
setSelectedNode, snapshotHistory,
fitViewPending, clearFitViewPending,
savedViewport, setSavedViewport,
copySelectedNodes, pasteNodes,
removeFromGroup,
removeNodesFromGroup,
} = useCanvasStore()
const { fitView, screenToFlowPosition, getIntersectingNodes } = useReactFlow<Node<NodeData>>()
const { fitView, getViewport, screenToFlowPosition, getIntersectingNodes } = useReactFlow<Node<NodeData>>()
// React Flow reads defaultViewport once, on mount. Leaving the canvas
// (Documentation) unmounts it, so hold the stored pan/zoom as it was at first
// render: later saves must not feed back into the prop.
const [initialViewport] = useState(savedViewport)
// Track the last cursor position over the canvas so paste lands under it.
const cursorRef = useRef<{ x: number; y: number } | null>(null)
@@ -93,11 +100,14 @@ export function CanvasContainer({ onConnect: onConnectProp, onEdgeDoubleClick, o
useEffect(() => {
if (!fitViewPending || nodes.length === 0) return
const id = setTimeout(() => {
fitView({ padding: 0.12, duration: 350 })
void fitView({ padding: 0.12, duration: 350 }).then(() => {
// Where the fit landed is the viewport to come back to.
setSavedViewport(getViewport())
})
clearFitViewPending()
}, 50)
return () => clearTimeout(id)
}, [fitViewPending, nodes.length, fitView, clearFitViewPending])
}, [fitViewPending, nodes.length, fitView, getViewport, setSavedViewport, clearFitViewPending])
const activeTheme = useThemeStore((s) => s.activeTheme)
const theme = THEMES[activeTheme]
@@ -125,6 +135,12 @@ export function CanvasContainer({ onConnect: onConnectProp, onEdgeDoubleClick, o
setSelectedNode(null)
}, [setSelectedNode])
// Every pan / zoom the user ends is remembered, so leaving the canvas and
// coming back does not reset the view to 1:1 at the origin.
const handleMoveEnd = useCallback((_: unknown, viewport: Viewport) => {
setSavedViewport(viewport)
}, [setSavedViewport])
const handleEdgeDoubleClick = useCallback((_: React.MouseEvent, edge: Edge<EdgeData>) => {
onEdgeDoubleClick?.(edge)
}, [onEdgeDoubleClick])
@@ -145,38 +161,61 @@ export function CanvasContainer({ onConnect: onConnectProp, onEdgeDoubleClick, o
const { guides, onNodeDrag, onNodeDragStop } = useAlignmentGuides()
// Drop a top-level node onto a group → ask App to confirm adding it. Runs
// before the alignment snap so detection uses the dropped position.
// Drop a selection onto a group → ask App to confirm adding it. Runs before
// the alignment snap so detection uses the dropped position. `dragNodes` is
// the whole multi-selection being dragged; `dragNode` (the one under the
// cursor) only picks the destination.
const handleNodeDragStop = useCallback<NonNullable<typeof onNodeDragStop>>((event, dragNode, dragNodes) => {
if (dragNode && dragNode.data.type !== 'group' && dragNode.data.type !== 'groupRect') {
// A single drag reports an empty `dragNodes` in some React Flow paths, so
// fall back to the dragged node itself.
const dragged = dragNodes && dragNodes.length > 0 ? dragNodes : dragNode ? [dragNode] : []
// Groups and zones are containers, never contents.
const movable = dragged.filter((n) => n.data.type !== 'group' && n.data.type !== 'groupRect')
if (dragNode && dragNode.data.type !== 'group' && dragNode.data.type !== 'groupRect' && movable.length > 0) {
const intersecting = getIntersectingNodes(dragNode)
const zoneParent = dragNode.parentId
? nodes.find((n) => n.id === dragNode.parentId && n.data.type === 'groupRect')
: undefined
if (zoneParent) {
// Zone children are not extent-clamped, so a drop outside the zone is
// how the user takes a node back out of it.
// how the user takes a node back out of it. The whole selection leaves
// with it, not only the node under the cursor.
if (!intersecting.some((n) => n.id === zoneParent.id)) {
removeFromGroup(zoneParent.id, dragNode.id)
const leaving = movable.filter((n) => n.parentId === zoneParent.id).map((n) => n.id)
// One history entry, so a single undo puts the whole selection back —
// symmetric with the batched add.
removeNodesFromGroup(zoneParent.id, leaving)
}
} else if (!dragNode.parentId) {
// Only free nodes join a new parent; one already nested elsewhere in the
// selection keeps its own parent.
const nodeIds = movable.filter((n) => !n.parentId).map((n) => n.id)
const group = intersecting.find((n) => n.data.type === 'group')
const container = intersecting.find((n) => n.id !== dragNode.id && n.data.container_mode === true)
// The destination can be part of the dragged selection (lasso over
// everything); it cannot also be one of its own new children.
const targetless = (targetId: string) => nodeIds.filter((id) => id !== targetId)
if (group) {
onRequestAddToGroup?.({ nodeId: dragNode.id, groupId: group.id })
const ids = targetless(group.id)
if (ids.length > 0) onRequestAddToGroup?.({ nodeIds: ids, groupId: group.id })
} else if (container) {
// Any node in container_mode (proxmox, docker_host, …) accepts children.
onRequestAddToContainer?.({ nodeId: dragNode.id, containerId: container.id })
const ids = targetless(container.id)
if (ids.length > 0) onRequestAddToContainer?.({ nodeIds: ids, containerId: container.id })
} else {
// Zones come last: they are the loosest container and the largest, so
// a group/container inside one still wins the drop.
const zone = intersecting.find((n) => n.data.type === 'groupRect')
if (zone) onRequestAddToZone?.({ nodeId: dragNode.id, zoneId: zone.id })
if (zone) {
const ids = targetless(zone.id)
if (ids.length > 0) onRequestAddToZone?.({ nodeIds: ids, zoneId: zone.id })
}
}
}
}
onNodeDragStop(event, dragNode, dragNodes)
}, [onRequestAddToGroup, onRequestAddToContainer, onRequestAddToZone, removeFromGroup, nodes, getIntersectingNodes, onNodeDragStop])
}, [onRequestAddToGroup, onRequestAddToContainer, onRequestAddToZone, removeNodesFromGroup, nodes, getIntersectingNodes, onNodeDragStop])
return (
<div ref={wrapperRef} className="w-full h-full" style={{ background: theme.colors.canvasBackground }} onMouseMove={onMouseMove}>
@@ -193,6 +232,8 @@ export function CanvasContainer({ onConnect: onConnectProp, onEdgeDoubleClick, o
onNodeDragStart={onNodeDragStart}
onNodeDrag={onNodeDrag}
onNodeDragStop={handleNodeDragStop}
onMoveEnd={handleMoveEnd}
defaultViewport={initialViewport ?? undefined}
nodeTypes={nodeTypes}
edgeTypes={edgeTypes}
deleteKeyCode={['Backspace', 'Delete']}
@@ -1,6 +1,7 @@
import { useCallback, useEffect, useRef, useState } from 'react'
import { ViewportPortal, useReactFlow, useStore } from '@xyflow/react'
import { useCanvasStore } from '@/stores/canvasStore'
import { resolveServerPath } from '@/utils/basePath'
interface ResizeState {
startMouseX: number
@@ -152,7 +153,7 @@ export function FloorMapLayer() {
onDoubleClick={locked ? undefined : (e) => { e.stopPropagation(); requestFloorMapEdit() }}
>
<img
src={imageData}
src={resolveServerPath(imageData)}
alt="Floor plan"
draggable={false}
style={{
@@ -1,5 +1,5 @@
import { describe, it, expect, vi, beforeEach } from 'vitest'
import { render } from '@testing-library/react'
import { act, render } from '@testing-library/react'
import { CanvasContainer } from '../CanvasContainer'
import { useCanvasStore } from '@/stores/canvasStore'
import { useThemeStore } from '@/stores/themeStore'
@@ -10,7 +10,11 @@ import type { NodeData, EdgeData } from '@/types'
let rfProps: Record<string, unknown> = {}
// Hoisted holder so the mock factory can read the configurable intersection set.
const rf = vi.hoisted(() => ({ intersecting: [] as unknown[] }))
const rf = vi.hoisted(() => ({
intersecting: [] as unknown[],
viewport: { x: 0, y: 0, zoom: 1 },
fitView: vi.fn(() => Promise.resolve(true)),
}))
vi.mock('@xyflow/react', () => ({
ReactFlow: (props: Record<string, unknown>) => {
@@ -25,7 +29,8 @@ vi.mock('@xyflow/react', () => ({
SelectionMode: { Partial: 'partial' },
Position: { Top: 'top', Right: 'right', Bottom: 'bottom', Left: 'left' },
useReactFlow: () => ({
fitView: vi.fn(),
fitView: rf.fitView,
getViewport: () => rf.viewport,
screenToFlowPosition: vi.fn(),
getIntersectingNodes: () => rf.intersecting,
setNodes: vi.fn(),
@@ -52,7 +57,12 @@ describe('CanvasContainer', () => {
beforeEach(() => {
rfProps = {}
rf.intersecting = []
useCanvasStore.setState({ nodes: [], edges: [], selectedNodeId: null })
rf.viewport = { x: 0, y: 0, zoom: 1 }
rf.fitView.mockClear()
useCanvasStore.setState({
nodes: [], edges: [], selectedNodeId: null,
savedViewport: null, fitViewPending: false,
})
useThemeStore.setState({ activeTheme: 'default' })
})
@@ -177,7 +187,7 @@ describe('CanvasContainer', () => {
rf.intersecting = [group]
render(<CanvasContainer onRequestAddToGroup={onRequestAddToGroup} />)
;(rfProps.onNodeDragStop as (...args: unknown[]) => unknown)({} as MouseEvent, node, [node])
expect(onRequestAddToGroup).toHaveBeenCalledWith({ nodeId: 'n1', groupId: 'g1' })
expect(onRequestAddToGroup).toHaveBeenCalledWith({ nodeIds: ['n1'], groupId: 'g1' })
})
it('does not fire onRequestAddToGroup when no group is under the node', () => {
@@ -219,7 +229,7 @@ describe('CanvasContainer', () => {
rf.intersecting = [containerNode('px1')]
render(<CanvasContainer onRequestAddToContainer={onRequestAddToContainer} />)
;(rfProps.onNodeDragStop as (...args: unknown[]) => unknown)({} as MouseEvent, node, [node])
expect(onRequestAddToContainer).toHaveBeenCalledWith({ nodeId: 'n1', containerId: 'px1' })
expect(onRequestAddToContainer).toHaveBeenCalledWith({ nodeIds: ['n1'], containerId: 'px1' })
})
it('prefers a group over a container when both intersect', () => {
@@ -229,7 +239,7 @@ describe('CanvasContainer', () => {
rf.intersecting = [containerNode('px1'), groupNode('g1')]
render(<CanvasContainer onRequestAddToGroup={onRequestAddToGroup} onRequestAddToContainer={onRequestAddToContainer} />)
;(rfProps.onNodeDragStop as (...args: unknown[]) => unknown)({} as MouseEvent, node, [node])
expect(onRequestAddToGroup).toHaveBeenCalledWith({ nodeId: 'n1', groupId: 'g1' })
expect(onRequestAddToGroup).toHaveBeenCalledWith({ nodeIds: ['n1'], groupId: 'g1' })
expect(onRequestAddToContainer).not.toHaveBeenCalled()
})
@@ -263,7 +273,7 @@ describe('CanvasContainer', () => {
rf.intersecting = [zoneNode('z1')]
render(<CanvasContainer onRequestAddToZone={onRequestAddToZone} />)
;(rfProps.onNodeDragStop as (...args: unknown[]) => unknown)({} as MouseEvent, node, [node])
expect(onRequestAddToZone).toHaveBeenCalledWith({ nodeId: 'n1', zoneId: 'z1' })
expect(onRequestAddToZone).toHaveBeenCalledWith({ nodeIds: ['n1'], zoneId: 'z1' })
})
it('prefers a group and a container over a zone when they overlap', () => {
@@ -273,7 +283,7 @@ describe('CanvasContainer', () => {
rf.intersecting = [zoneNode('z1'), containerNode('px1')]
render(<CanvasContainer onRequestAddToZone={onRequestAddToZone} onRequestAddToContainer={onRequestAddToContainer} />)
;(rfProps.onNodeDragStop as (...args: unknown[]) => unknown)({} as MouseEvent, node, [node])
expect(onRequestAddToContainer).toHaveBeenCalledWith({ nodeId: 'n1', containerId: 'px1' })
expect(onRequestAddToContainer).toHaveBeenCalledWith({ nodeIds: ['n1'], containerId: 'px1' })
expect(onRequestAddToZone).not.toHaveBeenCalled()
})
@@ -308,6 +318,64 @@ describe('CanvasContainer', () => {
expect(useCanvasStore.getState().nodes.find((n) => n.id === 'n1')?.parentId).toBe('z1')
})
// ── Multi-selection drops (#365) ──────────────────────────────────────────
it('fires onRequestAddToZone with every dragged node, not just the one under the cursor', () => {
const onRequestAddToZone = vi.fn()
const dragged = [makeNode('n1'), makeNode('n2'), makeNode('n3')]
rf.intersecting = [zoneNode('z1')]
render(<CanvasContainer onRequestAddToZone={onRequestAddToZone} />)
;(rfProps.onNodeDragStop as (...args: unknown[]) => unknown)({} as MouseEvent, dragged[0], dragged)
expect(onRequestAddToZone).toHaveBeenCalledWith({ nodeIds: ['n1', 'n2', 'n3'], zoneId: 'z1' })
})
it('leaves zones and groups out of a dragged selection', () => {
const onRequestAddToZone = vi.fn()
const dragged = [makeNode('n1'), zoneNode('z1'), groupNode('g1')]
rf.intersecting = [zoneNode('z2')]
render(<CanvasContainer onRequestAddToZone={onRequestAddToZone} />)
;(rfProps.onNodeDragStop as (...args: unknown[]) => unknown)({} as MouseEvent, dragged[0], dragged)
expect(onRequestAddToZone).toHaveBeenCalledWith({ nodeIds: ['n1'], zoneId: 'z2' })
})
it('never asks to add the destination container to itself', () => {
const onRequestAddToContainer = vi.fn()
const target = containerNode('px1')
const dragged = [makeNode('n1'), target]
rf.intersecting = [target]
render(<CanvasContainer onRequestAddToContainer={onRequestAddToContainer} />)
;(rfProps.onNodeDragStop as (...args: unknown[]) => unknown)({} as MouseEvent, dragged[0], dragged)
expect(onRequestAddToContainer).toHaveBeenCalledWith({ nodeIds: ['n1'], containerId: 'px1' })
})
it('keeps an already-parented node in the selection with its own parent', () => {
const onRequestAddToZone = vi.fn()
const dragged = [makeNode('n1'), { ...makeNode('n2'), parentId: 'pxOther' }]
rf.intersecting = [zoneNode('z1')]
render(<CanvasContainer onRequestAddToZone={onRequestAddToZone} />)
;(rfProps.onNodeDragStop as (...args: unknown[]) => unknown)({} as MouseEvent, dragged[0], dragged)
expect(onRequestAddToZone).toHaveBeenCalledWith({ nodeIds: ['n1'], zoneId: 'z1' })
})
it('detaches every dragged child when a selection leaves its zone', () => {
const zone = { ...zoneNode('z1'), position: { x: 100, y: 100 } }
const c1 = { ...makeNode('n1'), parentId: 'z1', position: { x: 20, y: 20 } }
const c2 = { ...makeNode('n2'), parentId: 'z1', position: { x: 40, y: 40 } }
useCanvasStore.setState({ nodes: [zone, c1, c2] })
rf.intersecting = []
render(<CanvasContainer />)
;(rfProps.onNodeDragStop as (...args: unknown[]) => unknown)({} as MouseEvent, c1, [c1, c2])
const after = useCanvasStore.getState().nodes
expect(after.find((n) => n.id === 'n1')!.parentId).toBeUndefined()
expect(after.find((n) => n.id === 'n2')!.parentId).toBeUndefined()
expect(after.find((n) => n.id === 'n2')!.position).toEqual({ x: 140, y: 140 })
// One history entry for the whole selection: a single undo puts both back.
useCanvasStore.getState().undo()
const restored = useCanvasStore.getState().nodes
expect(restored.find((n) => n.id === 'n1')!.parentId).toBe('z1')
expect(restored.find((n) => n.id === 'n2')!.parentId).toBe('z1')
})
// ── Canvas settings ───────────────────────────────────────────────────────
it('enables snapToGrid', () => {
@@ -386,4 +454,60 @@ describe('CanvasContainer', () => {
expect(snapshotHistory).toHaveBeenCalledOnce()
expect(result).toBe(true)
})
// ── Viewport persistence across unmount (#docs round-trip) ────────────────
it('onMoveEnd stores the viewport the user ended on', () => {
render(<CanvasContainer />)
act(() => {
;(rfProps.onMoveEnd as (...args: unknown[]) => unknown)({}, { x: -300, y: 120, zoom: 0.5 })
})
expect(useCanvasStore.getState().savedViewport).toEqual({ x: -300, y: 120, zoom: 0.5 })
})
it('mounts at the stored viewport — leaving for Documentation must not reset to 1:1', () => {
useCanvasStore.setState({ savedViewport: { x: -300, y: 120, zoom: 0.5 } })
render(<CanvasContainer />)
expect(rfProps.defaultViewport).toEqual({ x: -300, y: 120, zoom: 0.5 })
})
it('keeps the stored viewport across an unmount / remount round-trip', () => {
const first = render(<CanvasContainer />)
act(() => {
;(rfProps.onMoveEnd as (...args: unknown[]) => unknown)({}, { x: -80, y: 15, zoom: 0.75 })
})
first.unmount()
render(<CanvasContainer />)
expect(rfProps.defaultViewport).toEqual({ x: -80, y: 15, zoom: 0.75 })
})
it('leaves defaultViewport undefined on a first load, so React Flow fits', () => {
render(<CanvasContainer />)
expect(rfProps.defaultViewport).toBeUndefined()
})
it('does not feed a later pan back into defaultViewport (read once, on mount)', () => {
render(<CanvasContainer />)
act(() => {
;(rfProps.onMoveEnd as (...args: unknown[]) => unknown)({}, { x: -80, y: 15, zoom: 0.75 })
})
expect(rfProps.defaultViewport).toBeUndefined()
})
it('stores where fitView landed, so the fitted view survives the round-trip', async () => {
vi.useFakeTimers()
try {
rf.viewport = { x: -42, y: 7, zoom: 0.6 }
useCanvasStore.setState({ nodes: [makeNode('n1')], fitViewPending: true })
render(<CanvasContainer />)
await act(async () => {
await vi.advanceTimersByTimeAsync(100)
})
expect(rf.fitView).toHaveBeenCalledOnce()
expect(useCanvasStore.getState().savedViewport).toEqual({ x: -42, y: 7, zoom: 0.6 })
expect(useCanvasStore.getState().fitViewPending).toBe(false)
} finally {
vi.useRealTimers()
}
})
})
@@ -1,5 +1,5 @@
import { describe, it, expect, beforeEach, vi } from 'vitest'
import { render, screen, fireEvent } from '@testing-library/react'
import { render, screen, fireEvent, act } from '@testing-library/react'
import { FloorMapLayer } from '../FloorMapLayer'
import { useCanvasStore } from '@/stores/canvasStore'
import type { FloorMapConfig } from '@/types'
@@ -43,6 +43,16 @@ describe('FloorMapLayer', () => {
expect(container.querySelector('img')).toBeNull()
})
it('renders the stored image URL as-is at the root, base64 plans included', () => {
setFloorMap()
const { rerender } = render(<FloorMapLayer />)
expect(screen.getByAltText('Floor plan').getAttribute('src')).toBe('/api/v1/media/abc.png')
act(() => setFloorMap({ imageData: 'data:image/png;base64,abc' }))
rerender(<FloorMapLayer />)
expect(screen.getByAltText('Floor plan').getAttribute('src')).toBe('data:image/png;base64,abc')
})
it('hides resize handles until the plan is selected, then shows them (unlocked)', () => {
setFloorMap()
render(<FloorMapLayer />)
@@ -1,4 +1,6 @@
import { describe, it, expect } from 'vitest'
import { readFileSync } from 'node:fs'
import { resolve } from 'node:path'
import { render } from '@testing-library/react'
import { ReactFlowProvider } from '@xyflow/react'
import type { EdgeProps, Edge } from '@xyflow/react'
@@ -13,15 +15,18 @@ import type { EdgeData } from '@/types'
* CSS animations pause when the tab is hidden and don't leak — so the rendered
* output must contain a CSS `animation` on the path and zero <animate> nodes.
*/
function renderEdge(data: Partial<EdgeData> = {}) {
function renderEdge(
data: Partial<EdgeData> = {},
coords: { sourceY?: number; targetY?: number } = {},
) {
const props = {
id: 'e1',
source: 'a',
target: 'b',
sourceX: 0,
sourceY: 0,
sourceY: coords.sourceY ?? 0,
targetX: 100,
targetY: 100,
targetY: coords.targetY ?? 100,
sourcePosition: 'bottom',
targetPosition: 'top',
data: { type: 'ethernet', ...data } as EdgeData,
@@ -65,6 +70,53 @@ describe('HomelableEdge animation', () => {
expect(animated).toBeTruthy()
})
/**
* Regression (#395): the basic dash used to flip to `animation-direction:
* reverse` whenever the source sat below the target on screen, so an edge
* drawn bottom-to-top animated as if it went target→source. Direction comes
* from the data (the path is built source→target), never from the geometry.
*/
it('basic dash animates source→target when the target is below', () => {
const { container } = renderEdge({ animated: 'basic' }, { sourceY: 0, targetY: 100 })
const dashed = Array.from(container.querySelectorAll('path')).find((p) =>
(p.getAttribute('style') ?? '').includes('homelable-basic-dash'),
)
expect(dashed).toBeTruthy()
expect(dashed!.getAttribute('style') ?? '').not.toContain('reverse')
})
it('basic dash keeps the same direction when the target is above', () => {
const { container } = renderEdge({ animated: 'basic' }, { sourceY: 100, targetY: 0 })
const dashed = Array.from(container.querySelectorAll('path')).find((p) =>
(p.getAttribute('style') ?? '').includes('homelable-basic-dash'),
)
expect(dashed).toBeTruthy()
expect(dashed!.getAttribute('style') ?? '').not.toContain('reverse')
})
/**
* Regression (#395): the basic dash marched against snake and flow because
* its keyframes decreased stroke-dashoffset while theirs increase it. All
* three must move the same way along the path.
*/
it('basic dash keyframes march the same way as snake and flow', () => {
const css = readFileSync(resolve(__dirname, '../../../../index.css'), 'utf8')
const offsets = (name: string) => {
const block = css.match(new RegExp(`@keyframes ${name} \\{([^}]*\\}[^}]*)\\}`))
expect(block, `${name} keyframes not found`).toBeTruthy()
const values = [...block![1].matchAll(/stroke-dashoffset:\s*(-?\d+)/g)].map((m) => Number(m[1]))
expect(values).toHaveLength(2)
return values
}
const rising = (name: string) => {
const [from, to] = offsets(name)
return to > from
}
expect(rising('homelable-basic-dash')).toBe(true)
expect(rising('homelable-snake')).toBe(true)
expect(rising('homelable-flow')).toBe(true)
})
it('non-animated edge has no flow animation and no <animate>', () => {
const { container } = renderEdge({ animated: false })
expect(container.querySelector('animate')).toBeNull()
@@ -442,8 +442,11 @@ export function HomelableEdge({ id, source, target, sourceHandleId, targetHandle
strokeDasharray="5"
style={{
pointerEvents: 'none',
// Direction comes from the keyframes alone (see index.css), never
// from the endpoints' screen positions: an edge drawn bottom-to-top
// marches the same way as one drawn top-to-bottom, like the snake
// and flow modes.
animation: 'homelable-basic-dash 0.5s linear infinite',
animationDirection: sourceY <= targetY ? 'normal' : 'reverse',
}}
/>
)}

Some files were not shown because too many files have changed in this diff Show More