Compare commits

..
13 Commits
Author SHA1 Message Date
Pouzor 390a712f14 chore: bump version to 3.3.5
ha-relevant: no
2026-08-21 12:56:17 +02:00
Pouzor 9daea4c4d0 fix(scan): keep an edit, a discovery source and one name for a failed run
Three findings from the PR review.

A finishing deep rescan threw away an edit in progress. The reset effect in
InventoryDeviceModal was keyed on the `device` object, and the parent hands
down a fresh one whenever the row is refreshed — including from the poll's
own onSaved. Same device, new object, so the effect reset the form and left
edit mode, minutes into a scan the user was waiting on. It is keyed on the
id now: a refreshed row is not a different device, and only a different
device is a reason to throw the form away. The fresh row still reaches the
canvas and the grid — gating that behind edit mode would discard the scan
result instead, and a save unions the services server-side anyway.

A rescan tagged every device it touched as "arp"-discovered, so a Proxmox
guest, a rack mount or a hand-added host started answering the network
source filter. It carries its own source through instead.

The two background wrappers wrote status "failed" where the scanners write
"error" for the same condition. Harmonized on "error" — the one Scan
History filters and colours; "failed" showed up unlabelled. The frontend
union keeps 'failed' for rows already recorded.

That last one meant updating an existing assertion in test_scan_run.py: it
encoded the old spelling.

ha-relevant: yes
2026-08-21 12:49:48 +02:00
Pouzor 2e642815f4 fix(scan): stop a scan from repainting hand-picked service icons
merge_services did {**existing, **incoming}, so the fingerprint's guess at
an icon overwrote the one the user chose — and on a port no signature
covers it wrote None, clearing it outright. Since 3.3.0 the inventory row
is the only copy of a device's services, so every "Scan network" repainted
the service on every canvas drawing that device at once.

The scanner now merges with discovered=True: it still adds services and
refreshes what it knows, but leaves an established icon and category alone.
A user edit from the modal or a canvas changes them as before.

Blank incoming values no longer clear established ones either, on both
paths — an absent field is silence, not a reset. Same rule merge_properties
already follows.

ha-relevant: yes
2026-08-21 12:49:48 +02:00
Pouzor c6d6b525ea feat(scan): choose the port range before a deep scan
The Deep scan link on a device now opens a small dialog instead of firing
straight away. It is prefilled with the full 1-65535 range — that is still
the point of the feature — but a user who knows where a service lives can
narrow it and get an answer in seconds instead of minutes.

The dialog takes an nmap-style spec: a port, a range, or a comma list
(80,443,8000-9000). It shows the port count live, refuses to start on
something nmap could not use, and carries three presets (all / 1-1024 /
1-10000).

Backend:
- _parse_port_spec / _port_chunks generalize the slicing that used to be
  full-range only. Ranges are merged before slicing, so an overlapping
  spec is never scanned twice, and small ranges are packed into one nmap
  call instead of one call each. _deep_port_chunks still yields the same
  eight slices as before.
- run_device_scan takes ports=; it wins over full_ports.
- The retry-free flags (--max-retries 0 --min-rate 2000) now key on the
  total port count rather than on full_ports. They pay for themselves over
  thousands of ports on a lossy host; over a handful they only cost
  accuracy.
- RescanDeviceRequest.ports validates the spec — 422 rather than handing
  nmap a bad -p. Blank means the full sweep.

Frontend utils/portSpec.ts mirrors the backend parser so a typo is caught
before the request; the backend validates again because it is the one
calling nmap.

The existing rescan tests now go through the dialog: the click path
changed, so the start is two steps.

ha-relevant: yes
2026-08-21 12:49:48 +02:00
Pouzor fbb660504e fix(scan): slice the deep rescan instead of timing out the host
A deep rescan of a slow host came back with nothing at all: the run took
its full 600s ceiling and the device's services were unchanged, so a
service deleted by hand was never rediscovered.

nmap answers --host-timeout with "Skipping host <ip> due to host timeout"
and discards every port it had already found — the ceiling turned a slow
scan into one that reports nothing. What costs the time is a host that
drops packets: 8188 of 8192 ports filtered, each waiting out its probe.

- No --host-timeout on the deep discovery pass, ever.
- The full range runs as 8 slices of 8192 ports, one nmap call each,
  unioning the open ports. A slice that overruns costs its own ports, not
  all of them, and the loop has somewhere to notice a stop request.
- `scanner_deep_host_timeout` is now a total budget checked between
  slices (default 2700s), not an nmap flag. The first slice always runs.
- A partial sweep is reported rather than passed off as complete: the run
  finishes `done` carrying "Scanned 3/8 port ranges …", and the modal
  toasts a warning instead of success.
- Deep slices use --max-retries 0 --min-rate 2000. Measured against a
  dropping host, 8192 ports took 329s at --max-retries 1 and 164s at 0,
  finding the same ports; capping the RTT changed nothing. The range scan
  keeps nmap's default retries on its curated port list.

ha-relevant: yes
2026-08-21 12:49:48 +02:00
Pouzor 1be96d1045 feat(scan): deep-rescan one device from the inventory detail
Devices added before the scanner knew a service showed an empty Services
section with no way to refresh it (#350). The detail modal now starts a
full-port scan of that single device.

- `process_host` lifted out of `run_scan` so the range scan and the new
  single-device scan share the same match / merge / dedupe rules — a
  rescan unions services, it never replaces what the user added by hand.
- `run_device_scan`: no ping sweep, no mDNS, straight to the phase-2 nmap
  pass on the device IP over all 65535 TCP ports.
- `POST /scan/pending/{id}/rescan` records a normal ScanRun
  (`kind=device`, `ranges=["<ip>/32"]`), so stop, progress and Scan
  History work unchanged. One run per device at a time — a second request
  while the first is scanning is a 409. 404 unknown, 409 no-IP or hidden.
- `GET /scan/runs/{id}` so a caller can poll the run it started.
- Deep scan button in the Services section of the device detail, hidden
  without an IP and for Zigbee. Swaps to a stop control while running,
  folds the fresh services back in on completion (never over an edit in
  progress).

ha-relevant: yes
2026-08-21 12:49:48 +02:00
Pouzor 9bc02d61c2 fix(canvas): give a valueless property label the full node width
The property line caps its label at max-w-15 so a long key cannot crowd
out the value drawn beside it. When the value is empty there is nothing
to protect, but the cap still applied, truncating the label against
empty space.

Drop the cap when the value is blank and let the label truncate against
the node width instead. Same fix in BaseNode and ProxmoxGroupNode.

Closes #361

ha-relevant: yes
2026-08-17 20:26:29 +02:00
Pouzor 21a64e52ab fix(auth): accept the app's own origin in the OIDC CSRF check
A same-origin deployment behind a reverse proxy needs no CORS, so
CORS_ORIGINS is left at its localhost default — but browsers still send
Origin on unsafe methods, and OIDCCSRFMiddleware validated it against
CORS_ORIGINS alone. Every POST/PUT/DELETE came back 403 while reads
worked, so creating a canvas failed with no usable error.

The OIDC callback URL is served by this app, so the origin of
OIDC_REDIRECT_URI is the app's own and is always a legitimate CSRF
origin. origin_is_allowed now accepts it in addition to CORS_ORIGINS,
and settings validation warns at startup when CORS_ORIGINS omits it so
the misconfiguration is visible rather than silent.

Fixes #356

ha-relevant: no
2026-08-17 20:13:04 +02:00
Pouzor ad0b1b427d chore: bump version to 3.3.4
ha-relevant: no
2026-08-17 19:19:25 +02:00
Pouzor 4c460cec1c fix(scan): stop a re-scan from deleting a device's hand-added services
Since 3.3.0 the inventory row is the only copy of a device's services and
every canvas drawing it reads that list, but the scanner still assigned
`keep.services = fingerprint_ports(...)` — the pre-split behaviour, when a
node held its own copy. One re-scan therefore deleted every service the user
added by hand, on every canvas at once, and brought back the ones they had
deleted. It unions now, like every other writer of that field.

Two more things #347 turned up:

* A node whose view matches nothing the row still holds drew nothing at all —
  the row had been replaced under it, so every key was gone and every key was
  new, and `apply_view` hid the lot. Such a view says nothing about the list
  that replaced it, so it is treated as no view: the row is drawn. An empty
  view is untouched, being a real answer ("this canvas draws none of them").

* The 3.3.3 view seed recovers a node's arrangement from the pre-3.3.0 backup,
  which is 3.2.0-era and cannot know about a property added afterwards. On
  3.3.0-3.3.2 the row was the only place to add one and every canvas drew it,
  so seeding strictly from the backup took it off all of them at once. Those
  are appended visible, keeping the recovered order and hidden flags for
  everything the backup does know. Services keep the strict recovery: holding
  back what a scan fingerprinted is the whole point of the view.

ha-relevant: maybe
2026-08-17 19:05:13 +02:00
Pouzor 1aa88de491 chore: bump version to 3.3.3
ha-relevant: no
2026-08-17 14:09:01 +02:00
Pouzor d0679ad72b fix(inventory): stop the view seed from re-reading the backup every boot
The seed selected any node with a `device_id` and no view. Deleting a device
leaves that column dangling — SQLite runs with foreign keys off, so the
`ON DELETE SET NULL` never fires on an existing table — and such a node can be
seeded from nothing, so it stayed selected. The query matched it again on every
later start, re-opening the pre-upgrade backup and logging a recovery that
recovered nothing.

Narrowing the query to nodes whose row actually exists restores the invariant
the seed is documented to have: it runs on one boot, and never opens a file
again.

ha-relevant: no
2026-08-17 14:06:09 +02:00
Pouzor 5df70ce128 feat(canvas): give each node its own view of a device's services and properties
Since 3.3.0 the inventory row owns a device's services and properties, and
every node drawing that device rendered the row wholesale. One row shared by
several canvases meant one rendering: a service the scanner fingerprinted
appeared on every canvas at once (users reported Uptime Kuma and Synology DSM
on hosts running neither — both are port-only signatures), and a property added
on one schematic showed up on all the others.

Order and visibility are presentation, so they move to the node. `display_view`
records, per node, which of the row's services and properties it draws and in
what order, keyed by `port|protocol|name` and by lowercased property key so the
view survives an edit to the fact itself. The facts stay on the row: hiding is
per node, deleting is still device-wide.

An item the view does not list is reported hidden rather than dropped, so what
a scan finds next is one toggle away instead of pushed onto every canvas.

The wire shape is unchanged. A client already sends its lists in display order
with their `visible` flags, so the view is read back out of them rather than
asking for a second field, and `visible` is only stamped when something is
hidden.

Upgrades keep what each canvas showed:

* from 3.2.0, the backfill seeds each node's view from its own legacy columns
  before they are dropped;
* from 3.3.0-3.3.2 those columns are gone and the row holds the union of every
  canvas, so the layout is recovered from the backup `_backup_db` took before
  the 3.3.0 migration — the newest one whose `nodes` table still has the
  columns, read read-only and matched by node id;
* with no usable backup the row is the seed, so every canvas keeps showing
  exactly what it shows today and only later additions are held back.

Refs #347

ha-relevant: maybe
2026-08-17 14:06:09 +02:00
34 changed files with 3380 additions and 117 deletions
+32
View File
@@ -5,6 +5,38 @@ All notable changes to **Homelable** are documented here.
The format is loosely based on [Keep a Changelog](https://keepachangelog.com/),
and this project adheres to [Semantic Versioning](https://semver.org/).
## [3.3.5] - 2026-08-21
### Features
- Deep-scan a single device from its inventory detail: the Services section gets a **Deep scan** button that sweeps the device over a port range you pick, prefilled with the full 1-65535 range and narrowable when you know where the service lives. The run is a normal Scan Run, so stop, progress and Scan History work as usual, and the fresh services are merged in — never replacing what you added by hand. (#363)
### Fixes
- A deep rescan of a slow host no longer comes back empty. The run hit its timeout and nmap discarded every port it had already found; the full range is now scanned in slices, a partial sweep is reported as partial instead of passed off as complete, and the timeout is a total budget between slices rather than an nmap flag. (#363)
- A scan no longer repaints the service icons you picked by hand: the fingerprint's guess overwrote your choice, and cleared it outright on a port no signature covers. Since 3.3.0 the inventory row is the only copy, so one scan changed the service on every canvas at once. (#363)
- A rescan finishing while you edit a device no longer throws the edit away, and it no longer tags every device it touches as network-discovered — a Proxmox guest, a rack mount or a hand-added host keeps its own source. A failed run is also recorded as `error`, the status Scan History filters and colours. (#363)
- A property with no value gets the full node width for its label instead of being truncated against empty space. (#362)
- A same-origin OIDC deployment behind a reverse proxy no longer answers 403 to every write. The CSRF check validated the browser's `Origin` against `CORS_ORIGINS` alone; the app's own origin, taken from `OIDC_REDIRECT_URI`, is accepted too, and startup warns when `CORS_ORIGINS` omits it. (#360)
## [3.3.4] - 2026-08-17
### Fixes
- A network scan no longer deletes the services you added by hand. The scanner replaced a device's whole service list with what it fingerprinted, so one scan wiped manual entries on every canvas at once and brought back the ones you had deleted. It merges now. (#347)
- A node whose services were all replaced under it no longer renders empty: when nothing a node's view names is left on the device, the device's own list is drawn instead of hiding everything. (#347)
- Properties added while running 3.3.0–3.3.2 survive the 3.3.3 upgrade. The view recovered from the pre-3.3.0 backup could not know about them, so they were hidden on every canvas; they are kept visible, and the per-canvas arrangement the backup does know is still restored. (#347)
## [3.3.3] - 2026-08-17
### Features
- Each node now keeps its own view of a device's services and properties: order and visibility are per node, so a service the scanner fingerprinted no longer appears on every canvas drawing that device, and a property added on one schematic stays there. The facts still live on the inventory row — hiding is per node, deleting is device-wide. Upgrades keep what each canvas showed today. (#357)
### Fixes
- The one-off view seed no longer re-opens the pre-upgrade backup on every boot: a node pointing at a deleted device kept matching the seed query and logged a recovery that recovered nothing. (#357)
## [3.3.2] - 2026-08-17
### Fixes
+1 -1
View File
@@ -1 +1 @@
3.3.2
3.3.5
+126 -2
View File
@@ -32,7 +32,14 @@ from app.services.discovery_sources import add_source
from app.services.inventory_sync import find_device_for, merge_properties, merge_services
from app.services.mac_utils import normalize_mac
from app.services.node_dedupe import dedupe_nodes_by_device, find_duplicate_node
from app.services.scanner import DeepScanOptions, _valid_port_range, request_cancel, run_scan
from app.services.scanner import (
DeepScanOptions,
_valid_port_range,
_valid_port_spec,
request_cancel,
run_device_scan,
run_scan,
)
from app.services.zigbee_service import (
build_zigbee_properties,
merge_zigbee_properties,
@@ -176,7 +183,39 @@ async def _background_scan(
await db.rollback()
run = await db.get(ScanRun, run_id)
if run and run.status == "running":
run.status = "failed"
# "error", the same word run_scan / run_device_scan write when
# they fail themselves — one condition, one name. Scan History
# filters and colours that one; "failed" showed up unlabelled.
run.status = "error"
await db.commit()
async def _background_device_scan(
run_id: str,
device_id: str,
deep_scan: DeepScanOptions | None = None,
full_ports: bool = True,
ports: str | None = None,
) -> None:
async with AsyncSessionLocal() as db:
try:
await run_device_scan(
device_id,
db,
run_id,
deep_scan=deep_scan or DeepScanOptions(),
full_ports=full_ports,
ports=ports,
)
except Exception:
logger.exception("Device scan run %s failed unexpectedly", run_id)
await db.rollback()
run = await db.get(ScanRun, run_id)
if run and run.status == "running":
# "error", the same word run_scan / run_device_scan write when
# they fail themselves — one condition, one name. Scan History
# filters and colours that one; "failed" showed up unlabelled.
run.status = "error"
await db.commit()
@@ -215,6 +254,78 @@ async def trigger_scan(
return run
class RescanDeviceRequest(BaseModel):
"""Per-device deep rescan. Defaults to every TCP port — that is the point.
``ports`` narrows the sweep to what the user typed in the deep-scan dialog
(``80,443``, ``1-1024``); it wins over ``full_ports``.
"""
full_ports: bool = True
ports: str | None = None
http_probe_enabled: bool | None = None
verify_tls: bool | None = None
@field_validator("ports")
@classmethod
def _check_ports(cls, v: str | None) -> str | None:
if v is None or not v.strip():
return None
spec = v.strip()
if not _valid_port_spec(spec):
raise ValueError("Invalid port range")
return spec
@router.post("/pending/{device_id}/rescan", response_model=ScanRunResponse)
async def rescan_device(
device_id: str,
background_tasks: BackgroundTasks,
payload: RescanDeviceRequest | None = None,
db: AsyncSession = Depends(get_db),
_: str = Depends(get_current_user),
) -> ScanRun:
"""Deep-rescan one known device to refresh its services (issue #350).
Recorded as a ScanRun like any other scan, so progress, stop and history
work unchanged. One run per device at a time — a second request while the
first is still scanning is a 409, not a duplicate nmap over 65535 ports.
"""
device = await db.get(InventoryDevice, device_id)
if not device:
raise HTTPException(status_code=404, detail="Device not found")
if not device.ip:
raise HTTPException(
status_code=409, detail="Device has no IP address to scan"
)
if device.status == "hidden":
raise HTTPException(status_code=409, detail="Device is hidden")
target = f"{device.ip}/32"
running = (await db.execute(
select(ScanRun).where(ScanRun.status == "running", ScanRun.kind == "device")
)).scalars().all()
if any(target in (r.ranges or []) for r in running):
raise HTTPException(
status_code=409, detail="A scan is already running for this device"
)
p = payload or RescanDeviceRequest()
deep_scan = _resolve_deep_scan(
TriggerScanRequest(
http_probe_enabled=p.http_probe_enabled, verify_tls=p.verify_tls
)
)
run = ScanRun(status="running", kind="device", ranges=[target])
db.add(run)
await db.commit()
await db.refresh(run)
background_tasks.add_task(
_background_device_scan, run.id, device_id, deep_scan, p.full_ports, p.ports
)
return run
@router.post("/{run_id}/stop", response_model=dict)
async def stop_scan(
run_id: str,
@@ -933,6 +1044,19 @@ async def list_runs(db: AsyncSession = Depends(get_db), _: str = Depends(get_cur
return list(result.scalars().all())
@router.get("/runs/{run_id}", response_model=ScanRunResponse)
async def get_run(
run_id: str,
db: AsyncSession = Depends(get_db),
_: str = Depends(get_current_user),
) -> ScanRun:
"""One run, for a caller waiting on a scan it started (the device rescan)."""
run = await db.get(ScanRun, run_id)
if not run:
raise HTTPException(status_code=404, detail="Scan run not found")
return run
@router.get("/config", response_model=ScanConfig)
async def get_scan_config(_: str = Depends(get_current_user)) -> ScanConfig:
return ScanConfig(
+27
View File
@@ -2,12 +2,21 @@ import json
import logging
from pathlib import Path
from typing import Literal
from urllib.parse import urlsplit
from pydantic import Field, model_validator
from pydantic_settings import BaseSettings, SettingsConfigDict
logger = logging.getLogger(__name__)
def origin_of(url: str) -> str:
"""`scheme://host[:port]` of an absolute URL, or "" when it has none."""
parts = urlsplit(url)
if not parts.scheme or not parts.netloc:
return ""
return f"{parts.scheme}://{parts.netloc}"
def _read_version() -> str:
for candidate in [
Path(__file__).parent.parent.parent.parent / "VERSION", # repo root (dev)
@@ -88,6 +97,16 @@ class Settings(BaseSettings):
raise ValueError(f"{name} must use HTTPS when OIDC_COOKIE_SECURE=true")
if "*" in self.cors_origins:
raise ValueError("CORS_ORIGINS cannot contain '*' when AUTH_MODE=oidc")
app_origin = origin_of(self.oidc_redirect_uri)
allowed = {value.rstrip("/") for value in self.cors_origins}
if app_origin and app_origin not in allowed:
logger.warning(
"CORS_ORIGINS does not list %s (the origin of OIDC_REDIRECT_URI). "
"The CSRF check accepts that origin anyway, but add it to CORS_ORIGINS "
"so browser requests are not rejected: CORS_ORIGINS=[\"%s\"]",
app_origin,
app_origin,
)
return self
# Scanner
@@ -98,6 +117,14 @@ class Settings(BaseSettings):
# ports survive a timeout regardless; raise this on slow/overlay networks.
scanner_version_host_timeout: int = 60
# Per-device deep rescan: total time budget in seconds, checked between port
# slices. Unprivileged nmap falls back to a connect scan (-sT), and a host
# that drops packets makes every filtered port wait out its RTT — the whole
# range against such a host runs ~20-45 min. When the budget is spent the
# run keeps what it found and says how much it did not reach. This is NOT an
# nmap --host-timeout: that one discards a host's results wholesale.
scanner_deep_host_timeout: int = 2700
# Deep scan — persisted defaults (overridable per-scan from the scan dialog).
# http_ranges: extra nmap port ranges, opt-in, no default. Probe + TLS off by default.
scanner_http_ranges: list[str] = []
+8 -1
View File
@@ -14,7 +14,7 @@ from jwt import InvalidTokenError
from starlette.middleware.base import BaseHTTPMiddleware, RequestResponseEndpoint
from starlette.responses import JSONResponse
from app.core.config import settings
from app.core.config import origin_of, settings
ACCESS_TOKEN_USE = "access"
OIDC_SESSION_TOKEN_USE = "oidc_session"
@@ -153,6 +153,13 @@ def origin_is_allowed(origin: str | None) -> bool:
if not origin:
return False
allowed_origins = {value.rstrip("/") for value in settings.cors_origins}
# A same-origin deployment behind a reverse proxy needs no CORS, so
# CORS_ORIGINS is often left at its localhost default — yet browsers still
# send Origin on unsafe methods. The OIDC callback URL is served by this app,
# so its origin is the app's own and is always a legitimate CSRF origin.
app_origin = origin_of(settings.oidc_redirect_uri)
if app_origin:
allowed_origins.add(app_origin)
return origin.rstrip("/") in allowed_origins
+103 -2
View File
@@ -1,9 +1,10 @@
import json as _json
import logging
import shutil
import sqlite3
import uuid as _uuid_mod
from collections.abc import AsyncGenerator
from contextlib import suppress
from contextlib import closing, suppress
from pathlib import Path
from typing import Any
@@ -468,6 +469,10 @@ async def init_db() -> None:
"nodes.device_id.index",
"CREATE INDEX IF NOT EXISTS ix_nodes_device_id ON nodes(device_id)",
),
# Which of the row's services and properties this canvas shows, and
# in what order. Seeded from the row further down, once the backfill
# has had its say — see `_seed_node_views`.
("nodes.display_view", "ALTER TABLE nodes ADD COLUMN display_view JSON"),
):
await _try_migrate(conn, sql, label=label)
for label, sql in (
@@ -488,6 +493,7 @@ async def init_db() -> None:
await _backfill_node_devices()
await _drop_legacy_node_columns()
await _seed_node_views()
@@ -508,6 +514,7 @@ _NODE_COLUMNS_SQL = (
"label VARCHAR NOT NULL,"
"design_id VARCHAR REFERENCES designs(id) ON DELETE SET NULL,"
"device_id VARCHAR REFERENCES device_inventory(id) ON DELETE SET NULL,"
"display_view JSON,"
"pos_x FLOAT,"
"pos_y FLOAT,"
"parent_id VARCHAR REFERENCES nodes(id) ON DELETE CASCADE,"
@@ -526,7 +533,7 @@ _NODE_COLUMNS_SQL = (
)
_NODE_KEPT = (
"id, type, label, design_id, device_id, pos_x, pos_y, parent_id, container_mode, "
"id, type, label, design_id, device_id, display_view, pos_x, pos_y, parent_id, container_mode, "
"custom_colors, custom_icon, show_port_numbers, width, height, bottom_handles, "
"top_handles, left_handles, right_handles, created_at, updated_at"
)
@@ -651,6 +658,100 @@ async def _backfill_node_devices() -> None:
logger.warning("Inventory backfill failed: %s", exc)
def _pre_split_backup() -> Path | None:
"""The newest backup still holding the per-node device columns, if any.
`_backup_db` copies the database *before* the migrations of each new
version, so a user who upgraded to 3.3.0 has a `homelab.db.back-3.3.0`
carrying the last state in which `nodes` still owned its own services and
properties. That copy is the only record of which canvas showed what, since
3.3.0's backfill unioned them all onto one inventory row. Newest first: it is
the state closest to the upgrade, so it is what the user last saw.
"""
db_path = Path(settings.sqlite_path)
candidates = sorted(
db_path.parent.glob(f"{db_path.name}.back-*"),
key=lambda p: p.stat().st_mtime,
reverse=True,
)
for path in candidates:
try:
with closing(sqlite3.connect(f"file:{path}?mode=ro", uri=True)) as conn:
cols = {row[1] for row in conn.execute("PRAGMA table_info(nodes)")}
except sqlite3.Error:
continue
if {"services", "properties"} <= cols:
return path
return None
def _views_from_backup() -> dict[str, dict[str, Any]]:
"""What each node drew, read out of the pre-3.3.0 backup. Empty when there is none.
Keyed by node id, which is a uuid and stable across every version. A backup
that cannot be opened, or holds nodes this database no longer has, simply
contributes nothing — the caller falls back to the inventory row.
"""
path = _pre_split_backup()
if path is None:
return {}
try:
with closing(sqlite3.connect(f"file:{path}?mode=ro", uri=True)) as conn:
rows = conn.execute("SELECT id, services, properties FROM nodes").fetchall()
except sqlite3.Error as exc:
logger.warning("Could not read the pre-3.3.0 node lists from %s: %s", path.name, exc)
return {}
out: dict[str, dict[str, Any]] = {}
for node_id, services, properties in rows:
drawn: dict[str, Any] = {}
for kind, raw in (("services", services), ("properties", properties)):
if isinstance(raw, str | bytes):
with suppress(ValueError):
decoded = _json.loads(raw)
if isinstance(decoded, list):
drawn[kind] = decoded
if drawn:
out[node_id] = drawn
if out:
logger.info("Recovering the per-canvas service/property layout from %s", path.name)
return out
async def _seed_node_views() -> None:
"""Give pre-existing nodes an explicit view of their inventory row (3.3.3).
Order and visibility for services and properties moved to the node, so one
device drawn on two canvases can be rendered two ways. A node from before
that has no view, and where it comes from decides whether the user gets
their arrangement back:
* upgrading from 3.2.0, the backfill has already seeded it from the node's
own columns — nothing here to do;
* upgrading from 3.3.0-3.3.2, those columns are gone and the row holds the
union of every canvas, so the view is recovered from the backup taken
before the 3.3.0 migration. That is the difference between a canvas coming
back as the user left it and coming back showing every other canvas'
properties;
* with no usable backup, the row itself is the seed: what shows today keeps
showing, and only what the row gains *later* is held back.
Never fatal: without a view the node simply shows the whole row, which is
the behaviour it has now.
"""
# Imported here: app.services imports app.db.models, which imports this module.
from app.services.inventory_sync import seed_node_views
try:
async with AsyncSessionLocal() as session:
seeded = await seed_node_views(session, drawn=_views_from_backup)
if seeded:
await session.commit()
logger.info("Seeded the service/property view of %d node(s)", seeded)
except Exception as exc: # pragma: no cover - defensive, boot must not die
logger.warning("Seeding the node service/property views failed: %s", exc)
async def get_db() -> AsyncGenerator[AsyncSession, None]:
async with AsyncSessionLocal() as session:
yield session
+9
View File
@@ -49,6 +49,15 @@ class Node(Base):
device_id: Mapped[str | None] = mapped_column(
String, ForeignKey("device_inventory.id", ondelete="SET NULL"), index=True, nullable=True
)
# How this canvas renders the device's list facts: which services and which
# properties it shows, and in what order. The facts themselves stay on the
# inventory row, so the same device drawn on two canvases can show two
# different subsets — a scanner-guessed service on one, none on the other.
# {"services": [{"key": "443|tcp|https", "visible": true}, …],
# "properties": [{"key": "rack", "visible": false}, …]}
# NULL only for canvas furniture and for a node with no inventory row yet;
# `inventory_sync.link_facts` fills both lists as soon as there is one.
display_view: Mapped[dict[str, Any] | None] = mapped_column(JSON, nullable=True)
pos_x: Mapped[float] = mapped_column(Float, default=0)
pos_y: Mapped[float] = mapped_column(Float, default=0)
parent_id: Mapped[str | None] = mapped_column(String, ForeignKey("nodes.id", ondelete="CASCADE"))
+302 -8
View File
@@ -15,7 +15,7 @@ from __future__ import annotations
import json
import logging
from collections.abc import Mapping
from collections.abc import Callable, Mapping
from datetime import datetime
from typing import Any
@@ -203,8 +203,181 @@ def _service_key(svc: Any) -> Any:
return (svc.get("port"), svc.get("protocol"), (svc.get("service_name") or "").lower())
def merge_services(base: list[Any] | None, incoming: list[Any] | None) -> list[Any]:
"""Union two service lists on (port, protocol, name); incoming wins."""
# --- Per-node view of the device's list facts -----------------------------
#
# The row owns the services and the properties; a node owns which of them it
# shows and in what order. Both are keyed by a stable string so the view
# survives an edit to a service's path or a property's value.
VIEW_LISTS = ("services", "properties")
def _service_view_key(svc: Any) -> str:
port, protocol, name = _service_key(svc) if isinstance(svc, dict) else (None, None, repr(svc))
return f"{port}|{protocol}|{name}"
def _property_view_key(prop: Any) -> str:
if not isinstance(prop, dict):
return repr(prop)
return str(prop.get("key") or "").lower()
_VIEW_KEY = {"services": _service_view_key, "properties": _property_view_key}
def view_entries(items: list[Any] | None, kind: str) -> list[dict[str, Any]]:
"""One list of device facts as a node's view of it: order plus visibility.
A service carries no ``visible`` of its own — one that reached a canvas was
always drawn — so it defaults to shown. A property carries an explicit flag
and keeps it. Duplicate keys collapse: the view addresses the row, and the
row holds one entry per key.
"""
key_of = _VIEW_KEY[kind]
out: list[dict[str, Any]] = []
seen: set[str] = set()
for item in items or []:
key = key_of(item)
if key in seen:
continue
seen.add(key)
visible = bool(item.get("visible", True)) if isinstance(item, dict) else True
out.append({"key": key, "visible": visible})
return out
def view_from_facts(facts: Mapping[str, Any]) -> dict[str, list[dict[str, Any]]]:
"""The view a node payload implies — only for the lists it actually sent.
The wire shape has no separate view: a client sends its services and its
properties in display order, each with its ``visible`` flag, exactly as it
draws them. That *is* the view, so it is read back out here rather than
asking clients for a second field.
"""
return {kind: view_entries(facts[kind], kind) for kind in VIEW_LISTS if kind in facts}
def view_of_device(device: InventoryDevice) -> dict[str, list[dict[str, Any]]]:
"""A view showing everything the row currently holds — the seed for a new node."""
return {
"services": view_entries(device.services, "services"),
"properties": view_entries(device.properties, "properties"),
}
def next_view(
current: Mapping[str, Any] | None,
incoming: Mapping[str, Any],
device: InventoryDevice | None,
*,
strict: bool = False,
) -> dict[str, Any] | None:
"""This node's view after a write: what it sent, then the row for the rest.
A list the write did not carry keeps the view it had. A node linked to a row
always ends up with both lists, so "not in the view" can mean one thing
only: this canvas does not show it. That is what keeps a service a later scan
discovers off every canvas until someone turns it on.
A node getting its *first* view is the exception: an empty list there means
the writer had nothing to say about it, not that the user hid everything —
creating a node for an already-scanned device sends no services and must
still draw the ones the row holds. ``strict`` turns that off for the one
caller whose empty list is a real answer: the legacy backfill, where the
node's own columns are the whole of what that canvas used to show.
"""
if device is None:
return dict(current) if current else None
out: dict[str, Any] = dict(current or {})
first_view = current is None
seed = view_of_device(device)
for kind in VIEW_LISTS:
entries = incoming.get(kind)
if entries or (entries is not None and (strict or not first_view)):
out[kind] = entries
elif kind not in out:
out[kind] = seed[kind]
return out
def apply_view(items: list[Any] | None, entries: Any, kind: str) -> list[Any]:
"""The row's facts as one node draws them: its order, its visibility.
Without a view — furniture, or a node whose row was linked by an older
version — everything shows, in the row's own order. With one, an item the
view does not list is appended hidden rather than dropped, so a service a
scan added is one toggle away instead of invisible.
A *non-empty* view that matches nothing the row still holds is treated as
having no view at all. It means the row was replaced wholesale under the
node — every key gone, every key new — and hiding the lot would leave a node
drawing nothing while its view claims otherwise. An empty view is different:
it is a real answer ("this canvas draws none of them") and keeps hiding
everything.
"""
key_of = _VIEW_KEY[kind]
facts = list(items or [])
if not isinstance(entries, list):
return facts
by_key: dict[str, Any] = {}
for item in facts:
by_key.setdefault(key_of(item), item)
out: list[Any] = []
taken: set[str] = set()
for entry in entries:
if not isinstance(entry, dict):
continue
key = str(entry.get("key"))
item = by_key.get(key)
if item is None or key in taken:
continue # Deleted from the row since — the view catches up on write.
taken.add(key)
out.append(_stamped(item, bool(entry.get("visible", True))))
if entries and not taken:
return facts
for item in facts:
if key_of(item) not in taken:
out.append(_stamped(item, False))
return out
def _stamped(item: Any, visible: bool) -> Any:
"""``item`` carrying this node's verdict on whether it is drawn.
A shown item that never had a ``visible`` key does not gain one: services
have always travelled without it, and readers treat its absence as shown.
Only hiding is news, and properties keep the explicit flag they arrived with.
"""
if not isinstance(item, dict):
return item
if visible and "visible" not in item:
return dict(item)
return {**item, "visible": visible}
# How a service looks is the user's call. A fingerprint only ever guesses it
# from a port number and a banner, so a scan that re-finds a known service must
# not repaint the icon someone picked by hand — that happened on every "Scan
# network", across every canvas drawing the device, at once.
_CURATED_SERVICE_FIELDS = ("icon", "category")
def merge_services(
base: list[Any] | None,
incoming: list[Any] | None,
*,
discovered: bool = False,
) -> list[Any]:
"""Union two service lists on (port, protocol, name); incoming wins.
``discovered`` marks ``incoming`` as scanner output rather than a user edit:
it still adds services and refreshes facts, but leaves an established icon
and category alone. Either way a blank incoming value never clears one that
is already set — an absent field is silence, not a reset.
"""
out: list[Any] = [dict(s) if isinstance(s, dict) else s for s in (base or [])]
index = {_service_key(s): i for i, s in enumerate(out)}
for svc in incoming or []:
@@ -214,7 +387,12 @@ def merge_services(base: list[Any] | None, incoming: list[Any] | None) -> list[A
out.append(dict(svc) if isinstance(svc, dict) else svc)
index[key] = len(out) - 1
elif isinstance(svc, dict) and isinstance(out[pos], dict):
out[pos] = {**out[pos], **svc}
merged = {**out[pos], **svc}
for field_name in _CURATED_SERVICE_FIELDS:
established = out[pos].get(field_name)
if established and (discovered or not svc.get(field_name)):
merged[field_name] = established
out[pos] = merged
else:
out[pos] = svc
return out
@@ -378,6 +556,7 @@ async def link_facts(
replace_lists: bool = False,
only_changed: bool = False,
changed_fields: list[str] | None = None,
strict_view: bool = False,
) -> InventoryDevice | None:
"""Point one node at its inventory row, creating or merging as needed.
@@ -434,6 +613,13 @@ async def link_facts(
device.discovery_sources = add_source(device.discovery_sources, CANVAS_SOURCE)
node.device_id = device.id
# Order and visibility are this node's, not the device's, so they are taken
# from the full payload — never from the `changed_fields`/`only_changed`
# narrowing above, which exists to protect the *shared* row from a stale
# snapshot. A node's own view has no other writer to collide with.
node.display_view = next_view(
node.display_view, view_from_facts(facts), device, strict=strict_view
)
return device
@@ -444,10 +630,16 @@ def node_columns(payload: Mapping[str, Any]) -> dict[str, Any]:
inventory row now, so they are dropped here and applied through
:func:`link_facts` instead.
"""
allowed = {c.name for c in Node.__table__.columns}
allowed = {c.name for c in Node.__table__.columns} - _NODE_DERIVED_COLUMNS
return {k: v for k, v in payload.items() if k in allowed}
# Node columns the server derives rather than accepts: `display_view` is read
# back out of the services and properties a payload carries (see
# :func:`view_from_facts`), so a client sending one directly is ignored.
_NODE_DERIVED_COLUMNS = frozenset({"display_view"})
# Fields that are both a node column and a device fact: the node keeps a copy so
# a half-migrated database still renders, but the row is the truth.
_SHARED_FIELDS = ("label", "type")
@@ -502,6 +694,9 @@ def hydrated_node(node: Node, device: InventoryDevice | None) -> dict[str, Any]:
payload: dict[str, Any] = {
c.name: getattr(node, c.name) for c in node.__table__.columns
}
# An implementation detail of the split, not part of the wire shape: the
# view is reported *through* the services and properties it orders.
view = payload.pop("display_view", None) or {}
if device is None:
return payload
@@ -509,8 +704,8 @@ def hydrated_node(node: Node, device: InventoryDevice | None) -> dict[str, Any]:
payload[field] = getattr(device, field, None)
payload["label"] = device.label or node.label
payload["type"] = device.type or node.type
payload["services"] = device.services or []
payload["properties"] = device.properties or []
payload["services"] = apply_view(device.services, view.get("services"), "services")
payload["properties"] = apply_view(device.properties, view.get("properties"), "properties")
payload["show_hardware"] = bool(device.show_hardware)
payload["ieee_address"] = device.ieee_address
payload["status"] = device.status_live or "unknown"
@@ -591,6 +786,96 @@ async def _row_is_missing_facts(db: AsyncSession, device_id: str, facts: Mapping
)
def _with_later_properties(
view: dict[str, Any] | None, device: InventoryDevice
) -> dict[str, Any] | None:
"""``view``, plus the row's properties the pre-3.3.0 backup never saw.
They are what the user added while running 3.3.0-3.3.2 — the row was the
only place to add them then, and every canvas drew them. Recovering the old
view alone would take them off every canvas at once, which reads as data
loss. Appended in the row's order, after everything the backup placed.
"""
if view is None:
return None
entries = view.get("properties")
if not isinstance(entries, list):
return view
listed = {str(e.get("key")) for e in entries if isinstance(e, dict)}
later = [e for e in view_entries(device.properties, "properties") if e["key"] not in listed]
if not later:
return view
return {**view, "properties": [*entries, *later]}
async def seed_node_views(
db: AsyncSession, *, drawn: Callable[[], Mapping[str, Mapping[str, Any]]] | None = None
) -> int:
"""Give every linked node with no view one it can be held to.
Runs once, on the boot that adds `nodes.display_view`. Without it every
pre-existing node would fall through to "no view, show everything" and the
next scan would push a newly fingerprinted service onto all of them at once —
the leak this column exists to stop.
``drawn`` returns a map of node id to the services and properties that node
itself carried before 3.3.0 unioned them onto the row — the caller reads
them out of the pre-upgrade backup, and is only asked to when there is
actually a node to seed. Where it has an answer that answer wins, restoring
the arrangement the user made; where it does not, the row is the seed and
the canvas keeps showing exactly what it shows today. Does not commit.
The backup is 3.2.0-era, so it can only speak for what existed then. A
*property* the row has gained since — one the user added by hand while
running 3.3.0-3.3.2, when the row was all a canvas had — appears in no
backup entry, and seeding strictly from the backup would hide work the user
has been looking at for days. Those are appended visible, keeping the
recovered order and hidden flags for everything the backup does know.
Services are not treated that way: what a row gained since 3.3.0 is mostly a
scan's fingerprint, and holding it back is the whole point of the view.
"""
nodes = list(
(
await db.execute(
select(Node).where(
# A `device_id` naming a row that no longer exists cannot be
# seeded from anything, and leaving it in would match this
# query on every later boot — re-reading the backup file and
# logging a recovery that seeds nothing. Deleting a device
# leaves exactly that: SQLite runs with foreign keys off, so
# the `ON DELETE SET NULL` never fires on an existing table.
Node.device_id.in_(select(InventoryDevice.id)),
Node.display_view.is_(None),
)
)
)
.scalars()
.all()
)
if not nodes:
return 0
devices = await load_devices_for(db, nodes)
# Read the backup only now: on every later boot this function returns above
# and no file is opened at all.
was_drawn = drawn() if drawn is not None else {}
seeded = 0
for node in nodes:
device = devices.get(node.device_id or "")
if device is None: # pragma: no cover - the query already excluded these
continue
was = was_drawn.get(node.id)
# `strict`: an empty list in the backup is this canvas' real answer —
# it drew no service — and must not be read as "say nothing, show all".
if was:
view = next_view(None, view_from_facts(was), device, strict=True)
node.display_view = _with_later_properties(view, device)
else:
node.display_view = view_of_device(device)
seeded += 1
await db.flush()
return seeded
async def backfill_node_devices(db: AsyncSession) -> dict[str, int]:
"""Link every pre-3.3.0 canvas node to a Device Inventory row.
@@ -660,7 +945,16 @@ async def backfill_node_devices(db: AsyncSession) -> dict[str, int]:
# A node that is already linked only has its gaps filled: whatever
# is on the row was written after the migration and is newer.
device = await link_facts(
db, node, facts, overwrite_scalars=node.device_id is None
db,
node,
facts,
overwrite_scalars=node.device_id is None,
# The node's own columns are exactly what this canvas drew
# before the upgrade, empty lists included — so they define
# its view outright. Anything else the row carries (a service
# a scan fingerprinted, a property another canvas added)
# stays hidden here rather than appearing on every canvas.
strict_view=True,
)
if device is None:
continue
+352 -77
View File
@@ -7,6 +7,7 @@ import re
import socket
import subprocess
import threading
import time
from dataclasses import dataclass, field
from datetime import datetime, timezone
from typing import Any
@@ -19,6 +20,7 @@ from app.db.models import InventoryDevice, ScanRun
from app.services.discovery_sources import add_source
from app.services.fingerprint import fingerprint_ports, suggest_node_type
from app.services.http_probe import probe_open_ports
from app.services.inventory_sync import merge_services
from app.services.mac_utils import normalize_mac
logger = logging.getLogger(__name__)
@@ -61,8 +63,89 @@ def _valid_port_range(spec: str) -> bool:
return len(parts) == 1 or parts[0] <= parts[1]
def _build_port_spec(http_ranges: list[str] | None) -> str:
"""Combine the default port list with validated user ranges for nmap -p."""
# Every TCP port. Used by the per-device deep rescan, which trades minutes of
# nmap time for a service list that no curated port list can promise.
_FULL_PORTS = "1-65535"
# The deep rescan runs the full range in slices, one nmap call each, unioning
# what they find. A slice that runs long costs only its own ports — where a
# single 65535-port call that overruns costs everything (see _nmap_scan_single).
# It also gives the run somewhere to notice a stop request, and somewhere to
# give up when the budget is spent, without abandoning the ports already found.
_DEEP_CHUNK_SIZE = 8192
def _parse_port_spec(spec: str) -> list[tuple[int, int]]:
"""Parse an nmap ``-p`` spec into sorted, merged ``(start, end)`` ranges.
Accepts what the user can type in the deep-scan dialog: ``80``,
``8000-9000``, or a comma list of both. Returns ``[]`` for anything invalid
— the caller turns that into a 422 rather than handing nmap a bad ``-p``.
"""
ranges: list[tuple[int, int]] = []
for token in (t.strip() for t in spec.split(",")):
if not token or not _valid_port_range(token):
return []
parts = [int(p) for p in token.split("-")]
ranges.append((parts[0], parts[-1]))
if not ranges:
return []
ranges.sort()
merged = [ranges[0]]
for start, end in ranges[1:]:
last_start, last_end = merged[-1]
if start <= last_end + 1:
merged[-1] = (last_start, max(last_end, end))
else:
merged.append((start, end))
return merged
def _valid_port_spec(spec: str) -> bool:
"""True when ``spec`` is a usable comma list of ports/ranges."""
return bool(_parse_port_spec(spec))
def _port_chunks(spec: str, size: int = _DEEP_CHUNK_SIZE) -> list[str]:
"""Slice a port spec into nmap ``-p`` specs of at most ``size`` ports each.
Ranges are packed, not scanned one call per range: ``80,443`` is one chunk,
not two, while ``1-65535`` becomes eight. Each chunk is a scan boundary —
where a stop request lands and where the time budget is checked.
"""
chunks: list[str] = []
current: list[str] = []
budget = size
for start, end in _parse_port_spec(spec):
cursor = start
while cursor <= end:
take = min(budget, end - cursor + 1)
stop = cursor + take - 1
current.append(f"{cursor}-{stop}" if stop > cursor else str(cursor))
budget -= take
cursor = stop + 1
if budget == 0:
chunks.append(",".join(current))
current = []
budget = size
if current:
chunks.append(",".join(current))
return chunks
def _deep_port_chunks(size: int = _DEEP_CHUNK_SIZE) -> list[str]:
"""Slice the whole TCP range into nmap ``-p`` specs of ``size`` ports."""
return _port_chunks(_FULL_PORTS, size)
def _build_port_spec(http_ranges: list[str] | None, full: bool = False) -> str:
"""Combine the default port list with validated user ranges for nmap -p.
``full`` overrides everything with the whole TCP range — the deep rescan of
a single device, where completeness beats speed.
"""
if full:
return _FULL_PORTS
if not http_ranges:
return _EXTRA_PORTS
extra = [r.strip() for r in http_ranges if _valid_port_range(r.strip())]
@@ -237,7 +320,9 @@ async def _ping_sweep(target: str, run_id: str | None = None) -> dict[str, dict[
return alive
def _nmap_scan_single(host_dict: dict[str, Any], port_spec: str = _EXTRA_PORTS) -> dict[str, Any]:
def _nmap_scan_single(
host_dict: dict[str, Any], port_spec: str = _EXTRA_PORTS, bounded: bool = False
) -> dict[str, Any]:
"""
Phase 2 — single-IP port scan with service detection.
Runs in a thread (blocking). Returns the host dict enriched with open_ports.
@@ -261,8 +346,26 @@ def _nmap_scan_single(host_dict: dict[str, Any], port_spec: str = _EXTRA_PORTS)
# -sT without root but being explicit avoids edge cases.
scan_type = "-sS" if os.geteuid() == 0 else "-sT"
# --- Pass A: port discovery (no -sV, no host-timeout) ---
# --- Pass A: port discovery (no -sV) ---
# Default timing for the range scan: its ports are authoritative and a
# curated port list is fast either way.
#
# ``bounded`` is the deep rescan's timing, and carries NO --host-timeout on
# purpose: nmap answers a host timeout with "Skipping host <ip> due to host
# timeout" and discards *every* port it had already found, so a ceiling here
# turns a slow scan into one that reports nothing. The caller bounds total
# runtime by slicing the range instead.
#
# What costs the time is a host that drops packets: 8188 of 8192 ports
# filtered, each waiting out its probe. Measured against such a host, the
# retry pass is the whole cost — 8192 ports took 329s at --max-retries 1
# and 164s at 0, finding the same ports. Capping the RTT changed nothing
# (329s), so it is not set. Dropping retries risks missing a port that
# loses its one probe; that trade belongs to the deep scan alone, and the
# curated-port range scan keeps nmap's default retries.
discovery_args = f"{scan_type} --open -T4 -Pn -p {port_spec}"
if bounded:
discovery_args += " --max-retries 0 --min-rate 2000"
logger.debug("[Phase 2] %s discovery args: %s", ip, discovery_args)
nm_disc = nmap.PortScanner()
try:
@@ -501,6 +604,110 @@ async def _dedupe_pending_by_ip(db: AsyncSession) -> int:
return deleted
async def process_host(
db: AsyncSession,
host: dict[str, Any],
*,
hidden_ips: set[str],
deep_scan: DeepScanOptions,
discovery_source: str = "arp",
) -> str:
"""Fold one scanned host into ``device_inventory`` and commit.
Shared by the range scan and the single-device deep rescan, so both apply
the same matching, merge and de-duplication rules.
Returns ``"skipped"`` (hidden by the user), ``"created"`` (a new inventory
row) or ``"updated"`` (an existing row refreshed).
"""
ip = host["ip"]
# Skip only user-hidden devices. On-canvas devices are kept so they
# surface in the inventory with a canvas-presence badge.
if ip in hidden_ips:
logger.debug("Skipping %s — hidden by user", ip)
return "skipped"
open_ports = host["open_ports"]
# Deep-scan HTTP probe: enrich open ports with title/header signals so
# fingerprint can confirm services on custom ports. No-op when disabled
# or when the host has no open ports (e.g. mDNS-only discovery).
if deep_scan.http_probe_enabled and open_ports:
open_ports = await probe_open_ports(
ip, open_ports, verify_tls=deep_scan.verify_tls
)
norm_mac = normalize_mac(host.get("mac"))
services = fingerprint_ports(open_ports)
suggested_type = suggest_node_type(open_ports, norm_mac)
# One inventory row per device. Match by IP OR MAC across pending AND
# approved so a re-scan refreshes the existing row instead of spawning
# a duplicate — and so a device previously imported from Proxmox (which
# may have no IP but a known NIC MAC) reconciles with this scan instead
# of doubling up. Hidden rows are already skipped above.
match_cond = [InventoryDevice.ip == ip]
if norm_mac:
match_cond.append(InventoryDevice.mac == norm_mac)
existing_rows = (await db.execute(
select(InventoryDevice)
.where(or_(*match_cond), InventoryDevice.status != "hidden")
.order_by(InventoryDevice.discovered_at)
)).scalars().all()
if existing_rows:
# Prefer an approved row (it owns the canvas link semantics),
# otherwise the oldest. Collapse any leftover duplicates created
# by earlier scans.
keep = next((r for r in existing_rows if r.status == "approved"), existing_rows[0])
for dup in existing_rows:
if dup is not keep:
await db.delete(dup)
keep.ip = keep.ip or ip # fill an IP a Proxmox import lacked
keep.mac = norm_mac or keep.mac
keep.hostname = host.get("hostname") or keep.hostname
keep.os = host.get("os") or keep.os
# Union, never replace. Since 3.3.0 the row is the only copy of
# a device's services — every canvas drawing it reads this list
# — so overwriting it with the fingerprint would delete services
# the user added by hand, on every canvas at once. What a scan
# finds is added; what it no longer sees stays. A service the
# user does not want on a canvas is hidden by that node's view,
# which is where "I deleted this" belongs.
# discovered=: the fingerprint may add services and refresh what it
# knows, but never repaints an icon or category the user chose.
keep.services = merge_services(keep.services, services, discovered=True)
# Don't downgrade a Proxmox-typed guest (vm/lxc) to the generic
# scan guess; the importer knows the true type.
if not (keep.ieee_address or "").startswith("pve-"):
keep.suggested_type = suggested_type
# Merged row carries both sources (e.g. ["proxmox", "arp"]).
keep.discovery_sources = add_source(keep.discovery_sources, discovery_source)
# status preserved — an approved device stays approved.
# Stamp last_scan on the row so every canvas drawing the device
# shows when the scanner last observed it. The row is the one
# place that fact belongs; a node only draws it.
keep.last_scan = datetime.now(timezone.utc)
outcome = "updated"
else:
db.add(InventoryDevice(
ip=ip,
mac=norm_mac,
hostname=host.get("hostname"),
os=host.get("os"),
services=services,
suggested_type=suggested_type,
status="pending",
discovery_source=discovery_source,
discovery_sources=[discovery_source],
last_scan=datetime.now(timezone.utc),
))
outcome = "created"
await db.commit()
return outcome
async def run_scan(
ranges: list[str],
db: AsyncSession,
@@ -544,81 +751,17 @@ async def run_scan(
async def _process_host(host: dict[str, Any], discovery_source: str = "arp") -> None:
nonlocal devices_found
ip = host["ip"]
# Skip only user-hidden devices. On-canvas devices are kept so they
# surface in the inventory with a canvas-presence badge.
if ip in hidden_ips:
logger.debug("Skipping %s — hidden by user", ip)
outcome = await process_host(
db,
host,
hidden_ips=hidden_ips,
deep_scan=deep_scan,
discovery_source=discovery_source,
)
if outcome == "skipped":
return
open_ports = host["open_ports"]
# Deep-scan HTTP probe: enrich open ports with title/header signals so
# fingerprint can confirm services on custom ports. No-op when disabled
# or when the host has no open ports (e.g. mDNS-only discovery).
if deep_scan.http_probe_enabled and open_ports:
open_ports = await probe_open_ports(
ip, open_ports, verify_tls=deep_scan.verify_tls
)
norm_mac = normalize_mac(host.get("mac"))
services = fingerprint_ports(open_ports)
suggested_type = suggest_node_type(open_ports, norm_mac)
# One inventory row per device. Match by IP OR MAC across pending AND
# approved so a re-scan refreshes the existing row instead of spawning
# a duplicate — and so a device previously imported from Proxmox (which
# may have no IP but a known NIC MAC) reconciles with this scan instead
# of doubling up. Hidden rows are already skipped above.
match_cond = [InventoryDevice.ip == ip]
if norm_mac:
match_cond.append(InventoryDevice.mac == norm_mac)
existing_rows = (await db.execute(
select(InventoryDevice)
.where(or_(*match_cond), InventoryDevice.status != "hidden")
.order_by(InventoryDevice.discovered_at)
)).scalars().all()
if existing_rows:
# Prefer an approved row (it owns the canvas link semantics),
# otherwise the oldest. Collapse any leftover duplicates created
# by earlier scans.
keep = next((r for r in existing_rows if r.status == "approved"), existing_rows[0])
for dup in existing_rows:
if dup is not keep:
await db.delete(dup)
keep.ip = keep.ip or ip # fill an IP a Proxmox import lacked
keep.mac = norm_mac or keep.mac
keep.hostname = host.get("hostname") or keep.hostname
keep.os = host.get("os") or keep.os
keep.services = services
# Don't downgrade a Proxmox-typed guest (vm/lxc) to the generic
# scan guess; the importer knows the true type.
if not (keep.ieee_address or "").startswith("pve-"):
keep.suggested_type = suggested_type
# Merged row carries both sources (e.g. ["proxmox", "arp"]).
keep.discovery_sources = add_source(keep.discovery_sources, discovery_source)
# status preserved — an approved device stays approved.
# Stamp last_scan on the row so every canvas drawing the device
# shows when the scanner last observed it. The row is the one
# place that fact belongs; a node only draws it.
keep.last_scan = datetime.now(timezone.utc)
else:
db.add(InventoryDevice(
ip=ip,
mac=norm_mac,
hostname=host.get("hostname"),
os=host.get("os"),
services=services,
suggested_type=suggested_type,
status="pending",
discovery_source=discovery_source,
discovery_sources=[discovery_source],
last_scan=datetime.now(timezone.utc),
))
if outcome == "created":
devices_found += 1
await db.commit()
await broadcast_scan_update(run_id=run_id, devices_found=devices_found)
# nmap scan per CIDR — results stream in progressively
@@ -672,3 +815,135 @@ async def run_scan(
finally:
with _cancelled_lock:
_cancelled_runs.discard(run_id)
async def run_device_scan(
device_id: str,
db: AsyncSession,
run_id: str,
deep_scan: DeepScanOptions | None = None,
full_ports: bool = True,
ports: str | None = None,
) -> None:
"""Deep-rescan one known device and refresh its inventory row.
No ping sweep and no mDNS: the device is already known, so the IP goes
straight to the phase-2 port scan (``-Pn``). ``full_ports`` scans all 65535
TCP ports, which is the point of the feature — a device added before the
scanner knew a service, or listening on a port no curated list covers.
Minutes, not seconds; the run is cancellable like any other.
``ports`` narrows that to a user-chosen spec (``80,443``, ``1-1024``) and
wins over ``full_ports`` — the dialog prefills the full range, so a caller
that passes something else means it.
"""
from app.api.routes.status import broadcast_scan_update
deep_scan = deep_scan or DeepScanOptions()
port_spec = (
ports if ports else _build_port_spec(deep_scan.http_ranges, full=full_ports)
)
try:
device = await db.get(InventoryDevice, device_id)
if device is None or not device.ip:
raise ValueError("Device has no IP to scan")
host: dict[str, Any] = {
"ip": device.ip,
"mac": device.mac,
"hostname": device.hostname,
"os": device.os,
"open_ports": [],
}
# A rescan re-observes a device through the source it already has; it
# is not a network discovery. Hardcoding "arp" here would tell a
# Proxmox guest, a rack mount or a hand-added host that the network
# scanner found it, and it would then answer that source filter.
source = device.discovery_source or "arp"
# The full range goes out in slices so a stop request lands within one
# slice instead of at the end, and so a spent budget keeps the ports
# found so far. A curated port list is one call, as before.
if ports:
chunks = _port_chunks(ports)
elif full_ports:
chunks = _deep_port_chunks()
else:
chunks = [port_spec]
# Retry-free timing pays for itself over thousands of ports on a host
# that drops packets; over a handful it only costs accuracy.
total_ports = sum(end - start + 1 for start, end in _parse_port_spec(port_spec))
bounded = total_ports > _DEEP_CHUNK_SIZE
deadline = time.monotonic() + settings.scanner_deep_host_timeout
found: list[dict[str, Any]] = []
seen: set[tuple[str, int]] = set()
skipped = 0
for i, chunk in enumerate(chunks):
if _is_cancelled(run_id):
skipped = len(chunks) - i
break
if i and time.monotonic() > deadline:
skipped = len(chunks) - i
logger.warning(
"[Deep scan] %s — budget of %ds spent, %d port range(s) not scanned",
host["ip"], settings.scanner_deep_host_timeout, skipped,
)
break
# bounded: retry-free timing, for a host that drops packets.
scanned = await asyncio.to_thread(
_nmap_scan_single, dict(host), chunk, bounded
)
for port in scanned.get("open_ports") or []:
key = (port["protocol"], port["port"])
if key not in seen:
seen.add(key)
found.append(port)
host["mac"] = host["mac"] or scanned.get("mac")
host["os"] = scanned.get("os") or host["os"]
host["open_ports"] = found
# A partial sweep still says what it saw; the row unions it in.
partial = (
f"Scanned {len(chunks) - skipped}/{len(chunks)} port ranges "
f"({len(found)} open) — the rest was not reached"
if skipped
else None
)
devices_found = 0
if not _is_cancelled(run_id):
# The row is being rescanned on the user's request, so it is never
# "hidden" from itself — the route rejects hidden devices upfront.
outcome = await process_host(
db, host, hidden_ips=set(), deep_scan=deep_scan, discovery_source=source
)
if outcome == "created":
devices_found = 1
await broadcast_scan_update(run_id=run_id, devices_found=devices_found)
run = await db.get(ScanRun, run_id)
if run:
run.status = "cancelled" if _is_cancelled(run_id) else "done"
run.devices_found = devices_found
# Not a failure: a done run carrying an advisory, the way a Proxmox
# import reports what it could not see. Never let a partial sweep
# read as a complete one.
if partial and run.status == "done":
run.error = partial
run.finished_at = datetime.now(timezone.utc)
await db.commit()
except Exception as exc:
logger.error("Device scan failed: %s", exc)
await db.rollback()
run = await db.get(ScanRun, run_id)
if run:
run.status = "error"
run.error = str(exc)
run.finished_at = datetime.now(timezone.utc)
await db.commit()
finally:
with _cancelled_lock:
_cancelled_runs.discard(run_id)
@@ -9,6 +9,7 @@ those columns, so if they survive the upgrade every INSERT fails with
``NOT NULL constraint failed: nodes.status`` and approving a device is dead.
"""
import os
import shutil
os.environ.setdefault("SECRET_KEY", "test-only-secret-key-not-for-production")
@@ -188,3 +189,235 @@ async def test_a_node_the_backfill_cannot_link_still_leaves_nodes_insertable(db_
finally:
await check.dispose()
await engine.dispose()
async def test_upgrade_keeps_each_canvas_drawing_its_own_services(db_320):
"""3.3.3: order and visibility become the node's, and the upgrade freezes them.
Two canvases drew the same host with different service lists, and a scan had
already fingerprinted a third the user put on neither. All three converge on
one inventory row — so each node must come out of the upgrade still drawing
what it drew, with the scanner's guess hidden on both.
"""
db_path, engine = db_320
await _build_320(engine)
ssh = '[{"port": 22, "protocol": "tcp", "service_name": "ssh"}]'
both = '[{"port": 22, "protocol": "tcp", "service_name": "ssh"}, ' \
'{"port": 443, "protocol": "tcp", "service_name": "https"}]'
async with engine.begin() as conn:
await conn.exec_driver_sql(f"UPDATE nodes SET services = '{ssh}' WHERE id = 'n1'")
await conn.exec_driver_sql(f"UPDATE nodes SET services = '{both}' WHERE id = 'n2'")
await database.init_db()
from app.db.models import InventoryDevice, Node
session_factory = database.async_sessionmaker(engine, expire_on_commit=False)
async with session_factory() as session:
drawn = {}
for node_id in ("n1", "n2"):
node = await session.get(Node, node_id)
device = await session.get(InventoryDevice, node.device_id)
payload = inventory_sync.hydrated_node(node, device)
drawn[node_id] = [s["service_name"] for s in payload["services"] if s.get("visible", True)]
# Seeded, not left to "show the whole row".
assert node.display_view is not None
assert drawn == {"n1": ["ssh"], "n2": ["ssh", "https"]}
# What a scan finds next lands on the row, and on no canvas.
node = await session.get(Node, "n1")
device = await session.get(InventoryDevice, node.device_id)
device.services = [
*device.services,
{"port": 3001, "protocol": "tcp", "service_name": "Uptime Kuma"},
]
await session.commit()
payload = inventory_sync.hydrated_node(node, device)
assert [(s["service_name"], s.get("visible", True)) for s in payload["services"]] == [
("ssh", True), ("Uptime Kuma", False),
]
await engine.dispose()
async def _wind_back_to_3_3_2(engine, extra_service: dict) -> None:
"""A database that already took the 3.3.0 upgrade, leak included.
Both nodes point at one row holding the union of what each drew plus what a
scan found, and neither has a view — which is every 3.3.0-3.3.2 install.
"""
from app.db.models import InventoryDevice, Node
async with engine.begin() as conn:
await conn.exec_driver_sql("UPDATE nodes SET display_view = NULL")
session_factory = database.async_sessionmaker(engine, expire_on_commit=False)
async with session_factory() as session:
node = await session.get(Node, "n1")
one = await session.get(InventoryDevice, node.device_id)
other = await session.get(InventoryDevice, (await session.get(Node, "n2")).device_id)
for device in (one, other):
device.services = [
{"port": 22, "protocol": "tcp", "service_name": "ssh"},
{"port": 443, "protocol": "tcp", "service_name": "https"},
extra_service,
]
await session.commit()
async def test_a_database_already_on_3_3_recovers_its_layout_from_the_backup(db_320):
"""The second upgrade path: 3.3.0-3.3.2, where the legacy columns are gone.
3.3.0 unioned every canvas' services onto one row, so the row can no longer
say who drew what — but the backup taken before that migration still can.
Recovering from it is the difference between the user getting their canvases
back and getting each canvas showing every other canvas' services.
"""
_, engine = db_320
await _build_320(engine)
ssh = '[{"port": 22, "protocol": "tcp", "service_name": "ssh"}]'
both = '[{"port": 22, "protocol": "tcp", "service_name": "ssh"}, ' \
'{"port": 443, "protocol": "tcp", "service_name": "https"}]'
async with engine.begin() as conn:
await conn.exec_driver_sql(f"UPDATE nodes SET services = '{ssh}' WHERE id = 'n1'")
await conn.exec_driver_sql(f"UPDATE nodes SET services = '{both}' WHERE id = 'n2'")
await database.init_db() # 3.2.0 -> 3.3.x, and the backup that predates it.
kuma = {"port": 3001, "protocol": "tcp", "service_name": "Uptime Kuma"}
await _wind_back_to_3_3_2(engine, kuma)
await database.init_db()
from app.db.models import InventoryDevice, Node
session_factory = database.async_sessionmaker(engine, expire_on_commit=False)
async with session_factory() as session:
drawn = {}
for node_id in ("n1", "n2"):
node = await session.get(Node, node_id)
device = await session.get(InventoryDevice, node.device_id)
payload = inventory_sync.hydrated_node(node, device)
drawn[node_id] = [s["service_name"] for s in payload["services"] if s.get("visible", True)]
assert drawn == {"n1": ["ssh"], "n2": ["ssh", "https"]}
await engine.dispose()
async def test_a_property_added_while_on_3_3_survives_the_recovery(db_320):
"""The backup is 3.2.0-era and cannot know what the user added afterwards.
On 3.3.0-3.3.2 the row was the only place to add a property, and every
canvas drew it. Recovering the view strictly from the backup would hide it
on all of them at once — the "my properties disappeared" half of #347. The
scanner's service find is still held back: only properties are appended.
"""
_, engine = db_320
await _build_320(engine)
rack = '[{"key": "Rack", "value": "A", "icon": null, "visible": true}]'
ssh = '[{"port": 22, "protocol": "tcp", "service_name": "ssh"}]'
async with engine.begin() as conn:
await conn.exec_driver_sql(
f"UPDATE nodes SET properties = '{rack}', services = '{ssh}' WHERE id = 'n1'"
)
await database.init_db() # 3.2.0 -> 3.3.x, and the backup that predates it.
kuma = {"port": 3001, "protocol": "tcp", "service_name": "Uptime Kuma"}
await _wind_back_to_3_3_2(engine, kuma)
from app.db.models import InventoryDevice, Node
session_factory = database.async_sessionmaker(engine, expire_on_commit=False)
async with session_factory() as session:
node = await session.get(Node, "n1")
device = await session.get(InventoryDevice, node.device_id)
# What the user re-added by hand while running 3.3.x.
device.properties = [
*device.properties,
{"key": "Ports", "value": "8", "icon": None, "visible": True},
]
await session.commit()
await database.init_db()
async with session_factory() as session:
node = await session.get(Node, "n1")
device = await session.get(InventoryDevice, node.device_id)
payload = inventory_sync.hydrated_node(node, device)
assert [(p["key"], p.get("visible", True)) for p in payload["properties"]] == [
("Rack", True), ("Ports", True),
]
# And the scan's find is still off this canvas.
assert [s["service_name"] for s in payload["services"] if s.get("visible", True)] == ["ssh"]
await engine.dispose()
async def test_without_a_usable_backup_the_row_is_the_seed(db_320):
"""No backup to recover from: keep showing what the canvas shows today.
Nothing is taken away — the user simply keeps the merged list they have been
looking at since 3.3.0, and only what the row gains *after* this boot is
held back.
"""
db_path, engine = db_320
await _build_320(engine)
await database.init_db()
kuma = {"port": 3001, "protocol": "tcp", "service_name": "Uptime Kuma"}
await _wind_back_to_3_3_2(engine, kuma)
for backup in db_path.parent.glob(f"{db_path.name}.back-*"):
backup.unlink()
await database.init_db()
from app.db.models import InventoryDevice, Node
session_factory = database.async_sessionmaker(engine, expire_on_commit=False)
async with session_factory() as session:
node = await session.get(Node, "n1")
device = await session.get(InventoryDevice, node.device_id)
payload = inventory_sync.hydrated_node(node, device)
assert [s["service_name"] for s in payload["services"]] == ["ssh", "https", "Uptime Kuma"]
assert all(s.get("visible", True) for s in payload["services"])
# From here on the node is pinned: the next scan's find is held back.
device.services = [*device.services, {"port": 5001, "protocol": "tcp", "service_name": "Synology DSM HTTPS"}]
await session.commit()
payload = inventory_sync.hydrated_node(node, device)
assert [s["service_name"] for s in payload["services"] if not s.get("visible", True)] == [
"Synology DSM HTTPS"
]
await engine.dispose()
async def test_the_newest_backup_that_still_has_the_columns_is_the_one_read(db_320):
"""A 3.3.2 install has several backups; only some can answer.
`homelab.db.back-3.3.1` and `-3.3.2` were taken *after* the split and hold
nothing per node; `-3.3.0` was taken before it. Reading the wrong one would
either say nothing or resurrect a much older canvas.
"""
db_path, engine = db_320
await _build_320(engine)
async with engine.begin() as conn:
await conn.exec_driver_sql(
"UPDATE nodes SET services = "
"'[{\"port\": 22, \"protocol\": \"tcp\", \"service_name\": \"ssh\"}]' WHERE id = 'n1'"
)
# Ancient: same shape, but a canvas the user has long since moved on from.
old = db_path.parent / f"{db_path.name}.back-3.1.0"
shutil.copy2(db_path, old)
os.utime(old, (1, 1))
async with engine.begin() as conn:
await conn.exec_driver_sql(
"UPDATE nodes SET services = "
"'[{\"port\": 443, \"protocol\": \"tcp\", \"service_name\": \"https\"}]' WHERE id = 'n1'"
)
pre_split = db_path.parent / f"{db_path.name}.back-3.3.0"
shutil.copy2(db_path, pre_split)
await database.init_db() # drops the columns, and backs up under this version
# Post-split backups: newer, and unable to say who drew what.
for name in ("back-3.3.1", "back-3.3.2"):
shutil.copy2(db_path, db_path.parent / f"{db_path.name}.{name}")
assert database._pre_split_backup() == pre_split
assert database._views_from_backup()["n1"]["services"] == [
{"port": 443, "protocol": "tcp", "service_name": "https"}
]
await engine.dispose()
+151
View File
@@ -414,3 +414,154 @@ async def test_delete_pending_refuses_a_mounted_device(client: AsyncClient, head
assert res.status_code == 409
listed = (await client.get("/api/v1/scan/pending", headers=headers)).json()
assert [d["id"] for d in listed] == [pending_device.id]
# ---------------------------------------------------------------------------
# Per-device deep rescan (issue #350)
# ---------------------------------------------------------------------------
@pytest.mark.asyncio
async def test_rescan_device_requires_auth(client: AsyncClient, pending_device):
res = await client.post(f"/api/v1/scan/pending/{pending_device.id}/rescan")
assert res.status_code == 401
@pytest.mark.asyncio
async def test_rescan_device_creates_run_for_that_ip(client: AsyncClient, headers, pending_device):
with patch("app.api.routes.scan._background_device_scan", new_callable=AsyncMock) as bg:
res = await client.post(
f"/api/v1/scan/pending/{pending_device.id}/rescan", headers=headers
)
assert res.status_code == 200, res.text
data = res.json()
assert data["status"] == "running"
assert data["kind"] == "device"
# One host, not the configured ranges — the scan targets this device only.
assert data["ranges"] == ["192.168.1.100/32"]
bg.assert_called_once()
assert bg.call_args.args[1] == pending_device.id
# Full range unless the caller says otherwise.
assert bg.call_args.args[3] is True
@pytest.mark.asyncio
async def test_rescan_device_passes_the_requested_port_range(
client: AsyncClient, headers, pending_device
):
"""The dialog's range reaches the scanner verbatim."""
with patch("app.api.routes.scan._background_device_scan", new_callable=AsyncMock) as bg:
res = await client.post(
f"/api/v1/scan/pending/{pending_device.id}/rescan",
headers=headers,
json={"ports": " 80,443,8000-9000 "},
)
assert res.status_code == 200, res.text
assert bg.call_args.args[4] == "80,443,8000-9000"
@pytest.mark.asyncio
async def test_rescan_device_rejects_an_unusable_port_range(
client: AsyncClient, headers, pending_device
):
"""A bad spec is refused here, not handed to nmap."""
for bad in ["0", "65536", "100-50", "80,", "http"]:
res = await client.post(
f"/api/v1/scan/pending/{pending_device.id}/rescan",
headers=headers,
json={"ports": bad},
)
assert res.status_code == 422, f"{bad}: {res.text}"
@pytest.mark.asyncio
async def test_rescan_device_treats_a_blank_range_as_the_full_sweep(
client: AsyncClient, headers, pending_device
):
with patch("app.api.routes.scan._background_device_scan", new_callable=AsyncMock) as bg:
res = await client.post(
f"/api/v1/scan/pending/{pending_device.id}/rescan",
headers=headers,
json={"ports": " "},
)
assert res.status_code == 200, res.text
assert bg.call_args.args[4] is None
assert bg.call_args.args[3] is True
@pytest.mark.asyncio
async def test_rescan_device_unknown_id_404(client: AsyncClient, headers):
res = await client.post(f"/api/v1/scan/pending/{uuid.uuid4()}/rescan", headers=headers)
assert res.status_code == 404
@pytest.mark.asyncio
async def test_rescan_device_without_ip_409(client: AsyncClient, headers, db_session):
device = InventoryDevice(id=str(uuid.uuid4()), ip=None, hostname="zigbee-lamp", status="pending")
db_session.add(device)
await db_session.commit()
res = await client.post(f"/api/v1/scan/pending/{device.id}/rescan", headers=headers)
assert res.status_code == 409
assert "IP" in res.json()["detail"]
@pytest.mark.asyncio
async def test_rescan_device_hidden_409(client: AsyncClient, headers, db_session):
device = InventoryDevice(id=str(uuid.uuid4()), ip="192.168.1.77", status="hidden")
db_session.add(device)
await db_session.commit()
res = await client.post(f"/api/v1/scan/pending/{device.id}/rescan", headers=headers)
assert res.status_code == 409
@pytest.mark.asyncio
async def test_rescan_device_serialized_per_device(client: AsyncClient, headers, pending_device):
"""A second rescan while the first still runs is refused, not duplicated."""
with patch("app.api.routes.scan._background_device_scan", new_callable=AsyncMock):
first = await client.post(
f"/api/v1/scan/pending/{pending_device.id}/rescan", headers=headers
)
second = await client.post(
f"/api/v1/scan/pending/{pending_device.id}/rescan", headers=headers
)
assert first.status_code == 200
assert second.status_code == 409
assert "already running" in second.json()["detail"]
@pytest.mark.asyncio
async def test_rescan_device_allows_a_new_run_once_the_first_finished(
client: AsyncClient, headers, pending_device
):
with patch("app.api.routes.scan._background_device_scan", new_callable=AsyncMock):
first = await client.post(
f"/api/v1/scan/pending/{pending_device.id}/rescan", headers=headers
)
assert first.status_code == 200
run_id = first.json()["id"]
stopped = await client.post(f"/api/v1/scan/{run_id}/stop", headers=headers)
assert stopped.status_code == 200
again = await client.post(
f"/api/v1/scan/pending/{pending_device.id}/rescan", headers=headers
)
assert again.status_code == 200
@pytest.mark.asyncio
async def test_get_run_returns_status(client: AsyncClient, headers, pending_device):
with patch("app.api.routes.scan._background_device_scan", new_callable=AsyncMock):
started = await client.post(
f"/api/v1/scan/pending/{pending_device.id}/rescan", headers=headers
)
run_id = started.json()["id"]
res = await client.get(f"/api/v1/scan/runs/{run_id}", headers=headers)
assert res.status_code == 200
assert res.json()["id"] == run_id
assert res.json()["status"] == "running"
@pytest.mark.asyncio
async def test_get_run_unknown_id_404(client: AsyncClient, headers):
res = await client.get(f"/api/v1/scan/runs/{uuid.uuid4()}", headers=headers)
assert res.status_code == 404
+37 -5
View File
@@ -12,9 +12,14 @@ from app.services.scanner import _cancelled_runs, request_cancel, run_scan
@pytest.mark.asyncio
async def test_background_scan_marks_run_failed_on_exception(mem_db):
"""If run_scan() raises, the ScanRun must transition running → failed and the
session rollback path must execute without a follow-on exception."""
async def test_background_scan_marks_run_errored_on_exception(mem_db):
"""If run_scan() raises, the ScanRun must transition running → error and the
session rollback path must execute without a follow-on exception.
"error" and not "failed": one condition, one name — it is what run_scan
writes when it catches the failure itself, and the only one Scan History
filters and colours.
"""
from app.api.routes.scan import _background_scan
async with mem_db() as session:
@@ -36,13 +41,13 @@ async def test_background_scan_marks_run_failed_on_exception(mem_db):
async with mem_db() as session:
refreshed = await session.get(ScanRun, run_id)
assert refreshed is not None
assert refreshed.status == "failed"
assert refreshed.status == "error"
@pytest.mark.asyncio
async def test_background_scan_leaves_non_running_status_alone(mem_db):
"""If the run was already stopped/cancelled before run_scan failed, _background_scan
must NOT overwrite that terminal status with 'failed'."""
must NOT overwrite that terminal status with 'error'."""
from app.api.routes.scan import _background_scan
async with mem_db() as session:
@@ -436,3 +441,30 @@ async def test_run_scan_updates_existing_pending_device(db_session: AsyncSession
# Services and hostname should be updated
assert device.hostname == "myhost.lan"
assert any(s["port"] == 8096 for s in device.services)
@pytest.mark.asyncio
async def test_background_device_scan_marks_run_errored_on_exception(mem_db):
"""Same word as the network scan — a device run that blows up reads alike."""
from app.api.routes.scan import _background_device_scan
async with mem_db() as session:
run = ScanRun(status="running", kind="device", ranges=["10.0.0.5/32"])
session.add(run)
await session.commit()
run_id = run.id
with (
patch("app.api.routes.scan.AsyncSessionLocal", mem_db),
patch(
"app.api.routes.scan.run_device_scan",
new_callable=AsyncMock,
side_effect=RuntimeError("boom"),
),
):
await _background_device_scan(run_id, "d1")
async with mem_db() as session:
refreshed = await session.get(ScanRun, run_id)
assert refreshed is not None
assert refreshed.status == "error"
+74
View File
@@ -336,6 +336,80 @@ async def test_oidc_cookie_mutations_require_csrf_and_allowed_origin(client: Asy
assert (await client.get("/api/v1/auth/me")).status_code == 401
async def test_oidc_mutation_allows_app_origin_missing_from_cors_origins(
client: AsyncClient, oidc_settings
):
"""Regression for #356: a same-origin reverse-proxy deploy needs no CORS, so
CORS_ORIGINS is left at its localhost default while the browser still sends
Origin — every POST used to 403."""
oidc_settings.cors_origins = ["http://localhost:5173", "http://localhost:3000"]
fake_client = FakeOIDCClient(token={
"userinfo": {
"iss": "https://idp.example/application/o/homelable/",
"sub": "user-123",
"preferred_username": "alice",
},
})
with patch("app.api.routes.auth.get_oidc_client", return_value=fake_client):
await client.get("/api/v1/auth/oidc/callback")
csrf_token = (await client.get("/api/v1/auth/me")).json()["csrf_token"]
created = await client.post(
"/api/v1/designs",
json={"name": "From OIDC", "design_type": "network"},
headers={"Origin": "http://test", "X-Homelable-CSRF": csrf_token},
)
assert created.status_code == 201
# The fallback widens the allowlist by exactly one origin, nothing more.
evil = await client.post(
"/api/v1/designs",
json={"name": "Nope", "design_type": "network"},
headers={"Origin": "https://evil.example", "X-Homelable-CSRF": csrf_token},
)
assert evil.status_code == 403
def test_origin_is_allowed_falls_back_to_the_oidc_redirect_origin(oidc_settings):
from app.core.security import origin_is_allowed
oidc_settings.cors_origins = ["http://localhost:3000"]
oidc_settings.oidc_redirect_uri = "https://homelable.example.com/api/v1/auth/oidc/callback"
assert origin_is_allowed("https://homelable.example.com") is True
assert origin_is_allowed("https://homelable.example.com/") is True
assert origin_is_allowed("http://localhost:3000") is True
assert origin_is_allowed("https://evil.example") is False
assert origin_is_allowed(None) is False
# A redirect URI that is not an absolute URL contributes no origin.
oidc_settings.oidc_redirect_uri = "/api/v1/auth/oidc/callback"
assert origin_is_allowed("https://homelable.example.com") is False
def test_oidc_settings_warn_when_cors_origins_omits_the_app_origin(caplog):
from app.core.config import Settings
base = {
"secret_key": "x" * 32,
"auth_mode": "oidc",
"oidc_discovery_url": "https://idp.example/.well-known/openid-configuration",
"oidc_client_id": "homelable",
"oidc_client_secret": "secret",
"oidc_redirect_uri": "https://homelable.example.com/api/v1/auth/oidc/callback",
}
with caplog.at_level("WARNING"):
Settings(**base, cors_origins=["http://localhost:3000"])
assert "https://homelable.example.com" in caplog.text
assert "CORS_ORIGINS" in caplog.text
caplog.clear()
with caplog.at_level("WARNING"):
Settings(**base, cors_origins=["https://homelable.example.com/"])
assert caplog.text == ""
async def test_local_bearer_logout_does_not_require_csrf(client: AsyncClient, headers):
res = await client.post("/api/v1/auth/logout", headers=headers)
assert res.status_code == 204
+471
View File
@@ -17,9 +17,11 @@ from app.services.inventory_sync import (
backfill_node_devices,
changed_facts,
find_device_for,
hydrated_node,
link_facts,
merge_properties,
merge_services,
seed_node_views,
)
@@ -83,6 +85,49 @@ class TestMergeRules:
assert out[1]["port"] == 80
def test_a_scan_never_repaints_an_icon_the_user_picked(self):
"""Every "Scan network" used to overwrite a hand-picked brand icon."""
base = [
{"port": 80, "protocol": "tcp", "service_name": "http",
"icon": "brand:pihole", "category": "network"},
]
# What fingerprint_ports returns for the same port: its own guess.
incoming = [
{"port": 80, "protocol": "tcp", "service_name": "http",
"icon": "Globe", "category": "web"},
]
out = merge_services(base, incoming, discovered=True)
assert len(out) == 1
assert out[0]["icon"] == "brand:pihole"
assert out[0]["category"] == "network"
def test_a_scan_still_fills_an_icon_the_service_never_had(self):
out = merge_services(
[{"port": 80, "protocol": "tcp", "service_name": "http"}],
[{"port": 80, "protocol": "tcp", "service_name": "http",
"icon": "Globe", "category": "web"}],
discovered=True,
)
assert out[0]["icon"] == "Globe"
assert out[0]["category"] == "web"
def test_a_user_edit_still_changes_the_icon(self):
"""The guard is for scanner output only — an edit is an edit."""
out = merge_services(
[{"port": 80, "protocol": "tcp", "service_name": "http", "icon": "Globe"}],
[{"port": 80, "protocol": "tcp", "service_name": "http", "icon": "brand:pihole"}],
)
assert out[0]["icon"] == "brand:pihole"
def test_services_keep_an_established_icon_when_incoming_has_none(self):
"""A blank field is silence, not a reset — mirrors merge_properties."""
out = merge_services(
[{"port": 80, "protocol": "tcp", "service_name": "http", "icon": "brand:pihole"}],
[{"port": 80, "protocol": "tcp", "service_name": "http", "icon": None}],
)
assert out[0]["icon"] == "brand:pihole"
class TestChangedFacts:
"""What a save is allowed to write: its edit, not its whole snapshot."""
@@ -528,6 +573,61 @@ class TestBackfill:
rows = (await db_session.execute(select(InventoryDevice))).scalars().all()
assert len(rows) == 1
@pytest.mark.asyncio
async def test_each_canvas_keeps_showing_what_it_showed(self, db_session):
"""The upgrade must not redraw a canvas.
Two canvases drew the same host with different service lists, and the
scanner's row for it holds a third the user never put on either. They
converge on one row — so each node keeps a view of the subset it drew,
and the scanner's guess appears on neither.
"""
await self._legacy_nodes_table(db_session)
ssh = {"port": 22, "protocol": "tcp", "service_name": "ssh"}
https = {"port": 443, "protocol": "tcp", "service_name": "https"}
kuma = {"port": 3001, "protocol": "tcp", "service_name": "Uptime Kuma"}
db_session.add(InventoryDevice(id="d-1", ip="10.0.0.5", services=[kuma]))
await db_session.commit()
one, two = await _design(db_session, "Net"), await _design(db_session, "Rack")
a = await self._legacy_node(db_session, one, ip="10.0.0.5", services=[ssh])
b = await self._legacy_node(db_session, two, ip="10.0.0.5", services=[ssh, https])
await backfill_node_devices(db_session)
await db_session.commit()
device = await db_session.get(InventoryDevice, "d-1")
assert {s["service_name"] for s in device.services} == {"Uptime Kuma", "ssh", "https"}
drawn = {}
for node_id in (a, b):
node = await db_session.get(Node, node_id)
payload = hydrated_node(node, device)
drawn[node_id] = [s["service_name"] for s in payload["services"] if s.get("visible", True)]
assert drawn[a] == ["ssh"]
assert drawn[b] == ["ssh", "https"]
@pytest.mark.asyncio
async def test_a_node_that_drew_no_service_keeps_drawing_none(self, db_session):
"""An empty legacy list is an answer: that canvas showed no services."""
await self._legacy_nodes_table(db_session)
db_session.add(
InventoryDevice(
id="d-1", ip="10.0.0.5",
services=[{"port": 3001, "protocol": "tcp", "service_name": "Uptime Kuma"}],
)
)
await db_session.commit()
design = await _design(db_session)
node_id = await self._legacy_node(db_session, design, ip="10.0.0.5", services=[])
await backfill_node_devices(db_session)
await db_session.commit()
node = await db_session.get(Node, node_id)
device = await db_session.get(InventoryDevice, "d-1")
payload = hydrated_node(node, device)
assert [s.get("visible", True) for s in payload["services"]] == [False]
# --- routes -----------------------------------------------------------------
@@ -980,3 +1080,374 @@ class TestRoutesKeepTheLinkInStep:
assert res.status_code == 200
node = await db_session.get(Node, res.json()["node_id"])
assert node is not None and node.device_id == "d-1"
class TestPerNodeView:
"""Order and visibility belong to the node, the facts to the row.
The same device drawn on two canvases is one inventory row, so without a
per-node view every canvas showing it inherits every service a scan ever
fingerprinted and every property any other canvas added.
"""
async def _node_on(self, client, headers, design_id: str, **payload) -> dict:
body = {"type": "nas", "label": "NAS", "ip": "10.0.0.5", "design_id": design_id, "force": True}
body.update(payload)
res = await client.post("/api/v1/nodes", json=body, headers=body.pop("headers", None) or headers)
assert res.status_code == 201
return res.json()
@pytest.mark.asyncio
async def test_a_new_node_shows_what_the_row_already_holds(
self, client: AsyncClient, headers, db_session
):
"""An empty payload list means "I have nothing to say", not "hide it all"."""
db_session.add(
InventoryDevice(
id="d-1",
ip="10.0.0.5",
services=[{"port": 22, "protocol": "tcp", "service_name": "ssh"}],
properties=[{"key": "Rack", "value": "A1", "icon": None, "visible": True}],
)
)
await db_session.commit()
design = await _design(db_session)
node = await self._node_on(client, headers, design)
assert [s["service_name"] for s in node["services"]] == ["ssh"]
assert all(s.get("visible", True) for s in node["services"])
assert node["properties"][0]["visible"] is True
@pytest.mark.asyncio
async def test_hiding_a_service_on_one_node_leaves_the_other_alone(
self, client: AsyncClient, headers, db_session
):
db_session.add(
InventoryDevice(
id="d-1",
ip="10.0.0.5",
services=[
{"port": 22, "protocol": "tcp", "service_name": "ssh"},
{"port": 3001, "protocol": "tcp", "service_name": "Uptime Kuma"},
],
)
)
await db_session.commit()
one, two = await _design(db_session, "Net"), await _design(db_session, "Rack")
node_a = await self._node_on(client, headers, one)
node_b = await self._node_on(client, headers, two)
hidden = [
{"port": 22, "protocol": "tcp", "service_name": "ssh"},
{"port": 3001, "protocol": "tcp", "service_name": "Uptime Kuma", "visible": False},
]
res = await client.patch(
f"/api/v1/nodes/{node_a['id']}", json={"services": hidden}, headers=headers
)
assert res.status_code == 200
assert [(s["service_name"], s.get("visible", True)) for s in res.json()["services"]] == [
("ssh", True), ("Uptime Kuma", False),
]
other = (await client.get(f"/api/v1/nodes/{node_b['id']}", headers=headers)).json()
assert [s.get("visible", True) for s in other["services"]] == [True, True]
# The service itself is untouched — hiding is not deleting.
device = await db_session.get(InventoryDevice, "d-1")
await db_session.refresh(device)
assert len(device.services) == 2
@pytest.mark.asyncio
async def test_the_order_is_per_node_too(self, client: AsyncClient, headers, db_session):
db_session.add(
InventoryDevice(
id="d-1",
ip="10.0.0.5",
properties=[
{"key": "Rack", "value": "A1", "icon": None, "visible": True},
{"key": "Owner", "value": "me", "icon": None, "visible": True},
],
)
)
await db_session.commit()
one, two = await _design(db_session, "Net"), await _design(db_session, "Rack")
node_a = await self._node_on(client, headers, one)
node_b = await self._node_on(client, headers, two)
flipped = [
{"key": "Owner", "value": "me", "icon": None, "visible": True},
{"key": "Rack", "value": "A1", "icon": None, "visible": True},
]
res = await client.patch(
f"/api/v1/nodes/{node_a['id']}", json={"properties": flipped}, headers=headers
)
assert [p["key"] for p in res.json()["properties"]] == ["Owner", "Rack"]
other = (await client.get(f"/api/v1/nodes/{node_b['id']}", headers=headers)).json()
assert [p["key"] for p in other["properties"]] == ["Rack", "Owner"]
@pytest.mark.asyncio
async def test_a_service_the_row_gains_later_stays_off_the_canvas(
self, client: AsyncClient, headers, db_session
):
"""The leak this column exists to stop: a scan must not redraw every canvas."""
db_session.add(
InventoryDevice(
id="d-1", ip="10.0.0.5",
services=[{"port": 22, "protocol": "tcp", "service_name": "ssh"}],
)
)
await db_session.commit()
design = await _design(db_session)
node = await self._node_on(client, headers, design)
device = await db_session.get(InventoryDevice, "d-1")
device.services = [
*device.services,
{"port": 3001, "protocol": "tcp", "service_name": "Uptime Kuma"},
]
await db_session.commit()
after = (await client.get(f"/api/v1/nodes/{node['id']}", headers=headers)).json()
assert [(s["service_name"], s.get("visible", True)) for s in after["services"]] == [
("ssh", True), ("Uptime Kuma", False),
]
@pytest.mark.asyncio
async def test_removing_a_service_removes_it_from_the_device(
self, client: AsyncClient, headers, db_session
):
"""Delete is device-wide — hiding is what a single canvas does."""
db_session.add(
InventoryDevice(
id="d-1", ip="10.0.0.5",
services=[
{"port": 22, "protocol": "tcp", "service_name": "ssh"},
{"port": 3001, "protocol": "tcp", "service_name": "Uptime Kuma"},
],
)
)
await db_session.commit()
one, two = await _design(db_session, "Net"), await _design(db_session, "Rack")
node_a = await self._node_on(client, headers, one)
node_b = await self._node_on(client, headers, two)
await client.patch(
f"/api/v1/nodes/{node_a['id']}",
json={"services": [{"port": 22, "protocol": "tcp", "service_name": "ssh"}]},
headers=headers,
)
other = (await client.get(f"/api/v1/nodes/{node_b['id']}", headers=headers)).json()
assert [s["service_name"] for s in other["services"]] == ["ssh"]
@pytest.mark.asyncio
async def test_a_canvas_save_records_the_view_even_when_no_fact_changed(
self, client: AsyncClient, headers, db_session
):
"""`changed_facts` guards the shared row, not the node's own view.
A canvas save reporting no edited fact is exactly what hiding a service
looks like from the row's side — nothing about the device changed. The
view still has to land, or the toggle would not survive the save.
"""
db_session.add(
InventoryDevice(
id="d-1", ip="10.0.0.5",
services=[
{"port": 22, "protocol": "tcp", "service_name": "ssh"},
{"port": 3001, "protocol": "tcp", "service_name": "Uptime Kuma"},
],
)
)
await db_session.commit()
design = await _design(db_session)
node = await self._node_on(client, headers, design)
res = await client.post(
"/api/v1/canvas/save",
json={
"design_id": design,
"nodes": [{
"id": node["id"], "type": "nas", "label": "NAS", "ip": "10.0.0.5",
"device_id": "d-1", "changed_facts": [], "pos_x": 10.0, "pos_y": 20.0,
"services": [
{"port": 3001, "protocol": "tcp", "service_name": "Uptime Kuma", "visible": False},
{"port": 22, "protocol": "tcp", "service_name": "ssh"},
],
}],
"edges": [],
"viewport": {"x": 0, "y": 0, "zoom": 1},
},
headers=headers,
)
assert res.status_code == 200
loaded = (await client.get(f"/api/v1/canvas?design_id={design}", headers=headers)).json()
assert [(s["service_name"], s.get("visible", True)) for s in loaded["nodes"][0]["services"]] == [
("Uptime Kuma", False), ("ssh", True),
]
device = await db_session.get(InventoryDevice, "d-1")
await db_session.refresh(device)
assert [s["service_name"] for s in device.services] == ["ssh", "Uptime Kuma"]
@pytest.mark.asyncio
async def test_seeding_freezes_what_a_pre_upgrade_node_showed(self, db_session):
"""`display_view` arrives NULL on every existing node; the seed fills it."""
design = await _design(db_session)
db_session.add(
InventoryDevice(
id="d-1", ip="10.0.0.5",
services=[{"port": 22, "protocol": "tcp", "service_name": "ssh"}],
)
)
node = _node(design, device_id="d-1")
db_session.add(node)
await db_session.commit()
assert await seed_node_views(db_session) == 1
await db_session.commit()
assert node.display_view == {
"services": [{"key": "22|tcp|ssh", "visible": True}],
"properties": [],
}
# Idempotent: a second boot finds nothing without a view.
assert await seed_node_views(db_session) == 0
@pytest.mark.asyncio
async def test_a_node_whose_device_was_deleted_does_not_keep_the_seed_alive(self, db_session):
"""Deleting a device leaves `nodes.device_id` dangling — foreign keys are off.
Such a node can be seeded from nothing, so it must not keep matching the
query: it would re-open the pre-upgrade backup on every single boot and
log a recovery that recovers nothing.
"""
design = await _design(db_session)
db_session.add(_node(design, device_id="d-gone"))
await db_session.commit()
reads = []
def drawn():
reads.append(1)
return {}
assert await seed_node_views(db_session, drawn=drawn) == 0
assert reads == [] # the backup was never opened
@pytest.mark.asyncio
async def test_seeding_leaves_furniture_alone(self, db_session):
design = await _design(db_session)
node = _node(design, type="groupRect")
db_session.add(node)
await db_session.commit()
assert await seed_node_views(db_session) == 0
assert node.display_view is None
@pytest.mark.asyncio
async def test_a_view_matching_nothing_left_on_the_row_shows_the_row(self, db_session):
"""The row was replaced wholesale under the node — don't draw a blank node.
A view whose every key is gone tells us nothing about the list that
replaced it, and hiding all of it leaves a node showing no service while
"Show services" is on (#347). An *empty* view is a real answer and still
hides everything — that case is covered below.
"""
device = InventoryDevice(
id="d-1",
services=[
{"port": 8080, "protocol": "tcp", "service_name": "HTTP Alt"},
{"port": 5001, "protocol": "tcp", "service_name": "Synology DSM HTTPS"},
],
)
node = _node(
await _design(db_session),
device_id="d-1",
display_view={"services": [{"key": "9000|tcp|portainer", "visible": True}]},
)
assert [s["service_name"] for s in hydrated_node(node, device)["services"]] == [
"HTTP Alt", "Synology DSM HTTPS",
]
assert all(s.get("visible", True) for s in hydrated_node(node, device)["services"])
@pytest.mark.asyncio
async def test_an_empty_view_still_hides_everything(self, db_session):
""""This canvas draws none of them" is an answer, not a missing view."""
device = InventoryDevice(
id="d-1", services=[{"port": 22, "protocol": "tcp", "service_name": "ssh"}]
)
node = _node(await _design(db_session), device_id="d-1", display_view={"services": []})
assert hydrated_node(node, device)["services"] == [
{"port": 22, "protocol": "tcp", "service_name": "ssh", "visible": False}
]
@pytest.mark.asyncio
async def test_seeding_keeps_a_property_added_after_the_backup(self, db_session):
"""The backup is 3.2.0-era: it cannot know a property added since.
Adding one while on 3.3.0-3.3.2 wrote it to the row, and every canvas
drew it. Seeding strictly from the backup would take it off all of them
at once (#347), so it is appended visible after what the backup places.
"""
design = await _design(db_session)
db_session.add(
InventoryDevice(
id="d-1", ip="10.0.0.5",
properties=[
{"key": "Rack", "value": "A", "icon": None, "visible": True},
{"key": "Ports", "value": "8", "icon": None, "visible": True},
],
)
)
node = _node(design, device_id="d-1")
db_session.add(node)
await db_session.commit()
# The backup only knew "Rack", and this canvas had it hidden.
drawn = {node.id: {
"services": [],
"properties": [{"key": "Rack", "value": "A", "icon": None, "visible": False}],
}}
assert await seed_node_views(db_session, drawn=lambda: drawn) == 1
await db_session.commit()
assert node.display_view["properties"] == [
{"key": "rack", "visible": False}, # recovered, still hidden here
{"key": "ports", "visible": True}, # added since the backup, kept
]
@pytest.mark.asyncio
async def test_seeding_still_holds_back_a_service_found_since_the_backup(self, db_session):
"""Services are not treated that way: a scan's find stays off the canvas."""
design = await _design(db_session)
db_session.add(
InventoryDevice(
id="d-1", ip="10.0.0.5",
services=[
{"port": 22, "protocol": "tcp", "service_name": "ssh"},
{"port": 5001, "protocol": "tcp", "service_name": "Synology DSM HTTPS"},
],
)
)
node = _node(design, device_id="d-1")
db_session.add(node)
await db_session.commit()
drawn = {node.id: {
"services": [{"port": 22, "protocol": "tcp", "service_name": "ssh"}],
"properties": [],
}}
assert await seed_node_views(db_session, drawn=lambda: drawn) == 1
await db_session.commit()
assert node.display_view["services"] == [{"key": "22|tcp|ssh", "visible": True}]
@pytest.mark.asyncio
async def test_a_node_without_a_view_still_shows_everything(self, db_session):
"""No view — furniture, or a node an older version linked — is not "hide all"."""
device = InventoryDevice(
id="d-1", services=[{"port": 22, "protocol": "tcp", "service_name": "ssh"}]
)
node = _node(await _design(db_session), device_id="d-1")
assert hydrated_node(node, device)["services"] == [
{"port": 22, "protocol": "tcp", "service_name": "ssh"}
]
+448
View File
@@ -375,6 +375,79 @@ def test_nmap_scan_single_non_root_uses_connect_scan():
assert result["open_ports"][0]["banner"] == "nginx 1.24"
def test_nmap_scan_single_discovery_is_unbounded_by_default():
"""The range scan's discovery pass keeps its authoritative, untimed run."""
from app.services.scanner import _nmap_scan_single
host = {"ip": "192.168.1.14", "hostname": None, "mac": None, "os": None, "open_ports": []}
disc = _fake_scanner("192.168.1.14", {})
with patch("app.services.scanner.nmap.PortScanner", side_effect=[disc]), \
patch("app.services.scanner.os.geteuid", return_value=1000):
_nmap_scan_single(host)
args = disc.scan.call_args.kwargs["arguments"]
assert "--host-timeout" not in args
assert "--min-rate" not in args
def test_nmap_scan_single_bounded_never_sets_a_host_timeout():
"""A deep slice drops retries — never --host-timeout.
nmap answers a host timeout with "Skipping host <ip> due to host timeout"
and throws away every port it had already found, so a ceiling here turns a
slow scan into one that reports nothing. The time on a dropping host goes to
the retry pass — measured 2x — so that is what the deep scan gives up.
"""
from app.services.scanner import _nmap_scan_single
host = {"ip": "192.168.1.15", "hostname": None, "mac": None, "os": None, "open_ports": []}
disc = _fake_scanner("192.168.1.15", {})
with patch("app.services.scanner.nmap.PortScanner", side_effect=[disc]), \
patch("app.services.scanner.os.geteuid", return_value=1000):
_nmap_scan_single(host, "1-8192", True)
args = disc.scan.call_args.kwargs["arguments"]
assert "-p 1-8192" in args
assert "--host-timeout" not in args
assert "--max-retries 0" in args
def test_deep_port_chunks_cover_every_port_once():
from app.services.scanner import _deep_port_chunks
chunks = _deep_port_chunks(8192)
assert chunks[0] == "1-8192"
assert chunks[-1].endswith("-65535")
covered = []
for c in chunks:
lo, hi = (int(x) for x in c.split("-"))
covered.extend(range(lo, hi + 1))
assert covered == list(range(1, 65536))
def test_port_chunks_pack_ranges_and_honour_the_slice_size():
from app.services.scanner import _port_chunks
# Small ranges share one call instead of one call each.
assert _port_chunks("80,443,8000-9000") == ["80,443,8000-9000"]
# A range wider than the slice is cut at the slice boundary.
assert _port_chunks("1-100", 40) == ["1-40", "41-80", "81-100"]
# Overlapping input is merged before slicing, so no port is scanned twice.
assert _port_chunks("1-100,50-200", 1000) == ["1-200"]
assert _port_chunks("nonsense") == []
def test_parse_port_spec_rejects_what_nmap_could_not_use():
from app.services.scanner import _parse_port_spec, _valid_port_spec
for bad in ["", " ", "0", "65536", "100-50", "80,", "http", "-80"]:
assert _parse_port_spec(bad) == [], bad
assert _valid_port_spec(bad) is False, bad
assert _valid_port_spec("80,443,8000-9000") is True
# ---------------------------------------------------------------------------
# _nmap_scan
# ---------------------------------------------------------------------------
@@ -750,6 +823,48 @@ async def test_run_scan_merges_proxmox_row_by_mac(mem_db):
assert set(row.discovery_sources) == {"proxmox", "arp"} # both filters
@pytest.mark.asyncio
async def test_run_scan_keeps_services_the_fingerprint_cannot_see(mem_db):
"""A re-scan unions its fingerprint onto the row — it never replaces it.
Since 3.3.0 the row is the only copy of a device's services and every canvas
drawing it reads that list, so overwriting it with what nmap happened to
match would delete hand-added services everywhere at once (#347).
"""
from app.services.scanner import run_scan
run_id = _make_run_id()
async with mem_db() as session:
session.add(_make_scan_run(run_id))
session.add(InventoryDevice(
id="row-1", ip="192.168.1.60", status="approved",
discovery_source="arp", discovery_sources=["arp"],
services=[
{"port": 9000, "protocol": "tcp", "service_name": "Portainer", "path": "/#!/home"},
{"port": 22, "protocol": "tcp", "service_name": "ssh"},
],
))
await session.commit()
nmap_hosts = [{"ip": "192.168.1.60", "hostname": "docker.lan", "mac": None, "os": None,
"open_ports": [{"port": 22, "protocol": "tcp", "banner": "OpenSSH 9.2"}]}]
async with mem_db() as session:
with patch("app.services.scanner._nmap_scan", return_value=nmap_hosts), \
patch("app.services.scanner._mdns_discover", new_callable=AsyncMock, return_value=[]), \
patch("app.api.routes.status.broadcast_scan_update", new_callable=AsyncMock):
await run_scan(["192.168.1.0/24"], session, run_id)
async with mem_db() as session:
row = await session.get(InventoryDevice, "row-1")
by_port = {s["port"]: s for s in row.services}
assert by_port[9000]["service_name"] == "Portainer" # hand-added, untouched
assert by_port[9000]["path"] == "/#!/home"
assert 22 in by_port # what the scan saw is still there
assert row.status == "approved"
@pytest.mark.asyncio
async def test_run_scan_mdns_skipped_if_already_in_nmap(mem_db):
"""If nmap and mDNS both find the same IP, it should not be double-counted."""
@@ -988,3 +1103,336 @@ async def test_run_scan_no_probe_when_disabled(mem_db):
await run_scan(["192.168.1.0/24"], session, run_id)
probe.assert_not_called()
# ---------------------------------------------------------------------------
# run_device_scan — per-device deep rescan (issue #350)
# ---------------------------------------------------------------------------
def test_build_port_spec_full_covers_every_tcp_port():
from app.services.scanner import _build_port_spec
# full wins over the curated list *and* over user ranges — the deep rescan
# is only worth its minutes if it really scans everything.
assert _build_port_spec(None, full=True) == "1-65535"
assert _build_port_spec(["8000-8100"], full=True) == "1-65535"
@pytest.mark.asyncio
async def test_run_device_scan_refreshes_services_and_marks_run_done(mem_db):
from app.services.scanner import run_device_scan
run_id = _make_run_id()
async with mem_db() as session:
session.add(ScanRun(id=run_id, status="running", kind="device", ranges=["192.168.1.9/32"]))
session.add(InventoryDevice(id="d1", ip="192.168.1.9", status="approved", services=[]))
await session.commit()
seen_specs = []
def _scanned(host_dict, port_spec, bounded=False):
seen_specs.append(port_spec)
# A full-range rescan is always bounded, or it never ends.
assert bounded is True
# Only the slice holding 22 reports it — the union is the caller's job.
if port_spec == "1-8192":
host_dict["open_ports"] = [{"port": 22, "protocol": "tcp", "banner": "OpenSSH 9.2"}]
return host_dict
async with mem_db() as session:
with patch("app.services.scanner._nmap_scan_single", side_effect=_scanned), \
patch("app.api.routes.status.broadcast_scan_update", new_callable=AsyncMock):
await run_device_scan("d1", session, run_id)
async with mem_db() as session:
device = await session.get(InventoryDevice, "d1")
run = await session.get(ScanRun, run_id)
assert device is not None and run is not None
assert run.status == "done"
assert device.last_scan is not None
assert any(s.get("port") == 22 for s in device.services)
# A rescan refreshes an existing row; it never spawns a second one.
assert device.status == "approved"
@pytest.mark.asyncio
async def test_run_device_scan_keeps_hand_added_services(mem_db):
"""Regression: the rescan unions, it does not replace.
Services the user typed in by hand are the only copy that exists — every
canvas drawing the device reads this list.
"""
from app.services.scanner import run_device_scan
run_id = _make_run_id()
hand_added = {"port": 9000, "protocol": "tcp", "service_name": "My App"}
async with mem_db() as session:
session.add(ScanRun(id=run_id, status="running", kind="device", ranges=["192.168.1.9/32"]))
session.add(InventoryDevice(id="d1", ip="192.168.1.9", status="pending", services=[hand_added]))
await session.commit()
def _scanned(host_dict, port_spec, bounded=False):
host_dict["open_ports"] = [{"port": 22, "protocol": "tcp", "banner": "OpenSSH 9.2"}]
return host_dict
async with mem_db() as session:
with patch("app.services.scanner._nmap_scan_single", side_effect=_scanned), \
patch("app.api.routes.status.broadcast_scan_update", new_callable=AsyncMock):
await run_device_scan("d1", session, run_id)
async with mem_db() as session:
device = await session.get(InventoryDevice, "d1")
assert device is not None
ports = {s.get("port") for s in device.services}
assert ports == {22, 9000}
@pytest.mark.asyncio
async def test_scan_keeps_the_icon_the_user_picked_for_a_service(mem_db):
"""Regression: every scan used to repaint hand-picked service icons.
The fingerprint guesses an icon from the port; the user's choice is the
only one that means anything, so a rescan leaves it where it is.
"""
from app.services.scanner import run_device_scan
run_id = _make_run_id()
curated = {
"port": 22,
"protocol": "tcp",
"service_name": "ssh",
"icon": "brand:openssh",
"category": "remote",
}
async with mem_db() as session:
session.add(ScanRun(id=run_id, status="running", kind="device", ranges=["192.168.1.9/32"]))
session.add(InventoryDevice(id="d1", ip="192.168.1.9", status="approved", services=[curated]))
await session.commit()
def _scanned(host_dict, port_spec, bounded=False):
host_dict["open_ports"] = [{"port": 22, "protocol": "tcp", "banner": "OpenSSH 9.2"}]
return host_dict
async with mem_db() as session:
with patch("app.services.scanner._nmap_scan_single", side_effect=_scanned), \
patch("app.api.routes.status.broadcast_scan_update", new_callable=AsyncMock):
await run_device_scan("d1", session, run_id)
async with mem_db() as session:
device = await session.get(InventoryDevice, "d1")
assert device is not None
matches = [s for s in device.services if s.get("port") == 22]
# Merged in place — the name key is case-insensitive, so "SSH" from the
# signature does not append a second row next to the user's "ssh".
assert len(matches) == 1
ssh = matches[0]
assert ssh["icon"] == "brand:openssh"
assert ssh["category"] == "remote"
@pytest.mark.asyncio
async def test_run_device_scan_keeps_the_device_own_discovery_source(mem_db):
"""A rescan re-observes a device; it does not discover it on the network.
Tagging every rescanned device "arp" told a Proxmox guest or a hand-added
host that the network scanner found it, and it then answered that filter.
"""
from app.services.scanner import run_device_scan
run_id = _make_run_id()
async with mem_db() as session:
session.add(ScanRun(id=run_id, status="running", kind="device", ranges=["192.168.1.9/32"]))
session.add(InventoryDevice(
id="d1",
ip="192.168.1.9",
status="approved",
services=[],
discovery_source="proxmox",
discovery_sources=["proxmox"],
))
await session.commit()
def _scanned(host_dict, port_spec, bounded=False):
host_dict["open_ports"] = [{"port": 8006, "protocol": "tcp", "banner": ""}]
return host_dict
async with mem_db() as session:
with patch("app.services.scanner._nmap_scan_single", side_effect=_scanned), \
patch("app.api.routes.status.broadcast_scan_update", new_callable=AsyncMock):
await run_device_scan("d1", session, run_id, ports="8006")
async with mem_db() as session:
device = await session.get(InventoryDevice, "d1")
assert device is not None
assert device.discovery_sources == ["proxmox"]
assert device.discovery_source == "proxmox"
@pytest.mark.asyncio
async def test_run_device_scan_marks_run_error_when_device_has_no_ip(mem_db):
from app.services.scanner import run_device_scan
run_id = _make_run_id()
async with mem_db() as session:
session.add(ScanRun(id=run_id, status="running", kind="device", ranges=["/32"]))
session.add(InventoryDevice(id="d1", ip=None, status="pending", services=[]))
await session.commit()
async with mem_db() as session:
with patch("app.api.routes.status.broadcast_scan_update", new_callable=AsyncMock):
await run_device_scan("d1", session, run_id)
async with mem_db() as session:
run = await session.get(ScanRun, run_id)
assert run is not None
assert run.status == "error"
assert run.error is not None
@pytest.mark.asyncio
async def test_run_device_scan_skips_nmap_when_cancelled(mem_db):
from app.services.scanner import _cancelled_runs, request_cancel, run_device_scan
run_id = _make_run_id()
async with mem_db() as session:
session.add(ScanRun(id=run_id, status="running", kind="device", ranges=["192.168.1.9/32"]))
session.add(InventoryDevice(id="d1", ip="192.168.1.9", status="pending", services=[]))
await session.commit()
request_cancel(run_id)
single = MagicMock()
async with mem_db() as session:
with patch("app.services.scanner._nmap_scan_single", new=single), \
patch("app.api.routes.status.broadcast_scan_update", new_callable=AsyncMock):
await run_device_scan("d1", session, run_id)
async with mem_db() as session:
run = await session.get(ScanRun, run_id)
single.assert_not_called()
assert run is not None
assert run.status == "cancelled"
# The run cleans up its cancellation flag on the way out.
assert run_id not in _cancelled_runs
@pytest.mark.asyncio
async def test_run_device_scan_unions_ports_across_slices(mem_db):
"""Every slice contributes; a slow one costs only its own ports."""
from app.services.scanner import _deep_port_chunks, run_device_scan
run_id = _make_run_id()
async with mem_db() as session:
session.add(ScanRun(id=run_id, status="running", kind="device", ranges=["192.168.1.9/32"]))
session.add(InventoryDevice(id="d1", ip="192.168.1.9", status="pending", services=[]))
await session.commit()
def _scanned(host_dict, port_spec, bounded=False):
lo = int(port_spec.split("-")[0])
if lo == 1:
host_dict["open_ports"] = [{"port": 22, "protocol": "tcp", "banner": ""}]
elif lo == 8193:
host_dict["open_ports"] = [{"port": 8096, "protocol": "tcp", "banner": ""}]
return host_dict
async with mem_db() as session:
with patch("app.services.scanner._nmap_scan_single", side_effect=_scanned), \
patch("app.api.routes.status.broadcast_scan_update", new_callable=AsyncMock):
await run_device_scan("d1", session, run_id)
async with mem_db() as session:
device = await session.get(InventoryDevice, "d1")
run = await session.get(ScanRun, run_id)
assert device is not None and run is not None
assert {s.get("port") for s in device.services} == {22, 8096}
# A complete sweep carries no advisory.
assert run.error is None
assert len(_deep_port_chunks()) == 8
@pytest.mark.asyncio
async def test_run_device_scan_honours_a_requested_port_range(mem_db):
"""A user-chosen range replaces the full sweep — and skips retry-free timing.
A handful of ports is cheap enough to scan properly; the bounded flags only
pay off over thousands of them.
"""
from app.services.scanner import run_device_scan
run_id = _make_run_id()
async with mem_db() as session:
session.add(ScanRun(id=run_id, status="running", kind="device", ranges=["192.168.1.9/32"]))
session.add(InventoryDevice(id="d1", ip="192.168.1.9", status="pending", services=[]))
await session.commit()
calls = []
def _scanned(host_dict, port_spec, bounded=False):
calls.append((port_spec, bounded))
host_dict["open_ports"] = [{"port": 8096, "protocol": "tcp", "banner": ""}]
return host_dict
async with mem_db() as session:
with patch("app.services.scanner._nmap_scan_single", side_effect=_scanned), \
patch("app.api.routes.status.broadcast_scan_update", new_callable=AsyncMock):
await run_device_scan("d1", session, run_id, ports="8000-9000")
async with mem_db() as session:
device = await session.get(InventoryDevice, "d1")
run = await session.get(ScanRun, run_id)
assert calls == [("8000-9000", False)]
assert device is not None and run is not None
assert {s.get("port") for s in device.services} == {8096}
assert run.status == "done"
assert run.error is None
@pytest.mark.asyncio
async def test_run_device_scan_keeps_what_it_found_when_the_budget_runs_out(mem_db):
"""A spent budget stops the sweep — it never discards the ports found.
The earlier --host-timeout did exactly that (nmap skips the host wholesale),
which is why a deep scan could come back empty on a slow host.
"""
from app.services.scanner import run_device_scan
run_id = _make_run_id()
async with mem_db() as session:
session.add(ScanRun(id=run_id, status="running", kind="device", ranges=["192.168.1.9/32"]))
session.add(InventoryDevice(id="d1", ip="192.168.1.9", status="pending", services=[]))
await session.commit()
calls = []
def _scanned(host_dict, port_spec, bounded=False):
calls.append(port_spec)
host_dict["open_ports"] = [{"port": 22, "protocol": "tcp", "banner": ""}]
return host_dict
async with mem_db() as session:
# Budget already spent when the first slice returns.
with patch("app.services.scanner._nmap_scan_single", side_effect=_scanned), \
patch("app.services.scanner.settings") as mock_settings, \
patch("app.api.routes.status.broadcast_scan_update", new_callable=AsyncMock):
mock_settings.scanner_deep_host_timeout = -1
await run_device_scan("d1", session, run_id)
async with mem_db() as session:
device = await session.get(InventoryDevice, "d1")
run = await session.get(ScanRun, run_id)
assert calls == ["1-8192"]
assert device is not None and run is not None
assert {s.get("port") for s in device.services} == {22}
assert run.status == "done"
# Partial coverage is reported, never passed off as a full sweep.
assert run.error is not None
assert "1/8" in run.error
+2 -2
View File
@@ -1,12 +1,12 @@
{
"name": "frontend",
"version": "3.3.2",
"version": "3.3.5",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "frontend",
"version": "3.3.2",
"version": "3.3.5",
"dependencies": {
"@base-ui/react": "^1.2.0",
"@dagrejs/dagre": "^2.0.4",
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "frontend",
"private": true,
"version": "3.3.2",
"version": "3.3.5",
"type": "module",
"scripts": {
"dev": "vite",
+23
View File
@@ -109,6 +109,20 @@ export interface DeepScanConfig {
export type ScanConfigData = { ranges: string[] } & DeepScanConfig
/** A row of `scan_runs` — what `/scan/runs` and the device rescan return. */
export interface ScanRunSummary {
id: string
// 'failed' is legacy: runs recorded before the backend settled on 'error'
// for the same condition. Still read, never written.
status: 'running' | 'done' | 'cancelled' | 'error' | 'failed'
kind: string
ranges: string[]
devices_found: number
started_at: string
finished_at: string | null
error: string | null
}
// A device the backend refused to place because an equivalent node already
// exists on the target design (same ip/mac/ieee). `existing_node_id` points at
// the node already there so the UI can link to it.
@@ -158,8 +172,17 @@ export const scanApi = {
*/
updatePending: (id: string, data: Partial<Omit<InventoryEntry, 'id' | 'status' | 'discovered_at'>>) =>
api.patch<InventoryEntry>(`/scan/pending/${id}`, data),
/**
* Deep-rescan one known device: every TCP port, then re-fingerprint. Answers
* "this device predates the scanner knowing that service" (issue #350).
* Returns the ScanRun, so the caller polls `run` and can `stop` it.
* 409 when the device has no IP, is hidden, or is already being rescanned.
*/
rescanDevice: (id: string, opts?: { full_ports?: boolean; ports?: string; http_probe_enabled?: boolean; verify_tls?: boolean }) =>
api.post<ScanRunSummary>(`/scan/pending/${id}/rescan`, opts ?? {}),
hidden: () => api.get('/scan/hidden'),
runs: () => api.get('/scan/runs'),
run: (runId: string) => api.get<ScanRunSummary>(`/scan/runs/${runId}`),
clearPending: () => api.delete('/scan/pending'),
/** Remove one inventory entry. 409 when a rack still mounts it. */
deletePending: (id: string) => api.delete<{ deleted: boolean }>(`/scan/pending/${id}`),
@@ -40,7 +40,9 @@ export function BaseNode({ id, data, selected, icon: typeIcon, width, height }:
const colors = resolveNodeColors(data, activeTheme)
const statusColor = theme.colors.statusColors[data.status]
const isOnline = data.status === 'online'
const services = data.services ?? []
// Hiding a service is a property of this node, not of the device: another
// canvas drawing the same device answers for itself.
const services = (data.services ?? []).filter((svc) => svc.visible !== false)
const showServices = data.custom_colors?.show_services === true
const serviceHost = data.ip ? primaryIp(data.ip) : data.hostname
@@ -144,11 +146,12 @@ export function BaseNode({ id, data, selected, icon: typeIcon, width, height }:
<div className="flex flex-col gap-1 px-2.5 py-1.5 overflow-hidden">
{visibleProperties.map((prop) => {
const Icon = resolvePropertyIcon(prop.icon)
const hasValue = Boolean(prop.value.trim())
return (
<div key={prop.key} className="flex items-center gap-1 font-mono text-[10px] min-w-0 overflow-hidden" style={{ color: theme.colors.nodeSubtextColor }}>
{Icon && <Icon size={9} className="shrink-0" />}
<span className="truncate max-w-15 shrink-0" title={prop.key}>{prop.key}</span>
{prop.value.trim() && (
<span className={hasValue ? 'truncate max-w-15 shrink-0' : 'truncate min-w-0'} title={prop.key}>{prop.key}</span>
{hasValue && (
<span className="truncate min-w-0" title={prop.value}>· {prop.value}</span>
)}
</div>
@@ -107,6 +107,7 @@ export function ProxmoxGroupNode(props: NodeProps<Node<NodeData>>) {
{/* Properties */}
{data.properties?.filter((p) => p.visible).map((prop, i, arr) => {
const Icon = resolvePropertyIcon(prop.icon)
const hasValue = Boolean(prop.value.trim())
return (
<div
key={prop.key}
@@ -119,8 +120,8 @@ export function ProxmoxGroupNode(props: NodeProps<Node<NodeData>>) {
}}
>
{Icon && <Icon size={9} className="shrink-0" />}
<span className="truncate max-w-15 shrink-0" title={prop.key}>{prop.key}</span>
{prop.value.trim() && (
<span className={hasValue ? 'truncate max-w-15 shrink-0' : 'truncate min-w-0'} title={prop.key}>{prop.key}</span>
{hasValue && (
<span className="truncate min-w-0" title={prop.value}>· {prop.value}</span>
)}
</div>
@@ -0,0 +1,62 @@
/**
* A property with no value gives its whole line to the label.
*
* The label is capped at max-w-15 so a long key cannot crowd out the value
* beside it. With no value there is nothing to protect, and keeping the cap
* truncated the label against empty space (issue #361).
*/
import { describe, it, expect, vi } from 'vitest'
import { render } from '@testing-library/react'
import { ReactFlowProvider } from '@xyflow/react'
import { Server } from 'lucide-react'
import { BaseNode } from '../BaseNode'
import type { NodeData, NodeProperty } from '@/types'
vi.mock('@/stores/canvasStore', async () => {
const actual = await vi.importActual<typeof import('@/stores/canvasStore')>('@/stores/canvasStore')
return {
...actual,
useCanvasStore: (selector: (s: Record<string, unknown>) => unknown) =>
selector({ hideIp: false, serviceStatuses: {} }),
}
})
function renderNode(properties: NodeProperty[]) {
const data: NodeData = { label: 'NAS', type: 'nas', status: 'online', properties }
return render(
<ReactFlowProvider>
<BaseNode
{...({ id: 'n1', data, selected: false, icon: Server } as React.ComponentProps<typeof BaseNode>)}
/>
</ReactFlowProvider>,
)
}
function labelOf(container: HTMLElement, key: string) {
return [...container.querySelectorAll('span')].find((s) => s.textContent === key)
}
const prop = (key: string, value: string): NodeProperty => ({ key, value, visible: true })
describe('BaseNode properties', () => {
it('drops the label width cap when the value is empty', () => {
const { container } = renderNode([prop('A very long property label', '')])
const label = labelOf(container, 'A very long property label')
expect(label).toBeDefined()
expect(label!.className).not.toContain('max-w-15')
expect(label!.className).toContain('min-w-0')
})
it('treats a whitespace-only value as empty', () => {
const { container } = renderNode([prop('Label', ' ')])
const label = labelOf(container, 'Label')
expect(label!.className).not.toContain('max-w-15')
})
it('keeps the cap when a value shares the line', () => {
const { container } = renderNode([prop('Label', '42')])
const label = labelOf(container, 'Label')
expect(label!.className).toContain('max-w-15')
expect(container.textContent).toContain('· 42')
})
})
@@ -0,0 +1,63 @@
/**
* A service hidden on this node stays on the device.
*
* Order and visibility are the node's, the facts are the inventory row's, so
* the same device drawn on two canvases can show two different service lists.
* The node card only draws what its own copy says is visible.
*/
import { describe, it, expect, vi } from 'vitest'
import { render, screen } from '@testing-library/react'
import { ReactFlowProvider } from '@xyflow/react'
import { Server } from 'lucide-react'
import { BaseNode } from '../BaseNode'
import type { NodeData, ServiceInfo } from '@/types'
vi.mock('@/stores/canvasStore', async () => {
const actual = await vi.importActual<typeof import('@/stores/canvasStore')>('@/stores/canvasStore')
return {
...actual,
useCanvasStore: (selector: (s: Record<string, unknown>) => unknown) =>
selector({ hideIp: false, serviceStatuses: {} }),
}
})
const ssh: ServiceInfo = { port: 22, protocol: 'tcp', service_name: 'ssh' }
const kuma: ServiceInfo = { port: 3001, protocol: 'tcp', service_name: 'Uptime Kuma' }
function renderNode(services: ServiceInfo[]) {
const data: NodeData = {
label: 'NAS',
type: 'nas',
status: 'online',
services,
// Services only render when the node is set to show them.
custom_colors: { show_services: true },
}
return render(
<ReactFlowProvider>
<BaseNode
{...({ id: 'n1', data, selected: false, icon: Server } as React.ComponentProps<typeof BaseNode>)}
/>
</ReactFlowProvider>,
)
}
describe('BaseNode services', () => {
it('draws a service that carries no visible flag', () => {
renderNode([ssh, kuma])
expect(screen.getByText('ssh')).toBeInTheDocument()
expect(screen.getByText('Uptime Kuma')).toBeInTheDocument()
})
it('leaves out one this node hid, keeping the rest', () => {
renderNode([ssh, { ...kuma, visible: false }])
expect(screen.getByText('ssh')).toBeInTheDocument()
expect(screen.queryByText('Uptime Kuma')).not.toBeInTheDocument()
})
it('draws them in the order the node carries', () => {
const { container } = renderNode([kuma, ssh])
const names = [...container.querySelectorAll('span.font-medium')].map((n) => n.textContent)
expect(names.filter((n) => n === 'ssh' || n === 'Uptime Kuma')).toEqual(['Uptime Kuma', 'ssh'])
})
})
@@ -0,0 +1,130 @@
/**
* Deep scan — pick what to sweep before starting one.
*
* The full 65535-port range is the default and the point of the feature
* (issue #350), but it costs minutes per host. A user who already knows where
* a service lives, or who only wants the low ports back, types a narrower spec
* here instead of waiting for the whole range.
*/
import { useState } from 'react'
import { Radar } from 'lucide-react'
import { Dialog, DialogContent, DialogHeader, DialogTitle } from '@/components/ui/dialog'
import { Button } from '@/components/ui/button'
import { Input } from '@/components/ui/input'
import { Label } from '@/components/ui/label'
import { countPorts, isValidPortSpec, FULL_PORT_RANGE } from '@/utils/portSpec'
import modalStyles from './modal-interactive.module.css'
const PRESETS: Array<{ label: string; spec: string }> = [
{ label: 'All ports', spec: FULL_PORT_RANGE },
{ label: 'Well-known', spec: '1-1024' },
{ label: 'Common', spec: '1-10000' },
]
interface DeepScanModalProps {
open: boolean
target?: string | null
onClose: () => void
onStart: (ports: string) => void
}
export function DeepScanModal({ open, target, onClose, onStart }: DeepScanModalProps) {
// Prefilled with the full range. The caller mounts this only while it is
// open, so a previous narrow spec never becomes the next scan's default.
const [spec, setSpec] = useState(FULL_PORT_RANGE)
const trimmed = spec.trim()
const valid = isValidPortSpec(trimmed)
const total = countPorts(trimmed)
const handleSubmit = (e: React.FormEvent) => {
e.preventDefault()
if (!valid) return
onStart(trimmed)
onClose()
}
return (
<Dialog open={open} onOpenChange={(o) => !o && onClose()}>
<DialogContent className="sm:max-w-md bg-[#161b22] border-[#30363d]">
<DialogHeader>
<DialogTitle className="flex items-center gap-2 text-base">
<Radar size={15} className="text-[#00d4ff]" />
Deep scan{target ? ` — ${target}` : ''}
</DialogTitle>
</DialogHeader>
<form onSubmit={handleSubmit} className="flex flex-col gap-3">
<div className="flex flex-col gap-1.5">
<Label htmlFor="deep-scan-ports" className="text-xs text-muted-foreground">
Port range
</Label>
<Input
id="deep-scan-ports"
value={spec}
onChange={(e) => setSpec(e.target.value)}
autoFocus
spellCheck={false}
placeholder={FULL_PORT_RANGE}
data-testid="deep-scan-ports"
className="h-8 font-mono text-sm bg-[#21262d] border-[#30363d]"
/>
<div className="flex items-center justify-between gap-2 text-[10px]">
<span className="text-muted-foreground">
A port, a range, or a comma list — <span className="font-mono">80,443,8000-9000</span>
</span>
{trimmed !== '' && (
<span className={valid ? 'text-muted-foreground shrink-0' : 'text-[#f85149] shrink-0'}>
{valid ? `${total.toLocaleString('en-US')} ports` : 'Invalid range'}
</span>
)}
</div>
</div>
<div className="flex flex-wrap gap-1.5">
{PRESETS.map((preset) => (
<button
key={preset.spec}
type="button"
onClick={() => setSpec(preset.spec)}
className={`px-2 py-1 text-[10px] border transition-colors cursor-pointer ${modalStyles['modal-radius']} ${
trimmed === preset.spec
? 'border-[#00d4ff] text-[#00d4ff] bg-[#00d4ff]/10'
: 'border-[#30363d] text-muted-foreground hover:text-foreground'
}`}
>
{preset.label}
</button>
))}
</div>
<p className="text-[10px] text-muted-foreground">
Scanning runs in the background — the whole range takes several minutes. Found services are
merged into the device; nothing already recorded is removed.
</p>
<div className="flex justify-end gap-2 pt-1">
<Button
type="button"
variant="ghost"
size="sm"
className={`cursor-pointer ${modalStyles['modal-cancel-hover']}`}
onClick={onClose}
>
Cancel
</Button>
<Button
type="submit"
size="sm"
disabled={!valid}
data-testid="deep-scan-start"
className="bg-[#00d4ff] text-[#0d1117] hover:bg-[#00d4ff]/90 cursor-pointer disabled:opacity-50 disabled:cursor-not-allowed"
>
Start scan
</Button>
</div>
</form>
</DialogContent>
</Dialog>
)
}
@@ -14,7 +14,7 @@
* badges, then three columns — identity / operations / curation — so a device
* reads in one screen instead of a scrolling column of key-value pairs.
*/
import { useEffect, useState } from 'react'
import { useEffect, useRef, useState } from 'react'
import {
Check,
Copy,
@@ -23,9 +23,11 @@ import {
HeartPulse,
History,
Layers,
Loader2,
Network,
Pencil,
Plus,
Radar,
StickyNote,
Tags,
X,
@@ -40,6 +42,7 @@ import { Label } from '@/components/ui/label'
import { Select, SelectContent, SelectGroup, SelectItem, SelectLabel, SelectSeparator, SelectTrigger, SelectValue } from '@/components/ui/select'
import { PropertyList } from '@/components/common/PropertyList'
import { ServiceModal } from './ServiceModal'
import { DeepScanModal } from './DeepScanModal'
import { scanApi } from '@/api/client'
import { useCanvasStore } from '@/stores/canvasStore'
import { useThemeStore } from '@/stores/themeStore'
@@ -49,6 +52,7 @@ import { NODE_TYPE_DEFAULT_ICONS } from '@/utils/nodeIcons'
import { isRackDevice, orderedSources, SOURCE_META } from '@/utils/deviceSources'
import { DEVICE_TYPE_GROUPS } from '@/utils/nodeTypeGroups'
import { formatRelative, formatTimestamp } from '@/utils/timeFormat'
import { countPorts } from '@/utils/portSpec'
import { serviceToForm, type ServiceFormData, type ServiceSubmitData } from '@/utils/serviceForm'
import { NODE_TYPE_LABELS, type CheckMethod, type InventoryEntry, type NodeProperty, type NodeType, type ServiceInfo } from '@/types'
import modalStyles from './modal-interactive.module.css'
@@ -263,18 +267,89 @@ export function InventoryDeviceModal({ device, onClose, onApprove, onHide, onIgn
const [services, setServices] = useState<ServiceInfo[]>(device?.services ?? [])
const [svcModal, setSvcModal] = useState<{ index: number | null; form?: ServiceFormData } | null>(null)
const [saving, setSaving] = useState(false)
// The id of the deep rescan this modal started, while it is still running.
const [rescanRunId, setRescanRunId] = useState<string | null>(null)
const [rescanStarting, setRescanStarting] = useState(false)
// The port-range dialog, opened by the Deep scan link.
const [deepScanOpen, setDeepScanOpen] = useState(false)
const activeTheme = useThemeStore((s) => s.activeTheme)
// Kept in a ref so the poll effect below doesn't restart — and lose its
// timer — every time the parent re-renders with a new callback identity.
const onSavedRef = useRef(onSaved)
onSavedRef.current = onSaved
const editingRef = useRef(editing)
editingRef.current = editing
// Read by the reset effect, which keys on the id alone — the object itself
// must not be a dependency.
const deviceRef = useRef(device)
deviceRef.current = device
// Reset the form whenever the modal is pointed at another device — the
// Reset the form whenever the modal is pointed at *another* device — the
// component stays mounted across card clicks in the inventory grid.
//
// Keyed on the id, never on the object: the parent hands down a fresh
// `device` every time the row is refreshed from anywhere — a finishing deep
// rescan, a status push, a save — and resetting on those would drop an edit
// in progress along with the mode itself. A refreshed row is not a different
// device, and only a different device is a reason to throw the form away.
const deviceId = device?.id ?? null
useEffect(() => {
if (!device) return
setForm(toForm(device))
setProperties(device.properties ?? [])
setServices(device.services ?? [])
const d = deviceRef.current
if (!d) return
setForm(toForm(d))
setProperties(d.properties ?? [])
setServices(d.services ?? [])
setEditing(false)
setSvcModal(null)
}, [device])
setRescanRunId(null)
setDeepScanOpen(false)
}, [deviceId])
// Wait on a running deep rescan. A full 65535-port scan takes minutes, so the
// run is polled rather than awaited — the user can close the modal, and the
// scan keeps going and still shows under Scan History.
useEffect(() => {
if (!rescanRunId || !deviceId) return
let stopped = false
const timer = window.setInterval(async () => {
try {
const { data: run } = await scanApi.run(rescanRunId)
if (stopped || run.status === 'running') return
setRescanRunId(null)
if (run.status === 'error' || run.status === 'failed') {
toast.error(`Scan failed: ${run.error ?? 'unknown error'}`)
return
}
if (run.status === 'cancelled') {
toast.info('Scan stopped')
return
}
const { data: rows } = await scanApi.pending()
const fresh = (rows as InventoryEntry[]).find((d) => d.id === deviceId)
if (!fresh || stopped) return
// Never clobber an edit in progress — the user's unsaved services win.
if (!editingRef.current) setServices(fresh.services ?? [])
useCanvasStore.getState().applyDeviceFacts(fresh.id, deviceFactsToNodeData(fresh))
useCanvasStore.getState().notifyScanDeviceFound()
onSavedRef.current?.(fresh)
const n = fresh.services?.length ?? 0
const summary = `${n} service${n !== 1 ? 's' : ''}`
// A done run can still carry an advisory: the sweep ran out of budget
// before every port range. Saying "done" flat would read as complete.
if (run.error) {
toast.warning(`Scan partial — ${summary}. ${run.error}`)
} else {
toast.success(`Scan done — ${summary}`)
}
} catch {
// Transient failure: keep polling, the run is still on the server.
}
}, 3000)
return () => {
stopped = true
window.clearInterval(timer)
}
}, [rescanRunId, deviceId])
if (!device) return null
@@ -301,6 +376,31 @@ export function InventoryDeviceModal({ device, onClose, onApprove, onHide, onIgn
setServices(device.services ?? [])
}
const handleRescan = async (ports: string) => {
if (!device.ip || rescanRunId || rescanStarting) return
setRescanStarting(true)
try {
const res = await scanApi.rescanDevice(device.id, { ports })
setRescanRunId(res.data.id)
const n = countPorts(ports)
toast.info(`Deep scan started — ${n.toLocaleString('en-US')} ports, this takes a few minutes`)
} catch (err) {
const detail = (err as { response?: { data?: { detail?: string } } }).response?.data?.detail
toast.error(detail ?? 'Could not start the scan')
} finally {
setRescanStarting(false)
}
}
const handleStopRescan = async () => {
if (!rescanRunId) return
try {
await scanApi.stop(rescanRunId)
} catch {
toast.error('Could not stop the scan')
}
}
const handleSubmitService = (data: ServiceSubmitData) => {
const svc: ServiceInfo = {
...(data.port != null ? { port: data.port } : {}),
@@ -523,7 +623,10 @@ export function InventoryDeviceModal({ device, onClose, onApprove, onHide, onIgn
<PropertyList
properties={properties}
onChange={setProperties}
visibleLabel="Show on node"
// Whether a canvas draws a property is that node's own
// answer; the flag here only decides what a node drawing
// this device from now on starts out showing.
visibleLabel="Show on new nodes"
// Hardware is a property like any other; these are the keys
// the Proxmox import and the YAML import already mint.
suggestions={['CPU Model', 'CPU Cores', 'RAM', 'Disk']}
@@ -603,7 +706,36 @@ export function InventoryDeviceModal({ device, onClose, onApprove, onHide, onIgn
{/* Zigbee devices have no IP services — the section would always be empty. */}
{!isZigbee && (
<Section title={`Services found (${device.services.length})`} icon={Network}>
<Section
title={`Services found (${device.services.length})`}
icon={Network}
action={
// Deep rescan: the fix for a device added before the
// scanner knew its services, or one listening on a port
// no curated list covers (issue #350). Needs an IP.
device.ip ? (
rescanRunId ? (
<button
onClick={handleStopRescan}
data-testid="device-rescan-stop"
className="flex items-center gap-1 text-[10px] text-[#f85149] hover:text-[#f85149]/80 transition-colors cursor-pointer"
>
<Loader2 size={10} className="animate-spin" /> Scanning — stop
</button>
) : (
<button
onClick={() => setDeepScanOpen(true)}
disabled={rescanStarting}
data-testid="device-rescan"
title="Pick a port range and refresh the services"
className="flex items-center gap-1 text-[10px] text-[#00d4ff] hover:text-[#00d4ff]/80 transition-colors cursor-pointer disabled:opacity-50 disabled:cursor-not-allowed"
>
<Radar size={10} /> Deep scan
</button>
)
) : undefined
}
>
{device.services.length === 0 ? (
<Empty>No services detected</Empty>
) : (
@@ -753,6 +885,15 @@ export function InventoryDeviceModal({ device, onClose, onApprove, onHide, onIgn
)}
</div>
{deepScanOpen && (
<DeepScanModal
open
target={device.ip}
onClose={() => setDeepScanOpen(false)}
onStart={handleRescan}
/>
)}
{svcModal && (
<ServiceModal
key={`svc-${svcModal.index ?? 'new'}`}
@@ -0,0 +1,243 @@
/**
* Per-device deep rescan (issue #350) — the detail modal starts a full-port
* scan of one device, waits on the run, and folds the fresh services back in.
*/
import { describe, it, expect, beforeEach, afterEach, vi } from 'vitest'
import { act, render, screen, fireEvent, waitFor } from '@testing-library/react'
import { InventoryDeviceModal } from '../InventoryDeviceModal'
import type { InventoryEntry } from '@/types'
const mockRescanDevice = vi.fn()
const mockRun = vi.fn()
const mockPending = vi.fn()
const mockStop = vi.fn()
vi.mock('@/api/client', () => ({
scanApi: {
updatePending: vi.fn(),
rescanDevice: (...a: unknown[]) => mockRescanDevice(...a),
run: (...a: unknown[]) => mockRun(...a),
pending: (...a: unknown[]) => mockPending(...a),
stop: (...a: unknown[]) => mockStop(...a),
},
}))
vi.mock('sonner', async () => (await import('@/test/mocks')).mockSonner())
function makeDevice(overrides: Partial<InventoryEntry> = {}): InventoryEntry {
return {
id: 'dev-1',
ip: '192.168.1.100',
mac: 'aa:bb:cc:dd:ee:ff',
hostname: 'pve.local',
os: 'Linux',
services: [],
suggested_type: 'server',
status: 'pending',
discovered_at: '2024-01-15T10:30:00Z',
...overrides,
}
}
/** The link opens the port-range dialog; the scan starts from there. */
async function startScan() {
fireEvent.click(screen.getByTestId('device-rescan'))
await waitFor(() => expect(screen.getByTestId('deep-scan-start')).toBeInTheDocument())
// The start resolves a promise that sets state — act() keeps that update
// inside the test's control, fake timers or not.
await act(async () => { fireEvent.click(screen.getByTestId('deep-scan-start')) })
}
const noop = { onClose: vi.fn(), onApprove: vi.fn(), onHide: vi.fn(), onIgnore: vi.fn() }
beforeEach(() => {
vi.clearAllMocks()
mockRescanDevice.mockResolvedValue({ data: { id: 'run-1', status: 'running' } })
mockRun.mockResolvedValue({ data: { id: 'run-1', status: 'running', error: null } })
mockPending.mockResolvedValue({ data: [] })
mockStop.mockResolvedValue({ data: {} })
})
afterEach(() => {
vi.useRealTimers()
})
describe('InventoryDeviceModal — deep rescan', () => {
it('offers the deep scan only when the device has an IP', () => {
const { rerender } = render(<InventoryDeviceModal {...noop} device={makeDevice()} />)
expect(screen.getByTestId('device-rescan')).toBeInTheDocument()
rerender(<InventoryDeviceModal {...noop} device={makeDevice({ id: 'dev-2', ip: null })} />)
expect(screen.queryByTestId('device-rescan')).toBeNull()
})
it('is absent for a Zigbee device — it has no IP services at all', () => {
render(
<InventoryDeviceModal
{...noop}
device={makeDevice({ discovery_source: 'zigbee', ieee_address: '0x00124b' })}
/>
)
expect(screen.queryByTestId('device-rescan')).toBeNull()
})
it('scans every port and swaps to a stop control while it runs', async () => {
render(<InventoryDeviceModal {...noop} device={makeDevice()} />)
await startScan()
await waitFor(() => expect(screen.getByTestId('device-rescan-stop')).toBeInTheDocument())
expect(mockRescanDevice).toHaveBeenCalledWith('dev-1', { ports: '1-65535' })
expect(screen.queryByTestId('device-rescan')).toBeNull()
})
it('sends the range the user typed instead of the whole space', async () => {
render(<InventoryDeviceModal {...noop} device={makeDevice()} />)
fireEvent.click(screen.getByTestId('device-rescan'))
const input = await screen.findByTestId('deep-scan-ports')
expect(input).toHaveValue('1-65535')
fireEvent.change(input, { target: { value: '80,443,8000-9000' } })
fireEvent.click(screen.getByTestId('deep-scan-start'))
await waitFor(() =>
expect(mockRescanDevice).toHaveBeenCalledWith('dev-1', { ports: '80,443,8000-9000' })
)
})
it('refuses to start on a range nmap could not use', async () => {
render(<InventoryDeviceModal {...noop} device={makeDevice()} />)
fireEvent.click(screen.getByTestId('device-rescan'))
const input = await screen.findByTestId('deep-scan-ports')
fireEvent.change(input, { target: { value: '99999' } })
expect(screen.getByTestId('deep-scan-start')).toBeDisabled()
fireEvent.click(screen.getByTestId('deep-scan-start'))
expect(mockRescanDevice).not.toHaveBeenCalled()
})
it('stops the run through the same ScanRun the header uses', async () => {
render(<InventoryDeviceModal {...noop} device={makeDevice()} />)
await startScan()
await waitFor(() => expect(screen.getByTestId('device-rescan-stop')).toBeInTheDocument())
fireEvent.click(screen.getByTestId('device-rescan-stop'))
await waitFor(() => expect(mockStop).toHaveBeenCalledWith('run-1'))
})
it('keeps the button enabled again when the start is refused', async () => {
mockRescanDevice.mockRejectedValue({
response: { data: { detail: 'A scan is already running for this device' } },
})
const { toast } = await import('sonner')
render(<InventoryDeviceModal {...noop} device={makeDevice()} />)
await startScan()
await waitFor(() =>
expect(toast.error).toHaveBeenCalledWith('A scan is already running for this device')
)
expect(screen.getByTestId('device-rescan')).toBeInTheDocument()
})
it('folds the freshly found services back into the open modal', async () => {
vi.useFakeTimers({ shouldAdvanceTime: true })
const onSaved = vi.fn()
const fresh = makeDevice({
services: [{ port: 8096, protocol: 'tcp', service_name: 'Jellyfin' }],
})
render(<InventoryDeviceModal {...noop} onSaved={onSaved} device={makeDevice()} />)
await startScan()
await waitFor(() => expect(screen.getByTestId('device-rescan-stop')).toBeInTheDocument())
mockRun.mockResolvedValue({ data: { id: 'run-1', status: 'done', error: null } })
mockPending.mockResolvedValue({ data: [fresh] })
await act(async () => { await vi.advanceTimersByTimeAsync(3100) })
await waitFor(() => expect(onSaved).toHaveBeenCalledWith(fresh))
// Polling ends with the run — no second request once it is done.
const calls = mockRun.mock.calls.length
await act(async () => { await vi.advanceTimersByTimeAsync(6000) })
expect(mockRun.mock.calls.length).toBe(calls)
})
it('keeps an edit in progress when the scan lands', async () => {
// The parent patches the row in place on onSaved (`setSelected(saved)`),
// handing down a new object for the same device. That must not throw away
// a form the user is still filling in — a deep scan runs for minutes.
vi.useFakeTimers({ shouldAdvanceTime: true })
const onSaved = vi.fn()
const device = makeDevice()
const fresh = makeDevice({
services: [{ port: 8096, protocol: 'tcp', service_name: 'Jellyfin' }],
})
const { rerender } = render(
<InventoryDeviceModal {...noop} onSaved={onSaved} device={device} />
)
await startScan()
await waitFor(() => expect(screen.getByTestId('device-rescan-stop')).toBeInTheDocument())
fireEvent.click(screen.getByRole('button', { name: 'Edit' }))
fireEvent.change(screen.getByPlaceholderText('Display name'), { target: { value: 'Media box' } })
mockRun.mockResolvedValue({ data: { id: 'run-1', status: 'done', error: null } })
mockPending.mockResolvedValue({ data: [fresh] })
await act(async () => { await vi.advanceTimersByTimeAsync(3100) })
// The fresh row still reaches the canvas and the grid — the scan is not
// discarded just because a form is open.
await waitFor(() => expect(onSaved).toHaveBeenCalledWith(fresh))
rerender(<InventoryDeviceModal {...noop} onSaved={onSaved} device={fresh} />)
expect(screen.getByRole('button', { name: 'Save' })).toBeInTheDocument()
expect(screen.getByDisplayValue('Media box')).toBeInTheDocument()
})
it('still resets the form when pointed at another device', async () => {
const { rerender } = render(<InventoryDeviceModal {...noop} device={makeDevice()} />)
fireEvent.click(screen.getByRole('button', { name: 'Edit' }))
fireEvent.change(screen.getByPlaceholderText('Display name'), { target: { value: 'Media box' } })
rerender(<InventoryDeviceModal {...noop} device={makeDevice({ id: 'dev-2' })} />)
expect(screen.queryByRole('button', { name: 'Save' })).toBeNull()
expect(screen.queryByDisplayValue('Media box')).toBeNull()
})
it('says partial when the sweep ran out of budget', async () => {
vi.useFakeTimers({ shouldAdvanceTime: true })
const { toast } = await import('sonner')
const fresh = makeDevice({ services: [{ port: 22, protocol: 'tcp', service_name: 'SSH' }] })
render(<InventoryDeviceModal {...noop} device={makeDevice()} />)
await startScan()
await waitFor(() => expect(screen.getByTestId('device-rescan-stop')).toBeInTheDocument())
// A done run carrying an advisory — it scanned part of the range only.
mockRun.mockResolvedValue({
data: { id: 'run-1', status: 'done', error: 'Scanned 3/8 port ranges (1 open) — the rest was not reached' },
})
mockPending.mockResolvedValue({ data: [fresh] })
await act(async () => { await vi.advanceTimersByTimeAsync(3100) })
await waitFor(() =>
expect(toast.warning).toHaveBeenCalledWith(
expect.stringContaining('Scan partial — 1 service')
)
)
expect(toast.success).not.toHaveBeenCalled()
})
it('surfaces a failed run instead of silently ending', async () => {
vi.useFakeTimers({ shouldAdvanceTime: true })
const { toast } = await import('sonner')
render(<InventoryDeviceModal {...noop} device={makeDevice()} />)
await startScan()
await waitFor(() => expect(screen.getByTestId('device-rescan-stop')).toBeInTheDocument())
mockRun.mockResolvedValue({ data: { id: 'run-1', status: 'error', error: 'nmap missing' } })
await act(async () => { await vi.advanceTimersByTimeAsync(3100) })
await waitFor(() => expect(toast.error).toHaveBeenCalledWith('Scan failed: nmap missing'))
expect(mockPending).not.toHaveBeenCalled()
await waitFor(() => expect(screen.getByTestId('device-rescan')).toBeInTheDocument())
})
})
+23 -1
View File
@@ -134,6 +134,16 @@ export function DetailPanel({ onEdit, onOpenInventory }: DetailPanelProps) {
if (openSvcModal?.index === index) setSvcModal(null)
}
// Per node, not per device: the same device drawn on two canvases can show a
// service on one and hide it on the other. Only the order and the flag are the
// node's — the service itself still belongs to the inventory row.
const handleToggleService = (index: number) => {
snapshotHistory()
updateNode(node.id, {
services: services.map((svc, i) => (i === index ? { ...svc, visible: svc.visible === false } : svc)),
})
}
const handleStartEdit = (index: number) => {
const svc = services[index]
if (!svc) return
@@ -260,6 +270,7 @@ export function DetailPanel({ onEdit, onOpenInventory }: DetailPanelProps) {
setDragSvcIndex(null)
setDragOverSvcIndex(null)
}}
onToggleVisible={() => handleToggleService(i)}
onEdit={() => handleStartEdit(i)}
onRemove={() => handleRemoveService(i)}
/>
@@ -594,7 +605,7 @@ const CATEGORY_COLORS: Record<string, string> = {
web: '#00d4ff', database: '#a855f7', monitoring: '#39d353', storage: '#e3b341', security: '#f85149', remote: '#8b949e',
}
function ServiceBadge({ svc, host, status, draggable, isDragging, isDragOver, onDragStart, onDragEnter, onDragEnd, onDrop, onEdit, onRemove }: {
function ServiceBadge({ svc, host, status, draggable, isDragging, isDragOver, onDragStart, onDragEnter, onDragEnd, onDrop, onToggleVisible, onEdit, onRemove }: {
svc: ServiceInfo
host?: string
status?: ServiceStatus
@@ -605,6 +616,7 @@ function ServiceBadge({ svc, host, status, draggable, isDragging, isDragOver, on
onDragEnter: () => void
onDragEnd: () => void
onDrop: () => void
onToggleVisible: () => void
onEdit: () => void
onRemove: () => void
}) {
@@ -615,6 +627,8 @@ function ServiceBadge({ svc, host, status, draggable, isDragging, isDragOver, on
// A live offline service overrides the category colour with red.
const color = status === 'offline' ? '#f85149' : categoryColor
const pathLabel = svc.path?.trim() ? svc.path.trim() : ''
// Absent means shown: a service only carries the flag once it has been hidden.
const shown = svc.visible !== false
return (
<div
@@ -693,6 +707,14 @@ function ServiceBadge({ svc, host, status, draggable, isDragging, isDragOver, on
<span className="w-2.5" />
)}
<button
onClick={(e) => { e.preventDefault(); e.stopPropagation(); onToggleVisible() }}
className="text-[#8b949e] hover:text-[#00d4ff] ml-0.5 cursor-pointer transition-colors"
title={shown ? 'Hide on node' : 'Show on node'}
>
{shown ? <Eye size={10} /> : <EyeOff size={10} />}
</button>
<button
onClick={(e) => { e.preventDefault(); e.stopPropagation(); onEdit() }}
className="opacity-0 group-hover:opacity-100 transition-opacity text-[#8b949e] hover:text-[#00d4ff] ml-0.5 cursor-pointer"
@@ -546,6 +546,64 @@ describe('DetailPanel', () => {
expect(payload.services.map((s: { service_name: string }) => s.service_name)).toEqual(['C', 'A', 'B'])
})
it('hides a service on this node without touching the service itself', () => {
// Visibility is the node's answer, not the device's: the rest of the
// service record goes back unchanged so other canvases keep drawing it.
const updateNode = vi.fn()
vi.mocked(canvasStore.useCanvasStore).mockImplementation(
((sel?: (s: Record<string, unknown>) => unknown) => {
const state = {
nodes: [makeNode({ services: [{ port: 3001, protocol: 'tcp' as const, service_name: 'Uptime Kuma' }] })],
selectedNodeId: 'n1',
selectedNodeIds: [],
setSelectedNode: vi.fn(),
deleteNode: vi.fn(),
updateNode,
snapshotHistory: vi.fn(),
createGroup: vi.fn(),
ungroup: vi.fn(),
setNodeSize: vi.fn(),
serviceStatuses: {},
}
return sel ? sel(state) : state
}) as unknown as typeof canvasStore.useCanvasStore,
)
render(<DetailPanel onEdit={vi.fn()} />)
fireEvent.click(screen.getByTitle('Hide on node'))
expect(updateNode.mock.calls[0][1].services).toEqual([
{ port: 3001, protocol: 'tcp', service_name: 'Uptime Kuma', visible: false },
])
})
it('shows a hidden service again', () => {
const updateNode = vi.fn()
vi.mocked(canvasStore.useCanvasStore).mockImplementation(
((sel?: (s: Record<string, unknown>) => unknown) => {
const state = {
nodes: [makeNode({
services: [{ port: 3001, protocol: 'tcp' as const, service_name: 'Uptime Kuma', visible: false }],
})],
selectedNodeId: 'n1',
selectedNodeIds: [],
setSelectedNode: vi.fn(),
deleteNode: vi.fn(),
updateNode,
snapshotHistory: vi.fn(),
createGroup: vi.fn(),
ungroup: vi.fn(),
setNodeSize: vi.fn(),
serviceStatuses: {},
}
return sel ? sel(state) : state
}) as unknown as typeof canvasStore.useCanvasStore,
)
render(<DetailPanel onEdit={vi.fn()} />)
fireEvent.click(screen.getByTitle('Show on node'))
expect(updateNode.mock.calls[0][1].services[0].visible).toBe(true)
})
it('is not draggable with a single service', () => {
setupStore({ services: [{ port: 80, protocol: 'tcp', service_name: 'A' }] })
render(<DetailPanel onEdit={vi.fn()} />)
@@ -96,3 +96,49 @@ describe('canvasStore — applyDeviceFacts', () => {
expect(changedFactFields(nodes[0].data, factsBaseline.n1)).toEqual(['label'])
})
})
describe('canvasStore — applyDeviceFacts keeps each node\'s arrangement', () => {
beforeEach(resetStore)
const ssh = { port: 22, protocol: 'tcp' as const, service_name: 'ssh' }
const kuma = { port: 3001, protocol: 'tcp' as const, service_name: 'Uptime Kuma' }
const drawnAs = (id: string, services: typeof ssh[]) =>
makeNode(id, makeNodeData({ device_id: 'd-1', label: 'NAS', services }))
it('does not unhide a service the node hid', () => {
useCanvasStore.getState().loadCanvas([drawnAs('n1', [ssh, { ...kuma, visible: false }])], [])
// The inventory sends the row's own copy, which carries no such flag.
useCanvasStore.getState().applyDeviceFacts('d-1', { services: [ssh, kuma] })
expect(useCanvasStore.getState().nodes[0].data.services).toEqual([
ssh,
{ ...kuma, visible: false },
])
})
it('keeps this node\'s order while taking the row\'s values', () => {
useCanvasStore.getState().loadCanvas([drawnAs('n1', [kuma, ssh])], [])
useCanvasStore.getState().applyDeviceFacts('d-1', {
services: [{ ...ssh, path: '/admin' }, kuma],
})
expect(useCanvasStore.getState().nodes[0].data.services).toEqual([
kuma,
{ ...ssh, path: '/admin' },
])
})
it('brings a service the row gained in hidden', () => {
useCanvasStore.getState().loadCanvas([drawnAs('n1', [ssh])], [])
useCanvasStore.getState().applyDeviceFacts('d-1', { services: [ssh, kuma] })
expect(useCanvasStore.getState().nodes[0].data.services).toEqual([
ssh,
{ ...kuma, visible: false },
])
})
it('drops one the row lost', () => {
useCanvasStore.getState().loadCanvas([drawnAs('n1', [ssh, kuma])], [])
useCanvasStore.getState().applyDeviceFacts('d-1', { services: [ssh] })
expect(useCanvasStore.getState().nodes[0].data.services).toEqual([ssh])
})
})
+16 -1
View File
@@ -15,7 +15,13 @@ import { normalizeHandle, removedHandleIds, handleCountField, sideDefault, handl
import { applyOpacity } from '@/utils/colorUtils'
import { readHideIp, writeHideIp } from '@/utils/ipDisplay'
import { CONTAINER_MODE_TYPES } from '@/utils/virtualEdgeParent'
import { changedFactFields, factsBaselineOf, factsBaselines, type FactsBaseline } from '@/utils/deviceFacts'
import {
changedFactFields,
factsBaselineOf,
factsBaselines,
listArrangedForNode,
type FactsBaseline,
} from '@/utils/deviceFacts'
type HistoryEntry = { nodes: Node<NodeData>[]; edges: Edge<EdgeData>[] }
type Clipboard = { nodes: Node<NodeData>[]; edges: Edge<EdgeData>[] }
@@ -1012,6 +1018,15 @@ export const useCanvasStore = create<CanvasState>((rawSet) => {
if (Object.keys(applied).length === 0) return n
changed = true
const data = { ...n.data, ...applied }
// The row owns the facts, this node owns how they are laid out: an
// inventory edit updates the values in place rather than replacing the
// arrangement, and a service the row just gained arrives hidden.
if (Array.isArray(applied.services)) {
data.services = listArrangedForNode(n.data.services, applied.services)
}
if (Array.isArray(applied.properties)) {
data.properties = listArrangedForNode(n.data.properties, applied.properties)
}
// Rebase only what was applied: an untouched pending edit stays reported
// as this canvas' change so the next save still writes it.
const rebased = factsBaselineOf(data)
+4
View File
@@ -97,6 +97,10 @@ export interface ServiceInfo {
* several domains. Same accepted shapes as a node `ip`/`hostname`
* (`host`, `host:port`, `https://host/…`). */
host?: string
/** Whether the node drawing this device shows the service. Per node, not per
* device — the same device on another canvas keeps its own answer. Absent
* means shown: the flag only appears once the service has been hidden. */
visible?: boolean
}
export type ServiceStatus = 'online' | 'offline' | 'unknown'
@@ -48,6 +48,35 @@ describe('changedFactFields', () => {
expect(changedFactFields(data({ properties: [] }), base)).toEqual(['properties'])
})
it('does not call hiding a service an edit to the device', () => {
const services = [
{ port: 22, protocol: 'tcp' as const, service_name: 'ssh' },
{ port: 3001, protocol: 'tcp' as const, service_name: 'Uptime Kuma' },
]
const base = factsBaselineOf(data({ services }))
const hidden = [services[0], { ...services[1], visible: false }]
// Visibility belongs to this node — pushing it as a device edit would hide
// the service on every other canvas drawing the same row.
expect(changedFactFields(data({ services: hidden }), base)).toEqual([])
})
it('does not call reordering an edit to the device', () => {
const props = [
{ key: 'Rack', value: 'A1', icon: null, visible: true },
{ key: 'Owner', value: 'me', icon: null, visible: true },
]
const base = factsBaselineOf(data({ properties: props }))
expect(changedFactFields(data({ properties: [props[1], props[0]] }), base)).toEqual([])
})
it('still reports a real edit to a list item', () => {
const props = [{ key: 'Rack', value: 'A1', icon: null, visible: true }]
const base = factsBaselineOf(data({ properties: props }))
expect(changedFactFields(data({ properties: [{ ...props[0], value: 'B2' }] }), base)).toEqual([
'properties',
])
})
it('treats every fact as changed with no baseline — a new node has nothing to revert', () => {
expect(changedFactFields(data(), undefined)).toEqual([...DEVICE_FACT_FIELDS])
})
@@ -0,0 +1,39 @@
import { describe, it, expect } from 'vitest'
import { parsePortSpec, isValidPortSpec, countPorts, FULL_PORT_RANGE } from '../portSpec'
describe('parsePortSpec', () => {
it('reads a single port, a range and a comma list', () => {
expect(parsePortSpec('80')).toEqual([[80, 80]])
expect(parsePortSpec('1-1024')).toEqual([[1, 1024]])
expect(parsePortSpec('443,80')).toEqual([[80, 80], [443, 443]])
})
it('merges overlapping and adjacent ranges', () => {
expect(parsePortSpec('1-100,50-200')).toEqual([[1, 200]])
expect(parsePortSpec('1-100,101-200')).toEqual([[1, 200]])
expect(parsePortSpec('1-100,300-400')).toEqual([[1, 100], [300, 400]])
})
it('tolerates whitespace around tokens', () => {
expect(parsePortSpec(' 80 , 443 ')).toEqual([[80, 80], [443, 443]])
})
it('rejects what nmap could not use', () => {
for (const bad of ['', ' ', '0', '65536', '100-50', '80,', 'http', '80-', '-80', '1-2-3']) {
expect(parsePortSpec(bad)).toBeNull()
}
})
})
describe('isValidPortSpec / countPorts', () => {
it('counts merged ranges once', () => {
expect(countPorts('1-100,50-200')).toBe(200)
expect(countPorts('80,443')).toBe(2)
expect(countPorts(FULL_PORT_RANGE)).toBe(65535)
})
it('counts an invalid spec as nothing', () => {
expect(isValidPortSpec('nope')).toBe(false)
expect(countPorts('nope')).toBe(0)
})
})
+58 -2
View File
@@ -40,9 +40,35 @@ export type FactsBaseline = Record<string, string>
const encode = (value: unknown): string => JSON.stringify(value ?? null)
/**
* Which services and properties the node carries, ignoring how it draws them.
*
* Order and visibility belong to the node, not to the inventory row, so hiding
* a service or dragging a property into place is not an edit to the device and
* must not be reported as one — otherwise a rearranged canvas would push its
* arrangement onto every other canvas showing the same device.
*/
const keyOf = (item: Record<string, unknown>): string =>
'key' in item
? String(item.key ?? '').toLowerCase()
: `${item.port ?? ''}|${item.protocol ?? ''}|${String(item.service_name ?? '').toLowerCase()}`
const encodeFacts = (field: DeviceFactField, value: unknown): string => {
if (field !== 'services' && field !== 'properties') return encode(value)
const items = (Array.isArray(value) ? value : []).filter(
(item): item is Record<string, unknown> => typeof item === 'object' && item !== null,
)
const bare = items.map((item) => {
const rest = { ...item }
delete rest.visible
return rest
})
return encode([...bare].sort((a, b) => keyOf(a).localeCompare(keyOf(b))))
}
export function factsBaselineOf(data: Partial<NodeData>): FactsBaseline {
const out: FactsBaseline = {}
for (const field of DEVICE_FACT_FIELDS) out[field] = encode(data[field])
for (const field of DEVICE_FACT_FIELDS) out[field] = encodeFacts(field, data[field])
return out
}
@@ -64,7 +90,37 @@ export function changedFactFields(
baseline: FactsBaseline | undefined,
): DeviceFactField[] {
if (!baseline) return [...DEVICE_FACT_FIELDS]
return DEVICE_FACT_FIELDS.filter((field) => encode(data[field]) !== baseline[field])
return DEVICE_FACT_FIELDS.filter((field) => encodeFacts(field, data[field]) !== baseline[field])
}
/**
* The row's list, arranged the way this node already draws it.
*
* The facts are the row's, the order and the visibility are the node's, so an
* edit made in the Device Inventory must not reshuffle a canvas or unhide what
* it hid. Same rule the backend applies on read: what the node already lists
* keeps its place and its flag, what the row gained since is appended hidden,
* and what the row lost disappears.
*/
export function listArrangedForNode<T extends Record<string, unknown>>(
current: T[] | undefined,
incoming: T[],
): T[] {
if (!current?.length) return incoming
const seen = new Set<string>()
const by = new Map(incoming.map((item) => [keyOf(item), item]))
const out: T[] = []
for (const item of current) {
const key = keyOf(item)
const fresh = by.get(key)
if (!fresh || seen.has(key)) continue
seen.add(key)
out.push('visible' in item ? { ...fresh, visible: item.visible } : (fresh as T))
}
for (const item of incoming) {
if (!seen.has(keyOf(item))) out.push({ ...item, visible: false })
}
return out
}
/**
+50
View File
@@ -0,0 +1,50 @@
/**
* nmap-style port specs — `80`, `1-1024`, or a comma list of both.
*
* Mirrors `_parse_port_spec` in `backend/app/services/scanner.py`: the deep-scan
* dialog validates here so a typo is caught before a request, and the backend
* validates again because it is the one handing the spec to nmap.
*/
export const FULL_PORT_RANGE = '1-65535'
const TOKEN_RE = /^\d{1,5}(-\d{1,5})?$/
/**
* Sorted, merged `[start, end]` ranges, or `null` when the spec is unusable.
* An empty spec is unusable too — a scan of nothing is never what was meant.
*/
export function parsePortSpec(spec: string): Array<[number, number]> | null {
const tokens = spec.split(',').map((t) => t.trim())
if (tokens.length === 0 || tokens.some((t) => t === '')) return null
const ranges: Array<[number, number]> = []
for (const token of tokens) {
if (!TOKEN_RE.test(token)) return null
const parts = token.split('-').map(Number)
const start = parts[0]
const end = parts[parts.length - 1]
if (start < 1 || end > 65535 || start > end) return null
ranges.push([start, end])
}
ranges.sort((a, b) => a[0] - b[0])
const merged: Array<[number, number]> = [ranges[0]]
for (const [start, end] of ranges.slice(1)) {
const last = merged[merged.length - 1]
if (start <= last[1] + 1) last[1] = Math.max(last[1], end)
else merged.push([start, end])
}
return merged
}
export function isValidPortSpec(spec: string): boolean {
return parsePortSpec(spec) !== null
}
/** How many ports a spec covers — 0 when it is invalid. */
export function countPorts(spec: string): number {
const ranges = parsePortSpec(spec)
if (!ranges) return 0
return ranges.reduce((sum, [start, end]) => sum + (end - start + 1), 0)
}