Compare commits

..
Author SHA1 Message Date
Pouzor 322600f1c9 fix(zigbee): stop canvas imports dying on a proxy read timeout
A Zigbee2MQTT networkmap on a 200+ device mesh takes minutes to build.
Two separate failures fell out of that:

- POST /zigbee/import held the HTTP request open for the whole MQTT
  round-trip, so any reverse proxy in front of the API cut it first
  (Cloudflare returns a 524 at 120 s) and the browser never saw the map.
  It now registers a job, fetches in the background and answers 202; the
  client polls GET /zigbee/import/{job_id} until the payload is ready.
  Job results are transient and live in memory with a 15 min TTL — the
  same single-worker assumption the scheduler already makes. A failed
  fetch replays the status the synchronous route used to raise, so a bad
  broker is still a 502 and a slow mesh still a 504.

- The networkmap wait was hard-coded at 300 s with no way to raise it.
  It now reads ZIGBEE_NETWORKMAP_TIMEOUT, and the shared MQTT round-trip
  used by the Z-Wave import reads MQTT_RESPONSE_TIMEOUT. Both default to
  300 s, fall back to that if misconfigured to a non-positive value, and
  name themselves in the timeout message.

Also corrects the route and doc claims that the wait was 60 s.

The /import tests changed with the contract they cover, not to pass.

Fixes #380

ha-relevant: yes
2026-08-31 11:26:45 +02:00
Pouzor 9c2a1ba0eb docs(canvas): correct a stale comment on where a zone's height is stored
The zone size moved to the nodes.width/height columns; the comment still
described the custom_colors stash it replaced.

ha-relevant: yes
2026-08-27 01:45:23 +02:00
Pouzor 851141951d fix(canvas): store a zone's size in the width/height columns
Every node type persisted its size in `nodes.width` / `nodes.height` except
`groupRect`, which stashed it inside the `custom_colors` JSON next to its
colours. The columns already existed and were simply unused for zones, so
this was an inconsistency rather than a missing-column workaround, and it
put geometry in a blob that otherwise holds style.

The serializer now writes the columns for a zone too, and strips the legacy
`width`/`height` keys out of the blob so the two cannot drift apart and
leave an older canvas reading a stale size.

No data is lost on upgrade:

- `_backfill_zone_size` copies the blob geometry into the columns at
  startup. It only fills a column that is still NULL, so it cannot overwrite
  a size set since; it parses the JSON in Python rather than with
  `json_extract`, so it does not depend on the SQLite build carrying JSON1;
  and an unreadable row is skipped without costing the others their size.
  Re-running it is a no-op.
- the reader still falls back to the blob, covering a payload the backfill
  has not reached — an older server, or an import.

Standalone mode is unaffected: it stores React Flow nodes verbatim, so the
size was always on `node.width` / `node.height` there.

The four serializer tests that pinned the size to the blob now assert the
columns, since that is the behaviour being changed.

ha-relevant: yes
2026-08-27 01:45:23 +02:00
Pouzor f9f88c8fb0 refactor(canvas): drop a dead custom_colors.height write on zone growth
Growing a zone during a subnet import wrote the new height twice: to
`node.height`, the live field, and to `data.custom_colors.height`, which
nothing reads. The blob copy is produced by the serializer at save time
(rebuilt from `node.height`, so the value written here was overwritten
before it reached the API) and consumed at load time off the API payload.
Writing it from the store was invisible, and misleading in a blob that
otherwise holds colours and style.

The test asserting the dead field is replaced by a save/load round-trip
through the real serializer, which is what actually protects the height.

ha-relevant: yes
2026-08-27 01:45:23 +02:00
Pouzor 2bf9f1bca8 fix(canvas): keep a parent ahead of its children after a subnet import
The subnet import only guaranteed the zone preceded the nodes it pulled in.
An arrival that is itself a parent — a Proxmox host with nested VMs, whose
children stay put because they already have a parent — could end up listed
after those children, which React Flow renders detached with a
parent-not-found error. Nothing re-sorts on load, so the broken order
survived a save.

Reorder the whole array instead: every node now follows its parent, which
covers both directions at once. An already-valid list comes back untouched
and a parent cycle terminates rather than recursing.

ha-relevant: yes
2026-08-27 01:45:23 +02:00
Pouzor d9451f35dc feat(canvas): import devices into a zone by subnet
Closes #325.

A zone gains an "Import devices by subnet" action: type a CIDR, and every
free device whose IP falls in that range moves into the zone, laid out on a
grid in its free space.

The CIDR is an argument to a one-shot action, never a property of the zone —
it is not submitted with the form, not persisted, and cleared after each run.
So there is no schema change, no migration, and it works in standalone mode.
The trade-off is that a device scanned later does not join the zone by
itself; the user re-runs the import.

Deliberate rules:
- additive — running it twice with two subnets leaves both sets inside, and
  a device whose IP stops matching is never ejected
- only unparented devices move. A node nested in a group, a container host
  or another zone keeps the parent the user gave it
- canvas furniture (groupRect / group / text) is skipped, and a zone never
  swallows another zone
- one history entry per import, so a single Undo reverses the whole thing

Edit mode gets an Import button; add mode has none, since the zone does not
exist yet and a button would look broken — there the CIDR is applied right
after the zone is created.

IPv4 only: the modal rejects an IPv6 CIDR with a message rather than
silently matching nothing.

ha-relevant: yes
2026-08-27 01:45:23 +02:00
Pouzor 05c0da53e6 fix(scan): reconcile scan runs orphaned by a backend restart
A scan runs on a background thread inside the API process. If that process
dies mid-scan — an OOM kill, docker stop, a crash — the ScanRun row stays
"running" for ever, because nothing is left alive to finish it.

That row is not just cosmetic clutter in Scan History: the trigger endpoints
reject a new scan while one is "running" for the same target, so a single kill
locks that range out permanently.

Nothing can legitimately be "running" the moment we boot, so lifespan() now
marks every such row "error" — the same word run_scan and run_device_scan
write when they fail themselves — with finished_at and an explanatory message.

Reported alongside the OOM itself in #374, which is what produced the orphans.

Fixes #374

ha-relevant: maybe
2026-08-27 00:00:32 +02:00
Pouzor c29680a75f fix(status): stop an endless response body from OOM-killing the backend
Both HTTP paths buffered the whole response body before looking at it. An
endpoint that streams without end and sends no Content-Length — a Freebox
bandwidth-test port, an MJPEG camera, a log tail — grew the backend until the
cgroup limit killed it, every 5-10 minutes at a constant ~1.04 GB RSS.

httpx's timeout does not help: it applies per network operation, not to the
total time spent draining a socket that keeps delivering data.

- status_checker._http_get only needs the status line, so it now uses
  client.stream() and never reads the body at all.
- http_probe._probe_scheme needs at most _MAX_BODY_BYTES to hunt for <title>,
  so it streams and stops there. The cap existed already but was applied to
  resp.text, after the full body had been downloaded.

Regression tests serve an endless, Content-Length-less body through a
MockTransport and assert the read stays bounded. Both fail on the old code.

The existing mocks patched httpx.AsyncClient.get, which neither path calls
now; they are rebuilt on MockTransport — a real client over a fake network —
with every original assertion kept.

Fixes #375

ha-relevant: yes
2026-08-26 23:43:50 +02:00
Pouzor - Rémy JardientandGitHub eb7e74d36b Update README.md 2026-08-25 23:08:03 +02:00
37 changed files with 2366 additions and 169 deletions
+7
View File
@@ -89,6 +89,10 @@ MCP_SERVICE_KEY=svc_changeme
# ZIGBEE_BASE_TOPIC=zigbee2mqtt
# ZIGBEE_MQTT_TLS=false # true for TLS brokers (typically port 8883)
# ZIGBEE_MQTT_TLS_INSECURE=false # skip cert verify (self-signed only; requires TLS)
# Seconds to wait for the Z2M bridge to answer a networkmap request. Applies to
# manual imports too. Raise it on a large mesh (200+ devices can take minutes)
# if an import fails with "Timed out waiting for networkmap response".
# ZIGBEE_NETWORKMAP_TIMEOUT=300
# Z-Wave JS UI (zwavejs2mqtt) auto-sync — same MQTT secret/env rules as Zigbee.
# ZWAVE_MQTT_HOST=192.168.1.20
@@ -99,3 +103,6 @@ MCP_SERVICE_KEY=svc_changeme
# ZWAVE_GATEWAY_NAME=zwavejs2mqtt
# ZWAVE_MQTT_TLS=false # true for TLS brokers (typically port 8883)
# ZWAVE_MQTT_TLS_INSECURE=false # skip cert verify (self-signed only; requires TLS)
# Seconds to wait for the gateway to answer an MQTT request. Same rationale as
# ZIGBEE_NETWORKMAP_TIMEOUT above.
# MQTT_RESPONSE_TIMEOUT=300
+2 -2
View File
@@ -48,8 +48,8 @@ If you are running <img width="22" height="22" align="top" alt="New_Home_Assist
<p align="center">
<img src="docs/homelable1.png" alt="Homelable canvas overview" width="100%" />
<img alt="Homelable Device inventory" src="https://github.com/user-attachments/assets/f3903ac8-354d-4873-81ba-1914971890ed" />
<img width="49.5%" alt="Homelable Custom node" src="https://github.com/user-attachments/assets/813725b1-376b-4bad-bb1f-0985f3bc7546" />
<img width="49.5%" alt="Homelable Zigbee Network" src="https://github.com/user-attachments/assets/35e18d11-8363-498d-ae3d-642685cac76d" />
<img alt="Rack display" src="https://github.com/user-attachments/assets/43273605-4f46-4163-8aea-8bf8a76a3f76" />
</p>
+70 -17
View File
@@ -19,6 +19,8 @@ from app.schemas.zigbee import (
ZigbeeConfig,
ZigbeeCoordinatorOut,
ZigbeeEdgeOut,
ZigbeeImportJob,
ZigbeeImportJobResult,
ZigbeeImportPendingResponse,
ZigbeeImportRequest,
ZigbeeImportResponse,
@@ -27,6 +29,7 @@ from app.schemas.zigbee import (
ZigbeeTestConnectionRequest,
ZigbeeTestConnectionResponse,
)
from app.services.import_jobs import create_job, fail_job, finish_job, get_job
from app.services.node_dedupe import dedupe_nodes_by_device
from app.services.zigbee_service import (
build_zigbee_properties,
@@ -48,18 +51,70 @@ async def _is_drawn(db: AsyncSession, device_id: str) -> bool:
router = APIRouter()
@router.post("/import", response_model=ZigbeeImportResponse)
def _import_error(exc: BaseException) -> tuple[int, str]:
"""Map a fetch_networkmap failure to the (status, detail) the API reports."""
if isinstance(exc, ImportError):
return 500, str(exc)
if isinstance(exc, ConnectionError):
return 502, str(exc)
if isinstance(exc, TimeoutError):
return 504, str(exc)
if isinstance(exc, ValueError):
return 422, str(exc)
logger.exception("Unexpected error during Zigbee import", exc_info=exc)
return 500, "Unexpected error during Zigbee import"
@router.post("/import", response_model=ZigbeeImportJob, status_code=202)
async def import_zigbee_network(
payload: ZigbeeImportRequest,
background_tasks: BackgroundTasks,
_: str = Depends(get_current_user),
) -> ZigbeeImportResponse:
"""Fetch the Zigbee2MQTT network map and return nodes + edges ready for canvas drop.
) -> ZigbeeImportJob:
"""Start a canvas import and return a job id to poll.
Connects to the specified MQTT broker, publishes a networkmap request to
``<base_topic>/bridge/request/networkmap``, and waits up to 60 s for the
response (large meshes can take 30 s+). The devices are returned as typed homelable nodes with a
coordinator → router → end-device hierarchy.
``<base_topic>/bridge/request/networkmap`` and waits up to
``ZIGBEE_NETWORKMAP_TIMEOUT`` seconds (default 300) for the response.
The fetch runs in the background and the result is collected from
``GET /zigbee/import/{job_id}``: a 200+ device mesh takes minutes to answer,
which outlives the read timeout of any reverse proxy sitting in front of the
API. Polling keeps each request short.
"""
job = create_job()
background_tasks.add_task(_background_canvas_import, job.id, payload)
return ZigbeeImportJob(job_id=job.id, status=job.status)
@router.get("/import/{job_id}", response_model=ZigbeeImportJobResult)
async def get_zigbee_import_job(
job_id: str,
_: str = Depends(get_current_user),
) -> ZigbeeImportJobResult:
"""Poll a canvas import started by ``POST /zigbee/import``.
While running, returns ``status="running"`` with no payload. On success the
nodes and edges are carried in ``result``. A failed fetch is reported as the
status code the synchronous route used to raise, so the client keeps
distinguishing a bad broker (502) from a slow mesh (504).
"""
job = get_job(job_id)
if job is None:
raise HTTPException(status_code=404, detail="Import job not found or expired")
if job.status == "error":
raise HTTPException(
status_code=job.error_status or 500,
detail=job.error or "Zigbee import failed",
)
result = None
if job.status == "done" and job.result is not None:
result = ZigbeeImportResponse(**job.result)
return ZigbeeImportJobResult(job_id=job.id, status=job.status, result=result)
async def _background_canvas_import(job_id: str, payload: ZigbeeImportRequest) -> None:
"""Fetch the network map and park the canvas-ready payload on the job."""
try:
nodes_raw, edges_raw = await fetch_networkmap(
mqtt_host=payload.mqtt_host,
@@ -70,21 +125,19 @@ async def import_zigbee_network(
tls=payload.mqtt_tls,
tls_insecure=payload.mqtt_tls_insecure,
)
except ImportError as exc:
raise HTTPException(status_code=500, detail=str(exc)) from exc
except ConnectionError as exc:
raise HTTPException(status_code=502, detail=str(exc)) from exc
except TimeoutError as exc:
raise HTTPException(status_code=504, detail=str(exc)) from exc
except ValueError as exc:
raise HTTPException(status_code=422, detail=str(exc)) from exc
except Exception as exc:
logger.exception("Unexpected error during Zigbee import")
raise HTTPException(status_code=500, detail="Unexpected error during Zigbee import") from exc
status, detail = _import_error(exc)
fail_job(job_id, detail, status)
return
nodes = [ZigbeeNodeOut(**n) for n in nodes_raw]
edges = [ZigbeeEdgeOut(**e) for e in edges_raw]
return ZigbeeImportResponse(nodes=nodes, edges=edges, device_count=len(nodes))
finish_job(
job_id,
ZigbeeImportResponse(
nodes=nodes, edges=edges, device_count=len(nodes)
).model_dump(),
)
@router.post("/import-pending", response_model=ScanRunResponse)
+8
View File
@@ -179,6 +179,10 @@ class Settings(BaseSettings):
zigbee_mqtt_tls_insecure: bool = False
zigbee_sync_enabled: bool = False
zigbee_sync_interval: int = 3600 # seconds (floor 300 enforced on write)
# Seconds to wait for the Z2M bridge to answer a networkmap request. A mesh
# of 200+ devices can take several minutes to build the map, so raise this
# if imports fail with "Timed out waiting for networkmap response".
zigbee_networkmap_timeout: int = 300
# Z-Wave JS UI (zwavejs2mqtt) auto-sync import. Same secret/env rules.
zwave_mqtt_host: str = ""
@@ -192,6 +196,10 @@ class Settings(BaseSettings):
zwave_sync_enabled: bool = False
zwave_sync_interval: int = 3600 # seconds (floor 300 enforced on write)
# Seconds to wait for any MQTT gateway request/response round-trip
# (the Z-Wave node dump today). Same rationale as the Zigbee twin above.
mqtt_response_timeout: int = 300
def _override_path(self) -> Path:
return Path(self.sqlite_path).parent / "scan_config.json"
+61
View File
@@ -494,6 +494,7 @@ async def init_db() -> None:
await _backfill_node_devices()
await _drop_legacy_node_columns()
await _seed_node_views()
await _backfill_zone_size()
@@ -752,6 +753,66 @@ async def _seed_node_views() -> None:
logger.warning("Seeding the node service/property views failed: %s", exc)
async def _backfill_zone_size() -> None:
"""Move a zone's size out of the custom_colors blob into the real columns.
Every node type stored its size in `nodes.width` / `nodes.height` except
`groupRect`, which stashed it inside the `custom_colors` JSON alongside its
colours. The serializer writes the columns for zones too now, so a canvas
saved before this upgrade would come back at the default 360x240 without
this backfill.
Only fills a column that is still NULL, so it cannot overwrite a size the
user has set since, and re-running it is a no-op. Parsed in Python rather
than with `json_extract`, so it does not depend on the SQLite build being
compiled with JSON1.
Never fatal: the reader falls back to the blob, so the worst case of a
failure here is that the geometry keeps coming from where it always did.
"""
try:
async with engine.begin() as conn:
rows = (
await conn.exec_driver_sql(
"SELECT id, custom_colors FROM nodes "
"WHERE type = 'groupRect' AND custom_colors IS NOT NULL "
"AND (width IS NULL OR height IS NULL)"
)
).fetchall()
moved = 0
for node_id, blob in rows:
if isinstance(blob, str):
try:
blob = _json.loads(blob)
except ValueError:
continue
if not isinstance(blob, dict):
continue
width, height = blob.get("width"), blob.get("height")
# A bool is an int in Python; a size that is not a real number
# is left alone rather than written as garbage.
if not isinstance(width, int | float) or isinstance(width, bool):
width = None
if not isinstance(height, int | float) or isinstance(height, bool):
height = None
if width is None and height is None:
continue
await conn.exec_driver_sql(
"UPDATE nodes SET width = COALESCE(width, ?), height = COALESCE(height, ?) "
"WHERE id = ?",
(width, height, node_id),
)
moved += 1
if moved:
logger.info("Moved the size of %d zone(s) to the width/height columns", moved)
except Exception as exc: # pragma: no cover - defensive, boot must not die
logger.warning("Backfilling zone width/height failed: %s", exc)
async def get_db() -> AsyncGenerator[AsyncSession, None]:
async with AsyncSessionLocal() as session:
yield session
+6 -1
View File
@@ -28,7 +28,8 @@ from app.api.routes import settings as settings_routes
from app.core.config import settings
from app.core.scheduler import start_scheduler, stop_scheduler
from app.core.security import OIDCCSRFMiddleware
from app.db.database import init_db
from app.db.database import AsyncSessionLocal, init_db
from app.services.scanner import reconcile_orphan_runs
@asynccontextmanager
@@ -45,6 +46,10 @@ async def lifespan(app: FastAPI) -> AsyncGenerator[None, None]:
logging.getLogger("app.services.scanner").setLevel(logging.INFO)
await init_db()
settings.load_overrides()
# A scan runs on a background thread in this process, so anything still
# flagged "running" now was orphaned by a previous one that died mid-scan.
async with AsyncSessionLocal() as db:
await reconcile_orphan_runs(db)
start_scheduler()
yield
stop_scheduler()
+15
View File
@@ -73,6 +73,21 @@ class ZigbeeImportResponse(BaseModel):
device_count: int
class ZigbeeImportJob(BaseModel):
"""Handle returned when a canvas import is queued."""
job_id: str
status: str # running | done | error
class ZigbeeImportJobResult(BaseModel):
"""Poll response for a canvas import. ``result`` is set once done."""
job_id: str
status: str # running | done
result: ZigbeeImportResponse | None = None
class ZigbeeTestConnectionResponse(BaseModel):
connected: bool
message: str
+25 -4
View File
@@ -16,7 +16,11 @@ logger = logging.getLogger(__name__)
# Headers that commonly carry the application name.
_SIGNAL_HEADERS = ("Server", "X-Powered-By")
# Cap how much body we read when hunting for <title> — avoids large downloads.
# Cap how much body we read when hunting for <title>. This is a cap on the
# download itself, not just on what we scan: the body is streamed and the
# connection dropped once we hold this much. Some endpoints stream without end
# and send no Content-Length (bandwidth-test endpoints, MJPEG cameras), so
# buffering a full response would OOM the backend.
_MAX_BODY_BYTES = 64 * 1024
_TITLE_RE = re.compile(r"<title[^>]*>(.*?)</title>", re.IGNORECASE | re.DOTALL)
_PROBE_TIMEOUT = 3.0
@@ -24,6 +28,23 @@ _PROBE_TIMEOUT = 3.0
_NON_HTTP_PORTS = frozenset({22, 21, 23, 25, 53, 110, 143, 161, 162, 179, 445, 3306, 5432, 6379})
async def _read_capped(resp: httpx.Response) -> str:
"""Read at most _MAX_BODY_BYTES of a streaming response, then stop."""
chunks: list[bytes] = []
size = 0
async for chunk in resp.aiter_bytes():
chunks.append(chunk)
size += len(chunk)
if size >= _MAX_BODY_BYTES:
break
raw = b"".join(chunks)[:_MAX_BODY_BYTES]
encoding = resp.charset_encoding or "utf-8"
try:
return raw.decode(encoding, errors="replace")
except LookupError:
return raw.decode("utf-8", errors="replace")
def _extract_title(body: str) -> str | None:
m = _TITLE_RE.search(body)
if not m:
@@ -34,11 +55,11 @@ def _extract_title(body: str) -> str | None:
async def _probe_scheme(client: httpx.AsyncClient, url: str) -> dict[str, Any] | None:
try:
resp = await client.get(url, follow_redirects=True)
async with client.stream("GET", url, follow_redirects=True) as resp:
headers = {h: resp.headers[h] for h in _SIGNAL_HEADERS if h in resp.headers}
body = await _read_capped(resp)
except (httpx.HTTPError, OSError):
return None
headers = {h: resp.headers[h] for h in _SIGNAL_HEADERS if h in resp.headers}
body = resp.text[:_MAX_BODY_BYTES] if resp.text else ""
title = _extract_title(body)
if not title and not headers:
return None
+92
View File
@@ -0,0 +1,92 @@
"""In-process registry for long-running import jobs that return a payload.
Pending imports already run in the background and report through ``scan_runs``.
Canvas imports are different: the caller wants the fetched map *back* so it can
drop it on the canvas, which kept the request open for the whole MQTT
round-trip. On a large mesh that outlives any reverse proxy's read timeout
(Cloudflare cuts at 100 s and returns a 524), so the browser never sees the
result even though the fetch succeeded server-side.
The fix is to hand the client a job id immediately and let it poll. Results are
transient — a canvas import is meaningless once the user has closed the modal —
so they live in memory rather than the DB, and expire on a TTL. The backend runs
a single uvicorn worker, the same assumption the scheduler and ``BackgroundTasks``
already make.
"""
from __future__ import annotations
import time
import uuid
from dataclasses import dataclass, field
from typing import Any, Literal
JobStatus = Literal["running", "done", "error"]
# How long a finished job stays readable. Long enough for a client that polls
# slowly or reloads mid-import, short enough that a forgotten map is not held
# for the life of the process.
JOB_TTL_SECONDS = 900.0
@dataclass
class ImportJob:
id: str
status: JobStatus = "running"
result: dict[str, Any] | None = None
error: str | None = None
# HTTP status the synchronous route would have raised, so the client can
# keep reacting to failures the way it always has.
error_status: int | None = None
created_at: float = field(default_factory=time.monotonic)
finished_at: float | None = None
_jobs: dict[str, ImportJob] = {}
def _purge_expired(now: float) -> None:
for job_id, job in list(_jobs.items()):
if job.finished_at is not None and now - job.finished_at > JOB_TTL_SECONDS:
del _jobs[job_id]
def create_job() -> ImportJob:
"""Register a new running job and return it."""
now = time.monotonic()
_purge_expired(now)
job = ImportJob(id=str(uuid.uuid4()))
_jobs[job.id] = job
return job
def get_job(job_id: str) -> ImportJob | None:
"""Return a job, or None if it never existed or has expired."""
_purge_expired(time.monotonic())
return _jobs.get(job_id)
def finish_job(job_id: str, result: dict[str, Any]) -> None:
"""Mark a job done and attach its payload. No-op if it expired."""
job = _jobs.get(job_id)
if job is None:
return
job.status = "done"
job.result = result
job.finished_at = time.monotonic()
def fail_job(job_id: str, error: str, status: int) -> None:
"""Mark a job failed with a client-safe message. No-op if it expired."""
job = _jobs.get(job_id)
if job is None:
return
job.status = "error"
job.error = error
job.error_status = status
job.finished_at = time.monotonic()
def reset_jobs() -> None:
"""Drop every job. For tests."""
_jobs.clear()
+23 -4
View File
@@ -13,6 +13,8 @@ import logging
import ssl
from typing import Any
from app.core.config import settings
logger = logging.getLogger(__name__)
try:
@@ -21,7 +23,10 @@ except ImportError: # pragma: no cover
aiomqtt = None # type: ignore[assignment]
_CONNECTION_TIMEOUT = 5.0 # seconds to verify broker reachability
_RESPONSE_TIMEOUT = 300.0 # seconds to wait for a gateway response (large meshes are slow)
# Fallback wait for a gateway response. The effective value comes from
# ``settings.mqtt_response_timeout`` (env MQTT_RESPONSE_TIMEOUT), read at call
# time so an operator with a large mesh can raise it without a code change.
_RESPONSE_TIMEOUT = 300.0
def _sanitize_mqtt_error(exc: BaseException) -> str:
@@ -71,11 +76,14 @@ async def request_response(
password: str | None = None,
tls: bool = False,
tls_insecure: bool = False,
response_timeout: float = _RESPONSE_TIMEOUT,
response_timeout: float | None = None,
) -> dict[str, Any]:
"""Publish ``request_payload`` to ``request_topic`` and return the first
JSON message received on ``response_topic`` as a dict.
``response_timeout`` defaults to ``settings.mqtt_response_timeout``
(env ``MQTT_RESPONSE_TIMEOUT``, 300 s) — a large mesh can take minutes.
Raises:
ImportError: if aiomqtt is not installed.
TimeoutError: if no response arrives in time.
@@ -88,6 +96,14 @@ async def request_response(
"Install it with: pip install aiomqtt"
)
timeout = (
float(settings.mqtt_response_timeout)
if response_timeout is None
else response_timeout
)
if timeout <= 0:
timeout = _RESPONSE_TIMEOUT
response_payload: dict[str, Any] = {}
tls_context = _build_tls_context(tls_insecure) if tls else None
@@ -122,12 +138,15 @@ async def request_response(
raise ValueError(f"Malformed MQTT response: {exc}") from exc
return
await asyncio.wait_for(_wait_for_response(), timeout=response_timeout)
await asyncio.wait_for(_wait_for_response(), timeout=timeout)
except aiomqtt.MqttError as exc:
raise ConnectionError(_sanitize_mqtt_error(exc)) from exc
except asyncio.TimeoutError as exc:
raise TimeoutError("Timed out waiting for MQTT response") from exc
raise TimeoutError(
f"Timed out waiting for MQTT response after {timeout:g}s — "
"raise MQTT_RESPONSE_TIMEOUT if your mesh is large"
) from exc
if not response_payload:
raise ValueError("Empty MQTT response received")
+30
View File
@@ -947,3 +947,33 @@ async def run_device_scan(
finally:
with _cancelled_lock:
_cancelled_runs.discard(run_id)
async def reconcile_orphan_runs(db: AsyncSession) -> int:
"""
Mark every scan run still flagged `running` at startup as errored.
Scans live on a background thread inside this process, so nothing can
legitimately be `running` the moment we boot: any such row belongs to a
previous process that died mid-scan (an OOM kill, `docker stop`, a crash).
Left alone the row is immortal, and it also locks the target out — the
trigger endpoints reject a new scan while one is `running` for the same
range, so a single kill blocks re-scanning for good.
Returns the number of rows reconciled.
"""
orphans = (
await db.execute(select(ScanRun).where(ScanRun.status == "running"))
).scalars().all()
if not orphans:
return 0
now = datetime.now(timezone.utc)
for run in orphans:
# "error" is the word run_scan and run_device_scan already write when
# they fail; Scan History filters and colours that one.
run.status = "error"
run.error = "Interrupted: the backend restarted while this scan was running"
run.finished_at = now
await db.commit()
logger.warning("Reconciled %d scan run(s) left running by a previous process", len(orphans))
return len(orphans)
+10 -2
View File
@@ -103,8 +103,16 @@ async def _ping(host: str) -> bool:
async def _http_get(url: str, verify: bool = False) -> bool:
async with httpx.AsyncClient(verify=verify, timeout=5) as client:
resp = await client.get(url, follow_redirects=True)
# Only the status line matters here. A plain .get() buffers the whole body
# first, and some endpoints stream without end (bandwidth-test endpoints,
# MJPEG cameras, log tails) — enough to OOM the backend. timeout=5 does not
# save us: httpx applies it per network operation, not to the total time
# spent draining a socket that keeps delivering data. stream() closes the
# connection on exit without draining it.
async with (
httpx.AsyncClient(verify=verify, timeout=5) as client,
client.stream("GET", url, follow_redirects=True) as resp,
):
return resp.status_code < 500
+23 -3
View File
@@ -7,6 +7,7 @@ import json
import logging
from typing import Any
from app.core.config import settings
from app.services.mqtt_common import _build_tls_context, _sanitize_mqtt_error
logger = logging.getLogger(__name__)
@@ -19,7 +20,10 @@ except ImportError: # pragma: no cover
_NETWORKMAP_REQUEST_TOPIC = "{base_topic}/bridge/request/networkmap"
_NETWORKMAP_RESPONSE_TOPIC = "{base_topic}/bridge/response/networkmap"
_CONNECTION_TIMEOUT = 5.0 # seconds to verify broker reachability
_NETWORKMAP_TIMEOUT = 300.0 # seconds to wait for the networkmap response (large meshes can be slow)
# Fallback wait for the networkmap response. The effective value comes from
# ``settings.zigbee_networkmap_timeout`` (env ZIGBEE_NETWORKMAP_TIMEOUT), read at
# call time so an operator with a large mesh can raise it without a code change.
_NETWORKMAP_TIMEOUT = 300.0
# Re-exported for backwards compatibility — these now live in mqtt_common.
__all__ = ["_build_tls_context", "_sanitize_mqtt_error"]
@@ -238,9 +242,14 @@ async def fetch_networkmap(
password: str | None = None,
tls: bool = False,
tls_insecure: bool = False,
response_timeout: float | None = None,
) -> tuple[list[dict[str, Any]], list[dict[str, Any]]]:
"""Connect to the MQTT broker, request the Z2M networkmap, and return (nodes, edges).
``response_timeout`` defaults to ``settings.zigbee_networkmap_timeout``
(env ``ZIGBEE_NETWORKMAP_TIMEOUT``, 300 s) — a 200+ device mesh can take
minutes to answer.
Raises:
TimeoutError: if the broker does not respond in time.
ConnectionError: if the broker cannot be reached.
@@ -252,6 +261,14 @@ async def fetch_networkmap(
"Install it with: pip install aiomqtt"
)
timeout = (
float(settings.zigbee_networkmap_timeout)
if response_timeout is None
else response_timeout
)
if timeout <= 0:
timeout = _NETWORKMAP_TIMEOUT
request_topic = _NETWORKMAP_REQUEST_TOPIC.format(base_topic=base_topic)
response_topic = _NETWORKMAP_RESPONSE_TOPIC.format(base_topic=base_topic)
@@ -295,12 +312,15 @@ async def fetch_networkmap(
) from exc
return
await asyncio.wait_for(_wait_for_response(), timeout=_NETWORKMAP_TIMEOUT)
await asyncio.wait_for(_wait_for_response(), timeout=timeout)
except aiomqtt.MqttError as exc:
raise ConnectionError(_sanitize_mqtt_error(exc)) from exc
except asyncio.TimeoutError as exc:
raise TimeoutError("Timed out waiting for networkmap response") from exc
raise TimeoutError(
f"Timed out waiting for networkmap response after {timeout:g}s — "
"raise ZIGBEE_NETWORKMAP_TIMEOUT if your mesh is large"
) from exc
if not response_payload:
raise ValueError("Empty networkmap response received")
+104 -1
View File
@@ -8,7 +8,12 @@ from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.db.models import InventoryDevice, Node, ScanRun
from app.services.scanner import _cancelled_runs, request_cancel, run_scan
from app.services.scanner import (
_cancelled_runs,
reconcile_orphan_runs,
request_cancel,
run_scan,
)
@pytest.mark.asyncio
@@ -468,3 +473,101 @@ async def test_background_device_scan_marks_run_errored_on_exception(mem_db):
refreshed = await session.get(ScanRun, run_id)
assert refreshed is not None
assert refreshed.status == "error"
# --- reconcile_orphan_runs (issue #374) --------------------------------------
@pytest.mark.asyncio
async def test_reconcile_orphan_runs_marks_running_as_error(mem_db):
"""A run left "running" by a process that died mid-scan is immortal, and it
also locks its range out of future scans. Startup must reconcile it."""
async with mem_db() as session:
orphan = ScanRun(status="running", ranges=["10.0.0.0/24"])
session.add(orphan)
await session.commit()
orphan_id = orphan.id
assert await reconcile_orphan_runs(session) == 1
reconciled = await session.get(ScanRun, orphan_id)
# "error", not "failed" — one condition, one name, and the only status
# Scan History filters and colours.
assert reconciled.status == "error"
assert reconciled.finished_at is not None
assert "restarted" in reconciled.error
@pytest.mark.asyncio
async def test_reconcile_orphan_runs_leaves_finished_runs_alone(mem_db):
"""Only "running" is orphaned. Terminal rows must not be rewritten."""
async with mem_db() as session:
done = ScanRun(status="done", ranges=["10.0.0.0/24"], devices_found=7)
cancelled = ScanRun(status="cancelled", ranges=["10.0.1.0/24"])
errored = ScanRun(status="error", ranges=["10.0.2.0/24"], error="nmap missing")
session.add_all([done, cancelled, errored])
await session.commit()
ids = (done.id, cancelled.id, errored.id)
assert await reconcile_orphan_runs(session) == 0
for run_id, expected in zip(ids, ("done", "cancelled", "error"), strict=True):
assert (await session.get(ScanRun, run_id)).status == expected
# The pre-existing error message must survive untouched.
assert (await session.get(ScanRun, ids[2])).error == "nmap missing"
@pytest.mark.asyncio
async def test_reconcile_orphan_runs_handles_several_and_an_empty_table(mem_db):
async with mem_db() as session:
# Nothing to do on a fresh database.
assert await reconcile_orphan_runs(session) == 0
session.add_all([
ScanRun(status="running", ranges=["10.0.0.0/24"]),
ScanRun(status="running", kind="device", ranges=["10.0.0.5"]),
ScanRun(status="done", ranges=["10.0.1.0/24"]),
])
await session.commit()
assert await reconcile_orphan_runs(session) == 2
remaining = (
await session.execute(select(ScanRun).where(ScanRun.status == "running"))
).scalars().all()
assert remaining == []
@pytest.mark.asyncio
async def test_reconcile_orphan_runs_unblocks_a_new_scan(mem_db):
"""The point of the reconcile: the trigger endpoints reject a scan while one
is "running" for the same target, so an orphan blocks that range for good."""
async with mem_db() as session:
session.add(ScanRun(status="running", kind="device", ranges=["10.0.0.5"]))
await session.commit()
await reconcile_orphan_runs(session)
blocking = (
await session.execute(
select(ScanRun).where(
ScanRun.status == "running", ScanRun.kind == "device"
)
)
).scalars().all()
assert not any("10.0.0.5" in (r.ranges or []) for r in blocking)
@pytest.mark.asyncio
async def test_lifespan_reconciles_orphan_runs_at_startup():
"""The reconcile is worthless unless startup actually calls it."""
from app.main import app as fastapi_app
from app.main import lifespan
with patch("app.main.init_db", new=AsyncMock()), \
patch("app.main.start_scheduler"), \
patch("app.main.stop_scheduler"), \
patch("app.main.reconcile_orphan_runs", new=AsyncMock()) as reconcile:
async with lifespan(fastapi_app):
pass
reconcile.assert_awaited_once()
+131 -23
View File
@@ -1,10 +1,11 @@
"""Tests for the HTTP probe used by deep-scan service identification."""
from unittest.mock import AsyncMock, patch
from unittest.mock import patch
import httpx
import pytest
from app.services.http_probe import (
_MAX_BODY_BYTES,
_extract_title,
probe_open_ports,
probe_port,
@@ -15,6 +16,39 @@ def _response(text: str = "", headers: dict | None = None, status: int = 200) ->
return httpx.Response(status_code=status, text=text, headers=headers or {})
class _TransportClient:
"""
Patch target for httpx.AsyncClient that routes through a MockTransport.
Real client, fake network: the probe exercises the genuine httpx request
path (including .stream() and aiter_bytes()) instead of a mock that would
happily accept any call shape. `requests` records what was actually sent.
"""
def __init__(self, handler):
self._handler = handler
# Bound before patching, so building the real client here does not
# recurse back into this stand-in.
self._real = httpx.AsyncClient
self.kwargs: list[dict] = []
self.requests: list[httpx.Request] = []
def _record(self, request: httpx.Request) -> httpx.Response:
self.requests.append(request)
return self._handler(request)
def __call__(self, **kwargs):
self.kwargs.append(dict(kwargs))
kwargs.pop("verify", None)
return self._real(transport=httpx.MockTransport(self._record), **kwargs)
def _patch_client(handler) -> tuple:
"""Return (context manager, factory) patching http_probe's AsyncClient."""
factory = _TransportClient(handler)
return patch("app.services.http_probe.httpx.AsyncClient", factory), factory
# ── _extract_title ──────────────────────────────────────────────────────────
def test_extract_title_basic():
@@ -37,15 +71,18 @@ def test_extract_title_case_insensitive():
@pytest.mark.asyncio
async def test_probe_port_reads_title():
with patch("httpx.AsyncClient.get", new=AsyncMock(return_value=_response("<title>Jellyfin</title>"))):
ctx, _ = _patch_client(lambda req: _response("<title>Jellyfin</title>"))
with ctx:
result = await probe_port("10.0.0.5", 8096)
assert result == {"title": "Jellyfin", "headers": {}}
@pytest.mark.asyncio
async def test_probe_port_reads_headers():
resp = _response("", headers={"Server": "nginx", "X-Powered-By": "Express"})
with patch("httpx.AsyncClient.get", new=AsyncMock(return_value=resp)):
ctx, _ = _patch_client(
lambda req: _response("", headers={"Server": "nginx", "X-Powered-By": "Express"})
)
with ctx:
result = await probe_port("10.0.0.5", 3000)
assert result["headers"] == {"Server": "nginx", "X-Powered-By": "Express"}
@@ -53,30 +90,33 @@ async def test_probe_port_reads_headers():
@pytest.mark.asyncio
async def test_probe_port_falls_back_to_http():
# https raises, http succeeds
calls = {"n": 0}
async def fake_get(self, url, **kw):
calls["n"] += 1
if url.startswith("https"):
def handler(request):
if str(request.url).startswith("https"):
raise httpx.ConnectError("tls fail")
return _response("<title>HTTP App</title>")
with patch("httpx.AsyncClient.get", new=fake_get):
ctx, factory = _patch_client(handler)
with ctx:
result = await probe_port("10.0.0.5", 8080)
assert result["title"] == "HTTP App"
assert calls["n"] == 2 # tried https then http
assert len(factory.requests) == 2 # tried https then http
@pytest.mark.asyncio
async def test_probe_port_no_signal_returns_none():
with patch("httpx.AsyncClient.get", new=AsyncMock(return_value=_response(""))):
ctx, _ = _patch_client(lambda req: _response(""))
with ctx:
result = await probe_port("10.0.0.5", 8080)
assert result is None
@pytest.mark.asyncio
async def test_probe_port_timeout_returns_none():
with patch("httpx.AsyncClient.get", new=AsyncMock(side_effect=httpx.TimeoutException("slow"))):
def handler(request):
raise httpx.TimeoutException("slow")
ctx, _ = _patch_client(handler)
with ctx:
result = await probe_port("10.0.0.5", 8080)
assert result is None
@@ -84,33 +124,101 @@ async def test_probe_port_timeout_returns_none():
@pytest.mark.asyncio
async def test_probe_port_skips_non_http_ports():
# SSH should never trigger an HTTP request
get = AsyncMock()
with patch("httpx.AsyncClient.get", new=get):
ctx, factory = _patch_client(lambda req: _response("<title>nope</title>"))
with ctx:
result = await probe_port("10.0.0.5", 22)
assert result is None
get.assert_not_called()
assert factory.requests == []
@pytest.mark.asyncio
async def test_probe_port_verify_tls_flag_passed():
with patch("app.services.http_probe.httpx.AsyncClient") as client_cls:
instance = client_cls.return_value.__aenter__.return_value
instance.get = AsyncMock(return_value=_response("<title>X</title>"))
ctx, factory = _patch_client(lambda req: _response("<title>X</title>"))
with ctx:
await probe_port("10.0.0.5", 8443, verify_tls=True)
assert client_cls.call_args.kwargs["verify"] is True
assert factory.kwargs[0]["verify"] is True
# ── endless bodies (issue #375) ─────────────────────────────────────────────
_CHUNK_SIZE = 16 * 1024
def _endless_body(counter: dict, head: bytes = b""):
"""
A body that never ends and declares no Content-Length. Bounded at 512
chunks so a regression fails the test instead of hanging the suite.
"""
async def gen():
if head:
counter["bytes"] += len(head)
yield head
for _ in range(512):
counter["bytes"] += _CHUNK_SIZE
yield b"\0" * _CHUNK_SIZE
return gen()
@pytest.mark.asyncio
async def test_probe_caps_the_download_of_an_endless_body():
# _MAX_BODY_BYTES caps the download, not just the <title> scan: the body is
# streamed and the connection dropped once we hold enough. Allow one chunk
# of overshoot — the read stops on a chunk boundary.
counter = {"bytes": 0}
def handler(request):
# Plain HTTP only, like the reproducer, so the counter covers one probe.
if str(request.url).startswith("https"):
raise httpx.ConnectError("no tls")
return httpx.Response(
200,
headers={"Content-Type": "application/octet-stream"},
content=_endless_body(counter),
)
ctx, _ = _patch_client(handler)
with ctx:
result = await probe_port("10.0.0.5", 8095)
assert result is None # null bytes carry no title and no signal header
assert counter["bytes"] <= _MAX_BODY_BYTES + _CHUNK_SIZE
@pytest.mark.asyncio
async def test_probe_still_reads_a_title_from_an_endless_body():
# Stopping early must not cost us the signal: a <title> in the first chunk
# is still found even though the rest of the stream is abandoned.
counter = {"bytes": 0}
def handler(request):
if str(request.url).startswith("https"):
raise httpx.ConnectError("no tls")
return httpx.Response(
200,
headers={"Content-Type": "text/html"},
content=_endless_body(counter, head=b"<html><title>Jellyfin</title>"),
)
ctx, _ = _patch_client(handler)
with ctx:
result = await probe_port("10.0.0.5", 8096)
assert result["title"] == "Jellyfin"
assert counter["bytes"] <= _MAX_BODY_BYTES + _CHUNK_SIZE
# ── probe_open_ports ─────────────────────────────────────────────────────────
@pytest.mark.asyncio
async def test_probe_open_ports_enriches_each_port():
async def fake_get(self, url, **kw):
if ":8096" in url:
def handler(request):
if ":8096" in str(request.url):
return _response("<title>Jellyfin</title>")
return _response("")
ports = [{"port": 8096, "protocol": "tcp"}, {"port": 9999, "protocol": "tcp"}]
with patch("httpx.AsyncClient.get", new=fake_get):
ctx, _ = _patch_client(handler)
with ctx:
result = await probe_open_ports("10.0.0.5", ports)
by_port = {p["port"]: p for p in result}
+90
View File
@@ -0,0 +1,90 @@
"""Unit tests for the in-process canvas-import job registry."""
from __future__ import annotations
import pytest
from app.services import import_jobs
from app.services.import_jobs import (
create_job,
fail_job,
finish_job,
get_job,
reset_jobs,
)
@pytest.fixture(autouse=True)
def _clean() -> None:
reset_jobs()
def test_create_job_starts_running_with_unique_id() -> None:
a = create_job()
b = create_job()
assert a.id != b.id
assert a.status == "running"
assert a.result is None
assert a.error is None
def test_get_job_returns_the_registered_job() -> None:
job = create_job()
assert get_job(job.id) is job
def test_get_unknown_job_returns_none() -> None:
assert get_job("nope") is None
def test_finish_job_records_the_payload() -> None:
job = create_job()
finish_job(job.id, {"device_count": 3})
stored = get_job(job.id)
assert stored is not None
assert stored.status == "done"
assert stored.result == {"device_count": 3}
assert stored.finished_at is not None
def test_fail_job_records_message_and_status() -> None:
job = create_job()
fail_job(job.id, "broker unreachable", 502)
stored = get_job(job.id)
assert stored is not None
assert stored.status == "error"
assert stored.error == "broker unreachable"
assert stored.error_status == 502
def test_finish_and_fail_are_noops_for_unknown_ids() -> None:
finish_job("gone", {"device_count": 0})
fail_job("gone", "boom", 500)
assert get_job("gone") is None
def test_finished_jobs_expire_after_the_ttl(monkeypatch) -> None:
clock = {"now": 0.0}
monkeypatch.setattr(import_jobs.time, "monotonic", lambda: clock["now"])
job = create_job()
finish_job(job.id, {"device_count": 1})
clock["now"] = import_jobs.JOB_TTL_SECONDS + 1
assert get_job(job.id) is None
def test_running_jobs_never_expire(monkeypatch) -> None:
"""A slow mesh must not have its job purged out from under it."""
clock = {"now": 0.0}
monkeypatch.setattr(import_jobs.time, "monotonic", lambda: clock["now"])
job = create_job()
clock["now"] = import_jobs.JOB_TTL_SECONDS * 10
assert get_job(job.id) is not None
def test_reset_jobs_clears_everything() -> None:
job = create_job()
reset_jobs()
assert get_job(job.id) is None
+100
View File
@@ -2,13 +2,16 @@
from __future__ import annotations
import asyncio
import json
import ssl
from unittest.mock import patch
import pytest
from app.core.config import settings
from app.services.mqtt_common import (
_RESPONSE_TIMEOUT,
_build_tls_context,
_sanitize_mqtt_error,
request_response,
@@ -181,3 +184,100 @@ async def test_test_connection_failure() -> None:
mock_aiomqtt.MqttError = Exception
with pytest.raises(ConnectionError):
await _test_connection("bad", 1883)
# ---------------------------------------------------------------------------
# Response timeout — configurable via MQTT_RESPONSE_TIMEOUT (issue #380)
# ---------------------------------------------------------------------------
class _SilentClient:
"""An MQTT client that connects but never delivers the response message."""
async def __aenter__(self):
return self
async def __aexit__(self, *_):
pass
async def subscribe(self, *_a, **_kw) -> None:
pass
async def publish(self, *_a, **_kw) -> None:
pass
@property
def messages(self):
async def _never():
await asyncio.Event().wait()
yield # pragma: no cover
return _never()
class _NeverMqttError(Exception):
"""Stand-in for aiomqtt.MqttError that no test path actually raises."""
async def _round_trip(**kwargs):
return await request_response(
mqtt_host="host",
mqtt_port=1883,
request_topic="req",
response_topic="res",
request_payload={},
**kwargs,
)
@pytest.mark.asyncio
async def test_request_response_uses_settings_timeout(monkeypatch) -> None:
monkeypatch.setattr(settings, "mqtt_response_timeout", 0.01)
with patch("app.services.mqtt_common.aiomqtt") as mock_aiomqtt:
mock_aiomqtt.Client.return_value = _SilentClient()
mock_aiomqtt.MqttError = _NeverMqttError
with pytest.raises(TimeoutError) as ei:
await _round_trip()
msg = str(ei.value)
assert "0.01s" in msg
assert "MQTT_RESPONSE_TIMEOUT" in msg
@pytest.mark.asyncio
async def test_request_response_explicit_timeout_overrides_settings(monkeypatch) -> None:
monkeypatch.setattr(settings, "mqtt_response_timeout", 999)
with patch("app.services.mqtt_common.aiomqtt") as mock_aiomqtt:
mock_aiomqtt.Client.return_value = _SilentClient()
mock_aiomqtt.MqttError = _NeverMqttError
with pytest.raises(TimeoutError) as ei:
await _round_trip(response_timeout=0.01)
assert "0.01s" in str(ei.value)
@pytest.mark.asyncio
async def test_request_response_non_positive_setting_falls_back(monkeypatch) -> None:
"""A misconfigured 0 must not mean 'give up immediately'."""
monkeypatch.setattr(settings, "mqtt_response_timeout", 0)
captured: dict[str, float] = {}
async def _fake_wait_for(awaitable, timeout):
captured["timeout"] = timeout
awaitable.close()
raise asyncio.TimeoutError
with patch("app.services.mqtt_common.aiomqtt") as mock_aiomqtt:
mock_aiomqtt.Client.return_value = _SilentClient()
mock_aiomqtt.MqttError = _NeverMqttError
with (
patch("app.services.mqtt_common.asyncio.wait_for", _fake_wait_for),
pytest.raises(TimeoutError),
):
await _round_trip()
assert captured["timeout"] == _RESPONSE_TIMEOUT
+61 -8
View File
@@ -1,6 +1,7 @@
"""Tests for status_checker service: each check method."""
from unittest.mock import AsyncMock, MagicMock, patch
import httpx
import pytest
from app.services.status_checker import (
@@ -13,16 +14,31 @@ from app.services.status_checker import (
)
class _TransportClient:
"""
Patch target for httpx.AsyncClient that routes through a MockTransport.
Real client, fake network: _http_get exercises the genuine httpx request
path (including .stream()) instead of a mock that would happily accept any
call shape.
"""
def __init__(self, handler):
self._handler = handler
# Bound before patching, so building the real client here does not
# recurse back into this stand-in.
self._real = httpx.AsyncClient
self.kwargs = []
def __call__(self, **kwargs):
self.kwargs.append(dict(kwargs))
kwargs.pop("verify", None)
return self._real(transport=httpx.MockTransport(self._handler), **kwargs)
def _mock_httpx_client(status_code):
"""Build a stand-in for httpx.AsyncClient whose GET returns status_code."""
resp = MagicMock()
resp.status_code = status_code
client = MagicMock()
client.get = AsyncMock(return_value=resp)
ctx = MagicMock()
ctx.__aenter__ = AsyncMock(return_value=client)
ctx.__aexit__ = AsyncMock(return_value=False)
return MagicMock(return_value=ctx)
return _TransportClient(lambda request: httpx.Response(status_code))
# --- check_node dispatcher ---
@@ -554,6 +570,43 @@ async def test_check_services_empty_list():
# --- _http_get status-code interpretation (real primitive, mocked transport) ---
# --- Regression: an endless response body must not be buffered (issue #375) ---
_CHUNK = b"\0" * 65536
def _endless_body(counter: dict):
"""
A body that never ends and declares no Content-Length — the Freebox
bandwidth-test endpoint from issue #375. Bounded at 512 chunks (32 MiB) so
a regression fails the test instead of hanging the suite forever.
"""
async def gen():
for _ in range(512):
counter["chunks"] += 1
yield _CHUNK
return gen()
@pytest.mark.asyncio
async def test_http_get_does_not_read_the_body():
# _http_get only needs the status line. Buffering the body of an endless
# stream is what OOM-killed the backend, so assert not one byte is pulled.
counter = {"chunks": 0}
factory = _TransportClient(
lambda request: httpx.Response(
200,
headers={"Content-Type": "application/octet-stream"},
content=_endless_body(counter),
)
)
with patch("app.services.status_checker.httpx.AsyncClient", factory):
assert await _http_get("http://192.168.1.254:8095/") is True
assert counter["chunks"] == 0
@pytest.mark.asyncio
async def test_http_get_true_on_2xx():
with patch("app.services.status_checker.httpx.AsyncClient", _mock_httpx_client(200)):
+91 -40
View File
@@ -7,10 +7,34 @@ from unittest.mock import AsyncMock, patch
import pytest
from httpx import AsyncClient
from app.services.import_jobs import reset_jobs
# ---------------------------------------------------------------------------
# Fixtures
# ---------------------------------------------------------------------------
@pytest.fixture(autouse=True)
def _clear_import_jobs() -> None:
"""The canvas-import job registry is process-global; isolate each test."""
reset_jobs()
async def _start_canvas_import(
client: AsyncClient, headers: dict, body: dict
) -> str:
"""POST /zigbee/import and return the job id.
The route is fire-and-poll: it answers 202 immediately and the fetch runs as
a Starlette background task, which the ASGI transport completes before this
await returns — so the job is already settled by the time we poll.
"""
res = await client.post("/api/v1/zigbee/import", json=body, headers=headers)
assert res.status_code == 202, res.text
job_id: str = res.json()["job_id"]
assert res.json()["status"] == "running"
return job_id
# ---------------------------------------------------------------------------
# /api/v1/zigbee/test-connection
# ---------------------------------------------------------------------------
@@ -104,18 +128,18 @@ _SAMPLE_EDGES = [
async def test_import_success(client: AsyncClient, headers: dict) -> None:
with patch("app.api.routes.zigbee.fetch_networkmap") as mock_fetch:
mock_fetch.return_value = (_SAMPLE_NODES, _SAMPLE_EDGES)
res = await client.post(
"/api/v1/zigbee/import",
json={
"mqtt_host": "localhost",
"mqtt_port": 1883,
"base_topic": "zigbee2mqtt",
},
headers=headers,
job_id = await _start_canvas_import(
client,
headers,
{"mqtt_host": "localhost", "mqtt_port": 1883, "base_topic": "zigbee2mqtt"},
)
res = await client.get(f"/api/v1/zigbee/import/{job_id}", headers=headers)
assert res.status_code == 200
data = res.json()
body = res.json()
assert body["status"] == "done"
assert body["job_id"] == job_id
data = body["result"]
assert data["device_count"] == 2
assert len(data["nodes"]) == 2
assert len(data["edges"]) == 1
@@ -123,22 +147,57 @@ async def test_import_success(client: AsyncClient, headers: dict) -> None:
assert coordinator["ieee_address"] == "0x00000000"
@pytest.mark.asyncio
async def test_import_post_does_not_block_on_the_fetch(
client: AsyncClient, headers: dict
) -> None:
"""The POST answers 202 without a payload — that is what keeps a slow mesh
from outliving a reverse proxy's read timeout (issue #380)."""
with patch("app.api.routes.zigbee.fetch_networkmap") as mock_fetch:
mock_fetch.return_value = (_SAMPLE_NODES, _SAMPLE_EDGES)
res = await client.post(
"/api/v1/zigbee/import",
json={"mqtt_host": "localhost", "mqtt_port": 1883},
headers=headers,
)
assert res.status_code == 202
assert set(res.json()) == {"job_id", "status"}
@pytest.mark.asyncio
async def test_import_job_unknown_id_returns_404(
client: AsyncClient, headers: dict
) -> None:
res = await client.get("/api/v1/zigbee/import/no-such-job", headers=headers)
assert res.status_code == 404
@pytest.mark.asyncio
async def test_import_job_requires_auth(client: AsyncClient, headers: dict) -> None:
with patch("app.api.routes.zigbee.fetch_networkmap") as mock_fetch:
mock_fetch.return_value = ([], [])
job_id = await _start_canvas_import(
client, headers, {"mqtt_host": "localhost", "mqtt_port": 1883}
)
res = await client.get(f"/api/v1/zigbee/import/{job_id}")
assert res.status_code == 401
@pytest.mark.asyncio
async def test_import_with_credentials(client: AsyncClient, headers: dict) -> None:
with patch("app.api.routes.zigbee.fetch_networkmap") as mock_fetch:
mock_fetch.return_value = ([], [])
res = await client.post(
"/api/v1/zigbee/import",
json={
await _start_canvas_import(
client,
headers,
{
"mqtt_host": "localhost",
"mqtt_port": 1883,
"mqtt_username": "admin",
"mqtt_password": "secret",
"base_topic": "z2m",
},
headers=headers,
)
assert res.status_code == 200
mock_fetch.assert_called_once_with(
mqtt_host="localhost",
mqtt_port=1883,
@@ -154,11 +213,10 @@ async def test_import_with_credentials(client: AsyncClient, headers: dict) -> No
async def test_import_connection_error_returns_502(client: AsyncClient, headers: dict) -> None:
with patch("app.api.routes.zigbee.fetch_networkmap") as mock_fetch:
mock_fetch.side_effect = ConnectionError("broker unreachable")
res = await client.post(
"/api/v1/zigbee/import",
json={"mqtt_host": "bad-host", "mqtt_port": 1883},
headers=headers,
job_id = await _start_canvas_import(
client, headers, {"mqtt_host": "bad-host", "mqtt_port": 1883}
)
res = await client.get(f"/api/v1/zigbee/import/{job_id}", headers=headers)
assert res.status_code == 502
assert "broker unreachable" in res.json()["detail"]
@@ -167,23 +225,22 @@ async def test_import_connection_error_returns_502(client: AsyncClient, headers:
async def test_import_timeout_returns_504(client: AsyncClient, headers: dict) -> None:
with patch("app.api.routes.zigbee.fetch_networkmap") as mock_fetch:
mock_fetch.side_effect = TimeoutError("timed out")
res = await client.post(
"/api/v1/zigbee/import",
json={"mqtt_host": "localhost", "mqtt_port": 1883},
headers=headers,
job_id = await _start_canvas_import(
client, headers, {"mqtt_host": "localhost", "mqtt_port": 1883}
)
res = await client.get(f"/api/v1/zigbee/import/{job_id}", headers=headers)
assert res.status_code == 504
assert "timed out" in res.json()["detail"]
@pytest.mark.asyncio
async def test_import_malformed_payload_returns_422(client: AsyncClient, headers: dict) -> None:
with patch("app.api.routes.zigbee.fetch_networkmap") as mock_fetch:
mock_fetch.side_effect = ValueError("malformed response")
res = await client.post(
"/api/v1/zigbee/import",
json={"mqtt_host": "localhost", "mqtt_port": 1883},
headers=headers,
job_id = await _start_canvas_import(
client, headers, {"mqtt_host": "localhost", "mqtt_port": 1883}
)
res = await client.get(f"/api/v1/zigbee/import/{job_id}", headers=headers)
assert res.status_code == 422
@@ -201,13 +258,12 @@ async def test_import_empty_network(client: AsyncClient, headers: dict) -> None:
"""An empty Zigbee network (coordinator only) is a valid response."""
with patch("app.api.routes.zigbee.fetch_networkmap") as mock_fetch:
mock_fetch.return_value = ([], [])
res = await client.post(
"/api/v1/zigbee/import",
json={"mqtt_host": "localhost", "mqtt_port": 1883},
headers=headers,
job_id = await _start_canvas_import(
client, headers, {"mqtt_host": "localhost", "mqtt_port": 1883}
)
res = await client.get(f"/api/v1/zigbee/import/{job_id}", headers=headers)
assert res.status_code == 200
data = res.json()
data = res.json()["result"]
assert data["device_count"] == 0
assert data["nodes"] == []
assert data["edges"] == []
@@ -227,16 +283,11 @@ async def test_import_missing_mqtt_host(client: AsyncClient, headers: dict) -> N
async def test_import_with_tls_passes_flags(client: AsyncClient, headers: dict) -> None:
with patch("app.api.routes.zigbee.fetch_networkmap") as mock_fetch:
mock_fetch.return_value = ([], [])
res = await client.post(
"/api/v1/zigbee/import",
json={
"mqtt_host": "broker.example.com",
"mqtt_port": 8883,
"mqtt_tls": True,
},
headers=headers,
await _start_canvas_import(
client,
headers,
{"mqtt_host": "broker.example.com", "mqtt_port": 8883, "mqtt_tls": True},
)
assert res.status_code == 200
kwargs = mock_fetch.call_args.kwargs
assert kwargs["tls"] is True
assert kwargs["tls_insecure"] is False
+99
View File
@@ -2,6 +2,7 @@
from __future__ import annotations
import asyncio
import json
from typing import Any
from unittest.mock import patch
@@ -9,7 +10,9 @@ from unittest.mock import patch
import aiomqtt # noqa: F401
import pytest
from app.core.config import settings
from app.services.zigbee_service import (
_NETWORKMAP_TIMEOUT,
_find_parent_router,
_z2m_type_to_homelable,
fetch_networkmap,
@@ -433,6 +436,10 @@ async def test_test_mqtt_connection_failure() -> None:
await _test_mqtt_connection("bad-host", 1883)
class _NeverMqttError(Exception):
"""Stand-in for aiomqtt.MqttError that no test path actually raises."""
# ---------------------------------------------------------------------------
# TLS context
# ---------------------------------------------------------------------------
@@ -571,3 +578,95 @@ async def test_fetch_networkmap_does_not_leak_creds_in_connection_error() -> Non
assert "hunter2" not in msg
assert "admin" not in msg
assert msg == "Authentication failed"
# ---------------------------------------------------------------------------
# Networkmap response timeout — configurable via ZIGBEE_NETWORKMAP_TIMEOUT
# (issue #380: a 200+ device mesh needs longer than the hard-coded 300 s)
# ---------------------------------------------------------------------------
class _SilentClient:
"""An MQTT client that connects but never delivers the response message."""
async def __aenter__(self):
return self
async def __aexit__(self, *_):
pass
async def subscribe(self, *_a, **_kw) -> None:
pass
async def publish(self, *_a, **_kw) -> None:
pass
@property
def messages(self):
async def _never():
await asyncio.Event().wait()
yield # pragma: no cover
return _never()
@pytest.mark.asyncio
async def test_fetch_networkmap_uses_settings_timeout(monkeypatch) -> None:
monkeypatch.setattr(settings, "zigbee_networkmap_timeout", 0.01)
with patch("app.services.zigbee_service.aiomqtt") as mock_aiomqtt:
mock_aiomqtt.Client.return_value = _SilentClient()
mock_aiomqtt.MqttError = _NeverMqttError
with pytest.raises(TimeoutError) as ei:
await fetch_networkmap(
mqtt_host="host", mqtt_port=1883, base_topic="zigbee2mqtt"
)
msg = str(ei.value)
assert "0.01s" in msg
assert "ZIGBEE_NETWORKMAP_TIMEOUT" in msg
@pytest.mark.asyncio
async def test_fetch_networkmap_explicit_timeout_overrides_settings(monkeypatch) -> None:
monkeypatch.setattr(settings, "zigbee_networkmap_timeout", 999)
with patch("app.services.zigbee_service.aiomqtt") as mock_aiomqtt:
mock_aiomqtt.Client.return_value = _SilentClient()
mock_aiomqtt.MqttError = _NeverMqttError
with pytest.raises(TimeoutError) as ei:
await fetch_networkmap(
mqtt_host="host",
mqtt_port=1883,
base_topic="zigbee2mqtt",
response_timeout=0.01,
)
assert "0.01s" in str(ei.value)
@pytest.mark.asyncio
async def test_fetch_networkmap_non_positive_setting_falls_back(monkeypatch) -> None:
"""A misconfigured 0 must not mean 'give up immediately'."""
monkeypatch.setattr(settings, "zigbee_networkmap_timeout", 0)
captured: dict[str, float] = {}
async def _fake_wait_for(awaitable, timeout):
captured["timeout"] = timeout
awaitable.close()
raise asyncio.TimeoutError
with patch("app.services.zigbee_service.aiomqtt") as mock_aiomqtt:
mock_aiomqtt.Client.return_value = _SilentClient()
mock_aiomqtt.MqttError = _NeverMqttError
with (
patch("app.services.zigbee_service.asyncio.wait_for", _fake_wait_for),
pytest.raises(TimeoutError),
):
await fetch_networkmap(
mqtt_host="host", mqtt_port=1883, base_topic="zigbee2mqtt"
)
assert captured["timeout"] == _NETWORKMAP_TIMEOUT
+139
View File
@@ -0,0 +1,139 @@
"""Moving a zone's size out of the custom_colors blob into the real columns.
Every node type stored its size in `nodes.width` / `nodes.height` except
`groupRect`, which kept it inside the style JSON. Get this backfill wrong and
every zone a user ever drew comes back at the default 360x240, losing a layout
they arranged by hand.
"""
import json
import pytest
from sqlalchemy.ext.asyncio import create_async_engine
import app.db.database as database
pytestmark = pytest.mark.asyncio
_DDL = (
"CREATE TABLE nodes ("
"id VARCHAR PRIMARY KEY, type VARCHAR, label VARCHAR, "
"custom_colors JSON, width FLOAT, height FLOAT)"
)
async def _engine(tmp_path, rows):
engine = create_async_engine(f"sqlite+aiosqlite:///{tmp_path / 'zones.db'}")
async with engine.begin() as conn:
await conn.exec_driver_sql(_DDL)
for node_id, node_type, colors, width, height in rows:
await conn.exec_driver_sql(
"INSERT INTO nodes (id, type, label, custom_colors, width, height) "
"VALUES (?, ?, ?, ?, ?, ?)",
(node_id, node_type, node_id, json.dumps(colors) if colors else None, width, height),
)
return engine
async def _sizes(engine) -> dict[str, tuple]:
async with engine.begin() as conn:
rows = (await conn.exec_driver_sql("SELECT id, width, height FROM nodes")).fetchall()
return {r[0]: (r[1], r[2]) for r in rows}
async def _run(engine, monkeypatch) -> None:
monkeypatch.setattr(database, "engine", engine)
await database._backfill_zone_size()
async def test_moves_the_size_from_the_blob_to_the_columns(tmp_path, monkeypatch):
engine = await _engine(
tmp_path,
[("z1", "groupRect", {"width": 640, "height": 480, "border": "#0ff"}, None, None)],
)
await _run(engine, monkeypatch)
assert (await _sizes(engine))["z1"] == (640, 480)
# The style the blob actually owns is untouched.
async with engine.begin() as conn:
blob = (await conn.exec_driver_sql("SELECT custom_colors FROM nodes")).scalar()
assert json.loads(blob)["border"] == "#0ff"
await engine.dispose()
async def test_never_overwrites_a_size_already_in_the_columns(tmp_path, monkeypatch):
# A zone resized since the upgrade: the column is the truth, and a stale
# blob left over from before must not win.
engine = await _engine(
tmp_path,
[("z1", "groupRect", {"width": 111, "height": 222}, 800, None)],
)
await _run(engine, monkeypatch)
assert (await _sizes(engine))["z1"] == (800, 222)
await engine.dispose()
async def test_leaves_other_node_types_alone(tmp_path, monkeypatch):
# Only zones ever stashed their size in the blob. A width key on any other
# type is not geometry we should be moving.
engine = await _engine(tmp_path, [("s1", "server", {"width": 999}, None, None)])
await _run(engine, monkeypatch)
assert (await _sizes(engine))["s1"] == (None, None)
await engine.dispose()
async def test_skips_a_blob_with_no_usable_size(tmp_path, monkeypatch):
engine = await _engine(
tmp_path,
[
("colors_only", "groupRect", {"border": "#0ff"}, None, None),
("not_a_number", "groupRect", {"width": "wide", "height": True}, None, None),
("no_blob", "groupRect", None, None, None),
],
)
await _run(engine, monkeypatch)
sizes = await _sizes(engine)
assert sizes["colors_only"] == (None, None)
assert sizes["not_a_number"] == (None, None)
assert sizes["no_blob"] == (None, None)
await engine.dispose()
async def test_survives_a_corrupt_blob_without_killing_boot(tmp_path, monkeypatch):
engine = create_async_engine(f"sqlite+aiosqlite:///{tmp_path / 'zones.db'}")
async with engine.begin() as conn:
await conn.exec_driver_sql(_DDL)
await conn.exec_driver_sql(
"INSERT INTO nodes (id, type, label, custom_colors) VALUES ('bad', 'groupRect', 'b', '{oops')"
)
await conn.exec_driver_sql(
"INSERT INTO nodes (id, type, label, custom_colors) "
"VALUES ('good', 'groupRect', 'g', '{\"width\": 500, \"height\": 400}')"
)
await _run(engine, monkeypatch)
sizes = await _sizes(engine)
assert sizes["bad"] == (None, None)
# One unreadable row must not cost the others their size.
assert sizes["good"] == (500, 400)
await engine.dispose()
async def test_is_idempotent(tmp_path, monkeypatch):
engine = await _engine(
tmp_path,
[("z1", "groupRect", {"width": 640, "height": 480}, None, None)],
)
await _run(engine, monkeypatch)
await _run(engine, monkeypatch)
assert (await _sizes(engine))["z1"] == (640, 480)
await engine.dispose()
+17 -1
View File
@@ -55,9 +55,25 @@ Click **Fetch Devices**. Homelable will:
1. Connect to the broker
2. Subscribe to the response topic
3. Publish `{"type": "raw", "routes": false}` to the request topic
4. Wait up to 60 seconds for the network map response (large meshes can take 30 s+)
4. Wait up to `ZIGBEE_NETWORKMAP_TIMEOUT` seconds (default 300) for the network map response
5. Parse and group devices by type
The fetch runs server-side and the browser polls for the result, so a mesh that
takes minutes to answer cannot be cut short by a reverse proxy's read timeout.
### Large meshes and timeouts
A network of 200+ devices can take several minutes to build its map. Two knobs:
| Variable | Default | What it bounds |
|---|---|---|
| `ZIGBEE_NETWORKMAP_TIMEOUT` | `300` | Seconds to wait for the Z2M bridge to answer a networkmap request. Raise it if an import fails with *Timed out waiting for networkmap response*. |
| `MQTT_RESPONSE_TIMEOUT` | `300` | The same bound for the Z-Wave MQTT round-trip. |
Both apply to manual imports and to auto-sync. If you reverse-proxy the API,
these are the only timeouts that matter — the import request itself returns
immediately.
### 5. Select and add to canvas
Devices are grouped by type (Coordinator / Router / End Device).
+34 -3
View File
@@ -15,6 +15,7 @@ import { parseYamlToCanvas } from '@/utils/importYaml'
import { TooltipProvider } from '@/components/ui/tooltip'
import { Toaster } from '@/components/ui/sonner'
import { toast } from 'sonner'
import { isZoneSubnetCandidate } from '@/utils/subnet'
import { CanvasContainer } from '@/components/canvas/CanvasContainer'
import { Sidebar } from '@/components/panels/Sidebar'
import { Toolbar } from '@/components/panels/Toolbar'
@@ -64,7 +65,7 @@ import { buildProxmoxClusterEdges } from '@/components/proxmox/clusterEdges'
const STANDALONE = import.meta.env.VITE_STANDALONE === 'true'
export default function App() {
const { loadCanvas, applyLayout, markSaved, markUnsaved, hasUnsavedChanges, editSeq, selectedNodeId, selectedNodeIds, addNode, updateNode, deleteNode, onConnect, updateEdge, deleteEdge, setProxmoxContainerMode, setNodeZIndex, editingGroupRectId, setEditingGroupRectId, editingTextId, setEditingTextId, nodes, edges, snapshotHistory, undo, redo, addToGroup, addToContainer, addToZone, floorMap, setFloorMap } = useCanvasStore()
const { loadCanvas, applyLayout, markSaved, markUnsaved, hasUnsavedChanges, editSeq, selectedNodeId, selectedNodeIds, addNode, updateNode, deleteNode, onConnect, updateEdge, deleteEdge, setProxmoxContainerMode, setNodeZIndex, editingGroupRectId, setEditingGroupRectId, editingTextId, setEditingTextId, nodes, edges, snapshotHistory, undo, redo, addToGroup, addToContainer, addToZone, importZoneSubnet, floorMap, setFloorMap } = useCanvasStore()
const canvasRef = useRef<HTMLDivElement>(null)
const { isAuthenticated, isInitialized } = useAuthStore()
const authBootstrapStarted = useRef(false)
@@ -526,6 +527,26 @@ export default function App() {
toast.success(`Added "${data.label}"`)
}, [addNode, nodes, snapshotHistory])
// Subnet import is a one-shot action on an existing zone, so on the Add modal
// there is no zone to run it against yet: the CIDR is held here and applied
// once, right after the zone is created. Cleared whenever that modal closes.
const pendingZoneSubnet = useRef<string | null>(null)
const countSubnetMatches = useCallback(
(cidr: string, zoneId?: string) =>
nodes.filter((n) => isZoneSubnetCandidate(n, cidr, zoneId)).length,
[nodes],
)
const reportSubnetImport = useCallback((moved: number, cidr: string) => {
if (moved === 0) toast.info(`No unparented device in ${cidr}`)
else toast.success(`Moved ${moved} device${moved > 1 ? 's' : ''} from ${cidr} into the zone`)
}, [])
const handleImportSubnetIntoZone = useCallback((zoneId: string, cidr: string) => {
reportSubnetImport(importZoneSubnet(zoneId, cidr), cidr)
}, [importZoneSubnet, reportSubnetImport])
const handleAddGroupRect = useCallback((data: GroupRectFormData) => {
snapshotHistory()
const id = generateUUID()
@@ -556,7 +577,12 @@ export default function App() {
zIndex: data.z_order - 10,
}
addNode(newNode)
}, [addNode, snapshotHistory])
const cidr = pendingZoneSubnet.current
pendingZoneSubnet.current = null
// addNode has already committed, so the zone is in the store by now.
if (cidr) reportSubnetImport(importZoneSubnet(id, cidr), cidr)
}, [addNode, snapshotHistory, importZoneSubnet, reportSubnetImport])
const handleUpdateGroupRect = useCallback((data: GroupRectFormData) => {
if (!editingGroupRectId) return
@@ -1190,8 +1216,11 @@ export default function App() {
<GroupRectModal
open={addGroupRectOpen}
onClose={() => setAddGroupRectOpen(false)}
onClose={() => { pendingZoneSubnet.current = null; setAddGroupRectOpen(false) }}
onSubmit={handleAddGroupRect}
onImportSubnet={(cidr) => { pendingZoneSubnet.current = cidr }}
countSubnetMatches={(cidr) => countSubnetMatches(cidr)}
importOnSubmit
title="Add Zone"
/>
@@ -1202,6 +1231,8 @@ export default function App() {
onClose={() => setEditingGroupRectId(null)}
onSubmit={handleUpdateGroupRect}
onDelete={handleDeleteGroupRect}
onImportSubnet={(cidr) => { if (editingGroupRectId) handleImportSubnetIntoZone(editingGroupRectId, cidr) }}
countSubnetMatches={(cidr) => countSubnetMatches(cidr, editingGroupRectId ?? undefined)}
initial={(() => {
const n = editingGroupRectId ? nodes.find((nd) => nd.id === editingGroupRectId) : null
if (!n) return undefined
@@ -242,6 +242,11 @@ describe('api/client', () => {
expect(api.post).toHaveBeenCalledWith('/zigbee/import-pending', cfg)
})
it('zigbeeApi.getImportJob polls the job by id', () => {
mod.zigbeeApi.getImportJob('job-1')
expect(api.get).toHaveBeenCalledWith('/zigbee/import/job-1')
})
it('zwaveApi.testConnection/importNetwork/importToPending', () => {
const cfg = { mqtt_host: 'h', mqtt_port: 1883, prefix: 'zwave', gateway_name: 'zwavejs2mqtt' }
mod.zwaveApi.testConnection(cfg)
+16 -5
View File
@@ -305,6 +305,8 @@ export const racksApi = {
}),
}
export type ZigbeeImportJobStatus = 'running' | 'done' | 'error'
export interface ZigbeeConfigData {
mqtt_host: string
mqtt_port: number
@@ -358,11 +360,20 @@ export const zigbeeApi = {
mqtt_tls?: boolean
mqtt_tls_insecure?: boolean
}) =>
api.post<{
nodes: import('@/components/zigbee/types').ZigbeeNode[]
edges: import('@/components/zigbee/types').ZigbeeEdge[]
device_count: number
}>('/zigbee/import', data),
api.post<{ job_id: string; status: ZigbeeImportJobStatus }>('/zigbee/import', data),
// Poll a canvas import started by importNetwork. The fetch runs server-side
// so a slow mesh cannot outlive a reverse proxy's read timeout.
getImportJob: (jobId: string) =>
api.get<{
job_id: string
status: ZigbeeImportJobStatus
result: {
nodes: import('@/components/zigbee/types').ZigbeeNode[]
edges: import('@/components/zigbee/types').ZigbeeEdge[]
device_count: number
} | null
}>(`/zigbee/import/${jobId}`),
importToPending: (data: {
mqtt_host: string
@@ -7,6 +7,7 @@ import { Label } from '@/components/ui/label'
import { Select, SelectContent, SelectItem, SelectTrigger, SelectValue } from '@/components/ui/select'
import type { TextPosition } from '@/types'
import { hexToRgba, rgbaToHex8 } from '@/utils/colorUtils'
import { isValidCidr } from '@/utils/subnet'
import styles from './GroupRectModal.module.css'
export type BorderStyle = 'solid' | 'dashed' | 'dotted' | 'double' | 'none'
@@ -98,16 +99,51 @@ interface GroupRectModalProps {
onDelete?: () => void
initial?: Partial<GroupRectFormData>
title?: string
/** Run the subnet import for a valid CIDR. Omit to hide the whole section. */
onImportSubnet?: (cidr: string) => void
/** How many unparented devices a CIDR would pull in — drives the preview line. */
countSubnetMatches?: (cidr: string) => number
/**
* Add mode: the zone does not exist yet, so there is nothing to import into
* until it is created. The CIDR is handed over on submit instead of via an
* Import button, which would otherwise look like it did nothing.
*/
importOnSubmit?: boolean
}
export function GroupRectModal({ open, onClose, onSubmit, onDelete, initial, title = 'Add Zone' }: GroupRectModalProps) {
export function GroupRectModal({
open,
onClose,
onSubmit,
onDelete,
initial,
title = 'Add Zone',
onImportSubnet,
countSubnetMatches,
importOnSubmit = false,
}: GroupRectModalProps) {
const [form, setForm] = useState<GroupRectFormData>({ ...DEFAULT_FORM, ...initial })
const [subnet, setSubnet] = useState('')
const cidrValid = isValidCidr(subnet)
const showCidrError = subnet.trim() !== '' && !cidrValid
const canImport = cidrValid && !!onImportSubnet
const matchCount = cidrValid && countSubnetMatches ? countSubnetMatches(subnet) : null
const handleImport = () => {
if (!canImport) return
onImportSubnet!(subnet)
setSubnet('')
}
const set = <K extends keyof GroupRectFormData>(key: K, value: GroupRectFormData[K]) =>
setForm((f) => ({ ...f, [key]: value }))
const handleSubmit = (e: React.FormEvent) => {
e.preventDefault()
// Hand the CIDR over first: the caller queues it, then creates the zone and
// runs the import against it.
if (importOnSubmit && canImport) onImportSubnet!(subnet)
onSubmit(form)
onClose()
}
@@ -355,6 +391,57 @@ export function GroupRectModal({ open, onClose, onSubmit, onDelete, initial, tit
</div>{/* ── end RIGHT column ── */}
</div>{/* ── end 2-column grid ── */}
{/* ── Import devices by subnet ──
Deliberately NOT part of the form: the CIDR is an argument to a
one-shot action, never a property of the zone, so it is not
submitted, not persisted, and cleared after each run. */}
{onImportSubnet && (
<div className="flex flex-col gap-2 pt-3 border-t border-[#30363d]">
<div className="text-[11px] font-semibold uppercase tracking-wider text-muted-foreground/70">
Import devices by subnet
</div>
<div className="flex gap-2">
<Input
value={subnet}
onChange={(e) => setSubnet(e.target.value)}
placeholder="192.168.1.0/24"
aria-label="Subnet"
className={`bg-[#21262d] border-[#30363d] text-sm h-8 font-mono ${modalStyles['modal-radius']}`}
style={showCidrError ? { borderColor: '#f85149' } : undefined}
// Enter runs the import instead of submitting the whole form —
// except in add mode, where submitting IS how the import runs.
onKeyDown={(e) => {
if (e.key === 'Enter' && !importOnSubmit) {
e.preventDefault()
handleImport()
}
}}
/>
{!importOnSubmit && (
<Button
type="button"
size="sm"
variant="outline"
disabled={!canImport}
className="cursor-pointer border-[#30363d] bg-[#21262d] shrink-0"
onClick={handleImport}
>
Import
</Button>
)}
</div>
<p className="text-[11px] text-muted-foreground/70">
{showCidrError
? <span className="text-[#f85149]">Not a valid IPv4 CIDR — try 192.168.1.0/24</span>
: matchCount === null
? 'Moves every unparented device in that range into this zone. Nothing is removed.'
: matchCount === 0
? 'No unparented device in that range.'
: `${matchCount} device${matchCount > 1 ? 's' : ''} will move into this zone${importOnSubmit ? ' when you add it' : ''}.`}
</p>
</div>
)}
<div className="flex justify-between gap-2 pt-1">
{onDelete && (
<Button
@@ -345,3 +345,110 @@ describe('GroupRectModal font label rendering', () => {
expect(trigger.textContent).toContain('comic-sans-9000')
})
})
describe('GroupRectModal — subnet import', () => {
const setup = (props: Partial<React.ComponentProps<typeof GroupRectModal>> = {}) => {
const onImportSubnet = vi.fn()
const onSubmit = vi.fn()
const onClose = vi.fn()
render(
<GroupRectModal
open
onClose={onClose}
onSubmit={onSubmit}
onImportSubnet={onImportSubnet}
countSubnetMatches={() => 3}
title="Edit Zone"
{...props}
/>
)
return { onImportSubnet, onSubmit, onClose }
}
it('hides the whole section when no import handler is given', () => {
render(<GroupRectModal open onClose={vi.fn()} onSubmit={vi.fn()} />)
expect(screen.queryByText('Import devices by subnet')).toBeNull()
})
it('renders the field and disables Import until a CIDR is valid', () => {
setup()
const button = screen.getByRole('button', { name: 'Import' }) as HTMLButtonElement
expect(button.disabled).toBe(true)
fireEvent.change(screen.getByLabelText('Subnet'), { target: { value: '192.168.1.0/24' } })
expect(button.disabled).toBe(false)
})
it('explains an invalid CIDR instead of silently doing nothing', () => {
setup()
fireEvent.change(screen.getByLabelText('Subnet'), { target: { value: '192.168.1.0/99' } })
expect(screen.getByText(/Not a valid IPv4 CIDR/)).toBeInTheDocument()
expect((screen.getByRole('button', { name: 'Import' }) as HTMLButtonElement).disabled).toBe(true)
})
it('previews how many devices would move', () => {
setup()
fireEvent.change(screen.getByLabelText('Subnet'), { target: { value: '192.168.1.0/24' } })
expect(screen.getByText('3 devices will move into this zone.')).toBeInTheDocument()
})
it('imports without submitting or closing the form, and clears the field', () => {
const { onImportSubnet, onSubmit, onClose } = setup()
const field = screen.getByLabelText('Subnet') as HTMLInputElement
fireEvent.change(field, { target: { value: '192.168.1.0/24' } })
fireEvent.click(screen.getByRole('button', { name: 'Import' }))
expect(onImportSubnet).toHaveBeenCalledWith('192.168.1.0/24')
expect(onSubmit).not.toHaveBeenCalled()
expect(onClose).not.toHaveBeenCalled()
expect(field.value).toBe('')
})
it('runs the import on Enter rather than submitting the zone', () => {
const { onImportSubnet, onSubmit } = setup()
const field = screen.getByLabelText('Subnet')
fireEvent.change(field, { target: { value: '10.0.0.0/8' } })
fireEvent.keyDown(field, { key: 'Enter' })
expect(onImportSubnet).toHaveBeenCalledWith('10.0.0.0/8')
expect(onSubmit).not.toHaveBeenCalled()
})
it('never submits the CIDR as zone data — it is an argument, not a property', () => {
const { onSubmit } = setup()
fireEvent.change(screen.getByLabelText('Subnet'), { target: { value: '192.168.1.0/24' } })
fireEvent.click(screen.getByText('Save'))
expect(Object.keys(onSubmit.mock.calls[0][0])).not.toContain('subnet')
})
describe('add mode (importOnSubmit)', () => {
it('drops the Import button — there is no zone to import into yet', () => {
setup({ importOnSubmit: true, title: 'Add Zone' })
expect(screen.getByText('Import devices by subnet')).toBeInTheDocument()
expect(screen.queryByRole('button', { name: 'Import' })).toBeNull()
})
it('says the move happens on creation', () => {
setup({ importOnSubmit: true, title: 'Add Zone' })
fireEvent.change(screen.getByLabelText('Subnet'), { target: { value: '192.168.1.0/24' } })
expect(screen.getByText('3 devices will move into this zone when you add it.')).toBeInTheDocument()
})
it('hands the CIDR over before submitting, so the caller can apply it to the new zone', () => {
const { onImportSubnet, onSubmit } = setup({ importOnSubmit: true, title: 'Add Zone' })
fireEvent.change(screen.getByLabelText('Subnet'), { target: { value: '192.168.1.0/24' } })
fireEvent.click(screen.getByText('Add'))
expect(onImportSubnet).toHaveBeenCalledWith('192.168.1.0/24')
expect(onSubmit).toHaveBeenCalledOnce()
expect(onImportSubnet.mock.invocationCallOrder[0]).toBeLessThan(onSubmit.mock.invocationCallOrder[0])
})
it('submits normally when the field is left empty', () => {
const { onImportSubnet, onSubmit } = setup({ importOnSubmit: true, title: 'Add Zone' })
fireEvent.click(screen.getByText('Add'))
expect(onImportSubnet).not.toHaveBeenCalled()
expect(onSubmit).toHaveBeenCalledOnce()
})
})
})
@@ -1,10 +1,11 @@
import { useState } from 'react'
import { useEffect, useRef, useState } from 'react'
import { Network, Router, Cpu, CheckCircle2, XCircle, Loader2, Plus } from 'lucide-react'
import { Dialog, DialogContent, DialogHeader, DialogTitle, DialogFooter } from '@/components/ui/dialog'
import { Button } from '@/components/ui/button'
import { Input } from '@/components/ui/input'
import { Label } from '@/components/ui/label'
import { zigbeeApi } from '@/api/client'
import { pollImportJob, PollAbortedError } from '@/utils/importJobPoll'
import { toast } from 'sonner'
import type { ZigbeeNode, ZigbeeEdge } from './types'
@@ -68,6 +69,10 @@ export function ZigbeeImportModal({ open, onClose, onAddToCanvas, onInventoryImp
const [edges, setEdges] = useState<ZigbeeEdge[]>([])
const [checked, setChecked] = useState<Set<string>>(new Set())
const [importMode, setImportMode] = useState<ImportMode>('pending')
// Aborts the canvas-import poll loop when the modal closes or unmounts.
const pollAbort = useRef<AbortController | null>(null)
useEffect(() => () => pollAbort.current?.abort(), [])
const updateField = (field: keyof ConnectionForm, value: string) =>
setForm((f) => ({
@@ -143,17 +148,27 @@ export function ZigbeeImportModal({ open, onClose, onAddToCanvas, onInventoryImp
onInventoryImported?.(null)
handleClose()
} else {
const res = await zigbeeApi.importNetwork(buildPayload())
setDevices(res.data.nodes)
setEdges(res.data.edges)
setChecked(new Set(res.data.nodes.map((n) => n.id)))
if (res.data.device_count === 0) {
// The backend fetches the network map in the background and we poll for
// it — a large mesh takes minutes, which no reverse proxy will hold open.
pollAbort.current?.abort()
const controller = new AbortController()
pollAbort.current = controller
const start = await zigbeeApi.importNetwork(buildPayload())
const result = await pollImportJob(
async () => (await zigbeeApi.getImportJob(start.data.job_id)).data,
{ signal: controller.signal },
)
setDevices(result.nodes)
setEdges(result.edges)
setChecked(new Set(result.nodes.map((n) => n.id)))
if (result.device_count === 0) {
toast.info('No Zigbee devices found in the network map')
} else {
toast.success(`Found ${res.data.device_count} device${res.data.device_count !== 1 ? 's' : ''}`)
toast.success(`Found ${result.device_count} device${result.device_count !== 1 ? 's' : ''}`)
}
}
} catch (err: unknown) {
if (err instanceof PollAbortedError) return
toast.error(extractError(err) ?? 'Failed to fetch Zigbee devices')
} finally {
setLoading(false)
@@ -181,6 +196,8 @@ export function ZigbeeImportModal({ open, onClose, onAddToCanvas, onInventoryImp
}
const handleClose = () => {
pollAbort.current?.abort()
pollAbort.current = null
setDevices([])
setEdges([])
setChecked(new Set())
@@ -1,11 +1,12 @@
import { describe, it, expect, vi, beforeEach } from 'vitest'
import { render, screen, fireEvent, waitFor } from '@testing-library/react'
import { render, screen, fireEvent, waitFor, act } from '@testing-library/react'
import { ZigbeeImportModal } from '../ZigbeeImportModal'
vi.mock('@/api/client', () => ({
zigbeeApi: {
testConnection: vi.fn(),
importNetwork: vi.fn(),
getImportJob: vi.fn(),
importToPending: vi.fn(),
},
}))
@@ -51,6 +52,7 @@ describe('ZigbeeImportModal', () => {
beforeEach(() => {
vi.mocked(zigbeeApi.testConnection).mockReset()
vi.mocked(zigbeeApi.importNetwork).mockReset()
vi.mocked(zigbeeApi.getImportJob).mockReset()
vi.mocked(zigbeeApi.importToPending).mockReset()
vi.mocked(toast.success).mockReset()
vi.mocked(toast.error).mockReset()
@@ -115,16 +117,33 @@ describe('ZigbeeImportModal', () => {
fireEvent.click(screen.getByRole('radio', { name: /canvas directly/i }))
}
it('fetches devices and renders them grouped by type', async () => {
/** A canvas import now answers with a job id; the payload arrives from a poll. */
const mockCanvasImport = (result: {
nodes: typeof sampleNodes
edges: { source: string; target: string }[]
device_count: number
}) => {
vi.mocked(zigbeeApi.importNetwork).mockResolvedValue({
data: { nodes: sampleNodes, edges: [], device_count: 2 },
data: { job_id: 'job-1', status: 'running' },
} as never)
vi.mocked(zigbeeApi.getImportJob).mockResolvedValue({
data: { job_id: 'job-1', status: 'done', result },
} as never)
}
const startCanvasFetch = () => {
render(<ZigbeeImportModal {...defaultProps} />)
selectCanvasMode()
const hostInput = screen.getByPlaceholderText('192.168.1.x or mqtt.local')
fireEvent.change(hostInput, { target: { value: '192.168.1.100' } })
fireEvent.change(screen.getByPlaceholderText('192.168.1.x or mqtt.local'), {
target: { value: '192.168.1.100' },
})
fireEvent.click(screen.getByRole('button', { name: /fetch devices/i }))
}
it('fetches devices and renders them grouped by type', async () => {
mockCanvasImport({ nodes: sampleNodes, edges: [], device_count: 2 })
startCanvasFetch()
await waitFor(() => {
expect(screen.getByText('Coordinator')).toBeDefined()
@@ -134,15 +153,9 @@ describe('ZigbeeImportModal', () => {
})
it('shows info toast when no devices found', async () => {
vi.mocked(zigbeeApi.importNetwork).mockResolvedValue({
data: { nodes: [], edges: [], device_count: 0 },
} as never)
mockCanvasImport({ nodes: [], edges: [], device_count: 0 })
render(<ZigbeeImportModal {...defaultProps} />)
selectCanvasMode()
const hostInput = screen.getByPlaceholderText('192.168.1.x or mqtt.local')
fireEvent.change(hostInput, { target: { value: '192.168.1.100' } })
fireEvent.click(screen.getByRole('button', { name: /fetch devices/i }))
startCanvasFetch()
await waitFor(() => {
expect(toast.info).toHaveBeenCalledWith('No Zigbee devices found in the network map')
@@ -150,15 +163,13 @@ describe('ZigbeeImportModal', () => {
})
it('calls onAddToCanvas with selected devices and closes modal', async () => {
vi.mocked(zigbeeApi.importNetwork).mockResolvedValue({
data: { nodes: sampleNodes, edges: [{ source: '0x0000', target: '0x0001' }], device_count: 2 },
} as never)
mockCanvasImport({
nodes: sampleNodes,
edges: [{ source: '0x0000', target: '0x0001' }],
device_count: 2,
})
render(<ZigbeeImportModal {...defaultProps} />)
selectCanvasMode()
const hostInput = screen.getByPlaceholderText('192.168.1.x or mqtt.local')
fireEvent.change(hostInput, { target: { value: '192.168.1.100' } })
fireEvent.click(screen.getByRole('button', { name: /fetch devices/i }))
startCanvasFetch()
await waitFor(() => screen.getByText('Coordinator'))
@@ -207,17 +218,81 @@ describe('ZigbeeImportModal', () => {
})
it('switching to canvas mode calls importNetwork and not importToPending', async () => {
vi.mocked(zigbeeApi.importNetwork).mockResolvedValue({
data: { nodes: sampleNodes, edges: [], device_count: 2 },
} as never)
mockCanvasImport({ nodes: sampleNodes, edges: [], device_count: 2 })
render(<ZigbeeImportModal {...defaultProps} />)
fireEvent.click(screen.getByRole('radio', { name: /canvas directly/i }))
const hostInput = screen.getByPlaceholderText('192.168.1.x or mqtt.local')
fireEvent.change(hostInput, { target: { value: '192.168.1.100' } })
fireEvent.click(screen.getByRole('button', { name: /fetch devices/i }))
startCanvasFetch()
await waitFor(() => expect(zigbeeApi.importNetwork).toHaveBeenCalled())
expect(zigbeeApi.importToPending).not.toHaveBeenCalled()
})
it('keeps polling the job until the map is ready (issue #380)', async () => {
vi.useFakeTimers({ shouldAdvanceTime: true })
try {
vi.mocked(zigbeeApi.importNetwork).mockResolvedValue({
data: { job_id: 'job-1', status: 'running' },
} as never)
vi.mocked(zigbeeApi.getImportJob)
.mockResolvedValueOnce({
data: { job_id: 'job-1', status: 'running', result: null },
} as never)
.mockResolvedValueOnce({
data: {
job_id: 'job-1',
status: 'done',
result: { nodes: sampleNodes, edges: [], device_count: 2 },
},
} as never)
startCanvasFetch()
await waitFor(() => expect(zigbeeApi.getImportJob).toHaveBeenCalledTimes(1))
await act(async () => {
await vi.advanceTimersByTimeAsync(2000)
})
await waitFor(() => expect(screen.getByText('router_1')).toBeDefined())
expect(zigbeeApi.getImportJob).toHaveBeenCalledWith('job-1')
} finally {
vi.useRealTimers()
}
})
it('surfaces the backend detail when the job fails', async () => {
vi.mocked(zigbeeApi.importNetwork).mockResolvedValue({
data: { job_id: 'job-1', status: 'running' },
} as never)
vi.mocked(zigbeeApi.getImportJob).mockRejectedValue({
response: { status: 504, data: { detail: 'Timed out waiting for networkmap response' } },
})
startCanvasFetch()
await waitFor(() => {
expect(toast.error).toHaveBeenCalledWith('Timed out waiting for networkmap response')
})
})
it('stops polling and stays quiet when the modal is closed mid-import', async () => {
vi.mocked(zigbeeApi.importNetwork).mockResolvedValue({
data: { job_id: 'job-1', status: 'running' },
} as never)
vi.mocked(zigbeeApi.getImportJob).mockResolvedValue({
data: { job_id: 'job-1', status: 'running', result: null },
} as never)
const { unmount } = render(<ZigbeeImportModal {...defaultProps} />)
selectCanvasMode()
fireEvent.change(screen.getByPlaceholderText('192.168.1.x or mqtt.local'), {
target: { value: '192.168.1.100' },
})
fireEvent.click(screen.getByRole('button', { name: /fetch devices/i }))
await waitFor(() => expect(zigbeeApi.getImportJob).toHaveBeenCalled())
unmount()
const callsAtUnmount = vi.mocked(zigbeeApi.getImportJob).mock.calls.length
await new Promise((r) => setTimeout(r, 50))
expect(vi.mocked(zigbeeApi.getImportJob).mock.calls.length).toBe(callsAtUnmount)
expect(toast.error).not.toHaveBeenCalled()
})
})
@@ -1,6 +1,7 @@
import { describe, it, expect, beforeEach } from 'vitest'
import { useCanvasStore } from '@/stores/canvasStore'
import { makeNode } from '@/test/factories'
import { serializeNode, deserializeApiNode, type ApiNode } from '@/utils/canvasSerializer'
function resetStore() {
useCanvasStore.setState({
@@ -133,3 +134,187 @@ describe('canvasStore — zone (groupRect) parenting', () => {
expect(detached.position).toEqual({ x: 160, y: 220 })
})
})
describe('canvasStore — importZoneSubnet', () => {
beforeEach(resetStore)
const device = (id: string, ip: string | undefined, over: Partial<ReturnType<typeof makeNode>> = {}) => ({
...makeNode(id, { ip }),
position: { x: 900, y: 900 },
...over,
})
it('moves every free device in range into the zone and reports the count', () => {
useCanvasStore.setState({
nodes: [zone('z1'), device('n1', '192.168.1.10'), device('n2', '192.168.1.11')],
})
const moved = useCanvasStore.getState().importZoneSubnet('z1', '192.168.1.0/24')
expect(moved).toBe(2)
const nodes = useCanvasStore.getState().nodes
expect(nodes.find((n) => n.id === 'n1')!.parentId).toBe('z1')
expect(nodes.find((n) => n.id === 'n2')!.parentId).toBe('z1')
})
it('leaves out-of-range and address-less devices on the canvas', () => {
useCanvasStore.setState({
nodes: [zone('z1'), device('in', '192.168.1.10'), device('out', '10.0.0.1'), device('bare', undefined)],
})
expect(useCanvasStore.getState().importZoneSubnet('z1', '192.168.1.0/24')).toBe(1)
const nodes = useCanvasStore.getState().nodes
expect(nodes.find((n) => n.id === 'out')!.parentId).toBeUndefined()
expect(nodes.find((n) => n.id === 'bare')!.parentId).toBeUndefined()
})
it('never steals a node that already has a parent', () => {
useCanvasStore.setState({
nodes: [
zone('z1'),
zone('z2', 800, 100),
{ ...device('n1', '192.168.1.10'), parentId: 'z2' },
],
})
expect(useCanvasStore.getState().importZoneSubnet('z1', '192.168.1.0/24')).toBe(0)
expect(useCanvasStore.getState().nodes.find((n) => n.id === 'n1')!.parentId).toBe('z2')
})
it('never swallows another zone, even one carrying an IP', () => {
useCanvasStore.setState({
nodes: [zone('z1'), { ...zone('z2', 800, 100), data: { ...zone('z2').data, ip: '192.168.1.9' } }],
})
expect(useCanvasStore.getState().importZoneSubnet('z1', '192.168.1.0/24')).toBe(0)
expect(useCanvasStore.getState().nodes.find((n) => n.id === 'z2')!.parentId).toBeUndefined()
})
it('is a no-op for an invalid CIDR, a missing zone and a non-zone target', () => {
useCanvasStore.setState({ nodes: [zone('z1'), device('n1', '192.168.1.10'), device('plain', '192.168.1.11')] })
const before = useCanvasStore.getState().nodes
expect(useCanvasStore.getState().importZoneSubnet('z1', 'nonsense')).toBe(0)
expect(useCanvasStore.getState().importZoneSubnet('nope', '192.168.1.0/24')).toBe(0)
expect(useCanvasStore.getState().importZoneSubnet('plain', '192.168.1.0/24')).toBe(0)
expect(useCanvasStore.getState().nodes).toBe(before)
})
it('lays arrivals out on a non-overlapping grid inside the zone', () => {
useCanvasStore.setState({
nodes: [zone('z1'), device('n1', '192.168.1.10'), device('n2', '192.168.1.11')],
})
useCanvasStore.getState().importZoneSubnet('z1', '192.168.1.0/24')
const nodes = useCanvasStore.getState().nodes
const a = nodes.find((n) => n.id === 'n1')!.position
const b = nodes.find((n) => n.id === 'n2')!.position
expect(a).not.toEqual(b)
// Zone-relative and clear of the label band at the top.
for (const p of [a, b]) {
expect(p.x).toBeGreaterThanOrEqual(0)
expect(p.y).toBeGreaterThanOrEqual(40)
}
})
it('packs around the boxes already inside the zone', () => {
useCanvasStore.setState({
nodes: [
zone('z1'),
{ ...device('sitting', '10.0.0.1'), parentId: 'z1', position: { x: 16, y: 40 }, width: 160, height: 90 },
device('n1', '192.168.1.10'),
],
})
useCanvasStore.getState().importZoneSubnet('z1', '192.168.1.0/24')
const placed = useCanvasStore.getState().nodes.find((n) => n.id === 'n1')!.position
expect(placed).not.toEqual({ x: 16, y: 40 })
})
it('grows the zone when the arrivals overflow its height', () => {
const many = Array.from({ length: 12 }, (_, i) => device(`n${i}`, `192.168.1.${i + 10}`))
useCanvasStore.setState({ nodes: [zone('z1'), ...many] })
useCanvasStore.getState().importZoneSubnet('z1', '192.168.1.0/24')
const z = useCanvasStore.getState().nodes.find((n) => n.id === 'z1')!
expect(z.height!).toBeGreaterThan(300)
// The grown height has to survive a save/load round-trip.
const wire = serializeNode(z) as unknown as ApiNode
expect(wire.height).toBe(z.height)
const reloaded = deserializeApiNode(wire, new Map())
expect(reloaded.height).toBe(z.height)
})
it('keeps the parent ahead of its new children, as React Flow requires', () => {
useCanvasStore.setState({
nodes: [device('n1', '192.168.1.10'), zone('z1'), device('n2', '192.168.1.11')],
})
useCanvasStore.getState().importZoneSubnet('z1', '192.168.1.0/24')
const ids = useCanvasStore.getState().nodes.map((n) => n.id)
expect(ids.indexOf('z1')).toBeLessThan(ids.indexOf('n1'))
expect(ids.indexOf('z1')).toBeLessThan(ids.indexOf('n2'))
})
it('keeps an arrival that is itself a parent ahead of the children it leaves behind', () => {
// A Proxmox host matches the subnet; its VM does not move, because it
// already has a parent. The host must still be listed before the VM.
useCanvasStore.setState({
nodes: [
device('proxmox', '192.168.1.10'),
{ ...device('vm1', '10.0.0.1'), parentId: 'proxmox' },
zone('z1'),
],
})
expect(useCanvasStore.getState().importZoneSubnet('z1', '192.168.1.0/24')).toBe(1)
const nodes = useCanvasStore.getState().nodes
const ids = nodes.map((n) => n.id)
expect(nodes.find((n) => n.id === 'proxmox')!.parentId).toBe('z1')
expect(nodes.find((n) => n.id === 'vm1')!.parentId).toBe('proxmox')
expect(ids.indexOf('z1')).toBeLessThan(ids.indexOf('proxmox'))
expect(ids.indexOf('proxmox')).toBeLessThan(ids.indexOf('vm1'))
})
it('leaves an already-valid order untouched', () => {
useCanvasStore.setState({
nodes: [zone('z1'), device('a', '10.0.0.1'), device('b', '10.0.0.2'), device('hit', '192.168.1.10')],
})
useCanvasStore.getState().importZoneSubnet('z1', '192.168.1.0/24')
// Only the matched node moves in the array; the untouched ones keep their
// relative order.
const ids = useCanvasStore.getState().nodes.map((n) => n.id)
expect(ids.indexOf('a')).toBeLessThan(ids.indexOf('b'))
expect(ids[0]).toBe('z1')
})
it('is additive: a second subnet keeps the first import inside', () => {
useCanvasStore.setState({
nodes: [zone('z1'), device('n1', '192.168.1.10'), device('n2', '10.0.0.5')],
})
useCanvasStore.getState().importZoneSubnet('z1', '192.168.1.0/24')
useCanvasStore.getState().importZoneSubnet('z1', '10.0.0.0/8')
const nodes = useCanvasStore.getState().nodes
expect(nodes.find((n) => n.id === 'n1')!.parentId).toBe('z1')
expect(nodes.find((n) => n.id === 'n2')!.parentId).toBe('z1')
})
it('marks the canvas unsaved and undoes the whole import in one step', () => {
useCanvasStore.setState({
nodes: [zone('z1'), device('n1', '192.168.1.10'), device('n2', '192.168.1.11')],
})
useCanvasStore.getState().importZoneSubnet('z1', '192.168.1.0/24')
expect(useCanvasStore.getState().hasUnsavedChanges).toBe(true)
useCanvasStore.getState().undo()
const nodes = useCanvasStore.getState().nodes
expect(nodes.find((n) => n.id === 'n1')!.parentId).toBeUndefined()
expect(nodes.find((n) => n.id === 'n2')!.parentId).toBeUndefined()
})
})
+136 -1
View File
@@ -14,6 +14,7 @@ import { generateUUID } from '@/utils/uuid'
import { normalizeHandle, removedHandleIds, handleCountField, sideDefault, handleId, SIDES } from '@/utils/handleUtils'
import { applyOpacity } from '@/utils/colorUtils'
import { readHideIp, writeHideIp } from '@/utils/ipDisplay'
import { isValidCidr, isZoneSubnetCandidate } from '@/utils/subnet'
import { CONTAINER_MODE_TYPES } from '@/utils/virtualEdgeParent'
import {
changedFactFields,
@@ -29,6 +30,44 @@ type Clipboard = { nodes: Node<NodeData>[]; edges: Edge<EdgeData>[] }
/** Resolve a node's effective parent id from either the RF field or domain data. */
const parentIdOf = (n: Node<NodeData>): string | undefined => n.parentId ?? n.data.parent_id ?? undefined
/**
* Reorder so every node follows its parent, which is what React Flow needs to
* resolve nesting — a child listed first renders detached and logs a
* parent-not-found error. Order is otherwise preserved: a list that is already
* valid comes back untouched. A parent cycle terminates instead of recursing.
*/
function orderParentsFirst(nodes: Node<NodeData>[]): Node<NodeData>[] {
const byId = new Map(nodes.map((n) => [n.id, n]))
const emitted = new Set<string>()
const visiting = new Set<string>()
const out: Node<NodeData>[] = []
const visit = (n: Node<NodeData>) => {
if (emitted.has(n.id) || visiting.has(n.id)) return
visiting.add(n.id)
const parent = n.parentId ? byId.get(n.parentId) : undefined
if (parent) visit(parent)
visiting.delete(n.id)
emitted.add(n.id)
out.push(n)
}
for (const n of nodes) visit(n)
return out
}
// Zone subnet import: the grid the arrivals are packed on, in zone-relative
// pixels. A cell is one node box plus the gap that follows it.
const DEFAULT_ZONE_WIDTH = 360
const DEFAULT_ZONE_HEIGHT = 240
const ZONE_CELL_W = 180
const ZONE_CELL_H = 110
const ZONE_CELL_GAP = 20
const ZONE_PAD_X = 32
// Leaves the label band at the top of the zone clear.
const ZONE_PAD_TOP = 40
const ZONE_PAD_BOTTOM = 16
/**
* Whether a node change represents a real user edit that should dirty the canvas.
* Excludes:
@@ -198,6 +237,9 @@ interface CanvasState {
addToGroup: (groupId: string, childId: string) => void
addToContainer: (containerId: string, childId: string) => void
addToZone: (zoneId: string, childId: string) => void
/** Move every free node whose IP falls in `cidr` into the zone. Returns the
* count moved; 0 for an invalid CIDR or no match. */
importZoneSubnet: (zoneId: string, cidr: string) => number
removeFromGroup: (groupId: string, childId: string) => void
markSaved: () => void
markUnsaved: () => void
@@ -218,7 +260,7 @@ interface CanvasState {
applyAllCustomStyles: (def: CustomStyleDef) => void
}
export const useCanvasStore = create<CanvasState>((rawSet) => {
export const useCanvasStore = create<CanvasState>((rawSet, get) => {
// Wrap set so any update that flips hasUnsavedChanges to true also bumps
// editSeq. This centralises the "an edit happened" signal instead of touching
// every one of the ~two dozen mutating actions. Actions that update state
@@ -969,6 +1011,99 @@ export const useCanvasStore = create<CanvasState>((rawSet) => {
}
}),
// Pull every free node whose IP falls inside `cidr` into a zone, laying them
// out on a grid in the zone's free space. Deliberately additive and one-shot:
// the CIDR is never stored, nothing is ever ejected, and running it twice with
// two subnets leaves both sets inside. Undo reverses the whole import at once.
importZoneSubnet: (zoneId, cidr) => {
const state = get()
const zone = state.nodes.find((n) => n.id === zoneId)
if (!zone || zone.data.type !== 'groupRect') return 0
if (!isValidCidr(cidr)) return 0
const matches = state.nodes.filter((n) => isZoneSubnetCandidate(n, cidr, zoneId))
if (matches.length === 0) return 0
const moved = new Set(matches.map((n) => n.id))
const zoneWidth = zone.width ?? DEFAULT_ZONE_WIDTH
const zoneHeight = zone.height ?? DEFAULT_ZONE_HEIGHT
// Boxes already inside the zone, in zone-relative coordinates. New arrivals
// are packed around them rather than on top of them.
const occupied = state.nodes
.filter((n) => n.parentId === zoneId && !moved.has(n.id))
.map((n) => ({
x: n.position.x,
y: n.position.y,
w: n.width ?? ZONE_CELL_W - ZONE_CELL_GAP,
h: n.height ?? ZONE_CELL_H - ZONE_CELL_GAP,
}))
const columns = Math.max(1, Math.floor((zoneWidth - ZONE_PAD_X) / ZONE_CELL_W))
const overlaps = (x: number, y: number) =>
occupied.some(
(b) =>
x < b.x + b.w &&
x + ZONE_CELL_W - ZONE_CELL_GAP > b.x &&
y < b.y + b.h &&
y + ZONE_CELL_H - ZONE_CELL_GAP > b.y,
)
// Row-major scan for the first free cell per arrival. Rows keep going past
// the current height; the zone grows at the end to cover the last one used.
const placements = new Map<string, { x: number; y: number }>()
let cursor = 0
let maxBottom = 0
for (const node of matches) {
let x = 0
let y = 0
for (;;) {
x = ZONE_PAD_X / 2 + (cursor % columns) * ZONE_CELL_W
y = ZONE_PAD_TOP + Math.floor(cursor / columns) * ZONE_CELL_H
cursor += 1
if (!overlaps(x, y)) break
}
placements.set(node.id, { x, y })
occupied.push({ x, y, w: ZONE_CELL_W - ZONE_CELL_GAP, h: ZONE_CELL_H - ZONE_CELL_GAP })
maxBottom = Math.max(maxBottom, y + ZONE_CELL_H)
}
const grownHeight = Math.max(zoneHeight, maxBottom + ZONE_PAD_BOTTOM)
set((s) => {
const updated = s.nodes.map((n) => {
// `height` is the only field to write: the serializer persists it to
// the real `nodes.height` column, and keeps it out of the style blob.
if (n.id === zoneId) {
return grownHeight === zoneHeight ? n : { ...n, height: grownHeight }
}
const at = placements.get(n.id)
if (!at) return n
return {
...n,
parentId: zoneId,
position: at,
selected: false,
data: { ...n.data, parent_id: zoneId },
}
})
return {
// React Flow requires a parent to precede its children in the array.
// Reordering the whole list covers both directions at once: the zone
// ahead of its new children, and an arrival that is itself a parent
// (a Proxmox host with nested VMs, whose children stay put because
// they already have a parent) ahead of the children it left behind.
nodes: orderParentsFirst(updated),
hasUnsavedChanges: true,
past: [...s.past.slice(-49), { nodes: s.nodes, edges: s.edges }],
future: [],
}
})
return matches.length
},
// Release a single child from a group back to the canvas. Group stays.
removeFromGroup: (groupId, childId) =>
set((state) => {
@@ -165,7 +165,7 @@ describe('serializeNode — regular node', () => {
// ── serializeNode — groupRect ─────────────────────────────────────────────────
describe('serializeNode — groupRect', () => {
it('stores dimensions inside custom_colors', () => {
it('stores dimensions in the width/height columns, like every other node type', () => {
const node = makeRfNode({
type: 'groupRect',
data: { label: 'Zone A', type: 'groupRect', status: 'unknown', services: [] },
@@ -173,8 +173,25 @@ describe('serializeNode — groupRect', () => {
height: 250,
})
const result = serializeNode(node)
expect((result.custom_colors as Record<string, unknown>).width).toBe(400)
expect((result.custom_colors as Record<string, unknown>).height).toBe(250)
expect(result.width).toBe(400)
expect(result.height).toBe(250)
})
it('keeps the size out of the style blob, so the two cannot drift apart', () => {
const node = makeRfNode({
type: 'groupRect',
// A zone loaded before the move still carries the legacy keys in memory.
data: {
label: 'Z', type: 'groupRect', status: 'unknown', services: [],
custom_colors: { border: '#aaa', width: 111, height: 222 },
},
width: 400,
height: 250,
})
const cc = serializeNode(node).custom_colors as Record<string, unknown>
expect(cc.width).toBeUndefined()
expect(cc.height).toBeUndefined()
expect(cc.border).toBe('#aaa')
})
it('prefers explicit width/height over measured, falling back to measured per-axis', () => {
@@ -184,15 +201,15 @@ describe('serializeNode — groupRect', () => {
}
const result = serializeNode(node)
// Explicit width wins; height has no explicit value so it uses measured.
expect((result.custom_colors as Record<string, unknown>).width).toBe(400)
expect((result.custom_colors as Record<string, unknown>).height).toBe(260)
expect(result.width).toBe(400)
expect(result.height).toBe(260)
})
it('falls back to defaults when no dimensions available', () => {
const node = makeRfNode({ type: 'groupRect', data: { label: 'Z', type: 'groupRect', status: 'unknown', services: [] } })
const result = serializeNode(node)
expect((result.custom_colors as Record<string, unknown>).width).toBe(360)
expect((result.custom_colors as Record<string, unknown>).height).toBe(240)
expect(result.width).toBe(360)
expect(result.height).toBe(240)
})
it('preserves existing custom_colors fields alongside dimensions', () => {
@@ -205,8 +222,8 @@ describe('serializeNode — groupRect', () => {
const cc = result.custom_colors as Record<string, unknown>
expect(cc.border).toBe('#aaa')
expect(cc.z_order).toBe(2)
expect(cc.width).toBe(300)
expect(cc.height).toBe(200)
expect(result.width).toBe(300)
expect(result.height).toBe(200)
})
})
@@ -420,9 +437,9 @@ describe('deserializeApiNode — regular node', () => {
describe('deserializeApiNode — groupRect', () => {
const emptyMap = new Map<string, boolean>()
it('restores width/height from custom_colors', () => {
it('restores width/height from the columns', () => {
const result = deserializeApiNode(
makeApiNode({ type: 'groupRect', custom_colors: { width: 400, height: 250, z_order: 2 } }),
makeApiNode({ type: 'groupRect', width: 400, height: 250, custom_colors: { z_order: 2 } }),
emptyMap,
)
expect(result.width).toBe(400)
@@ -430,7 +447,27 @@ describe('deserializeApiNode — groupRect', () => {
expect(result.zIndex).toBe(-8)
})
it('defaults to 360x240 when custom_colors has no dimensions', () => {
it('still reads a legacy canvas that kept its size in custom_colors', () => {
// Saved before the size moved to the columns, and not yet migrated: no
// fallback here means the zone comes back at the default 360x240.
const result = deserializeApiNode(
makeApiNode({ type: 'groupRect', custom_colors: { width: 400, height: 250, z_order: 2 } }),
emptyMap,
)
expect(result.width).toBe(400)
expect(result.height).toBe(250)
})
it('lets the columns win over a stale legacy blob', () => {
const result = deserializeApiNode(
makeApiNode({ type: 'groupRect', width: 500, height: 300, custom_colors: { width: 111, height: 222 } }),
emptyMap,
)
expect(result.width).toBe(500)
expect(result.height).toBe(300)
})
it('defaults to 360x240 when neither source has dimensions', () => {
const result = deserializeApiNode(makeApiNode({ type: 'groupRect' }), emptyMap)
expect(result.width).toBe(360)
expect(result.height).toBe(240)
@@ -0,0 +1,102 @@
import { describe, it, expect, vi } from 'vitest'
import { pollImportJob, PollAbortedError, type ImportJobState } from '../importJobPoll'
interface Payload { device_count: number }
const done = (device_count: number): ImportJobState<Payload> => ({
status: 'done',
result: { device_count },
})
const running: ImportJobState<Payload> = { status: 'running', result: null }
/** Resolves instantly so no test waits on a real timer. */
const noSleep = () => Promise.resolve()
describe('pollImportJob', () => {
it('returns the result when the first poll is already done', async () => {
const fetchJob = vi.fn().mockResolvedValue(done(2))
await expect(pollImportJob(fetchJob, { sleep: noSleep })).resolves.toEqual({ device_count: 2 })
expect(fetchJob).toHaveBeenCalledTimes(1)
})
it('keeps polling while the job is running', async () => {
const fetchJob = vi
.fn()
.mockResolvedValueOnce(running)
.mockResolvedValueOnce(running)
.mockResolvedValueOnce(done(7))
await expect(pollImportJob(fetchJob, { sleep: noSleep })).resolves.toEqual({ device_count: 7 })
expect(fetchJob).toHaveBeenCalledTimes(3)
})
it('waits the configured interval between polls', async () => {
const sleep = vi.fn().mockResolvedValue(undefined)
const fetchJob = vi.fn().mockResolvedValueOnce(running).mockResolvedValueOnce(done(1))
await pollImportJob(fetchJob, { intervalMs: 1234, sleep })
expect(sleep).toHaveBeenCalledTimes(1)
expect(sleep.mock.calls[0][0]).toBe(1234)
})
it('propagates a rejected poll so the caller sees the backend status', async () => {
const err = Object.assign(new Error('boom'), {
response: { status: 504, data: { detail: 'Timed out' } },
})
const fetchJob = vi.fn().mockResolvedValueOnce(running).mockRejectedValueOnce(err)
await expect(pollImportJob(fetchJob, { sleep: noSleep })).rejects.toBe(err)
})
it('rejects with PollAbortedError when aborted before the first poll', async () => {
const controller = new AbortController()
controller.abort()
const fetchJob = vi.fn().mockResolvedValue(done(1))
await expect(
pollImportJob(fetchJob, { signal: controller.signal, sleep: noSleep }),
).rejects.toBeInstanceOf(PollAbortedError)
expect(fetchJob).not.toHaveBeenCalled()
})
it('rejects with PollAbortedError when aborted while a poll is in flight', async () => {
const controller = new AbortController()
const fetchJob = vi.fn().mockImplementation(async () => {
controller.abort()
return running
})
await expect(
pollImportJob(fetchJob, { signal: controller.signal, sleep: noSleep }),
).rejects.toBeInstanceOf(PollAbortedError)
expect(fetchJob).toHaveBeenCalledTimes(1)
})
it('rejects when the job reports done with no payload', async () => {
const fetchJob = vi.fn().mockResolvedValue({ status: 'done', result: null })
await expect(pollImportJob(fetchJob, { sleep: noSleep })).rejects.toThrow(
'Import finished without a result',
)
})
it('default sleep resolves after the interval and rejects on abort', async () => {
vi.useFakeTimers()
try {
const fetchJob = vi.fn().mockResolvedValueOnce(running).mockResolvedValueOnce(done(1))
const promise = pollImportJob(fetchJob, { intervalMs: 50 })
await vi.advanceTimersByTimeAsync(60)
await expect(promise).resolves.toEqual({ device_count: 1 })
const controller = new AbortController()
const aborting = pollImportJob(
vi.fn().mockResolvedValue(running),
{ intervalMs: 50, signal: controller.signal },
)
await vi.advanceTimersByTimeAsync(1)
controller.abort()
await expect(aborting).rejects.toBeInstanceOf(PollAbortedError)
} finally {
vi.useRealTimers()
}
})
})
+110
View File
@@ -0,0 +1,110 @@
import { describe, it, expect } from 'vitest'
import { ipToInt, parseCidr, isValidCidr, ipInSubnet, isZoneSubnetCandidate } from '@/utils/subnet'
describe('ipToInt', () => {
it('converts a dotted quad', () => {
expect(ipToInt('0.0.0.0')).toBe(0)
expect(ipToInt('255.255.255.255')).toBe(4294967295)
expect(ipToInt('192.168.1.42')).toBe(3232235818)
})
it('tolerates surrounding whitespace', () => {
expect(ipToInt(' 10.0.0.1 ')).toBe(ipToInt('10.0.0.1'))
})
it('rejects malformed addresses', () => {
for (const bad of ['', '1.2.3', '1.2.3.4.5', '256.0.0.1', '1.2.3.-1', 'a.b.c.d', '1.2.3.4/24', '::1']) {
expect(ipToInt(bad)).toBeNull()
}
})
})
describe('parseCidr', () => {
it('masks the host bits off, so any address in the range parses to the network', () => {
expect(parseCidr('192.168.1.42/24')).toEqual(parseCidr('192.168.1.0/24'))
})
it('treats a bare address as /32', () => {
expect(parseCidr('10.0.0.7')).toEqual({ base: ipToInt('10.0.0.7'), bits: 32 })
})
it('handles /0 without the shift wrapping to -1', () => {
expect(parseCidr('0.0.0.0/0')).toEqual({ base: 0, bits: 0 })
expect(parseCidr('192.168.1.1/0')).toEqual({ base: 0, bits: 0 })
})
it('rejects junk', () => {
for (const bad of ['', ' ', '192.168.1.0/33', '192.168.1.0/x', '192.168.1.0/24/8', '192.168.1.0/', 'not-an-ip/24']) {
expect(parseCidr(bad)).toBeNull()
}
})
it('rejects IPv6 — out of scope, so the UI can say why', () => {
expect(parseCidr('2001:db8::/32')).toBeNull()
expect(isValidCidr('2001:db8::/32')).toBe(false)
})
})
describe('ipInSubnet', () => {
it('matches inside a /24 and rejects the neighbours', () => {
expect(ipInSubnet('192.168.1.1', '192.168.1.0/24')).toBe(true)
expect(ipInSubnet('192.168.1.255', '192.168.1.0/24')).toBe(true)
expect(ipInSubnet('192.168.2.1', '192.168.1.0/24')).toBe(false)
expect(ipInSubnet('192.168.0.255', '192.168.1.0/24')).toBe(false)
})
it('honours wider and narrower prefixes', () => {
expect(ipInSubnet('10.4.9.2', '10.0.0.0/8')).toBe(true)
expect(ipInSubnet('11.4.9.2', '10.0.0.0/8')).toBe(false)
expect(ipInSubnet('10.0.0.7', '10.0.0.7/32')).toBe(true)
expect(ipInSubnet('10.0.0.8', '10.0.0.7/32')).toBe(false)
expect(ipInSubnet('8.8.8.8', '0.0.0.0/0')).toBe(true)
})
it('strips a prefix or port suffix off the node address', () => {
expect(ipInSubnet('192.168.1.5/24', '192.168.1.0/24')).toBe(true)
expect(ipInSubnet('192.168.1.5:8006', '192.168.1.0/24')).toBe(true)
})
it('never matches a missing or unparseable address', () => {
expect(ipInSubnet(undefined, '192.168.1.0/24')).toBe(false)
expect(ipInSubnet(null, '192.168.1.0/24')).toBe(false)
expect(ipInSubnet('', '192.168.1.0/24')).toBe(false)
expect(ipInSubnet('fe80::1', '192.168.1.0/24')).toBe(false)
})
it('never matches against an invalid CIDR', () => {
expect(ipInSubnet('192.168.1.5', 'nonsense')).toBe(false)
})
})
describe('isZoneSubnetCandidate', () => {
const node = (over: Partial<{ id: string; parentId?: string; type: string; ip?: string }> = {}) => ({
id: over.id ?? 'n1',
parentId: over.parentId,
data: { type: over.type ?? 'server', ip: 'ip' in over ? over.ip : '192.168.1.5' },
})
it('accepts a free, addressed device in range', () => {
expect(isZoneSubnetCandidate(node(), '192.168.1.0/24', 'z1')).toBe(true)
})
it('leaves an already-parented node with the parent the user gave it', () => {
expect(isZoneSubnetCandidate(node({ parentId: 'g1' }), '192.168.1.0/24', 'z1')).toBe(false)
})
it('skips canvas furniture', () => {
for (const type of ['groupRect', 'group', 'text']) {
expect(isZoneSubnetCandidate(node({ type }), '192.168.1.0/24', 'z1')).toBe(false)
}
})
it('never treats the zone itself as a candidate', () => {
expect(isZoneSubnetCandidate(node({ id: 'z1', type: 'server' }), '192.168.1.0/24', 'z1')).toBe(false)
})
it('rejects an out-of-range or address-less node', () => {
expect(isZoneSubnetCandidate(node({ ip: '10.0.0.1' }), '192.168.1.0/24', 'z1')).toBe(false)
expect(isZoneSubnetCandidate(node({ ip: undefined }), '192.168.1.0/24', 'z1')).toBe(false)
})
})
+24 -5
View File
@@ -64,6 +64,18 @@ export interface ApiEdge {
// ── Serialization (RF node → API save payload) ───────────────────────────────
/** Drop the legacy geometry keys from a zone's style blob — its size lives in
* the `width`/`height` columns now, and a stale copy here would be the one an
* older canvas reads back. */
function omitZoneSize(
colors: NodeData['custom_colors'],
): Record<string, unknown> {
if (!colors) return {}
return Object.fromEntries(
Object.entries(colors).filter(([key]) => key !== 'width' && key !== 'height'),
)
}
export function serializeNode(
n: Node<NodeData>,
factsBaseline?: FactsBaseline,
@@ -89,10 +101,14 @@ export function serializeNode(
custom_icon: null,
pos_x: n.position.x,
pos_y: n.position.y,
// A zone's size goes in the real columns, like every other node type.
// It used to live in the custom_colors blob; `width`/`height` are
// stripped from it below so the two cannot drift apart, and a canvas
// saved before the change is migrated by `_backfill_zone_size`.
width: n.width ?? n.measured?.width ?? 360,
height: n.height ?? n.measured?.height ?? 240,
custom_colors: {
...n.data.custom_colors,
width: n.width ?? n.measured?.width ?? 360,
height: n.height ?? n.measured?.height ?? 240,
...omitZoneSize(n.data.custom_colors),
// Stash collapse state inside custom_colors so the API/YAML blob does
// not need a new column. Hoisted back to `data.collapsed` on load.
collapsed: n.data.collapsed ?? false,
@@ -182,8 +198,11 @@ export function deserializeApiNode(
): Node<NodeData> {
const normalizedType = n.type === 'docker' ? 'docker_host' : n.type
if (n.type === 'groupRect') {
const w = (n.custom_colors?.width as number | undefined) ?? 360
const h = (n.custom_colors?.height as number | undefined) ?? 240
// Prefer the real columns; fall back to the custom_colors stash for a
// canvas saved before the size moved out of it, and which the backend
// backfill has not reached (a payload from an older server, an import).
const w = n.width ?? (n.custom_colors?.width as number | undefined) ?? 360
const h = n.height ?? (n.custom_colors?.height as number | undefined) ?? 240
const z = (n.custom_colors?.z_order as number | undefined) ?? 1
// Hoist persisted collapse flag from the custom_colors stash to a
// first-class field on NodeData. Tolerates legacy saves that already had
+73
View File
@@ -0,0 +1,73 @@
/**
* Polling helper for backend import jobs that answer immediately and finish later.
*
* A Zigbee canvas import has to fetch a Z2M networkmap, which on a large mesh
* takes minutes — longer than the read timeout of any reverse proxy in front of
* the API. The backend therefore returns a job id and does the work in the
* background; the client polls short requests until the payload is ready.
*/
export class PollAbortedError extends Error {
constructor() {
super('Import polling aborted')
this.name = 'PollAbortedError'
}
}
export interface ImportJobState<T> {
status: string
result: T | null
}
export interface PollImportJobOptions {
/** Delay between polls, in ms. */
intervalMs?: number
/** Abort the loop (modal closed, component unmounted). */
signal?: AbortSignal
/** Injected in tests so no timer actually runs. */
sleep?: (ms: number, signal?: AbortSignal) => Promise<void>
}
const defaultSleep = (ms: number, signal?: AbortSignal) =>
new Promise<void>((resolve, reject) => {
const timer = setTimeout(() => {
signal?.removeEventListener('abort', onAbort)
resolve()
}, ms)
const onAbort = () => {
clearTimeout(timer)
reject(new PollAbortedError())
}
if (signal?.aborted) {
onAbort()
return
}
signal?.addEventListener('abort', onAbort, { once: true })
})
/**
* Poll `fetchJob` until it reports a terminal status, then resolve its result.
*
* A failed job surfaces as a rejected `fetchJob` call (the backend replays the
* original status code), so errors propagate untouched to the caller. Rejects
* with `PollAbortedError` if the signal fires, and with a plain Error if the
* job reports done without a payload.
*/
export async function pollImportJob<T>(
fetchJob: () => Promise<ImportJobState<T>>,
{ intervalMs = 2000, signal, sleep = defaultSleep }: PollImportJobOptions = {},
): Promise<T> {
for (;;) {
if (signal?.aborted) throw new PollAbortedError()
const job = await fetchJob()
if (signal?.aborted) throw new PollAbortedError()
if (job.status !== 'running') {
if (job.result == null) throw new Error('Import finished without a result')
return job.result
}
await sleep(intervalMs, signal)
}
}
+105
View File
@@ -0,0 +1,105 @@
/**
* IPv4 subnet matching, used by the zone "Import devices by subnet" action.
*
* IPv4 only on purpose: a homelab zone is drawn around a LAN segment, and the
* canvas has no IPv6 grouping story yet. `isValidCidr` rejects an IPv6 CIDR so
* the modal can say why instead of silently matching nothing.
*/
/** A parsed CIDR: the network address as a 32-bit int, plus the prefix length. */
export interface ParsedCidr {
base: number
bits: number
}
/**
* "192.168.1.42" → 3232235818. Null for anything that is not four decimal
* octets in 0..255 — no leading zeros, no shorthand.
*/
export function ipToInt(ip: string): number | null {
const parts = ip.trim().split('.')
if (parts.length !== 4) return null
let acc = 0
for (const part of parts) {
if (!/^\d{1,3}$/.test(part)) return null
const n = Number(part)
if (n > 255) return null
acc = acc * 256 + n
}
return acc
}
/**
* Parse "192.168.1.0/24". The host bits of the given address are masked off, so
* "192.168.1.42/24" and "192.168.1.0/24" parse to the same network.
*
* A bare address with no "/" is treated as /32 — one host.
*/
export function parseCidr(cidr: string): ParsedCidr | null {
const trimmed = cidr.trim()
if (!trimmed) return null
const [addr, prefix, ...rest] = trimmed.split('/')
if (rest.length > 0) return null
const ip = ipToInt(addr)
if (ip === null) return null
let bits = 32
if (prefix !== undefined) {
if (!/^\d{1,2}$/.test(prefix)) return null
bits = Number(prefix)
if (bits > 32) return null
}
// A /0 mask would be `-1 << 32`, which JS evaluates as `-1 << 0` = -1 — the
// shift count wraps mod 32. Special-cased so "0.0.0.0/0" matches everything.
const mask = bits === 0 ? 0 : (0xffffffff << (32 - bits)) >>> 0
return { base: (ip & mask) >>> 0, bits }
}
export function isValidCidr(cidr: string): boolean {
return parseCidr(cidr) !== null
}
/**
* Does `ip` fall inside `cidr`?
*
* `ip` comes straight off node data, so it may carry a prefix ("10.0.0.5/24")
* or a port ("10.0.0.5:8006"); both suffixes are stripped. Missing or
* unparseable addresses never match.
*/
export function ipInSubnet(ip: string | null | undefined, cidr: string): boolean {
const parsed = parseCidr(cidr)
if (!parsed) return false
if (!ip) return false
const bare = ip.trim().split('/')[0].split(':')[0]
const value = ipToInt(bare)
if (value === null) return false
const mask = parsed.bits === 0 ? 0 : (0xffffffff << (32 - parsed.bits)) >>> 0
return ((value & mask) >>> 0) === parsed.base
}
/**
* The rule the zone subnet import runs on, shared by the store action and the
* modal's match-count preview so the number shown is the number that moves.
*
* A candidate must be free (no parent — a node already nested in a group, a
* container host or another zone keeps the parent the user gave it), must not
* be canvas furniture (which describes nothing physical and so has no IP worth
* matching), and must have an IP inside the range.
*/
const FURNITURE_TYPES = new Set(['groupRect', 'group', 'text'])
export function isZoneSubnetCandidate(
node: { id: string; parentId?: string; data: { type: string; ip?: string } },
cidr: string,
zoneId?: string,
): boolean {
if (node.id === zoneId) return false
if (node.parentId) return false
if (FURNITURE_TYPES.has(node.data.type)) return false
return ipInSubnet(node.data.ip, cidr)
}