## Overview Adds **AI document classification** and a **classification-aware Files sidebar**: uploaded documents are automatically tagged with document-type labels (Invoice, Contract, Lab report, …), and the sidebar groups files under editable parent categories so a large library stays navigable. > [!IMPORTANT] > **This feature only runs in the SaaS build.** Classification depends on the AI engine and team-scoped label storage, so it's gated to SaaS end-to-end: > - The sidebar grouping is a `saas/`-layer override of the `fileSidebarGrouping` seam; every other build (OSS core, self-hosted proprietary, desktop) gets the null stub and renders the **unchanged flat, recency-sorted list** — no categories, no "Other", no picker. > - The classify/labels backend endpoints are gated on `policies.enabled` (on in SaaS) and live in `app/proprietary`, so they're absent from pure OSS and dormant in self-hosted unless explicitly enabled. > - The Python classifier is reached only via that gated path. > > Shared-layer changes that do compile everywhere are inert without the engine (dormant schema/field additions) or intentional (`GetInfoOnPDF` surfacing custom metadata). ## What it does - **Classifier (engine):** reads the first/last two pages of a PDF and assigns document-type labels from an allowed vocabulary. Labels are deliberately document-*type* descriptors — no deep-content/PII detection, since only a page window is read. - **Team label vocabulary:** ~270 built-in defaults across ~15 families, seeded per team. Editable by team leaders/admins in the Classification policy settings (import/export/reset). Team-scoped and shared; **per-user personal labels are intentionally out of scope** — the vocabulary is team-level only. - **Sidebar categories:** files group under parent categories (Financial, Legal, Medical, …), busiest-first, collapsible, with a "Recent" group on top and an "Other" group for anything uncategorised. The category structure (names, icons, membership, custom categories) is **device-local and user-editable** via a "Customize" picker — the only per-user personalization; it never changes the team's label vocabulary. - Classification results are written to PDF metadata (`StirlingPDFClassification`), read back to keep files in their groups without re-parsing. ## Architecture Spans all three layers, mirroring the existing policy/source subsystem conventions: - **`frontend/editor`** — sidebar grouping seam + SaaS override, category manager, labels editor, icon palette, file grouping, tests, `en-US` i18n. - **`app/proprietary` + `app/common` + `app/core`** — `ClassifyLabelController`, team-scoped `ClassificationLabelStore` (Jpa + in-process impls, same shape as `PolicyStore`/`SourceStore`), metadata read/write. - **`engine`** — the document-classifier agent, contracts, routes, tests. ## Screenshots **Files sidebar — grouped by category (SaaS)** ### Loading view <img width="2056" height="1046" alt="Screenshot 2026-07-07 at 5 12 56 PM" src="https://github.com/user-attachments/assets/1d712da5-50ae-4349-b0cd-e62665c3ec0c" /> ### Organized in the sidebar <img width="2056" height="1045" alt="Screenshot 2026-07-07 at 5 14 05 PM" src="https://github.com/user-attachments/assets/3ea4fe21-da51-4cea-bc3a-18ce040d3d05" /> **Customize categories picker** ### Personal settings to change how labels are grouped in an individual users editor <img width="2056" height="1044" alt="Screenshot 2026-07-07 at 5 52 42 PM" src="https://github.com/user-attachments/assets/40be03ce-0f63-4d1e-b58b-cec045d01cb2" /> **Classification labels editor (team settings)** <img width="2056" height="1042" alt="Screenshot 2026-07-07 at 5 53 00 PM" src="https://github.com/user-attachments/assets/337b0739-15c9-4749-9c6b-22e3b20825b8" /> ## Testing - Frontend `task frontend:check` — green (editor + portal tests, typecheck across all flavors, lint, label-drift guard). - Backend `task backend:check` (proprietary) and `:saas:test` — green. - Engine `task engine:check` — green.
54 lines
1.9 KiB
Python
54 lines
1.9 KiB
Python
from __future__ import annotations
|
|
|
|
from pydantic import Field
|
|
|
|
from stirling.models import ApiModel
|
|
|
|
from .documents import PageText
|
|
|
|
|
|
class LabelOption(ApiModel):
|
|
"""One allowed label: a stable ``id`` (the identity) plus the human ``name``
|
|
the model actually reasons over. The model picks names; the engine maps its
|
|
answer back to ids, so the result is language- and rename-stable."""
|
|
|
|
id: str = Field(min_length=1)
|
|
name: str = Field(min_length=1)
|
|
|
|
|
|
class ClassifyDocumentRequest(ApiModel):
|
|
"""Classify one document from its page text.
|
|
|
|
The caller sends the page text directly — typically just the bounded window
|
|
(first/last pages), since the classifier reads no more than that. There is no
|
|
ingestion or RAG step.
|
|
|
|
``labels`` is the allowed vocabulary for this request — the caller (the
|
|
backend) always supplies it from the team's stored label set, each an
|
|
``{id, name}`` pair. The engine holds no vocabulary of its own, so at least
|
|
one label is required.
|
|
"""
|
|
|
|
file_name: str = Field(min_length=1)
|
|
pages: list[PageText] = Field(default_factory=list)
|
|
labels: list[LabelOption] = Field(min_length=1)
|
|
|
|
|
|
class DocumentClassificationResponse(ApiModel):
|
|
"""Terminal classification result.
|
|
|
|
``labels`` is the subset of the allowed vocabulary the model assigned to the
|
|
document, as label **ids**: at most five entries, deduplicated, in the
|
|
model's order. An empty list is a valid answer — nothing in the vocabulary
|
|
fit. This is a plain answer from a dedicated endpoint — it carries no
|
|
``outcome`` discriminator (it isn't one of the orchestrator's
|
|
WorkflowOutcome-routed union responses).
|
|
"""
|
|
|
|
labels: list[str] = Field(default_factory=list)
|
|
|
|
|
|
# Only one response shape today; kept as a named alias so routes and agents have
|
|
# a stable response type to import.
|
|
ClassifyDocumentResponse = DocumentClassificationResponse
|