Classifier Policy (#6898)
## Overview Adds **AI document classification** and a **classification-aware Files sidebar**: uploaded documents are automatically tagged with document-type labels (Invoice, Contract, Lab report, …), and the sidebar groups files under editable parent categories so a large library stays navigable. > [!IMPORTANT] > **This feature only runs in the SaaS build.** Classification depends on the AI engine and team-scoped label storage, so it's gated to SaaS end-to-end: > - The sidebar grouping is a `saas/`-layer override of the `fileSidebarGrouping` seam; every other build (OSS core, self-hosted proprietary, desktop) gets the null stub and renders the **unchanged flat, recency-sorted list** — no categories, no "Other", no picker. > - The classify/labels backend endpoints are gated on `policies.enabled` (on in SaaS) and live in `app/proprietary`, so they're absent from pure OSS and dormant in self-hosted unless explicitly enabled. > - The Python classifier is reached only via that gated path. > > Shared-layer changes that do compile everywhere are inert without the engine (dormant schema/field additions) or intentional (`GetInfoOnPDF` surfacing custom metadata). ## What it does - **Classifier (engine):** reads the first/last two pages of a PDF and assigns document-type labels from an allowed vocabulary. Labels are deliberately document-*type* descriptors — no deep-content/PII detection, since only a page window is read. - **Team label vocabulary:** ~270 built-in defaults across ~15 families, seeded per team. Editable by team leaders/admins in the Classification policy settings (import/export/reset). Team-scoped and shared; **per-user personal labels are intentionally out of scope** — the vocabulary is team-level only. - **Sidebar categories:** files group under parent categories (Financial, Legal, Medical, …), busiest-first, collapsible, with a "Recent" group on top and an "Other" group for anything uncategorised. The category structure (names, icons, membership, custom categories) is **device-local and user-editable** via a "Customize" picker — the only per-user personalization; it never changes the team's label vocabulary. - Classification results are written to PDF metadata (`StirlingPDFClassification`), read back to keep files in their groups without re-parsing. ## Architecture Spans all three layers, mirroring the existing policy/source subsystem conventions: - **`frontend/editor`** — sidebar grouping seam + SaaS override, category manager, labels editor, icon palette, file grouping, tests, `en-US` i18n. - **`app/proprietary` + `app/common` + `app/core`** — `ClassifyLabelController`, team-scoped `ClassificationLabelStore` (Jpa + in-process impls, same shape as `PolicyStore`/`SourceStore`), metadata read/write. - **`engine`** — the document-classifier agent, contracts, routes, tests. ## Screenshots **Files sidebar — grouped by category (SaaS)** ### Loading view <img width="2056" height="1046" alt="Screenshot 2026-07-07 at 5 12 56 PM" src="https://github.com/user-attachments/assets/1d712da5-50ae-4349-b0cd-e62665c3ec0c" /> ### Organized in the sidebar <img width="2056" height="1045" alt="Screenshot 2026-07-07 at 5 14 05 PM" src="https://github.com/user-attachments/assets/3ea4fe21-da51-4cea-bc3a-18ce040d3d05" /> **Customize categories picker** ### Personal settings to change how labels are grouped in an individual users editor <img width="2056" height="1044" alt="Screenshot 2026-07-07 at 5 52 42 PM" src="https://github.com/user-attachments/assets/40be03ce-0f63-4d1e-b58b-cec045d01cb2" /> **Classification labels editor (team settings)** <img width="2056" height="1042" alt="Screenshot 2026-07-07 at 5 53 00 PM" src="https://github.com/user-attachments/assets/337b0739-15c9-4749-9c6b-22e3b20825b8" /> ## Testing - Frontend `task frontend:check` — green (editor + portal tests, typecheck across all flavors, lint, label-drift guard). - Backend `task backend:check` (proprietary) and `:saas:test` — green. - Engine `task engine:check` — green.
This commit is contained in:
@@ -0,0 +1,86 @@
|
||||
// Default classification labels. The SOURCE OF TRUTH is the co-located static
|
||||
// JSON (`classificationLabels.json`), imported here and shaped into typed
|
||||
// objects — edit THAT file, not this one. This is the ONLY copy of the label
|
||||
// data: it seeds a team's editable set and drives the sidebar's grouping,
|
||||
// icons, and display names. Neither the backend nor the engine keeps a copy.
|
||||
//
|
||||
// NOTE: this is only the built-in default vocabulary. A team's own
|
||||
// (admin-editable) labels live in the backend store and are what the backend
|
||||
// sends to the engine per classify request — the engine holds no vocabulary of
|
||||
// its own. Edits to a team's set reach Python on the next run, independent of
|
||||
// this file.
|
||||
//
|
||||
// `labels` is the flat set: each has a stable `id` (slug — the value on the wire,
|
||||
// in storage and keyed on) and a human `name` (display, translatable via
|
||||
// `classification.labels.<id>`); `icon` is presentational only, the engine never
|
||||
// sees it. `families` are presentational sidebar roll-ups (referencing labels by
|
||||
// id) the classifier never sees.
|
||||
|
||||
import labelsData from "@app/data/classificationLabels.json";
|
||||
|
||||
export interface ClassificationLabel {
|
||||
/** Stable identity (slug) — the value on the wire, stored on the doc, and
|
||||
* keyed on. Independent of the (translatable) display name. */
|
||||
id: string;
|
||||
/** Human display name; the en-US default for `classification.labels.<id>`. */
|
||||
name: string;
|
||||
/** Material Symbols icon key (see `labelIcons.ts`). */
|
||||
icon?: string;
|
||||
}
|
||||
|
||||
export interface LabelFamily {
|
||||
/** Stable identity for sidebar prefs — never rename once shipped. */
|
||||
id: string;
|
||||
/** Group header text shown in the sidebar and the group picker. */
|
||||
name: string;
|
||||
/** Material Symbols icon key (see `labelIcons.ts`). */
|
||||
icon: string;
|
||||
/** The built-in labels this family rolls up in the sidebar. */
|
||||
labels: ClassificationLabel[];
|
||||
}
|
||||
|
||||
/** Shape of `classificationLabels.json` — the flat label set plus the
|
||||
* presentational family grouping (which references labels by id). */
|
||||
interface LabelsFile {
|
||||
labels: ClassificationLabel[];
|
||||
families: { id: string; name: string; icon: string; labelIds: string[] }[];
|
||||
}
|
||||
|
||||
const data = labelsData as LabelsFile;
|
||||
|
||||
/** Flat default label set — file order, as the classifier/team-seed sees it. */
|
||||
export const DEFAULT_CLASSIFICATION_LABELS: ClassificationLabel[] = data.labels;
|
||||
|
||||
const LABEL_BY_ID = new Map(data.labels.map((label) => [label.id, label]));
|
||||
|
||||
/**
|
||||
* Built-in families with their labels resolved from the flat set by id. Throws
|
||||
* at module load if a family references an unknown id, so a bad hand-edit of the
|
||||
* JSON fails fast rather than silently dropping a label from its group.
|
||||
*/
|
||||
export const LABEL_FAMILIES: LabelFamily[] = data.families.map((family) => ({
|
||||
id: family.id,
|
||||
name: family.name,
|
||||
icon: family.icon,
|
||||
labels: family.labelIds.map((id) => {
|
||||
const label = LABEL_BY_ID.get(id);
|
||||
if (!label) {
|
||||
throw new Error(
|
||||
`Classification family "${family.id}" references unknown label id "${id}"`,
|
||||
);
|
||||
}
|
||||
return label;
|
||||
}),
|
||||
}));
|
||||
|
||||
/**
|
||||
* Stable slug id from a label's canonical (English) name — used to derive an id
|
||||
* for a NEW custom label the user types. Built-in ids are authored in the JSON;
|
||||
* this must stay in sync with the slug rule used to generate them.
|
||||
*/
|
||||
export function labelId(name: string): string {
|
||||
return name
|
||||
.toLowerCase()
|
||||
.replace(/[^a-z0-9]+/g, "-")
|
||||
.replace(/^-+|-+$/g, "");
|
||||
}
|
||||
Reference in New Issue
Block a user