Compare commits

..
Author SHA1 Message Date
Connor Yoh 238a8bf81c Open processor "Open in browser" CTAs in a new tab
The home-hero and download-modal "Open in browser" buttons used
window.location.href, which replaced the current tab. Switch to
window.open(..., "_blank", "noopener,noreferrer") to match the
existing EditorStatusCard behaviour.
2026-07-15 18:09:51 +01:00
Anthony StirlingandGitHub bef83d80a1 Patch CVEs in engine Python dependencies (#6804)
# Description of Changes

- Patch CVEs in engine Python dependencies (43 alerts): `starlette`
1.3.1, `cryptography` 49.0.0, `pyjwt` 2.13.0, `urllib3` 2.7.0, `aiohttp`
3.14.1, `python-multipart` 0.0.32, `langchain-core` 1.4.8, `langsmith`
0.9.1, `authlib` 1.7.2, `requests` 2.34.2, `idna` 3.18, `pytest` 9.1.1,
`pygments` 2.20.0, `pydantic-settings` 2.14.2
- Cap `pydantic-ai` `<2.0.0` and bump to 1.107.0 (1.99.0 patches
CVE-2026-46678; 2.0 is a separate major migration)

---

## Checklist

### General

- [x] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [x] I have performed a self-review of my own code
- [x] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs (if functionality has heavily
changed)
- [ ] I have read the section Add New Translation Tags (for new
translation tags only)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached

### Testing (if applicable)

- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [ ] I have tested my changes locally
2026-07-15 14:49:33 +00:00
Anthony StirlingandGitHub 8ed1ff152b Hide processor pipelines stat boxes during loading/empty state (#6983)
# Description of Changes

Gate the processor Pipelines KPI strip on having real pipelines, so the
loading and empty states no longer flash a row of empty `—` stat boxes.
Matches the Policies view; Sources is intentionally left as-is.

---

## Checklist

### General

- [x] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [x] I have performed a self-review of my own code
- [x] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [x] I have run `task check` to verify linters, typechecks, and tests
pass
- [x] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
2026-07-15 14:49:25 +00:00
Anthony StirlingandGitHub d2c8bbfcb2 Portal Sources: consistent SVG source icons (#6995)
# Description of Changes

Replaces the odd glyph icons in the Sources connect flow with centred
stroke SVGs, and fixes the connect-modal type-tile alignment and the
inconsistent Folder chip width.

Part of a portal (processor) UI-consistency pass, split into small
focused PRs.

## Before / after

<img width="2816" height="1224" alt="before-sources-light"
src="https://github.com/user-attachments/assets/22814817-8123-4bc7-b40b-5c2ba2881415"
/>
<img width="2880" height="2000" alt="after-connect-modal"
src="https://github.com/user-attachments/assets/0e51ca78-cda0-4909-bff3-912ecd7925a3"
/>
<img width="2816" height="1224" alt="after-sources-light"
src="https://github.com/user-attachments/assets/b9fd4b8a-c45a-4518-9676-e871bbabd9b8"
/>
<img width="2880" height="2000" alt="before-connect-modal"
src="https://github.com/user-attachments/assets/55058c40-d152-4c5e-a8a0-1ca5549a121c"
/>


---

## Checklist

### General

- [x] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [x] I have performed a self-review of my own code
- [x] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [x] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [x] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
2026-07-15 14:49:21 +00:00
Anthony StirlingandGitHub da1be650ea Portal Billing: use design tokens for colours (#6996)
# Description of Changes

Swaps off-palette hard-coded blues/cyans on the usage & billing page for
shared design tokens, so button and meter colours match the rest of the
portal in both themes.

Part of a portal (processor) UI-consistency pass, split into small
focused PRs.

## Before / after
<img width="2816" height="1000" alt="after-billing-free"
src="https://github.com/user-attachments/assets/1f6acad3-56a1-47a8-b5f5-8cc25565492b"
/>
<img width="2816" height="2048" alt="after-billing-subscribed-dark"
src="https://github.com/user-attachments/assets/35a75057-bdf5-4143-bdfa-4dbe94b5429e"
/>
<img width="2816" height="2048" alt="after-billing-subscribed-light"
src="https://github.com/user-attachments/assets/f9e7eab8-7d01-4d29-b842-b5d38adf7780"
/>
<img width="2816" height="1000" alt="before-billing-free"
src="https://github.com/user-attachments/assets/c7e5eca6-a57a-4959-9781-fd664dbca576"
/>
<img width="2816" height="2004" alt="before-billing-subscribed-dark"
src="https://github.com/user-attachments/assets/d7fc08be-7b91-4803-bd50-3301b886e1c0"
/>
<img width="2816" height="2004" alt="before-billing-subscribed-light"
src="https://github.com/user-attachments/assets/ff430ec1-c3bb-4517-b6b5-2150c566eca4"
/>


---

## Checklist

### General

- [x] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [x] I have performed a self-review of my own code
- [x] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [x] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [x] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
2026-07-15 14:49:15 +00:00
Anthony StirlingandGitHub 5cb5a866ca Portal: remove unused Components section (#7005)
# Description of Changes

Deletes the unused Components catalogue section (view, cards, SDK mocks)
along with its sidebar entry and route.

Part of a portal (processor) UI-consistency pass, split into small
focused PRs.

## Before / after

<!-- paste before / after screenshots here -->

---

## Checklist

### General

- [x] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [x] I have performed a self-review of my own code
- [x] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [x] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [x] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
2026-07-15 14:49:13 +00:00
Anthony StirlingandGitHub 83e5319661 Reduce CI cost: disable Depot, gate arm64/Tauri PR builds, self-testing CI routing (#7028)
## What

CI cost/routing cleanup. Four changes, each reversible with no code
deleted.

### 1. Disable Depot repo-wide (reversible)
Depot ran on trusted (non-fork) triggers via the `is_fork` output of
`_runner-pick.yml`, driving both the `depot-*` runner selection and the
Depot docker build actions. It's now disabled everywhere behind a single
kill-switch:

- `_runner-pick.yml` gains a dedicated `use_depot` output, forced
`false` via `DEPOT_ENABLED=false`. `is_fork` stays truthful for trust
gating (e.g. `build-enterprise` skipping on forks).
- All `runs-on:` and `USE_DEPOT:` expressions now key off `use_depot`,
so every job falls back to `ubuntu-latest` + buildx.
- `settings.gradle` Depot remote build cache (`cache.depot.dev`) gated
behind `depotCacheEnabled = false`.

**Switch back on:** set `DEPOT_ENABLED=true` in `_runner-pick.yml` (and
`depotCacheEnabled = true` in `settings.gradle`). Depot then reactivates
on trusted triggers exactly as before.

### 2. arm64 PR docker build only on Dockerfile changes
`test-build-docker.yml` was building `linux/amd64,linux/arm64/v8` on
every PR matching the broad `project` filter. With Depot off, the arm64
leg runs under slow QEMU emulation on every code PR. New `dockerfiles`
path filter (`docker/**/Dockerfile*`) gates the arm64 leg: normal code
PRs build amd64 only; PRs that touch a Dockerfile still build amd64 +
arm64. arm64 is still fully exercised on the base-image publish and on
release.

### 3. Tauri PR build -> Linux only, unsigned, deb-only
The PR path built the full 3-OS matrix (Windows + macOS-universal +
Linux), plus the flaky Linux AppImage pass (#6127). PRs now build Linux
only (fastest + cheapest to compile) via a new `minimal` input on
`tauri-build.yml`: Linux deb only, no rpm, no AppImage. The full signed
multi-OS matrix still runs on release, and nightly still warms the Rust
cache with all-OS defaults (unchanged).

Tradeoff: Windows/macOS desktop build breaks are caught by nightly
(all-OS) rather than the introducing PR.

### 4. CI self-testing routing
Editing `build.yml` only matched the `project` filter, so a change to
how e2e / enterprise / tauri / engine jobs are dispatched didn't
actually run those jobs. Added a `ci` anchor (`build.yml` +
`.github/config/.files.yaml`) that every job-gating area filter now
includes, so editing the router or the filter config runs every job.
Also added the orphaned reusable workflows (`e2e-*`,
`frontend-validation`, `docker-compose-tests`, `test-build-docker`,
`check-openapi`, `check-licence`) to their area filters so editing a
reusable workflow self-tests.

## Validation
- All workflow YAML + `.files.yaml` parse; anchor resolution verified
(every job-gating filter resolves to include the `ci` paths).
- Gradle evaluates `settings.gradle` cleanly; `spotlessGradleCheck`
passes.
2026-07-15 14:49:06 +00:00
James BruntonandGitHub ed58d90ab8 Remove policies feature flag (#7031)
# Description of Changes
Removes the feature flags for enabling policies on both the backend and
frontend. We shouldn't be releasing another self-hosted release that
doesn't include policies, so it makes sense to do this now. Builds that
don't have the Processor will just not run policies because they won't
have any. Beyond that, the API should always be available, but checks
whether the user actually has the entitlements to run policies (whether
they have credits/a payment method available)
2026-07-15 14:25:25 +00:00
James BruntonandGitHub 4ee54243f3 Remove encryption from stored policies JSON (#7035)
# Description of Changes
The policies stored in the DB are currently encrypted at rest, because
one version in the past included S3 keys. These are now stored properly
in the credentials system and I've manually removed the only policy that
used S3 (it was very recently released). Since there's no S3 (or other)
credentials in the policies stored JSON now, we might as well just
decrypt them. This PR pairs with #7034 to fix the issues - #7034 makes
it resilient to crashing when attempting to load encrypted JSON that's
been encrypted with the wrong key, and this makes it so if it does load
any encrypted policies, they'll be re-saved decrypted, so we should have
a vanishingly small number of encrypted policies over time.
2026-07-15 12:28:29 +00:00
James BruntonandGitHub 350c0b796e Upgrade to TS7 official release (#6958)
# Description of Changes
Convert from the TS7 release candidate to the TS7 official release,
keeping TS6 around in [the compatibility mode suggested by
Microsoft](https://devblogs.microsoft.com/typescript/announcing-typescript-7-0/#running-side-by-side-with-typescript-6.0)
so ESLint and our scripts which rely on the TS API still work.
2026-07-15 12:21:34 +00:00
Anthony StirlingandGitHub 663bb32b2c Skip unreadable policy source/policy rows instead of crashing :) (#7034)
# Description of Changes

Thanks james for the prod issue  :) 

---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
2026-07-15 12:19:06 +00:00
EthanHealy01andGitHub b86e963c9d initial colors and theme improvements (#7009)
## What this does

Consolidates the frontend's colour/theme system into a small,
well-defined token layer and reworks the theme picker. The goal was a
minimal, scalable set of semantic tokens that the editor **and** the
Processor/portal (and Storybook) all share, plus a theme model that's
easy to reason about.

## Token architecture (`core/theme/`)

A four-file layer, imported once via `index.css`:

| File | Role |
|---|---|
| `primitives.css` | The raw palette — the **only** place literal
colours live (neutral ramps `--p-gray-*`/`--p-zinc-*` + status hues). |
| `colors.css` | ~21 semantic `--c-*` tokens (surfaces, text, borders,
primary, status) mapped from primitives per theme. **Reference these.**
|
| `compat.css` | Legacy names (`--bg-*`, `--text-*`, `--color-*`)
aliased onto `--c-*` via `:root:root` so ~200 existing files keep
working. |
| `dimensions.css` | All non-colour tokens (spacing, radius, z-index,
type, motion) — single source, resolving prior collisions. |

A blocking linter (`scripts/lint/theme-lint.mjs`, run in
`frontend:lint`) enforces "literals only in `primitives.css`" within
`core/theme/`, and has a non-blocking WCAG contrast report. See
`core/theme/README.md`.

## Theme model

- **Mode** (`light` / `dark` / `system`) and **accent** are independent.
Each mode has its own accent (`lightPrimary` / `darkPrimary`).
- The editor is always `data-app-theme="custom"`; `ThemeProvider`
injects the accent as `--user-primary` and sets `data-accent`.
- **Two accent states:**
- A **colour** (preset or custom hex) → tints every surface that hue
(whole-app theming).
- The **`default`** sentinel → neutral surfaces (white/grey light, zinc
black/grey dark) with blue buttons, no tint. (`data-accent="default"`
opts surfaces out of the tint.)
- Accent contrast guardrails (`utils/customPrimary.ts`): lightness
clamps so an accent can't collapse into the base, a contrast-picked
on-primary foreground, and an accent-as-foreground variant so accent
text is never dark-on-dark.

## Theme picker (Settings → General)

- 3×5 grid: a distinct **Default** icon chip (not a colour) + 14 curated
accents, in a dropdown per mode.
- **Custom** colour via the shared `ColorInput`, with a live gamut clamp
(`clampValue`) that refuses white/grey/black — the picker handle sticks
at the boundary and preserves the working hue at achromatic extremes.
- "Restore theme to default" resets both modes.

## Other

- Dark mode is a true neutral zinc (no navy "midnight" tint); the
Mantine dark ramp and Tailwind dark channels were neutralised to match.
- Pre-paint inline script in `index.html` applies theme + accent before
first paint (no FOUC); portal and editor now share the same
`preferences.theme` source of truth.
- High-visibility surfaces migrated to tokens (FAB, landing upload
buttons, portal hero banners); scattered per-component colour swaps were
intentionally **left for a follow-up** to keep this PR focused.

## Testing

- `task frontend:check:all` (typecheck all variants + eslint + prettier
+ colour-lint) green.
- Verified light/dark, default vs tinted accents, and the custom clamp
via computed styles in the dev preview.

> Note: the `prerender-og` build step failing in the e2e/deploy jobs is
unrelated to this diff — it's in `vite.config.ts` (untouched here) and
builds cleanly locally.
2026-07-15 12:11:00 +00:00
EthanHealy01andGitHub 7f01bcdc44 Classifier setup as a processor policy (#7012)
## Overview

Adds a **Classification policy** to the processor's policy catalogue,
set up the same way as the Security policy. This moves classifier
configuration out of the editor (where the labels UI landed in #6898 and
was then removed with the rest of the editor's policy-management surface
in #6932) and into the processor, which is now the single place policies
are configured.

## What it does

- **Classification card** in the processor policy catalogue. Always
shown, but **setup is locked until the backend reports the AI engine is
on** — so admins can see the capability they're missing rather than it
being hidden entirely.
- **Setup wizard** mirrors Security: the workflow step shows the team's
**classification label editor** (reused
`LabelsEditor`/`LabelsEditorModal` — add box, chip grid, per-label icon
picker, import/export, reset) instead of tool toggles, since classify is
a single non-configurable step.
- On enable, the team's label vocabulary is **seeded with the 268
built-in defaults** (clobber-safe: only when the team has none). On
upload the document is classified against the team's labels and tagged;
on SaaS with the engine on, files group by category in the editor
sidebar.

## Reuse & consolidation

- Reuses the existing labels table, `labelsFile` helpers, and default
vocabulary. Labels read/write through the processor's own
`apiClient.local` (not the editor's axios client) so auth/base routing
stays explicit; the wire shape is shared.
- Consolidates policy-category icons into a shared, **id-keyed**
`policyCategoryIcon` util (outline glyphs) used by both the editor and
the processor, replacing the processor's emoji-glyph map (and the stray
`schedule` key that rendered a bare dot).

## Testing

- `task frontend:typecheck:{core,proprietary,portal}`,
`frontend:lint:eslint`, `frontend:test` (156 files / 1305 tests) — all
green.
- Verified in Storybook: the Classification card renders, the setup
wizard shows the label editor (268 defaults), and the full labels editor
opens with icons/import/export/reset. Added an MSW handler for the
app-config + labels endpoints and a `Classification` wizard story.

## Notes for reviewers

- The AI-engine gate reads the public `/api/v1/config/app-config`;
classification labels use `/api/v1/classification/labels` (team-scoped,
team-lead/admin-gated, `policies.enabled`); the classify step hits
`/api/v1/ai/tools/classify-and-label` — all pre-existing backend from
#6898.
- Known parity behavior (matches the editor hook): a transient failure
loading team labels falls back to showing the defaults; not changed here
to avoid diverging the two hooks.
2026-07-15 11:04:30 +00:00
EthanHealy01andGitHub 0570c4c4d9 Create-PDF engine: render from a structured document (#7018) 2026-07-14 12:30:33 +00:00
316 changed files with 6203 additions and 11537 deletions
+29 -3
View File
@@ -1,16 +1,34 @@
# CI routing infra. Editing the top-level router (build.yml) or this filter
# config re-runs every area's jobs, so every job-gating filter below includes
# *ci. That makes a change to how jobs are dispatched actually exercise those
# jobs (self-testing), instead of a router edit only matching the project filter.
ci: &ci
- .github/workflows/build.yml
- .github/config/.files.yaml
build: &build
- *ci
- build.gradle
- app/(common|core|proprietary|saas)/build.gradle
- Taskfile.yml
- .taskfiles/backend.yml
- .github/workflows/check-licence.yml
openapi: &openapi
- *ci
- *build
- app/(common|core|proprietary|saas)/src/main/java/**
- .github/workflows/check-openapi.yml
docker-base: &docker-base
- docker/base/Dockerfile
# Dockerfiles only (base + embedded + unoserver). Gates the slow multi-arch
# (arm64) leg of the PR docker test build: arm64 is only rebuilt when a
# Dockerfile itself changes, not on every code PR.
dockerfiles: &dockerfiles
- docker/**/Dockerfile*
docker: &docker
- docker/embedded/Dockerfile
- docker/embedded/Dockerfile.fat
@@ -23,13 +41,11 @@ docker: &docker
- *docker-base
project: &project
- *ci
- app/(common|core|proprietary|saas)/src/(main|test)/java/**
- *build
- "app/(common|core|proprietary|saas)/src/(main|test)/resources/**/!(messages_*.properties|*.md)*"
- exampleYmlFiles/**
- gradle/**
- libs/**
- "testing/**/!(requirements*.txt|requirements*.in)*"
- *docker
- *docker-base
- gradle.properties
@@ -45,8 +61,11 @@ project: &project
- .taskfiles/docker.yml
- scripts/db-migration/**
- .github/workflows/db-migration-test.yml
- .github/workflows/docker-compose-tests.yml
- .github/workflows/test-build-docker.yml
frontend: &frontend
- *ci
- frontend/**
- .github/workflows/testdriver.yml
- testing/**
@@ -63,10 +82,14 @@ frontend: &frontend
- Taskfile.yml
- .taskfiles/frontend.yml
- .taskfiles/e2e.yml
- .github/workflows/frontend-validation.yml
- .github/workflows/e2e-stubbed.yml
- .github/workflows/e2e-live.yml
# Files that affect the Tauri desktop bundle. Gate the multi-OS Tauri build
# job on changes to any of these.
tauri: &tauri
- *ci
- frontend/editor/src-tauri/**
- frontend/editor/src/desktop/**
- frontend/editor/tsconfig.desktop.vite.json
@@ -81,6 +104,7 @@ tauri: &tauri
# the engine validation job on changes to engine sources or to the Java
# tool surfaces it generates models from.
engine: &engine
- *ci
- engine/**
- app/(common|core|proprietary|saas)/src/main/java/**
- .github/workflows/ai-engine.yml
@@ -93,6 +117,7 @@ engine: &engine
# tasks that drive generation. Deliberately excludes the broad frontend/docker/
# testing globs, so a CSS-only PR does not boot the backend to rebuild the spec.
generated-models: &generated-models
- *ci
- *openapi
- frontend/editor/scripts/generate-tool-api-types.mts
- frontend/editor/src/core/types/toolApiTypes.ts
@@ -115,6 +140,7 @@ licenses-backend: &licenses-backend
# Files that can affect premium / enterprise behaviour. Gate the enterprise
# Playwright job on changes to any of these on PRs.
proprietary: &proprietary
- *ci
- app/proprietary/**
- frontend/editor/src/proprietary/**
- frontend/editor/src/core/tests/enterprise/**
+4 -5
View File
@@ -29,7 +29,7 @@ jobs:
check-pr:
if: (github.event_name == 'pull_request' && github.event.action != 'closed') || github.event_name == 'workflow_dispatch'
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
runs-on: ${{ needs.pick.outputs.use_depot == 'true' && 'depot-ubuntu-24.04-4' || 'ubuntu-latest' }}
outputs:
should_deploy: ${{ steps.decide.outputs.should_deploy }}
is_fork: ${{ steps.resolve.outputs.is_fork }}
@@ -102,7 +102,7 @@ jobs:
deploy-v2-pr:
needs: [pick, check-pr]
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
runs-on: ${{ needs.pick.outputs.use_depot == 'true' && 'depot-ubuntu-24.04-4' || 'ubuntu-latest' }}
if: needs.check-pr.outputs.should_deploy == 'true' && (needs.check-pr.outputs.is_fork == 'false' || needs.check-pr.outputs.allow_fork == 'true')
# Concurrency control - only one deployment per PR at a time
concurrency:
@@ -114,7 +114,7 @@ jobs:
pull-requests: write
id-token: write
env:
USE_DEPOT: ${{ needs.pick.outputs.is_fork != 'true' }}
USE_DEPOT: ${{ needs.pick.outputs.use_depot == 'true' }}
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
# Single source of truth for whether this preview embeds the admin portal:
# drives the image build-arg and the deployment comment.
@@ -297,7 +297,6 @@ jobs:
- /stirling/V2-PR-${{ needs.check-pr.outputs.pr_number }}/storage:/storage:rw
environment:
DISABLE_ADDITIONAL_FEATURES: "false"
POLICIES_ENABLED: "true"
STIRLING_BILLING_ACCOUNT_LINK_ENABLED: "true"
SECURITY_ENABLELOGIN: "true"
SECURITY_INITIALLOGIN_USERNAME: "${{ secrets.TEST_LOGIN_USERNAME }}"
@@ -476,7 +475,7 @@ jobs:
cleanup-v2-deployment:
if: github.event.action == 'closed'
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
runs-on: ${{ needs.pick.outputs.use_depot == 'true' && 'depot-ubuntu-24.04-4' || 'ubuntu-latest' }}
permissions:
contents: read
issues: write
@@ -39,7 +39,7 @@ jobs:
check-comment:
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
runs-on: ${{ needs.pick.outputs.use_depot == 'true' && 'depot-ubuntu-24.04-4' || 'ubuntu-latest' }}
permissions:
issues: write
if: |
@@ -180,13 +180,13 @@ jobs:
deploy-pr:
needs: [pick, check-comment]
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
runs-on: ${{ needs.pick.outputs.use_depot == 'true' && 'depot-ubuntu-24.04-4' || 'ubuntu-latest' }}
permissions:
issues: write
pull-requests: write
id-token: write
env:
USE_DEPOT: ${{ needs.pick.outputs.is_fork != 'true' }}
USE_DEPOT: ${{ needs.pick.outputs.use_depot == 'true' }}
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
steps:
@@ -511,7 +511,7 @@ jobs:
handle-label-commands:
if: ${{ github.event.issue.pull_request != null }}
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
runs-on: ${{ needs.pick.outputs.use_depot == 'true' && 'depot-ubuntu-24.04-4' || 'ubuntu-latest' }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
+41 -17
View File
@@ -5,6 +5,11 @@ name: _runner-pick
# can pick a runner class without each one duplicating the 200-char gate
# expression in their own `runs-on:`.
#
# It also owns the single Depot kill-switch (use_depot). Depot is currently
# disabled repo-wide; downstream jobs gate their Depot runner/build usage on
# use_depot so nothing has to be deleted to turn Depot off. Flip DEPOT_ENABLED
# in the decide step to switch Depot back on.
#
# Caller pattern:
#
# jobs:
@@ -13,12 +18,15 @@ name: _runner-pick
#
# real-work:
# needs: pick
# runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-8' }}
# runs-on: ${{ needs.pick.outputs.use_depot == 'true' && 'depot-ubuntu-24.04-8' || 'ubuntu-latest' }}
# steps: [...]
#
# Output:
# is_fork: "true" when the trigger is a pull_request from a fork or an
# untrusted author_association, "false" otherwise.
# Outputs:
# is_fork: "true" when the trigger is a pull_request from a fork or an
# untrusted author_association, "false" otherwise. Use this for
# trust gating (skipping secret-dependent jobs on forks).
# use_depot: "true" when downstream jobs should use Depot runners/builders.
# Currently forced "false" (Depot disabled repo-wide).
on:
workflow_call:
@@ -26,6 +34,9 @@ on:
is_fork:
description: '"true" if the trigger is an untrusted fork PR.'
value: ${{ jobs.pick.outputs.is_fork }}
use_depot:
description: '"true" when downstream jobs should use Depot. Currently forced off.'
value: ${{ jobs.pick.outputs.use_depot }}
permissions:
contents: read
@@ -36,6 +47,7 @@ jobs:
timeout-minutes: 1
outputs:
is_fork: ${{ steps.decide.outputs.is_fork }}
use_depot: ${{ steps.decide.outputs.use_depot }}
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
@@ -50,21 +62,33 @@ jobs:
AUTHOR_ASSOC: ${{ github.event.pull_request.author_association }}
run: |
set -eu
# Depot kill-switch. Depot is disabled repo-wide: no job uses Depot
# runners or the Depot build actions while this is false. All the
# Depot wiring is left in place - set DEPOT_ENABLED=true to switch it
# back on (it then activates on trusted, non-fork triggers as before).
DEPOT_ENABLED=false
if [ -z "${PR_NUMBER:-}" ]; then
# Not a pull_request event at all (push, schedule, workflow_dispatch,
# workflow_call from a non-PR trigger) -> trusted by default.
echo "is_fork=false" >> "$GITHUB_OUTPUT"
exit 0
is_fork=false
elif [ "${HEAD_REPO_FORK}" = "true" ]; then
is_fork=true
else
case "${AUTHOR_ASSOC}" in
OWNER|MEMBER|COLLABORATOR) is_fork=false ;;
*) is_fork=true ;;
esac
fi
if [ "${HEAD_REPO_FORK}" = "true" ]; then
echo "is_fork=true" >> "$GITHUB_OUTPUT"
exit 0
# Depot only ever ran on trusted triggers, so gate it on both the
# kill-switch and is_fork.
if [ "${DEPOT_ENABLED}" = "true" ] && [ "${is_fork}" = "false" ]; then
use_depot=true
else
use_depot=false
fi
case "${AUTHOR_ASSOC}" in
OWNER|MEMBER|COLLABORATOR)
echo "is_fork=false" >> "$GITHUB_OUTPUT"
;;
*)
echo "is_fork=true" >> "$GITHUB_OUTPUT"
;;
esac
echo "is_fork=${is_fork}" >> "$GITHUB_OUTPUT"
echo "use_depot=${use_depot}" >> "$GITHUB_OUTPUT"
+1 -1
View File
@@ -24,7 +24,7 @@ jobs:
build:
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-8' }}
runs-on: ${{ needs.pick.outputs.use_depot == 'true' && 'depot-ubuntu-24.04-8' || 'ubuntu-latest' }}
env:
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
strategy:
+1 -1
View File
@@ -54,7 +54,7 @@ jobs:
# (nor DEPOT_TOKEN), so the suite can't boot premium and would fail. See the
# header comment. GitHub reports the skipped reusable workflow as success.
if: needs.pick.outputs.is_fork != 'true'
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || format('depot-ubuntu-24.04-{0}', inputs.depot_cores || '8') }}
runs-on: ${{ needs.pick.outputs.use_depot == 'true' && format('depot-ubuntu-24.04-{0}', inputs.depot_cores || '8') || 'ubuntu-latest' }}
timeout-minutes: 45
env:
PREMIUM_KEY: ${{ secrets.PREMIUM_KEY_ENTERPRISE }}
+9
View File
@@ -41,6 +41,7 @@ jobs:
openapi: ${{ steps.changes.outputs.openapi }}
frontend: ${{ steps.changes.outputs.frontend }}
docker-base: ${{ steps.changes.outputs.docker-base }}
dockerfiles: ${{ steps.changes.outputs.dockerfiles }}
tauri: ${{ steps.changes.outputs.tauri }}
engine: ${{ steps.changes.outputs.engine }}
generated-models: ${{ steps.changes.outputs.generated-models }}
@@ -153,6 +154,7 @@ jobs:
secrets: inherit
with:
docker-base-changed: ${{ needs.files-changed.outputs.docker-base }}
dockerfiles-changed: ${{ needs.files-changed.outputs.dockerfiles }}
tauri-build:
if: needs.files-changed.outputs.tauri == 'true'
@@ -162,6 +164,13 @@ jobs:
pull-requests: write
uses: ./.github/workflows/tauri-build.yml
secrets: inherit
# PR smoke build: Linux only (fastest + cheapest to compile), unsigned,
# deb-only, no AppImage. The full signed multi-OS matrix runs on release;
# nightly still warms the Rust cache with all-OS defaults.
with:
platform: linux
sign: false
minimal: true
ai-engine:
if: needs.files-changed.outputs.engine == 'true'
+1 -1
View File
@@ -15,7 +15,7 @@ jobs:
check-generate-openapi-docs:
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
runs-on: ${{ needs.pick.outputs.use_depot == 'true' && 'depot-ubuntu-24.04-4' || 'ubuntu-latest' }}
env:
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
steps:
+1 -1
View File
@@ -34,7 +34,7 @@ jobs:
aggregate:
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
runs-on: ${{ needs.pick.outputs.use_depot == 'true' && 'depot-ubuntu-24.04-4' || 'ubuntu-latest' }}
timeout-minutes: 15
steps:
- name: Harden Runner
+1 -1
View File
@@ -17,7 +17,7 @@ jobs:
migration-test:
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-8' }}
runs-on: ${{ needs.pick.outputs.use_depot == 'true' && 'depot-ubuntu-24.04-8' || 'ubuntu-latest' }}
timeout-minutes: 30
env:
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
+2 -2
View File
@@ -15,7 +15,7 @@ jobs:
deploy-v2-on-push:
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
runs-on: ${{ needs.pick.outputs.use_depot == 'true' && 'depot-ubuntu-24.04-4' || 'ubuntu-latest' }}
concurrency:
group: deploy-v2-push-V2
cancel-in-progress: true
@@ -23,7 +23,7 @@ jobs:
contents: read
id-token: write
env:
USE_DEPOT: ${{ needs.pick.outputs.is_fork != 'true' }}
USE_DEPOT: ${{ needs.pick.outputs.use_depot == 'true' }}
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
steps:
+1 -1
View File
@@ -26,7 +26,7 @@ jobs:
docker-compose-tests:
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || format('depot-ubuntu-24.04-{0}', inputs.depot_cores || '4') }}
runs-on: ${{ needs.pick.outputs.use_depot == 'true' && format('depot-ubuntu-24.04-{0}', inputs.depot_cores || '4') || 'ubuntu-latest' }}
permissions:
actions: write
contents: read
+1 -1
View File
@@ -21,7 +21,7 @@ jobs:
playwright-e2e-live:
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || format('depot-ubuntu-24.04-{0}', inputs.depot_cores || '8') }}
runs-on: ${{ needs.pick.outputs.use_depot == 'true' && format('depot-ubuntu-24.04-{0}', inputs.depot_cores || '8') || 'ubuntu-latest' }}
timeout-minutes: 30
steps:
- name: Harden Runner
+1 -1
View File
@@ -21,7 +21,7 @@ jobs:
playwright-e2e:
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || format('depot-ubuntu-24.04-{0}', inputs.depot_cores || '8') }}
runs-on: ${{ needs.pick.outputs.use_depot == 'true' && format('depot-ubuntu-24.04-{0}', inputs.depot_cores || '8') || 'ubuntu-latest' }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
@@ -25,7 +25,7 @@ jobs:
files-changed:
name: detect what files changed
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
runs-on: ${{ needs.pick.outputs.use_depot == 'true' && 'depot-ubuntu-24.04-4' || 'ubuntu-latest' }}
timeout-minutes: 3
outputs:
licenses-frontend: ${{ steps.changes.outputs.licenses-frontend }}
@@ -49,7 +49,7 @@ jobs:
if: needs.files-changed.outputs.licenses-frontend == 'true'
name: Generate Frontend License Report
needs: [pick, files-changed]
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
runs-on: ${{ needs.pick.outputs.use_depot == 'true' && 'depot-ubuntu-24.04-4' || 'ubuntu-latest' }}
permissions:
contents: write
pull-requests: write
@@ -320,7 +320,7 @@ jobs:
if: needs.files-changed.outputs.licenses-backend == 'true'
needs: [pick, files-changed]
name: Generate Backend License Report
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
runs-on: ${{ needs.pick.outputs.use_depot == 'true' && 'depot-ubuntu-24.04-4' || 'ubuntu-latest' }}
permissions:
contents: write
pull-requests: write
+1 -1
View File
@@ -16,7 +16,7 @@ jobs:
frontend-validation:
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
runs-on: ${{ needs.pick.outputs.use_depot == 'true' && 'depot-ubuntu-24.04-4' || 'ubuntu-latest' }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
+3 -3
View File
@@ -42,7 +42,7 @@ jobs:
determine-matrix:
if: ${{ vars.CI_PROFILE != 'lite' }}
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
runs-on: ${{ needs.pick.outputs.use_depot == 'true' && 'depot-ubuntu-24.04-4' || 'ubuntu-latest' }}
outputs:
matrix: ${{ steps.set-matrix.outputs.matrix }}
version: ${{ steps.versionNumber.outputs.versionNumber }}
@@ -113,7 +113,7 @@ jobs:
build-jars:
needs: [pick, determine-matrix]
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
runs-on: ${{ needs.pick.outputs.use_depot == 'true' && 'depot-ubuntu-24.04-4' || 'ubuntu-latest' }}
env:
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
strategy:
@@ -639,7 +639,7 @@ jobs:
collect-and-release:
needs: [pick, determine-matrix, build, build-jars]
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
runs-on: ${{ needs.pick.outputs.use_depot == 'true' && 'depot-ubuntu-24.04-4' || 'ubuntu-latest' }}
permissions:
contents: write
steps:
+1 -1
View File
@@ -19,7 +19,7 @@ jobs:
playwright-all-browsers:
name: Playwright (chromium + firefox + webkit)
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
runs-on: ${{ needs.pick.outputs.use_depot == 'true' && 'depot-ubuntu-24.04-4' || 'ubuntu-latest' }}
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
+1 -1
View File
@@ -28,7 +28,7 @@ jobs:
push:
if: ${{ vars.CI_PROFILE != 'lite' }}
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
runs-on: ${{ needs.pick.outputs.use_depot == 'true' && 'depot-ubuntu-24.04-4' || 'ubuntu-latest' }}
env:
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
steps:
+20 -9
View File
@@ -21,6 +21,11 @@ on:
required: false
type: boolean
default: true
minimal:
description: "Fast smoke build: Linux deb only, skip rpm and the flaky AppImage pass. Used by PR builds."
required: false
type: boolean
default: false
workflow_dispatch:
inputs:
platform:
@@ -38,6 +43,11 @@ on:
required: false
default: true
type: boolean
minimal:
description: "Fast smoke build: Linux deb only, skip rpm and the flaky AppImage pass."
required: false
default: false
type: boolean
permissions:
contents: read
@@ -386,10 +396,10 @@ jobs:
with:
projectPath: ./frontend/editor
tauriScript: npx tauri
# Linux: build deb+rpm only here. AppImage runs in its own
# continue-on-error step below so its persistent linuxdeploy
# failure (#6127 onwards) does not tank deb/rpm uploads.
args: ${{ matrix.platform == 'ubuntu-22.04' && '--bundles deb,rpm' || matrix.args }}
# Linux: build deb+rpm only here (deb-only on minimal smoke builds).
# AppImage runs in its own continue-on-error step below so its
# persistent linuxdeploy failure (#6127 onwards) does not tank uploads.
args: ${{ matrix.platform == 'ubuntu-22.04' && (inputs.minimal && '--bundles deb' || '--bundles deb,rpm') || matrix.args }}
- name: Build Tauri app (unsigned)
if: ${{ !inputs.sign }}
@@ -406,15 +416,16 @@ jobs:
with:
projectPath: ./frontend/editor
tauriScript: npx tauri
# Linux: build deb+rpm only here. AppImage runs in its own
# continue-on-error step below so its persistent linuxdeploy
# failure (#6127 onwards) does not tank deb/rpm uploads.
args: ${{ matrix.platform == 'ubuntu-22.04' && '--bundles deb,rpm' || matrix.args }}
# Linux: build deb+rpm only here (deb-only on minimal smoke builds).
# AppImage runs in its own continue-on-error step below so its
# persistent linuxdeploy failure (#6127 onwards) does not tank uploads.
args: ${{ matrix.platform == 'ubuntu-22.04' && (inputs.minimal && '--bundles deb' || '--bundles deb,rpm') || matrix.args }}
# AppImage is decoupled so its linuxdeploy run gets a fresh process
# (rpm scratch state torn down) and its failure can't tank deb/rpm.
# Skipped on minimal smoke builds (flaky + slow, deb is enough to verify).
- name: Build Tauri app (Linux AppImage)
if: matrix.platform == 'ubuntu-22.04'
if: matrix.platform == 'ubuntu-22.04' && !inputs.minimal
continue-on-error: true
uses: tauri-apps/tauri-action@84b9d35b5fc46c1e45415bdb6144030364f7ebc5 # v0.6.2
env:
+19 -5
View File
@@ -12,6 +12,11 @@ on:
required: false
type: string
default: "false"
dockerfiles-changed:
description: "Whether any Dockerfile changed (forwarded from files-changed). Gates the slow arm64 build leg."
required: false
type: string
default: "false"
depot_cores:
description: "Depot runner vCPU count (used in runs-on). Override for benchmarking."
required: false
@@ -41,12 +46,12 @@ jobs:
# `task backend:build:ci` produce equivalent JARs (verify before wiring).
test-build-docker-images:
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || format('depot-ubuntu-24.04-{0}', inputs.depot_cores || '8') }}
runs-on: ${{ needs.pick.outputs.use_depot == 'true' && format('depot-ubuntu-24.04-{0}', inputs.depot_cores || '8') || 'ubuntu-latest' }}
permissions:
contents: read
id-token: write
env:
USE_DEPOT: ${{ needs.pick.outputs.is_fork != 'true' && inputs.docker-base-changed != 'true' }}
USE_DEPOT: ${{ needs.pick.outputs.use_depot == 'true' && inputs.docker-base-changed != 'true' }}
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
strategy:
fail-fast: false
@@ -146,13 +151,22 @@ jobs:
# GITHUB_EVENT_NAME is already provided by the runner.
env:
DOCKER_BASE_CHANGED: ${{ inputs.docker-base-changed }}
DOCKERFILES_CHANGED: ${{ inputs.dockerfiles-changed }}
run: |
if [ "$GITHUB_EVENT_NAME" = "pull_request" ] && [ "$DOCKER_BASE_CHANGED" = "true" ]; then
# Base Dockerfile changed: build against the locally-built base,
# which only exists for amd64.
echo "base_image=stirling-pdf-base:pr-test" >> "$GITHUB_OUTPUT"
echo "platforms=linux/amd64" >> "$GITHUB_OUTPUT"
else
elif [ "$DOCKERFILES_CHANGED" = "true" ]; then
# A Dockerfile changed: also verify the arm64 build (slow QEMU leg).
echo "base_image=stirlingtools/stirling-pdf-base:latest" >> "$GITHUB_OUTPUT"
echo "platforms=linux/amd64,linux/arm64/v8" >> "$GITHUB_OUTPUT"
else
# No Dockerfile change: amd64 only. arm64 is exercised on the base
# image publish and on release, not on every code PR.
echo "base_image=stirlingtools/stirling-pdf-base:latest" >> "$GITHUB_OUTPUT"
echo "platforms=linux/amd64" >> "$GITHUB_OUTPUT"
fi
# Base-changed PRs build the embedded image with the local docker driver
@@ -214,12 +228,12 @@ jobs:
test-build-unoserver-image:
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || format('depot-ubuntu-24.04-{0}', inputs.depot_cores || '8') }}
runs-on: ${{ needs.pick.outputs.use_depot == 'true' && format('depot-ubuntu-24.04-{0}', inputs.depot_cores || '8') || 'ubuntu-latest' }}
permissions:
contents: read
id-token: write
env:
USE_DEPOT: ${{ needs.pick.outputs.is_fork != 'true' && inputs.docker-base-changed != 'true' }}
USE_DEPOT: ${{ needs.pick.outputs.use_depot == 'true' && inputs.docker-base-changed != 'true' }}
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
steps:
- name: Harden Runner
+5 -5
View File
@@ -26,12 +26,12 @@ jobs:
deploy:
if: ${{ vars.CI_PROFILE != 'lite' }}
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
runs-on: ${{ needs.pick.outputs.use_depot == 'true' && 'depot-ubuntu-24.04-4' || 'ubuntu-latest' }}
permissions:
contents: read
id-token: write
env:
USE_DEPOT: ${{ needs.pick.outputs.is_fork != 'true' }}
USE_DEPOT: ${{ needs.pick.outputs.use_depot == 'true' }}
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
steps:
- name: Harden Runner
@@ -154,7 +154,7 @@ jobs:
if: always()
name: detect what files changed
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
runs-on: ${{ needs.pick.outputs.use_depot == 'true' && 'depot-ubuntu-24.04-4' || 'ubuntu-latest' }}
timeout-minutes: 3
outputs:
frontend: ${{ steps.changes.outputs.frontend }}
@@ -175,7 +175,7 @@ jobs:
test:
if: needs.files-changed.outputs.frontend == 'true'
needs: [pick, deploy, files-changed]
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
runs-on: ${{ needs.pick.outputs.use_depot == 'true' && 'depot-ubuntu-24.04-4' || 'ubuntu-latest' }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
@@ -209,7 +209,7 @@ jobs:
cleanup:
needs: [pick, deploy, test]
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
runs-on: ${{ needs.pick.outputs.use_depot == 'true' && 'depot-ubuntu-24.04-4' || 'ubuntu-latest' }}
if: always()
steps:
+2 -4
View File
@@ -26,7 +26,6 @@ tasks:
AIENGINE_ENABLED: '{{.AIENGINE_ENABLED}}'
AIENGINE_TIMEOUTSECONDS: '{{.AIENGINE_TIMEOUTSECONDS}}'
SECURITY_ENABLELOGIN: '{{.SECURITY_ENABLELOGIN}}'
POLICIES_ENABLED: '{{.POLICIES_ENABLED}}'
dev:proprietary:
desc: "Start backend dev server in proprietary mode"
@@ -41,13 +40,12 @@ tasks:
AIENGINE_ENABLED: '{{.AIENGINE_ENABLED | default "false"}}'
AIENGINE_TIMEOUTSECONDS: '{{.AIENGINE_TIMEOUTSECONDS | default "120"}}'
SECURITY_ENABLELOGIN: '{{.SECURITY_ENABLELOGIN | default ""}}'
POLICIES_ENABLED: '{{.POLICIES_ENABLED | default ""}}'
env:
SERVER_PORT: '{{.PORT}}'
cmds:
- cmd: '{{if .AIENGINE_URL}}AIENGINE_URL={{.AIENGINE_URL}} AIENGINE_ENABLED={{.AIENGINE_ENABLED}} AIENGINE_TIMEOUTSECONDS={{.AIENGINE_TIMEOUTSECONDS}} {{end}}{{if .SECURITY_ENABLELOGIN}}SECURITY_ENABLELOGIN={{.SECURITY_ENABLELOGIN}} {{end}}{{if .POLICIES_ENABLED}}POLICIES_ENABLED={{.POLICIES_ENABLED}} {{end}}cmd /c ".\gradlew.bat :stirling-pdf:bootRun"'
- cmd: '{{if .AIENGINE_URL}}AIENGINE_URL={{.AIENGINE_URL}} AIENGINE_ENABLED={{.AIENGINE_ENABLED}} AIENGINE_TIMEOUTSECONDS={{.AIENGINE_TIMEOUTSECONDS}} {{end}}{{if .SECURITY_ENABLELOGIN}}SECURITY_ENABLELOGIN={{.SECURITY_ENABLELOGIN}} {{end}}cmd /c ".\gradlew.bat :stirling-pdf:bootRun"'
platforms: [windows]
- cmd: '{{if .AIENGINE_URL}}AIENGINE_URL={{.AIENGINE_URL}} AIENGINE_ENABLED={{.AIENGINE_ENABLED}} AIENGINE_TIMEOUTSECONDS={{.AIENGINE_TIMEOUTSECONDS}} {{end}}{{if .SECURITY_ENABLELOGIN}}SECURITY_ENABLELOGIN={{.SECURITY_ENABLELOGIN}} {{end}}{{if .POLICIES_ENABLED}}POLICIES_ENABLED={{.POLICIES_ENABLED}} {{end}}./gradlew :stirling-pdf:bootRun'
- cmd: '{{if .AIENGINE_URL}}AIENGINE_URL={{.AIENGINE_URL}} AIENGINE_ENABLED={{.AIENGINE_ENABLED}} AIENGINE_TIMEOUTSECONDS={{.AIENGINE_TIMEOUTSECONDS}} {{end}}{{if .SECURITY_ENABLELOGIN}}SECURITY_ENABLELOGIN={{.SECURITY_ENABLELOGIN}} {{end}}./gradlew :stirling-pdf:bootRun'
platforms: [linux, darwin]
dev:bundled:
+14 -3
View File
@@ -203,6 +203,19 @@ tasks:
cmds:
- task: lint:eslint
- task: lint:dpdm
- task: lint:colors
lint:colors:
desc: "Enforce theme tokens — colours in core/theme route through the palette"
deps: [install]
cmds:
- node editor/scripts/lint/theme-lint.mjs
contrast:
desc: "Report low-contrast theme token pairs (warning only, never blocks)"
deps: [install]
cmds:
- node editor/scripts/lint/theme-lint.mjs contrast
lint:eslint:
desc: "Run ESLint linting"
@@ -249,10 +262,8 @@ tasks:
typecheck:_run:
internal: true
env:
CI: '{{ .CI | default "false" }}'
cmds:
- '{{ if eq .CI "true" }}npx tsc{{ else }}npx tsgo{{ end }} --noEmit --project {{.PROJECT}}'
- 'npx tsc --noEmit --project {{.PROJECT}}'
typecheck:core:
desc: "Typecheck core build variant"
+2
View File
@@ -155,6 +155,8 @@ The project structure is defined in `engine/pyproject.toml`. Any new dependencie
For a broader explanation of the frontend layering and override architecture, read @frontend/editor/DeveloperGuide.md
Before touching colours or theming (tokens, dark mode, accent colours), read @frontend/editor/src/core/theme/README.md — it explains the palette/`--c-*` token system and the rule that literal colours live only in `primitives.css`.
```typescript
// ✅ CORRECT - Use @app/* for all imports
import { AppLayout } from "@app/components/AppLayout";
-1
View File
@@ -90,7 +90,6 @@ tasks:
vars:
PORT: '{{.BACKEND_PORT}}'
SECURITY_ENABLELOGIN: "true"
POLICIES_ENABLED: "true"
- task: frontend:dev:proprietary
vars:
PORT: '{{.EDITOR_PORT}}'
+12
View File
@@ -208,6 +208,18 @@
"moduleName": ".*",
"moduleLicense": "The W3C License"
},
{
"moduleName": "com.google.re2j:re2j",
"moduleLicense": "Go License"
},
{
"moduleName": "com.hubspot:algebra",
"moduleLicense": null
},
{
"moduleName": "com.hubspot.immutables:immutables-exceptions",
"moduleLicense": null
},
{
"moduleName": ".*",
"moduleLicense": "UnRar License"
@@ -206,11 +206,6 @@ public class ApplicationProperties {
@Data
public static class Policies {
/**
* Master switch for the policy + sources subsystem (the PAYG-metered automation surface).
*/
private boolean enabled = false;
/**
* Absolute directories that policy folder input sources and output sinks may read from or
* write to. Empty (the default) disables folder access entirely, so a policy can never be
+14
View File
@@ -66,6 +66,20 @@ dependencies {
implementation "com.google.code.gson:gson:${gsonVersion}"
// jinjava/jjwt transitively request older Jackson 2 versions; declare the current
// version directly so it is selected consistently (root build.gradle pins are the fallback).
runtimeOnly "com.fasterxml.jackson.core:jackson-core:${jackson2Version}"
runtimeOnly "com.fasterxml.jackson.core:jackson-databind:${jackson2Version}"
implementation("com.hubspot.jinjava:jinjava:${jinjavaVersion}") {
// Compile-time-only annotation artifacts (class-retention annotations, not needed at
// runtime) whose declared licences (LGPL / none) fail the licence compatibility check.
exclude group: 'com.google.code.findbugs', module: 'annotations'
exclude group: 'org.derive4j', module: 'derive4j-annotation'
exclude group: 'com.hubspot.immutables', module: 'hubspot-style'
exclude group: 'com.hubspot.immutables', module: 'immutable-collection-encodings'
}
api 'io.micrometer:micrometer-registry-prometheus'
api "io.jsonwebtoken:jjwt-api:${jwtVersion}"
@@ -35,6 +35,7 @@ import stirling.software.proprietary.billing.ContentHasher;
import stirling.software.proprietary.billing.DocumentUnitCalculator;
import stirling.software.proprietary.billing.DocumentUnitCalculator.FileSize;
import stirling.software.proprietary.billing.UnitCalcPolicy;
import stirling.software.proprietary.policy.controller.PolicyRunRoutes;
import stirling.software.proprietary.security.model.ApiKeyAuthenticationToken;
/**
@@ -84,7 +85,13 @@ public class InstanceEntitlementInterceptor implements HandlerInterceptor {
instanceof ApiKeyAuthenticationToken;
BillingCategory category = BillableOperationClassifier.categorize(request, apiKey);
request.setAttribute(ATTR_CATEGORY, category);
decision = gate.evaluate(category != BillingCategory.BYPASSED);
// A policy run kicks off billable automation, so block it up front when unentitled
// rather than after its first tool. It carries no automation header itself (category
// BYPASSED), so it's gated here but metered only via its dispatched sub-steps - keeping
// the BYPASSED meter category avoids double-counting.
boolean billable =
category != BillingCategory.BYPASSED || PolicyRunRoutes.matches(request);
decision = gate.evaluate(billable);
} catch (RuntimeException e) {
// Fail open: an inability to resolve entitlement (e.g. a DB or SaaS blip) must never
// turn into a hard block on billable work.
@@ -0,0 +1,61 @@
package stirling.software.proprietary.classification;
import java.io.InputStream;
import java.util.List;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.core.io.ClassPathResource;
import org.springframework.stereotype.Component;
import lombok.extern.slf4j.Slf4j;
import stirling.software.proprietary.classification.model.ClassificationLabel;
import stirling.software.proprietary.classification.model.ClassificationLabels;
import tools.jackson.databind.ObjectMapper;
/**
* Supplies the classification vocabulary the classify tool sends to the AI engine. The set is a
* fixed, built-in list bundled with the application ({@code
* classification/classification-labels.json}) and shared by everyone — there is no per-team
* customization or database. Loaded once at startup.
*/
@Slf4j
@Component
public class ClassificationLabelProvider {
private static final String RESOURCE = "classification/classification-labels.json";
private final List<ClassificationLabel> labels;
// Explicit @Autowired: the class has a second (private) constructor for tests, so Spring
// can't infer which to use without it.
@Autowired
public ClassificationLabelProvider(ObjectMapper objectMapper) {
this(load(objectMapper));
}
private ClassificationLabelProvider(List<ClassificationLabel> labels) {
this.labels = List.copyOf(labels);
}
/** Build a provider with an explicit label set (tests). */
public static ClassificationLabelProvider withLabels(List<ClassificationLabel> labels) {
return new ClassificationLabelProvider(labels);
}
/** The built-in vocabulary, in file order. */
public List<ClassificationLabel> labels() {
return labels;
}
private static List<ClassificationLabel> load(ObjectMapper objectMapper) {
try (InputStream in = new ClassPathResource(RESOURCE).getInputStream()) {
ClassificationLabels parsed = objectMapper.readValue(in, ClassificationLabels.class);
return parsed.labels();
} catch (Exception e) {
log.error("Failed to load classification labels from {}", RESOURCE, e);
return List.of();
}
}
}
@@ -1,136 +0,0 @@
package stirling.software.proprietary.classification;
import org.springframework.boot.autoconfigure.condition.ConditionalOnBooleanProperty;
import org.springframework.http.HttpStatus;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.DeleteMapping;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PutMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.server.ResponseStatusException;
import io.swagger.v3.oas.annotations.Hidden;
import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.tags.Tag;
import lombok.RequiredArgsConstructor;
import stirling.software.common.model.ApplicationProperties;
import stirling.software.common.service.UserServiceInterface;
import stirling.software.proprietary.classification.model.ClassificationLabels;
import stirling.software.proprietary.classification.model.LabelsValidator;
import stirling.software.proprietary.classification.store.ClassificationLabelStore;
import stirling.software.proprietary.classification.store.TeamLabelsEntity;
import stirling.software.proprietary.policy.config.PolicyManagementAuthority;
/**
* Read/write the team's classification label set — the flat vocabulary the document classifier runs
* against. Shared and team-scoped exactly like policies: every user reads their own team's labels,
* and only a user who may edit policies (a team leader on SaaS, the global admin self-hosted; see
* {@link PolicyManagementAuthority}) may change it — gated only when login is enabled, since
* single-user deployments trust the local operator. A team with no stored labels reads as {@code
* 204}; that team has no vocabulary, so its documents are not classified (there is no built-in
* default on the backend or the engine — the label data lives only in the frontend).
*/
@RestController
@RequestMapping("/api/v1/classification/labels")
@Hidden
@RequiredArgsConstructor
@Tag(name = "Classification", description = "Team-scoped document-classification labels")
@ConditionalOnBooleanProperty(name = "policies.enabled")
public class ClassificationLabelsController {
private final ClassificationLabelStore labelStore;
private final PolicyManagementAuthority policyManagementAuthority;
private final ApplicationProperties applicationProperties;
private final UserServiceInterface userService;
@GetMapping
@Operation(
summary = "Get the team's classification labels",
description =
"Returns the caller's team label set, or 204 when the team has none (its"
+ " documents are then not classified).")
public ResponseEntity<ClassificationLabels> getTeamLabels() {
return labelStore
.findByTeam(currentTeamId())
.map(ResponseEntity::ok)
.orElseGet(() -> ResponseEntity.noContent().build());
}
@PutMapping(consumes = MediaType.APPLICATION_JSON_VALUE)
@Operation(
summary = "Save the team's classification labels",
description =
"Validates and stores the label set for the caller's team, shared by everyone"
+ " on the team. Requires the policy-editor role for the team.")
public ResponseEntity<ClassificationLabels> saveTeamLabels(
@RequestBody ClassificationLabels labels) {
requireEditingAllowed();
validate(labels);
ClassificationLabels saved = labelStore.save(currentTeamId(), labels, currentUsername());
return ResponseEntity.ok(saved);
}
@DeleteMapping
@Operation(
summary = "Reset the team's classification labels",
description =
"Removes the team's stored label set; its documents are then not classified"
+ " until labels are saved again. Requires the policy-editor role for the"
+ " team.")
public ResponseEntity<Void> resetTeamLabels() {
requireEditingAllowed();
labelStore.deleteByTeam(currentTeamId());
return ResponseEntity.noContent().build();
}
private static void validate(ClassificationLabels labels) {
try {
LabelsValidator.validate(labels);
} catch (IllegalArgumentException e) {
throw new ResponseStatusException(HttpStatus.BAD_REQUEST, e.getMessage());
}
}
/**
* Editing the team labels requires the editor role for the caller's team — the same gate
* policies use (team leader on SaaS, global admin self-hosted). Single-user deployments (login
* disabled) have no such role, so they trust the local operator.
*/
private void requireEditingAllowed() {
if (!applicationProperties.getSecurity().isEnableLogin()) {
return;
}
if (!policyManagementAuthority.canEditPolicies()) {
throw new ResponseStatusException(
HttpStatus.FORBIDDEN,
"The team classification labels may only be changed by a team leader");
}
}
/**
* The caller's team key. With login disabled the single operator owns the {@link
* TeamLabelsEntity#NO_TEAM} sentinel row; with login enabled a caller with no resolvable team
* is an error rather than being dropped into the shared sentinel bucket (which would let
* unteamed users read and overwrite each other's "team" labels).
*/
private Long currentTeamId() {
Long teamId = policyManagementAuthority.currentUserTeamId();
if (teamId != null) {
return teamId;
}
if (!applicationProperties.getSecurity().isEnableLogin()) {
return TeamLabelsEntity.NO_TEAM;
}
throw new ResponseStatusException(
HttpStatus.UNAUTHORIZED, "Could not resolve the current user's team");
}
private String currentUsername() {
return userService == null ? null : userService.getCurrentUsername();
}
}
@@ -3,10 +3,10 @@ package stirling.software.proprietary.classification.model;
import java.util.List;
/**
* A flat multi-label classification vocabulary — the set of labels a document may be assigned.
* Stored per team (admin-edited, shared by everyone on the team); the classifier runs against these
* label names. A team with no stored set has no vocabulary, so its documents are not classified —
* neither the backend nor the engine holds a default of its own.
* A flat multi-label classification vocabulary — the set of labels a document may be assigned. The
* classifier runs against these label names. The vocabulary is a fixed, built-in set shared by
* everyone (see {@link stirling.software.proprietary.classification.ClassificationLabelProvider});
* this record is the JSON parse target for that bundled resource.
*/
public record ClassificationLabels(List<ClassificationLabel> labels) {
@@ -1,71 +0,0 @@
package stirling.software.proprietary.classification.model;
import java.util.HashSet;
import java.util.Locale;
import java.util.Set;
import java.util.regex.Pattern;
/**
* Structural validation for a user- or admin-supplied label set, run before it is stored so a
* malformed vocabulary can never reach the classifier. Mirrors the invariants the engine relies on:
* non-blank ids and names, each unique within the set (ids exactly, names case-insensitively).
*/
public final class LabelsValidator {
private LabelsValidator() {}
// Generous upper bounds so a legitimate label set is never blocked, but a single team or user
// can't store an unbounded blob that would bloat the row, balloon the classifier prompt, or
// exhaust memory on deserialize.
static final int MAX_LABELS = 500;
static final int MAX_TEXT_LENGTH = 128;
// Icon is a Material Symbols key (lowercase, digits, hyphens). Enforce the SHAPE server-side —
// the exact allowlist lives in the frontend — so a client bypassing the UI can't store
// arbitrary
// text that would render as garbage (or worse) in every teammate's sidebar.
private static final Pattern ICON_KEY = Pattern.compile("^[a-z0-9-]+$");
/**
* @throws IllegalArgumentException with a human-readable message when the label set is invalid.
*/
public static void validate(ClassificationLabels labels) {
if (labels == null || labels.labels() == null) {
throw new IllegalArgumentException("Labels are required");
}
if (labels.labels().size() > MAX_LABELS) {
throw new IllegalArgumentException("Too many labels (max " + MAX_LABELS + ")");
}
Set<String> ids = new HashSet<>();
Set<String> names = new HashSet<>();
for (ClassificationLabel label : labels.labels()) {
requireText(label.id(), "Label id");
requireText(label.name(), "Label name");
if (label.icon() != null && !label.icon().isEmpty()) {
if (label.icon().length() > MAX_TEXT_LENGTH) {
throw new IllegalArgumentException(
"Label icon is too long (max " + MAX_TEXT_LENGTH + " characters)");
}
if (!ICON_KEY.matcher(label.icon()).matches()) {
throw new IllegalArgumentException("Invalid label icon: " + label.icon());
}
}
if (!ids.add(label.id().trim())) {
throw new IllegalArgumentException("Duplicate label id: " + label.id());
}
if (!names.add(label.name().trim().toLowerCase(Locale.ROOT))) {
throw new IllegalArgumentException("Duplicate label name: " + label.name());
}
}
}
private static void requireText(String value, String field) {
if (value == null || value.isBlank()) {
throw new IllegalArgumentException(field + " must not be blank");
}
if (value.trim().length() > MAX_TEXT_LENGTH) {
throw new IllegalArgumentException(
field + " is too long (max " + MAX_TEXT_LENGTH + " characters)");
}
}
}
@@ -1,22 +0,0 @@
package stirling.software.proprietary.classification.store;
import java.util.Optional;
import stirling.software.proprietary.classification.model.ClassificationLabels;
/**
* Stores one {@link ClassificationLabels} set per team. A {@code null} teamId addresses the
* unteamed set (login disabled / no resolvable team), mirroring how the policy store treats a null
* team.
*/
public interface ClassificationLabelStore {
/** The team's stored labels, or empty when it has none (callers then skip classification). */
Optional<ClassificationLabels> findByTeam(Long teamId);
/** Create or replace the team's labels. Returns the stored value. */
ClassificationLabels save(Long teamId, ClassificationLabels labels, String updatedBy);
/** Remove the team's labels (reset to default). Returns whether a set existed. */
boolean deleteByTeam(Long teamId);
}
@@ -1,36 +0,0 @@
package stirling.software.proprietary.classification.store;
import java.util.Map;
import java.util.Optional;
import java.util.concurrent.ConcurrentHashMap;
import stirling.software.proprietary.classification.model.ClassificationLabels;
/**
* In-memory {@link ClassificationLabelStore} for tests and any future no-database mode. {@link
* JpaClassificationLabelStore} is the runtime bean.
*/
public class InProcessClassificationLabelStore implements ClassificationLabelStore {
private final Map<Long, ClassificationLabels> byTeam = new ConcurrentHashMap<>();
@Override
public Optional<ClassificationLabels> findByTeam(Long teamId) {
return Optional.ofNullable(byTeam.get(key(teamId)));
}
@Override
public ClassificationLabels save(Long teamId, ClassificationLabels labels, String updatedBy) {
byTeam.put(key(teamId), labels);
return labels;
}
@Override
public boolean deleteByTeam(Long teamId) {
return byTeam.remove(key(teamId)) != null;
}
private static long key(Long teamId) {
return teamId == null ? TeamLabelsEntity.NO_TEAM : teamId;
}
}
@@ -1,76 +0,0 @@
package stirling.software.proprietary.classification.store;
import java.time.Instant;
import java.util.Optional;
import org.springframework.boot.autoconfigure.condition.ConditionalOnBooleanProperty;
import org.springframework.stereotype.Service;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import stirling.software.proprietary.classification.model.ClassificationLabels;
import tools.jackson.core.JacksonException;
import tools.jackson.databind.ObjectMapper;
/**
* Durable {@link ClassificationLabelStore} backed by JPA; the runtime store. Gated on {@code
* policies.enabled} — stored labels only matter when the Classification policy can run — so it
* shares the policy subsystem's on/off switch. Each label set is persisted as JSON via {@link
* TeamLabelsEntity}.
*/
@Slf4j
@Service
@RequiredArgsConstructor
@ConditionalOnBooleanProperty(name = "policies.enabled")
public class JpaClassificationLabelStore implements ClassificationLabelStore {
private final TeamLabelsRepository teamRepository;
private final ObjectMapper objectMapper;
@Override
public Optional<ClassificationLabels> findByTeam(Long teamId) {
return teamRepository
.findById(key(teamId))
.flatMap(entity -> parse(entity.getLabelsJson(), "team " + teamId));
}
@Override
public ClassificationLabels save(Long teamId, ClassificationLabels labels, String updatedBy) {
TeamLabelsEntity entity = new TeamLabelsEntity();
entity.setTeamId(key(teamId));
entity.setLabelsJson(objectMapper.writeValueAsString(labels));
entity.setUpdatedAt(Instant.now());
entity.setUpdatedBy(updatedBy);
teamRepository.save(entity);
return labels;
}
@Override
public boolean deleteByTeam(Long teamId) {
long id = key(teamId);
if (!teamRepository.existsById(id)) {
return false;
}
teamRepository.deleteById(id);
return true;
}
private Optional<ClassificationLabels> parse(String json, String owner) {
try {
return Optional.of(objectMapper.readValue(json, ClassificationLabels.class));
} catch (JacksonException e) {
// A stored label set that no longer parses (corruption / manual DB edit) must not break
// classification: drop it so the caller treats the team as having no labels (and skips
// classification) rather than surfacing a 500 on every upload.
log.warn("Discarding unparseable stored labels for {}: {}", owner, e.getMessage());
return Optional.empty();
}
}
/** Map the nullable team id onto the entity's non-null key (sentinel for the unteamed case). */
private static long key(Long teamId) {
return teamId == null ? TeamLabelsEntity.NO_TEAM : teamId;
}
}
@@ -1,47 +0,0 @@
package stirling.software.proprietary.classification.store;
import java.io.Serializable;
import java.time.Instant;
import jakarta.persistence.Column;
import jakarta.persistence.Entity;
import jakarta.persistence.Id;
import jakarta.persistence.Table;
import lombok.Getter;
import lombok.NoArgsConstructor;
import lombok.Setter;
/**
* JPA row for a team's classification labels — one row per team. The label set lives as JSON in
* {@code labelsJson} (authoritative on read). {@code teamId} is the natural key; the sentinel
* {@link #NO_TEAM} stands in for the unteamed (login-disabled / self-hosted single-team) case,
* since a primary key can't be null (policies store a nullable {@code team_id}, but this table is
* keyed one-per-team). Kept decoupled from the security entities — {@code teamId} is a plain value,
* not a foreign key — so classification can be enabled or disabled without touching them.
*/
@Entity
@Table(name = "classification_labels")
@NoArgsConstructor
@Getter
@Setter
public class TeamLabelsEntity implements Serializable {
private static final long serialVersionUID = 1L;
/** Sentinel key for the unteamed label set (login disabled / no resolvable team). */
public static final long NO_TEAM = 0L;
@Id
@Column(name = "team_id")
private long teamId;
@Column(name = "labels_json", columnDefinition = "text")
private String labelsJson;
@Column(name = "updated_at")
private Instant updatedAt;
@Column(name = "updated_by")
private String updatedBy;
}
@@ -1,7 +0,0 @@
package stirling.software.proprietary.classification.store;
import org.springframework.data.jpa.repository.JpaRepository;
import org.springframework.stereotype.Repository;
@Repository
public interface TeamLabelsRepository extends JpaRepository<TeamLabelsEntity, Long> {}
@@ -54,21 +54,10 @@ public class CustomAuditEventRepository implements AuditEventRepository {
return;
}
String rid = MDC.get("requestId");
String apiKeyLabel =
MDC.get(
stirling.software.proprietary.security.service
.ApiKeyAuthenticationService.AUDIT_LABEL_MDC_KEY);
if (rid != null || apiKeyLabel != null) {
if (rid != null) {
clean = new java.util.HashMap<>(clean);
if (rid != null) {
clean.put("requestId", rid);
}
// Named key that made the request; surfaces as the doc source in the processor
// feed.
if (apiKeyLabel != null) {
clean.put("__apiKeyLabel", apiKeyLabel);
}
clean.put("requestId", rid);
}
String source = MDC.get("auditSource");
@@ -31,10 +31,9 @@ import stirling.software.common.service.PdfMetadataService;
import stirling.software.common.service.UserServiceInterface;
import stirling.software.common.util.TempFileManager;
import stirling.software.common.util.WebResponseUtils;
import stirling.software.proprietary.classification.ClassificationLabelProvider;
import stirling.software.proprietary.classification.model.ClassificationLabel;
import stirling.software.proprietary.classification.store.ClassificationLabelStore;
import stirling.software.proprietary.model.api.ai.AiPageText;
import stirling.software.proprietary.policy.config.PolicyManagementAuthority;
import stirling.software.proprietary.service.AiEngineClient;
import stirling.software.proprietary.service.PdfContentExtractor;
@@ -46,7 +45,7 @@ import tools.jackson.databind.node.ObjectNode;
* Dispatchable tool that classifies a PDF and writes the result into its metadata.
*
* <p>Runs as a Classification-policy pipeline step: it reads a bounded page window, asks the AI
* engine to classify the document against the caller's team label set, and stores the engine's JSON
* engine to classify the document against the built-in label set, and stores the engine's JSON
* answer — minus the transport-only {@code outcome} field — in the custom Info-dictionary key
* {@link PdfMetadataService#CLASSIFICATION_KEY}. Returns the labelled PDF. Not intended for direct
* client use.
@@ -72,13 +71,9 @@ public class ClassifyLabelController {
private final UserServiceInterface userService;
/**
* Present only when the policy subsystem is enabled ({@code policies.enabled}); the store and
* team authority are gated on it. Null otherwise, in which case there are no team labels to
* classify against and the document is passed through unlabelled.
* The fixed, built-in vocabulary shared by everyone — see {@link ClassificationLabelProvider}.
*/
private final ClassificationLabelStore labelStore;
private final PolicyManagementAuthority policyManagementAuthority;
private final ClassificationLabelProvider labelProvider;
public ClassifyLabelController(
CustomPDFDocumentFactory pdfDocumentFactory,
@@ -87,18 +82,16 @@ public class ClassifyLabelController {
PdfMetadataService pdfMetadataService,
AiEngineClient aiEngineClient,
ObjectMapper objectMapper,
@Autowired(required = false) UserServiceInterface userService,
@Autowired(required = false) ClassificationLabelStore labelStore,
@Autowired(required = false) PolicyManagementAuthority policyManagementAuthority) {
ClassificationLabelProvider labelProvider,
@Autowired(required = false) UserServiceInterface userService) {
this.pdfDocumentFactory = pdfDocumentFactory;
this.tempFileManager = tempFileManager;
this.pdfContentExtractor = pdfContentExtractor;
this.pdfMetadataService = pdfMetadataService;
this.aiEngineClient = aiEngineClient;
this.objectMapper = objectMapper;
this.labelProvider = labelProvider;
this.userService = userService;
this.labelStore = labelStore;
this.policyManagementAuthority = policyManagementAuthority;
}
@PostMapping(value = "/classify-and-label", consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
@@ -116,9 +109,9 @@ public class ClassifyLabelController {
List<EngineLabel> allowed = resolveAllowedLabels();
if (allowed.isEmpty()) {
// No vocabulary to classify against (the team stored no labels): pass the file
// through unlabelled rather than ask the engine to classify against nothing.
log.debug("[classify-and-label] {} has no team labels; skipping", fileName);
// No vocabulary to classify against: pass the file through unlabelled rather than
// ask the engine to classify against nothing.
log.debug("[classify-and-label] {} has no labels; skipping", fileName);
return WebResponseUtils.pdfDocToWebResponse(document, fileName, tempFileManager);
}
@@ -175,24 +168,13 @@ public class ClassifyLabelController {
}
/**
* The allowed labels for the caller's team as {@code {id, name}} pairs, de-duplicated by id.
* The engine shows the model the names and returns the ids (icons are presentational and never
* sent). Returns an empty list — the caller then skips classification — when the policy
* subsystem is disabled (no store) or the team has no stored labels. The engine holds no
* default vocabulary of its own, so a team's stored labels are the only source.
* The built-in vocabulary as {@code {id, name}} pairs, de-duplicated by id. The engine shows
* the model the names and returns the ids (icons are presentational and never sent). The engine
* holds no default vocabulary of its own, so this bundled set is the only source.
*/
private List<EngineLabel> resolveAllowedLabels() {
if (labelStore == null) {
return List.of();
}
Long teamId =
policyManagementAuthority == null
? null
: policyManagementAuthority.currentUserTeamId();
Map<String, EngineLabel> byId = new LinkedHashMap<>();
labelStore.findByTeam(teamId).ifPresent(labels -> collectLabels(labels.labels(), byId));
collectLabels(labelProvider.labels(), byId);
return List.copyOf(byId.values());
}
@@ -8,12 +8,14 @@ import java.util.List;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.springframework.core.io.Resource;
import org.springframework.http.HttpStatus;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.server.ResponseStatusException;
import io.github.pixee.security.Filenames;
import io.swagger.v3.oas.annotations.Hidden;
@@ -24,18 +26,24 @@ import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import stirling.software.common.configuration.RuntimePathConfig;
import stirling.software.common.model.ApplicationProperties;
import stirling.software.common.service.CustomPDFDocumentFactory;
import stirling.software.common.util.ProcessExecutor;
import stirling.software.common.util.TempFile;
import stirling.software.common.util.TempFileManager;
import stirling.software.common.util.WebResponseUtils;
import stirling.software.proprietary.model.api.ai.create.AiDocument;
import stirling.software.proprietary.service.AiDocumentHtmlRenderer;
import tools.jackson.core.JacksonException;
import tools.jackson.databind.ObjectMapper;
/**
* Dispatchable tool that converts an AI-generated HTML string to a PDF via WeasyPrint.
* Dispatchable tool that converts an AI-generated document model to a PDF via WeasyPrint.
*
* <p>Called by {@link stirling.software.proprietary.service.AiWorkflowService} when the engine
* emits a {@code CREATE_PDF_FROM_HTML_AGENT} plan step. The HTML comes from a trusted Jinja
* template so sanitization is intentionally skipped.
* emits a {@code CREATE_PDF_FROM_HTML_AGENT} plan step. The engine supplies the document as
* structured fields; the HTML is built here from a fixed template.
*/
@Slf4j
@Hidden
@@ -48,6 +56,9 @@ public class CreatePdfAgentController {
private final TempFileManager tempFileManager;
private final CustomPDFDocumentFactory pdfDocumentFactory;
private final RuntimePathConfig runtimePathConfig;
private final ApplicationProperties applicationProperties;
private final ObjectMapper objectMapper;
private final AiDocumentHtmlRenderer htmlRenderer;
/**
* Returns true only when WeasyPrint is definitively unavailable — either the binary could not
@@ -74,32 +85,42 @@ public class CreatePdfAgentController {
value = "/create-pdf-from-html-agent",
consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
@Operation(
summary = "Convert AI-generated HTML to a PDF",
summary = "Convert an AI-generated document to a PDF",
description =
"Accepts an HTML document as a plain-text parameter and returns a PDF."
+ " This endpoint is dispatched by the AI workflow orchestrator as a"
+ " plan step; it is not intended for direct client use.")
public ResponseEntity<Resource> createPdfFromHtml(
@RequestParam("htmlContent") String htmlContent,
@RequestParam("filename") String filename)
"Accepts a structured document as a JSON parameter and returns a PDF. This"
+ " endpoint is dispatched by the AI workflow orchestrator as a plan"
+ " step; it is not intended for direct client use.")
public ResponseEntity<Resource> createPdf(
@RequestParam("document") String document, @RequestParam("filename") String filename)
throws Exception {
if (!applicationProperties.getAiEngine().isEnabled()) {
throw new ResponseStatusException(HttpStatus.NOT_FOUND);
}
AiDocument model;
try {
model = objectMapper.readValue(document, AiDocument.class);
} catch (JacksonException e) {
throw new ResponseStatusException(HttpStatus.BAD_REQUEST);
}
String html = htmlRenderer.render(model);
log.info(
"[create-pdf-agent] converting HTML to PDF via WeasyPrint — html_bytes={}",
htmlContent.length());
"[create-pdf-agent] converting document to PDF via WeasyPrint — html_bytes={}",
html.length());
try (TempFile htmlFile = tempFileManager.createManagedTempFile(".html");
TempFile pdfFile = tempFileManager.createManagedTempFile(".pdf")) {
Files.writeString(htmlFile.getPath(), htmlContent, StandardCharsets.UTF_8);
Files.writeString(htmlFile.getPath(), html, StandardCharsets.UTF_8);
List<String> command = new ArrayList<>();
command.add(runtimePathConfig.getWeasyPrintPath());
command.add("-e");
command.add("utf-8");
command.add("-v");
// SSRF: the HTML is self-contained and the engine validates style colours, so no
// external url() reaches WeasyPrint. For full isolation, run it network-isolated.
command.add(htmlFile.getAbsolutePath());
command.add(pdfFile.getAbsolutePath());
@@ -126,8 +147,8 @@ public class CreatePdfAgentController {
// avoids materialising the whole document as a byte[] twice (read-all + re-serialise),
// which matters for large generated documents.
TempFile tempOut = tempFileManager.createManagedTempFile(".pdf");
try (PDDocument document = pdfDocumentFactory.load(pdfFile.getPath())) {
document.save(tempOut.getPath().toFile());
try (PDDocument pdDocument = pdfDocumentFactory.load(pdfFile.getPath())) {
pdDocument.save(tempOut.getPath().toFile());
} catch (Exception e) {
tempOut.close();
throw e;
@@ -1,54 +0,0 @@
package stirling.software.proprietary.controller.api;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.DeleteMapping;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestParam;
import io.swagger.v3.oas.annotations.Operation;
import lombok.RequiredArgsConstructor;
import stirling.software.common.annotations.api.ProprietaryUiDataApi;
import stirling.software.proprietary.model.api.apikey.CreateApiKeyRequest;
import stirling.software.proprietary.model.api.apikey.CreatedApiKeyDto;
import stirling.software.proprietary.model.api.apikey.PortalApiKeysResponse;
import stirling.software.proprietary.security.service.ApiKeyManagementService;
/**
* Real backing for the portal Infrastructure → API Keys tab: list/create/revoke named, personal API
* keys. Replaces the former portal-only mock endpoint. Not gated behind an Enterprise license - API
* keys are a core auth feature available on every self-hosted instance.
*/
@ProprietaryUiDataApi
@RequiredArgsConstructor
public class PortalApiKeysController {
private final ApiKeyManagementService apiKeyManagementService;
// tier accepted for endpoint symmetry with the other infra tabs; ignored here.
@GetMapping("/infrastructure/api-keys")
@Operation(summary = "List API keys", description = "The caller's personal API keys.")
public ResponseEntity<PortalApiKeysResponse> list(
@RequestParam(value = "tier", required = false) String tier) {
return ResponseEntity.ok(apiKeyManagementService.listVisibleKeys());
}
@PostMapping("/infrastructure/api-keys")
@Operation(
summary = "Create an API key",
description = "Mints a personal key and returns its one-time secret.")
public ResponseEntity<CreatedApiKeyDto> create(@RequestBody CreateApiKeyRequest request) {
return ResponseEntity.ok(apiKeyManagementService.createKey(request));
}
@DeleteMapping("/infrastructure/api-keys/{id}")
@Operation(summary = "Revoke an API key", description = "Disables a key the caller owns.")
public ResponseEntity<Void> revoke(@PathVariable("id") Long id) {
apiKeyManagementService.revokeKey(id);
return ResponseEntity.noContent().build();
}
}
@@ -0,0 +1,35 @@
package stirling.software.proprietary.integration.crypto;
import jakarta.persistence.AttributeConverter;
import jakarta.persistence.Converter;
/**
* For columns that were once whole-blob encrypted but no longer hold secrets.
*
* <p>Writes plaintext, so the value is readable by any instance regardless of the per-installation
* encryption key. Any value that isn't our ciphertext (already-plaintext JSON, or ciphertext from a
* key we don't hold) is returned as-is; the latter is the caller's to reject.
*/
@Converter
public class LegacyDecryptStringConverter implements AttributeConverter<String, String> {
@Override
public String convertToDatabaseColumn(String attribute) {
return attribute;
}
@Override
public String convertToEntityAttribute(String dbData) {
// Plaintext JSON can never be our Base64 ciphertext ('{' is not in the Base64 alphabet), so
// skip the decrypt attempt for it.
if (dbData == null || dbData.stripLeading().startsWith("{")) {
return dbData;
}
try {
return CredentialEncryption.decrypt(dbData);
} catch (IllegalArgumentException | IllegalStateException e) {
// Legacy ciphertext we can't read (key we don't hold) - the caller's to reject.
return dbData;
}
}
}
@@ -1,31 +0,0 @@
package stirling.software.proprietary.integration.crypto;
import jakarta.persistence.AttributeConverter;
import jakarta.persistence.Converter;
/**
* {@link EncryptedStringConverter} for columns that held plaintext before encryption shipped:
* writes are always encrypted, but a stored value that is not valid ciphertext is returned as-is,
* so pre-encryption rows keep loading and become encrypted on their next save. The discrimination
* is exact for JSON payloads, which can never be mistaken for ciphertext ('{' is not in the Base64
* alphabet). The trade-off is that a genuinely corrupted ciphertext surfaces as garbage to the
* caller's parser instead of failing here.
*/
@Converter
public class LenientEncryptedStringConverter implements AttributeConverter<String, String> {
@Override
public String convertToDatabaseColumn(String attribute) {
return CredentialEncryption.encrypt(attribute);
}
@Override
public String convertToEntityAttribute(String dbData) {
try {
return CredentialEncryption.decrypt(dbData);
} catch (IllegalArgumentException | IllegalStateException e) {
// Not ciphertext: legacy plaintext from before encryption shipped.
return dbData;
}
}
}
@@ -24,8 +24,8 @@ import stirling.software.proprietary.security.model.User;
import stirling.software.proprietary.security.service.UserService;
/**
* API-key auth for the MCP endpoint: validates a Stirling API key and binds the request to that
* user with the MCP scopes.
* API-key auth for the MCP endpoint: validates a Stirling per-user API key and binds the request to
* that user with the MCP scopes.
*/
@Slf4j
public class McpApiKeyAuthFilter extends OncePerRequestFilter {
@@ -0,0 +1,35 @@
package stirling.software.proprietary.model.api.ai.create;
import java.util.List;
import lombok.Data;
@Data
public class AiDocument {
private String title;
private String subtitle;
private String referenceNumber;
private Style style;
private List<Section> sections;
@Data
public static class Style {
private String primaryColor;
private String backgroundColor;
private String bodyTextColor;
}
@Data
public static class Section {
private String type;
private String heading;
private String body;
private List<List<String>> pairs;
private List<String> columns;
private List<List<String>> rows;
private List<String> totalRow;
private List<String> items;
private List<String> signatories;
}
}
@@ -1,4 +0,0 @@
package stirling.software.proprietary.model.api.apikey;
/** Create-key request body from the portal: just a display name for the new personal key. */
public record CreateApiKeyRequest(String name) {}
@@ -1,7 +0,0 @@
package stirling.software.proprietary.model.api.apikey;
import lombok.Builder;
/** Returned once when a key is created: the row plus the plaintext secret, never persisted. */
@Builder
public record CreatedApiKeyDto(PortalApiKeyDto key, String secret) {}
@@ -1,21 +0,0 @@
package stirling.software.proprietary.model.api.apikey;
import lombok.Builder;
/**
* One API key as shown in the portal Infrastructure → API Keys tab. Never carries the secret; that
* is returned once from {@link CreatedApiKeyDto} at creation time.
*/
@Builder
public record PortalApiKeyDto(
String id,
String name,
String prefix,
String created,
String lastUsed,
/** "active" | "revoked". */
String status,
long usageToday,
long usageMonth,
/** Lifetime request count for the key. */
long usageTotal) {}
@@ -1,9 +0,0 @@
package stirling.software.proprietary.model.api.apikey;
import java.util.List;
import lombok.Builder;
/** Payload for the API Keys tab: the personal keys the caller owns. */
@Builder
public record PortalApiKeysResponse(List<PortalApiKeyDto> keys) {}
@@ -6,7 +6,6 @@ import java.util.Arrays;
import java.util.List;
import java.util.Optional;
import org.springframework.boot.autoconfigure.condition.ConditionalOnBooleanProperty;
import org.springframework.core.env.Environment;
import org.springframework.stereotype.Component;
@@ -30,7 +29,6 @@ import stirling.software.proprietary.policy.source.SourceStore;
* defended: an operator who roots an allowlist on a symlink to a sensitive location is trusted.
*/
@Component
@ConditionalOnBooleanProperty(name = "policies.enabled")
public class FolderAccessGuard {
public static final String FOLDER_TYPE = "folder";
@@ -3,7 +3,6 @@ package stirling.software.proprietary.policy.config;
import java.util.List;
import java.util.Objects;
import org.springframework.boot.autoconfigure.condition.ConditionalOnBooleanProperty;
import org.springframework.stereotype.Component;
import lombok.RequiredArgsConstructor;
@@ -23,7 +22,6 @@ import stirling.software.proprietary.policy.store.PolicyStore;
*/
@Component
@RequiredArgsConstructor
@ConditionalOnBooleanProperty(name = "policies.enabled")
public class PolicyAccessGuard {
private final UserServiceInterface userService;
@@ -7,7 +7,6 @@ import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import org.springframework.boot.autoconfigure.condition.ConditionalOnBooleanProperty;
import org.springframework.core.io.FileSystemResource;
import org.springframework.core.io.Resource;
import org.springframework.http.HttpStatus;
@@ -85,7 +84,6 @@ import stirling.software.proprietary.util.SecretMasker;
@Hidden
@RequiredArgsConstructor
@Tag(name = "Policies", description = "Run tool pipelines on the backend")
@ConditionalOnBooleanProperty(name = "policies.enabled")
public class PolicyController {
private final PolicyRunner policyRunner;
@@ -0,0 +1,85 @@
package stirling.software.proprietary.policy.controller;
import org.springframework.web.servlet.HandlerMapping;
import jakarta.servlet.http.HttpServletRequest;
/**
* Policy execute-route namespace under {@code /api/v1/policies} - the paths that actually run an
* automation ({@code /run}, {@code /run/stream}, {@code /{id}/run}, {@code /{id}/trigger}).
*
* <p>Single source of truth for both PAYG entitlement gates, so a caller without billing is blocked
* at the start of a run rather than partway through: the saas {@code EntitlementGuard} gates these
* on {@code FeatureGate.AUTOMATION}, and the self-hosted account-link {@code
* InstanceEntitlementInterceptor} treats them as billable. Read/list policy endpoints are
* deliberately excluded so the UI can still show policies and prompt on use.
*
* <p>This is the sole gate between an unentitled caller and a billable run, so the match is exact
* (not a loose suffix) and segment-anchored. {@code PolicyRunRoutesTest} asserts it against every
* mapping on {@code PolicyController}, so a new execute route that isn't classified here fails the
* build rather than silently running for free.
*/
public final class PolicyRunRoutes {
private static final String BASE = "/api/v1/policies";
private PolicyRunRoutes() {}
/**
* True when the request resolved to a policy execute endpoint. Prefers the matched route
* pattern (context-path independent, set by Spring MVC) and falls back to the raw request URI.
*/
public static boolean matches(HttpServletRequest request) {
Object pattern = request.getAttribute(HandlerMapping.BEST_MATCHING_PATTERN_ATTRIBUTE);
String path = pattern instanceof String s ? s : request.getRequestURI();
String rel = relativeToBase(path);
return rel != null && isExecuteRoute(rel);
}
/**
* The path relative to {@code /api/v1/policies}, or null if the request isn't under that base.
* Segment-anchored (the char after the base must be {@code /} or end-of-string) so a sibling
* like {@code /api/v1/policies-x/...} never matches; tolerates a leading context path.
*/
private static String relativeToBase(String path) {
if (path == null) {
return null;
}
int base = path.indexOf(BASE);
if (base < 0) {
return null;
}
int end = base + BASE.length();
if (end < path.length() && path.charAt(end) != '/') {
return null;
}
return path.substring(end);
}
/**
* The execute routes only: {@code /run}, {@code /run/stream}, and the single-segment {@code
* /{id}/run} / {@code /{id}/trigger} (template or concrete id). Read/list/CRUD routes - {@code
* /run/{runId}}, {@code /runs}, {@code /overview}, {@code /triggers}, {@code /{id}}, {@code
* /order}, {@code /{id}/processed-history}, the base list/create - are all excluded.
*/
private static boolean isExecuteRoute(String rel) {
return rel.equals("/run")
|| rel.equals("/run/stream")
|| isSingleIdRoute(rel, "run")
|| isSingleIdRoute(rel, "trigger");
}
/**
* True for exactly {@code /{oneSegment}/<verb>} (the id being a template or a concrete value).
*/
private static boolean isSingleIdRoute(String rel, String verb) {
String suffix = "/" + verb;
if (!rel.endsWith(suffix)) {
return false;
}
String idSegment = rel.substring(0, rel.length() - suffix.length());
return idSegment.length() > 1
&& idSegment.charAt(0) == '/'
&& idSegment.indexOf('/', 1) < 0;
}
}
@@ -9,7 +9,6 @@ import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ExecutorService;
import org.slf4j.MDC;
import org.springframework.boot.autoconfigure.condition.ConditionalOnBooleanProperty;
import org.springframework.core.io.Resource;
import org.springframework.http.ResponseEntity;
import org.springframework.security.core.Authentication;
@@ -57,7 +56,6 @@ import stirling.software.proprietary.service.DownstreamEntitlementError;
@Slf4j
@Service
@RequiredArgsConstructor
@ConditionalOnBooleanProperty(name = "policies.enabled")
public class PolicyEngine {
// Admission weight for one run. Weighted heavy: a run chains many tools and holds intermediate
@@ -9,7 +9,6 @@ import java.util.concurrent.Executors;
import java.util.concurrent.ScheduledExecutorService;
import java.util.concurrent.TimeUnit;
import org.springframework.boot.autoconfigure.condition.ConditionalOnBooleanProperty;
import org.springframework.stereotype.Service;
import jakarta.annotation.PreDestroy;
@@ -29,7 +28,6 @@ import stirling.software.proprietary.policy.model.PolicyRun;
*/
@Slf4j
@Service
@ConditionalOnBooleanProperty(name = "policies.enabled")
public class PolicyRunRegistry {
private final Map<String, PolicyRun> runs = new ConcurrentHashMap<>();
@@ -5,7 +5,6 @@ import java.util.ArrayList;
import java.util.List;
import java.util.function.Consumer;
import org.springframework.boot.autoconfigure.condition.ConditionalOnBooleanProperty;
import org.springframework.stereotype.Service;
import lombok.RequiredArgsConstructor;
@@ -35,7 +34,6 @@ import stirling.software.proprietary.policy.source.SourceStore;
@Slf4j
@Service
@RequiredArgsConstructor
@ConditionalOnBooleanProperty(name = "policies.enabled")
public class PolicyRunner {
private final PolicyEngine policyEngine;
@@ -2,7 +2,6 @@ package stirling.software.proprietary.policy.engine;
import java.util.List;
import org.springframework.boot.autoconfigure.condition.ConditionalOnBooleanProperty;
import org.springframework.stereotype.Service;
import lombok.RequiredArgsConstructor;
@@ -25,7 +24,6 @@ import stirling.software.proprietary.policy.trigger.PolicyTrigger;
*/
@Service
@RequiredArgsConstructor
@ConditionalOnBooleanProperty(name = "policies.enabled")
public class PolicyValidator {
private final List<PolicyTrigger> triggers;
@@ -14,7 +14,6 @@ import java.util.Map;
import java.util.function.Supplier;
import java.util.stream.Stream;
import org.springframework.boot.autoconfigure.condition.ConditionalOnBooleanProperty;
import org.springframework.core.io.FileSystemResource;
import org.springframework.core.io.Resource;
import org.springframework.stereotype.Service;
@@ -42,7 +41,6 @@ import stirling.software.proprietary.policy.model.PolicyInputs;
@Slf4j
@Service
@RequiredArgsConstructor
@ConditionalOnBooleanProperty(name = "policies.enabled")
public class FolderInputSource implements InputSource {
private static final String TYPE = FolderAccessGuard.FOLDER_TYPE;
@@ -6,7 +6,6 @@ import java.io.InputStream;
import java.util.ArrayList;
import java.util.List;
import org.springframework.boot.autoconfigure.condition.ConditionalOnBooleanProperty;
import org.springframework.core.io.AbstractResource;
import org.springframework.core.io.Resource;
import org.springframework.stereotype.Service;
@@ -49,7 +48,6 @@ import software.amazon.awssdk.services.s3.model.S3Object;
@Slf4j
@Service
@RequiredArgsConstructor
@ConditionalOnBooleanProperty(name = "policies.enabled")
public class S3InputSource implements InputSource {
private static final String TYPE = "s3";
@@ -7,7 +7,6 @@ import java.util.Map;
import java.util.function.Supplier;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.autoconfigure.condition.ConditionalOnBooleanProperty;
import org.springframework.boot.context.event.ApplicationReadyEvent;
import org.springframework.context.event.EventListener;
import org.springframework.dao.DataIntegrityViolationException;
@@ -24,7 +23,6 @@ import lombok.extern.slf4j.Slf4j;
*/
@Slf4j
@Service
@ConditionalOnBooleanProperty(name = "policies.enabled")
public class JpaProcessedLedger implements ProcessedLedger {
private static final int STAMP_CHUNK = 500;
@@ -15,7 +15,6 @@ import java.util.List;
import java.util.UUID;
import java.util.stream.Stream;
import org.springframework.boot.autoconfigure.condition.ConditionalOnBooleanProperty;
import org.springframework.core.io.Resource;
import org.springframework.http.MediaType;
import org.springframework.http.MediaTypeFactory;
@@ -41,7 +40,6 @@ import stirling.software.proprietary.policy.model.OutputSpec;
@Slf4j
@Service
@RequiredArgsConstructor
@ConditionalOnBooleanProperty(name = "policies.enabled")
public class FolderOutputSink implements PolicyOutputSink {
static final String TYPE = FolderAccessGuard.FOLDER_TYPE;
@@ -5,7 +5,6 @@ import java.io.InputStream;
import java.util.ArrayList;
import java.util.List;
import org.springframework.boot.autoconfigure.condition.ConditionalOnBooleanProperty;
import org.springframework.core.io.Resource;
import org.springframework.http.MediaType;
import org.springframework.http.MediaTypeFactory;
@@ -23,7 +22,6 @@ import stirling.software.proprietary.policy.model.OutputSpec;
*/
@Service
@RequiredArgsConstructor
@ConditionalOnBooleanProperty(name = "policies.enabled")
public class InlineOutputSink implements PolicyOutputSink {
private static final String TYPE = "inline";
@@ -13,7 +13,6 @@ import java.util.HexFormat;
import java.util.List;
import java.util.UUID;
import org.springframework.boot.autoconfigure.condition.ConditionalOnBooleanProperty;
import org.springframework.core.io.Resource;
import org.springframework.http.MediaType;
import org.springframework.http.MediaTypeFactory;
@@ -54,7 +53,6 @@ import software.amazon.awssdk.services.s3.model.S3Exception;
@Slf4j
@Service
@RequiredArgsConstructor
@ConditionalOnBooleanProperty(name = "policies.enabled")
public class S3OutputSink implements PolicyOutputSink {
private static final String TYPE = "s3";
@@ -5,7 +5,6 @@ import java.util.HashMap;
import java.util.List;
import java.util.Map;
import org.springframework.boot.autoconfigure.condition.ConditionalOnBooleanProperty;
import org.springframework.stereotype.Service;
import lombok.RequiredArgsConstructor;
@@ -29,7 +28,6 @@ import stirling.software.proprietary.policy.store.PolicyStore;
*/
@Service
@RequiredArgsConstructor
@ConditionalOnBooleanProperty(name = "policies.enabled")
public class PolicyOverviewService {
private final PolicyStore policyStore;
@@ -4,7 +4,6 @@ import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import org.springframework.boot.autoconfigure.condition.ConditionalOnBooleanProperty;
import org.springframework.boot.context.event.ApplicationReadyEvent;
import org.springframework.context.event.EventListener;
import org.springframework.stereotype.Component;
@@ -32,8 +31,8 @@ import tools.jackson.databind.ObjectMapper;
* One-time, idempotent extraction of legacy embedded S3 credentials into stored connections:
* sources and policy outputs written before connections shipped carry bucket/credentials in their
* own options; this rewrites each to reference a (deduplicated) S3 {@link IntegrationConfig} and
* keeps only per-use options (prefix, mode). MUST be programmatic - the option JSON is encrypted at
* the application layer, so no SQL migration can read it.
* keeps only per-use options (prefix, mode). MUST be programmatic - it parses and rewrites the
* option JSON (and decrypts any legacy ciphertext row on read), which no SQL migration can do.
*
* <p>Idempotent by construction: rewritten rows no longer embed credentials, so re-runs find
* nothing to do. Connections are deduplicated against both this run's extractions and existing S3
@@ -44,7 +43,6 @@ import tools.jackson.databind.ObjectMapper;
@Slf4j
@Component
@RequiredArgsConstructor
@ConditionalOnBooleanProperty(name = "policies.enabled")
public class EmbeddedS3CredentialMigration {
private static final ObjectMapper OBJECT_MAPPER = new ObjectMapper();
@@ -4,7 +4,6 @@ import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import org.springframework.boot.autoconfigure.condition.ConditionalOnBooleanProperty;
import org.springframework.stereotype.Component;
import lombok.RequiredArgsConstructor;
@@ -23,7 +22,6 @@ import stirling.software.proprietary.policy.store.PolicyStore;
*/
@Component
@RequiredArgsConstructor
@ConditionalOnBooleanProperty(name = "policies.enabled")
public class PolicyS3ConnectionUsageCheck implements IntegrationConfigUsageCheck {
private final SourceStore sourceStore;
@@ -6,7 +6,6 @@ import java.util.concurrent.ConcurrentHashMap;
import java.util.function.Function;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.autoconfigure.condition.ConditionalOnBooleanProperty;
import org.springframework.stereotype.Service;
import jakarta.annotation.PreDestroy;
@@ -32,7 +31,6 @@ import software.amazon.awssdk.services.s3.S3Configuration;
* users rather than the operator.
*/
@Service
@ConditionalOnBooleanProperty(name = "policies.enabled")
public class S3ConnectionPool {
private final ApplicationProperties applicationProperties;
@@ -3,7 +3,6 @@ package stirling.software.proprietary.policy.s3;
import java.util.LinkedHashMap;
import java.util.Map;
import org.springframework.boot.autoconfigure.condition.ConditionalOnBooleanProperty;
import org.springframework.security.core.Authentication;
import org.springframework.security.core.context.SecurityContextHolder;
import org.springframework.security.core.userdetails.UserDetails;
@@ -39,7 +38,6 @@ import tools.jackson.databind.ObjectMapper;
@Service
@RequiredArgsConstructor
@Transactional(readOnly = true)
@ConditionalOnBooleanProperty(name = "policies.enabled")
public class S3ConnectionResolver {
static final String CONNECTION_ID_OPTION = "connectionId";
@@ -3,7 +3,6 @@ package stirling.software.proprietary.policy.s3;
import java.net.URI;
import java.util.Map;
import org.springframework.boot.autoconfigure.condition.ConditionalOnBooleanProperty;
import org.springframework.stereotype.Component;
import lombok.RequiredArgsConstructor;
@@ -21,7 +20,6 @@ import stirling.software.proprietary.integration.service.IntegrationConfigValida
*/
@Component
@RequiredArgsConstructor
@ConditionalOnBooleanProperty(name = "policies.enabled")
public class S3IntegrationValidator implements IntegrationConfigValidator {
private final ApplicationProperties applicationProperties;
@@ -10,7 +10,6 @@ import java.util.function.IntSupplier;
import java.util.function.Supplier;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.autoconfigure.condition.ConditionalOnBooleanProperty;
import org.springframework.dao.DataIntegrityViolationException;
import org.springframework.scheduling.annotation.Scheduled;
import org.springframework.stereotype.Service;
@@ -24,7 +23,6 @@ import org.springframework.stereotype.Service;
* table stays bounded (~one row per source per active hour, for at most 30 days).
*/
@Service
@ConditionalOnBooleanProperty(name = "policies.enabled")
public class JpaSourceDocCounter implements SourceDocCounter {
private final SourceDocCountRepository countRepository;
@@ -4,10 +4,10 @@ import java.util.List;
import java.util.Optional;
import java.util.UUID;
import org.springframework.boot.autoconfigure.condition.ConditionalOnBooleanProperty;
import org.springframework.stereotype.Service;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import tools.jackson.databind.ObjectMapper;
@@ -15,9 +15,9 @@ import tools.jackson.databind.ObjectMapper;
* Durable {@link SourceStore} backed by JPA; the runtime store. Sources are persisted as JSON via
* {@link SourceEntity}, with scalar columns kept in sync for querying.
*/
@Slf4j
@Service
@RequiredArgsConstructor
@ConditionalOnBooleanProperty(name = "policies.enabled")
public class JpaSourceStore implements SourceStore {
private final SourceRepository repository;
@@ -53,17 +53,20 @@ public class JpaSourceStore implements SourceStore {
@Override
public Optional<Source> get(String id) {
return repository.findById(id).map(this::toSource);
return repository.findById(id).flatMap(this::toSource);
}
@Override
public List<Source> all() {
return repository.findAll().stream().map(this::toSource).toList();
return repository.findAll().stream().map(this::toSource).flatMap(Optional::stream).toList();
}
@Override
public List<Source> findByTeam(Long teamId) {
return repository.findByTeam(teamId).stream().map(this::toSource).toList();
return repository.findByTeam(teamId).stream()
.map(this::toSource)
.flatMap(Optional::stream)
.toList();
}
@Override
@@ -75,7 +78,20 @@ public class JpaSourceStore implements SourceStore {
return true;
}
private Source toSource(SourceEntity entity) {
return objectMapper.readValue(entity.getSourceJson(), Source.class);
// Skip (don't fail) rows whose JSON can't be read - e.g. written by another app version/key.
// One unreadable row must never abort a bulk read or crash startup.
private Optional<Source> toSource(SourceEntity entity) {
try {
return Optional.of(objectMapper.readValue(entity.getSourceJson(), Source.class));
} catch (Exception e) {
log.error(
"Skipping unreadable policy source id={} name={}: stored JSON could not be"
+ " parsed ({}). Likely written by a different app version or"
+ " encryption key.",
entity.getId(),
entity.getName(),
e.getMessage());
return Optional.empty();
}
}
}
@@ -3,7 +3,6 @@ package stirling.software.proprietary.policy.source;
import java.util.List;
import java.util.Objects;
import org.springframework.boot.autoconfigure.condition.ConditionalOnBooleanProperty;
import org.springframework.stereotype.Component;
import lombok.RequiredArgsConstructor;
@@ -20,7 +19,6 @@ import stirling.software.proprietary.policy.config.PolicyManagementAuthority;
*/
@Component
@RequiredArgsConstructor
@ConditionalOnBooleanProperty(name = "policies.enabled")
public class SourceAccessGuard {
private final UserServiceInterface userService;
@@ -3,7 +3,6 @@ package stirling.software.proprietary.policy.source;
import java.util.List;
import java.util.Map;
import org.springframework.boot.autoconfigure.condition.ConditionalOnBooleanProperty;
import org.springframework.http.HttpStatus;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
@@ -43,7 +42,6 @@ import stirling.software.proprietary.util.SecretMasker;
@Hidden
@RequiredArgsConstructor
@Tag(name = "Sources", description = "Reusable policy input connections")
@ConditionalOnBooleanProperty(name = "policies.enabled")
public class SourceController {
private final SourceStore sourceStore;
@@ -12,7 +12,7 @@ import lombok.Getter;
import lombok.NoArgsConstructor;
import lombok.Setter;
import stirling.software.proprietary.integration.crypto.LenientEncryptedStringConverter;
import stirling.software.proprietary.integration.crypto.LegacyDecryptStringConverter;
/**
* JPA row for a {@link Source}. The whole source lives as JSON in {@code sourceJson} (authoritative
@@ -48,9 +48,9 @@ public class SourceEntity implements Serializable {
@Column(name = "enabled")
private boolean enabled;
// Encrypted at rest: source options carry user-supplied credentials (e.g. an S3 secret
// access key). Lenient so rows written before encryption shipped still load.
@Convert(converter = LenientEncryptedStringConverter.class)
// Plaintext at rest: the S3 credentials that used to live here now sit in a referenced
// IntegrationConfig connection (still encrypted). Decrypts legacy ciphertext on read.
@Convert(converter = LegacyDecryptStringConverter.class)
@Column(name = "source_json", columnDefinition = "text")
private String sourceJson;
}
@@ -6,7 +6,6 @@ import java.util.HashMap;
import java.util.List;
import java.util.Map;
import org.springframework.boot.autoconfigure.condition.ConditionalOnBooleanProperty;
import org.springframework.stereotype.Service;
import lombok.RequiredArgsConstructor;
@@ -24,7 +23,6 @@ import stirling.software.proprietary.util.SecretMasker;
*/
@Service
@RequiredArgsConstructor
@ConditionalOnBooleanProperty(name = "policies.enabled")
public class SourceOverviewService {
private final SourceStore sourceStore;
@@ -5,11 +5,11 @@ import java.util.Objects;
import java.util.Optional;
import java.util.UUID;
import org.springframework.boot.autoconfigure.condition.ConditionalOnBooleanProperty;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import stirling.software.proprietary.policy.model.Policy;
@@ -19,9 +19,9 @@ import tools.jackson.databind.ObjectMapper;
* Durable {@link PolicyStore} backed by JPA; the runtime store. Policies are persisted as JSON via
* {@link PolicyEntity}, with scalar columns kept in sync for querying.
*/
@Slf4j
@Service
@RequiredArgsConstructor
@ConditionalOnBooleanProperty(name = "policies.enabled")
public class JpaPolicyStore implements PolicyStore {
private final PolicyRepository repository;
@@ -98,23 +98,30 @@ public class JpaPolicyStore implements PolicyStore {
@Override
public Optional<Policy> get(String id) {
return repository.findById(id).map(this::toPolicy);
return repository.findById(id).flatMap(this::toPolicy);
}
@Override
public List<Policy> all() {
return repository.findAllOrdered().stream().map(this::toPolicy).toList();
return repository.findAllOrdered().stream()
.map(this::toPolicy)
.flatMap(Optional::stream)
.toList();
}
@Override
public List<Policy> findByTeam(Long teamId) {
return repository.findByTeam(teamId).stream().map(this::toPolicy).toList();
return repository.findByTeam(teamId).stream()
.map(this::toPolicy)
.flatMap(Optional::stream)
.toList();
}
@Override
public List<Policy> findByTriggerType(String triggerType) {
return repository.findByTriggerTypeAndEnabledTrue(triggerType).stream()
.map(this::toPolicy)
.flatMap(Optional::stream)
.toList();
}
@@ -127,7 +134,19 @@ public class JpaPolicyStore implements PolicyStore {
return true;
}
private Policy toPolicy(PolicyEntity entity) {
return objectMapper.readValue(entity.getPolicyJson(), Policy.class);
// Skip (don't fail) rows whose JSON can't be read - e.g. written by another app version/key.
// One unreadable row must never abort a bulk read or crash startup.
private Optional<Policy> toPolicy(PolicyEntity entity) {
try {
return Optional.of(objectMapper.readValue(entity.getPolicyJson(), Policy.class));
} catch (Exception e) {
log.error(
"Skipping unreadable policy id={} name={}: stored JSON could not be parsed"
+ " ({}). Likely written by a different app version or encryption key.",
entity.getId(),
entity.getName(),
e.getMessage());
return Optional.empty();
}
}
}
@@ -12,7 +12,7 @@ import lombok.Getter;
import lombok.NoArgsConstructor;
import lombok.Setter;
import stirling.software.proprietary.integration.crypto.LenientEncryptedStringConverter;
import stirling.software.proprietary.integration.crypto.LegacyDecryptStringConverter;
/**
* JPA row for a {@link stirling.software.proprietary.policy.model.Policy}. The whole policy lives
@@ -58,9 +58,9 @@ public class PolicyEntity implements Serializable {
@Column(name = "sort_order")
private Integer sortOrder;
// Encrypted at rest: output options carry user-supplied credentials (e.g. an S3 secret
// access key). Lenient so rows written before encryption shipped still load.
@Convert(converter = LenientEncryptedStringConverter.class)
// Plaintext at rest: the S3 credentials that used to live here now sit in a referenced
// IntegrationConfig connection (still encrypted). Decrypts legacy ciphertext on read.
@Convert(converter = LegacyDecryptStringConverter.class)
@Column(name = "policy_json", columnDefinition = "text")
private String policyJson;
}
@@ -20,7 +20,6 @@ import java.util.concurrent.Executors;
import java.util.concurrent.ScheduledExecutorService;
import java.util.concurrent.TimeUnit;
import org.springframework.boot.autoconfigure.condition.ConditionalOnBooleanProperty;
import org.springframework.stereotype.Service;
import lombok.RequiredArgsConstructor;
@@ -50,7 +49,6 @@ import stirling.software.proprietary.policy.store.PolicyStore;
@Slf4j
@Service
@RequiredArgsConstructor
@ConditionalOnBooleanProperty(name = "policies.enabled")
public class FolderWatchTrigger implements PolicyTrigger {
private static final String TYPE = "folder-watch";
@@ -2,7 +2,6 @@ package stirling.software.proprietary.policy.trigger;
import java.util.List;
import org.springframework.boot.autoconfigure.condition.ConditionalOnBooleanProperty;
import org.springframework.context.SmartLifecycle;
import org.springframework.stereotype.Service;
@@ -13,7 +12,6 @@ import lombok.extern.slf4j.Slf4j;
@Slf4j
@Service
@RequiredArgsConstructor
@ConditionalOnBooleanProperty(name = "policies.enabled")
public class PolicyTriggerManager implements SmartLifecycle {
private final List<PolicyTrigger> triggers;
@@ -10,7 +10,6 @@ import java.util.concurrent.Executors;
import java.util.concurrent.ScheduledExecutorService;
import java.util.concurrent.TimeUnit;
import org.springframework.boot.autoconfigure.condition.ConditionalOnBooleanProperty;
import org.springframework.stereotype.Service;
import lombok.RequiredArgsConstructor;
@@ -32,7 +31,6 @@ import tools.jackson.databind.ObjectMapper;
@Slf4j
@Service
@RequiredArgsConstructor
@ConditionalOnBooleanProperty(name = "policies.enabled")
public class ScheduleTrigger implements PolicyTrigger {
private static final String TYPE = "schedule";
@@ -37,8 +37,7 @@ import stirling.software.common.model.exception.UnsupportedProviderException;
"stirling.software.proprietary.policy.ledger",
"stirling.software.proprietary.accountlink",
"stirling.software.proprietary.access.repository",
"stirling.software.proprietary.integration.repository",
"stirling.software.proprietary.classification.store"
"stirling.software.proprietary.integration.repository"
})
@EntityScan({
"stirling.software.proprietary.security.model",
@@ -50,8 +49,7 @@ import stirling.software.common.model.exception.UnsupportedProviderException;
"stirling.software.proprietary.policy.ledger",
"stirling.software.proprietary.accountlink",
"stirling.software.proprietary.access.model",
"stirling.software.proprietary.integration.model",
"stirling.software.proprietary.classification.store"
"stirling.software.proprietary.integration.model"
})
public class DatabaseConfig {
@@ -57,7 +57,6 @@ import stirling.software.proprietary.security.oauth2.TauriAuthorizationRequestRe
import stirling.software.proprietary.security.saml2.CustomSaml2AuthenticationFailureHandler;
import stirling.software.proprietary.security.saml2.CustomSaml2AuthenticationSuccessHandler;
import stirling.software.proprietary.security.saml2.CustomSaml2ResponseAuthenticationConverter;
import stirling.software.proprietary.security.service.ApiKeyAuthenticationService;
import stirling.software.proprietary.security.service.CustomOAuth2UserService;
import stirling.software.proprietary.security.service.CustomUserDetailsService;
import stirling.software.proprietary.security.service.JwtServiceInterface;
@@ -485,14 +484,12 @@ public class SecurityConfiguration {
}
@Bean
public JwtAuthenticationFilter jwtAuthenticationFilter(
ApiKeyAuthenticationService apiKeyAuthenticationService) {
public JwtAuthenticationFilter jwtAuthenticationFilter() {
return new JwtAuthenticationFilter(
jwtService,
userService,
userDetailsService,
jwtAuthenticationEntryPoint,
securityProperties,
apiKeyAuthenticationService);
securityProperties);
}
}
@@ -11,7 +11,6 @@ import java.sql.SQLException;
import java.util.Map;
import java.util.Optional;
import org.slf4j.MDC;
import org.springframework.security.authentication.UsernamePasswordAuthenticationToken;
import org.springframework.security.core.Authentication;
import org.springframework.security.core.AuthenticationException;
@@ -34,9 +33,8 @@ import stirling.software.common.model.ApplicationProperties;
import stirling.software.common.model.exception.UnsupportedProviderException;
import stirling.software.proprietary.security.model.ApiKeyAuthenticationToken;
import stirling.software.proprietary.security.model.AuthenticationType;
import stirling.software.proprietary.security.model.User;
import stirling.software.proprietary.security.model.exception.AuthenticationFailureException;
import stirling.software.proprietary.security.service.ApiKeyAuthenticationService;
import stirling.software.proprietary.security.service.ApiKeyAuthenticationService.ApiKeyAuthentication;
import stirling.software.proprietary.security.service.CustomUserDetailsService;
import stirling.software.proprietary.security.service.JwtServiceInterface;
import stirling.software.proprietary.security.service.UserService;
@@ -50,15 +48,11 @@ public class JwtAuthenticationFilter extends OncePerRequestFilter {
private final CustomUserDetailsService userDetailsService;
private final AuthenticationEntryPoint authenticationEntryPoint;
private final ApplicationProperties.Security securityProperties;
private final ApiKeyAuthenticationService apiKeyAuthenticationService;
@Override
protected void doFilterInternal(
HttpServletRequest request, HttpServletResponse response, FilterChain filterChain)
throws ServletException, IOException {
// Start clean so a pooled thread can't inherit a prior request's key label. This filter
// runs before UserAuthenticationFilter, so in JWT mode it owns the API-key label lifecycle.
MDC.remove(ApiKeyAuthenticationService.AUDIT_LABEL_MDC_KEY);
if (!jwtService.isJwtEnabled()) {
filterChain.doFilter(request, response);
return;
@@ -137,14 +131,9 @@ public class JwtAuthenticationFilter extends OncePerRequestFilter {
if (apiKey != null && !apiKey.isBlank()) {
try {
// Resolve through the shared service so the multi-key table (then the legacy
// per-user key) is consulted and per-key usage is recorded; the key runs as its
// owner. It also yields a per-key label for the processor's document
// attribution.
Optional<ApiKeyAuthentication> resolved =
apiKeyAuthenticationService.authenticate(apiKey);
Optional<User> user = userService.getUserByApiKey(apiKey);
if (resolved.isEmpty()) {
if (user.isEmpty()) {
handleAuthenticationFailure(
request,
response,
@@ -154,13 +143,8 @@ public class JwtAuthenticationFilter extends OncePerRequestFilter {
authentication =
new ApiKeyAuthenticationToken(
resolved.get().user(), apiKey, resolved.get().authorities());
user.get(), apiKey, user.get().getAuthorities());
SecurityContextHolder.getContext().setAuthentication(authentication);
if (resolved.get().auditLabel() != null) {
MDC.put(
ApiKeyAuthenticationService.AUDIT_LABEL_MDC_KEY,
resolved.get().auditLabel());
}
return true;
} catch (AuthenticationException e) {
handleAuthenticationFailure(
@@ -6,7 +6,6 @@ import java.io.IOException;
import java.util.List;
import java.util.Optional;
import org.slf4j.MDC;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.context.annotation.Lazy;
import org.springframework.context.annotation.Profile;
@@ -34,8 +33,6 @@ import stirling.software.common.util.RequestUriUtils;
import stirling.software.proprietary.security.model.ApiKeyAuthenticationToken;
import stirling.software.proprietary.security.model.User;
import stirling.software.proprietary.security.saml2.CustomSaml2AuthenticatedPrincipal;
import stirling.software.proprietary.security.service.ApiKeyAuthenticationService;
import stirling.software.proprietary.security.service.ApiKeyAuthenticationService.ApiKeyAuthentication;
import stirling.software.proprietary.security.service.UserService;
import stirling.software.proprietary.security.session.SessionPersistentRegistry;
@@ -44,24 +41,18 @@ import stirling.software.proprietary.security.session.SessionPersistentRegistry;
@Profile("!saas")
public class UserAuthenticationFilter extends OncePerRequestFilter {
/** MDC key carrying the resolved key's label into audit events for the processor feed. */
public static final String API_KEY_LABEL_MDC = ApiKeyAuthenticationService.AUDIT_LABEL_MDC_KEY;
private final ApplicationProperties.Security securityProp;
private final UserService userService;
private final ApiKeyAuthenticationService apiKeyAuthenticationService;
private final SessionPersistentRegistry sessionPersistentRegistry;
private final boolean loginEnabledValue;
public UserAuthenticationFilter(
@Lazy ApplicationProperties.Security securityProp,
@Lazy UserService userService,
ApiKeyAuthenticationService apiKeyAuthenticationService,
SessionPersistentRegistry sessionPersistentRegistry,
@Qualifier("loginEnabled") boolean loginEnabledValue) {
this.securityProp = securityProp;
this.userService = userService;
this.apiKeyAuthenticationService = apiKeyAuthenticationService;
this.sessionPersistentRegistry = sessionPersistentRegistry;
this.loginEnabledValue = loginEnabledValue;
}
@@ -71,14 +62,6 @@ public class UserAuthenticationFilter extends OncePerRequestFilter {
HttpServletRequest request, HttpServletResponse response, FilterChain filterChain)
throws ServletException, IOException {
// Start each request clean so a pooled thread can't inherit a prior request's key label -
// but keep a label an upstream filter (JwtAuthenticationFilter) already set for a request
// it API-key-authenticated, otherwise per-key attribution is lost on the JWT path.
if (!(SecurityContextHolder.getContext().getAuthentication()
instanceof ApiKeyAuthenticationToken)) {
MDC.remove(API_KEY_LABEL_MDC);
}
if (!loginEnabledValue) {
// If login is not enabled, just pass all requests without authentication
filterChain.doFilter(request, response);
@@ -106,23 +89,18 @@ public class UserAuthenticationFilter extends OncePerRequestFilter {
String apiKey = request.getHeader("X-API-KEY");
if (apiKey != null && !apiKey.trim().isEmpty()) {
try {
// Resolves the multi-key table then the legacy key, records usage, and yields a
// per-key label for the processor's document-source attribution.
Optional<ApiKeyAuthentication> resolved =
apiKeyAuthenticationService.authenticate(apiKey);
if (resolved.isEmpty()) {
// Use API key to authenticate. This requires you to have an authentication
// provider for API keys.
Optional<User> user = userService.getUserByApiKey(apiKey);
if (user.isEmpty()) {
response.setStatus(HttpStatus.UNAUTHORIZED.value());
response.getWriter().write("Invalid API Key.");
return;
}
User user = resolved.get().user();
authentication =
new ApiKeyAuthenticationToken(
user, apiKey, resolved.get().authorities());
user.get(), apiKey, user.get().getAuthorities());
SecurityContextHolder.getContext().setAuthentication(authentication);
if (resolved.get().auditLabel() != null) {
MDC.put(API_KEY_LABEL_MDC, resolved.get().auditLabel());
}
} catch (AuthenticationException e) {
// If API key authentication fails, deny the request
response.setStatus(HttpStatus.UNAUTHORIZED.value());
@@ -11,6 +11,7 @@ import org.springframework.http.HttpStatus;
import org.springframework.security.core.Authentication;
import org.springframework.security.core.GrantedAuthority;
import org.springframework.security.core.context.SecurityContextHolder;
import org.springframework.security.core.userdetails.UserDetails;
import org.springframework.stereotype.Component;
import org.springframework.web.filter.OncePerRequestFilter;
@@ -57,25 +58,23 @@ public class UserBasedRateLimitingFilter extends OncePerRequestFilter {
filterChain.doFilter(request, response);
return;
}
// Bucket by the resolved user (the auth filter runs first and populates the context, even
// for X-API-KEY requests), so all of a user's API keys share ONE per-user quota - minting
// extra keys can't multiply the daily limit. Fall back to the raw key / IP only when the
// request is unauthenticated.
String identifier = null;
Authentication authentication = SecurityContextHolder.getContext().getAuthentication();
if (authentication != null
&& authentication.isAuthenticated()
&& !"anonymousUser".equals(authentication.getName())) {
identifier = authentication.getName();
}
if (identifier == null) {
String apiKey = request.getHeader("X-API-KEY");
if (apiKey != null && !apiKey.trim().isEmpty()) {
identifier = "API_KEY_" + apiKey;
} else {
identifier = request.getRemoteAddr();
// Check for API key in the request headers
String apiKey = request.getHeader("X-API-KEY");
if (apiKey != null && !apiKey.trim().isEmpty()) {
identifier = // Prefix to distinguish between API keys and usernames
"API_KEY_" + apiKey;
} else {
Authentication authentication = SecurityContextHolder.getContext().getAuthentication();
if (authentication != null && authentication.isAuthenticated()) {
UserDetails userDetails = (UserDetails) authentication.getPrincipal();
identifier = userDetails.getUsername();
}
}
// If neither API key nor an authenticated user is present, use IP address
if (identifier == null) {
identifier = request.getRemoteAddr();
}
Role userRole =
getRoleFromAuthentication(SecurityContextHolder.getContext().getAuthentication());
if (request.getHeader("X-API-KEY") != null) {
@@ -1,72 +0,0 @@
package stirling.software.proprietary.security.model;
import java.io.Serializable;
import java.time.Instant;
import jakarta.persistence.*;
import lombok.AllArgsConstructor;
import lombok.Builder;
import lombok.Getter;
import lombok.NoArgsConstructor;
import lombok.Setter;
/**
* A named, personal API key belonging to a user. The raw secret is shown once at creation and never
* stored; only its SHA-256 hash is persisted, so a leaked database row cannot be replayed. Distinct
* from the legacy single {@code users.apiKey} column, which stays a per-user key for backward
* compatibility and is lazily represented here.
*/
@Entity
@Table(
name = "api_keys",
indexes = {
@Index(name = "idx_api_key_hash", columnList = "key_hash", unique = true),
@Index(name = "idx_api_key_owner", columnList = "owner_user_id")
})
@Getter
@Setter
@Builder
@NoArgsConstructor
@AllArgsConstructor
public class ApiKey implements Serializable {
private static final long serialVersionUID = 1L;
@Id
@GeneratedValue(strategy = GenerationType.IDENTITY)
@Column(name = "id")
private Long id;
@Column(name = "name", nullable = false, length = 100)
private String name;
/** SHA-256 hex of the raw key; the raw value is never persisted. */
@Column(name = "key_hash", nullable = false, unique = true, length = 64)
private String keyHash;
/** Non-secret leading fragment of the raw key, shown in listings (e.g. {@code sk_a1b2c3d4}). */
@Column(name = "prefix", nullable = false, length = 32)
private String prefix;
/** The user who created and owns the key; the key authenticates as this user. */
@Column(name = "owner_user_id", nullable = false)
private Long ownerUserId;
@Column(name = "enabled", nullable = false)
private boolean enabled;
@Column(name = "created_at", nullable = false)
private Instant createdAt;
@Column(name = "last_used_at")
private Instant lastUsedAt;
@Column(name = "revoked_at")
private Instant revokedAt;
/** Active = enabled and not revoked; only active keys authenticate. */
public boolean isActive() {
return enabled && revokedAt == null;
}
}
@@ -5,7 +5,6 @@ import java.util.Collection;
import org.springframework.security.authentication.AbstractAuthenticationToken;
import org.springframework.security.core.GrantedAuthority;
/** Authentication produced from an {@code X-API-KEY} header; runs as the key's owner. */
public class ApiKeyAuthenticationToken extends AbstractAuthenticationToken {
private final Object principal;
@@ -1,45 +0,0 @@
package stirling.software.proprietary.security.model;
import java.io.Serializable;
import jakarta.persistence.Column;
import jakarta.persistence.Entity;
import jakarta.persistence.Id;
import jakarta.persistence.IdClass;
import jakarta.persistence.Table;
import lombok.Getter;
import lombok.NoArgsConstructor;
import lombok.Setter;
/**
* One UTC day's request tally for an API key. Rolling "today"/"this month" usage is summed from
* these rows, keeping the table at one row per key per active day rather than one per request.
*/
@Entity
@Table(name = "api_key_daily_usage")
@IdClass(ApiKeyDailyUsageId.class)
@Getter
@Setter
@NoArgsConstructor
public class ApiKeyDailyUsage implements Serializable {
private static final long serialVersionUID = 1L;
@Id
@Column(name = "api_key_id")
private Long apiKeyId;
@Id
@Column(name = "epoch_day")
private long epochDay;
@Column(name = "count")
private long count;
public ApiKeyDailyUsage(Long apiKeyId, long epochDay, long count) {
this.apiKeyId = apiKeyId;
this.epochDay = epochDay;
this.count = count;
}
}
@@ -1,36 +0,0 @@
package stirling.software.proprietary.security.model;
import java.io.Serializable;
import java.util.Objects;
/** Composite key for {@link ApiKeyDailyUsage}: one row per key per UTC day. */
public class ApiKeyDailyUsageId implements Serializable {
private static final long serialVersionUID = 1L;
private Long apiKeyId;
private long epochDay;
public ApiKeyDailyUsageId() {}
public ApiKeyDailyUsageId(Long apiKeyId, long epochDay) {
this.apiKeyId = apiKeyId;
this.epochDay = epochDay;
}
@Override
public boolean equals(Object o) {
if (this == o) {
return true;
}
if (!(o instanceof ApiKeyDailyUsageId other)) {
return false;
}
return epochDay == other.epochDay && Objects.equals(apiKeyId, other.apiKeyId);
}
@Override
public int hashCode() {
return Objects.hash(apiKeyId, epochDay);
}
}
@@ -1,55 +0,0 @@
package stirling.software.proprietary.security.repository;
import java.util.Collection;
import java.util.List;
import org.springframework.data.jpa.repository.JpaRepository;
import org.springframework.data.jpa.repository.Modifying;
import org.springframework.data.jpa.repository.Query;
import org.springframework.data.repository.query.Param;
import org.springframework.stereotype.Repository;
import stirling.software.proprietary.security.model.ApiKeyDailyUsage;
import stirling.software.proprietary.security.model.ApiKeyDailyUsageId;
@Repository
public interface ApiKeyDailyUsageRepository
extends JpaRepository<ApiKeyDailyUsage, ApiKeyDailyUsageId> {
/** Atomically bump today's tally; returns 0 when no row exists yet (caller then inserts). */
@Modifying
@Query(
"UPDATE ApiKeyDailyUsage u SET u.count = u.count + 1 "
+ "WHERE u.apiKeyId = :apiKeyId AND u.epochDay = :epochDay")
int incrementIfPresent(@Param("apiKeyId") Long apiKeyId, @Param("epochDay") long epochDay);
@Query(
"SELECT COALESCE(SUM(u.count), 0) FROM ApiKeyDailyUsage u "
+ "WHERE u.apiKeyId = :apiKeyId AND u.epochDay >= :fromDayInclusive")
long sumSince(
@Param("apiKeyId") Long apiKeyId, @Param("fromDayInclusive") long fromDayInclusive);
@Query(
"SELECT u.count FROM ApiKeyDailyUsage u "
+ "WHERE u.apiKeyId = :apiKeyId AND u.epochDay = :epochDay")
Long countForDay(@Param("apiKeyId") Long apiKeyId, @Param("epochDay") long epochDay);
/** Batched today-count for many keys in one query (avoids N+1 when listing keys). */
@Query(
"SELECT u.apiKeyId AS apiKeyId, u.count AS total FROM ApiKeyDailyUsage u "
+ "WHERE u.apiKeyId IN :ids AND u.epochDay = :epochDay")
List<ApiKeyUsageSum> countForDayByIds(
@Param("ids") Collection<Long> ids, @Param("epochDay") long epochDay);
/** Batched trailing-window sum for many keys in one query. */
@Query(
"SELECT u.apiKeyId AS apiKeyId, SUM(u.count) AS total FROM ApiKeyDailyUsage u "
+ "WHERE u.apiKeyId IN :ids AND u.epochDay >= :fromDayInclusive "
+ "GROUP BY u.apiKeyId")
List<ApiKeyUsageSum> sumSinceByIds(
@Param("ids") Collection<Long> ids, @Param("fromDayInclusive") long fromDayInclusive);
void deleteByApiKeyId(Long apiKeyId);
List<ApiKeyDailyUsage> findByApiKeyId(Long apiKeyId);
}
@@ -1,19 +0,0 @@
package stirling.software.proprietary.security.repository;
import java.util.List;
import java.util.Optional;
import org.springframework.data.jpa.repository.JpaRepository;
import org.springframework.stereotype.Repository;
import stirling.software.proprietary.security.model.ApiKey;
@Repository
public interface ApiKeyRepository extends JpaRepository<ApiKey, Long> {
Optional<ApiKey> findByKeyHash(String keyHash);
boolean existsByKeyHash(String keyHash);
List<ApiKey> findByOwnerUserIdOrderByCreatedAtDesc(Long ownerUserId);
}
@@ -1,8 +0,0 @@
package stirling.software.proprietary.security.repository;
/** Projection: a key id and a usage total, for batching per-key usage into one query. */
public interface ApiKeyUsageSum {
Long getApiKeyId();
Long getTotal();
}
@@ -1,107 +0,0 @@
package stirling.software.proprietary.security.service;
import java.time.Instant;
import java.util.Collection;
import java.util.Optional;
import org.springframework.security.core.GrantedAuthority;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
import lombok.RequiredArgsConstructor;
import stirling.software.proprietary.security.database.repository.UserRepository;
import stirling.software.proprietary.security.model.ApiKey;
import stirling.software.proprietary.security.model.User;
import stirling.software.proprietary.security.repository.ApiKeyRepository;
/**
* Resolves an incoming {@code X-API-KEY} to its owning user and records per-key usage. Depends only
* on repositories (never {@code UserService}) so {@code UserService} can delegate here without a
* bean cycle.
*
* <p>Resolution order: the multi-key {@code api_keys} table first (by hash), then the legacy
* per-user {@code users.apiKey} column. Legacy keys therefore keep working unchanged. Every key is
* personal and authenticates as its owner with the owner's authorities.
*/
@Service
@RequiredArgsConstructor
public class ApiKeyAuthenticationService {
/**
* MDC key that carries the resolved key's label into audit events so the processor's Documents
* feed can attribute a document to the specific key. Set by the auth filters (both flavors),
* read by {@code CustomAuditEventRepository}.
*/
public static final String AUDIT_LABEL_MDC_KEY = "apiKeyLabel";
private final ApiKeyRepository apiKeyRepository;
private final ApiKeyUsageRecorder usageRecorder;
private final UserRepository userRepository;
/** The user a raw key authenticates as, or empty if it matches no active key. */
public Optional<User> resolveUser(String rawKey) {
return authenticate(rawKey).map(ApiKeyAuthentication::user);
}
/**
* Resolve a raw key, recording usage as a side effect. Returns the owning user, a display label
* for the resolved key ({@code null} for the legacy per-user key), and the owner's authorities.
*/
public Optional<ApiKeyAuthentication> authenticate(String rawKey) {
if (rawKey == null || rawKey.isBlank()) {
return Optional.empty();
}
ApiKey key = apiKeyRepository.findByKeyHash(ApiKeyHasher.hash(rawKey)).orElse(null);
if (key != null) {
if (!key.isActive()) {
return Optional.empty();
}
User owner = userRepository.findById(key.getOwnerUserId()).orElse(null);
if (owner == null || !owner.isEnabled()) {
return Optional.empty();
}
usageRecorder.record(key.getId());
return Optional.of(
new ApiKeyAuthentication(owner, auditLabel(key), owner.getAuthorities()));
}
// Legacy single per-user key: keep working, always a personal key for its user.
return userRepository
.findByApiKey(rawKey)
.filter(User::isEnabled)
.map(user -> new ApiKeyAuthentication(user, null, user.getAuthorities()));
}
/** "Production ingest (sk_a1b2c3d4)" - shown against API-sourced docs in the processor feed. */
private static String auditLabel(ApiKey key) {
return key.getName() + " (" + key.getPrefix() + ")";
}
/**
* Revoke the {@code api_keys} row that mirrors a given raw key, if any. Called when the legacy
* per-user key is rotated so the migrated shadow row can't keep authenticating the old secret.
*/
@Transactional
public void revokeMigratedKey(String rawKey) {
if (rawKey == null || rawKey.isBlank()) {
return;
}
apiKeyRepository
.findByKeyHash(ApiKeyHasher.hash(rawKey))
.filter(ApiKey::isActive)
.ifPresent(
k -> {
k.setEnabled(false);
k.setRevokedAt(Instant.now());
apiKeyRepository.save(k);
});
}
/**
* A resolved key: the user, an optional processor-feed label, and the authorities to run as.
*/
public record ApiKeyAuthentication(
User user, String auditLabel, Collection<? extends GrantedAuthority> authorities) {}
}
@@ -1,50 +0,0 @@
package stirling.software.proprietary.security.service;
import java.nio.charset.StandardCharsets;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import java.security.SecureRandom;
import java.util.HexFormat;
/** Generates opaque API-key secrets and hashes them for storage/lookup. */
public final class ApiKeyHasher {
/** Human-recognisable prefix so a leaked string is identifiable as a Stirling API key. */
public static final String KEY_PREFIX = "sk_";
/** Chars of the raw key kept for non-secret display (includes the {@code sk_} prefix). */
private static final int DISPLAY_PREFIX_LENGTH = 11;
private static final SecureRandom RANDOM = new SecureRandom();
private ApiKeyHasher() {}
/** A fresh opaque secret: {@code sk_} followed by 40 hex chars of cryptographic randomness. */
public static String generateRawKey() {
byte[] bytes = new byte[20];
RANDOM.nextBytes(bytes);
return KEY_PREFIX + HexFormat.of().formatHex(bytes);
}
/** SHA-256 hex of a raw key; the value stored and looked up, never the raw key. */
public static String hash(String rawKey) {
try {
byte[] digest =
MessageDigest.getInstance("SHA-256")
.digest(rawKey.getBytes(StandardCharsets.UTF_8));
return HexFormat.of().formatHex(digest);
} catch (NoSuchAlgorithmException e) {
throw new IllegalStateException("SHA-256 unavailable", e);
}
}
/** Leading, non-secret fragment shown in listings (e.g. {@code sk_a1b2c3d4}). */
public static String displayPrefix(String rawKey) {
if (rawKey == null) {
return "";
}
return rawKey.length() <= DISPLAY_PREFIX_LENGTH
? rawKey
: rawKey.substring(0, DISPLAY_PREFIX_LENGTH);
}
}
@@ -1,31 +0,0 @@
package stirling.software.proprietary.security.service;
import org.springframework.stereotype.Component;
import org.springframework.transaction.annotation.Propagation;
import org.springframework.transaction.annotation.Transactional;
import lombok.RequiredArgsConstructor;
import stirling.software.proprietary.security.model.ApiKey;
import stirling.software.proprietary.security.repository.ApiKeyRepository;
/**
* Inserts the shadow {@code api_keys} row that mirrors a user's legacy {@code users.apiKey} in its
* OWN ({@code REQUIRES_NEW}) transaction. Kept a separate bean so the write is isolated from the
* caller's listing transaction: when two concurrent first-loads race to insert the same hash, the
* loser's unique-key clash rolls back only this insert instead of poisoning the caller's
* transaction (on Postgres a failed statement aborts the whole transaction). The {@code
* DataIntegrityViolationException} is left to propagate so the caller can treat it as "already
* migrated".
*/
@Component
@RequiredArgsConstructor
class ApiKeyLegacyMigrator {
private final ApiKeyRepository apiKeyRepository;
@Transactional(propagation = Propagation.REQUIRES_NEW)
public void insertMigratedKey(ApiKey key) {
apiKeyRepository.saveAndFlush(key);
}
}
@@ -1,235 +0,0 @@
package stirling.software.proprietary.security.service;
import java.time.Instant;
import java.time.ZoneOffset;
import java.time.format.DateTimeFormatter;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import org.springframework.dao.DataIntegrityViolationException;
import org.springframework.http.HttpStatus;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
import org.springframework.web.server.ResponseStatusException;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import stirling.software.proprietary.model.api.apikey.CreateApiKeyRequest;
import stirling.software.proprietary.model.api.apikey.CreatedApiKeyDto;
import stirling.software.proprietary.model.api.apikey.PortalApiKeyDto;
import stirling.software.proprietary.model.api.apikey.PortalApiKeysResponse;
import stirling.software.proprietary.security.database.repository.UserRepository;
import stirling.software.proprietary.security.model.ApiKey;
import stirling.software.proprietary.security.model.User;
import stirling.software.proprietary.security.repository.ApiKeyDailyUsageRepository;
import stirling.software.proprietary.security.repository.ApiKeyRepository;
/**
* Portal-facing CRUD for named, personal API keys: lists, creates, and revokes the caller's own
* keys. Every key belongs to exactly one user and authenticates as that user; there is no sharing.
*
* <p>Every pre-existing single {@code users.apiKey} is lazily represented as a key owned by that
* user, so historic keys list uniformly.
*/
@Slf4j
@Service
@RequiredArgsConstructor
public class ApiKeyManagementService {
private static final DateTimeFormatter CREATED_FORMAT =
DateTimeFormatter.ofPattern("yyyy-MM-dd").withZone(ZoneOffset.UTC);
private static final DateTimeFormatter LAST_USED_FORMAT =
DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm").withZone(ZoneOffset.UTC);
private static final int MONTH_WINDOW_DAYS = 30;
/** Bounds a key name so it can't bloat storage or the audit/processor feed. */
private static final int MAX_NAME_LENGTH = 100;
/** Caps active keys per user so key creation can't be used to multiply rate-limit budget. */
private static final int MAX_ACTIVE_KEYS_PER_USER = 50;
private final ApiKeyRepository apiKeyRepository;
private final ApiKeyDailyUsageRepository usageRepository;
private final UserRepository userRepository;
private final UserService userService;
private final ApiKeyLegacyMigrator legacyMigrator;
/** All keys the caller owns. */
@Transactional
public PortalApiKeysResponse listVisibleKeys() {
User caller = requireCaller();
migrateLegacyKey(caller);
List<ApiKey> visible =
apiKeyRepository.findByOwnerUserIdOrderByCreatedAtDesc(caller.getId());
// Batch usage for all keys into three queries rather than two-per-key (avoids N+1).
long today = Instant.now().atZone(ZoneOffset.UTC).toLocalDate().toEpochDay();
List<Long> ids = visible.stream().map(ApiKey::getId).toList();
Map<Long, Long> todayById = new HashMap<>();
Map<Long, Long> monthById = new HashMap<>();
Map<Long, Long> totalById = new HashMap<>();
if (!ids.isEmpty()) {
usageRepository
.countForDayByIds(ids, today)
.forEach(r -> todayById.put(r.getApiKeyId(), r.getTotal()));
usageRepository
.sumSinceByIds(ids, today - (MONTH_WINDOW_DAYS - 1))
.forEach(r -> monthById.put(r.getApiKeyId(), r.getTotal()));
usageRepository
.sumSinceByIds(ids, Long.MIN_VALUE)
.forEach(r -> totalById.put(r.getApiKeyId(), r.getTotal()));
}
List<PortalApiKeyDto> keys =
visible.stream()
.map(
k ->
toDto(
k,
zeroIfNull(todayById.get(k.getId())),
zeroIfNull(monthById.get(k.getId())),
zeroIfNull(totalById.get(k.getId()))))
.toList();
return PortalApiKeysResponse.builder().keys(keys).build();
}
private static long zeroIfNull(Long value) {
return value == null ? 0L : value;
}
/** Create a personal key and return its one-time secret. */
@Transactional
public CreatedApiKeyDto createKey(CreateApiKeyRequest request) {
User caller = requireCaller();
String name = request == null ? null : request.name();
if (name == null || name.isBlank()) {
throw new ResponseStatusException(HttpStatus.BAD_REQUEST, "Key name is required");
}
if (name.trim().length() > MAX_NAME_LENGTH) {
throw new ResponseStatusException(
HttpStatus.BAD_REQUEST,
"Key name must be " + MAX_NAME_LENGTH + " characters or fewer");
}
long activeOwned =
apiKeyRepository.findByOwnerUserIdOrderByCreatedAtDesc(caller.getId()).stream()
.filter(ApiKey::isActive)
.count();
if (activeOwned >= MAX_ACTIVE_KEYS_PER_USER) {
throw new ResponseStatusException(
HttpStatus.TOO_MANY_REQUESTS,
"You have reached the maximum of "
+ MAX_ACTIVE_KEYS_PER_USER
+ " active API keys; revoke one before creating another");
}
String rawKey = ApiKeyHasher.generateRawKey();
ApiKey saved =
apiKeyRepository.save(
ApiKey.builder()
.name(name.trim())
.keyHash(ApiKeyHasher.hash(rawKey))
.prefix(ApiKeyHasher.displayPrefix(rawKey))
.ownerUserId(caller.getId())
.enabled(true)
.createdAt(Instant.now())
.build());
return CreatedApiKeyDto.builder().key(toDto(saved, 0L, 0L, 0L)).secret(rawKey).build();
}
/** Soft-revoke a key the caller owns; also clears the legacy column if it is that key. */
@Transactional
public void revokeKey(Long id) {
User caller = requireCaller();
ApiKey key =
apiKeyRepository
.findById(id)
.orElseThrow(
() -> new ResponseStatusException(HttpStatus.NOT_FOUND, "No key"));
if (!key.getOwnerUserId().equals(caller.getId())) {
// Not-found rather than forbidden so a caller can't probe other users' key ids.
throw new ResponseStatusException(HttpStatus.NOT_FOUND, "No key");
}
key.setEnabled(false);
key.setRevokedAt(Instant.now());
apiKeyRepository.save(key);
clearLegacyColumnIfMatches(key);
}
/** Represent a user's pre-existing single key as a row so it lists uniformly. */
private void migrateLegacyKey(User user) {
String legacy = user.getApiKey();
if (legacy == null || legacy.isBlank()) {
return;
}
String hash = ApiKeyHasher.hash(legacy);
if (apiKeyRepository.existsByKeyHash(hash)) {
return;
}
try {
// Insert in its own transaction so a concurrent-insert clash can't poison this
// listing transaction (see ApiKeyLegacyMigrator).
legacyMigrator.insertMigratedKey(
ApiKey.builder()
.name("Default key")
.keyHash(hash)
.prefix(ApiKeyHasher.displayPrefix(legacy))
.ownerUserId(user.getId())
.enabled(true)
.createdAt(Instant.now())
.build());
} catch (DataIntegrityViolationException alreadyMigrated) {
// A concurrent first-load won the race and inserted the same hash; that's fine.
log.debug("Legacy key already migrated concurrently for user {}", user.getId());
}
}
/**
* If a revoked key is the owner's legacy {@code users.apiKey}, null it so it stops resolving.
*/
private void clearLegacyColumnIfMatches(ApiKey key) {
userRepository
.findById(key.getOwnerUserId())
.ifPresent(
owner -> {
String legacy = owner.getApiKey();
if (legacy != null
&& ApiKeyHasher.hash(legacy).equals(key.getKeyHash())) {
owner.setApiKey(null);
userRepository.save(owner);
}
});
}
private PortalApiKeyDto toDto(ApiKey key, long usageToday, long usageMonth, long usageTotal) {
return PortalApiKeyDto.builder()
.id(String.valueOf(key.getId()))
.name(key.getName())
.prefix(key.getPrefix())
.created(
key.getCreatedAt() == null ? "" : CREATED_FORMAT.format(key.getCreatedAt()))
.lastUsed(
key.getLastUsedAt() == null
? "Never"
: LAST_USED_FORMAT.format(key.getLastUsedAt()))
.status(key.isActive() ? "active" : "revoked")
.usageToday(usageToday)
.usageMonth(usageMonth)
.usageTotal(usageTotal)
.build();
}
private User requireCaller() {
String username = userService.getCurrentUsername();
if (username == null || username.isBlank() || "anonymousUser".equalsIgnoreCase(username)) {
throw new ResponseStatusException(HttpStatus.UNAUTHORIZED, "Not authenticated");
}
return userService
.findByUsernameIgnoreCase(username)
.orElseThrow(
() -> new ResponseStatusException(HttpStatus.UNAUTHORIZED, "Unknown user"));
}
}
@@ -1,60 +0,0 @@
package stirling.software.proprietary.security.service;
import java.time.Instant;
import java.time.ZoneOffset;
import org.springframework.scheduling.annotation.Async;
import org.springframework.stereotype.Service;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
/**
* Records per-key usage off the request thread. Kept a separate bean so the {@code @Async} proxy is
* honoured (a self-invocation from the resolver would run inline). Best-effort: never fails a
* request. The actual writes go through {@link ApiKeyUsageWriter} so each step commits in its own
* transaction and a first-write race can't drop a count.
*/
@Slf4j
@Service
@RequiredArgsConstructor
public class ApiKeyUsageRecorder {
private final ApiKeyUsageWriter writer;
/** Bump today's tally for the key and stamp last-used. */
@Async("auditExecutor")
public void record(Long apiKeyId) {
if (apiKeyId == null) {
return;
}
try {
long epochDay = Instant.now().atZone(ZoneOffset.UTC).toLocalDate().toEpochDay();
// First writer of the day inserts the row; everyone else (and the loser of an insert
// race) increments. Separate transactions mean a unique-key clash never rolls back an
// already-counted request.
if (writer.increment(apiKeyId, epochDay) == 0
&& !firstUseInserted(apiKeyId, epochDay)) {
writer.increment(apiKeyId, epochDay);
}
writer.stampLastUsed(apiKeyId);
} catch (Exception e) {
log.debug("Failed to record API key usage for id={}", apiKeyId, e);
}
}
/**
* Whether we inserted the day's first row. A lost insert race can surface either as a {@code
* false} return or - when the failed flush marked the REQUIRES_NEW transaction rollback-only,
* so its commit throws - as an exception; both mean "someone else inserted", so we treat any
* failure as not-inserted and let the caller fall back to an increment rather than dropping the
* count.
*/
private boolean firstUseInserted(Long apiKeyId, long epochDay) {
try {
return writer.tryInsertFirstUse(apiKeyId, epochDay);
} catch (RuntimeException raced) {
return false;
}
}
}
@@ -1,59 +0,0 @@
package stirling.software.proprietary.security.service;
import java.time.Instant;
import org.springframework.dao.DataIntegrityViolationException;
import org.springframework.stereotype.Component;
import org.springframework.transaction.annotation.Propagation;
import org.springframework.transaction.annotation.Transactional;
import lombok.RequiredArgsConstructor;
import stirling.software.proprietary.security.model.ApiKeyDailyUsage;
import stirling.software.proprietary.security.repository.ApiKeyDailyUsageRepository;
import stirling.software.proprietary.security.repository.ApiKeyRepository;
/**
* Per-step transactional writes for {@link ApiKeyUsageRecorder}. Each method runs in its own
* ({@code REQUIRES_NEW}) transaction so a unique-key clash when two requests race to insert the
* day's first row rolls back only that failed insert - never an already-counted request or the
* last-used stamp.
*/
@Component
@RequiredArgsConstructor
class ApiKeyUsageWriter {
private final ApiKeyRepository apiKeyRepository;
private final ApiKeyDailyUsageRepository usageRepository;
/** Bump today's tally if the row already exists; returns rows updated (0 if none yet). */
@Transactional(propagation = Propagation.REQUIRES_NEW)
public int increment(Long apiKeyId, long epochDay) {
return usageRepository.incrementIfPresent(apiKeyId, epochDay);
}
/**
* Insert today's row with a count of 1. Flushes so a concurrent first-write's unique-key clash
* surfaces here (returning false) instead of at commit; the caller then increments instead.
*/
@Transactional(propagation = Propagation.REQUIRES_NEW)
public boolean tryInsertFirstUse(Long apiKeyId, long epochDay) {
try {
usageRepository.saveAndFlush(new ApiKeyDailyUsage(apiKeyId, epochDay, 1));
return true;
} catch (DataIntegrityViolationException raced) {
return false;
}
}
@Transactional(propagation = Propagation.REQUIRES_NEW)
public void stampLastUsed(Long apiKeyId) {
apiKeyRepository
.findById(apiKeyId)
.ifPresent(
key -> {
key.setLastUsedAt(Instant.now());
apiKeyRepository.save(key);
});
}
}

Some files were not shown because too many files have changed in this diff Show More