Compare commits

...
Author SHA1 Message Date
Anthony StirlingandGitHub bf77d67c00 Merge branch 'main' into desktopInstallButton 2026-05-28 15:57:43 +01:00
Anthony StirlingandGitHub c80a5db5f5 folder and file fixes (#6461) 2026-05-28 15:57:35 +01:00
Anthony StirlingandGitHub 4fa67afc3d Fix Tauri artifact copy path so installers upload (smoke + release) (#6466)
## Summary
Regression from #6404 (Restructure/frontend editor). Two CI workflows
copy the built installers to the wrong directory, so installer artifacts
(MSI / DMG / DEB / RPM / AppImage) silently vanish:

- **`tauri-build.yml`** (PR/desktop smoke builds) - uploads zero
installer artifacts.
- **`multiOSReleases.yml`** (production releases) - the empty artifacts
are downloaded by `create-release` and fed to `action-gh-release`, so a
release would publish **only the JARs, no desktop installers**.

## Root cause
#6404 moved the Tauri project from `frontend/` to `frontend/editor/` and
updated every **absolute** path (`projectPath`, `cd`, `Get-ChildItem`)
to add the `editor/` segment - but left the **relative** copy targets
`../../../dist`. Those resolve against the (now one level deeper)
working dir after `cd ./frontend/editor/src-tauri/target`:

| | resolves to |
|---|---|
| before #6404 (`frontend/src-tauri/target`) | repo-root `dist/` ✅ |
| after #6404 (`frontend/editor/src-tauri/target`) | `frontend/dist/` ❌
(missing) |

The `cp` fails, repo-root `dist/` (from `mkdir -p ./dist`) stays empty,
and the upload finds nothing. `find -exec cp` failing is non-fatal, so
jobs still report success - that's why it went unnoticed. No release has
shipped broken yet: the last release (v2.11.0, 2026-05-19) predates
#6404 (2026-05-22).

## Fix
Copy to an absolute `$GITHUB_WORKSPACE/dist` in both workflows so the
`cd` can't drift the destination again. This matches where the upload /
signature-verify steps already read from.

## Evidence (run 26574078559, all 3 OS legs)
```
cp: cannot create regular file '../../../dist/Stirling-PDF-windows-x86_64.msi': No such file or directory
##[warning]No files were found with the provided path: ./dist/*. No artifacts will be uploaded.
```
The Tauri builds themselves succeeded - only the copy/upload was broken.

## Test plan
- [ ] `tauri-build` on this PR uploads non-empty `Stirling-PDF-<name>`
artifacts on Windows/macOS/Linux.
- [ ] Next release (or a `workflow_dispatch` of multiOSReleases)
attaches MSI/DMG/DEB/RPM/AppImage to the release.
2026-05-28 15:57:01 +01:00
8bd78d2624 Add landscape page size options (#6248)
# Description of Changes

Adds orientation (portrait/landscape) to the Adjust Page Scale tool.

- Orientation as a separate parameter (per review), sent through to the
backend
- ScalePagesController simplified; PDFWithPageSize gains the orientation
field
- Regenerated tool_models.py; frontend + backend tests added

---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.

---------

Co-authored-by: ConnorYoh <40631091+ConnorYoh@users.noreply.github.com>
2026-05-28 14:16:15 +00:00
Anthony StirlingandGitHub b3c4b8b463 Add S3 storage and cluster artifact backend (#6457) 2026-05-28 13:06:27 +01:00
James BruntonandGitHub 57af5b9dc2 Fix Tauri testing (#6462)
# Description of Changes
#6402 introduced a Rust test `refresh_token_fallback.rs`, but it wasn't
moved properly after the restructure of the `frontend/` folder in #6404.
This PR moves the file to the right place, and also hooks up Task and CI
rules for `cargo test` since nothing was actually running the test in
the first place.
2026-05-28 11:05:56 +00:00
James BruntonandGitHub 44fbf8c587 Various bug fixes found while testing SaaS build (#6459)
# Description of Changes
Various fixes and improvements I made while testing the SaaS code:
- Changes the new `.env.saas` file to live in `app/` and match the
semantics of the other `.env` files
- Adds top-level `task dev:saas` command to spawn SaaS frontend &
backend
- Deletes dead SaaS code and improves some overriding logic
- Fixes refreshing issue when coming back to the tab
- Fix the Compare tool's selection logic
- Make Compare handle error cases properly
- Fixes the location of the "Dismiss All Errors" button (was rendering
on top of the top-bar with a transparent background previously so it
looked rubbish)
- Fixes file selection in PDF Editor
2026-05-28 11:05:30 +00:00
Anthony StirlingandGitHub 76840d8a57 Add CI DB migration smoke test against v2.0/v2.5/v2.10 updates (#6453) 2026-05-28 11:36:07 +01:00
James BruntonandGitHub d459ded168 Add cancel button to kill long-running AI tasks (#6351)
# Description of Changes
Adds a cancel button to the AI chat to allow the user to abort
long-running AI tasks. Just disconnects the SSE stream (all the backend
code already interrupts when it notices the stream is dead).
2026-05-28 09:25:23 +00:00
ConnorYohandGitHub 43b67d213d feat(oauth2): opt-in claim-dump diagnostics for OIDC login failures (#6456)
# Description of Changes

## What & why

Customers using ADFS (or any generic OIDC provider that doesn't emit
`email`) hit `Attribute value for 'email' cannot be null` during OAuth2
login with no visibility into what claims the provider actually sent.
The only available remedy was guessing at
`security.oauth2.useAsUsername` until something worked.

This PR adds a new opt-in `security.oauth2.debugLogging` flag (default
`false`). When enabled, `CustomOAuth2UserService` logs:

- All ID token claims (sorted, with values)
- All UserInfo endpoint claims (if any)
- The merged attribute key set Spring exposes to `getAttribute()`
- The value the configured `useAsUsername` actually resolved to
- A **`Hint:`** line listing the claim keys present in the token that
map to a valid `UsernameAttribute` enum value — i.e. exactly what the
operator could put in `useAsUsername` to make login work

Logged at `INFO` on the success path and `ERROR` on failure (inside the
existing `catch (IllegalArgumentException)` block that throws
`OAuth2AuthenticationException`). The block is wrapped with a `[OAUTH2
DEBUG] ... [/OAUTH2 DEBUG]` banner and ends with a PII warning so
operators don't leave it on in production.

Default off → zero observable change for anyone not actively
troubleshooting.

## Files changed

| File | Why |
|---|---|
| `app/common/.../ApplicationProperties.java` | New `debugLogging` field
on the `OAUTH2` config class with javadoc warning about PII |
| `app/core/src/main/resources/settings.yml.template` | Documents
`oauth2.debugLogging` so it appears on next startup |
| `app/proprietary/.../security/service/CustomOAuth2UserService.java` |
Emits the claim dump + suggestion hint when the flag is on |
|
`app/proprietary/.../security/service/CustomOAuth2UserServiceDebugLoggingTest.java`
(new) | Unit test: mocks the OIDC delegate, asserts off-path is silent
and on-path emits the dump with the right Hint contents |

## End-to-end verification

Ran the bundled `testing/compose/docker-compose-keycloak-oauth.yml`
Keycloak realm, configured `security.oauth2.useAsUsername: mail`
(Keycloak emits `email`, not `mail`) and `provider: demarest` (matches
the original customer bug report). Triggered the OAuth flow at
`http://localhost:8080/oauth2/authorization/demarest` and confirmed:

- The ERROR-level dump fires with the full 19-claim ID token decoded
- `-- Value at 'mail' : <NULL — this is why login fails>` correctly
identifies the missing claim
- `-- Hint:` correctly suggests `[email, family_name, given_name,
preferred_username]` (the four keys present that map to valid
`UsernameAttribute` values)
- Auth still fails with the original `OAuth2AuthenticationException` —
no change to control flow, just added diagnostic logging

Unit test (`CustomOAuth2UserServiceDebugLoggingTest`) covers both
branches.

## Reviewer notes

- **No new public APIs.** The flag is config-only; no servlet endpoints
exposed.
- **PII is logged when the flag is on.** This is the whole point —
operators need to see the claims to fix their config — but it's gated,
defaults off, and the dump self-documents with a `WARNING: ... Set
security.oauth2.debugLogging=false once troubleshooting is complete.`
footer.
- **Why log everything, not just sub/email?** Because the operator
doesn't know in advance which claim they actually want. ADFS uses `upn`
in some configs and `preferred_username` in others; Azure AD uses `oid`;
the customer here had neither. Dumping the full set is the only way to
make the diagnostic self-service.
- **Out of scope for this PR (follow-ups):**
- The `UsernameAttribute` enum doesn't include `upn` / `unique_name`
(common ADFS claims). If the customer's token only has `upn`, the Hint
will be empty even though the operator can see `upn` in the dump. Worth
a separate PR to extend the enum.
- The known-provider validator in `Provider.java` (rejects e.g.
`useAsUsername: mail` for `provider: keycloak` at startup) bypasses our
diagnostic for those provider names. ADFS customers using `provider:
<name>` fall into the `default` branch so are not affected — but it's a
sharp edge worth documenting.

---

## Checklist

### General

- [x] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [x] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable) — N/A, backend-only change
- [x] I have performed a self-review of my own code
- [x] My changes generate no new warnings

### Documentation

- [ ] Doc-repo update (if functionality has heavily changed) —
diagnostic flag is self-documenting via the `settings.yml.template`
comment and the in-log warning; happy to add a doc-repo entry if
reviewers want one
- [ ] Translation tags — N/A

### UI Changes (if applicable)

- [ ] N/A — backend-only

### Testing (if applicable)

- [x] Unit test added (`CustomOAuth2UserServiceDebugLoggingTest`)
covering on/off paths and Hint correctness
- [x] End-to-end verified locally against bundled Keycloak compose with
intentionally misconfigured `useAsUsername`
- [x] Full `:proprietary:test` suite passes
2026-05-27 13:01:51 +00:00
Anthony StirlingandGitHub d42b779644 Add server-side folders and files page UI (#6383) 2026-05-27 12:52:46 +01:00
Anthony StirlingandGitHub 4564ed5bec Add cluster backplane abstraction and interfaces (#6449) 2026-05-27 11:54:59 +01:00
7743720f0a chore: fix "Endoints" typo in health route comment (#6446)
`app/common/src/main/java/stirling/software/common/util/RequestUriUtils.java:175`
section comment said "Health Endoints" → "Health Endpoints".
Comment-only.

Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
2026-05-26 23:30:57 +01:00
Saul IfshinandGitHub 620f246ca4 test(frontend): add URL-encoded filename parsing case (#6436)
# Description of Changes

<!--
Please provide a summary of the changes, including:

- What was changed
- Why the change was made
- Any challenges encountered

Closes #(issue_number)
-->

---

## Checklist

### General

- [x] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [x] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [x] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [x] I have performed a self-review of my own code
- [x] My changes generate no new warnings

### Documentation

- [x] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [x] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [x] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [x] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [x] I have run `task check` to verify linters, typechecks, and tests
pass
- [x] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
2026-05-26 23:29:32 +01:00
Saul IfshinandGitHub dd24440b73 test(frontend): cover MIME precedence in non-PDF type detection (#6438)
# Description of Changes

<!--
Please provide a summary of the changes, including:

- What was changed
- Why the change was made
- Any challenges encountered

Closes #(issue_number)
-->

---

## Checklist

### General

- [x] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [x] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [x] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [x] I have performed a self-review of my own code
- [x] My changes generate no new warnings

### Documentation

- [x] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [x] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [x] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [x] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [x] I have run `task check` to verify linters, typechecks, and tests
pass
- [x] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
2026-05-26 23:28:17 +01:00
LudyandGitHub c0374266e7 ci: harden workflow runners and update checkout action pins (#6445)
# Description of Changes

Add runner hardening and housekeeping across workflows.

- .github/dependabot.yml: add /docker/unoserver and /engine to
Dependabot update paths.
- .github/workflows/_runner-pick.yml and .github/workflows/build.yml:
add step-security/harden-runner (egress-policy: audit) to audit outbound
calls from runners.
- .github/workflows/ai-engine.yml, .github/workflows/aur-publish.yml,
.github/workflows/package-managers.yml: update actions/checkout usage to
the newer v6.0.2 reference.
- .github/workflows/package-managers.yml: minor YAML formatting tidy
(release types array).

These changes improve CI security by auditing egress, update the
checkout action to a newer release, and expand Dependabot coverage.

---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
2026-05-26 23:26:20 +01:00
Anthony StirlingandGitHub 730ed43af9 Fix login loop (#6402) 2026-05-26 20:05:25 +01:00
Anthony Stirling 930dc5c018 Use universal Mac installer + centralized installer URLs 2026-05-26 19:44:10 +01:00
Anthony StirlingandGitHub c2cc7ede10 Merge branch 'main' into desktopInstallButton 2026-05-26 17:50:56 +01:00
Anthony StirlingandGitHub 56ff1e5092 impl migration to pdfium for split (#6410)
## Summary

Migrates `SplitPDFController`, `SplitPdfByChaptersController`,
`SplitPdfBySizeController` from PDFBox to JPDFium.
`SplitPdfBySectionsController` and `AutoSplitPdfController` are
intentionally left on PDFBox (require JPDFium 1.0.2 features that don't
exist yet).

## Benchmark (audited on `audit/jpdfium-split`, file
`app/core/src/test/java/stirling/software/SPDF/bench/SplitBenchmark.java`)

| Workload | PDFBox heap | JPDFium heap | PDFBox wall | JPDFium wall |
|---|---|---|---|---|
| 100 pp, chunk 10 | +21-26 MB | **+0.02 MB** | 80-106 ms | **25 ms** |
| 300 pp, chunk 10 | +59 MB | **+1.0 MB** | 232 ms | **76 ms** |

**98-99.9% heap reduction. 3-4.2x faster wall.**

## Hybrid

- AcroForm-bearing splits keep PDFBox
`FormUtils.pruneOrphanedFormFields` post-pass (FPDF_ImportPagesByIndex
drops AcroForm dict). Sub-bench shows +1.0 MB / +27 ms - tightly
bounded.
- Metadata extraction stays on PDFBox.
- `SplitPdfBySectionsController` - JPDFium `PdfPageSplitter` only does
2-up halving, not arbitrary MxN.
- `AutoSplitPdfController` - needs PDFRenderer + zxing for QR markers.

## Test plan

- [ ] 30/30 unit tests pass with PDFBox `Loader.loadPDF` as the oracle
(assert page counts + document totals)
- [ ] Existing cucumber feature `split.feature` continues to pass
- [ ] AcroForm-bearing PDF round-trips without orphaned widgets (covered
by existing FormUtils tests)
2026-05-26 17:50:13 +01:00
ConnorYohandGitHub 05b80fbe4f Working local Saas (#6450)
Env file for setting backend saas and taskfile for running it
2026-05-26 14:40:44 +00:00
Anthony Stirling a95db1aa8b Restore download URL fallback for non-desktop installs 2026-05-26 15:14:29 +01:00
Anthony Stirling 45da220060 desktop install button 2026-05-26 15:13:59 +01:00
Reece BrowneandGitHub 4047d02086 chore: address restructure PR review feedback (#6423) 2026-05-26 14:12:05 +01:00
LudyandGitHub 5f78083470 Fix unresolved Material Symbols icon names in frontend (#6443) 2026-05-25 17:02:59 +01:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>Anthony Stirling
fc048872d7 build(deps): bump idna from 3.12 to 3.15 in /testing/cucumber in the pip group across 1 directory (#6403)
Bumps the pip group with 1 update in the /testing/cucumber directory:
[idna](https://github.com/kjd/idna).

Updates `idna` from 3.12 to 3.15
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/kjd/idna/blob/master/HISTORY.md">idna's
changelog</a>.</em></p>
<blockquote>
<h2>3.15 (2026-05-12)</h2>
<ul>
<li>Enforce DNS-length cap on individual labels early in
<code>check_label</code>,
short-circuiting contextual-rule processing for oversized input
while staying compatible with UTS 46 usage.</li>
<li>Tidy core helpers: hoist bidi category sets to module-level
frozensets (avoiding per-codepoint list construction), simplify
length checks, and reuse the shared <code>_unicode_dots_re</code> from
<code>idna.core</code> in the codec module.</li>
<li>Use <code>raise ... from err</code> for proper exception chaining
and
switch internal string formatting to f-strings.</li>
<li>Allow <code>flit_core</code> 4.x in the build backend.</li>
<li>Expand the ruff lint set (flake8-bugbear, flake8-simplify,
pyupgrade, perflint) and apply the surfaced fixes; pin lint CI
to Python 3.14.</li>
<li>Add Dependabot configuration for GitHub Actions.</li>
<li>Convert README and HISTORY from reStructuredText to Markdown.</li>
<li>Reference CVE-2026-45409 for the 3.14 advisory in place of the
initial GHSA identifier.</li>
</ul>
<p>Thanks to Felix Yan, Stan Ulbrych, and metsw24-max for
contributions to this release.</p>
<h2>3.14 (2026-05-10)</h2>
<ul>
<li>Removed opportunity to process long inputs into quadratic
time by rejecting oversize inputs up-front. Closes a bypass
of the CVE-2024-3651 mitigation. [CVE-2026-45409]</li>
</ul>
<p>Thanks to Stan Ulbrych for reporting the issue.</p>
<h2>3.13 (2026-04-22)</h2>
<ul>
<li>Correct classification error for codepoint U+A7F1</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/kjd/idna/commit/af30a092e158181d0b35ac66dfa813788126bdd8"><code>af30a09</code></a>
Release 3.15</li>
<li><a
href="https://github.com/kjd/idna/commit/30314d4628744ca14cf2b5820564e5127a9f86f2"><code>30314d4</code></a>
Pre-release 3.15rc0</li>
<li><a
href="https://github.com/kjd/idna/commit/05d4b219aa9eddc47371fcbd2000f0301016f3e9"><code>05d4b21</code></a>
Merge pull request <a
href="https://redirect.github.com/kjd/idna/issues/237">#237</a> from
kjd/convert-docs-to-markdown</li>
<li><a
href="https://github.com/kjd/idna/commit/2987fdba1962bbb2358399e0084ba062b98a0bee"><code>2987fdb</code></a>
Convert README and HISTORY from reStructuredText to Markdown</li>
<li><a
href="https://github.com/kjd/idna/commit/59fa8002d514bf4a5ce7b58f67b9ec587d53fa9c"><code>59fa800</code></a>
Merge pull request <a
href="https://redirect.github.com/kjd/idna/issues/236">#236</a> from
kjd/dependabot/github_actions/actions-f3e34333ea</li>
<li><a
href="https://github.com/kjd/idna/commit/def69834ced5d4b3c50439d8b99c4c856ec19ca2"><code>def6983</code></a>
Merge branch 'master' into
dependabot/github_actions/actions-f3e34333ea</li>
<li><a
href="https://github.com/kjd/idna/commit/bbd8004a797185d8c56bb555cd5c88fde05e0631"><code>bbd8004</code></a>
Merge pull request <a
href="https://redirect.github.com/kjd/idna/issues/234">#234</a> from
StanFromIreland/patch-1</li>
<li><a
href="https://github.com/kjd/idna/commit/edd07c05024344a6ccb517414ccb36683aee99fc"><code>edd07c0</code></a>
Bump github/codeql-action from 3.35.2 to 4.35.2 in the actions
group</li>
<li><a
href="https://github.com/kjd/idna/commit/5557db030c11bdec50d62aa5f631d705d33ba123"><code>5557db0</code></a>
Merge branch 'master' into patch-1</li>
<li><a
href="https://github.com/kjd/idna/commit/f11746cf4981d25123ef7830d3ee60f07de8ae3d"><code>f11746c</code></a>
Merge pull request <a
href="https://redirect.github.com/kjd/idna/issues/235">#235</a> from
StanFromIreland/patch-2</li>
<li>Additional commits viewable in <a
href="https://github.com/kjd/idna/compare/v3.12...v3.15">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=idna&package-manager=pip&previous-version=3.12&new-version=3.15)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore <dependency name> major version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's major version (unless you unignore this specific
dependency's major version or upgrade to it yourself)
- `@dependabot ignore <dependency name> minor version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's minor version (unless you unignore this specific
dependency's minor version or upgrade to it yourself)
- `@dependabot ignore <dependency name>` will close this group update PR
and stop Dependabot creating any more for the specific dependency
(unless you unignore this specific dependency or upgrade to it yourself)
- `@dependabot unignore <dependency name>` will remove all of the ignore
conditions of the specified dependency
- `@dependabot unignore <dependency name> <ignore condition>` will
remove the ignore condition of the specified dependency and ignore
conditions
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/Stirling-Tools/Stirling-PDF/network/alerts).

</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Anthony Stirling <77850077+Frooodle@users.noreply.github.com>
2026-05-24 09:51:19 +00:00
dependabot[bot]andGitHub 6ee533c427 build(deps): bump ws from 8.20.0 to 8.20.1 in /frontend (#6400)
Signed-off-by: dependabot[bot] <support@github.com>
2026-05-22 22:47:49 +01:00
dependabot[bot]andGitHub a13683e84a build(deps): bump protobufjs from 7.5.6 to 7.6.0 in /frontend (#6401)
Signed-off-by: dependabot[bot] <support@github.com>
2026-05-22 22:44:02 +01:00
914b474d84 build(deps-dev): bump ip-address from 10.1.0 to 10.2.0 in /frontend (#6319)
Co-authored-by: Anthony Stirling <77850077+Frooodle@users.noreply.github.com>
Signed-off-by: dependabot[bot] <support@github.com>
2026-05-22 22:43:11 +01:00
dependabot[bot]andGitHub 51f69a718f build(deps): bump svelte from 5.55.0 to 5.55.9 in /frontend (#6409)
Signed-off-by: dependabot[bot] <support@github.com>
2026-05-22 22:42:51 +01:00
dependabot[bot]andGitHub 7f597051e4 build(deps): bump reportlab from 4.5.0 to 4.5.1 in /testing/cucumber (#6414)
Signed-off-by: dependabot[bot] <support@github.com>
2026-05-22 22:42:26 +01:00
dependabot[bot]andGitHub e113df6252 build(deps): bump requests from 2.33.1 to 2.34.2 in /testing/cucumber (#6416)
Signed-off-by: dependabot[bot] <support@github.com>
2026-05-22 22:42:11 +01:00
dependabot[bot]andGitHub 06b4a4184b build(deps): bump com.diffplug.spotless from 8.4.0 to 8.5.0 (#6417)
Signed-off-by: dependabot[bot] <support@github.com>
2026-05-22 22:41:52 +01:00
dependabot[bot]andGitHub 19982d7207 build(deps): bump actions/ai-inference from 2.0.7 to 2.1.0 (#6415)
Signed-off-by: dependabot[bot] <support@github.com>
2026-05-22 22:30:44 +01:00
dependabot[bot]andGitHub cdfefb02bc build(deps): bump github/codeql-action from 4.35.4 to 4.35.5 (#6421)
Signed-off-by: dependabot[bot] <support@github.com>
2026-05-22 22:30:28 +01:00
dependabot[bot]andGitHub 5ba6a65c1a build(deps): bump step-security/harden-runner from 2.19.1 to 2.19.3 (#6422)
Signed-off-by: dependabot[bot] <support@github.com>
2026-05-22 22:30:02 +01:00
ConnorYohandGitHub 017c8d59fa feat(ai): add Contradiction Agent on a new ChunkedMapper primitive (#6369)
## Summary

Adds a new AI specialist that finds **textual contradictions** across
one or more PDFs — conflicting claims, recommendations, points of view,
contested facts — built entirely in Python on top of the new
`DocumentService` + `ChunkedReasoner` stack from #6314. Replaces the
closed #6304, which was started before #6314 landed and therefore
over-engineered (Java orchestrator, two-round handshake, resume
artifact, discriminated-union lift).

Two commits:
1. **`refactor(engine): extract ChunkedMapper[T] from ChunkedReasoner`**
— pure refactor, public API of ChunkedReasoner unchanged. New
`ChunkedMapper[T: BaseModel]` is a generic parallel-chunk primitive
(slicing, semaphore, time-bounded extraction, cancellation drain,
progress events) that's now a peer to ChunkedReasoner rather than locked
inside it. The compression loop stays on ChunkedReasoner where it
belongs.
2. **`feat(ai): add Contradiction Agent on ChunkedMapper`** — the agent
itself, plus integrations into `PdfReviewAgent` and `PdfQuestionAgent`.

## Architecture

- **Python-only.** No Java code. No `AgentToolId.CONTRADICTION_AGENT`.
No dedicated HTTP endpoint. No resume artifact, no discriminated-union
lift in `contracts/common.py`. Detector runs inside the Python engine
and the Python engine alone.
- **Review path** (`PdfReviewAgent`): a new
`ContradictionIntentClassifier` fires on contradiction-flavoured
prompts; agent runs detection synchronously and emits a single
`EditPlanResponse(steps=[ADD_COMMENTS])`. Single-turn flow — no resume.
- **Question path** (`PdfQuestionAgent`): a new
`ContradictionCapability` joins `RagCapability` and
`WholeDocReaderCapability` in the smart-model toolset, exposing
`find_contradictions(query)`. The smart model picks it from the toolset
alongside `search_knowledge` and `read_full_document`.

## Inside `ContradictionDetector.detect()`

1. `DocumentService.read_pages(file_id)` → ordered `list[Page]`.
2. `ChunkedMapper[_ExtractedClaims].map_pages(...)` — char-budgeted
multi-page slicing; each slice runs the claim-extractor LLM in parallel
under a semaphore.
3. Page-traceability: the extractor returns `_ExtractedClaim.page`
(which `[Page N]` marker the claim came from). The wrapper validates
`page ∈ chunk.pages`; if not, mechanical fallback searches the chunk's
page text for the verbatim quote and reassigns. If still no match, drop
the claim.
4. `Claim.anchor_quality: Literal[\"verbatim\", \"paraphrased\"]` is set
by a substring check against the declared page's text. Verbatim quotes
feed `anchor_text` for snap-to-quote add-comments placement; paraphrased
ones fall back to margin geometry.
5. Subject canonicalisation: ONE fast-model LLM call collapses synonyms
across the document. Fails open to lexical bucketing.
6. Pre-filters: drop identical-quote pairs; drop same-page same-polarity
paraphrases.
7. Per-bucket pair detection in parallel (separate semaphore, cap 5).
Buckets > 12 claims chunk into windows of 12 with overlap 2; pairs
deduped across overlapping windows by frozen `(i, j)` index pair.
8. Summary fast-model call with fallback string on error.

## Prompt-injection hardening

Every prompt that interpolates user-supplied or PDF-extracted text wraps
content in `<user_message>` / `<verdict>` / `<content>` tags with an
explicit SECURITY preamble instructing the model to treat tagged content
as data only.

## Limitations

- **Combined math + contradiction intent**: when both intent classifiers
fire on the same prompt, contradiction takes precedence and the math
intent is silently dropped. Documented in the Review module docstring
and pinned by
`test_review_integration.py::test_contradiction_precedence_over_math`.
- **Cross-window contradiction reach**: within a subject bucket, pairs
more than ~10 claim indices apart in the same chunked window may be
missed by the overlap-2 strategy. Documented in `test_detector.py`.
Acceptable for v1.

## Settings (engine/src/stirling/config/settings.py)

```python
contradiction_detect_concurrency = 5     # per-bucket detector semaphore
contradiction_bucket_chunk_size = 12     # max claims per detector call
contradiction_bucket_chunk_overlap = 2   # overlap for >threshold buckets
```

`chars_per_slice` and extraction concurrency are reused from the
existing `chunked_reasoner_*` settings.

## Test plan

- [x] `uv run pytest tests/ -v` — **245/245 pass** (210 pre-existing +
35 new)
- [x] `uv run ruff check src/ tests/` — clean
- [x] `uv run pyright src/stirling/agents/contradiction/
src/stirling/contracts/contradiction.py` — 0 errors
- [x] `./gradlew :proprietary:test` — green; no Java was touched, but
verified untouched
- [x] Page-traceability tests cover: valid page kept, hallucinated page
dropped, mechanical-reassign on misattribution, anchor-quality verbatim
vs paraphrased
- [x] Review integration: ADD_COMMENTS plan with two paired CommentSpecs
per contradiction; NeedIngestResponse precheck; precedence vs math
intent pinned
- [x] Question integration: all three capabilities wired into
smart-model toolset; `find_contradictions` returns formatted report text
- [x] ChunkedMapper standalone: slicing, multi-chunk ordering, worker
failures, timeouts, cancellation drain, semaphore saturation
- [x] ChunkedReasoner regression: all pre-existing tests pass unchanged
after the internal split

## Relationship to closed #6304

#6304 was closed in favour of this PR. The closed PR predated #6314 and
modelled the agent as a Java-orchestrated two-round examine/deliberate
flow with its own HTTP endpoint and a discriminated-union resume
artifact. With #6314 making full ordered page text available to the
engine via `DocumentService.read_pages`, none of that is needed. Net
effect: drop ~600 lines of Java, drop the two-round handshake, drop the
`ToolReportArtifact` lift, while ending up with a more scalable agent
(chunk-based instead of page-based extraction; tested to
ChunkedReasoner-equivalent scale).
2026-05-22 13:23:52 +00:00
301 changed files with 27578 additions and 2746 deletions
+2 -2
View File
@@ -34,8 +34,8 @@ frontend/playwright-report/
# Tauri/desktop builds
src-tauri/target/
src-tauri/dist/
frontend/src-tauri/target/
frontend/src-tauri/dist/
frontend/editor/src-tauri/target/
frontend/editor/src-tauri/dist/
# IDE and editor
.idea/
+2
View File
@@ -38,6 +38,8 @@ project: &project
- frontend/**
- docker/**
- scripts/RestartHelper.java
- scripts/db-migration/**
- .github/workflows/db-migration-test.yml
frontend: &frontend
- frontend/**
+2
View File
@@ -25,6 +25,8 @@ updates:
- "/docker/frontend"
- "/docker/base"
- "/docker/engine"
- "/docker/unoserver"
- "/engine"
schedule:
interval: "weekly"
cooldown:
+2 -2
View File
@@ -69,8 +69,8 @@ labels:
- label: 'Tauri'
files:
- 'frontend/src-tauri/**'
- 'frontend/src-tauri/.*'
- 'frontend/editor/src-tauri/**'
- 'frontend/editor/src-tauri/.*'
- label: 'engine'
files:
+2 -2
View File
@@ -13,7 +13,7 @@ Usage:
"""
# Sample for Windows:
# python .github/scripts/check_language_toml.py --reference-file frontend/public/locales/en-GB/translation.toml --branch "" --files frontend/public/locales/de-DE/translation.toml frontend/public/locales/fr-FR/translation.toml
# python .github/scripts/check_language_toml.py --reference-file frontend/editor/public/locales/en-GB/translation.toml --branch "" --files frontend/editor/public/locales/de-DE/translation.toml frontend/editor/public/locales/fr-FR/translation.toml
import argparse
import glob
@@ -308,7 +308,7 @@ def check_for_differences(reference_file, file_list, branch, actor):
report.append("## ❌ Overall Check Status: **_Failed_**")
report.append("")
report.append(
f"@{actor} please check your translation if it conforms to the standard. Follow the format of [en-GB/translation.toml](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/frontend/public/locales/en-GB/translation.toml)"
f"@{actor} please check your translation if it conforms to the standard. Follow the format of [en-GB/translation.toml](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/frontend/editor/public/locales/en-GB/translation.toml)"
)
else:
report.append("## ✅ Overall Check Status: **_Success_**")
+5 -4
View File
@@ -39,7 +39,7 @@ jobs:
pr_ref: ${{ steps.resolve.outputs.ref }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -119,7 +119,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -287,6 +287,7 @@ jobs:
- /stirling/V2-PR-${{ needs.check-pr.outputs.pr_number }}/data:/usr/share/tessdata:rw
- /stirling/V2-PR-${{ needs.check-pr.outputs.pr_number }}/config:/configs:rw
- /stirling/V2-PR-${{ needs.check-pr.outputs.pr_number }}/logs:/logs:rw
- /stirling/V2-PR-${{ needs.check-pr.outputs.pr_number }}/storage:/storage:rw
environment:
DISABLE_ADDITIONAL_FEATURES: "false"
SECURITY_ENABLELOGIN: "true"
@@ -309,7 +310,7 @@ jobs:
ssh -i ../private.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -T ${{ secrets.NEW_VPS_USERNAME }}@${{ secrets.NEW_VPS_HOST }} << ENDSSH
# Create V2 PR-specific directories
mkdir -p /stirling/V2-PR-${{ needs.check-pr.outputs.pr_number }}/{data,config,logs}
mkdir -p /stirling/V2-PR-${{ needs.check-pr.outputs.pr_number }}/{data,config,logs,storage}
# Move docker-compose file to correct location
mv /tmp/docker-compose-v2.yml /stirling/V2-PR-${{ needs.check-pr.outputs.pr_number }}/docker-compose.yml
@@ -385,7 +386,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -74,7 +74,7 @@ jobs:
enable_prototypes: ${{ steps.check-prototypes-flag.outputs.enable_prototypes }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -191,7 +191,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -514,7 +514,7 @@ jobs:
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
+1 -1
View File
@@ -21,7 +21,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
+5
View File
@@ -37,6 +37,11 @@ jobs:
outputs:
is_fork: ${{ steps.decide.outputs.is_fork }}
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
- name: Classify the trigger
id: decide
env:
+2 -2
View File
@@ -21,12 +21,12 @@ jobs:
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
- name: Checkout code
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Install uv
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
+2 -2
View File
@@ -19,7 +19,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -87,7 +87,7 @@ jobs:
- name: AI PR Title Analysis
if: steps.actor.outputs.is_repo_dev == 'true'
id: ai-title-analysis
uses: actions/ai-inference@e09e65981758de8b2fdab13c2bfb7c7d5493b0b6 # v2.0.7
uses: actions/ai-inference@17ff458cb182449bbb2e43701fcd98f6af8f6570 # v2.1.0
with:
model: openai/gpt-4o
system-prompt-file: ".github/config/system-prompt.txt"
+3 -3
View File
@@ -26,7 +26,7 @@ jobs:
jar_sha256: ${{ steps.hashes.outputs.jar_sha256 }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -70,12 +70,12 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
- name: Checkout repository (for PKGBUILD templates)
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Update stirling-pdf-desktop PKGBUILD
env:
+1 -1
View File
@@ -16,7 +16,7 @@ jobs:
pull-requests: write
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
+1 -1
View File
@@ -28,7 +28,7 @@ jobs:
spring-security: [true, false]
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
- name: Checkout repository
+1 -1
View File
@@ -61,7 +61,7 @@ jobs:
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
- name: Checkout repository
+20 -1
View File
@@ -46,7 +46,7 @@ jobs:
proprietary: ${{ steps.changes.outputs.proprietary }}
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
- name: Checkout repository
@@ -68,6 +68,18 @@ jobs:
uses: ./.github/workflows/backend-build.yml
secrets: inherit
db-migration-test:
# Boots the current bootJar against H2 fixtures captured from past
# releases (v2.0.0 / v2.5.0 / v2.10.0) and verifies admin login still
# works after Hibernate's ddl-auto=update migrates the schema. Gated on
# the `project` filter so doc-only PRs skip this ~5-minute job.
if: needs.files-changed.outputs.project == 'true'
needs: [files-changed]
permissions:
contents: read
uses: ./.github/workflows/db-migration-test.yml
secrets: inherit
check-generateOpenApiDocs:
if: needs.files-changed.outputs.openapi == 'true'
needs: [files-changed]
@@ -184,6 +196,7 @@ jobs:
needs:
- files-changed
- build
- db-migration-test
- check-generateOpenApiDocs
- frontend-validation
- playwright-e2e
@@ -198,11 +211,17 @@ jobs:
- dependency-review
runs-on: ubuntu-latest
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
- name: Verify every required job passed (or was legitimately skipped)
env:
RESULTS: |
files-changed=${{ needs.files-changed.result }}
build=${{ needs.build.result }}
db-migration-test=${{ needs.db-migration-test.result }}
check-generateOpenApiDocs=${{ needs.check-generateOpenApiDocs.result }}
frontend-validation=${{ needs.frontend-validation.result }}
playwright-e2e=${{ needs.playwright-e2e.result }}
+1 -1
View File
@@ -15,7 +15,7 @@ jobs:
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
+1 -1
View File
@@ -20,7 +20,7 @@ jobs:
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
+1 -1
View File
@@ -27,7 +27,7 @@ jobs:
pull-requests: write # Allow writing to pull requests
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
+93
View File
@@ -0,0 +1,93 @@
name: DB migration smoke test
# Boots the current Stirling-PDF JAR against H2 fixtures captured from past
# releases (v2.0.0 / v2.5.0 / v2.10.0) and verifies admin login still works.
# Catches schema changes that would break existing user databases under
# Hibernate's `ddl-auto=update` upgrade path.
on:
workflow_call:
permissions:
contents: read
jobs:
pick:
uses: ./.github/workflows/_runner-pick.yml
migration-test:
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-8' }}
timeout-minutes: 30
env:
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Set up JDK 25
uses: actions/setup-java@be666c2fcd27ec809703dec50e508c2fdc7f6654 # v5.2.0
with:
java-version: 25
distribution: temurin
- name: Cache Gradle dependency artifacts
uses: actions/cache@27d5ce7f107fe9357f9df03efb73ab90386fccae # v5.0.5
with:
path: |
~/.gradle/wrapper
~/.gradle/caches/modules-2/files-2.1
~/.gradle/caches/modules-2/metadata-2.*
key: gradle-deps-${{ runner.os }}-jdk-25-${{ hashFiles('**/gradle/wrapper/gradle-wrapper.properties', '**/*.gradle', '**/*.gradle.kts', 'settings.gradle', 'settings.gradle.kts', 'gradle/libs.versions.toml') }}
- name: Setup Gradle
uses: gradle/actions/setup-gradle@50e97c2cd7a37755bbfafc9c5b7cafaece252f6e # v6.1.0
with:
gradle-version: 9.3.1
cache-disabled: true
# No `-PnoSpotless` here yet because the upstream cache layer matches the
# backend build's; reuse keeps cold-cache cost identical.
- name: Build Stirling-PDF JAR
env:
MAVEN_USER: ${{ secrets.MAVEN_USER }}
MAVEN_PASSWORD: ${{ secrets.MAVEN_PASSWORD }}
MAVEN_PUBLIC_URL: ${{ secrets.MAVEN_PUBLIC_URL }}
run: ./gradlew :stirling-pdf:bootJar -PnoSpotless --no-daemon
- name: Locate built JAR
id: jar
run: |
jar=$(find app/core/build/libs -maxdepth 1 -name 'Stirling-PDF*.jar' -o -name 'stirling-pdf*.jar' 2>/dev/null \
| grep -vE '(-plain|-sources)\.jar$' | head -n 1)
if [[ -z "$jar" ]]; then
echo "::error::No JAR under app/core/build/libs"
ls -lah app/core/build/libs || true
exit 1
fi
# Absolute path - the migration script pushd's into a temp workdir
# before invoking java, which would dangle a relative path.
jar=$(realpath "$jar")
echo "path=$jar" >> "$GITHUB_OUTPUT"
echo "Built JAR: $jar"
- name: Run migration smoke test
env:
STIRLING_JAR: ${{ steps.jar.outputs.path }}
run: bash scripts/db-migration/run-migration-test.sh
- name: Upload app logs on failure
if: failure()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: db-migration-app-logs
# Path matches the preserved workdir in run-migration-test.sh -
# only failing fixtures leave a directory behind.
path: /tmp/stirling-migration-failed-*/app.log
retention-days: 7
if-no-files-found: warn
+1 -1
View File
@@ -14,7 +14,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
+1 -1
View File
@@ -28,7 +28,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
+1 -1
View File
@@ -36,7 +36,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
+1 -1
View File
@@ -25,7 +25,7 @@ jobs:
timeout-minutes: 30
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
- name: Checkout repository
+1 -1
View File
@@ -24,7 +24,7 @@ jobs:
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || format('depot-ubuntu-24.04-{0}', inputs.depot_cores || '8') }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
- name: Checkout repository
@@ -32,7 +32,7 @@ jobs:
licenses-backend: ${{ steps.changes.outputs.licenses-backend }}
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -56,7 +56,7 @@ jobs:
repository-projects: write # Required for enabling automerge
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -322,7 +322,7 @@ jobs:
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
+1 -1
View File
@@ -19,7 +19,7 @@ jobs:
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
- name: Checkout repository
+1 -1
View File
@@ -15,7 +15,7 @@ jobs:
issues: write
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
+13 -11
View File
@@ -48,7 +48,7 @@ jobs:
version: ${{ steps.versionNumber.outputs.versionNumber }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -133,7 +133,7 @@ jobs:
file_suffix: "-server"
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -198,7 +198,7 @@ jobs:
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
allowed-endpoints: >
@@ -586,21 +586,23 @@ jobs:
if: always() && steps.digicert-setup.conclusion != 'failure'
shell: bash
run: |
mkdir -p ./dist
# Absolute dist path so the cd below can't break the copy targets.
DIST="$GITHUB_WORKSPACE/dist"
mkdir -p "$DIST"
cd ./frontend/editor/src-tauri/target
# Find and rename artifacts based on platform
if [ "${{ matrix.platform }}" = "windows-latest" ]; then
# Only ship the MSI installer on Windows. The loose exe and WiX toolset exes
# are not the user-facing installer - the MSI contains the signed inner exe.
find . -name "*.msi" -exec cp {} "../../../dist/Stirling-PDF-${{ matrix.name }}.msi" \;
find . -name "*.msi" -exec cp {} "$DIST/Stirling-PDF-${{ matrix.name }}.msi" \;
elif [ "${{ matrix.platform }}" = "macos-15" ]; then
find . -name "*.dmg" -exec cp {} "../../../dist/Stirling-PDF-${{ matrix.name }}.dmg" \;
find . -name "*.app" -exec cp -r {} "../../../dist/Stirling-PDF-${{ matrix.name }}.app" \;
find . -name "*.dmg" -exec cp {} "$DIST/Stirling-PDF-${{ matrix.name }}.dmg" \;
find . -name "*.app" -exec cp -r {} "$DIST/Stirling-PDF-${{ matrix.name }}.app" \;
else
find . -name "*.deb" -exec cp {} "../../../dist/Stirling-PDF-${{ matrix.name }}.deb" \;
find . -name "*.rpm" -exec cp {} "../../../dist/Stirling-PDF-${{ matrix.name }}.rpm" \;
find . -name "*.AppImage" -exec cp {} "../../../dist/Stirling-PDF-${{ matrix.name }}.AppImage" \;
find . -name "*.deb" -exec cp {} "$DIST/Stirling-PDF-${{ matrix.name }}.deb" \;
find . -name "*.rpm" -exec cp {} "$DIST/Stirling-PDF-${{ matrix.name }}.rpm" \;
find . -name "*.AppImage" -exec cp {} "$DIST/Stirling-PDF-${{ matrix.name }}.AppImage" \;
fi
- name: Upload build artifacts
@@ -619,7 +621,7 @@ jobs:
contents: write
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
+1 -1
View File
@@ -22,7 +22,7 @@ jobs:
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
+4 -4
View File
@@ -2,7 +2,7 @@ name: Update Package Manager Manifests
on:
release:
types: [ released ]
types: [released]
workflow_dispatch:
inputs:
version:
@@ -28,7 +28,7 @@ jobs:
jar_sha256: ${{ steps.hashes.outputs.jar_sha256 }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -79,12 +79,12 @@ jobs:
contents: write
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
- name: Checkout homebrew-stirling-pdf tap (also hosts Scoop bucket)
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
repository: Stirling-Tools/homebrew-stirling-pdf
token: ${{ secrets.HOMEBREW_TAP_TOKEN }}
+1 -1
View File
@@ -19,7 +19,7 @@ jobs:
PIP_DISABLE_PIP_VERSION_CHECK: "1"
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
+1 -1
View File
@@ -43,7 +43,7 @@ jobs:
echo "version=${VERSION}" >> $GITHUB_OUTPUT
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
+1 -1
View File
@@ -53,7 +53,7 @@ jobs:
RUN_UNOSERVER: ${{ github.event_name != 'workflow_dispatch' || inputs.build_unoserver }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
+1 -1
View File
@@ -18,7 +18,7 @@ jobs:
packages: write
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
+2 -2
View File
@@ -35,7 +35,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -75,6 +75,6 @@ jobs:
# Upload the results to GitHub's code scanning dashboard.
- name: "Upload to code-scanning"
uses: github/codeql-action/upload-sarif@68bde559dea0fdcac2102bfdf6230c5f70eb485e # v3.29.5
uses: github/codeql-action/upload-sarif@9e0d7b8d25671d64c341c19c0152d693099fb5ba # v3.29.5
with:
sarif_file: results.sarif
+1 -1
View File
@@ -17,7 +17,7 @@ jobs:
pull-requests: write
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
+1 -1
View File
@@ -33,7 +33,7 @@ jobs:
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
+1 -1
View File
@@ -35,7 +35,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
+19 -10
View File
@@ -4,6 +4,10 @@ name: Build Tauri Applications
# Linux). Called from build.yml on PRs that touch desktop sources (gated
# via the `tauri` filter in .github/config/.files.yaml). Also runnable
# on demand via workflow_dispatch with a per-platform selector.
#
# Note: editing this file is itself enough to make the `tauri` path filter
# match, which is how non-desktop PRs (e.g. backend-only fixes) opt into a
# desktop smoke build.
on:
workflow_call:
inputs:
@@ -37,7 +41,7 @@ jobs:
matrix: ${{ steps.set-matrix.outputs.matrix }}
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -85,7 +89,7 @@ jobs:
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -153,6 +157,9 @@ jobs:
JPDFIUM_PLATFORMS: ${{ matrix.jpdfium_platforms }}
run: task desktop:prepare
- name: Run Tauri/Cargo tests
run: task desktop:test
# DigiCert KeyLocker Setup (Cloud HSM)
- name: Setup DigiCert KeyLocker
id: digicert-setup
@@ -413,20 +420,22 @@ jobs:
- name: Rename artifacts
shell: bash
run: |
mkdir -p ./dist
# Absolute dist path so the cd below can't break the copy targets.
DIST="$GITHUB_WORKSPACE/dist"
mkdir -p "$DIST"
cd ./frontend/editor/src-tauri/target
# Find and rename artifacts based on platform
if [ "${{ matrix.platform }}" = "windows-latest" ]; then
# Only ship the MSI installer. The loose exe and WiX toolset exes
# are not the user-facing installer - the MSI contains the signed inner exe.
find . -name "*.msi" -exec cp {} "../../../dist/Stirling-PDF-${{ matrix.name }}.msi" \;
find . -name "*.msi" -exec cp {} "$DIST/Stirling-PDF-${{ matrix.name }}.msi" \;
elif [ "${{ matrix.platform }}" = "macos-15" ]; then
find . -name "*.dmg" -exec cp {} "../../../dist/Stirling-PDF-${{ matrix.name }}.dmg" \;
find . -name "*.dmg" -exec cp {} "$DIST/Stirling-PDF-${{ matrix.name }}.dmg" \;
else
find . -name "*.deb" -exec cp {} "../../../dist/Stirling-PDF-${{ matrix.name }}.deb" \;
find . -name "*.rpm" -exec cp {} "../../../dist/Stirling-PDF-${{ matrix.name }}.rpm" \;
find . -name "*.AppImage" -exec cp {} "../../../dist/Stirling-PDF-${{ matrix.name }}.AppImage" \;
find . -name "*.deb" -exec cp {} "$DIST/Stirling-PDF-${{ matrix.name }}.deb" \;
find . -name "*.rpm" -exec cp {} "$DIST/Stirling-PDF-${{ matrix.name }}.rpm" \;
find . -name "*.AppImage" -exec cp {} "$DIST/Stirling-PDF-${{ matrix.name }}.AppImage" \;
fi
# Verify the MSI AND the inner exe extracted from it are signed.
@@ -556,7 +565,7 @@ jobs:
pull-requests: write
steps:
- name: Harden the runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -647,7 +656,7 @@ jobs:
if: always()
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
+2 -2
View File
@@ -63,7 +63,7 @@ jobs:
cache-scope: stirling-pdf-fat
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -208,7 +208,7 @@ jobs:
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
+4 -4
View File
@@ -35,7 +35,7 @@ jobs:
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -160,7 +160,7 @@ jobs:
frontend: ${{ steps.changes.outputs.frontend }}
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -178,7 +178,7 @@ jobs:
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -214,7 +214,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
+8 -1
View File
@@ -23,6 +23,10 @@ customFiles/
configs/
watchedFolders/
clientWebUI/
# Scratch dir used by local fixture-regeneration runs (see
# app/proprietary/src/test/resources/db-migration-fixtures/README.md).
# Holds downloaded JARs and disposable workdirs. Never committed.
.alpha-local/
!cucumber/
!cucumber/exampleFiles/
!cucumber/exampleFiles/example_html.zip
@@ -157,7 +161,7 @@ app/proprietary/build
common/build
proprietary/build
stirling-pdf/build
frontend/src-tauri/provisioner/target
frontend/editor/src-tauri/provisioner/target
# Byte-compiled / optimized / DLL files
__pycache__/
@@ -273,3 +277,6 @@ docs/type3/signatures/
# Playwright MCP screenshots / traces
.playwright-mcp/
*.playwright-mcp.png
# Local screenshot artifacts from *-screenshots.spec.ts
frontend/screenshots/
+1 -1
View File
@@ -1,6 +1,6 @@
repos:
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.15.11
rev: v0.15.14
hooks:
- id: ruff
args:
+19
View File
@@ -39,6 +39,25 @@ tasks:
- cmd: ./gradlew clean bootRun -PbuildWithFrontend=true
platforms: [linux, darwin]
dev:saas:
desc: "Start backend in SaaS flavor against Supabase"
# `dotenv:` reads from the root Taskfile's directory (".") because this
# subtaskfile is included with `dir: .`.
dotenv: ['app/.env.saas.local', 'app/.env.saas']
ignore_error: true
vars:
PORT: '{{.PORT | default "8080"}}'
# Override to "" to run the pure `saas` profile against your own SAAS_DB_*.
PROFILES: '{{.PROFILES | default "dev"}}'
env:
SERVER_PORT: '{{.PORT}}'
STIRLING_FLAVOR: saas
cmds:
- cmd: cmd /c ".\gradlew.bat :stirling-pdf:bootRun {{if .PROFILES}}--args=\"--spring.profiles.include={{.PROFILES}}\"{{end}}"
platforms: [windows]
- cmd: ./gradlew :stirling-pdf:bootRun {{if .PROFILES}}--args='--spring.profiles.include={{.PROFILES}}'{{end}}
platforms: [linux, darwin]
build:
desc: "Full backend build"
cmds:
+33 -15
View File
@@ -31,52 +31,67 @@ tasks:
provisioner:
desc: "Build installer provisioner"
platforms: [windows]
dir: editor
cmds:
- node editor/scripts/build-provisioner.mjs
- node scripts/build-provisioner.mjs
dev:
desc: "Start Tauri desktop dev mode"
deps: [prepare]
ignore_error: true
dir: editor
cmds:
- cd editor && npx tauri dev --no-watch
- npx tauri dev --no-watch
build:
desc: "Build Tauri desktop app (production)"
deps: [prepare]
dir: editor
cmds:
- cd editor && npx tauri build
- npx tauri build
build:dev:
desc: "Build Tauri desktop app (dev, no bundling)"
deps: [prepare]
dir: editor
cmds:
- cd editor && npx tauri build --no-bundle
- npx tauri build --no-bundle
build:dev:mac:
desc: "Build Tauri desktop .app bundle (macOS)"
deps: [prepare]
dir: editor
cmds:
- cd editor && npx tauri build --bundles app
- npx tauri build --bundles app
build:dev:windows:
desc: "Build Tauri desktop NSIS installer (Windows)"
deps: [prepare]
dir: editor
cmds:
- cd editor && npx tauri build --bundles nsis
- npx tauri build --bundles nsis
build:dev:linux:
desc: "Build Tauri desktop AppImage (Linux)"
deps: [prepare]
dir: editor
cmds:
- cd editor && npx tauri build --bundles appimage
- npx tauri build --bundles appimage
test:
desc: "Run Tauri/Cargo tests"
deps: [prepare]
dir: editor/src-tauri
cmds:
- cargo test
clean:
desc: "Clean Tauri/Cargo build artifacts"
dir: editor
cmds:
- task: jlink:clean
- cd editor/src-tauri && cargo clean
- rm -rf editor/dist editor/build
- cd src-tauri && cargo clean
- rm -rf dist build
# ============================================================
# JLink — Build bundled Java runtime for Tauri
@@ -106,9 +121,10 @@ tasks:
jlink:runtime:
desc: "Create custom JRE with jlink"
deps: [jlink:jar]
dir: editor/src-tauri
cmds:
- rm -rf editor/src-tauri/runtime/jre
- mkdir -p editor/src-tauri/runtime
- rm -rf runtime/jre
- mkdir -p runtime
- >-
jlink
--add-modules {{.JLINK_MODULES}}
@@ -116,14 +132,15 @@ tasks:
--compress=zip-6
--no-header-files
--no-man-pages
--output editor/src-tauri/runtime/jre
--output runtime/jre
status:
- test -d editor/src-tauri/runtime/jre
jlink:clean:
desc: "Remove JLink runtime and bundled JARs"
dir: editor/src-tauri
cmds:
- rm -rf editor/src-tauri/libs editor/src-tauri/runtime
- rm -rf libs runtime
# macOS-only. Replaces jlink:runtime's single-arch JRE with a universal
# (arm64 + x86_64) one for the universal Tauri shell. Runs the x86_64
@@ -134,8 +151,9 @@ tasks:
desc: "Create universal (arm64+x86_64) JRE for the macOS Tauri build"
deps: [jlink:jar]
platforms: [darwin]
dir: editor
env:
JLINK_MODULES: "{{.JLINK_MODULES}}"
OUTPUT_DIR: editor/src-tauri/runtime/jre
OUTPUT_DIR: src-tauri/runtime/jre
cmds:
- editor/scripts/build-universal-mac-jre.sh
- scripts/build-universal-mac-jre.sh
+17
View File
@@ -58,6 +58,23 @@ tasks:
BACKEND_URL: 'http://localhost:{{.BACKEND_PORT}}'
OPEN: "true"
dev:saas:
desc: "Start SaaS backend + frontend concurrently on free ports"
vars:
PORTS:
sh: '{{if eq OS "windows"}}{{.FIND_FREE_PORT_PS}} 8080 5173{{else}}{{.FIND_FREE_PORT_SH}} 8080 5173{{end}}'
BACKEND_PORT: '{{index (splitList "\n" .PORTS) 0}}'
FRONTEND_PORT: '{{index (splitList "\n" .PORTS) 1}}'
deps:
- task: backend:dev:saas
vars:
PORT: '{{.BACKEND_PORT}}'
- task: frontend:dev:saas
vars:
PORT: '{{.FRONTEND_PORT}}'
BACKEND_URL: 'http://localhost:{{.BACKEND_PORT}}'
OPEN: "true"
dev:all:
desc: "Start backend + frontend + engine concurrently on free ports"
vars:
+34
View File
@@ -0,0 +1,34 @@
###############################################################################
# Stirling-PDF SaaS environment defaults.
#
# This file is committed and provides non-secret defaults loaded by
# `task backend:dev:saas`. Put real values for secrets (passwords, project
# refs, edge function secrets) in `.env.saas.local` - any variable set there
# takes precedence over what's defined here.
#
# DO NOT commit `.env.saas.local`. Only `.env.saas` is checked in.
###############################################################################
# ---------- Supabase project ----------
# Project reference (the subdomain part of <ref>.supabase.co). Required.
# Set in .env.saas.local.
SAAS_DB_PROJECT_REF=
# Edge function secret used by billing/license rollup calls. Set in .env.saas.local.
SUPABASE_EDGE_FUNCTION_SECRET=
# ---------- Database (saas profile) ----------
# Direct JDBC URL to the Supabase Postgres. Required when running the plain
# `saas` profile (i.e. without `--spring.profiles.include=dev`).
# Example: jdbc:postgresql://db.<project-ref>.supabase.co:5432/postgres
SAAS_DB_URL=
SAAS_DB_USERNAME=postgres
SAAS_DB_PASSWORD=
# ---------- Database (dev profile overrides) ----------
# Used when `--spring.profiles.include=dev` is active. The dev profile
# defaults the URL/username to the shared dev Supabase project, but the
# password must still be provided in .env.saas.local.
SAAS_DEV_DB_URL=
SAAS_DEV_DB_USERNAME=postgres
SAAS_DEV_DB_PASSWORD=
+3
View File
@@ -0,0 +1,3 @@
# Whitelist committed env defaults. `.env.saas.local` (and any other .env*)
# stays ignored via the root .gitignore.
!.env.saas
+4
View File
@@ -44,6 +44,10 @@
"moduleName": ".*",
"moduleLicense": "The MIT License"
},
{
"moduleName": ".*",
"moduleLicense": "MIT-0"
},
{
"moduleName": "com.github.jai-imageio:jai-imageio-core",
"moduleLicense": "LICENSE.txt"
+5 -2
View File
@@ -60,7 +60,7 @@ dependencies {
exclude group: 'com.google.code.gson', module: 'gson'
}
api 'com.stirling:jpdfium:1.0.0'
api 'com.stirling:jpdfium:1.0.1'
// -PjpdfiumPlatforms=all|<csv of linux-x64,linux-arm64,darwin-x64,darwin-arm64,windows-x64>
def jpdfiumPlatformsProp = (project.findProperty('jpdfiumPlatforms') ?: 'all').toString().trim()
@@ -75,9 +75,12 @@ dependencies {
}
logger.lifecycle("JPDFium native platforms: ${jpdfiumPlatforms.join(', ')}")
jpdfiumPlatforms.each { platform ->
runtimeOnly "com.stirling:jpdfium-natives-${platform}:1.0.0"
runtimeOnly "com.stirling:jpdfium-natives-${platform}:1.0.1"
}
// Bucket4j (local in-process token bucket for RateLimitStore default impl)
implementation 'com.bucket4j:bucket4j_jdk17-core:8.19.0'
// ArchUnit: enforces module dependency direction (see ArchitectureTest)
testImplementation 'com.tngtech.archunit:archunit-junit5:1.4.2'
}
@@ -0,0 +1,24 @@
package stirling.software.common.cluster;
/** Health and identity facade for the active cluster backplane. */
public interface ClusterBackplane {
/** Returns {@code true} when the backplane is reachable; used for health endpoints. */
boolean isHealthy();
/** Returns {@code "inprocess"} or {@code "valkey"}. */
String backplaneType();
/** Returns this JVM's stable node id (matches {@code Cluster.resolvedNodeId()}). */
String localNodeId();
/**
* Whether this JVM should run the local {@link
* stirling.software.common.service.TaskManager#cleanupOldJobs()} loop. Distributed backplanes
* own job expiry via their own TTL, so they should override this to return {@code false}.
* Defaults to {@code true} so in-process behavior is preserved without an explicit override.
*/
default boolean shouldRunLocalCleanup() {
return true;
}
}
@@ -0,0 +1,63 @@
package stirling.software.common.cluster;
import org.springframework.context.annotation.Configuration;
import jakarta.annotation.PostConstruct;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import stirling.software.common.model.ApplicationProperties;
import stirling.software.common.model.ApplicationProperties.Cluster;
/**
* Validates that cluster mode is internally consistent.
*
* <p>Cluster settings are bound on the central {@link ApplicationProperties} under {@code
* cluster.*}; this class reads {@link ApplicationProperties#getCluster()} and runs guards in {@link
* PostConstruct}. When {@code cluster.enabled=false} (the default) all checks are skipped so a
* single-instance install needs no new config.
*/
@Slf4j
@Configuration
@RequiredArgsConstructor
public class ClusterConfig {
private final ApplicationProperties applicationProperties;
@PostConstruct
void validate() {
Cluster cluster = applicationProperties.getCluster();
if (!cluster.isEnabled()) {
return;
}
String backplane = cluster.getBackplane();
if ("valkey".equalsIgnoreCase(backplane)) {
String url = cluster.getValkey() == null ? null : cluster.getValkey().getUrl();
if (url == null || url.isBlank()) {
throw new IllegalStateException(
"cluster.enabled=true with backplane=valkey requires"
+ " cluster.valkey.url to be set (e.g."
+ " redis://valkey:6379).");
}
} else if ("inprocess".equalsIgnoreCase(backplane)) {
// enabled+inprocess only coordinates the local JVM; cross-node lookups will 410.
log.warn(
"cluster.enabled=true with backplane=inprocess - only the local"
+ " JVM is coordinated. Cross-node lookups and the file proxy will fail."
+ " Use backplane=valkey for real multi-node deployments.");
} else {
// Fail fast on typos like "valky" so Spring doesn't later report a cryptic
// "no ClusterBackplane bean" - the operator-facing error names the bad value.
throw new IllegalStateException(
"cluster.enabled=true with unknown backplane '"
+ backplane
+ "'. Valid values: inprocess | valkey.");
}
log.info(
"Cluster mode enabled (backplane={}, nodeRole={}, nodeId={}).",
backplane,
cluster.resolvedRole(),
cluster.resolvedNodeId());
}
}
@@ -0,0 +1,12 @@
package stirling.software.common.cluster;
import java.time.Instant;
/**
* Snapshot of a peer node as recorded in the {@link InstanceRegistry}.
*
* @param internalAddress {@code host:port} the node listens on for {@code /internal/cluster/**}
* @param role one of {@code WEB}, {@code WORKER}, {@code BOTH}
*/
public record ClusterNode(
String nodeId, String internalAddress, Instant lastHeartbeat, String role) {}
@@ -0,0 +1,21 @@
package stirling.software.common.cluster;
import java.time.Duration;
import java.util.Optional;
/** Cluster-wide mutual exclusion primitive; non-reentrant by contract. */
public interface DistributedLock {
Optional<LockHandle> tryAcquire(String lockKey, Duration leaseTime);
interface LockHandle extends AutoCloseable {
void release();
boolean renew(Duration leaseTime);
@Override
default void close() {
release();
}
}
}
@@ -0,0 +1,45 @@
package stirling.software.common.cluster;
import java.io.IOException;
import java.io.InputStream;
import java.nio.file.Files;
import java.nio.file.Path;
/** Low-level storage seam for result/job files. */
public interface FileStore {
/** Stored file record. */
record Stored(String fileId, long size) {}
/** Store the given stream and return a generated file id and total bytes written. */
Stored store(InputStream in, String originalName) throws IOException;
/**
* Store the file at {@code source} and return a generated file id and total bytes written.
*
* <p>Default implementation opens {@code source} as a stream and delegates to {@link
* #store(InputStream, String)}. Local-disk implementations should override to use a direct
* file-to-file copy ({@code Files.copy(source, dest)} can use {@code sendfile(2)} on Linux),
* which avoids the two-memory-copy hit of streaming a disk-backed upload through the JVM heap.
*/
default Stored store(Path source, String originalName) throws IOException {
try (InputStream in = Files.newInputStream(source)) {
return store(in, originalName);
}
}
/** Open the stored file for streaming reads. Caller closes. */
InputStream retrieve(String fileId) throws IOException;
/** Load the stored file into a byte array. */
byte[] retrieveBytes(String fileId) throws IOException;
/** Size of the stored file in bytes. */
long size(String fileId) throws IOException;
/** Delete the stored file. Returns true if a file was removed. */
boolean delete(String fileId);
/** Whether the file id exists in the store. */
boolean exists(String fileId);
}
@@ -0,0 +1,18 @@
package stirling.software.common.cluster;
import java.time.Duration;
import java.util.Collection;
import java.util.Optional;
/** Maps {@code nodeId} to its internal cluster address, with TTL'd heartbeats. */
public interface InstanceRegistry {
/** Register or refresh this node. Idempotent so a wiped backplane self-heals on next tick. */
void register(ClusterNode node, Duration heartbeatTtl);
Optional<ClusterNode> lookup(String nodeId);
Collection<ClusterNode> activeNodes();
void deregister(String nodeId);
}
@@ -0,0 +1,24 @@
package stirling.software.common.cluster;
import java.time.Duration;
import java.util.Collection;
import java.util.Optional;
/** Cluster-visible storage for job status and result metadata, with TTL'd entries. */
public interface JobStore {
/** Persist or overwrite a job entry. {@code ttl} sets the lifetime of the entry. */
void put(JobStoreEntry entry, Duration ttl);
Optional<JobStoreEntry> get(String jobId);
void delete(String jobId);
boolean exists(String jobId);
/** Reverse lookup: which job owns this result file id? */
Optional<String> findJobIdByFileId(String fileId);
/** Snapshot of every active entry. Used by admin/stats endpoints; may be O(n). */
Collection<JobStoreEntry> all();
}
@@ -0,0 +1,30 @@
package stirling.software.common.cluster;
import java.time.Instant;
import java.util.List;
import java.util.Map;
/**
* Cluster-visible projection of a job's status and result metadata, as persisted in {@link
* JobStore}.
*
* @param owningNodeId the node id that originally executed the job
*/
public record JobStoreEntry(
String jobId,
JobState state,
String owningNodeId,
Instant createdAt,
Instant completedAt,
String error,
List<String> fileIds,
Map<String, String> resultMeta) {
/** Lifecycle states for a job as observed by the cluster. */
public enum JobState {
PENDING,
RUNNING,
COMPLETE,
FAILED
}
}
@@ -0,0 +1,16 @@
package stirling.software.common.cluster;
import java.time.Duration;
import java.util.Optional;
/** Short-TTL namespaced key/value cache backed by the cluster backplane. */
public interface KeyValueCache {
void put(String namespace, String key, String value, Duration ttl);
Optional<String> get(String namespace, String key);
void evict(String namespace, String key);
void evictNamespace(String namespace);
}
@@ -0,0 +1,18 @@
package stirling.software.common.cluster;
import java.time.Duration;
/** Token-bucket rate limiting backed by the cluster backplane. */
public interface RateLimitStore {
/**
* Attempt to consume one token from the bucket identified by {@code bucketKey}.
*
* @param bucketKey opaque key identifying the bucket (e.g. {@code api:user:123})
* @param capacity bucket capacity
* @param refillPeriod time window over which {@code capacity} tokens refill
*/
RateLimitDecision tryConsume(String bucketKey, long capacity, Duration refillPeriod);
record RateLimitDecision(boolean allowed, long remainingTokens, long nanosToWaitForRefill) {}
}
@@ -0,0 +1,7 @@
package stirling.software.common.cluster;
/** Records one increment per sticky-session miss (a 410 Gone for a job owned by another node). */
@FunctionalInterface
public interface StickyMissRecorder {
void recordStickyMiss();
}
@@ -0,0 +1,31 @@
package stirling.software.common.cluster.inprocess;
import lombok.extern.slf4j.Slf4j;
import stirling.software.common.cluster.ClusterBackplane;
import stirling.software.common.model.ApplicationProperties;
@Slf4j
public class InProcessClusterBackplane implements ClusterBackplane {
private final ApplicationProperties applicationProperties;
public InProcessClusterBackplane(ApplicationProperties applicationProperties) {
this.applicationProperties = applicationProperties;
}
@Override
public boolean isHealthy() {
return true;
}
@Override
public String backplaneType() {
return "inprocess";
}
@Override
public String localNodeId() {
return applicationProperties.getCluster().resolvedNodeId();
}
}
@@ -0,0 +1,65 @@
package stirling.software.common.cluster.inprocess;
import org.springframework.boot.autoconfigure.condition.ConditionalOnExpression;
import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import lombok.extern.slf4j.Slf4j;
import stirling.software.common.cluster.ClusterBackplane;
import stirling.software.common.cluster.DistributedLock;
import stirling.software.common.cluster.InstanceRegistry;
import stirling.software.common.cluster.JobStore;
import stirling.software.common.cluster.KeyValueCache;
import stirling.software.common.cluster.RateLimitStore;
import stirling.software.common.model.ApplicationProperties;
/**
* Default cluster backplane wiring: every interface gets an {@code InProcess*} bean. Active when
* cluster mode is off or {@code cluster.backplane=inprocess}.
*/
@Slf4j
@Configuration
@ConditionalOnExpression(
"!${cluster.enabled:false} ||"
+ " '${cluster.backplane:inprocess}'.equalsIgnoreCase('inprocess')")
public class InProcessClusterConfiguration {
@Bean
@ConditionalOnMissingBean
public ClusterBackplane clusterBackplane(ApplicationProperties applicationProperties) {
log.info("Cluster backplane: in-process (single node)");
return new InProcessClusterBackplane(applicationProperties);
}
@Bean
@ConditionalOnMissingBean
public JobStore jobStore() {
return new InProcessJobStore();
}
@Bean
@ConditionalOnMissingBean
public RateLimitStore rateLimitStore() {
return new InProcessRateLimitStore();
}
@Bean
@ConditionalOnMissingBean
public DistributedLock distributedLock() {
return new InProcessDistributedLock();
}
@Bean
@ConditionalOnMissingBean
public KeyValueCache keyValueCache() {
return new InProcessKeyValueCache();
}
@Bean
@ConditionalOnMissingBean
public InstanceRegistry instanceRegistry() {
return new InProcessInstanceRegistry();
}
}
@@ -0,0 +1,128 @@
package stirling.software.common.cluster.inprocess;
import java.time.Duration;
import java.util.Optional;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.atomic.AtomicLong;
import stirling.software.common.cluster.DistributedLock;
/**
* In-process {@link DistributedLock}, non-reentrant per the interface contract, with lease-expiry
* semantics that mirror a SET-NX-EX style distributed backend.
*
* <p>Each lock state carries a per-acquire {@code ownerToken} and an {@code expiryNanos}; another
* caller can take over once the lease has elapsed even if the original holder never called {@link
* LockHandle#release()}. This matters mostly for parity with the Valkey-backed implementation
* (Redis {@code SETEX} auto-expires the key); within a single JVM a crashed holder takes its lock
* state with it, but tests and code that rely on the {@code leaseTime} parameter still need it to
* be honored.
*
* <p>Expiry is lazy: an expired lock state lingers in the map until the next acquire attempt for
* the same key replaces it. Per-key cleanup also happens on explicit {@link LockHandle#release()},
* so a balanced acquire/release workload keeps the map size bounded.
*/
public class InProcessDistributedLock implements DistributedLock {
private final ConcurrentHashMap<String, LockState> locks = new ConcurrentHashMap<>();
private final AtomicLong tokenSeq = new AtomicLong();
/**
* Lease state for a single acquired lock. {@code ownerToken} prevents a former holder from
* releasing or renewing a lock now owned by someone else after lease expiry; {@code
* expiryNanos} is read/written only inside {@link ConcurrentHashMap#compute} so the bin lock
* provides the necessary happens-before guarantee.
*/
private static final class LockState {
final long ownerToken;
long expiryNanos;
LockState(long ownerToken, long expiryNanos) {
this.ownerToken = ownerToken;
this.expiryNanos = expiryNanos;
}
}
@Override
public Optional<LockHandle> tryAcquire(String lockKey, Duration leaseTime) {
long token = tokenSeq.incrementAndGet();
long nowNanos = System.nanoTime();
long expiryNanos = nowNanos + leaseTime.toNanos();
boolean[] acquired = {false};
locks.compute(
lockKey,
(k, existing) -> {
if (existing == null || existing.expiryNanos - nowNanos <= 0L) {
// No lock, or the previous lease has expired - we take it. Subtraction
// form avoids the long-overflow trap that would bite a naive
// expiryNanos <= nowNanos comparison around System.nanoTime() rollover.
acquired[0] = true;
return new LockState(token, expiryNanos);
}
return existing;
});
if (!acquired[0]) {
return Optional.empty();
}
return Optional.of(new InProcessHandle(lockKey, token));
}
private void releaseInternal(String lockKey, long token) {
locks.compute(
lockKey,
(k, existing) -> {
if (existing == null || existing.ownerToken != token) {
// Already removed, expired-and-replaced, or never ours.
return existing;
}
return null;
});
}
private boolean renewInternal(String lockKey, long token, Duration leaseTime) {
long nowNanos = System.nanoTime();
boolean[] renewed = {false};
locks.compute(
lockKey,
(k, existing) -> {
if (existing == null
|| existing.ownerToken != token
|| existing.expiryNanos - nowNanos <= 0L) {
// Lock is gone or expired; renewal is a no-op so the caller can detect it.
return existing;
}
existing.expiryNanos = nowNanos + leaseTime.toNanos();
renewed[0] = true;
return existing;
});
return renewed[0];
}
private final class InProcessHandle implements LockHandle {
private final String lockKey;
private final long token;
private boolean released;
InProcessHandle(String lockKey, long token) {
this.lockKey = lockKey;
this.token = token;
}
@Override
public synchronized void release() {
if (released) {
return;
}
released = true;
releaseInternal(lockKey, token);
}
@Override
public synchronized boolean renew(Duration leaseTime) {
if (released) {
return false;
}
return renewInternal(lockKey, token, leaseTime);
}
}
}
@@ -0,0 +1,42 @@
package stirling.software.common.cluster.inprocess;
import java.time.Duration;
import java.util.Collection;
import java.util.Collections;
import java.util.Optional;
import java.util.concurrent.atomic.AtomicReference;
import stirling.software.common.cluster.ClusterNode;
import stirling.software.common.cluster.InstanceRegistry;
public class InProcessInstanceRegistry implements InstanceRegistry {
private final AtomicReference<ClusterNode> self = new AtomicReference<>();
@Override
public void register(ClusterNode node, Duration heartbeatTtl) {
self.set(node);
}
@Override
public Optional<ClusterNode> lookup(String nodeId) {
ClusterNode current = self.get();
return current != null && current.nodeId().equals(nodeId)
? Optional.of(current)
: Optional.empty();
}
@Override
public Collection<ClusterNode> activeNodes() {
ClusterNode current = self.get();
return current == null ? Collections.emptyList() : Collections.singletonList(current);
}
@Override
public void deregister(String nodeId) {
ClusterNode current = self.get();
if (current != null && current.nodeId().equals(nodeId)) {
self.set(null);
}
}
}
@@ -0,0 +1,96 @@
package stirling.software.common.cluster.inprocess;
import java.time.Duration;
import java.time.Instant;
import java.util.ArrayList;
import java.util.Collection;
import java.util.List;
import java.util.Map;
import java.util.Optional;
import java.util.concurrent.ConcurrentHashMap;
import lombok.extern.slf4j.Slf4j;
import stirling.software.common.cluster.JobStore;
import stirling.software.common.cluster.JobStoreEntry;
@Slf4j
public class InProcessJobStore implements JobStore {
private final ConcurrentHashMap<String, Holder> entries = new ConcurrentHashMap<>();
@Override
public void put(JobStoreEntry entry, Duration ttl) {
Instant expiry = ttl == null ? Instant.MAX : Instant.now().plus(ttl);
entries.put(entry.jobId(), new Holder(entry, expiry));
}
@Override
public Optional<JobStoreEntry> get(String jobId) {
Holder h = entries.get(jobId);
if (h == null) {
return Optional.empty();
}
if (h.isExpired()) {
entries.remove(jobId, h);
return Optional.empty();
}
return Optional.of(h.entry);
}
@Override
public void delete(String jobId) {
entries.remove(jobId);
}
@Override
public boolean exists(String jobId) {
return get(jobId).isPresent();
}
@Override
public Optional<String> findJobIdByFileId(String fileId) {
for (Map.Entry<String, Holder> e : entries.entrySet()) {
Holder h = e.getValue();
if (h.isExpired()) {
continue;
}
List<String> fileIds = h.entry.fileIds();
if (fileIds != null && fileIds.contains(fileId)) {
return Optional.of(e.getKey());
}
}
return Optional.empty();
}
@Override
public Collection<JobStoreEntry> all() {
List<JobStoreEntry> result = new ArrayList<>(entries.size());
for (Holder h : entries.values()) {
if (!h.isExpired()) {
result.add(h.entry);
}
}
return result;
}
/** Drop entries whose TTL has elapsed. Called by the {@code TaskManager} cleanup scheduler. */
public int purgeExpired() {
int removed = 0;
Instant now = Instant.now();
for (Map.Entry<String, Holder> e : entries.entrySet()) {
if (!e.getValue().expiry.equals(Instant.MAX) && e.getValue().expiry.isBefore(now)) {
if (entries.remove(e.getKey(), e.getValue())) {
removed++;
}
}
}
return removed;
}
private record Holder(JobStoreEntry entry, Instant expiry) {
boolean isExpired() {
return !expiry.equals(Instant.MAX) && expiry.isBefore(Instant.now());
}
}
}
@@ -0,0 +1,55 @@
package stirling.software.common.cluster.inprocess;
import java.time.Duration;
import java.time.Instant;
import java.util.Map;
import java.util.Optional;
import java.util.concurrent.ConcurrentHashMap;
import stirling.software.common.cluster.KeyValueCache;
public class InProcessKeyValueCache implements KeyValueCache {
private final ConcurrentHashMap<String, ConcurrentHashMap<String, Expiring>> namespaces =
new ConcurrentHashMap<>();
@Override
public void put(String namespace, String key, String value, Duration ttl) {
Instant expiry = ttl == null ? Instant.MAX : Instant.now().plus(ttl);
namespaces
.computeIfAbsent(namespace, n -> new ConcurrentHashMap<>())
.put(key, new Expiring(value, expiry));
}
@Override
public Optional<String> get(String namespace, String key) {
Map<String, Expiring> ns = namespaces.get(namespace);
if (ns == null) {
return Optional.empty();
}
Expiring e = ns.get(key);
if (e == null) {
return Optional.empty();
}
if (e.expiry.isBefore(Instant.now())) {
ns.remove(key, e);
return Optional.empty();
}
return Optional.of(e.value);
}
@Override
public void evict(String namespace, String key) {
Map<String, Expiring> ns = namespaces.get(namespace);
if (ns != null) {
ns.remove(key);
}
}
@Override
public void evictNamespace(String namespace) {
namespaces.remove(namespace);
}
private record Expiring(String value, Instant expiry) {}
}
@@ -0,0 +1,49 @@
package stirling.software.common.cluster.inprocess;
import java.time.Duration;
import java.util.Collections;
import java.util.LinkedHashMap;
import java.util.Map;
import io.github.bucket4j.Bandwidth;
import io.github.bucket4j.Bucket;
import io.github.bucket4j.ConsumptionProbe;
import io.github.bucket4j.local.LocalBucketBuilder;
import stirling.software.common.cluster.RateLimitStore;
/** Bucket4j-backed token bucket implementation of {@link RateLimitStore}. */
public class InProcessRateLimitStore implements RateLimitStore {
/** Cap to bound memory; oldest accessed buckets are evicted. */
private static final int MAX_BUCKETS = 10_000;
private final Map<String, Bucket> buckets =
Collections.synchronizedMap(
new LinkedHashMap<String, Bucket>(256, 0.75f, true) {
@Override
protected boolean removeEldestEntry(Map.Entry<String, Bucket> eldest) {
return size() > MAX_BUCKETS;
}
});
@Override
public RateLimitDecision tryConsume(String bucketKey, long capacity, Duration refillPeriod) {
String compositeKey = bucketKey + "|" + capacity + "|" + refillPeriod.toNanos();
Bucket bucket =
buckets.computeIfAbsent(compositeKey, k -> buildBucket(capacity, refillPeriod));
ConsumptionProbe probe = bucket.tryConsumeAndReturnRemaining(1);
return new RateLimitDecision(
probe.isConsumed(),
probe.getRemainingTokens(),
probe.isConsumed() ? 0L : probe.getNanosToWaitForRefill());
}
private static Bucket buildBucket(long capacity, Duration refillPeriod) {
Bandwidth limit =
Bandwidth.builder().capacity(capacity).refillGreedy(capacity, refillPeriod).build();
LocalBucketBuilder builder = Bucket.builder();
builder.addLimit(limit);
return builder.build();
}
}
@@ -0,0 +1,128 @@
package stirling.software.common.cluster.inprocess;
import java.io.BufferedInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.UUID;
import lombok.extern.slf4j.Slf4j;
import stirling.software.common.cluster.FileStore;
/** Local-disk {@link FileStore} storing files under a base directory keyed by a UUID file id. */
@Slf4j
public class LocalDiskFileStore implements FileStore {
private final String baseDirPath;
public LocalDiskFileStore(String baseDirPath) {
this.baseDirPath = baseDirPath;
}
@Override
public Stored store(InputStream in, String originalName) throws IOException {
String fileId = UUID.randomUUID().toString();
Path filePath = resolve(fileId);
Files.createDirectories(filePath.getParent());
boolean success = false;
try {
long size = Files.copy(in, filePath);
success = true;
return new Stored(fileId, size);
} finally {
if (!success) {
try {
Files.deleteIfExists(filePath);
} catch (IOException cleanupEx) {
log.warn(
"Failed to clean up partial file {} after store failure",
filePath,
cleanupEx);
}
}
}
}
/**
* File-to-file copy. {@link Files#copy(Path, Path, java.nio.file.CopyOption...)} can use {@code
* sendfile(2)} on Linux for a zero-copy kernel transfer when source and destination share a
* filesystem, avoiding the streaming overhead of pulling the bytes through the JVM heap. Reads
* the source size before copying so the post-copy stat is unnecessary.
*/
@Override
public Stored store(Path source, String originalName) throws IOException {
String fileId = UUID.randomUUID().toString();
Path filePath = resolve(fileId);
Files.createDirectories(filePath.getParent());
long size = Files.size(source);
boolean success = false;
try {
Files.copy(source, filePath);
success = true;
return new Stored(fileId, size);
} finally {
if (!success) {
try {
Files.deleteIfExists(filePath);
} catch (IOException cleanupEx) {
log.warn(
"Failed to clean up partial file {} after store failure",
filePath,
cleanupEx);
}
}
}
}
@Override
public InputStream retrieve(String fileId) throws IOException {
return new BufferedInputStream(Files.newInputStream(resolve(fileId)));
}
@Override
public byte[] retrieveBytes(String fileId) throws IOException {
Path filePath = resolve(fileId);
if (!Files.exists(filePath)) {
throw new IOException("File not found with ID: " + fileId);
}
return Files.readAllBytes(filePath);
}
@Override
public long size(String fileId) throws IOException {
Path filePath = resolve(fileId);
if (!Files.exists(filePath)) {
throw new IOException("File not found with ID: " + fileId);
}
return Files.size(filePath);
}
@Override
public boolean delete(String fileId) {
try {
return Files.deleteIfExists(resolve(fileId));
} catch (IOException e) {
log.error("Error deleting file with ID: {}", fileId, e);
return false;
}
}
@Override
public boolean exists(String fileId) {
return Files.exists(resolve(fileId));
}
public Path resolve(String fileId) {
if (fileId.contains("..") || fileId.contains("/") || fileId.contains("\\")) {
throw new IllegalArgumentException("Invalid file ID");
}
Path basePath = Path.of(baseDirPath).normalize().toAbsolutePath();
Path resolvedPath = basePath.resolve(fileId).normalize();
if (!resolvedPath.startsWith(basePath)) {
throw new IllegalArgumentException("File ID resolves to an invalid path");
}
return resolvedPath;
}
}
@@ -0,0 +1,29 @@
package stirling.software.common.cluster.inprocess;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import stirling.software.common.cluster.FileStore;
/**
* Always-on wiring for the per-node local-disk {@link FileStore}. Active when {@code
* cluster.artifactStore=local} (the default; {@code matchIfMissing=true}). The S3 artifact-store
* supplies its own bean when {@code cluster.artifactStore=s3}.
*/
@Configuration
@ConditionalOnProperty(
prefix = "cluster",
name = "artifactStore",
havingValue = "local",
matchIfMissing = true)
public class LocalDiskFileStoreConfiguration {
@Bean
@ConditionalOnMissingBean
public FileStore fileStore(@Value("${stirling.tempDir:/tmp/stirling-files}") String tempDir) {
return new LocalDiskFileStore(tempDir);
}
}
@@ -13,6 +13,7 @@ import java.util.Arrays;
import java.util.Collection;
import java.util.List;
import java.util.Locale;
import java.util.UUID;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.context.annotation.Bean;
@@ -77,6 +78,7 @@ public class ApplicationProperties {
private PdfEditor pdfEditor = new PdfEditor();
private AiEngine aiEngine = new AiEngine();
private InternalApi internalApi = new InternalApi();
private Cluster cluster = new Cluster();
@Bean
public PropertySource<?> dynamicYamlPropertySource(ConfigurableEnvironment environment)
@@ -254,6 +256,106 @@ public class ApplicationProperties {
private int longRunningTimeoutSeconds = 600;
}
/**
* Cluster backplane configuration. All keys live under the top-level {@code cluster.*} prefix
* (e.g. env var {@code CLUSTER_ENABLED}). The master switch is {@link #enabled} and defaults to
* off; when off the in-process backplane is wired and no other cluster keys are required.
*/
@Data
public static class Cluster {
/** Master switch. When {@code false} (default) the in-process backplane is wired. */
private boolean enabled = false;
/** Backplane implementation selector. Valid values: {@code inprocess} | {@code valkey}. */
private String backplane = "inprocess";
/**
* Transient cluster job-artifact store selector. Valid values: {@code local} | {@code s3}.
*
* <p>This is distinct from {@code storage.provider}, which selects the backend for
* persistent user-uploaded files. The two switches exist because the user-facing storage
* feature is optional ({@code storage.enabled=false} is common) but every multi-node
* cluster still needs a shared artifact store to serve cross-node downloads. Both
* implementations share credentials from {@code storage.s3.*} when set to {@code s3}.
*/
private String artifactStore = "local";
private Valkey valkey = new Valkey();
private Node node = new Node();
private transient String cachedNodeId;
public NodeRole resolvedRole() {
if (node == null || node.getRole() == null) {
return NodeRole.BOTH;
}
String value = node.getRole().trim().toUpperCase(Locale.ROOT);
try {
return NodeRole.valueOf(value);
} catch (IllegalArgumentException ex) {
return NodeRole.BOTH;
}
}
public synchronized String resolvedNodeId() {
if (node != null && node.getId() != null && !node.getId().isBlank()) {
return node.getId();
}
if (cachedNodeId == null) {
cachedNodeId = UUID.randomUUID().toString();
}
return cachedNodeId;
}
public enum NodeRole {
WEB,
WORKER,
BOTH
}
@Data
public static class Valkey {
/**
* {@code redis://host:6379} or {@code rediss://...} for TLS. Required when cluster mode
* is on and backplane is valkey.
*/
private String url = "";
private Tls tls = new Tls();
@Data
public static class Tls {
/**
* When {@code true}, skip Valkey/Redis TLS certificate verification (dev/test
* only). Leave {@code false} in production.
*/
private boolean skipCertVerification = false;
}
}
@Data
public static class Node {
/** Optional explicit node id. Blank = auto-generated UUID at startup. */
private String id = "";
/** {@code web} | {@code worker} | {@code both}. */
private String role = "both";
/**
* Internal cluster address advertised in the instance registry (host:port). Blank =
* derived at startup.
*/
private String internalAddress = "";
/** {@code http} | {@code https} - scheme used when peers call this node. */
private String scheme = "http";
/** Heartbeat publish interval for the instance registry, in milliseconds. */
private long heartbeatIntervalMs = 5000;
}
}
/**
* HTTP timeouts for loopback calls to internal Stirling API endpoints, used by the AI workflow
* executor and the pipeline processor. A bounded read timeout prevents a hung tool (e.g. an
@@ -426,6 +528,16 @@ public class ApplicationProperties {
private String provider;
private Client client = new Client();
/**
* When true, the OAuth2/OIDC login flow logs the full set of ID token and UserInfo
* claims at INFO level (and again at ERROR level if the username attribute cannot be
* resolved). Used to diagnose provider misconfiguration (for example ADFS not returning
* an {@code email} claim). WARNING: writes PII (sub, email, name) to application logs.
* Leave disabled in production; enable only while actively troubleshooting and disable
* again afterwards.
*/
private Boolean debugLogging = false;
public void setScopes(String scopes) {
List<String> scopesList =
Arrays.stream(scopes.split(",")).map(String::trim).toList();
@@ -676,6 +788,7 @@ public class ApplicationProperties {
private boolean enabled = false;
private String provider = "local";
private Local local = new Local();
private S3 s3 = new S3();
private Quotas quotas = new Quotas();
private Sharing sharing = new Sharing();
private Signing signing = new Signing();
@@ -685,6 +798,57 @@ public class ApplicationProperties {
private String basePath = InstallationPathConfig.getPath() + "storage";
}
@Data
public static class S3 {
/**
* Optional custom endpoint (e.g. {@code https://<account>.r2.cloudflarestorage.com},
* {@code https://<project>.supabase.co/storage/v1/s3}, or {@code http://localhost:9000}
* for MinIO). Blank = use AWS regional default.
*/
private String endpoint = "";
private String bucket = "";
private String region = "us-east-1";
private String accessKey = "";
private String secretKey = "";
/**
* When {@code true} use path-style URLs ({@code <endpoint>/<bucket>/<key>}) instead of
* virtual-hosted ({@code <bucket>.<endpoint>/<key>}). MinIO and most S3-compatible
* gateways require path-style; AWS S3 prefers virtual-hosted.
*/
private boolean pathStyleAccess = false;
/**
* When {@code false} (default), {@code endpoint} hostnames that resolve to private,
* loopback, or link-local addresses are rejected at startup to block SSRF attacks via
* the cloud metadata service (e.g. {@code http://169.254.169.254/}). Set to {@code
* true} to opt in for MinIO / in-cluster S3 endpoints on private networks.
*/
private boolean allowPrivateEndpoints = false;
/**
* Controls when the SDK adds an {@code x-amz-checksum-*} header on PUT/UploadPart.
* Default {@code WHEN_SUPPORTED} (the SDK default since 2.30) makes the SDK send a
* CRC32 checksum on every upload - this works on AWS S3, MinIO, current Supabase,
* Backblaze B2 (post-July-2025), and modern R2. Set to {@code WHEN_REQUIRED} to
* suppress the auto-checksum on vendors that reject unknown {@code x-amz-checksum-*}
* headers (older Backblaze B2, some R2 corner cases, GCS S3 endpoint). Invalid values
* fall back to {@code WHEN_SUPPORTED}.
*/
private String requestChecksumCalculation = "WHEN_SUPPORTED";
/**
* Controls when the SDK validates returned {@code x-amz-checksum-*} headers on GET
* responses. Default {@code WHEN_SUPPORTED}. Set to {@code WHEN_REQUIRED} if your
* vendor never returns these headers and you see false-positive checksum-mismatch
* errors. Invalid values fall back to {@code WHEN_SUPPORTED}.
*/
private String responseChecksumValidation = "WHEN_SUPPORTED";
}
@Data
public static class Sharing {
private boolean enabled = false;
@@ -1,15 +1,13 @@
package stirling.software.common.service;
import java.io.BufferedInputStream;
import java.io.BufferedOutputStream;
import java.io.ByteArrayInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.UUID;
import java.io.PipedInputStream;
import java.io.PipedOutputStream;
import java.util.concurrent.Executors;
import java.util.concurrent.atomic.AtomicReference;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.core.io.Resource;
import org.springframework.stereotype.Service;
import org.springframework.web.multipart.MultipartFile;
@@ -18,9 +16,11 @@ import org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBo
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import stirling.software.common.cluster.FileStore;
/**
* Service for storing and retrieving files with unique file IDs. Used by the AutoJobPostMapping
* system to handle file references.
* system to handle file references. Disk I/O is delegated to the injected {@link FileStore} bean.
*/
@Service
@RequiredArgsConstructor
@@ -30,251 +30,150 @@ public class FileStorage {
/** Holds the result of a stream-to-disk store operation: the file ID and the bytes written. */
public record StoredFile(String fileId, long size) {}
@Value("${stirling.tempDir:/tmp/stirling-files}")
private String tempDirPath;
private final FileOrUploadService fileOrUploadService;
private final FileStore fileStore;
/**
* Store a file and return its unique ID
*
* @param file The file to store
* @return The unique ID assigned to the file
* @throws IOException If there is an error storing the file
*/
public String storeFile(MultipartFile file) throws IOException {
String fileId = generateFileId();
Path filePath = getFilePath(fileId);
// Ensure the directory exists
Files.createDirectories(filePath.getParent());
// Transfer the file to the storage location
file.transferTo(filePath.toFile());
log.debug("Stored file with ID: {}", fileId);
return fileId;
}
/**
* Store a byte array as a file and return its unique ID
*
* @param bytes The byte array to store
* @param originalName The original name of the file (for extension)
* @return The unique ID assigned to the file
* @throws IOException If there is an error storing the file
*/
public String storeBytes(byte[] bytes, String originalName) throws IOException {
String fileId = generateFileId();
Path filePath = getFilePath(fileId);
// Ensure the directory exists
Files.createDirectories(filePath.getParent());
// Write the bytes to the file
Files.write(filePath, bytes);
log.debug("Stored byte array with ID: {}", fileId);
return fileId;
}
/**
* Retrieve a file by its ID as a MultipartFile
*
* @param fileId The ID of the file to retrieve
* @return The file as a MultipartFile
* @throws IOException If the file doesn't exist or can't be read
*/
public MultipartFile retrieveFile(String fileId) throws IOException {
Path filePath = getFilePath(fileId);
if (!Files.exists(filePath)) {
throw new IOException("File not found with ID: " + fileId);
// Fast path: when Spring buffered the multipart to disk (typical for large uploads), the
// backing Resource exposes a real File. Hand the Path to the FileStore so it can do a
// file-to-file copy (Linux sendfile, no copy through Java heap) rather than streaming
// the bytes through an 8K buffer. Falls back to the InputStream path for in-memory
// multiparts, exotic Resource impls, and anything that does not back onto a File.
Resource res;
try {
res = file.getResource();
} catch (RuntimeException ignored) {
res = null;
}
if (res != null && res.isFile()) {
try {
FileStore.Stored stored =
fileStore.store(res.getFile().toPath(), file.getOriginalFilename());
log.debug("Stored file with ID: {} (fast path)", stored.fileId());
return stored.fileId();
} catch (IOException ex) {
// Some Resource impls advertise isFile()=true but throw on getFile(); fall through.
log.debug("Resource fast path failed, falling back to stream copy", ex);
}
}
try (InputStream in = file.getInputStream()) {
FileStore.Stored stored = fileStore.store(in, file.getOriginalFilename());
log.debug("Stored file with ID: {}", stored.fileId());
return stored.fileId();
}
}
byte[] fileData = Files.readAllBytes(filePath);
public String storeBytes(byte[] bytes, String originalName) throws IOException {
FileStore.Stored stored = fileStore.store(new ByteArrayInputStream(bytes), originalName);
log.debug("Stored byte array with ID: {}", stored.fileId());
return stored.fileId();
}
public MultipartFile retrieveFile(String fileId) throws IOException {
byte[] fileData = fileStore.retrieveBytes(fileId);
return fileOrUploadService.toMockMultipartFile(fileId, fileData);
}
/**
* Retrieve a file by its ID as a byte array
*
* @param fileId The ID of the file to retrieve
* @return The file as a byte array
* @throws IOException If the file doesn't exist or can't be read
*/
public byte[] retrieveBytes(String fileId) throws IOException {
Path filePath = getFilePath(fileId);
if (!Files.exists(filePath)) {
throw new IOException("File not found with ID: " + fileId);
}
return Files.readAllBytes(filePath);
return fileStore.retrieveBytes(fileId);
}
/**
* Retrieve a file by its ID as a streaming InputStream. The caller is responsible for closing
* the returned stream.
*
* @param fileId The ID of the file to retrieve
* @return A buffered InputStream for the file
* @throws IOException If the file doesn't exist or can't be read
*/
public InputStream retrieveInputStream(String fileId) throws IOException {
Path filePath = getFilePath(fileId);
// Let Files.newInputStream throw NoSuchFileException naturally — avoids TOCTOU race
// between exists-check and open when another thread may delete concurrently.
return new BufferedInputStream(Files.newInputStream(filePath));
return fileStore.retrieve(fileId);
}
/**
* Store data from an InputStream as a file and return its unique ID and byte count. Streams
* directly to disk without buffering the entire content in heap.
*
* @param inputStream The input stream to read from
* @param originalName The original name of the file (unused, kept for API symmetry)
* @return A {@link StoredFile} containing the file ID and the number of bytes written
* @throws IOException If there is an error storing the file
*/
public StoredFile storeInputStream(InputStream inputStream, String originalName)
throws IOException {
String fileId = generateFileId();
Path filePath = getFilePath(fileId);
Files.createDirectories(filePath.getParent());
long size = Files.copy(inputStream, filePath);
log.debug("Stored input stream with ID: {}", fileId);
return new StoredFile(fileId, size);
FileStore.Stored stored = fileStore.store(inputStream, originalName);
log.debug("Stored input stream with ID: {}", stored.fileId());
return new StoredFile(stored.fileId(), stored.size());
}
public String storeFromStreamingBody(StreamingResponseBody body, String originalName)
throws IOException {
String fileId = generateFileId();
Path filePath = getFilePath(fileId);
Files.createDirectories(filePath.getParent());
boolean success = false;
try (OutputStream os = new BufferedOutputStream(Files.newOutputStream(filePath))) {
body.writeTo(os);
success = true;
} finally {
if (!success) {
// Hold Throwable not IOException: an unchecked failure (NPE, IllegalState, OOM, etc.)
// from the body writer would otherwise close the pipe with EOF and the consumer would
// return a truncated file with no error surfaced to the caller.
AtomicReference<Throwable> bodyError = new AtomicReference<>();
try (PipedOutputStream out = new PipedOutputStream();
PipedInputStream in = new PipedInputStream(out, 8192)) {
var executor = Executors.newSingleThreadExecutor(Thread.ofVirtual().factory());
java.util.concurrent.Future<?> task = null;
try {
task =
executor.submit(
() -> {
try {
body.writeTo(out);
} catch (Throwable ex) {
bodyError.set(ex);
} finally {
try {
out.close();
} catch (IOException ignored) {
// closed on the consumer side too
}
}
});
FileStore.Stored stored = fileStore.store(in, originalName);
Throwable writerErr = bodyError.get();
if (writerErr != null) {
// Body failed mid-write: the FileStore persisted a truncated entry.
// Best-effort delete so we don't leak partial files; never let cleanup
// mask the original writer error.
try {
fileStore.delete(stored.fileId());
} catch (RuntimeException cleanupEx) {
log.warn(
"Failed to delete partial file {} after writer error: {}",
stored.fileId(),
cleanupEx.getMessage());
}
if (writerErr instanceof IOException ioe) {
throw ioe;
}
throw new IOException(
"StreamingResponseBody writer failed: " + writerErr.getMessage(),
writerErr);
}
log.debug("Stored StreamingResponseBody with ID: {}", stored.fileId());
return stored.fileId();
} finally {
// Interrupt and join the writer task: shutdown() alone returns immediately and a
// failed store leaves the writer running, leaking a thread per failed upload.
if (task != null && !task.isDone()) {
task.cancel(true);
}
executor.shutdown();
try {
Files.deleteIfExists(filePath);
} catch (IOException cleanupEx) {
log.warn(
"Failed to clean up partial file {} after store failure",
filePath,
cleanupEx);
if (!executor.awaitTermination(5, java.util.concurrent.TimeUnit.SECONDS)) {
executor.shutdownNow();
}
} catch (InterruptedException ie) {
executor.shutdownNow();
Thread.currentThread().interrupt();
}
}
}
log.debug("Stored StreamingResponseBody with ID: {}", fileId);
return fileId;
}
/**
* Persist a {@link Resource} body to disk, returning the generated file ID. Used by the async
* job pipeline to capture {@code ResponseEntity<Resource>} results produced by controllers.
*/
public String storeFromResource(Resource resource, String originalName) throws IOException {
String fileId = generateFileId();
Path filePath = getFilePath(fileId);
Files.createDirectories(filePath.getParent());
boolean success = false;
try (InputStream in = resource.getInputStream()) {
Files.copy(in, filePath);
success = true;
} finally {
if (!success) {
try {
Files.deleteIfExists(filePath);
} catch (IOException cleanupEx) {
log.warn(
"Failed to clean up partial file {} after store failure",
filePath,
cleanupEx);
}
}
FileStore.Stored stored = fileStore.store(in, originalName);
log.debug("Stored Resource with ID: {}", stored.fileId());
return stored.fileId();
}
log.debug("Stored Resource with ID: {}", fileId);
return fileId;
}
/**
* Delete a file by its ID
*
* @param fileId The ID of the file to delete
* @return true if the file was deleted, false otherwise
*/
public boolean deleteFile(String fileId) {
try {
Path filePath = getFilePath(fileId);
return Files.deleteIfExists(filePath);
} catch (IOException e) {
log.error("Error deleting file with ID: {}", fileId, e);
return false;
}
return fileStore.delete(fileId);
}
/**
* Check if a file exists by its ID
*
* @param fileId The ID of the file to check
* @return true if the file exists, false otherwise
*/
public boolean fileExists(String fileId) {
Path filePath = getFilePath(fileId);
return Files.exists(filePath);
return fileStore.exists(fileId);
}
/**
* Get the size of a file by its ID without loading the content into memory
*
* @param fileId The ID of the file
* @return The size of the file in bytes
* @throws IOException If the file doesn't exist or can't be read
*/
public long getFileSize(String fileId) throws IOException {
Path filePath = getFilePath(fileId);
if (!Files.exists(filePath)) {
throw new IOException("File not found with ID: " + fileId);
}
return Files.size(filePath);
}
/**
* Get the path for a file ID
*
* @param fileId The ID of the file
* @return The path to the file
* @throws IllegalArgumentException if fileId contains path traversal characters or resolves
* outside base directory
*/
private Path getFilePath(String fileId) {
// Validate fileId to prevent path traversal
if (fileId.contains("..") || fileId.contains("/") || fileId.contains("\\")) {
throw new IllegalArgumentException("Invalid file ID");
}
Path basePath = Path.of(tempDirPath).normalize().toAbsolutePath();
Path resolvedPath = basePath.resolve(fileId).normalize();
// Ensure resolved path is within the base directory
if (!resolvedPath.startsWith(basePath)) {
throw new IllegalArgumentException("File ID resolves to an invalid path");
}
return resolvedPath;
}
/**
* Generate a unique file ID
*
* @return A unique file ID
*/
private String generateFileId() {
return UUID.randomUUID().toString();
return fileStore.size(fileId);
}
}
@@ -3,9 +3,13 @@ package stirling.software.common.service;
import java.io.BufferedInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.time.Duration;
import java.time.Instant;
import java.time.LocalDateTime;
import java.time.ZoneId;
import java.time.temporal.ChronoUnit;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Locale;
import java.util.Map;
@@ -16,6 +20,7 @@ import java.util.concurrent.TimeUnit;
import java.util.zip.ZipEntry;
import java.util.zip.ZipInputStream;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.http.MediaType;
import org.springframework.stereotype.Service;
@@ -26,6 +31,10 @@ import jakarta.annotation.PreDestroy;
import lombok.extern.slf4j.Slf4j;
import stirling.software.common.cluster.ClusterBackplane;
import stirling.software.common.cluster.JobStore;
import stirling.software.common.cluster.JobStoreEntry;
import stirling.software.common.cluster.JobStoreEntry.JobState;
import stirling.software.common.model.job.JobResult;
import stirling.software.common.model.job.JobStats;
import stirling.software.common.model.job.ResultFile;
@@ -40,20 +49,20 @@ public class TaskManager {
private int jobResultExpiryMinutes = 30;
private final FileStorage fileStorage;
private final JobStore jobStore;
private final ClusterBackplane clusterBackplane;
private final ScheduledExecutorService cleanupExecutor =
Executors.newSingleThreadScheduledExecutor(
Thread.ofVirtual().name("task-cleanup-", 0).factory());
/** Initialize the task manager and start the cleanup scheduler */
public TaskManager(FileStorage fileStorage) {
@Autowired
public TaskManager(
FileStorage fileStorage, JobStore jobStore, ClusterBackplane clusterBackplane) {
this.fileStorage = fileStorage;
this.jobStore = jobStore;
this.clusterBackplane = clusterBackplane;
// Schedule periodic cleanup of old job results
cleanupExecutor.scheduleAtFixedRate(
this::cleanupOldJobs,
10, // Initial delay
10, // Interval
TimeUnit.MINUTES);
cleanupExecutor.scheduleAtFixedRate(this::cleanupOldJobs, 10, 10, TimeUnit.MINUTES);
log.debug(
"Task manager initialized with job result expiry of {} minutes",
@@ -66,7 +75,9 @@ public class TaskManager {
* @param jobId The job ID
*/
public void createTask(String jobId) {
jobResults.put(jobId, JobResult.createNew(jobId));
JobResult result = JobResult.createNew(jobId);
jobResults.put(jobId, result);
writeThrough(jobId, result);
log.debug("Created task with job ID: {}", jobId);
}
@@ -79,6 +90,7 @@ public class TaskManager {
public void setResult(String jobId, Object result) {
JobResult jobResult = getOrCreateJobResult(jobId);
jobResult.completeWithResult(result);
writeThrough(jobId, jobResult);
log.debug("Set result for job ID: {}", jobId);
}
@@ -101,6 +113,7 @@ public class TaskManager {
extractZipToIndividualFiles(fileId, originalFileName);
if (!extractedFiles.isEmpty()) {
jobResult.completeWithFiles(extractedFiles);
writeThrough(jobId, jobResult);
log.debug(
"Set multiple file results for job ID: {} with {} files extracted from"
+ " ZIP",
@@ -127,6 +140,7 @@ public class TaskManager {
"Failed to get file size for job {}: {}. Using size 0.", jobId, e.getMessage());
jobResult.completeWithSingleFile(fileId, originalFileName, contentType, 0);
}
writeThrough(jobId, jobResult);
}
/**
@@ -138,6 +152,7 @@ public class TaskManager {
public void setMultipleFileResults(String jobId, List<ResultFile> resultFiles) {
JobResult jobResult = getOrCreateJobResult(jobId);
jobResult.completeWithFiles(resultFiles);
writeThrough(jobId, jobResult);
log.debug(
"Set multiple file results for job ID: {} with {} files",
jobId,
@@ -153,6 +168,7 @@ public class TaskManager {
public void setError(String jobId, String error) {
JobResult jobResult = getOrCreateJobResult(jobId);
jobResult.failWithError(error);
writeThrough(jobId, jobResult);
log.debug("Set error for job ID: {}: {}", jobId, error);
}
@@ -169,6 +185,7 @@ public class TaskManager {
// If no result or error has been set, mark it as complete with an empty result
jobResult.completeWithResult("Task completed successfully");
}
writeThrough(jobId, jobResult);
log.debug("Marked job ID: {} as complete", jobId);
}
@@ -205,6 +222,7 @@ public class TaskManager {
JobResult jobResult = jobResults.get(jobId);
if (jobResult != null) {
jobResult.addNote(note);
writeThrough(jobId, jobResult);
log.debug("Added note to job ID: {}: {}", jobId, note);
return true;
}
@@ -295,8 +313,11 @@ public class TaskManager {
return jobResults.computeIfAbsent(jobId, JobResult::createNew);
}
/** Clean up old completed job results */
/** Clean up old completed job results. No-op in cluster mode; the backplane TTL owns expiry. */
public void cleanupOldJobs() {
if (clusterBackplane != null && !clusterBackplane.shouldRunLocalCleanup()) {
return;
}
LocalDateTime expiryThreshold =
LocalDateTime.now().minus(jobResultExpiryMinutes, ChronoUnit.MINUTES);
int removedCount = 0;
@@ -315,6 +336,9 @@ public class TaskManager {
// Remove the job result
jobResults.remove(entry.getKey());
if (jobStore != null) {
jobStore.delete(entry.getKey());
}
removedCount++;
}
}
@@ -327,6 +351,53 @@ public class TaskManager {
}
}
/** Mirror the in-memory {@code JobResult} into the cluster-visible {@link JobStore}. */
private void writeThrough(String jobId, JobResult result) {
if (jobStore == null) {
return;
}
try {
jobStore.put(toEntry(jobId, result), Duration.ofMinutes(jobResultExpiryMinutes));
} catch (RuntimeException ex) {
log.warn("JobStore write-through failed for job {}: {}", jobId, ex.getMessage());
}
}
private JobStoreEntry toEntry(String jobId, JobResult result) {
JobState state;
if (result.isComplete()) {
state = result.getError() != null ? JobState.FAILED : JobState.COMPLETE;
} else {
state = JobState.PENDING;
}
Instant createdAt = toInstant(result.getCreatedAt());
Instant completedAt = toInstant(result.getCompletedAt());
List<String> fileIds = new ArrayList<>();
if (result.hasFiles()) {
for (ResultFile rf : result.getAllResultFiles()) {
fileIds.add(rf.getFileId());
}
}
Map<String, String> meta = new HashMap<>();
if (result.getNotes() != null && !result.getNotes().isEmpty()) {
meta.put("notesCount", Integer.toString(result.getNotes().size()));
}
String owningNodeId = clusterBackplane == null ? "local" : clusterBackplane.localNodeId();
return new JobStoreEntry(
jobId,
state,
owningNodeId,
createdAt,
completedAt,
result.getError(),
fileIds,
meta);
}
private Instant toInstant(LocalDateTime ldt) {
return ldt == null ? null : ldt.atZone(ZoneId.systemDefault()).toInstant();
}
/** Shutdown the cleanup executor */
@PreDestroy
public void shutdown() {
@@ -370,7 +441,7 @@ public class TaskManager {
while ((entry = zipIn.getNextEntry()) != null) {
if (!entry.isDirectory()) {
String contentType = determineContentType(entry.getName());
// storeInputStream returns the fileId and byte count — no extra stat needed
// storeInputStream returns the fileId and byte count - no extra stat needed
FileStorage.StoredFile stored =
fileStorage.storeInputStream(zipIn, entry.getName());
@@ -458,7 +529,8 @@ public class TaskManager {
}
/**
* Find the job key that owns a given file ID.
* Find the job key that owns a given file ID. Checks the local in-memory map first, then falls
* back to the cluster-visible {@link JobStore}.
*
* @param fileId file identifier to look up
* @return scoped job key if found, otherwise null
@@ -474,6 +546,18 @@ public class TaskManager {
}
}
}
if (jobStore != null) {
// Propagate JobStore failures: returning null on a backplane outage would conflate
// "no such file" with "lookup unavailable" and the caller would respond 404 to a
// transient blip that should be retried. Let Spring's exception handler surface a
// 5xx so clients know to retry.
try {
return jobStore.findJobIdByFileId(fileId).orElse(null);
} catch (RuntimeException e) {
log.warn("JobStore findJobIdByFileId failed for {}: {}", fileId, e.getMessage());
throw e;
}
}
return null;
}
}
@@ -83,7 +83,16 @@ public class RequestUriUtils {
return false;
}
// Blocklist of backend/non-frontend paths that should still go through filters
// Blocklist of backend/non-frontend paths that should still go through filters.
//
// `/files` was historically a backend route; it is now a frontend route
// owned by HomePage / FileManagerView. Direct-nav or refresh on /files
// (or /files/<folder-uuid>) was returning the Spring auth filter's 401
// JSON instead of serving index.html, so the SPA never got a chance to
// mount and the user saw a raw error response. There are no `/files`
// backend mappings at the servlet root - the real storage endpoints
// live under `/api/v1/storage/files`, which is filtered out a few lines
// up by the `startsWith("/api/")` guard.
String[] backendOnlyPrefixes = {
"/register",
"/pipeline",
@@ -91,7 +100,6 @@ public class RequestUriUtils {
"/pdfjs-legacy",
"/fonts",
"/images",
"/files",
"/css",
"/js",
"/swagger",
@@ -172,7 +180,7 @@ public class RequestUriUtils {
"/api/v1/ui-data/footer-info") // Public footer configuration
|| trimmedUri.startsWith("/api/v1/invite/validate")
|| trimmedUri.startsWith("/api/v1/invite/accept")
// Health Endoints
// Health Endpoints
|| trimmedUri.startsWith("/actuator/health")
|| trimmedUri.startsWith("/health")
|| trimmedUri.startsWith("/healthz")
@@ -181,7 +189,7 @@ public class RequestUriUtils {
|| trimmedUri.startsWith(
"/api/v1/mobile-scanner/") // Mobile scanner endpoints (no auth)
|| trimmedUri.startsWith("/v1/api-docs")
// Workflow participant endpoints — access controlled by share tokens, not login
// Workflow participant endpoints - access controlled by share tokens, not login
|| trimmedUri.startsWith("/api/v1/workflow/participant/")
// Share-link SPA bootstrap; data APIs remain protected
|| trimmedUri.matches("^/share/[^/]+/?$");
@@ -56,4 +56,17 @@ class ArchitectureTest {
.resideInAPackage("stirling.software.saas..");
rule.check(commonClasses);
}
@Test
void clusterInterfacesHaveNoImplementationDependencies() {
ArchRule rule =
noClasses()
.that()
.resideInAPackage("stirling.software.common.cluster..")
.should()
.dependOnClassesThat()
.resideInAnyPackage(
"stirling.software.proprietary..", "stirling.software.saas..");
rule.check(commonClasses);
}
}
@@ -0,0 +1,51 @@
package stirling.software.common.cluster;
import static org.junit.jupiter.api.Assertions.assertEquals;
import java.time.Instant;
import java.util.List;
import java.util.Map;
import org.junit.jupiter.api.Test;
class BackplaneContractCompilationTest {
@Test
void jobStoreEntryRecordRoundTrips() {
Instant now = Instant.now();
JobStoreEntry entry =
new JobStoreEntry(
"job-1",
JobStoreEntry.JobState.PENDING,
"node-a",
now,
null,
null,
List.of("file-1"),
Map.of("k", "v"));
assertEquals("job-1", entry.jobId());
assertEquals(JobStoreEntry.JobState.PENDING, entry.state());
assertEquals("node-a", entry.owningNodeId());
assertEquals(now, entry.createdAt());
assertEquals(List.of("file-1"), entry.fileIds());
assertEquals("v", entry.resultMeta().get("k"));
}
@Test
void clusterNodeRecordRoundTrips() {
Instant heartbeat = Instant.now();
ClusterNode node = new ClusterNode("node-a", "10.0.0.1:8080", heartbeat, "BOTH");
assertEquals("node-a", node.nodeId());
assertEquals("10.0.0.1:8080", node.internalAddress());
assertEquals(heartbeat, node.lastHeartbeat());
assertEquals("BOTH", node.role());
}
@Test
void rateLimitDecisionRecordRoundTrips() {
RateLimitStore.RateLimitDecision d = new RateLimitStore.RateLimitDecision(true, 7, 0L);
assertEquals(true, d.allowed());
assertEquals(7, d.remainingTokens());
assertEquals(0L, d.nanosToWaitForRefill());
}
}
@@ -0,0 +1,65 @@
package stirling.software.common.cluster;
import static org.junit.jupiter.api.Assertions.assertDoesNotThrow;
import static org.junit.jupiter.api.Assertions.assertThrows;
import java.lang.reflect.Method;
import org.junit.jupiter.api.Test;
import stirling.software.common.model.ApplicationProperties;
import stirling.software.common.model.ApplicationProperties.Cluster;
class ClusterConfigValidationTest {
@Test
void validationPassesWhenDisabled() {
ApplicationProperties props = new ApplicationProperties();
ClusterConfig config = new ClusterConfig(props);
assertDoesNotThrow(() -> invokeValidate(config));
}
@Test
void validationFailsWhenValkeyEnabledWithoutUrl() {
ApplicationProperties props = new ApplicationProperties();
Cluster cluster = props.getCluster();
cluster.setEnabled(true);
cluster.setBackplane("valkey");
ClusterConfig config = new ClusterConfig(props);
assertThrows(IllegalStateException.class, () -> invokeValidate(config));
}
@Test
void validationPassesWhenValkeyEnabledWithUrl() {
ApplicationProperties props = new ApplicationProperties();
Cluster cluster = props.getCluster();
cluster.setEnabled(true);
cluster.setBackplane("valkey");
cluster.getValkey().setUrl("redis://localhost:6379");
ClusterConfig config = new ClusterConfig(props);
assertDoesNotThrow(() -> invokeValidate(config));
}
@Test
void validationPassesWhenInProcessEnabled() {
ApplicationProperties props = new ApplicationProperties();
Cluster cluster = props.getCluster();
cluster.setEnabled(true);
cluster.setBackplane("inprocess");
ClusterConfig config = new ClusterConfig(props);
assertDoesNotThrow(() -> invokeValidate(config));
}
private void invokeValidate(ClusterConfig config) throws Exception {
Method m = ClusterConfig.class.getDeclaredMethod("validate");
m.setAccessible(true);
try {
m.invoke(config);
} catch (java.lang.reflect.InvocationTargetException ex) {
if (ex.getCause() instanceof RuntimeException re) {
throw re;
}
throw ex;
}
}
}
@@ -0,0 +1,62 @@
package stirling.software.common.cluster;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import org.junit.jupiter.api.Test;
import stirling.software.common.model.ApplicationProperties;
import stirling.software.common.model.ApplicationProperties.Cluster;
class ClusterPropertiesTest {
@Test
void defaultsAreDisabledAndInprocess() {
Cluster props = new ApplicationProperties().getCluster();
assertFalse(props.isEnabled());
assertEquals("inprocess", props.getBackplane());
assertEquals("local", props.getArtifactStore());
assertEquals(Cluster.NodeRole.BOTH, props.resolvedRole());
assertEquals("", props.getValkey().getUrl());
assertFalse(props.getValkey().getTls().isSkipCertVerification());
assertEquals("both", props.getNode().getRole());
assertEquals("http", props.getNode().getScheme());
assertEquals(5000L, props.getNode().getHeartbeatIntervalMs());
}
@Test
void resolvedRoleParsesCaseInsensitively() {
Cluster props = new ApplicationProperties().getCluster();
props.getNode().setRole("WEB");
assertEquals(Cluster.NodeRole.WEB, props.resolvedRole());
props.getNode().setRole("web");
assertEquals(Cluster.NodeRole.WEB, props.resolvedRole());
props.getNode().setRole("Worker");
assertEquals(Cluster.NodeRole.WORKER, props.resolvedRole());
props.getNode().setRole("garbage");
assertEquals(Cluster.NodeRole.BOTH, props.resolvedRole());
props.getNode().setRole(null);
assertEquals(Cluster.NodeRole.BOTH, props.resolvedRole());
}
@Test
void resolvedNodeIdIsStableAcrossCalls() {
Cluster props = new ApplicationProperties().getCluster();
String first = props.resolvedNodeId();
String second = props.resolvedNodeId();
assertNotNull(first);
assertEquals(first, second);
}
@Test
void resolvedNodeIdHonoursExplicitId() {
Cluster props = new ApplicationProperties().getCluster();
props.getNode().setId("abc");
assertEquals("abc", props.resolvedNodeId());
}
}
@@ -0,0 +1,90 @@
package stirling.software.common.cluster;
import static org.assertj.core.api.Assertions.assertThat;
import org.junit.jupiter.api.Test;
import org.springframework.boot.autoconfigure.context.PropertyPlaceholderAutoConfiguration;
import org.springframework.boot.test.context.runner.ApplicationContextRunner;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import stirling.software.common.cluster.inprocess.InProcessClusterConfiguration;
import stirling.software.common.model.ApplicationProperties;
/**
* Verifies the {@link InProcessClusterConfiguration} conditional wiring: in-process beans wire when
* cluster mode is off or {@code backplane=inprocess}, and are skipped when {@code
* backplane=valkey}.
*/
class InProcessConfigurationConditionalTest {
private final ApplicationContextRunner runner =
new ApplicationContextRunner()
.withConfiguration(
org.springframework.boot.autoconfigure.AutoConfigurations.of(
PropertyPlaceholderAutoConfiguration.class))
.withUserConfiguration(
TestAppPropertiesConfig.class,
ClusterConfig.class,
InProcessClusterConfiguration.class);
@Test
void inProcessBeansWireWhenClusterDisabled() {
runner.run(
context ->
assertThat(context)
.hasNotFailed()
.hasSingleBean(ClusterBackplane.class)
.hasSingleBean(JobStore.class)
.hasSingleBean(RateLimitStore.class)
.hasSingleBean(DistributedLock.class)
.hasSingleBean(KeyValueCache.class)
.hasSingleBean(InstanceRegistry.class));
}
@Test
void inProcessBeansWireWhenEnabledWithInProcessBackplane() {
runner.withPropertyValues("cluster.enabled=true", "cluster.backplane=inprocess")
.run(
context ->
assertThat(context)
.hasNotFailed()
.hasSingleBean(ClusterBackplane.class)
.hasSingleBean(JobStore.class)
.hasSingleBean(RateLimitStore.class)
.hasSingleBean(DistributedLock.class)
.hasSingleBean(KeyValueCache.class)
.hasSingleBean(InstanceRegistry.class));
}
@Test
void inProcessBeansSkippedWhenEnabledWithDistributedBackplane() {
runner.withPropertyValues(
"cluster.enabled=true",
"cluster.backplane=valkey",
"cluster.valkey.url=redis://localhost:6379")
.run(
context ->
assertThat(context)
.hasNotFailed()
.doesNotHaveBean(ClusterBackplane.class)
.doesNotHaveBean(JobStore.class)
.doesNotHaveBean(RateLimitStore.class)
.doesNotHaveBean(DistributedLock.class)
.doesNotHaveBean(KeyValueCache.class)
.doesNotHaveBean(InstanceRegistry.class));
}
/**
* Hand-rolled {@link ApplicationProperties} bean: the production class loads YAML at startup
* via a {@code @PostConstruct} hook that isn't appropriate for the slice runner, so we wire a
* defaults-only instance here.
*/
@Configuration
static class TestAppPropertiesConfig {
@Bean
ApplicationProperties applicationProperties() {
return new ApplicationProperties();
}
}
}
@@ -0,0 +1,167 @@
package stirling.software.common.cluster.inprocess;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertTrue;
import java.time.Duration;
import java.util.Optional;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicBoolean;
import org.junit.jupiter.api.Test;
import stirling.software.common.cluster.DistributedLock;
class InProcessDistributedLockTest {
@Test
void acquireReleaseAcquire() {
DistributedLock lock = new InProcessDistributedLock();
DistributedLock.LockHandle h1 = lock.tryAcquire("k", Duration.ofSeconds(30)).orElseThrow();
h1.release();
assertTrue(lock.tryAcquire("k", Duration.ofSeconds(30)).isPresent());
}
@Test
void reentryFromSameThreadFails() {
DistributedLock lock = new InProcessDistributedLock();
DistributedLock.LockHandle h1 = lock.tryAcquire("k", Duration.ofSeconds(30)).orElseThrow();
Optional<DistributedLock.LockHandle> reentry = lock.tryAcquire("k", Duration.ofSeconds(30));
assertFalse(reentry.isPresent(), "in-process lock must be non-reentrant");
h1.release();
// After release, anyone can acquire again.
assertTrue(lock.tryAcquire("k", Duration.ofSeconds(30)).isPresent());
}
@Test
void secondAcquireFromAnotherThreadFails() throws InterruptedException {
DistributedLock lock = new InProcessDistributedLock();
DistributedLock.LockHandle h1 = lock.tryAcquire("k", Duration.ofSeconds(30)).orElseThrow();
CountDownLatch done = new CountDownLatch(1);
AtomicBoolean acquired = new AtomicBoolean(true);
Thread t =
new Thread(
() -> {
Optional<DistributedLock.LockHandle> attempt =
lock.tryAcquire("k", Duration.ofSeconds(30));
acquired.set(attempt.isPresent());
attempt.ifPresent(DistributedLock.LockHandle::release);
done.countDown();
});
t.start();
assertTrue(done.await(2, TimeUnit.SECONDS));
assertFalse(acquired.get());
h1.release();
}
@Test
void leaseExpiryAllowsTakeoverEvenWithoutRelease() throws InterruptedException {
// Acquire with a short lease, never call release, then try to acquire again after the
// lease has elapsed. Matches Redis SET-NX-EX semantics - the second caller gets the lock
// because the first lease auto-expired. 250ms lease + 350ms wait gives CI generous slack.
DistributedLock lock = new InProcessDistributedLock();
DistributedLock.LockHandle h1 = lock.tryAcquire("k", Duration.ofMillis(250)).orElseThrow();
Thread.sleep(350);
Optional<DistributedLock.LockHandle> takeover =
lock.tryAcquire("k", Duration.ofSeconds(30));
assertTrue(
takeover.isPresent(),
"expired lease must release the lock so a new caller can take over");
// Calling release() on the original handle after takeover must be a no-op (token check).
h1.release();
// The takeover holder is still the legitimate owner.
assertFalse(lock.tryAcquire("k", Duration.ofSeconds(30)).isPresent());
takeover.get().release();
}
@Test
void renewExtendsLease() throws InterruptedException {
// Acquire with a short lease, renew it before it expires, then verify the lock is still
// held past the original expiry point. 200ms initial + renew to 2s + wait 350ms.
DistributedLock lock = new InProcessDistributedLock();
DistributedLock.LockHandle h1 = lock.tryAcquire("k", Duration.ofMillis(200)).orElseThrow();
assertTrue(h1.renew(Duration.ofSeconds(2)), "renew on a held lease must succeed");
Thread.sleep(350);
assertFalse(
lock.tryAcquire("k", Duration.ofSeconds(30)).isPresent(),
"renew should have pushed expiry well past the original 200ms");
h1.release();
}
@Test
void renewAfterReleaseFails() {
DistributedLock lock = new InProcessDistributedLock();
DistributedLock.LockHandle h1 = lock.tryAcquire("k", Duration.ofSeconds(30)).orElseThrow();
h1.release();
assertFalse(h1.renew(Duration.ofSeconds(30)), "renew on a released handle must fail");
}
/**
* Concurrency stress: many threads contending on the same key with each holder respecting the
* lease (hold &lt;&lt; lease). The lock behaves as a strict mutex in this regime so asserting
* mutual exclusion is meaningful. A separate test ({@link
* #leaseExpiryAllowsTakeoverEvenWithoutRelease}) covers the takeover-across-expiry branch,
* which legitimately allows two holders momentarily and is split-brain behaviour inherent to
* any lease-based lock.
*/
@Test
void concurrentContentionPreservesMutualExclusion() throws InterruptedException {
DistributedLock lock = new InProcessDistributedLock();
int threads = 16;
int attemptsPerThread = 200;
// Lease far exceeds any plausible hold time, so the takeover branch never triggers in
// this test and the lock acts as a strict mutex.
Duration lease = Duration.ofSeconds(5);
java.util.concurrent.atomic.AtomicInteger concurrentHolders =
new java.util.concurrent.atomic.AtomicInteger();
java.util.concurrent.atomic.AtomicInteger maxConcurrent =
new java.util.concurrent.atomic.AtomicInteger();
java.util.concurrent.atomic.AtomicInteger acquires =
new java.util.concurrent.atomic.AtomicInteger();
java.util.concurrent.atomic.AtomicReference<Throwable> firstFailure =
new java.util.concurrent.atomic.AtomicReference<>();
CountDownLatch start = new CountDownLatch(1);
CountDownLatch done = new CountDownLatch(threads);
for (int i = 0; i < threads; i++) {
new Thread(
() -> {
try {
start.await();
for (int j = 0; j < attemptsPerThread; j++) {
Optional<DistributedLock.LockHandle> h =
lock.tryAcquire("hot", lease);
if (h.isPresent()) {
int now = concurrentHolders.incrementAndGet();
maxConcurrent.accumulateAndGet(now, Math::max);
acquires.incrementAndGet();
// Trivial critical section; well within lease.
concurrentHolders.decrementAndGet();
h.get().release();
}
}
} catch (Throwable t) {
firstFailure.compareAndSet(null, t);
} finally {
done.countDown();
}
},
"lock-stress-" + i)
.start();
}
start.countDown();
assertTrue(done.await(30, TimeUnit.SECONDS), "stress workers must finish in time");
org.junit.jupiter.api.Assertions.assertNull(firstFailure.get(), "no worker may throw");
org.junit.jupiter.api.Assertions.assertEquals(
1,
maxConcurrent.get(),
"mutual exclusion violated: more than one holder observed simultaneously");
assertTrue(
acquires.get() > 0,
"at least some acquires must succeed under contention (saw "
+ acquires.get()
+ ")");
}
}
@@ -0,0 +1,28 @@
package stirling.software.common.cluster.inprocess;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertTrue;
import java.time.Duration;
import java.time.Instant;
import org.junit.jupiter.api.Test;
import stirling.software.common.cluster.ClusterNode;
class InProcessInstanceRegistryTest {
@Test
void registerThenLookupAndActiveNodes() {
InProcessInstanceRegistry registry = new InProcessInstanceRegistry();
ClusterNode node = new ClusterNode("node-1", "127.0.0.1:8080", Instant.now(), "BOTH");
registry.register(node, Duration.ofSeconds(30));
assertTrue(registry.lookup("node-1").isPresent());
assertEquals("node-1", registry.lookup("node-1").get().nodeId());
assertEquals(1, registry.activeNodes().size());
registry.deregister("node-1");
assertTrue(registry.lookup("node-1").isEmpty());
}
}
@@ -0,0 +1,91 @@
package stirling.software.common.cluster.inprocess;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertTrue;
import java.time.Duration;
import java.time.Instant;
import java.util.List;
import java.util.Map;
import org.junit.jupiter.api.Test;
import stirling.software.common.cluster.JobStoreEntry;
class InProcessJobStoreTest {
private final InProcessJobStore store = new InProcessJobStore();
@Test
void putGetDeleteExistsRoundTrip() {
JobStoreEntry entry = entry("job-1");
store.put(entry, Duration.ofMinutes(30));
assertTrue(store.exists("job-1"));
assertEquals(entry, store.get("job-1").orElseThrow());
store.delete("job-1");
assertFalse(store.exists("job-1"));
}
@Test
void ttlExpiry() throws InterruptedException {
store.put(entry("job-2"), Duration.ofMillis(50));
Thread.sleep(100);
assertFalse(store.get("job-2").isPresent());
}
@Test
void purgeExpiredRemovesOnlyStaleEntries() throws InterruptedException {
store.put(entry("job-fresh"), Duration.ofMinutes(30));
store.put(entry("job-stale"), Duration.ofMillis(20));
Thread.sleep(80);
int removed = store.purgeExpired();
assertEquals(1, removed);
assertTrue(store.exists("job-fresh"));
}
@Test
void findJobIdByFileIdReturnsTheRightJob() {
store.put(
new JobStoreEntry(
"job-a",
JobStoreEntry.JobState.COMPLETE,
"node-1",
Instant.now(),
Instant.now(),
null,
List.of("file-1", "file-2"),
Map.of()),
Duration.ofMinutes(30));
store.put(
new JobStoreEntry(
"job-b",
JobStoreEntry.JobState.COMPLETE,
"node-1",
Instant.now(),
Instant.now(),
null,
List.of("file-3"),
Map.of()),
Duration.ofMinutes(30));
assertEquals("job-a", store.findJobIdByFileId("file-1").orElseThrow());
assertEquals("job-b", store.findJobIdByFileId("file-3").orElseThrow());
assertFalse(store.findJobIdByFileId("missing").isPresent());
}
private JobStoreEntry entry(String id) {
return new JobStoreEntry(
id,
JobStoreEntry.JobState.PENDING,
"node-1",
Instant.now(),
null,
null,
List.of(),
Map.of());
}
}
@@ -0,0 +1,41 @@
package stirling.software.common.cluster.inprocess;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import java.time.Duration;
import org.junit.jupiter.api.Test;
import stirling.software.common.cluster.KeyValueCache;
class InProcessKeyValueCacheTest {
@Test
void putGetEvict() {
KeyValueCache cache = new InProcessKeyValueCache();
cache.put("apikey", "a", "userA", Duration.ofMinutes(1));
assertEquals("userA", cache.get("apikey", "a").orElseThrow());
cache.evict("apikey", "a");
assertFalse(cache.get("apikey", "a").isPresent());
}
@Test
void ttlExpiry() throws InterruptedException {
KeyValueCache cache = new InProcessKeyValueCache();
cache.put("ns", "k", "v", Duration.ofMillis(40));
Thread.sleep(80);
assertFalse(cache.get("ns", "k").isPresent());
}
@Test
void evictNamespace() {
KeyValueCache cache = new InProcessKeyValueCache();
cache.put("ns", "a", "1", Duration.ofMinutes(1));
cache.put("ns", "b", "2", Duration.ofMinutes(1));
cache.evictNamespace("ns");
assertFalse(cache.get("ns", "a").isPresent());
assertFalse(cache.get("ns", "b").isPresent());
}
}
@@ -0,0 +1,56 @@
package stirling.software.common.cluster.inprocess;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertTrue;
import java.time.Duration;
import org.junit.jupiter.api.Test;
import stirling.software.common.cluster.RateLimitStore;
import stirling.software.common.cluster.RateLimitStore.RateLimitDecision;
class InProcessRateLimitStoreTest {
@Test
void firstNConsumesAllowed() {
RateLimitStore store = new InProcessRateLimitStore();
for (int i = 0; i < 5; i++) {
assertTrue(store.tryConsume("k", 5, Duration.ofSeconds(60)).allowed(), "i=" + i);
}
assertFalse(store.tryConsume("k", 5, Duration.ofSeconds(60)).allowed());
}
@Test
void remainingTokensDecrements() {
RateLimitStore store = new InProcessRateLimitStore();
RateLimitDecision d1 = store.tryConsume("k", 5, Duration.ofSeconds(60));
RateLimitDecision d2 = store.tryConsume("k", 5, Duration.ofSeconds(60));
assertTrue(d1.allowed());
assertTrue(d2.allowed());
assertEquals(4, d1.remainingTokens());
assertEquals(3, d2.remainingTokens());
}
@Test
void refillRestoresTokens() throws InterruptedException {
RateLimitStore store = new InProcessRateLimitStore();
// Capacity 2 with smooth refill over 100 ms -> ~1 token per 50 ms.
for (int i = 0; i < 2; i++) {
assertTrue(store.tryConsume("k", 2, Duration.ofMillis(100)).allowed());
}
assertFalse(store.tryConsume("k", 2, Duration.ofMillis(100)).allowed());
Thread.sleep(150);
assertTrue(store.tryConsume("k", 2, Duration.ofMillis(100)).allowed());
}
@Test
void deniedConsumeReportsWaitNanos() {
RateLimitStore store = new InProcessRateLimitStore();
assertTrue(store.tryConsume("wait", 1, Duration.ofSeconds(10)).allowed());
RateLimitDecision denied = store.tryConsume("wait", 1, Duration.ofSeconds(10));
assertFalse(denied.allowed());
assertTrue(denied.nanosToWaitForRefill() > 0L);
}
}
@@ -0,0 +1,42 @@
package stirling.software.common.cluster.inprocess;
import static org.junit.jupiter.api.Assertions.assertArrayEquals;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertTrue;
import java.io.ByteArrayInputStream;
import java.io.IOException;
import java.nio.file.Path;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.io.TempDir;
import stirling.software.common.cluster.FileStore;
class LocalDiskFileStoreTest {
@Test
void storeRetrieveSizeDeleteExistsRoundTrip(@TempDir Path dir) throws IOException {
LocalDiskFileStore store = new LocalDiskFileStore(dir.toString());
byte[] payload = "hello-bytes".getBytes();
FileStore.Stored stored = store.store(new ByteArrayInputStream(payload), "x.txt");
assertEquals(payload.length, stored.size());
assertTrue(store.exists(stored.fileId()));
assertEquals(payload.length, store.size(stored.fileId()));
assertArrayEquals(payload, store.retrieveBytes(stored.fileId()));
assertTrue(store.delete(stored.fileId()));
assertFalse(store.exists(stored.fileId()));
}
@Test
void traversalIdsAreRejected(@TempDir Path dir) {
LocalDiskFileStore store = new LocalDiskFileStore(dir.toString());
assertThrows(IllegalArgumentException.class, () -> store.resolve("../foo"));
assertThrows(IllegalArgumentException.class, () -> store.resolve("a/b"));
assertThrows(IllegalArgumentException.class, () -> store.resolve("a\\b"));
}
}
@@ -0,0 +1,27 @@
package stirling.software.common.service;
import static org.junit.jupiter.api.Assertions.assertArrayEquals;
import static org.mockito.Mockito.mock;
import java.io.IOException;
import java.nio.file.Path;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.io.TempDir;
import stirling.software.common.cluster.inprocess.LocalDiskFileStore;
class FileStorageDelegationTest {
@Test
void storeBytesThenRetrieveBytesRoundTripsThroughFileStore(@TempDir Path tempDir)
throws IOException {
FileStorage fs =
new FileStorage(
mock(FileOrUploadService.class),
new LocalDiskFileStore(tempDir.toString()));
byte[] payload = "round-trip".getBytes();
String id = fs.storeBytes(payload, "x.bin");
assertArrayEquals(payload, fs.retrieveBytes(id));
}
}
@@ -3,6 +3,7 @@ package stirling.software.common.service;
import static org.junit.jupiter.api.Assertions.*;
import static org.mockito.Mockito.*;
import java.io.ByteArrayInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.nio.charset.StandardCharsets;
@@ -13,29 +14,30 @@ import java.util.stream.Stream;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.io.TempDir;
import org.mockito.InjectMocks;
import org.mockito.Mock;
import org.mockito.MockitoAnnotations;
import org.springframework.core.io.ByteArrayResource;
import org.springframework.core.io.Resource;
import org.springframework.http.MediaType;
import org.springframework.test.util.ReflectionTestUtils;
import org.springframework.web.multipart.MultipartFile;
import stirling.software.common.cluster.inprocess.LocalDiskFileStore;
class FileStorageTest {
@TempDir Path tempDir;
@Mock private FileOrUploadService fileOrUploadService;
@InjectMocks private FileStorage fileStorage;
private FileStorage fileStorage;
private MultipartFile mockFile;
@BeforeEach
void setUp() {
void setUp() throws IOException {
MockitoAnnotations.openMocks(this);
ReflectionTestUtils.setField(fileStorage, "tempDirPath", tempDir.toString());
fileStorage =
new FileStorage(fileOrUploadService, new LocalDiskFileStore(tempDir.toString()));
// Create a mock MultipartFile
mockFile = mock(MultipartFile.class);
@@ -47,17 +49,7 @@ class FileStorageTest {
void testStoreFile() throws IOException {
// Arrange
byte[] fileContent = "Test PDF content".getBytes();
when(mockFile.getBytes()).thenReturn(fileContent);
// Set up mock to handle transferTo by writing the file
doAnswer(
invocation -> {
java.io.File file = invocation.getArgument(0);
Files.write(file.toPath(), fileContent);
return null;
})
.when(mockFile)
.transferTo(any(java.io.File.class));
when(mockFile.getInputStream()).thenReturn(new ByteArrayInputStream(fileContent));
// Act
String fileId = fileStorage.storeFile(mockFile);
@@ -65,7 +57,7 @@ class FileStorageTest {
// Assert
assertNotNull(fileId);
assertTrue(Files.exists(tempDir.resolve(fileId)));
verify(mockFile).transferTo(any(java.io.File.class));
assertArrayEquals(fileContent, Files.readAllBytes(tempDir.resolve(fileId)));
}
@Test
@@ -247,11 +239,11 @@ class FileStorageTest {
filesBefore = s.count();
}
// Act + Assert: IOException must propagate out — not be swallowed.
// Act + Assert: IOException must propagate out - not be swallowed.
assertThrows(
IOException.class, () -> fileStorage.storeFromResource(flakyResource, "n.pdf"));
// Assert: no partial file lingers under the storage directory — the finally
// Assert: no partial file lingers under the storage directory - the finally
// branch's deleteIfExists must have cleaned it up.
long filesAfter;
try (Stream<Path> s = Files.list(tempDir)) {
@@ -0,0 +1,120 @@
package stirling.software.common.service;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.Mockito.never;
import static org.mockito.Mockito.spy;
import static org.mockito.Mockito.verify;
import java.time.LocalDateTime;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.mockito.Mock;
import org.mockito.MockitoAnnotations;
import org.springframework.test.util.ReflectionTestUtils;
import stirling.software.common.cluster.ClusterBackplane;
import stirling.software.common.cluster.JobStoreEntry;
import stirling.software.common.cluster.JobStoreEntry.JobState;
import stirling.software.common.cluster.inprocess.InProcessClusterBackplane;
import stirling.software.common.cluster.inprocess.InProcessJobStore;
import stirling.software.common.model.ApplicationProperties;
import stirling.software.common.model.job.JobResult;
class TaskManagerJobStoreDelegationTest {
@Mock private FileStorage fileStorage;
private InProcessJobStore jobStore;
private ClusterBackplane backplane;
private TaskManager taskManager;
@BeforeEach
void setUp() {
MockitoAnnotations.openMocks(this);
jobStore = spy(new InProcessJobStore());
backplane = new InProcessClusterBackplane(new ApplicationProperties());
taskManager = new TaskManager(fileStorage, jobStore, backplane);
ReflectionTestUtils.setField(taskManager, "jobResultExpiryMinutes", 30);
}
@Test
void createTaskWritesPendingEntry() {
taskManager.createTask("job-1");
JobStoreEntry entry = jobStore.get("job-1").orElseThrow();
assertEquals(JobState.PENDING, entry.state());
assertEquals(backplane.localNodeId(), entry.owningNodeId());
}
@Test
void setCompleteFlipsToComplete() {
taskManager.createTask("job-2");
taskManager.setResult("job-2", "ok");
taskManager.setComplete("job-2");
JobStoreEntry entry = jobStore.get("job-2").orElseThrow();
assertEquals(JobState.COMPLETE, entry.state());
}
@Test
void setErrorFlipsToFailed() {
taskManager.createTask("job-3");
taskManager.setError("job-3", "boom");
JobStoreEntry entry = jobStore.get("job-3").orElseThrow();
assertEquals(JobState.FAILED, entry.state());
assertEquals("boom", entry.error());
}
@Test
void cleanupOldJobsIsNoopWhenBackplaneIsNotInProcess() {
ClusterBackplane mockedValkeyBackplane =
new ClusterBackplane() {
@Override
public boolean isHealthy() {
return true;
}
@Override
public String backplaneType() {
return "valkey";
}
@Override
public String localNodeId() {
return "node-1";
}
@Override
public boolean shouldRunLocalCleanup() {
return false;
}
};
TaskManager tm = new TaskManager(fileStorage, jobStore, mockedValkeyBackplane);
ReflectionTestUtils.setField(tm, "jobResultExpiryMinutes", 30);
tm.createTask("job-4");
tm.setComplete("job-4");
ageJobPastExpiry(tm, "job-4");
tm.cleanupOldJobs();
// cleanup must short-circuit before touching jobStore in cluster mode; the backplane
// TTL owns expiry there. If the gate fired correctly, delete is never called.
verify(jobStore, never()).delete(any());
}
@Test
void cleanupOldJobsDeletesFromJobStoreWhenBackplaneIsInProcess() {
taskManager.createTask("job-5");
taskManager.setComplete("job-5");
ageJobPastExpiry(taskManager, "job-5");
taskManager.cleanupOldJobs();
verify(jobStore).delete("job-5");
}
@SuppressWarnings("unchecked")
private static void ageJobPastExpiry(TaskManager tm, String jobId) {
var jobResults =
(java.util.Map<String, JobResult>) ReflectionTestUtils.getField(tm, "jobResults");
JobResult result = jobResults.get(jobId);
ReflectionTestUtils.setField(result, "completedAt", LocalDateTime.now().minusHours(2));
ReflectionTestUtils.setField(result, "complete", true);
}
}
@@ -1,20 +1,28 @@
package stirling.software.common.service;
import static org.junit.jupiter.api.Assertions.*;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.ArgumentMatchers.anyString;
import static org.mockito.Mockito.*;
import java.time.LocalDateTime;
import java.util.Map;
import java.util.Optional;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.mockito.ArgumentCaptor;
import org.mockito.InjectMocks;
import org.mockito.Mock;
import org.mockito.MockitoAnnotations;
import org.springframework.http.MediaType;
import org.springframework.test.util.ReflectionTestUtils;
import stirling.software.common.cluster.ClusterBackplane;
import stirling.software.common.cluster.JobStore;
import stirling.software.common.cluster.JobStoreEntry;
import stirling.software.common.cluster.JobStoreEntry.JobState;
import stirling.software.common.model.job.JobResult;
import stirling.software.common.model.job.JobStats;
import stirling.software.common.model.job.ResultFile;
@@ -22,6 +30,8 @@ import stirling.software.common.model.job.ResultFile;
class TaskManagerTest {
@Mock private FileStorage fileStorage;
@Mock private JobStore jobStore;
@Mock private ClusterBackplane clusterBackplane;
@InjectMocks private TaskManager taskManager;
@@ -30,6 +40,10 @@ class TaskManagerTest {
@BeforeEach
void setUp() {
closeable = MockitoAnnotations.openMocks(this);
// Treat the backplane as in-process so cleanupOldJobs is not short-circuited.
lenient().when(clusterBackplane.backplaneType()).thenReturn("inprocess");
lenient().when(clusterBackplane.localNodeId()).thenReturn("test-node");
lenient().when(clusterBackplane.shouldRunLocalCleanup()).thenReturn(true);
ReflectionTestUtils.setField(taskManager, "jobResultExpiryMinutes", 30);
}
@@ -270,6 +284,33 @@ class TaskManagerTest {
verify(fileStorage).deleteFile("file-id");
}
@Test
void testCleanupOldJobs_NoOpWhenBackplaneOwnsExpiry() {
// When the backplane reports it should NOT run local cleanup (e.g. a distributed
// backplane with its own TTL), the cleanup loop must leave local state untouched.
when(clusterBackplane.shouldRunLocalCleanup()).thenReturn(false);
// Seed an old completed job that would normally be removed.
String oldJobId = "old-job-distributed";
taskManager.createTask(oldJobId);
JobResult oldJob = taskManager.getJobResult(oldJobId);
ReflectionTestUtils.setField(oldJob, "completedAt", LocalDateTime.now().minusHours(1));
ReflectionTestUtils.setField(oldJob, "complete", true);
Map<String, JobResult> jobResultsMap =
(Map<String, JobResult>) ReflectionTestUtils.getField(taskManager, "jobResults");
assertNotNull(jobResultsMap);
assertTrue(jobResultsMap.containsKey(oldJobId));
// Act
taskManager.cleanupOldJobs();
// Assert: nothing was removed locally, and no jobStore.delete was issued.
assertTrue(jobResultsMap.containsKey(oldJobId));
verify(jobStore, never()).delete(anyString());
verify(fileStorage, never()).deleteFile(anyString());
}
@Test
void testShutdown() {
// This mainly tests that the shutdown method doesn't throw exceptions
@@ -310,4 +351,33 @@ class TaskManagerTest {
// Assert
assertFalse(result);
}
@Test
void testWriteThroughOnUpdate() {
// Mutating calls must write through to the injected JobStore.
String jobId = "write-through-job";
taskManager.createTask(jobId);
taskManager.setResult(jobId, "done");
ArgumentCaptor<JobStoreEntry> captor = ArgumentCaptor.forClass(JobStoreEntry.class);
verify(jobStore, atLeast(2)).put(captor.capture(), any());
JobStoreEntry last = captor.getValue();
assertEquals(jobId, last.jobId());
assertEquals(JobState.COMPLETE, last.state());
assertEquals("test-node", last.owningNodeId());
}
@Test
void testFindJobKeyByFileId_FallsBackToJobStore() {
// When the file id is not in the local map, TaskManager delegates to JobStore.
String fileId = "remote-file-id";
String expectedJobKey = "remote-job-key";
when(jobStore.findJobIdByFileId(fileId)).thenReturn(Optional.of(expectedJobKey));
String actual = taskManager.findJobKeyByFileId(fileId);
assertEquals(expectedJobKey, actual);
verify(jobStore).findJobIdByFileId(fileId);
}
}
@@ -98,6 +98,17 @@ class RequestUriUtilsTest {
assertTrue(RequestUriUtils.isFrontendRoute("", "/split-pdf"));
}
@Test
void testIsFrontendRoute_filesRouteOwnedByFrontend() {
// /files and /files/<folder-uuid> are FileManagerView routes - they
// must fall through to the SPA index.html, not get blocked by the
// backend auth filter. Regression test for direct-nav/refresh on
// the file manager returning a 401 JSON.
assertTrue(RequestUriUtils.isFrontendRoute("", "/files"));
assertTrue(
RequestUriUtils.isFrontendRoute("", "/files/3331910a-4155-4f71-8111-e38c896bc458"));
}
@Test
void testIsFrontendRoute_pathWithExtension() {
assertFalse(RequestUriUtils.isFrontendRoute("", "/some/file.pdf"));
@@ -183,7 +194,7 @@ class RequestUriUtilsTest {
@Test
void testIsPublicAuthEndpoint_shareRootNotPublic() {
// Avoid matching bare "/share" or "/share/" — must have a token segment
// Avoid matching bare "/share" or "/share/" - must have a token segment
assertFalse(RequestUriUtils.isPublicAuthEndpoint("/share", ""));
assertFalse(RequestUriUtils.isPublicAuthEndpoint("/share/", ""));
}
@@ -197,7 +208,7 @@ class RequestUriUtilsTest {
@Test
void testIsPublicAuthEndpoint_shareApiStillProtected() {
// Share-link data APIs must NOT be public — they enforce auth + access checks
// Share-link data APIs must NOT be public - they enforce auth + access checks
assertFalse(RequestUriUtils.isPublicAuthEndpoint("/api/v1/storage/share-links/abc123", ""));
assertFalse(
RequestUriUtils.isPublicAuthEndpoint(
+12 -8
View File
@@ -196,9 +196,13 @@ if (buildPrototypes) {
}
def frontendBuildTask = "frontend:build:${frontendMode}"
// Workspace root holds package.json and node_modules (shared across editor /
// future portal). Editor-specific paths (src, public, dist, tauri) live one
// level deeper under frontend/editor/. When the portal lands as an embedded
// app, add a sibling frontendPortalDir / frontendPortalDistDir alongside.
def frontendDir = file('../../frontend')
def frontendEditorDir = file('../../frontend/editor')
def frontendDistDir = file('../../frontend/editor/dist')
def frontendEditorDistDir = file('../../frontend/editor/dist')
def resourcesStaticDir = file('src/main/resources/static')
def generatedFrontendPaths = [
'assets',
@@ -263,13 +267,13 @@ tasks.register('npmBuild', Exec) {
doNotTrackState("Frontend build depends on untracked npmInstall task")
enabled = buildWithFrontend
group = 'frontend'
description = 'Build frontend application'
description = 'Build editor frontend application'
workingDir file('../..')
commandLine = ['task', frontendBuildTask]
inputs.dir(new File(frontendEditorDir, 'src'))
inputs.dir(new File(frontendEditorDir, 'public'))
inputs.file(new File(frontendDir, 'package.json'))
outputs.dir(frontendDistDir)
outputs.dir(frontendEditorDistDir)
// Show live output
standardOutput = System.out
@@ -280,17 +284,17 @@ tasks.register('npmBuild', Exec) {
environment 'VITE_API_BASE_URL', '/'
doFirst {
println "Building frontend application for production (mode=${frontendMode}, VITE_API_BASE_URL=/)"
println "Building editor frontend application for production (mode=${frontendMode}, VITE_API_BASE_URL=/)"
}
}
tasks.register('copyFrontendAssets', Copy) {
enabled = buildWithFrontend
group = 'frontend'
description = 'Copy frontend build to static resources'
description = 'Copy editor frontend build to static resources'
dependsOn npmBuild
dependsOn cleanFrontendAssets
from(frontendDistDir) {
from(frontendEditorDistDir) {
// Exclude files that conflict with backend static resources
exclude 'robots.txt' // Backend already has this
exclude 'favicon.ico' // Backend already has this
@@ -298,7 +302,7 @@ tasks.register('copyFrontendAssets', Copy) {
into resourcesStaticDir
duplicatesStrategy = DuplicatesStrategy.INCLUDE // Let frontend overwrite when needed
doFirst {
println "Copying frontend build from ${frontendDistDir} to ${resourcesStaticDir}..."
println "Copying frontend build from ${frontendEditorDistDir} to ${resourcesStaticDir}..."
println "Backend static resources will be preserved"
}
doLast {
@@ -330,7 +334,7 @@ tasks.named('copyFrontendAssets').configure {
}
if (buildWithFrontend) {
println "Frontend build enabled - JAR will include React frontend (mode=${frontendMode})"
println "Editor frontend build enabled - JAR will include React frontend (mode=${frontendMode})"
processResources.dependsOn copyFrontendAssets
} else {
println "Frontend build disabled - JAR will be backend-only with API landing page"
@@ -40,7 +40,8 @@ public class ScalePagesController {
private final CustomPDFDocumentFactory pdfDocumentFactory;
private final TempFileManager tempFileManager;
private static PDRectangle getTargetSize(String targetPDRectangle, PDDocument sourceDocument) {
private static PDRectangle getTargetSize(
String targetPDRectangle, String orientation, PDDocument sourceDocument) {
if ("KEEP".equals(targetPDRectangle)) {
if (sourceDocument.getNumberOfPages() == 0) {
throw ExceptionUtils.createInvalidPageSizeException("KEEP");
@@ -57,18 +58,19 @@ public class ScalePagesController {
}
Map<String, PDRectangle> sizeMap = getSizeMap();
if (sizeMap.containsKey(targetPDRectangle)) {
return sizeMap.get(targetPDRectangle);
PDRectangle base = sizeMap.get(targetPDRectangle);
if (base == null) {
throw ExceptionUtils.createInvalidPageSizeException(targetPDRectangle);
}
throw ExceptionUtils.createInvalidPageSizeException(targetPDRectangle);
if ("LANDSCAPE".equalsIgnoreCase(orientation)) {
return new PDRectangle(base.getHeight(), base.getWidth());
}
return base;
}
private static Map<String, PDRectangle> getSizeMap() {
Map<String, PDRectangle> sizeMap = new HashMap<>();
// Portrait sizes (A0-A6)
sizeMap.put("A0", PDRectangle.A0);
sizeMap.put("A1", PDRectangle.A1);
sizeMap.put("A2", PDRectangle.A2);
@@ -76,42 +78,8 @@ public class ScalePagesController {
sizeMap.put("A4", PDRectangle.A4);
sizeMap.put("A5", PDRectangle.A5);
sizeMap.put("A6", PDRectangle.A6);
// Landscape sizes (A0-A6)
sizeMap.put(
"A0_LANDSCAPE",
new PDRectangle(PDRectangle.A0.getHeight(), PDRectangle.A0.getWidth()));
sizeMap.put(
"A1_LANDSCAPE",
new PDRectangle(PDRectangle.A1.getHeight(), PDRectangle.A1.getWidth()));
sizeMap.put(
"A2_LANDSCAPE",
new PDRectangle(PDRectangle.A2.getHeight(), PDRectangle.A2.getWidth()));
sizeMap.put(
"A3_LANDSCAPE",
new PDRectangle(PDRectangle.A3.getHeight(), PDRectangle.A3.getWidth()));
sizeMap.put(
"A4_LANDSCAPE",
new PDRectangle(PDRectangle.A4.getHeight(), PDRectangle.A4.getWidth()));
sizeMap.put(
"A5_LANDSCAPE",
new PDRectangle(PDRectangle.A5.getHeight(), PDRectangle.A5.getWidth()));
sizeMap.put(
"A6_LANDSCAPE",
new PDRectangle(PDRectangle.A6.getHeight(), PDRectangle.A6.getWidth()));
// Portrait US sizes
sizeMap.put("LETTER", PDRectangle.LETTER);
sizeMap.put("LEGAL", PDRectangle.LEGAL);
// Landscape US sizes
sizeMap.put(
"LETTER_LANDSCAPE",
new PDRectangle(PDRectangle.LETTER.getHeight(), PDRectangle.LETTER.getWidth()));
sizeMap.put(
"LEGAL_LANDSCAPE",
new PDRectangle(PDRectangle.LEGAL.getHeight(), PDRectangle.LEGAL.getWidth()));
return sizeMap;
}
@@ -128,13 +96,14 @@ public class ScalePagesController {
throws IOException {
MultipartFile file = request.getFileInput();
String targetPDRectangle = request.getPageSize();
String orientation = request.getOrientation();
float scaleFactor = request.getScaleFactor();
try (PDDocument sourceDocument = pdfDocumentFactory.load(file);
PDDocument outputDocument =
pdfDocumentFactory.createNewDocumentBasedOnOldDocument(sourceDocument)) {
PDRectangle targetSize = getTargetSize(targetPDRectangle, sourceDocument);
PDRectangle targetSize = getTargetSize(targetPDRectangle, orientation, sourceDocument);
// Create LayerUtility once outside the loop for better performance
LayerUtility layerUtility = new LayerUtility(outputDocument);
@@ -3,6 +3,7 @@ package stirling.software.SPDF.controller.api;
import java.io.File;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.StandardCopyOption;
import java.util.ArrayList;
import java.util.HashSet;
@@ -30,12 +31,13 @@ import stirling.software.common.annotations.AutoJobPostMapping;
import stirling.software.common.annotations.api.GeneralApi;
import stirling.software.common.enumeration.ResourceWeight;
import stirling.software.common.service.CustomPDFDocumentFactory;
import stirling.software.common.util.ExceptionUtils;
import stirling.software.common.util.FormUtils;
import stirling.software.common.util.GeneralUtils;
import stirling.software.common.util.TempFile;
import stirling.software.common.util.TempFileManager;
import stirling.software.common.util.WebResponseUtils;
import stirling.software.jpdfium.PdfDocument;
import stirling.software.jpdfium.PdfSplit;
@GeneralApi
@Slf4j
@@ -91,10 +93,14 @@ public class SplitPDFController {
try (ZipOutputStream zipOut =
new ZipOutputStream(Files.newOutputStream(outputTempFile.getPath()))) {
if (hasForm) {
writeSplitsViaReload(
// JPDFium's FPDF_ImportPagesByIndex drops the AcroForm dictionary, which
// breaks Fill Forms downstream. Fall back to the PDFBox load-and-remove
// path so the AcroForm (with only fields whose widgets remain on kept
// pages) is preserved.
writeSplitsViaPdfBox(
sourceTempFile.getFile(), pageNumbers, baseFilename, zipOut);
} else {
writeSplitsViaSharedSource(
writeSplitsViaJpdfium(
sourceTempFile.getFile(), pageNumbers, baseFilename, zipOut);
}
}
@@ -109,7 +115,26 @@ public class SplitPDFController {
}
}
private void writeSplitsViaReload(
private void writeSplitsViaJpdfium(
File source, List<Integer> pageNumbers, String baseFilename, ZipOutputStream zipOut)
throws IOException {
try (PdfDocument sourceDoc = PdfDocument.open(source.toPath())) {
int previousPageNumber = 0;
for (int splitIndex = 0; splitIndex < pageNumbers.size(); splitIndex++) {
int splitPoint = pageNumbers.get(splitIndex);
try (TempFile splitTemp = new TempFile(tempFileManager, ".pdf")) {
try (PdfDocument splitDoc =
PdfSplit.extractPageRange(sourceDoc, previousPageNumber, splitPoint)) {
splitDoc.save(splitTemp.getPath());
}
writeEntry(zipOut, baseFilename, splitIndex + 1, splitTemp.getPath());
}
previousPageNumber = splitPoint + 1;
}
}
}
private void writeSplitsViaPdfBox(
File source, List<Integer> pageNumbers, String baseFilename, ZipOutputStream zipOut)
throws IOException {
int previousPageNumber = 0;
@@ -129,33 +154,15 @@ public class SplitPDFController {
}
FormUtils.pruneOrphanedFormFields(splitDoc);
writeEntry(zipOut, baseFilename, splitIndex + 1, splitDoc);
} catch (Exception e) {
ExceptionUtils.logException("document splitting and saving", e);
throw e;
}
}
}
private void writeSplitsViaSharedSource(
File source, List<Integer> pageNumbers, String baseFilename, ZipOutputStream zipOut)
private void writeEntry(ZipOutputStream zipOut, String baseFilename, int index, Path pdfPath)
throws IOException {
try (PDDocument sourceDoc = pdfDocumentFactory.load(source)) {
int previousPageNumber = 0;
for (int splitIndex = 0; splitIndex < pageNumbers.size(); splitIndex++) {
int splitPoint = pageNumbers.get(splitIndex);
try (PDDocument splitDoc =
pdfDocumentFactory.createNewDocumentBasedOnOldDocument(sourceDoc)) {
for (int i = previousPageNumber; i <= splitPoint; i++) {
splitDoc.addPage(sourceDoc.getPage(i));
}
previousPageNumber = splitPoint + 1;
writeEntry(zipOut, baseFilename, splitIndex + 1, splitDoc);
} catch (Exception e) {
ExceptionUtils.logException("document splitting and saving", e);
throw e;
}
}
}
zipOut.putNextEntry(new ZipEntry(baseFilename + "_" + index + ".pdf"));
Files.copy(pdfPath, zipOut);
zipOut.closeEntry();
}
private void writeEntry(ZipOutputStream zipOut, String baseFilename, int index, PDDocument doc)
@@ -1,6 +1,10 @@
package stirling.software.SPDF.controller.api;
import java.io.File;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.StandardCopyOption;
import java.util.ArrayList;
import java.util.List;
import java.util.Locale;
@@ -8,9 +12,7 @@ import java.util.zip.ZipEntry;
import java.util.zip.ZipOutputStream;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.apache.pdfbox.pdmodel.PDPage;
import org.apache.pdfbox.pdmodel.interactive.documentnavigation.outline.PDDocumentOutline;
import org.apache.pdfbox.pdmodel.interactive.documentnavigation.outline.PDOutlineItem;
import org.apache.pdfbox.pdmodel.interactive.form.PDAcroForm;
import org.springframework.core.io.Resource;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
@@ -35,10 +37,13 @@ import stirling.software.common.model.PdfMetadata;
import stirling.software.common.service.CustomPDFDocumentFactory;
import stirling.software.common.service.PdfMetadataService;
import stirling.software.common.util.ExceptionUtils;
import stirling.software.common.util.FormUtils;
import stirling.software.common.util.GeneralUtils;
import stirling.software.common.util.TempFile;
import stirling.software.common.util.TempFileManager;
import stirling.software.common.util.WebResponseUtils;
import stirling.software.jpdfium.PdfDocument;
import stirling.software.jpdfium.PdfSplit;
@GeneralApi
@Slf4j
@@ -51,72 +56,36 @@ public class SplitPdfByChaptersController {
private final TempFileManager tempFileManager;
private static List<Bookmark> extractOutlineItems(
PDDocument sourceDocument,
PDOutlineItem current,
List<Bookmark> bookmarks,
PDOutlineItem nextParent,
private static void collectBookmarks(
List<stirling.software.jpdfium.doc.Bookmark> source,
List<Bookmark> out,
int level,
int maxLevel)
throws Exception {
while (current != null) {
String currentTitle = current.getTitle().replace("/", "");
int firstPage =
sourceDocument.getPages().indexOf(current.findDestinationPage(sourceDocument));
PDOutlineItem child = current.getFirstChild();
PDOutlineItem nextSibling = current.getNextSibling();
int endPage;
if (child != null && level < maxLevel) {
endPage =
sourceDocument
.getPages()
.indexOf(child.findDestinationPage(sourceDocument));
} else if (nextSibling != null) {
endPage =
sourceDocument
.getPages()
.indexOf(nextSibling.findDestinationPage(sourceDocument));
} else if (nextParent != null) {
endPage =
sourceDocument
.getPages()
.indexOf(nextParent.findDestinationPage(sourceDocument));
} else {
endPage = -2;
/*
happens when we have something like this:
Outline Item 2
Outline Item 2.1
Outline Item 2.1.1
Outline Item 2.2
Outline 2.2.1
Outline 2.2.2 <--- this item neither has an immediate next parent nor an immediate next sibling
Outline Item 3
*/
int maxLevel) {
for (stirling.software.jpdfium.doc.Bookmark bm : source) {
if (!bm.isInternal()) {
continue;
}
if (!bookmarks.isEmpty()
&& bookmarks.get(bookmarks.size() - 1).getEndPage() == -2
&& firstPage
>= bookmarks
.get(bookmarks.size() - 1)
.getStartPage()) { // for handling the above-mentioned case
Bookmark previousBookmark = bookmarks.get(bookmarks.size() - 1);
previousBookmark.setEndPage(firstPage);
String title = bm.title() == null ? "" : bm.title().replace("/", "");
int firstPage = Math.max(0, bm.pageIndex());
out.add(new Bookmark(title, firstPage, -2));
if (bm.hasChildren() && level < maxLevel) {
collectBookmarks(bm.children(), out, level + 1, maxLevel);
}
bookmarks.add(new Bookmark(currentTitle, firstPage, endPage));
// Recursively process children
if (child != null && level < maxLevel) {
extractOutlineItems(
sourceDocument, child, bookmarks, nextSibling, level + 1, maxLevel);
}
current = nextSibling;
}
return bookmarks;
}
private static void assignEndPages(List<Bookmark> bookmarks, int totalPages) {
for (int i = 0; i < bookmarks.size(); i++) {
Bookmark current = bookmarks.get(i);
int next = -1;
for (int j = i + 1; j < bookmarks.size(); j++) {
if (bookmarks.get(j).getStartPage() >= current.getStartPage()) {
next = bookmarks.get(j).getStartPage();
break;
}
}
current.setEndPage(next == -1 ? totalPages : next);
}
}
@AutoJobPostMapping(
@@ -134,46 +103,41 @@ public class SplitPdfByChaptersController {
MultipartFile file = request.getFileInput();
boolean includeMetadata = Boolean.TRUE.equals(request.getIncludeMetadata());
Integer bookmarkLevel =
request.getBookmarkLevel(); // levels start from 0 (top most bookmarks)
Integer bookmarkLevel = request.getBookmarkLevel();
if (bookmarkLevel < 0) {
throw ExceptionUtils.createIllegalArgumentException(
"error.invalidArgument", "Invalid argument: {0}", "bookmark level");
}
try (PDDocument sourceDocument = pdfDocumentFactory.load(file)) {
PDDocumentOutline outline = sourceDocument.getDocumentCatalog().getDocumentOutline();
try (TempFile sourceTempFile = new TempFile(tempFileManager, ".pdf")) {
Files.copy(
file.getInputStream(),
sourceTempFile.getPath(),
StandardCopyOption.REPLACE_EXISTING);
if (outline == null) {
log.warn("No outline found for {}", file.getOriginalFilename());
throw ExceptionUtils.createIllegalArgumentException(
"error.pdfBookmarksNotFound", "No PDF bookmarks/outline found in document");
}
List<Bookmark> bookmarks = new ArrayList<>();
try {
bookmarks =
extractOutlineItems(
sourceDocument,
outline.getFirstChild(),
bookmarks,
outline.getFirstChild().getNextSibling(),
0,
bookmarkLevel);
// to handle last page edge case
bookmarks.get(bookmarks.size() - 1).setEndPage(sourceDocument.getNumberOfPages());
} catch (Exception e) {
ExceptionUtils.logException("outline extraction", e);
throw e;
int totalPages;
try (PdfDocument sourceDocument = PdfDocument.open(sourceTempFile.getPath())) {
totalPages = sourceDocument.pageCount();
List<stirling.software.jpdfium.doc.Bookmark> roots = sourceDocument.bookmarks();
if (roots == null || roots.isEmpty()) {
log.warn("No outline found for {}", file.getOriginalFilename());
throw ExceptionUtils.createIllegalArgumentException(
"error.pdfBookmarksNotFound",
"No PDF bookmarks/outline found in document");
}
collectBookmarks(roots, bookmarks, 0, bookmarkLevel);
if (bookmarks.isEmpty()) {
log.warn("No outline found for {}", file.getOriginalFilename());
throw ExceptionUtils.createIllegalArgumentException(
"error.pdfBookmarksNotFound",
"No PDF bookmarks/outline found in document");
}
assignEndPages(bookmarks, totalPages);
}
boolean allowDuplicates = Boolean.TRUE.equals(request.getAllowDuplicates());
if (!allowDuplicates) {
/*
duplicates are generated when multiple bookmarks correspond to the same page,
if the user doesn't want duplicates mergeBookmarksThatCorrespondToSamePage() method will merge the titles of all
the bookmarks that correspond to the same page, and treat them as a single bookmark
*/
bookmarks = mergeBookmarksThatCorrespondToSamePage(bookmarks);
}
for (Bookmark bookmark : bookmarks) {
@@ -184,7 +148,23 @@ public class SplitPdfByChaptersController {
bookmark.getEndPage());
}
TempFile zipTempFile = createZipFile(sourceDocument, bookmarks, includeMetadata);
PdfMetadata metadata = null;
boolean hasForm = false;
if (includeMetadata) {
try (PDDocument metaDoc = pdfDocumentFactory.load(sourceTempFile.getFile())) {
metadata = pdfMetadataService.extractMetadataFromPdf(metaDoc);
PDAcroForm acroForm = metaDoc.getDocumentCatalog().getAcroForm(null);
hasForm = acroForm != null;
}
} else {
try (PDDocument acroDoc = pdfDocumentFactory.load(sourceTempFile.getFile(), true)) {
hasForm = acroDoc.getDocumentCatalog().getAcroForm(null) != null;
}
}
TempFile zipTempFile =
createZipFile(
sourceTempFile.getFile(), bookmarks, metadata, totalPages, hasForm);
String filename = GeneralUtils.generateFilename(file.getOriginalFilename(), "");
return WebResponseUtils.zipFileToWebResponse(zipTempFile, filename + ".zip");
}
@@ -216,43 +196,40 @@ public class SplitPdfByChaptersController {
}
private TempFile createZipFile(
PDDocument sourceDocument, List<Bookmark> bookmarks, boolean includeMetadata)
File sourceFile,
List<Bookmark> bookmarks,
PdfMetadata metadata,
int totalPages,
boolean hasForm)
throws Exception {
PdfMetadata metadata =
includeMetadata ? pdfMetadataService.extractMetadataFromPdf(sourceDocument) : null;
String fileNumberFormatter = "%0" + (Integer.toString(bookmarks.size()).length()) + "d ";
TempFile zipTempFile = new TempFile(tempFileManager, ".zip");
try {
try (ZipOutputStream zipOut =
new ZipOutputStream(Files.newOutputStream(zipTempFile.getPath()))) {
try (ZipOutputStream zipOut =
new ZipOutputStream(Files.newOutputStream(zipTempFile.getPath()))) {
if (hasForm) {
// JPDFium's FPDF_ImportPagesByIndex drops the AcroForm dictionary. For form
// PDFs, do the per-chapter extract via PDFBox so form fields survive the split.
for (int i = 0; i < bookmarks.size(); i++) {
Bookmark bookmark = bookmarks.get(i);
try (PDDocument splitDocument = new PDDocument()) {
boolean isSinglePage = (bookmark.getStartPage() == bookmark.getEndPage());
for (int pg = bookmark.getStartPage();
pg < bookmark.getEndPage() + (isSinglePage ? 1 : 0);
pg++) {
PDPage page = sourceDocument.getPage(pg);
splitDocument.addPage(page);
log.debug("Adding page {} to split document", pg);
}
if (includeMetadata) {
pdfMetadataService.setMetadataToPdf(splitDocument, metadata);
}
// split files will be named as "[FILE_NUMBER] [BOOKMARK_TITLE].pdf"
String fileName =
String.format(Locale.ROOT, fileNumberFormatter, i)
+ bookmark.getTitle()
+ ".pdf";
zipOut.putNextEntry(new ZipEntry(fileName));
splitDocument.save(zipOut);
zipOut.closeEntry();
log.debug("Wrote split document {} to zip file", fileName);
} catch (Exception e) {
ExceptionUtils.logException("document splitting and saving", e);
throw e;
writeChapterViaPdfBox(
sourceFile,
bookmarks.get(i),
i,
fileNumberFormatter,
metadata,
zipOut,
totalPages);
}
} else {
try (PdfDocument sourceDocument = PdfDocument.open(sourceFile.toPath())) {
for (int i = 0; i < bookmarks.size(); i++) {
writeChapterViaJpdfium(
sourceDocument,
bookmarks.get(i),
i,
fileNumberFormatter,
metadata,
zipOut,
totalPages);
}
}
}
@@ -265,6 +242,97 @@ public class SplitPdfByChaptersController {
throw e;
}
}
private void writeChapterViaJpdfium(
PdfDocument sourceDocument,
Bookmark bookmark,
int index,
String fileNumberFormatter,
PdfMetadata metadata,
ZipOutputStream zipOut,
int totalPages)
throws Exception {
int[] range = clampRange(bookmark, totalPages);
int from = range[0];
int to = range[1];
try (TempFile splitTemp = new TempFile(tempFileManager, ".pdf")) {
try (PdfDocument splitDoc = PdfSplit.extractPageRange(sourceDocument, from, to)) {
splitDoc.save(splitTemp.getPath());
}
Path finalPath = splitTemp.getPath();
TempFile metaTemp = null;
try {
if (metadata != null) {
metaTemp = new TempFile(tempFileManager, ".pdf");
try (PDDocument doc = pdfDocumentFactory.load(splitTemp.getFile())) {
pdfMetadataService.setMetadataToPdf(doc, metadata);
doc.save(metaTemp.getFile());
}
finalPath = metaTemp.getPath();
}
writeZipEntry(zipOut, fileNumberFormatter, index, bookmark.getTitle(), finalPath);
} finally {
if (metaTemp != null) {
metaTemp.close();
}
}
}
}
private void writeChapterViaPdfBox(
File sourceFile,
Bookmark bookmark,
int index,
String fileNumberFormatter,
PdfMetadata metadata,
ZipOutputStream zipOut,
int totalPages)
throws Exception {
int[] range = clampRange(bookmark, totalPages);
int from = range[0];
int to = range[1];
try (PDDocument doc = pdfDocumentFactory.load(sourceFile)) {
for (int p = doc.getNumberOfPages() - 1; p >= 0; p--) {
if (p < from || p > to) {
doc.removePage(p);
}
}
FormUtils.pruneOrphanedFormFields(doc);
if (metadata != null) {
pdfMetadataService.setMetadataToPdf(doc, metadata);
}
String fileName =
String.format(Locale.ROOT, fileNumberFormatter, index)
+ bookmark.getTitle()
+ ".pdf";
zipOut.putNextEntry(new ZipEntry(fileName));
doc.save(zipOut);
zipOut.closeEntry();
log.debug("Wrote split document {} to zip file", fileName);
}
}
private void writeZipEntry(
ZipOutputStream zipOut,
String fileNumberFormatter,
int index,
String title,
Path pdfPath)
throws IOException {
String fileName = String.format(Locale.ROOT, fileNumberFormatter, index) + title + ".pdf";
zipOut.putNextEntry(new ZipEntry(fileName));
Files.copy(pdfPath, zipOut);
zipOut.closeEntry();
log.debug("Wrote split document {} to zip file", fileName);
}
private static int[] clampRange(Bookmark bookmark, int totalPages) {
boolean isSinglePage = bookmark.getStartPage() == bookmark.getEndPage();
int from = Math.min(Math.max(0, bookmark.getStartPage()), totalPages - 1);
int rawEnd = isSinglePage ? bookmark.getEndPage() : bookmark.getEndPage() - 1;
int to = Math.min(Math.max(from, rawEnd), totalPages - 1);
return new int[] {from, to};
}
}
@Data
@@ -1,9 +1,9 @@
package stirling.software.SPDF.controller.api;
import java.io.ByteArrayOutputStream;
import java.io.File;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.StandardCopyOption;
import java.util.ArrayList;
import java.util.HashSet;
@@ -13,7 +13,6 @@ import java.util.zip.ZipEntry;
import java.util.zip.ZipOutputStream;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.apache.pdfbox.pdmodel.PDPage;
import org.springframework.core.io.Resource;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
@@ -37,6 +36,8 @@ import stirling.software.common.util.GeneralUtils;
import stirling.software.common.util.TempFile;
import stirling.software.common.util.TempFileManager;
import stirling.software.common.util.WebResponseUtils;
import stirling.software.jpdfium.PdfDocument;
import stirling.software.jpdfium.PdfSplit;
@GeneralApi
@Slf4j
@@ -75,23 +76,27 @@ public class SplitPdfBySizeController {
sourceTempFile.getPath(),
StandardCopyOption.REPLACE_EXISTING);
try (PDDocument sourceDocument =
pdfDocumentFactory.load(sourceTempFile.getFile(), true)) {
boolean hasForm = sourceDocument.getDocumentCatalog().getAcroForm(null) != null;
List<List<Integer>> ranges = computeRanges(request, sourceDocument);
boolean hasForm;
try (PDDocument acroDoc = pdfDocumentFactory.load(sourceTempFile.getFile(), true)) {
hasForm = acroDoc.getDocumentCatalog().getAcroForm(null) != null;
}
try (PdfDocument sourceDocument = PdfDocument.open(sourceTempFile.getPath())) {
List<int[]> ranges = computeRanges(request, sourceDocument);
int fileIndex = 1;
for (List<Integer> range : ranges) {
if (range.isEmpty()) {
for (int[] range : ranges) {
if (range.length == 0) {
continue;
}
if (hasForm) {
writeRangeViaReload(
sourceTempFile.getFile(), range, zipOut, filename, fileIndex++);
} else {
writeRangeViaSharedSource(
sourceDocument, range, zipOut, filename, fileIndex++);
}
writeRange(
sourceDocument,
sourceTempFile.getFile(),
range,
zipOut,
filename,
fileIndex++,
hasForm);
}
}
}
@@ -104,29 +109,54 @@ public class SplitPdfBySizeController {
}
}
private List<List<Integer>> computeRanges(
SplitPdfBySizeOrCountRequest request, PDDocument sourceDocument) throws IOException {
private List<int[]> computeRanges(SplitPdfBySizeOrCountRequest request, PdfDocument sourceDoc)
throws IOException {
int type = request.getSplitType();
String value = request.getSplitValue();
if (type == 0) {
return computeSizeRanges(sourceDocument, GeneralUtils.convertSizeToBytes(value));
return computeSizeRanges(sourceDoc, GeneralUtils.convertSizeToBytes(value));
} else if (type == 1) {
return computePageCountRanges(sourceDocument, Integer.parseInt(value));
return computePageCountRanges(sourceDoc, Integer.parseInt(value));
} else if (type == 2) {
return computeDocCountRanges(sourceDocument, Integer.parseInt(value));
return computeDocCountRanges(sourceDoc, Integer.parseInt(value));
}
throw ExceptionUtils.createIllegalArgumentException(
"error.invalidArgument", "Invalid argument: {0}", "split type: " + type);
}
private void writeRangeViaReload(
private void writeRange(
PdfDocument sourceDoc,
File sourceFile,
List<Integer> keepIndices,
int[] range,
ZipOutputStream zipOut,
String baseFilename,
int fileIndex,
boolean hasForm)
throws IOException {
if (hasForm) {
// JPDFium's FPDF_ImportPagesByIndex drops the AcroForm dictionary, breaking form
// fields downstream. For form-bearing PDFs, do the extract via PDFBox so the
// AcroForm survives (pruneOrphanedFormFields removes references to dropped pages).
writeRangeViaPdfBox(sourceFile, range, zipOut, baseFilename, fileIndex);
} else {
try (TempFile splitTemp = new TempFile(tempFileManager, ".pdf")) {
extractRangeToFile(sourceDoc, range, splitTemp.getPath());
writeEntry(zipOut, baseFilename, fileIndex, splitTemp.getPath());
}
}
}
private void writeRangeViaPdfBox(
File sourceFile,
int[] range,
ZipOutputStream zipOut,
String baseFilename,
int fileIndex)
throws IOException {
Set<Integer> keep = new HashSet<>(keepIndices);
Set<Integer> keep = new HashSet<>();
for (int p : range) {
keep.add(p);
}
try (PDDocument doc = pdfDocumentFactory.load(sourceFile)) {
for (int i = doc.getNumberOfPages() - 1; i >= 0; i--) {
if (!keep.contains(i)) {
@@ -134,50 +164,41 @@ public class SplitPdfBySizeController {
}
}
FormUtils.pruneOrphanedFormFields(doc);
writeEntry(zipOut, baseFilename, fileIndex, doc);
zipOut.putNextEntry(new ZipEntry(baseFilename + "_" + fileIndex + ".pdf"));
doc.save(zipOut);
zipOut.closeEntry();
}
}
private void writeRangeViaSharedSource(
PDDocument sourceDocument,
List<Integer> keepIndices,
ZipOutputStream zipOut,
String baseFilename,
int fileIndex)
private void extractRangeToFile(PdfDocument sourceDoc, int[] range, Path outputPath)
throws IOException {
try (PDDocument doc =
pdfDocumentFactory.createNewDocumentBasedOnOldDocument(sourceDocument)) {
for (int p : keepIndices) {
doc.addPage(sourceDocument.getPage(p));
}
writeEntry(zipOut, baseFilename, fileIndex, doc);
int from = range[0];
int to = range[range.length - 1];
try (PdfDocument split = PdfSplit.extractPageRange(sourceDoc, from, to)) {
split.save(outputPath);
}
}
private void writeEntry(
ZipOutputStream zipOut, String baseFilename, int fileIndex, PDDocument doc)
ZipOutputStream zipOut, String baseFilename, int fileIndex, Path pdfPath)
throws IOException {
zipOut.putNextEntry(new ZipEntry(baseFilename + "_" + fileIndex + ".pdf"));
doc.save(zipOut);
Files.copy(pdfPath, zipOut);
zipOut.closeEntry();
}
/** Page-index ranges each output should contain. AcroForm overhead isn't modeled. */
private List<List<Integer>> computeSizeRanges(PDDocument sourceDocument, long maxBytes)
throws IOException {
List<List<Integer>> ranges = new ArrayList<>();
List<Integer> currentRange = new ArrayList<>();
int totalPages = sourceDocument.getNumberOfPages();
/** Returns contiguous page-index ranges fitting within {@code maxBytes}. */
private List<int[]> computeSizeRanges(PdfDocument sourceDoc, long maxBytes) throws IOException {
List<int[]> ranges = new ArrayList<>();
int totalPages = sourceDoc.pageCount();
int baseCheckFrequency = 5;
PDDocument scratch = new PDDocument();
try {
int rangeStart = 0;
int rangeEnd = -1;
try (TempFile probe = new TempFile(tempFileManager, ".pdf")) {
File probeFile = probe.getFile();
for (int pageIndex = 0; pageIndex < totalPages; pageIndex++) {
PDPage page = sourceDocument.getPage(pageIndex);
scratch.addPage(new PDPage(page.getCOSObject()));
currentRange.add(pageIndex);
int pageAdded = currentRange.size();
rangeEnd = pageIndex;
int pageAdded = rangeEnd - rangeStart + 1;
boolean shouldCheckSize =
(pageAdded % baseCheckFrequency == 0)
|| (pageIndex == totalPages - 1)
@@ -185,117 +206,110 @@ public class SplitPdfBySizeController {
if (!shouldCheckSize) {
continue;
}
long actualSize;
try (ByteArrayOutputStream out = new ByteArrayOutputStream()) {
scratch.save(out);
actualSize = out.size();
}
long actualSize = saveRange(sourceDoc, rangeStart, rangeEnd, probeFile);
if (actualSize > maxBytes) {
if (scratch.getNumberOfPages() > 1) {
scratch.removePage(scratch.getNumberOfPages() - 1);
currentRange.remove(currentRange.size() - 1);
pageIndex--; // retry this page in the next chunk
if (pageAdded > 1) {
rangeEnd = pageIndex - 1;
pageIndex--;
}
ranges.add(new ArrayList<>(currentRange));
currentRange.clear();
scratch.close();
scratch = new PDDocument();
ranges.add(buildRange(rangeStart, rangeEnd));
rangeStart = rangeEnd + 1;
rangeEnd = rangeStart - 1;
} else if (pageIndex < totalPages - 1 && actualSize < maxBytes * 0.75) {
int extraPagesAdded =
lookAheadFit(scratch, sourceDocument, pageIndex, maxBytes);
for (int i = 0; i < extraPagesAdded; i++) {
int extra = pageIndex + 1 + i;
scratch.addPage(new PDPage(sourceDocument.getPage(extra).getCOSObject()));
currentRange.add(extra);
}
pageIndex += extraPagesAdded;
int extra =
lookAheadFit(
sourceDoc,
rangeStart,
pageIndex,
maxBytes,
totalPages,
probeFile);
pageIndex += extra;
rangeEnd = pageIndex;
}
}
if (!currentRange.isEmpty()) {
ranges.add(new ArrayList<>(currentRange));
}
} finally {
scratch.close();
}
if (rangeEnd >= rangeStart) {
ranges.add(buildRange(rangeStart, rangeEnd));
}
return ranges;
}
/** Speculatively tries up to 5 next pages; returns how many fit under {@code maxBytes}. */
private int lookAheadFit(PDDocument scratch, PDDocument source, int pageIndex, long maxBytes)
private long saveRange(PdfDocument sourceDoc, int from, int to, File output)
throws IOException {
int totalPages = source.getNumberOfPages();
int pagesToLookAhead = Math.min(5, totalPages - pageIndex - 1);
if (pagesToLookAhead == 0) {
return 0;
try (PdfDocument split = PdfSplit.extractPageRange(sourceDoc, from, to)) {
split.save(output.toPath());
}
int extraPagesAdded = 0;
try (PDDocument testDoc = new PDDocument()) {
for (int i = 0; i < scratch.getNumberOfPages(); i++) {
testDoc.addPage(new PDPage(scratch.getPage(i).getCOSObject()));
}
for (int i = 0; i < pagesToLookAhead; i++) {
testDoc.addPage(new PDPage(source.getPage(pageIndex + 1 + i).getCOSObject()));
long testSize;
try (ByteArrayOutputStream out = new ByteArrayOutputStream()) {
testDoc.save(out);
testSize = out.size();
}
if (testSize > maxBytes) {
break;
}
extraPagesAdded++;
}
}
return extraPagesAdded;
return output.length();
}
private List<List<Integer>> computePageCountRanges(PDDocument sourceDocument, int pageCount) {
private int lookAheadFit(
PdfDocument sourceDoc,
int rangeStart,
int currentEnd,
long maxBytes,
int totalPages,
File probeFile)
throws IOException {
int pagesToLookAhead = Math.min(5, totalPages - currentEnd - 1);
int extra = 0;
for (int i = 0; i < pagesToLookAhead; i++) {
int trialEnd = currentEnd + 1 + i;
long size = saveRange(sourceDoc, rangeStart, trialEnd, probeFile);
if (size > maxBytes) {
break;
}
extra++;
}
return extra;
}
private List<int[]> computePageCountRanges(PdfDocument sourceDoc, int pageCount) {
if (pageCount <= 0) {
throw ExceptionUtils.createIllegalArgumentException(
"error.invalidArgument", "Invalid argument: {0}", "page count: " + pageCount);
}
int totalPages = sourceDocument.getNumberOfPages();
List<List<Integer>> ranges = new ArrayList<>();
List<Integer> current = new ArrayList<>(pageCount);
for (int i = 0; i < totalPages; i++) {
current.add(i);
if (current.size() == pageCount) {
ranges.add(current);
current = new ArrayList<>(pageCount);
}
}
if (!current.isEmpty()) {
ranges.add(current);
int totalPages = sourceDoc.pageCount();
List<int[]> ranges = new ArrayList<>();
int start = 0;
while (start < totalPages) {
int end = Math.min(start + pageCount - 1, totalPages - 1);
ranges.add(buildRange(start, end));
start = end + 1;
}
return ranges;
}
private List<List<Integer>> computeDocCountRanges(
PDDocument sourceDocument, int documentCount) {
private List<int[]> computeDocCountRanges(PdfDocument sourceDoc, int documentCount) {
if (documentCount <= 0) {
throw ExceptionUtils.createIllegalArgumentException(
"error.invalidArgument",
"Invalid argument: {0}",
"document count: " + documentCount);
}
int totalPages = sourceDocument.getNumberOfPages();
int totalPages = sourceDoc.pageCount();
int pagesPerDocument = totalPages / documentCount;
int extraPages = totalPages % documentCount;
List<List<Integer>> ranges = new ArrayList<>();
List<int[]> ranges = new ArrayList<>();
int cursor = 0;
for (int i = 0; i < documentCount; i++) {
int pagesToAdd = pagesPerDocument + (i < extraPages ? 1 : 0);
List<Integer> range = new ArrayList<>(pagesToAdd);
for (int j = 0; j < pagesToAdd; j++) {
range.add(cursor++);
if (pagesToAdd == 0) {
continue;
}
ranges.add(range);
int end = cursor + pagesToAdd - 1;
ranges.add(buildRange(cursor, end));
cursor = end + 1;
}
return ranges;
}
private static int[] buildRange(int start, int end) {
int[] range = new int[end - start + 1];
for (int i = 0; i < range.length; i++) {
range[i] = start + i;
}
return range;
}
}
@@ -12,6 +12,8 @@ import org.springframework.web.bind.annotation.RequestParam;
import io.swagger.v3.oas.annotations.Hidden;
import jakarta.servlet.http.HttpServletRequest;
import lombok.extern.slf4j.Slf4j;
import stirling.software.SPDF.config.EndpointConfiguration;
@@ -91,6 +93,44 @@ public class ConfigController {
return null;
}
/**
* Resolve the frontend URL the client should advertise to phones / share-link recipients.
* Priority: explicit system.frontendUrl, then the Host the user is already using to reach this
* server (works for Docker, reverse proxies, and bare-metal LANs), then a detected site-local
* IPv4, then empty.
*/
// visible for testing
String resolveFrontendUrl(HttpServletRequest request, AppConfig appConfig) {
String configured = applicationProperties.getSystem().getFrontendUrl();
if (configured != null && !configured.isBlank()) {
return configured;
}
if (request != null) {
String host = request.getServerName();
if (host != null && !host.isBlank() && !isLoopbackHost(host)) {
String scheme = request.getScheme();
int port = request.getServerPort();
boolean defaultPort =
("http".equals(scheme) && port == 80)
|| ("https".equals(scheme) && port == 443);
return defaultPort ? scheme + "://" + host : scheme + "://" + host + ":" + port;
}
}
String localIp = GeneralUtils.getLocalNetworkIp();
if (localIp != null) {
String scheme = appConfig.getBackendUrl().startsWith("https") ? "https" : "http";
return scheme + "://" + localIp + ":" + appConfig.getServerPort();
}
return "";
}
private static boolean isLoopbackHost(String host) {
return "localhost".equalsIgnoreCase(host)
|| "127.0.0.1".equals(host)
|| "::1".equals(host)
|| "0:0:0:0:0:0:0:1".equals(host);
}
/** Check if running Enterprise edition dynamically. */
private Boolean isRunningEE() {
// Use LicenseService for fresh license status if available
@@ -107,7 +147,7 @@ public class ConfigController {
}
@GetMapping("/app-config")
public ResponseEntity<Map<String, Object>> getAppConfig() {
public ResponseEntity<Map<String, Object>> getAppConfig(HttpServletRequest request) {
Map<String, Object> configData = new HashMap<>();
try {
@@ -124,17 +164,7 @@ public class ConfigController {
configData.put("serverPort", appConfig.getServerPort());
String frontendUrl = applicationProperties.getSystem().getFrontendUrl();
if ((frontendUrl == null || frontendUrl.isBlank())
&& Boolean.parseBoolean(
System.getProperty("STIRLING_PDF_TAURI_MODE", "false"))) {
String localIp = GeneralUtils.getLocalNetworkIp();
if (localIp != null) {
String scheme =
appConfig.getBackendUrl().startsWith("https") ? "https" : "http";
frontendUrl = scheme + "://" + localIp + ":" + appConfig.getServerPort();
}
}
configData.put("frontendUrl", frontendUrl != null ? frontendUrl : "");
configData.put("frontendUrl", resolveFrontendUrl(request, appConfig));
// Add mobile scanner settings
configData.put(
@@ -160,14 +160,19 @@ public class ReactRoutingController {
return ResponseEntity.ok().contentType(MediaType.TEXT_HTML).body(cachedCallbackHtml);
}
// `files` was historically a backend static-asset directory and was therefore
// in the exclusion list - removing it lets /files and /files/<folder-uuid>
// forward to the SPA index.html, which is what FileManagerView expects.
// (Real storage endpoints live under /api/v1/storage/files, already
// excluded by the leading `api` token in the same regex.)
@GetMapping(
"/{path:^(?!api|static|robots\\.txt|favicon\\.ico|manifest.*\\.json|pipeline|pdfjs|pdfjs-legacy|pdfium|vendor|fonts|images|files|css|js|assets|locales|modern-logo|classic-logo|Login|og_images|samples)[^\\.]*$}")
"/{path:^(?!api|static|robots\\.txt|favicon\\.ico|manifest.*\\.json|pipeline|pdfjs|pdfjs-legacy|pdfium|vendor|fonts|images|css|js|assets|locales|modern-logo|classic-logo|Login|og_images|samples)[^\\.]*$}")
public ResponseEntity<String> forwardRootPaths(HttpServletRequest request) throws IOException {
return serveIndexHtml(request);
}
@GetMapping(
"/{path:^(?!api|static|pipeline|pdfjs|pdfjs-legacy|pdfium|vendor|fonts|images|files|css|js|assets|locales|modern-logo|classic-logo|Login|og_images|samples)[^\\.]*}/{subpath:^(?!.*\\.).*$}")
"/{path:^(?!api|static|pipeline|pdfjs|pdfjs-legacy|pdfium|vendor|fonts|images|css|js|assets|locales|modern-logo|classic-logo|Login|og_images|samples)[^\\.]*}/{subpath:^(?!.*\\.).*$}")
public ResponseEntity<String> forwardNestedPaths(HttpServletRequest request)
throws IOException {
return serveIndexHtml(request);
@@ -22,6 +22,7 @@ import org.springframework.web.bind.annotation.ExceptionHandler;
import org.springframework.web.bind.annotation.RestControllerAdvice;
import org.springframework.web.multipart.MaxUploadSizeExceededException;
import org.springframework.web.multipart.support.MissingServletRequestPartException;
import org.springframework.web.server.ResponseStatusException;
import org.springframework.web.servlet.NoHandlerFoundException;
import jakarta.servlet.http.HttpServletRequest;
@@ -196,12 +197,12 @@ public class GlobalExceptionHandler {
/**
* Checks whether the given IOException indicates that the client disconnected before the
* response could be written (broken pipe, connection reset, etc.). When this happens there is
* no point in serialising a {@link ProblemDetail} body because the socket is already closed —
* no point in serialising a {@link ProblemDetail} body because the socket is already closed -
* and attempting to do so may trigger a secondary {@code HttpMessageNotWritableException} if
* the response Content-Type was already committed as a non-JSON type (e.g. image/png).
*/
private static boolean isClientDisconnectException(IOException ex) {
// Walk the causal chain — Jetty/Tomcat may wrap the low-level SocketException
// Walk the causal chain - Jetty/Tomcat may wrap the low-level SocketException
Throwable current = ex;
while (current != null) {
String msg = current.getMessage();
@@ -1040,6 +1041,43 @@ public class GlobalExceptionHandler {
* @param request the HTTP servlet request
* @return ProblemDetail with appropriate HTTP status
*/
/**
* Handle ResponseStatusException explicitly so its embedded HTTP status reaches the client
* instead of being swallowed by the {@code RuntimeException} catch-all (which would downgrade
* every controller-thrown 400/404/409 to a generic 500). Folder/file storage controllers and
* any other code that throws {@code ResponseStatusException} relies on this handler taking
* precedence.
*/
@ExceptionHandler(ResponseStatusException.class)
public ResponseEntity<ProblemDetail> handleResponseStatusException(
ResponseStatusException ex, HttpServletRequest request) {
HttpStatus status =
HttpStatus.resolve(ex.getStatusCode().value()) != null
? HttpStatus.valueOf(ex.getStatusCode().value())
: HttpStatus.INTERNAL_SERVER_ERROR;
String reason = ex.getReason() != null ? ex.getReason() : status.getReasonPhrase();
ProblemDetail problemDetail = createBaseProblemDetail(status, reason, request);
problemDetail.setType(URI.create("/errors/" + status.value()));
problemDetail.setTitle(status.getReasonPhrase());
problemDetail.setProperty("title", status.getReasonPhrase());
// 5xx is operator-relevant; 4xx is a normal client-rejection - log at the right level.
if (status.is5xxServerError()) {
log.error(
"ResponseStatusException {} at {}: {}",
status.value(),
request.getRequestURI(),
reason,
ex);
} else {
log.debug(
"ResponseStatusException {} at {}: {}",
status.value(),
request.getRequestURI(),
reason);
}
return ResponseEntity.status(status).contentType(PROBLEM_JSON).body(problemDetail);
}
@ExceptionHandler(RuntimeException.class)
public ResponseEntity<ProblemDetail> handleRuntimeException(
RuntimeException ex, HttpServletRequest request) {
@@ -18,4 +18,11 @@ public class PDFWithPageSize extends PDFFile {
requiredMode = Schema.RequiredMode.REQUIRED,
allowableValues = {"A0", "A1", "A2", "A3", "A4", "A5", "A6", "LETTER", "LEGAL", "KEEP"})
private String pageSize;
@Schema(
description =
"Orientation to apply to the target page size. Ignored when pageSize is KEEP.",
defaultValue = "PORTRAIT",
allowableValues = {"PORTRAIT", "LANDSCAPE"})
private String orientation = "PORTRAIT";
}

Some files were not shown because too many files have changed in this diff Show More