Compare commits

...
Author SHA1 Message Date
EthanHealy01 600add1474 run frontend fix 2026-05-27 12:42:22 +01:00
EthanHealy01 8451b06d53 chore: set aiEngine env vars automatically in dev:all
When AIENGINE_URL is provided (i.e. task dev:all), also set
AIENGINE_ENABLED=true and AIENGINE_TIMEOUTSECONDS=120 so the backend
picks up the full AI engine config without needing it in settings.yml.
2026-05-27 12:12:42 +01:00
EthanHealy01 7bb3024dc9 feat: add midnight theme and fix theme locations
Adds midnight dark theme, fixes broken theme variable locations, adds
clean-unused-theme-vars script, and updates RainbowThemeProvider and
related theme constants.
2026-05-27 11:37:49 +01:00
EthanHealy01 18cd2baec0 feat: add Agents UI to proprietary right sidebar
Adds agents panel to the right sidebar, gated behind aiEngineEnabled.
Includes RightSidebar refactor, FullscreenToolPanel, core stubs, and
proprietary AgentsPanel with Stirling hero CTA and coming-soon agents.
2026-05-27 11:14:19 +01:00
7743720f0a chore: fix "Endoints" typo in health route comment (#6446)
`app/common/src/main/java/stirling/software/common/util/RequestUriUtils.java:175`
section comment said "Health Endoints" → "Health Endpoints".
Comment-only.

Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
2026-05-26 23:30:57 +01:00
Saul IfshinandGitHub 620f246ca4 test(frontend): add URL-encoded filename parsing case (#6436)
# Description of Changes

<!--
Please provide a summary of the changes, including:

- What was changed
- Why the change was made
- Any challenges encountered

Closes #(issue_number)
-->

---

## Checklist

### General

- [x] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [x] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [x] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [x] I have performed a self-review of my own code
- [x] My changes generate no new warnings

### Documentation

- [x] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [x] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [x] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [x] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [x] I have run `task check` to verify linters, typechecks, and tests
pass
- [x] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
2026-05-26 23:29:32 +01:00
Saul IfshinandGitHub dd24440b73 test(frontend): cover MIME precedence in non-PDF type detection (#6438)
# Description of Changes

<!--
Please provide a summary of the changes, including:

- What was changed
- Why the change was made
- Any challenges encountered

Closes #(issue_number)
-->

---

## Checklist

### General

- [x] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [x] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [x] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [x] I have performed a self-review of my own code
- [x] My changes generate no new warnings

### Documentation

- [x] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [x] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [x] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [x] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [x] I have run `task check` to verify linters, typechecks, and tests
pass
- [x] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
2026-05-26 23:28:17 +01:00
LudyandGitHub c0374266e7 ci: harden workflow runners and update checkout action pins (#6445)
# Description of Changes

Add runner hardening and housekeeping across workflows.

- .github/dependabot.yml: add /docker/unoserver and /engine to
Dependabot update paths.
- .github/workflows/_runner-pick.yml and .github/workflows/build.yml:
add step-security/harden-runner (egress-policy: audit) to audit outbound
calls from runners.
- .github/workflows/ai-engine.yml, .github/workflows/aur-publish.yml,
.github/workflows/package-managers.yml: update actions/checkout usage to
the newer v6.0.2 reference.
- .github/workflows/package-managers.yml: minor YAML formatting tidy
(release types array).

These changes improve CI security by auditing egress, update the
checkout action to a newer release, and expand Dependabot coverage.

---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have run `task check` to verify linters, typechecks, and tests
pass
- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#7-testing)
for more details.
2026-05-26 23:26:20 +01:00
Anthony StirlingandGitHub 730ed43af9 Fix login loop (#6402) 2026-05-26 20:05:25 +01:00
Anthony StirlingandGitHub 56ff1e5092 impl migration to pdfium for split (#6410)
## Summary

Migrates `SplitPDFController`, `SplitPdfByChaptersController`,
`SplitPdfBySizeController` from PDFBox to JPDFium.
`SplitPdfBySectionsController` and `AutoSplitPdfController` are
intentionally left on PDFBox (require JPDFium 1.0.2 features that don't
exist yet).

## Benchmark (audited on `audit/jpdfium-split`, file
`app/core/src/test/java/stirling/software/SPDF/bench/SplitBenchmark.java`)

| Workload | PDFBox heap | JPDFium heap | PDFBox wall | JPDFium wall |
|---|---|---|---|---|
| 100 pp, chunk 10 | +21-26 MB | **+0.02 MB** | 80-106 ms | **25 ms** |
| 300 pp, chunk 10 | +59 MB | **+1.0 MB** | 232 ms | **76 ms** |

**98-99.9% heap reduction. 3-4.2x faster wall.**

## Hybrid

- AcroForm-bearing splits keep PDFBox
`FormUtils.pruneOrphanedFormFields` post-pass (FPDF_ImportPagesByIndex
drops AcroForm dict). Sub-bench shows +1.0 MB / +27 ms - tightly
bounded.
- Metadata extraction stays on PDFBox.
- `SplitPdfBySectionsController` - JPDFium `PdfPageSplitter` only does
2-up halving, not arbitrary MxN.
- `AutoSplitPdfController` - needs PDFRenderer + zxing for QR markers.

## Test plan

- [ ] 30/30 unit tests pass with PDFBox `Loader.loadPDF` as the oracle
(assert page counts + document totals)
- [ ] Existing cucumber feature `split.feature` continues to pass
- [ ] AcroForm-bearing PDF round-trips without orphaned widgets (covered
by existing FormUtils tests)
2026-05-26 17:50:13 +01:00
ConnorYohandGitHub 05b80fbe4f Working local Saas (#6450)
Env file for setting backend saas and taskfile for running it
2026-05-26 14:40:44 +00:00
Reece BrowneandGitHub 4047d02086 chore: address restructure PR review feedback (#6423) 2026-05-26 14:12:05 +01:00
LudyandGitHub 5f78083470 Fix unresolved Material Symbols icon names in frontend (#6443) 2026-05-25 17:02:59 +01:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>Anthony Stirling
fc048872d7 build(deps): bump idna from 3.12 to 3.15 in /testing/cucumber in the pip group across 1 directory (#6403)
Bumps the pip group with 1 update in the /testing/cucumber directory:
[idna](https://github.com/kjd/idna).

Updates `idna` from 3.12 to 3.15
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/kjd/idna/blob/master/HISTORY.md">idna's
changelog</a>.</em></p>
<blockquote>
<h2>3.15 (2026-05-12)</h2>
<ul>
<li>Enforce DNS-length cap on individual labels early in
<code>check_label</code>,
short-circuiting contextual-rule processing for oversized input
while staying compatible with UTS 46 usage.</li>
<li>Tidy core helpers: hoist bidi category sets to module-level
frozensets (avoiding per-codepoint list construction), simplify
length checks, and reuse the shared <code>_unicode_dots_re</code> from
<code>idna.core</code> in the codec module.</li>
<li>Use <code>raise ... from err</code> for proper exception chaining
and
switch internal string formatting to f-strings.</li>
<li>Allow <code>flit_core</code> 4.x in the build backend.</li>
<li>Expand the ruff lint set (flake8-bugbear, flake8-simplify,
pyupgrade, perflint) and apply the surfaced fixes; pin lint CI
to Python 3.14.</li>
<li>Add Dependabot configuration for GitHub Actions.</li>
<li>Convert README and HISTORY from reStructuredText to Markdown.</li>
<li>Reference CVE-2026-45409 for the 3.14 advisory in place of the
initial GHSA identifier.</li>
</ul>
<p>Thanks to Felix Yan, Stan Ulbrych, and metsw24-max for
contributions to this release.</p>
<h2>3.14 (2026-05-10)</h2>
<ul>
<li>Removed opportunity to process long inputs into quadratic
time by rejecting oversize inputs up-front. Closes a bypass
of the CVE-2024-3651 mitigation. [CVE-2026-45409]</li>
</ul>
<p>Thanks to Stan Ulbrych for reporting the issue.</p>
<h2>3.13 (2026-04-22)</h2>
<ul>
<li>Correct classification error for codepoint U+A7F1</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/kjd/idna/commit/af30a092e158181d0b35ac66dfa813788126bdd8"><code>af30a09</code></a>
Release 3.15</li>
<li><a
href="https://github.com/kjd/idna/commit/30314d4628744ca14cf2b5820564e5127a9f86f2"><code>30314d4</code></a>
Pre-release 3.15rc0</li>
<li><a
href="https://github.com/kjd/idna/commit/05d4b219aa9eddc47371fcbd2000f0301016f3e9"><code>05d4b21</code></a>
Merge pull request <a
href="https://redirect.github.com/kjd/idna/issues/237">#237</a> from
kjd/convert-docs-to-markdown</li>
<li><a
href="https://github.com/kjd/idna/commit/2987fdba1962bbb2358399e0084ba062b98a0bee"><code>2987fdb</code></a>
Convert README and HISTORY from reStructuredText to Markdown</li>
<li><a
href="https://github.com/kjd/idna/commit/59fa8002d514bf4a5ce7b58f67b9ec587d53fa9c"><code>59fa800</code></a>
Merge pull request <a
href="https://redirect.github.com/kjd/idna/issues/236">#236</a> from
kjd/dependabot/github_actions/actions-f3e34333ea</li>
<li><a
href="https://github.com/kjd/idna/commit/def69834ced5d4b3c50439d8b99c4c856ec19ca2"><code>def6983</code></a>
Merge branch 'master' into
dependabot/github_actions/actions-f3e34333ea</li>
<li><a
href="https://github.com/kjd/idna/commit/bbd8004a797185d8c56bb555cd5c88fde05e0631"><code>bbd8004</code></a>
Merge pull request <a
href="https://redirect.github.com/kjd/idna/issues/234">#234</a> from
StanFromIreland/patch-1</li>
<li><a
href="https://github.com/kjd/idna/commit/edd07c05024344a6ccb517414ccb36683aee99fc"><code>edd07c0</code></a>
Bump github/codeql-action from 3.35.2 to 4.35.2 in the actions
group</li>
<li><a
href="https://github.com/kjd/idna/commit/5557db030c11bdec50d62aa5f631d705d33ba123"><code>5557db0</code></a>
Merge branch 'master' into patch-1</li>
<li><a
href="https://github.com/kjd/idna/commit/f11746cf4981d25123ef7830d3ee60f07de8ae3d"><code>f11746c</code></a>
Merge pull request <a
href="https://redirect.github.com/kjd/idna/issues/235">#235</a> from
StanFromIreland/patch-2</li>
<li>Additional commits viewable in <a
href="https://github.com/kjd/idna/compare/v3.12...v3.15">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=idna&package-manager=pip&previous-version=3.12&new-version=3.15)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore <dependency name> major version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's major version (unless you unignore this specific
dependency's major version or upgrade to it yourself)
- `@dependabot ignore <dependency name> minor version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's minor version (unless you unignore this specific
dependency's minor version or upgrade to it yourself)
- `@dependabot ignore <dependency name>` will close this group update PR
and stop Dependabot creating any more for the specific dependency
(unless you unignore this specific dependency or upgrade to it yourself)
- `@dependabot unignore <dependency name>` will remove all of the ignore
conditions of the specified dependency
- `@dependabot unignore <dependency name> <ignore condition>` will
remove the ignore condition of the specified dependency and ignore
conditions
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/Stirling-Tools/Stirling-PDF/network/alerts).

</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Anthony Stirling <77850077+Frooodle@users.noreply.github.com>
2026-05-24 09:51:19 +00:00
dependabot[bot]andGitHub 6ee533c427 build(deps): bump ws from 8.20.0 to 8.20.1 in /frontend (#6400)
Signed-off-by: dependabot[bot] <support@github.com>
2026-05-22 22:47:49 +01:00
dependabot[bot]andGitHub a13683e84a build(deps): bump protobufjs from 7.5.6 to 7.6.0 in /frontend (#6401)
Signed-off-by: dependabot[bot] <support@github.com>
2026-05-22 22:44:02 +01:00
914b474d84 build(deps-dev): bump ip-address from 10.1.0 to 10.2.0 in /frontend (#6319)
Co-authored-by: Anthony Stirling <77850077+Frooodle@users.noreply.github.com>
Signed-off-by: dependabot[bot] <support@github.com>
2026-05-22 22:43:11 +01:00
dependabot[bot]andGitHub 51f69a718f build(deps): bump svelte from 5.55.0 to 5.55.9 in /frontend (#6409)
Signed-off-by: dependabot[bot] <support@github.com>
2026-05-22 22:42:51 +01:00
dependabot[bot]andGitHub 7f597051e4 build(deps): bump reportlab from 4.5.0 to 4.5.1 in /testing/cucumber (#6414)
Signed-off-by: dependabot[bot] <support@github.com>
2026-05-22 22:42:26 +01:00
dependabot[bot]andGitHub e113df6252 build(deps): bump requests from 2.33.1 to 2.34.2 in /testing/cucumber (#6416)
Signed-off-by: dependabot[bot] <support@github.com>
2026-05-22 22:42:11 +01:00
dependabot[bot]andGitHub 06b4a4184b build(deps): bump com.diffplug.spotless from 8.4.0 to 8.5.0 (#6417)
Signed-off-by: dependabot[bot] <support@github.com>
2026-05-22 22:41:52 +01:00
dependabot[bot]andGitHub 19982d7207 build(deps): bump actions/ai-inference from 2.0.7 to 2.1.0 (#6415)
Signed-off-by: dependabot[bot] <support@github.com>
2026-05-22 22:30:44 +01:00
dependabot[bot]andGitHub cdfefb02bc build(deps): bump github/codeql-action from 4.35.4 to 4.35.5 (#6421)
Signed-off-by: dependabot[bot] <support@github.com>
2026-05-22 22:30:28 +01:00
dependabot[bot]andGitHub 5ba6a65c1a build(deps): bump step-security/harden-runner from 2.19.1 to 2.19.3 (#6422)
Signed-off-by: dependabot[bot] <support@github.com>
2026-05-22 22:30:02 +01:00
ConnorYohandGitHub 017c8d59fa feat(ai): add Contradiction Agent on a new ChunkedMapper primitive (#6369)
## Summary

Adds a new AI specialist that finds **textual contradictions** across
one or more PDFs — conflicting claims, recommendations, points of view,
contested facts — built entirely in Python on top of the new
`DocumentService` + `ChunkedReasoner` stack from #6314. Replaces the
closed #6304, which was started before #6314 landed and therefore
over-engineered (Java orchestrator, two-round handshake, resume
artifact, discriminated-union lift).

Two commits:
1. **`refactor(engine): extract ChunkedMapper[T] from ChunkedReasoner`**
— pure refactor, public API of ChunkedReasoner unchanged. New
`ChunkedMapper[T: BaseModel]` is a generic parallel-chunk primitive
(slicing, semaphore, time-bounded extraction, cancellation drain,
progress events) that's now a peer to ChunkedReasoner rather than locked
inside it. The compression loop stays on ChunkedReasoner where it
belongs.
2. **`feat(ai): add Contradiction Agent on ChunkedMapper`** — the agent
itself, plus integrations into `PdfReviewAgent` and `PdfQuestionAgent`.

## Architecture

- **Python-only.** No Java code. No `AgentToolId.CONTRADICTION_AGENT`.
No dedicated HTTP endpoint. No resume artifact, no discriminated-union
lift in `contracts/common.py`. Detector runs inside the Python engine
and the Python engine alone.
- **Review path** (`PdfReviewAgent`): a new
`ContradictionIntentClassifier` fires on contradiction-flavoured
prompts; agent runs detection synchronously and emits a single
`EditPlanResponse(steps=[ADD_COMMENTS])`. Single-turn flow — no resume.
- **Question path** (`PdfQuestionAgent`): a new
`ContradictionCapability` joins `RagCapability` and
`WholeDocReaderCapability` in the smart-model toolset, exposing
`find_contradictions(query)`. The smart model picks it from the toolset
alongside `search_knowledge` and `read_full_document`.

## Inside `ContradictionDetector.detect()`

1. `DocumentService.read_pages(file_id)` → ordered `list[Page]`.
2. `ChunkedMapper[_ExtractedClaims].map_pages(...)` — char-budgeted
multi-page slicing; each slice runs the claim-extractor LLM in parallel
under a semaphore.
3. Page-traceability: the extractor returns `_ExtractedClaim.page`
(which `[Page N]` marker the claim came from). The wrapper validates
`page ∈ chunk.pages`; if not, mechanical fallback searches the chunk's
page text for the verbatim quote and reassigns. If still no match, drop
the claim.
4. `Claim.anchor_quality: Literal[\"verbatim\", \"paraphrased\"]` is set
by a substring check against the declared page's text. Verbatim quotes
feed `anchor_text` for snap-to-quote add-comments placement; paraphrased
ones fall back to margin geometry.
5. Subject canonicalisation: ONE fast-model LLM call collapses synonyms
across the document. Fails open to lexical bucketing.
6. Pre-filters: drop identical-quote pairs; drop same-page same-polarity
paraphrases.
7. Per-bucket pair detection in parallel (separate semaphore, cap 5).
Buckets > 12 claims chunk into windows of 12 with overlap 2; pairs
deduped across overlapping windows by frozen `(i, j)` index pair.
8. Summary fast-model call with fallback string on error.

## Prompt-injection hardening

Every prompt that interpolates user-supplied or PDF-extracted text wraps
content in `<user_message>` / `<verdict>` / `<content>` tags with an
explicit SECURITY preamble instructing the model to treat tagged content
as data only.

## Limitations

- **Combined math + contradiction intent**: when both intent classifiers
fire on the same prompt, contradiction takes precedence and the math
intent is silently dropped. Documented in the Review module docstring
and pinned by
`test_review_integration.py::test_contradiction_precedence_over_math`.
- **Cross-window contradiction reach**: within a subject bucket, pairs
more than ~10 claim indices apart in the same chunked window may be
missed by the overlap-2 strategy. Documented in `test_detector.py`.
Acceptable for v1.

## Settings (engine/src/stirling/config/settings.py)

```python
contradiction_detect_concurrency = 5     # per-bucket detector semaphore
contradiction_bucket_chunk_size = 12     # max claims per detector call
contradiction_bucket_chunk_overlap = 2   # overlap for >threshold buckets
```

`chars_per_slice` and extraction concurrency are reused from the
existing `chunked_reasoner_*` settings.

## Test plan

- [x] `uv run pytest tests/ -v` — **245/245 pass** (210 pre-existing +
35 new)
- [x] `uv run ruff check src/ tests/` — clean
- [x] `uv run pyright src/stirling/agents/contradiction/
src/stirling/contracts/contradiction.py` — 0 errors
- [x] `./gradlew :proprietary:test` — green; no Java was touched, but
verified untouched
- [x] Page-traceability tests cover: valid page kept, hallucinated page
dropped, mechanical-reassign on misattribution, anchor-quality verbatim
vs paraphrased
- [x] Review integration: ADD_COMMENTS plan with two paired CommentSpecs
per contradiction; NeedIngestResponse precheck; precedence vs math
intent pinned
- [x] Question integration: all three capabilities wired into
smart-model toolset; `find_contradictions` returns formatted report text
- [x] ChunkedMapper standalone: slicing, multi-chunk ordering, worker
failures, timeouts, cancellation drain, semaphore saturation
- [x] ChunkedReasoner regression: all pre-existing tests pass unchanged
after the internal split

## Relationship to closed #6304

#6304 was closed in favour of this PR. The closed PR predated #6314 and
modelled the agent as a Java-orchestrated two-round examine/deliberate
flow with its own HTTP endpoint and a discriminated-union resume
artifact. With #6314 making full ordered page text available to the
engine via `DocumentService.read_pages`, none of that is needed. Net
effect: drop ~600 lines of Java, drop the two-round handshake, drop the
`ToolReportArtifact` lift, while ending up with a more scalable agent
(chunk-based instead of page-based extraction; tested to
ChunkedReasoner-equivalent scale).
2026-05-22 13:23:52 +00:00
0a50e765b7 Restructure/frontend editor (#6404)
## Move editor under `frontend/editor/`

Pure restructure: `frontend/` becomes the workspace, `frontend/editor/`
holds
  the PDF editor. 1775 file renames + 40 wiring edits. No logic changes.

  ### Why

`frontend/` is currently the editor — its `src/`, `public/`,
`src-tauri/`,
  config files all sit at the root. Promoting `frontend/` to a
workspace and putting the editor in a sibling folder leaves room for
future
apps to drop in alongside it, sharing one `package.json` /
`node_modules` /
  lint config / Storybook.

  ### What moves

  frontend/
  ├── editor/                ← NEW: everything editor-specific
  │   ├── src/               ← was frontend/src/
  │   ├── public/            ← was frontend/public/
  │   ├── src-tauri/         ← was frontend/src-tauri/
│ ├── index.html, vite.config.ts, vitest.config.ts, playwright.config.ts
  │   ├── tsconfig*.json, tailwind.config.js, postcss.config.js
  │   ├── scripts/
  │   ├── .env, .env.desktop, .env.saas
  │   └── DeveloperGuide.md
├── package.json, package-lock.json, node_modules/ ← workspace install
  ├── eslint.config.mjs, .prettierrc, .prettierignore ← shared tooling
  ├── .gitignore
  └── README.md

  ### Wiring edits (40 files)

  - `.taskfiles/frontend.yml`, `desktop.yml`, `e2e.yml`
  - `build.gradle`, `app/core/build.gradle`
- `eslint.config.mjs`, `frontend/package.json`, `.gitignore`,
`.prettierignore`
  - `docker/frontend/Dockerfile`
  - 8 `.github/workflows/*.yml`, plus `.github/dependabot.yml`,
    `.github/config/.files.yaml`, `.github/labeler-config-srvaroa.yml`
  - `scripts/translations/**`
- Docs: `AGENTS.md`, `CLAUDE.md`, `ADDING_TOOLS.md`,
`DeveloperGuide.md`,
`WINDOWS_SIGNING.md`, `devGuide/HowToAddNewLanguage.md`,
`frontend/README.md`,
    `frontend/editor/DeveloperGuide.md`

Plus 3 renamed + edited: `editor/vite.config.ts` (env path +
node_modules
  walk-up), `editor/scripts/setup-env.mts` (renamed from `.ts` for
`import.meta.url`), `editor/scripts/build-provisioner.mjs` (resolve
src-tauri
  relative to script).

  ### Verification

  | Check | Result |
  |---|---|
  | `task frontend:typecheck:all` (6 variants) | exit 0 |
  | `task frontend:lint` (eslint + dpdm) | exit 0 |
  | `task frontend:format:check` | exit 0 |
  | `task frontend:test` | 657 tests pass, 50 files |
| `task frontend:build:{core,proprietary,saas,desktop,prototypes}` | all
green |
| `task desktop:build` | full Tauri pipeline →
`Stirling-PDF_2.11.0_x64_en-US.msi` |
  | `playwright test --list --project=stubbed` | 172 tests discovered |

`task desktop:build` exercises the heaviest path — Rust + WiX + MSI
bundle
against the moved `editor/src-tauri/`. If anything in the restructure
was
  wrong it wouldn't have built.

  ### Test plan

  - [ ] `frontend-validation.yml` green
  - [ ] `e2e-stubbed.yml` green
  - [ ] `tauri-build.yml` green on at least one platform
  - [ ] `check_toml.yml` runs on a translation-touching PR

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-22 13:40:34 +01:00
Anthony StirlingandGitHub 48027ee9d6 JDK25 - Integrate Stirling-Tools JPDFium for PDF operations (#6387) 2026-05-22 13:19:46 +01:00
22d56bb1fb test Depot runners for team commits (#6394)
Co-authored-by: ConnorYoh <40631091+ConnorYoh@users.noreply.github.com>
2026-05-21 16:54:20 +01:00
9d081d1792 SaaS Consolidation (#6384)
Co-authored-by: ConnorYoh <40631091+ConnorYoh@users.noreply.github.com>
2026-05-21 16:05:35 +01:00
089de247b4 Updated Spanish translation (#6309)
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Ludy <Ludy87@users.noreply.github.com>
Co-authored-by: Anthony Stirling <77850077+Frooodle@users.noreply.github.com>
2026-05-21 13:16:07 +01:00
dependabot[bot]andGitHub 0c3a512b21 build(deps): bump softprops/action-gh-release from 2.6.1 to 3.0.0 (#6346)
Signed-off-by: dependabot[bot] <support@github.com>
2026-05-20 14:34:16 +01:00
dependabot[bot]andGitHub 707cfa3140 build(deps): bump devalue from 5.6.4 to 5.8.1 in /frontend (#6367)
Signed-off-by: dependabot[bot] <support@github.com>
2026-05-20 14:18:43 +01:00
dependabot[bot]andGitHub ea91a35518 build(deps): bump org.postgresql:postgresql from 42.7.10 to 42.7.11 (#6375)
Signed-off-by: dependabot[bot] <support@github.com>
2026-05-20 14:17:36 +01:00
dependabot[bot]andGitHub 5a5aada2d8 build(deps): bump pypdf from 6.10.2 to 6.11.0 in /testing/cucumber (#6376)
Signed-off-by: dependabot[bot] <support@github.com>
2026-05-20 14:17:18 +01:00
dependabot[bot]andGitHub a4ef70eaf8 build(deps): bump sigstore/cosign-installer from 4.1.1 to 4.1.2 (#6377)
Signed-off-by: dependabot[bot] <support@github.com>
2026-05-20 14:17:04 +01:00
dependabot[bot]andGitHub d76d25654a build(deps): bump actions/dependency-review-action from 4.9.0 to 5.0.0 (#6378)
Signed-off-by: dependabot[bot] <support@github.com>
2026-05-20 14:16:48 +01:00
dependabot[bot]andGitHub 31b00b7d06 build(deps): bump github/codeql-action from 4.35.3 to 4.35.4 (#6379)
Signed-off-by: dependabot[bot] <support@github.com>
2026-05-20 14:16:42 +01:00
dependabot[bot]andGitHub f00c12ad8a build(deps): bump step-security/harden-runner from 2.19.0 to 2.19.1 (#6380)
Signed-off-by: dependabot[bot] <support@github.com>
2026-05-20 14:16:38 +01:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
1f5c046362 build(deps): bump tokio from 1.49.0 to 1.50.0 in /frontend/src-tauri in the tokio group across 1 directory (#6296)
Bumps the tokio group with 1 update in the /frontend/src-tauri
directory: [tokio](https://github.com/tokio-rs/tokio).

Updates `tokio` from 1.49.0 to 1.50.0
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/tokio-rs/tokio/releases">tokio's
releases</a>.</em></p>
<blockquote>
<h2>Tokio v1.50.0</h2>
<h1>1.50.0 (Mar 3rd, 2026)</h1>
<h3>Added</h3>
<ul>
<li>net: add <code>TcpStream::set_zero_linger</code> (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7837">#7837</a>)</li>
<li>rt: add <code>is_rt_shutdown_err</code> (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7771">#7771</a>)</li>
</ul>
<h3>Changed</h3>
<ul>
<li>io: add optimizer hint that <code>memchr</code> returns in-bounds
pointer (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7792">#7792</a>)</li>
<li>io: implement vectored writes for <code>write_buf</code> (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7871">#7871</a>)</li>
<li>runtime: panic when <code>event_interval</code> is set to 0 (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7838">#7838</a>)</li>
<li>runtime: shorten default thread name to fit in Linux limit (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7880">#7880</a>)</li>
<li>signal: remember the result of <code>SetConsoleCtrlHandler</code>
(<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7833">#7833</a>)</li>
<li>signal: specialize windows <code>Registry</code> (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7885">#7885</a>)</li>
</ul>
<h3>Fixed</h3>
<ul>
<li>io: always cleanup <code>AsyncFd</code> registration list on
deregister (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7773">#7773</a>)</li>
<li>macros: remove (most) local <code>use</code> declarations in
<code>tokio::select!</code> (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7929">#7929</a>)</li>
<li>net: fix <code>GET_BUF_SIZE</code> constant for <code>target_os =
&quot;android&quot;</code> (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7889">#7889</a>)</li>
<li>runtime: avoid redundant unpark in current_thread scheduler (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7834">#7834</a>)</li>
<li>runtime: don't park in <code>current_thread</code> if
<code>before_park</code> defers waker (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7835">#7835</a>)</li>
<li>io: fix write readiness on ESP32 on short writes (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7872">#7872</a>)</li>
<li>runtime: wake deferred tasks before entering
<code>block_in_place</code> (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7879">#7879</a>)</li>
<li>sync: drop rx waker when oneshot receiver is dropped (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7886">#7886</a>)</li>
<li>runtime: fix double increment of <code>num_idle_threads</code> on
shutdown (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7910">#7910</a>,
<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7918">#7918</a>,
<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7922">#7922</a>)</li>
</ul>
<h3>Unstable</h3>
<ul>
<li>fs: check for io-uring opcode support (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7815">#7815</a>)</li>
<li>runtime: avoid lock acquisition after uring init (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7850">#7850</a>)</li>
</ul>
<h3>Documented</h3>
<ul>
<li>docs: update outdated unstable features section (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7839">#7839</a>)</li>
<li>io: clarify the behavior of <code>AsyncWriteExt::shutdown()</code>
(<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7908">#7908</a>)</li>
<li>io: explain how to flush stdout/stderr (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7904">#7904</a>)</li>
<li>io: fix incorrect and confusing <code>AsyncWrite</code>
documentation (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7875">#7875</a>)</li>
<li>rt: clarify the documentation of <code>Runtime::spawn</code> (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7803">#7803</a>)</li>
<li>rt: fix missing quotation in docs (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7925">#7925</a>)</li>
<li>runtime: correct the default thread name in docs (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7896">#7896</a>)</li>
<li>runtime: fix <code>event_interval</code> doc (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7932">#7932</a>)</li>
<li>sync: clarify RwLock fairness documentation (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7919">#7919</a>)</li>
<li>sync: clarify that <code>recv</code> returns <code>None</code> once
closed and no more messages (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7920">#7920</a>)</li>
<li>task: clarify when to use <code>spawn_blocking</code> vs dedicated
threads (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7923">#7923</a>)</li>
<li>task: doc that task drops before <code>JoinHandle</code> completion
(<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7825">#7825</a>)</li>
<li>signal: guarantee that listeners never return <code>None</code> (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7869">#7869</a>)</li>
<li>task: fix task module feature flags in docs (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7891">#7891</a>)</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/tokio-rs/tokio/commit/0273e45ead199dac7725faee1e3dc35a9c8753ab"><code>0273e45</code></a>
chore: prepare Tokio v1.50.0 (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7934">#7934</a>)</li>
<li><a
href="https://github.com/tokio-rs/tokio/commit/e3ee4e58dc9bb7accf26dfd51b0a2146922b5269"><code>e3ee4e5</code></a>
chore: prepare tokio-macros v2.6.1 (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7943">#7943</a>)</li>
<li><a
href="https://github.com/tokio-rs/tokio/commit/8c980ea75a0f8dd2799403777db700c2e8f4cda4"><code>8c980ea</code></a>
io: add <code>write_all_vectored</code> to <code>tokio-util</code> (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7768">#7768</a>)</li>
<li><a
href="https://github.com/tokio-rs/tokio/commit/e35fd6d6b7d9a8ba37ee621835ef91372c2565cb"><code>e35fd6d</code></a>
ci: fix patch during clippy step (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7935">#7935</a>)</li>
<li><a
href="https://github.com/tokio-rs/tokio/commit/03fe44c10302fdb55c29dbe5b08d4f8769c80272"><code>03fe44c</code></a>
runtime: fix <code>event_interval</code> doc (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7932">#7932</a>)</li>
<li><a
href="https://github.com/tokio-rs/tokio/commit/d18e5dfbb0cdc28725bebb28cde80a6c11ee32bc"><code>d18e5df</code></a>
io: fix race in <code>Mock::poll_write</code> (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7882">#7882</a>)</li>
<li><a
href="https://github.com/tokio-rs/tokio/commit/f21f2693f02aec9a876ac2bd21566c85e15b682e"><code>f21f269</code></a>
runtime: fix race condition during the blocking pool shutdown (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7922">#7922</a>)</li>
<li><a
href="https://github.com/tokio-rs/tokio/commit/d81e8f0acbdd7d866bce4f733b3545fd834c7840"><code>d81e8f0</code></a>
macros: remove (most) local <code>use</code> declarations in
<code>tokio::select!</code> (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7929">#7929</a>)</li>
<li><a
href="https://github.com/tokio-rs/tokio/commit/25e7f2641ef2555d688c267059431a2802805f1d"><code>25e7f26</code></a>
rt: fix missing quotation in docs (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7925">#7925</a>)</li>
<li><a
href="https://github.com/tokio-rs/tokio/commit/e1a91ef114a301b542d810abab9956f2868861b9"><code>e1a91ef</code></a>
util: fix typo in docs (<a
href="https://redirect.github.com/tokio-rs/tokio/issues/7926">#7926</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/tokio-rs/tokio/compare/tokio-1.49.0...tokio-1.50.0">compare
view</a></li>
</ul>
</details>
<br />

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-20 14:16:18 +01:00
dependabot[bot]andGitHub 1787443102 build(deps): bump fast-uri from 3.0.6 to 3.1.2 in /devTools (#6332)
Signed-off-by: dependabot[bot] <support@github.com>
2026-05-20 14:15:08 +01:00
dependabot[bot]andGitHub 8fb281525b build(deps): bump quinn-proto from 0.11.13 to 0.11.14 in /frontend/src-tauri (#6322)
Signed-off-by: dependabot[bot] <support@github.com>
2026-05-20 14:13:31 +01:00
LudyandGitHub 4c3be4992f deps: align Bouncy Castle modules to shared version property (#6362) 2026-05-20 14:11:43 +01:00
dependabot[bot]andGitHub 2790467241 build(deps): bump urllib3 from 2.6.3 to 2.7.0 in /testing/cucumber in the pip group across 1 directory (#6352)
Signed-off-by: dependabot[bot] <support@github.com>
2026-05-20 14:11:11 +01:00
dependabot[bot]andGitHub dd770dfa76 build(deps): bump rustls-webpki from 0.103.9 to 0.103.13 in /frontend/src-tauri (#6323)
Signed-off-by: dependabot[bot] <support@github.com>
2026-05-20 14:10:56 +01:00
ConnorYohandGitHub b146d9994d fix(task): make task dev / task dev:all work on Windows (#6392)
## Summary

`#6145` (port picker) and `#6244` (gradle unification) combined to break
`task dev` / `task dev:all` on Windows. Two independent regressions,
both addressed here.

### 1. `find-free-port.ps1` panic (`#6145`)

```
$ task --dry dev
The argument 'scriptsfind-free-port.ps1' to the -File parameter does not exist.
panic: ended up with a non-nil exitStatus.err but a zero exitStatus.code
```

- **Backslash stripped.** `Taskfile.yml` had
`scripts\find-free-port.ps1`. go-task pipes the `sh:` block through
mvdan/sh, which treats `\` as a POSIX escape and silently drops it,
leaving `scriptsfind-free-port.ps1`. PowerShell can't find the file,
exits non-zero, mvdan/sh panics on the inconsistent exit status.
Switched to a forward slash.
- **`-Preferred 8080,5173` is brittle.** Relies on PowerShell parsing
the comma-list into `[int[]]`. Dropped the named flag; switched the
script's `param` to `[Parameter(ValueFromRemainingArguments =
$true)][int[]]$Preferred`; pass each port as its own positional token
(matching how the bash variant is called).

### 2. `bash gradlew` can't find Java on Windows (`#6244`)

```
[backend:dev] ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH.
```

`#6244` unified all gradle invocations under `bash gradlew` on the
assumption that Git-Bash inherits Windows-side env. It doesn't (and
neither does WSL bash, which can also shadow `bash` on a developer's
PATH). The standard Adoptium/Temurin installer sets `JAVA_HOME` in the
Windows env only, so `bash gradlew` fails before Spring Boot even loads.

Restored the per-platform branch for every backend task: `cmd /c
".\gradlew.bat ..."` on Windows, `./gradlew ...` on Linux/macOS. Two
Windows-specific gotchas to be aware of for future edits:

- mvdan/sh strips `\` outside of double quotes, so `cmd /c
.\gradlew.bat` ends up as `.gradlew.bat`. The entire payload must be
wrapped in double quotes (`cmd /c "..."`).
- Modern Windows excludes cwd from cmd.exe's search path, so the leading
`.\` is required — bare `cmd /c gradlew.bat` errors with "is not
recognized" even from the repo root.

## Test plan

- [x] `task --dry dev` / `task --dry dev:all` on Windows resolve ports
cleanly and dispatch the inner tasks
- [x] `powershell -NoProfile -File scripts/find-free-port.ps1 8080 5173
5001` prints three ports
- [x] `task backend:dev PORT=8081` on Windows starts Spring Boot in ~9s;
`/api/v1/info/status` returns `{"status":"UP"}`
- [x] `task dev:all` on Windows brings up all three services healthy:
backend `/api/v1/info/status` 200, engine `/health` 200, frontend `/`
200
- [ ] Linux / macOS path unaffected (only the Windows branches changed
in behaviour)
2026-05-20 12:57:41 +00:00
2399da6893 migrate exportUpdatedPages from bytes to stream (#6201)
Co-authored-by: ConnorYoh <40631091+ConnorYoh@users.noreply.github.com>
2026-05-19 23:03:44 +01:00
EthanHealy01andGitHub 2c15044dfa Snag/disable logo drag (#6393) 2026-05-19 18:13:58 +00:00
ConnorYohandGitHub 31f6ea4b25 perf(frontend): stabilize hot-path context subscriptions to fix excessive rerenders (#6373)
## Summary

The frontend was rerendering excessively across many interactions —
typing, clicking tools, opening modals, toggling the sidebar — because
**multiple compounding ref-instability cascades defeated `memo()` checks
in hot paths**. This PR fixes the cascades structurally.

Seven focused commits, low-to-high blast radius:

1. `perf(contexts): memoize BannerContext provider value`
2. `perf(contexts): memoize CommentAuthor and ActiveDocument provider
values`
3. `perf(contexts): memoize AppConfigContext provider value`
4. `perf(useToolManagement): stop spreading tool entries to keep refs
stable` — **root-cause fix**
5. `refactor(ToolPicker): hoist module-scope styles and helpers`
6. `feat(ToolWorkflowContext): add ref-stable Actions and Data subset
contexts` — additive
7. `perf(tools): migrate hot consumers to slim contexts and wrap in
memo()`

(Plus `style: apply prettier formatting` for CI.)

## What was wrong

Whenever something high-up in the tree caused a render, a chain of
unstable references propagated downward and forced every `ToolButton` to
re-execute its full body (hooks, derived computations, hook
subscriptions to other contexts). The chain:

- **4 unstable Context providers** (`Banner`, `CommentAuthor`,
`ActiveDocument`, `AppConfig`) were passing fresh `value={{ … }}`
objects on every render. Every consumer rerendered on every ancestor
render.
- **`useToolManagement.toolRegistry`** spread `{...baseTool, name,
description}` — a no-op spread that manufactured a new tool object
identity on every memo recompute.
- **The big `ToolWorkflowContext`** (25+ fields including
`state.searchQuery`) rebuilt its entire value on every
keystroke/click/toggle, forcing every `useToolWorkflow()` consumer (~36
files) to rerender.
- **`useToolNavigation`** transitively subscribed every `ToolButton` to
the full workflow context.
- **`ToolButton` & `ToolPicker`** weren't `memo()`-wrapped, so nothing
checked.
- **`ToolPanel`** passed inline `onSelect={(id) =>
handleToolSelect(...)}` — fresh ref every render, defeats child
memoization.
- **`ToolPicker`** allocated inline styles / `[]` / `toTitleCase` inside
the function body — churned `useToolSections`'s internal memo.

## Interaction matrix — what improves

The PR fixes the underlying ref-stability problem; the same fix benefits
*every* interaction that previously triggered the cascade:

| Interaction | Before | After |
|---|---|---|
| **Typing in tool search** | All visible buttons rerender per keystroke
| Only buttons whose matched-text changes rerender |
| **Clicking a tool** | All 36 `useToolWorkflow()` consumers rerender |
Only previously-selected and newly-selected buttons rerender (via
`isSelected` prop) |
| **Toggling sidebar / panel mode / reader mode** | Every tool button
rerenders | Tool components stay still (slim context doesn't see UI
state) |
| **Switching workbench / navigation** | `handleToolSelect` identity
changes → cascades through `onSelect` props | Ref-stabilized in Actions
context. Identity stable. Children's memo bails |
| **Modal/dialog open/close** | AppConfig churns → every `useAppConfig`
consumer rerenders (ToolButton reads `premiumEnabled`) | AppConfig
memoized; consumers rerender only when config changes |
| **Banner show/hide** | BannerProvider value churns → every consumer
rerenders on any ancestor render | Memoized; AppLayout rerenders only
when banner content changes |
| **Any state update high in the tree** | Compounding cascade defeats
memo everywhere | Stable subscriptions; memo bails out |

## Evidence

Per-keystroke prop instability on `ToolButton` (cleanest measurable
signal, captured via custom memo comparators logging which prop refs
differ):

| | `tool` ref diffs | `onSelect` ref diffs | `matchedSynonym` value
diffs | Total |
|---|---|---|---|---|
| Before | 18 | 18 | 6 | **42** |
| After  | 0  | 0  | 6 | **6 (all legitimate)** |

→ **86% reduction** in spurious per-keystroke prop instability. The 6
remaining matched-synonym diffs are correct (different substring
highlighted per keystroke).

Context value rebuild counts during a keystroke (verified with
instrumented `useMemo` factories): `useToolWorkflowData=0`,
`useToolWorkflowActions=0`, `AppConfigContext=0`.

The same stabilization applies to click/toggle/modal interactions — they
were all driven by the same cascading invalidations.

## Honest caveat on render-count metrics

`React.Profiler` counts and function-body execution counts in **dev
mode** came back identical before vs after (StrictMode + concurrent
rendering + Mantine internal commits dominate the numbers). The PR's
value is measured against the **prop-stability signal** above, not
Profiler counts. Production builds — where StrictMode doesn't
double-render and Mantine internals aren't constantly committing — will
show memo bail out properly.

## Risk × benefit

| # | Commit | Risk | Benefit |
|---|--------|------|---------|
| 1 | BannerContext memo | ⬛ Trivial | 🟦 Small |
| 2 | CommentAuthor + ActiveDocument memo | ⬛ Trivial | 🟦 Small |
| 3 | AppConfig memo | ⬛ Trivial | 🟦 Moderate (wide consumer base) |
| 4 | useToolManagement spread removal | ⬛ Trivial | 🟥 **High (root
cause)** |
| 5 | ToolPicker hoist | ⬛ Trivial | 🟦 Small |
| 6 | ToolWorkflowContext split | 🟧 Low-Med | 🟥 **High (foundation)** |
| 7 | Hot consumer migration + memo | 🟧 Low-Med | 🟥 **High
(actualization)** |

Commit 6 introduces an invariant: ref-stabilized callbacks in the
Actions context must only be invoked from event handlers (post-commit),
never during render. All current call sites comply.

## Test plan

- [x] `npx playwright test --project=stubbed` — 145 / 6 skipped / 0
failed before and after.
- [x] Targeted regression: `main-dashboard`, `tool-search`, `navigation`
— 11/11 passing.
- [x] CI passing on commits (one infrastructure flake on
`docker-compose-tests` — "No space left on device" — unrelated;
rerunning).
- [ ] Manual sanity check in a dev build after merge.

## What this enables

The same Actions + Data subset-context pattern can be applied to
`FileContext`, `NavigationContext`, and other big contexts. The
foundation is in place.
2026-05-19 15:56:50 +00:00
Anthony StirlingandGitHub f9fbc37800 add translations (#6390) 2026-05-19 11:25:13 +01:00
6b9567cf38 Split and delete forms (#6277)
Delete orphaned forms when removing pages and maintain forms correctly
when splitting

---------

Co-authored-by: ConnorYoh <40631091+ConnorYoh@users.noreply.github.com>
2026-05-18 11:14:55 +00:00
c731d5fd5d UI redesign staging (#6149)
Co-authored-by: Reece Browne <74901996+reecebrowne@users.noreply.github.com>
Co-authored-by: James Brunton <jbrunton96@gmail.com>
2026-05-15 15:36:50 +01:00
James BruntonandGitHub beb99e273b Improve edit agent's knowledge of tools (#6356)
# Description of Changes
Give Edit Agent access to descriptions of the request from the Java API.
This opens the door to us better documenting our Java APIs to give the
stirling engine better knowledge of what the various tools are and how
to use them.

Also improves the tool selection sub-agent to get the tool parameters
and descriptions so it can more intelligently decide which operations
should be used to fulfil the user's request. Also provides it more
encouragement to string together multiple operations if necessary.
2026-05-14 18:30:39 +00:00
ece1bb6865 Feature/pdf to markdown agent (#6271)
Co-authored-by: James Brunton <jbrunton96@gmail.com>
2026-05-14 16:20:45 +00:00
2164 changed files with 46172 additions and 9854 deletions
+2 -2
View File
@@ -34,8 +34,8 @@ frontend/playwright-report/
# Tauri/desktop builds
src-tauri/target/
src-tauri/dist/
frontend/src-tauri/target/
frontend/src-tauri/dist/
frontend/editor/src-tauri/target/
frontend/editor/src-tauri/dist/
# IDE and editor
.idea/
+34
View File
@@ -0,0 +1,34 @@
###############################################################################
# Stirling-PDF SaaS local environment template.
#
# Copy this file to `.env.saas.local` (gitignored) and fill in real values.
# Loaded by `task backend:dev:saas` via Taskfile's `dotenv:` directive, then
# read by Spring Boot's `${...}` placeholders in application-saas.properties
# and application-dev.properties.
#
# DO NOT commit `.env.saas.local`. Only `.env.saas.example` is checked in.
###############################################################################
# ---------- Supabase project ----------
# Project reference (the subdomain part of <ref>.supabase.co). Required.
# Example dev project:
SAAS_DB_PROJECT_REF=
# Edge function secret used by billing/license rollup calls.
SUPABASE_EDGE_FUNCTION_SECRET=
# ---------- Database (saas profile) ----------
# Direct JDBC URL to the Supabase Postgres. Required when running the plain
# `saas` profile (i.e. without `--spring.profiles.include=dev`).
# Example: jdbc:postgresql://db.<project-ref>.supabase.co:5432/postgres
SAAS_DB_URL=
SAAS_DB_USERNAME=postgres
SAAS_DB_PASSWORD=
# ---------- Database (dev profile overrides) ----------
# Used when `--spring.profiles.include=dev` is active. The dev profile
# defaults the URL/username to the shared dev Supabase project, but the
# password must still be provided here.
SAAS_DEV_DB_URL=
SAAS_DEV_DB_USERNAME=postgres
SAAS_DEV_DB_PASSWORD=
+7 -8
View File
@@ -8,7 +8,6 @@ openapi: &openapi
docker-base: &docker-base
- docker/base/Dockerfile
- ".github/workflows/push-docker-base.yml"
docker: &docker
- docker/embedded/Dockerfile
@@ -58,12 +57,12 @@ frontend: &frontend
# Files that affect the Tauri desktop bundle. Gate the multi-OS Tauri build
# job on changes to any of these.
tauri: &tauri
- frontend/src-tauri/**
- frontend/src/desktop/**
- frontend/tsconfig.desktop.json
- frontend/editor/src-tauri/**
- frontend/editor/src/desktop/**
- frontend/editor/tsconfig.desktop.vite.json
- frontend/package.json
- frontend/package-lock.json
- frontend/vite.config.ts
- frontend/editor/vite.config.ts
- .github/workflows/tauri-build.yml
# Files that affect the AI engine (Python tool models, fixers, tests). Gate
@@ -78,7 +77,7 @@ licenses-frontend: &licenses-frontend
- ".github/workflows/frontend-backend-licenses-update.yml"
- "frontend/package.json"
- "frontend/package-lock.json"
- "frontend/scripts/generate-licenses.js"
- "frontend/editor/scripts/generate-licenses.js"
licenses-backend: &licenses-backend
- ".github/workflows/frontend-backend-licenses-update.yml"
@@ -88,8 +87,8 @@ licenses-backend: &licenses-backend
# Playwright job on changes to any of these on PRs.
proprietary: &proprietary
- app/proprietary/**
- frontend/src/proprietary/**
- frontend/src/core/tests/enterprise/**
- frontend/editor/src/proprietary/**
- frontend/editor/src/core/tests/enterprise/**
- testing/compose/docker-compose-keycloak-oauth.yml
- testing/compose/docker-compose-keycloak-saml.yml
- testing/compose/keycloak-realm-oauth.json
+5 -3
View File
@@ -25,6 +25,8 @@ updates:
- "/docker/frontend"
- "/docker/base"
- "/docker/engine"
- "/docker/unoserver"
- "/engine"
schedule:
interval: "weekly"
cooldown:
@@ -126,9 +128,9 @@ updates:
- package-ecosystem: cargo
directories:
- /frontend/src-tauri
- /frontend/src-tauri/thumbnail-handler
- /frontend/src-tauri/provisioner
- /frontend/editor/src-tauri
- /frontend/editor/src-tauri/thumbnail-handler
- /frontend/editor/src-tauri/provisioner
schedule:
interval: "weekly"
cooldown:
+3 -3
View File
@@ -51,7 +51,7 @@ labels:
- label: 'Translation'
files:
- 'frontend/public/locales/[a-zA-Z]{2}-[a-zA-Z\-]{2,7}/translation.toml'
- 'frontend/editor/public/locales/[a-zA-Z]{2}-[a-zA-Z\-]{2,7}/translation.toml'
- 'scripts/ignore_translation.toml'
- 'scripts/remove_translation_keys.sh'
- 'scripts/replace_translation_line.sh'
@@ -69,8 +69,8 @@ labels:
- label: 'Tauri'
files:
- 'frontend/src-tauri/**'
- 'frontend/src-tauri/.*'
- 'frontend/editor/src-tauri/**'
- 'frontend/editor/src-tauri/.*'
- label: 'engine'
files:
+35 -9
View File
@@ -23,9 +23,13 @@ permissions:
pull-requests: write
jobs:
pick:
uses: ./.github/workflows/_runner-pick.yml
check-pr:
if: (github.event_name == 'pull_request' && github.event.action != 'closed') || github.event_name == 'workflow_dispatch'
runs-on: ubuntu-latest
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
outputs:
should_deploy: ${{ steps.decide.outputs.should_deploy }}
is_fork: ${{ steps.resolve.outputs.is_fork }}
@@ -35,7 +39,7 @@ jobs:
pr_ref: ${{ steps.resolve.outputs.ref }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -97,8 +101,8 @@ jobs:
echo "allow_fork=${allow_fork:-false}" >> $GITHUB_OUTPUT
deploy-v2-pr:
needs: check-pr
runs-on: ubuntu-latest
needs: [pick, check-pr]
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
if: needs.check-pr.outputs.should_deploy == 'true' && (needs.check-pr.outputs.is_fork == 'false' || needs.check-pr.outputs.allow_fork == 'true')
# Concurrency control - only one deployment per PR at a time
concurrency:
@@ -108,10 +112,14 @@ jobs:
contents: read
issues: write
pull-requests: write
id-token: write
env:
USE_DEPOT: ${{ needs.pick.outputs.is_fork != 'true' }}
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -179,7 +187,12 @@ jobs:
token: ${{ secrets.GITHUB_TOKEN }}
fetch-depth: 0 # Fetch full history for commit hash detection
- name: Set up Depot CLI
if: env.USE_DEPOT == 'true'
uses: depot/setup-action@15c09a5f77a0840ad4bce955686522a257853461 # v1.0.0
- name: Set up Docker Buildx
if: env.USE_DEPOT != 'true'
uses: docker/setup-buildx-action@4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd # v4.0.0
- name: Get version number
@@ -224,8 +237,20 @@ jobs:
echo "Image needs to be built"
fi
- name: Build and push V2 image
if: steps.check-image.outputs.exists == 'false'
- name: Build and push V2 image (Depot)
if: env.USE_DEPOT == 'true' && steps.check-image.outputs.exists == 'false'
uses: depot/build-push-action@5f3b3c2e5a00f0093de47f657aeaefcedff27d18 # v1.16.0
with:
project: ${{ vars.DEPOT_PROJECT_ID }}
context: .
file: ./docker/embedded/Dockerfile
push: true
tags: ${{ secrets.DOCKER_HUB_USERNAME }}/test:v2-${{ steps.commit-hash.outputs.app_short }}
build-args: VERSION_TAG=v2-alpha
platforms: linux/amd64
- name: Build and push V2 image (Docker fork fallback)
if: env.USE_DEPOT != 'true' && steps.check-image.outputs.exists == 'false'
uses: docker/build-push-action@bcafcacb16a39f128d818304e6c9c0c18556b85f # v7.1.0
with:
context: .
@@ -351,7 +376,8 @@ jobs:
cleanup-v2-deployment:
if: github.event.action == 'closed'
runs-on: ubuntu-latest
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
permissions:
contents: read
issues: write
@@ -359,7 +385,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -34,8 +34,12 @@ permissions:
pull-requests: read
jobs:
pick:
uses: ./.github/workflows/_runner-pick.yml
check-comment:
runs-on: ubuntu-latest
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
permissions:
issues: write
if: |
@@ -70,7 +74,7 @@ jobs:
enable_prototypes: ${{ steps.check-prototypes-flag.outputs.enable_prototypes }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -175,15 +179,19 @@ jobs:
}
deploy-pr:
needs: check-comment
runs-on: ubuntu-latest
needs: [pick, check-comment]
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
permissions:
issues: write
pull-requests: write
id-token: write
env:
USE_DEPOT: ${{ needs.pick.outputs.is_fork != 'true' }}
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -232,7 +240,12 @@ jobs:
MAVEN_PUBLIC_URL: ${{ secrets.MAVEN_PUBLIC_URL }}
STIRLING_PDF_DESKTOP_UI: false
- name: Set up Depot CLI
if: env.USE_DEPOT == 'true'
uses: depot/setup-action@15c09a5f77a0840ad4bce955686522a257853461 # v1.0.0
- name: Set up Docker Buildx
if: env.USE_DEPOT != 'true'
uses: docker/setup-buildx-action@4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd # v4.0.0
- name: Login to Docker Hub
@@ -241,7 +254,22 @@ jobs:
username: ${{ secrets.DOCKER_HUB_USERNAME }}
password: ${{ secrets.DOCKER_HUB_API }}
- name: Build and push PR-specific image
- name: Build and push PR-specific image (Depot)
if: env.USE_DEPOT == 'true'
uses: depot/build-push-action@5f3b3c2e5a00f0093de47f657aeaefcedff27d18 # v1.16.0
with:
project: ${{ vars.DEPOT_PROJECT_ID }}
context: .
file: ./docker/embedded/Dockerfile
push: true
tags: ${{ secrets.DOCKER_HUB_USERNAME }}/test:pr-${{ needs.check-comment.outputs.pr_number }}
build-args: |
VERSION_TAG=alpha
PROTOTYPES_BUILD=${{ needs.check-comment.outputs.enable_prototypes }}
platforms: linux/amd64
- name: Build and push PR-specific image (Docker fork fallback)
if: env.USE_DEPOT != 'true'
uses: docker/build-push-action@bcafcacb16a39f128d818304e6c9c0c18556b85f # v7.1.0
with:
context: .
@@ -255,8 +283,19 @@ jobs:
PROTOTYPES_BUILD=${{ needs.check-comment.outputs.enable_prototypes }}
platforms: linux/amd64
- name: Build and push engine image
if: needs.check-comment.outputs.enable_prototypes == 'true'
- name: Build and push engine image (Depot)
if: env.USE_DEPOT == 'true' && needs.check-comment.outputs.enable_prototypes == 'true'
uses: depot/build-push-action@5f3b3c2e5a00f0093de47f657aeaefcedff27d18 # v1.16.0
with:
project: ${{ vars.DEPOT_PROJECT_ID }}
context: ./engine
file: ./engine/Dockerfile
push: true
tags: ${{ secrets.DOCKER_HUB_USERNAME }}/test:engine-pr-${{ needs.check-comment.outputs.pr_number }}
platforms: linux/amd64
- name: Build and push engine image (Docker fork fallback)
if: env.USE_DEPOT != 'true' && needs.check-comment.outputs.enable_prototypes == 'true'
uses: docker/build-push-action@bcafcacb16a39f128d818304e6c9c0c18556b85f # v7.1.0
with:
context: ./engine
@@ -471,10 +510,11 @@ jobs:
handle-label-commands:
if: ${{ github.event.issue.pull_request != null }}
runs-on: ubuntu-latest
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
+1 -1
View File
@@ -21,7 +21,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
+70
View File
@@ -0,0 +1,70 @@
name: _runner-pick
# Tiny reusable workflow that classifies the trigger as either a "fork PR
# from an untrusted contributor" or a "trusted commit" so downstream jobs
# can pick a runner class without each one duplicating the 200-char gate
# expression in their own `runs-on:`.
#
# Caller pattern:
#
# jobs:
# pick:
# uses: ./.github/workflows/_runner-pick.yml
#
# real-work:
# needs: pick
# runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-8' }}
# steps: [...]
#
# Output:
# is_fork: "true" when the trigger is a pull_request from a fork or an
# untrusted author_association, "false" otherwise.
on:
workflow_call:
outputs:
is_fork:
description: '"true" if the trigger is an untrusted fork PR.'
value: ${{ jobs.pick.outputs.is_fork }}
permissions:
contents: read
jobs:
pick:
runs-on: ubuntu-latest
timeout-minutes: 1
outputs:
is_fork: ${{ steps.decide.outputs.is_fork }}
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
- name: Classify the trigger
id: decide
env:
PR_NUMBER: ${{ github.event.pull_request.number }}
HEAD_REPO_FORK: ${{ github.event.pull_request.head.repo.fork }}
AUTHOR_ASSOC: ${{ github.event.pull_request.author_association }}
run: |
set -eu
if [ -z "${PR_NUMBER:-}" ]; then
# Not a pull_request event at all (push, schedule, workflow_dispatch,
# workflow_call from a non-PR trigger) -> trusted by default.
echo "is_fork=false" >> "$GITHUB_OUTPUT"
exit 0
fi
if [ "${HEAD_REPO_FORK}" = "true" ]; then
echo "is_fork=true" >> "$GITHUB_OUTPUT"
exit 0
fi
case "${AUTHOR_ASSOC}" in
OWNER|MEMBER|COLLABORATOR)
echo "is_fork=false" >> "$GITHUB_OUTPUT"
;;
*)
echo "is_fork=true" >> "$GITHUB_OUTPUT"
;;
esac
+4 -2
View File
@@ -17,14 +17,16 @@ jobs:
permissions:
contents: read
pull-requests: write
env:
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
- name: Checkout code
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Install uv
uses: astral-sh/setup-uv@08807647e7069bb48b6ef5acd8ec9567f424441b # v8.1.0
+2 -2
View File
@@ -19,7 +19,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -87,7 +87,7 @@ jobs:
- name: AI PR Title Analysis
if: steps.actor.outputs.is_repo_dev == 'true'
id: ai-title-analysis
uses: actions/ai-inference@e09e65981758de8b2fdab13c2bfb7c7d5493b0b6 # v2.0.7
uses: actions/ai-inference@17ff458cb182449bbb2e43701fcd98f6af8f6570 # v2.1.0
with:
model: openai/gpt-4o
system-prompt-file: ".github/config/system-prompt.txt"
+4 -4
View File
@@ -26,7 +26,7 @@ jobs:
jar_sha256: ${{ steps.hashes.outputs.jar_sha256 }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -70,12 +70,12 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
- name: Checkout repository (for PKGBUILD templates)
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Update stirling-pdf-desktop PKGBUILD
env:
@@ -106,7 +106,7 @@ jobs:
- name: Publish stirling-pdf-desktop to AUR
if: ${{ github.event_name == 'release' || inputs.dry_run == false }}
uses: KSXGitHub/github-actions-deploy-aur@v4.1.3
uses: KSXGitHub/github-actions-deploy-aur@da03e160361ce01bf087e790b6ffd196d7dccff7 # v4.1.3
with:
pkgname: stirling-pdf-desktop
pkgbuild: .github/aur/stirling-pdf-desktop/PKGBUILD
+1 -1
View File
@@ -16,7 +16,7 @@ jobs:
pull-requests: write
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
+11 -5
View File
@@ -1,7 +1,7 @@
name: Backend build, format check, and coverage
# Reusable workflow called from build.yml. Runs the full backend build matrix
# (JDK 21/25 × spring-security on/off), Spotless formatting check, JUnit, and
# Reusable workflow called from build.yml. Runs the backend build matrix
# (JDK 25 × spring-security on/off), Spotless formatting check, JUnit, and
# posts Jacoco coverage to PRs.
on:
workflow_call:
@@ -13,16 +13,22 @@ permissions:
pull-requests: write
jobs:
pick:
uses: ./.github/workflows/_runner-pick.yml
build:
runs-on: ubuntu-latest
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-8' }}
env:
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
strategy:
fail-fast: false
matrix:
jdk-version: [21, 25]
jdk-version: [25]
spring-security: [true, false]
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
- name: Checkout repository
+20 -3
View File
@@ -17,11 +17,23 @@ name: Enterprise E2E (Playwright)
on:
workflow_call:
inputs:
depot_cores:
description: "Depot runner vCPU count (used in runs-on). Override for benchmarking."
required: false
type: string
default: "8"
push:
branches: ["main"]
schedule:
- cron: "0 4 * * *"
workflow_dispatch:
inputs:
depot_cores:
description: "Depot runner vCPU count (used in runs-on). Override for benchmarking."
required: false
type: string
default: "8"
# No `concurrency:` block here on purpose. When this workflow is called via
# workflow_call from build.yml, ${{ github.workflow }}/event_name/pr_number
@@ -35,16 +47,21 @@ permissions:
contents: read
jobs:
pick:
uses: ./.github/workflows/_runner-pick.yml
playwright-e2e-enterprise:
runs-on: ubuntu-latest
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || format('depot-ubuntu-24.04-{0}', inputs.depot_cores || '8') }}
timeout-minutes: 45
env:
PREMIUM_KEY: ${{ secrets.PREMIUM_KEY_ENTERPRISE }}
PREMIUM_ENABLED: "true"
SYSTEM_ENABLEANALYTICS: "false"
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
- name: Checkout repository
@@ -268,5 +285,5 @@ jobs:
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: playwright-report-enterprise-${{ github.run_id }}
path: frontend/playwright-report/
path: frontend/editor/playwright-report/
retention-days: 7
+7 -1
View File
@@ -46,7 +46,7 @@ jobs:
proprietary: ${{ steps.changes.outputs.proprietary }}
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
- name: Checkout repository
@@ -135,6 +135,7 @@ jobs:
permissions:
contents: read
packages: read
id-token: write
uses: ./.github/workflows/test-build-docker.yml
secrets: inherit
with:
@@ -197,6 +198,11 @@ jobs:
- dependency-review
runs-on: ubuntu-latest
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
- name: Verify every required job passed (or was legitimately skipped)
env:
RESULTS: |
+3 -1
View File
@@ -11,9 +11,11 @@ permissions:
jobs:
check-licence:
runs-on: ubuntu-latest
env:
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
+8 -2
View File
@@ -10,11 +10,17 @@ permissions:
contents: read
jobs:
pick:
uses: ./.github/workflows/_runner-pick.yml
check-generate-openapi-docs:
runs-on: ubuntu-latest
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
env:
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
+6 -6
View File
@@ -6,7 +6,7 @@ on:
pull_request_target:
types: [opened, synchronize, reopened]
paths:
- "frontend/public/locales/*/translation.toml"
- "frontend/editor/public/locales/*/translation.toml"
- ".github/scripts/check_language_toml.py"
- ".github/workflows/check_toml.yml"
@@ -27,7 +27,7 @@ jobs:
pull-requests: write # Allow writing to pull requests
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -76,7 +76,7 @@ jobs:
exit 1
fi
# Get changed files and filter for TOML translation files
gh pr view ${{ steps.get-pr-data.outputs.pr_number }} --json files -q ".files[].path" | grep -E '^frontend/public/locales/[a-zA-Z-]+/translation\.toml$' > changed_files.txt || echo "No matching TOML files found in PR"
gh pr view ${{ steps.get-pr-data.outputs.pr_number }} --json files -q ".files[].path" | grep -E '^frontend/editor/public/locales/[a-zA-Z-]+/translation\.toml$' > changed_files.txt || echo "No matching TOML files found in PR"
# Check if any files were found
if [ ! -s changed_files.txt ]; then
echo "No TOML translation files changed in this PR"
@@ -166,12 +166,12 @@ jobs:
// Determine reference file
let referenceFilePath;
if (changedFiles.includes("frontend/public/locales/en-GB/translation.toml")) {
if (changedFiles.includes("frontend/editor/public/locales/en-GB/translation.toml")) {
console.log("Using PR branch reference file.");
const { data: fileContent } = await github.rest.repos.getContent({
owner: prRepoOwner,
repo: prRepoName,
path: "frontend/public/locales/en-GB/translation.toml",
path: "frontend/editor/public/locales/en-GB/translation.toml",
ref: branch,
});
@@ -183,7 +183,7 @@ jobs:
const { data: fileContent } = await github.rest.repos.getContent({
owner: repoOwner,
repo: repoName,
path: "frontend/public/locales/en-GB/translation.toml",
path: "frontend/editor/public/locales/en-GB/translation.toml",
ref: "main",
});
+2 -2
View File
@@ -14,13 +14,13 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
- name: "Checkout Repository"
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: "Dependency Review"
uses: actions/dependency-review-action@2031cfc080254a8a887f58cffee85186f0e49e48 # v4.9.0
uses: actions/dependency-review-action@a1d282b36b6f3519aa1f3fc636f609c47dddb294 # v5.0.0
with:
config-file: "./.github/config/dependency-review-config.yml"
+49 -6
View File
@@ -10,22 +10,37 @@ permissions:
contents: read
jobs:
pick:
uses: ./.github/workflows/_runner-pick.yml
deploy-v2-on-push:
runs-on: ubuntu-latest
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
concurrency:
group: deploy-v2-push-V2
cancel-in-progress: true
permissions:
contents: read
id-token: write
env:
USE_DEPOT: ${{ needs.pick.outputs.is_fork != 'true' }}
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
- name: Checkout code
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Set up Depot CLI
if: env.USE_DEPOT == 'true'
uses: depot/setup-action@15c09a5f77a0840ad4bce955686522a257853461 # v1.0.0
- name: Set up Docker Buildx
if: env.USE_DEPOT != 'true'
uses: docker/setup-buildx-action@4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd # v4.0.0
- name: Get commit hashes for frontend and backend
@@ -90,8 +105,22 @@ jobs:
username: ${{ secrets.DOCKER_HUB_USERNAME }}
password: ${{ secrets.DOCKER_HUB_API }}
- name: Build and push frontend image
if: steps.check-frontend.outputs.exists == 'false'
- name: Build and push frontend image (Depot)
if: env.USE_DEPOT == 'true' && steps.check-frontend.outputs.exists == 'false'
uses: depot/build-push-action@5f3b3c2e5a00f0093de47f657aeaefcedff27d18 # v1.16.0
with:
project: ${{ vars.DEPOT_PROJECT_ID }}
context: .
file: ./docker/frontend/Dockerfile
push: true
tags: |
${{ secrets.DOCKER_HUB_USERNAME }}/test:v2-frontend-${{ steps.commit-hashes.outputs.frontend_short }}
${{ secrets.DOCKER_HUB_USERNAME }}/test:v2-frontend-latest
build-args: VERSION_TAG=v2-alpha
platforms: linux/amd64
- name: Build and push frontend image (Docker fork fallback)
if: env.USE_DEPOT != 'true' && steps.check-frontend.outputs.exists == 'false'
uses: docker/build-push-action@bcafcacb16a39f128d818304e6c9c0c18556b85f # v7.1.0
with:
context: .
@@ -105,8 +134,22 @@ jobs:
build-args: VERSION_TAG=v2-alpha
platforms: linux/amd64
- name: Build and push backend image
if: steps.check-backend.outputs.exists == 'false'
- name: Build and push backend image (Depot)
if: env.USE_DEPOT == 'true' && steps.check-backend.outputs.exists == 'false'
uses: depot/build-push-action@5f3b3c2e5a00f0093de47f657aeaefcedff27d18 # v1.16.0
with:
project: ${{ vars.DEPOT_PROJECT_ID }}
context: .
file: ./docker/backend/Dockerfile
push: true
tags: |
${{ secrets.DOCKER_HUB_USERNAME }}/test:v2-backend-${{ steps.commit-hashes.outputs.backend_short }}
${{ secrets.DOCKER_HUB_USERNAME }}/test:v2-backend-latest
build-args: VERSION_TAG=v2-alpha
platforms: linux/amd64
- name: Build and push backend image (Docker fork fallback)
if: env.USE_DEPOT != 'true' && steps.check-backend.outputs.exists == 'false'
uses: docker/build-push-action@bcafcacb16a39f128d818304e6c9c0c18556b85f # v7.1.0
with:
context: .
+13 -2
View File
@@ -11,21 +11,32 @@ on:
required: false
type: string
default: "false"
depot_cores:
description: "Depot runner vCPU count (used in runs-on). Override for benchmarking. Tuned to 4 because bench showed 16 was within noise of 4."
required: false
type: string
default: "4"
permissions:
contents: read
jobs:
pick:
uses: ./.github/workflows/_runner-pick.yml
docker-compose-tests:
runs-on: ubuntu-latest
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || format('depot-ubuntu-24.04-{0}', inputs.depot_cores || '4') }}
permissions:
actions: write
contents: read
checks: write
env:
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
+13 -3
View File
@@ -5,17 +5,27 @@ name: Playwright E2E (live backend)
# server.
on:
workflow_call:
inputs:
depot_cores:
description: "Depot runner vCPU count (used in runs-on). Override for benchmarking."
required: false
type: string
default: "8"
permissions:
contents: read
jobs:
pick:
uses: ./.github/workflows/_runner-pick.yml
playwright-e2e-live:
runs-on: ubuntu-latest
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || format('depot-ubuntu-24.04-{0}', inputs.depot_cores || '8') }}
timeout-minutes: 30
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
- name: Checkout repository
@@ -60,5 +70,5 @@ jobs:
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: playwright-report-live-${{ github.run_id }}
path: frontend/playwright-report/
path: frontend/editor/playwright-report/
retention-days: 7
+13 -3
View File
@@ -5,16 +5,26 @@ name: Playwright E2E (stubbed)
# mocks API responses in the browser.
on:
workflow_call:
inputs:
depot_cores:
description: "Depot runner vCPU count (used in runs-on). Override for benchmarking. Tuned to 8 to match the other playwright workflows; bench showed flat scaling above 8."
required: false
type: string
default: "8"
permissions:
contents: read
jobs:
pick:
uses: ./.github/workflows/_runner-pick.yml
playwright-e2e:
runs-on: ubuntu-latest
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || format('depot-ubuntu-24.04-{0}', inputs.depot_cores || '8') }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
- name: Checkout repository
@@ -40,5 +50,5 @@ jobs:
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: playwright-report-stubbed-${{ github.run_id }}
path: frontend/playwright-report/
path: frontend/editor/playwright-report/
retention-days: 7
@@ -19,16 +19,20 @@ permissions:
contents: read
jobs:
pick:
uses: ./.github/workflows/_runner-pick.yml
files-changed:
name: detect what files changed
runs-on: ubuntu-latest
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
timeout-minutes: 3
outputs:
licenses-frontend: ${{ steps.changes.outputs.licenses-frontend }}
licenses-backend: ${{ steps.changes.outputs.licenses-backend }}
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -44,15 +48,15 @@ jobs:
generate-frontend-license-report:
if: needs.files-changed.outputs.licenses-frontend == 'true'
name: Generate Frontend License Report
needs: files-changed
runs-on: ubuntu-latest
needs: [pick, files-changed]
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
permissions:
contents: write
pull-requests: write
repository-projects: write # Required for enabling automerge
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -114,23 +118,23 @@ jobs:
env:
PR_IS_FORK: "true"
run: |
node base/frontend/scripts/generate-licenses.js \
--input frontend/src/assets/3rdPartyLicenses.json
node base/frontend/editor/scripts/generate-licenses.js \
--input frontend/editor/src/assets/3rdPartyLicenses.json
- name: Copy postprocessed artifacts back (fork PRs)
if: github.event_name == 'pull_request' && github.event.pull_request.head.repo.fork == true
run: |
mkdir -p frontend/src/assets
if [ -f "base/frontend/src/assets/3rdPartyLicenses.json" ]; then
cp base/frontend/src/assets/3rdPartyLicenses.json frontend/src/assets/3rdPartyLicenses.json
mkdir -p frontend/editor/src/assets
if [ -f "base/frontend/editor/src/assets/3rdPartyLicenses.json" ]; then
cp base/frontend/editor/src/assets/3rdPartyLicenses.json frontend/editor/src/assets/3rdPartyLicenses.json
fi
if [ -f "base/frontend/src/assets/license-warnings.json" ]; then
cp base/frontend/src/assets/license-warnings.json frontend/src/assets/license-warnings.json
if [ -f "base/frontend/editor/src/assets/license-warnings.json" ]; then
cp base/frontend/editor/src/assets/license-warnings.json frontend/editor/src/assets/license-warnings.json
fi
- name: Check for license warnings
run: |
if [ -f "frontend/src/assets/license-warnings.json" ]; then
if [ -f "frontend/editor/src/assets/license-warnings.json" ]; then
echo "LICENSE_WARNINGS_EXIST=true" >> $GITHUB_ENV
else
echo "LICENSE_WARNINGS_EXIST=false" >> $GITHUB_ENV
@@ -178,10 +182,10 @@ jobs:
echo ""
if [ "${LICENSE_WARNINGS_EXIST}" = "true" ]; then
echo "❌ **Failed** – incompatible or unknown licenses found."
if [ -f "frontend/src/assets/license-warnings.json" ]; then
if [ -f "frontend/editor/src/assets/license-warnings.json" ]; then
echo ""
echo "### Warnings"
jq -r '.warnings[] | "- \(.message)"' frontend/src/assets/license-warnings.json || true
jq -r '.warnings[] | "- \(.message)"' frontend/editor/src/assets/license-warnings.json || true
fi
else
echo "✅ **Passed** – no license warnings detected."
@@ -207,7 +211,7 @@ jobs:
const fs = require('fs');
let warningDetails = '';
try {
const warnings = JSON.parse(fs.readFileSync('frontend/src/assets/license-warnings.json', 'utf8'));
const warnings = JSON.parse(fs.readFileSync('frontend/editor/src/assets/license-warnings.json', 'utf8'));
warningDetails = warnings.warnings.map(w => `- ${w.message}`).join('\n');
} catch (e) {
warningDetails = 'Unable to read warning details';
@@ -247,7 +251,7 @@ jobs:
- name: Commit changes (Push only)
if: github.event_name == 'push'
run: |
git add frontend/src/assets/3rdPartyLicenses.json
git add frontend/editor/src/assets/3rdPartyLicenses.json
# Note: Do NOT commit license-warnings.json - it's only for PR review
git diff --staged --quiet || echo "CHANGES_DETECTED=true" >> $GITHUB_ENV
@@ -265,7 +269,7 @@ jobs:
The following licenses may require review for corporate compatibility:
$(cat frontend/src/assets/license-warnings.json | jq -r '.warnings[].message')
$(cat frontend/editor/src/assets/license-warnings.json | jq -r '.warnings[].message')
Please review these licenses to ensure they are acceptable for your use case."
fi
@@ -307,16 +311,18 @@ jobs:
generate-backend-license-report:
if: needs.files-changed.outputs.licenses-backend == 'true'
needs: files-changed
needs: [pick, files-changed]
name: Generate Backend License Report
runs-on: ubuntu-latest
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
permissions:
contents: write
pull-requests: write
repository-projects: write # Required for enabling automerge
env:
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
+7 -3
View File
@@ -11,11 +11,15 @@ permissions:
pull-requests: write
jobs:
pick:
uses: ./.github/workflows/_runner-pick.yml
frontend-validation:
runs-on: ubuntu-latest
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
- name: Checkout repository
@@ -109,5 +113,5 @@ jobs:
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: frontend-build
path: frontend/dist/
path: frontend/editor/dist/
retention-days: 3
+1 -1
View File
@@ -15,7 +15,7 @@ jobs:
issues: write
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
+34 -21
View File
@@ -36,15 +36,19 @@ permissions:
contents: read
jobs:
pick:
uses: ./.github/workflows/_runner-pick.yml
determine-matrix:
if: ${{ vars.CI_PROFILE != 'lite' }}
runs-on: ubuntu-latest
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
outputs:
matrix: ${{ steps.set-matrix.outputs.matrix }}
version: ${{ steps.versionNumber.outputs.versionNumber }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -90,26 +94,28 @@ jobs:
if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
case "${{ github.event.inputs.platform }}" in
"windows")
echo 'matrix={"include":[{"platform":"windows-latest","args":"--target x86_64-pc-windows-msvc","name":"windows-x86_64"}]}' >> $GITHUB_OUTPUT
echo 'matrix={"include":[{"platform":"windows-latest","args":"--target x86_64-pc-windows-msvc","name":"windows-x86_64","jpdfium_platforms":"windows-x64"}]}' >> $GITHUB_OUTPUT
;;
"macos")
echo 'matrix={"include":[{"platform":"macos-15","args":"--target universal-apple-darwin","name":"macos-universal"}]}' >> $GITHUB_OUTPUT
echo 'matrix={"include":[{"platform":"macos-15","args":"--target universal-apple-darwin","name":"macos-universal","jpdfium_platforms":"darwin-arm64,darwin-x64"}]}' >> $GITHUB_OUTPUT
;;
"linux")
echo 'matrix={"include":[{"platform":"ubuntu-22.04","args":"","name":"linux-x86_64"}]}' >> $GITHUB_OUTPUT
echo 'matrix={"include":[{"platform":"ubuntu-22.04","args":"","name":"linux-x86_64","jpdfium_platforms":"linux-x64"}]}' >> $GITHUB_OUTPUT
;;
*)
echo 'matrix={"include":[{"platform":"windows-latest","args":"--target x86_64-pc-windows-msvc","name":"windows-x86_64"},{"platform":"macos-15","args":"--target universal-apple-darwin","name":"macos-universal"},{"platform":"ubuntu-22.04","args":"","name":"linux-x86_64"}]}' >> $GITHUB_OUTPUT
echo 'matrix={"include":[{"platform":"windows-latest","args":"--target x86_64-pc-windows-msvc","name":"windows-x86_64","jpdfium_platforms":"windows-x64"},{"platform":"macos-15","args":"--target universal-apple-darwin","name":"macos-universal","jpdfium_platforms":"darwin-arm64,darwin-x64"},{"platform":"ubuntu-22.04","args":"","name":"linux-x86_64","jpdfium_platforms":"linux-x64"}]}' >> $GITHUB_OUTPUT
;;
esac
else
# For push/release events, build all platforms
echo 'matrix={"include":[{"platform":"windows-latest","args":"--target x86_64-pc-windows-msvc","name":"windows-x86_64"},{"platform":"macos-15","args":"--target universal-apple-darwin","name":"macos-universal"},{"platform":"ubuntu-22.04","args":"","name":"linux-x86_64"}]}' >> $GITHUB_OUTPUT
echo 'matrix={"include":[{"platform":"windows-latest","args":"--target x86_64-pc-windows-msvc","name":"windows-x86_64","jpdfium_platforms":"windows-x64"},{"platform":"macos-15","args":"--target universal-apple-darwin","name":"macos-universal","jpdfium_platforms":"darwin-arm64,darwin-x64"},{"platform":"ubuntu-22.04","args":"","name":"linux-x86_64","jpdfium_platforms":"linux-x64"}]}' >> $GITHUB_OUTPUT
fi
build-jars:
needs: determine-matrix
runs-on: ubuntu-latest
needs: [pick, determine-matrix]
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
env:
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
strategy:
matrix:
variant:
@@ -127,7 +133,7 @@ jobs:
file_suffix: "-server"
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -189,9 +195,10 @@ jobs:
SM_API_KEY: ${{ secrets.SM_API_KEY }}
WINDOWS_CERTIFICATE: ${{ secrets.WINDOWS_CERTIFICATE }}
RELEASE_GPG_PRIVATE_KEY: ${{ secrets.RELEASE_GPG_PRIVATE_KEY }}
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
allowed-endpoints: >
@@ -256,15 +263,17 @@ jobs:
if: matrix.platform == 'macos-15'
env:
AARCH64_JAVA_HOME: ${{ env.JAVA_HOME }}
JPDFIUM_PLATFORMS: ${{ matrix.jpdfium_platforms }}
run: task desktop:jlink:universal-mac
- name: Prepare desktop build
run: task desktop:prepare
env:
MAVEN_USER: ${{ secrets.MAVEN_USER }}
MAVEN_PASSWORD: ${{ secrets.MAVEN_PASSWORD }}
MAVEN_PUBLIC_URL: ${{ secrets.MAVEN_PUBLIC_URL }}
DISABLE_ADDITIONAL_FEATURES: true
JPDFIUM_PLATFORMS: ${{ matrix.jpdfium_platforms }}
run: task desktop:prepare
# DigiCert KeyLocker Setup (Cloud HSM)
- name: Setup DigiCert KeyLocker
@@ -418,7 +427,7 @@ jobs:
env:
KEYPAIR_ALIAS: ${{ secrets.SM_KEYPAIR_ALIAS }}
run: |
cat > ./frontend/src-tauri/tauri.windows.conf.json <<EOF
cat > ./frontend/editor/src-tauri/tauri.windows.conf.json <<EOF
{
"bundle": {
"windows": {
@@ -431,7 +440,11 @@ jobs:
}
EOF
echo "Generated tauri.windows.conf.json (alias masked):"
sed "s/${KEYPAIR_ALIAS}/***/g" ./frontend/src-tauri/tauri.windows.conf.json
sed "s/${KEYPAIR_ALIAS}/***/g" ./frontend/editor/src-tauri/tauri.windows.conf.json
- name: Sign JPDFium dylibs inside bootJar (macOS only)
if: matrix.platform == 'macos-15' && env.APPLE_CERTIFICATE != ''
run: bash frontend/scripts/sign-jpdfium-dylibs-in-bootjar.sh
- name: Import release GPG signing key (Linux)
if: matrix.platform == 'ubuntu-22.04' && env.RELEASE_GPG_PRIVATE_KEY != '' && (github.event_name == 'release' || (github.event_name == 'workflow_dispatch' && github.event.inputs.sign != 'false') || github.ref == 'refs/heads/V2-master')
@@ -479,7 +492,7 @@ jobs:
SM_CODE_SIGNING_CERT_SHA1_HASH: ${{ secrets.SM_CODE_SIGNING_CERT_SHA1_HASH }}
CI: true
with:
projectPath: ./frontend
projectPath: ./frontend/editor
tauriScript: npx tauri
args: ${{ matrix.args }}
@@ -505,7 +518,7 @@ jobs:
$allSigned = $true
# Check MSI installer (outer wrapper - what users download)
$msiFiles = Get-ChildItem -Path "./frontend/src-tauri/target" -Filter "*.msi" -Recurse -File
$msiFiles = Get-ChildItem -Path "./frontend/editor/src-tauri/target" -Filter "*.msi" -Recurse -File
if ($msiFiles.Count -eq 0) {
Write-Host "[ERROR] No MSI found under target/"
exit 1
@@ -574,7 +587,7 @@ jobs:
shell: bash
run: |
mkdir -p ./dist
cd ./frontend/src-tauri/target
cd ./frontend/editor/src-tauri/target
# Find and rename artifacts based on platform
if [ "${{ matrix.platform }}" = "windows-latest" ]; then
@@ -600,13 +613,13 @@ jobs:
create-release:
if: (github.event_name == 'workflow_dispatch' && github.event.inputs.test_mode != 'true') || github.event_name == 'release' || github.ref == 'refs/heads/V2-master'
needs: [determine-matrix, build, build-jars]
runs-on: ubuntu-latest
needs: [pick, determine-matrix, build, build-jars]
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
permissions:
contents: write
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -638,7 +651,7 @@ jobs:
run: ls -R ./artifacts
- name: Upload binaries to Release
uses: softprops/action-gh-release@153bb8e04406b158c6c84fc1615b65b24149a1fe # v2.6.1
uses: softprops/action-gh-release@b4309332981a82ec1c5618f44dd2e27cc8bfbfda # v3.0.0
with:
tag_name: v${{ needs.determine-matrix.outputs.version }}
generate_release_notes: true
+7 -3
View File
@@ -13,12 +13,16 @@ permissions:
contents: read
jobs:
pick:
uses: ./.github/workflows/_runner-pick.yml
playwright-all-browsers:
name: Playwright (chromium + firefox + webkit)
runs-on: ubuntu-latest
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -45,5 +49,5 @@ jobs:
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: playwright-nightly-${{ github.run_id }}
path: frontend/playwright-report/
path: frontend/editor/playwright-report/
retention-days: 14
+4 -4
View File
@@ -2,7 +2,7 @@ name: Update Package Manager Manifests
on:
release:
types: [ released ]
types: [released]
workflow_dispatch:
inputs:
version:
@@ -28,7 +28,7 @@ jobs:
jar_sha256: ${{ steps.hashes.outputs.jar_sha256 }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -79,12 +79,12 @@ jobs:
contents: write
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
- name: Checkout homebrew-stirling-pdf tap (also hosts Scoop bucket)
uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
repository: Stirling-Tools/homebrew-stirling-pdf
token: ${{ secrets.HOMEBREW_TAP_TOKEN }}
+1 -1
View File
@@ -19,7 +19,7 @@ jobs:
PIP_DISABLE_PIP_VERSION_CHECK: "1"
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
+2 -2
View File
@@ -43,7 +43,7 @@ jobs:
echo "version=${VERSION}" >> $GITHUB_OUTPUT
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -100,7 +100,7 @@ jobs:
sbom: true
- name: Install cosign
uses: sigstore/cosign-installer@cad07c2e89fa2edd6e2d7bab4c1aa38e53f76003 # v4.1.1
uses: sigstore/cosign-installer@6f9f17788090df1f26f669e9d70d6ae9567deba6 # v4.1.2
with:
cosign-release: "v2.4.1"
+3 -3
View File
@@ -53,7 +53,7 @@ jobs:
RUN_UNOSERVER: ${{ github.event_name != 'workflow_dispatch' || inputs.build_unoserver }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -96,13 +96,13 @@ jobs:
- name: Install cosign
if: github.ref == 'refs/heads/master' || github.ref == 'refs/heads/V2-master'
uses: sigstore/cosign-installer@cad07c2e89fa2edd6e2d7bab4c1aa38e53f76003 # v4.1.1
uses: sigstore/cosign-installer@6f9f17788090df1f26f669e9d70d6ae9567deba6 # v4.1.2
with:
cosign-release: "v2.4.1"
- name: Install cosign
if: github.ref == 'refs/heads/master' || github.ref == 'refs/heads/V2-master'
uses: sigstore/cosign-installer@cad07c2e89fa2edd6e2d7bab4c1aa38e53f76003 # v4.1.1
uses: sigstore/cosign-installer@6f9f17788090df1f26f669e9d70d6ae9567deba6 # v4.1.2
with:
cosign-release: "v2.4.1"
+1 -1
View File
@@ -18,7 +18,7 @@ jobs:
packages: write
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
+2 -2
View File
@@ -35,7 +35,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -75,6 +75,6 @@ jobs:
# Upload the results to GitHub's code scanning dashboard.
- name: "Upload to code-scanning"
uses: github/codeql-action/upload-sarif@e46ed2cbd01164d986452f91f178727624ae40d7 # v3.29.5
uses: github/codeql-action/upload-sarif@9e0d7b8d25671d64c341c19c0152d693099fb5ba # v3.29.5
with:
sarif_file: results.sarif
+1 -1
View File
@@ -17,7 +17,7 @@ jobs:
pull-requests: write
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
+8 -2
View File
@@ -22,12 +22,18 @@ permissions:
contents: read
jobs:
pick:
uses: ./.github/workflows/_runner-pick.yml
push:
if: ${{ vars.CI_PROFILE != 'lite' }}
runs-on: ubuntu-latest
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
env:
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
+6 -6
View File
@@ -11,7 +11,7 @@ on:
- "app/core/build.gradle"
- "app/proprietary/build.gradle"
- "README.md"
- "frontend/public/locales/*/translation.toml"
- "frontend/editor/public/locales/*/translation.toml"
- "app/core/src/main/resources/static/3rdPartyLicenses.json"
- "scripts/ignore_translation.toml"
@@ -35,7 +35,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -62,7 +62,7 @@ jobs:
- name: Sync translation TOML files
run: |
python .github/scripts/check_language_toml.py --reference-file "frontend/public/locales/en-GB/translation.toml" --branch main
python .github/scripts/check_language_toml.py --reference-file "frontend/editor/public/locales/en-GB/translation.toml" --branch main
- name: pre-commit run
run: |
@@ -70,7 +70,7 @@ jobs:
- name: Commit translation files
run: |
git add frontend/public/locales/*/translation.toml
git add frontend/editor/public/locales/*/translation.toml
git diff --staged --quiet || git commit -m ":memo: Sync translation files (TOML)" || echo "No changes detected"
- name: Sync README.md
@@ -100,7 +100,7 @@ jobs:
This Pull Request was automatically generated to synchronize updates to translation files and documentation. Below are the details of the changes made:
#### **1. Synchronization of Translation Files**
- Updated translation files (`frontend/public/locales/*/translation.toml`) to reflect changes in the reference file `en-GB/translation.toml`.
- Updated translation files (`frontend/editor/public/locales/*/translation.toml`) to reflect changes in the reference file `en-GB/translation.toml`.
- Ensured consistency and synchronization across all supported language files.
- Highlighted any missing or incomplete translations.
- **Format**: TOML
@@ -125,5 +125,5 @@ jobs:
sign-commits: true
add-paths: |
README.md
frontend/public/locales/*/translation.toml
frontend/editor/public/locales/*/translation.toml
scripts/ignore_translation.toml
+26 -23
View File
@@ -4,6 +4,10 @@ name: Build Tauri Applications
# Linux). Called from build.yml on PRs that touch desktop sources (gated
# via the `tauri` filter in .github/config/.files.yaml). Also runnable
# on demand via workflow_dispatch with a per-platform selector.
#
# Note: editing this file is itself enough to make the `tauri` path filter
# match, which is how non-desktop PRs (e.g. backend-only fixes) opt into a
# desktop smoke build.
on:
workflow_call:
inputs:
@@ -37,7 +41,7 @@ jobs:
matrix: ${{ steps.set-matrix.outputs.matrix }}
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -47,12 +51,10 @@ jobs:
APPLE_CERTIFICATE: ${{ secrets.APPLE_CERTIFICATE }}
PLATFORM: ${{ inputs.platform }}
run: |
WINDOWS='{"platform":"windows-latest","args":"--target x86_64-pc-windows-msvc","name":"windows-x86_64"}'
MACOS='{"platform":"macos-15","args":"--target universal-apple-darwin","name":"macos-universal"}'
LINUX='{"platform":"ubuntu-22.04","args":"","name":"linux-x86_64"}'
WINDOWS='{"platform":"windows-latest","args":"--target x86_64-pc-windows-msvc","name":"windows-x86_64","jpdfium_platforms":"windows-x64"}'
MACOS='{"platform":"macos-15","args":"--target universal-apple-darwin","name":"macos-universal","jpdfium_platforms":"darwin-arm64,darwin-x64"}'
LINUX='{"platform":"ubuntu-22.04","args":"","name":"linux-x86_64","jpdfium_platforms":"linux-x64"}'
# Resolve requested platform — populated by either workflow_dispatch
# or workflow_call inputs; both paths default to "all".
case "$PLATFORM" in
windows) ENTRIES=("$WINDOWS") ;;
macos) ENTRIES=("$MACOS") ;;
@@ -84,9 +86,10 @@ jobs:
WINDOWS_CERTIFICATE: ${{ secrets.WINDOWS_CERTIFICATE }}
APPLE_CERTIFICATE: ${{ secrets.APPLE_CERTIFICATE }}
RELEASE_GPG_PRIVATE_KEY: ${{ secrets.RELEASE_GPG_PRIVATE_KEY }}
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -112,10 +115,6 @@ jobs:
toolchain: stable
targets: ${{ matrix.platform == 'macos-15' && 'aarch64-apple-darwin,x86_64-apple-darwin' || '' }}
# x86_64 JDK is set up first so the aarch64 step below can leave its
# JAVA_HOME as the active one. The macOS universal JRE build needs
# jmods from both arches; the x64 path is captured into the env
# before the second setup-java overwrites JAVA_HOME.
- name: Set up x86_64 JDK 25 (macOS universal JRE)
if: matrix.platform == 'macos-15'
uses: actions/setup-java@be666c2fcd27ec809703dec50e508c2fdc7f6654 # v5.2.0
@@ -142,21 +141,21 @@ jobs:
- name: Setup Task
uses: go-task/setup-task@3be4020d41929789a01026e0e427a4321ce0ad44 # v2.0.0
# Build the universal JRE before desktop:prepare so the jlink:runtime
# task short-circuits on its `test -d runtime/jre` status check.
- name: Build universal macOS JRE
if: matrix.platform == 'macos-15'
env:
AARCH64_JAVA_HOME: ${{ env.JAVA_HOME }}
JPDFIUM_PLATFORMS: ${{ matrix.jpdfium_platforms }}
run: task desktop:jlink:universal-mac
- name: Prepare desktop build
run: task desktop:prepare
env:
MAVEN_USER: ${{ secrets.MAVEN_USER }}
MAVEN_PASSWORD: ${{ secrets.MAVEN_PASSWORD }}
MAVEN_PUBLIC_URL: ${{ secrets.MAVEN_PUBLIC_URL }}
DISABLE_ADDITIONAL_FEATURES: true
JPDFIUM_PLATFORMS: ${{ matrix.jpdfium_platforms }}
run: task desktop:prepare
# DigiCert KeyLocker Setup (Cloud HSM)
- name: Setup DigiCert KeyLocker
@@ -269,6 +268,10 @@ jobs:
echo "APPLE_SIGNING_IDENTITY=$CERT_ID" >> $GITHUB_ENV
echo "Certificate imported successfully."
- name: Sign JPDFium dylibs inside bootJar (macOS only)
if: matrix.platform == 'macos-15' && env.APPLE_CERTIFICATE != ''
run: bash frontend/scripts/sign-jpdfium-dylibs-in-bootjar.sh
- name: Check DMG creation dependencies (macOS only)
if: matrix.platform == 'macos-15'
run: |
@@ -299,7 +302,7 @@ jobs:
env:
KEYPAIR_ALIAS: ${{ secrets.SM_KEYPAIR_ALIAS }}
run: |
cat > ./frontend/src-tauri/tauri.windows.conf.json <<EOF
cat > ./frontend/editor/src-tauri/tauri.windows.conf.json <<EOF
{
"bundle": {
"windows": {
@@ -357,7 +360,7 @@ jobs:
SM_CODE_SIGNING_CERT_SHA1_HASH: ${{ secrets.SM_CODE_SIGNING_CERT_SHA1_HASH }}
CI: true
with:
projectPath: ./frontend
projectPath: ./frontend/editor
tauriScript: npx tauri
# Linux: build deb+rpm only here. AppImage runs in its own
# continue-on-error step below so its persistent linuxdeploy
@@ -382,7 +385,7 @@ jobs:
VITE_SAAS_BACKEND_API_URL: ${{ secrets.VITE_SAAS_BACKEND_API_URL || 'https://api.stirlingpdf.com' }}
CI: true
with:
projectPath: ./frontend
projectPath: ./frontend/editor
tauriScript: npx tauri
args: --bundles appimage
@@ -400,7 +403,7 @@ jobs:
if: matrix.platform == 'macos-15'
run: |
echo "🔍 Verifying notarization status..."
cd ./frontend/src-tauri/target
cd ./frontend/editor/src-tauri/target
DMG_FILE=$(find . -name "*.dmg" | head -1)
if [ -n "$DMG_FILE" ]; then
echo "Found DMG: $DMG_FILE"
@@ -415,7 +418,7 @@ jobs:
shell: bash
run: |
mkdir -p ./dist
cd ./frontend/src-tauri/target
cd ./frontend/editor/src-tauri/target
# Find and rename artifacts based on platform
if [ "${{ matrix.platform }}" = "windows-latest" ]; then
@@ -505,7 +508,7 @@ jobs:
- name: Verify build artifacts
shell: bash
run: |
cd ./frontend/src-tauri/target
cd ./frontend/editor/src-tauri/target
# Check for expected artifacts based on platform
if [ "${{ matrix.platform }}" = "windows-latest" ]; then
@@ -536,7 +539,7 @@ jobs:
- name: Test artifact sizes
shell: bash
run: |
cd ./frontend/src-tauri/target
cd ./frontend/editor/src-tauri/target
echo "Artifact sizes for ${{ matrix.name }}:"
find . -name "*.exe" -o -name "*.dmg" -o -name "*.deb" -o -name "*.rpm" -o -name "*.AppImage" -o -name "*.msi" | while read file; do
if [ -f "$file" ]; then
@@ -557,7 +560,7 @@ jobs:
pull-requests: write
steps:
- name: Harden the runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -648,7 +651,7 @@ jobs:
if: always()
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
+70 -6
View File
@@ -12,11 +12,19 @@ on:
required: false
type: string
default: "false"
depot_cores:
description: "Depot runner vCPU count (used in runs-on). Override for benchmarking."
required: false
type: string
default: "8"
permissions:
contents: read
jobs:
pick:
uses: ./.github/workflows/_runner-pick.yml
# TODO: extract a pre-matrix `prepare` job that runs once and produces
# shared artifacts for the three matrix entries below to consume:
# 1. `task backend:build` — currently runs 3× in parallel with
@@ -32,7 +40,14 @@ jobs:
# spring-security=true matrix entry if `task backend:build` and
# `task backend:build:ci` produce equivalent JARs (verify before wiring).
test-build-docker-images:
runs-on: ubuntu-latest
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || format('depot-ubuntu-24.04-{0}', inputs.depot_cores || '8') }}
permissions:
contents: read
id-token: write
env:
USE_DEPOT: ${{ needs.pick.outputs.is_fork != 'true' && inputs.docker-base-changed != 'true' }}
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
strategy:
fail-fast: false
matrix:
@@ -48,7 +63,7 @@ jobs:
cache-scope: stirling-pdf-fat
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -105,10 +120,16 @@ jobs:
DISABLE_ADDITIONAL_FEATURES: true
STIRLING_PDF_DESKTOP_UI: false
- name: Set up Depot CLI
if: env.USE_DEPOT == 'true'
uses: depot/setup-action@15c09a5f77a0840ad4bce955686522a257853461 # v1.0.0
- name: Set up QEMU
if: env.USE_DEPOT != 'true'
uses: docker/setup-qemu-action@ce360397dd3f832beb865e1373c09c0e9f86d70a # v4.0.0
- name: Set up Docker Buildx
if: env.USE_DEPOT != 'true'
id: buildx
uses: docker/setup-buildx-action@4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd # v4.0.0
@@ -134,7 +155,22 @@ jobs:
echo "platforms=linux/amd64,linux/arm64/v8" >> "$GITHUB_OUTPUT"
fi
- name: Build ${{ matrix.docker-rev }}
- name: Build ${{ matrix.docker-rev }} (Depot)
if: env.USE_DEPOT == 'true'
uses: depot/build-push-action@5f3b3c2e5a00f0093de47f657aeaefcedff27d18 # v1.16.0
with:
project: ${{ vars.DEPOT_PROJECT_ID }}
context: .
file: ./${{ matrix.docker-rev }}
push: false
platforms: ${{ steps.build-params.outputs.platforms }}
build-args: |
BASE_IMAGE=${{ steps.build-params.outputs.base_image }}
provenance: true
sbom: true
- name: Build ${{ matrix.docker-rev }} (Docker fork fallback)
if: env.USE_DEPOT != 'true'
uses: docker/build-push-action@bcafcacb16a39f128d818304e6c9c0c18556b85f # v7.1.0
with:
builder: ${{ steps.buildx.outputs.name }}
@@ -162,24 +198,52 @@ jobs:
if-no-files-found: warn
test-build-unoserver-image:
runs-on: ubuntu-latest
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || format('depot-ubuntu-24.04-{0}', inputs.depot_cores || '8') }}
permissions:
contents: read
id-token: write
env:
USE_DEPOT: ${{ needs.pick.outputs.is_fork != 'true' && inputs.docker-base-changed != 'true' }}
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@8d3c67de8e2fe68ef647c8db1e6a09f647780f40 # v2.19.0
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
- name: Checkout Repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Set up Depot CLI
if: env.USE_DEPOT == 'true'
uses: depot/setup-action@15c09a5f77a0840ad4bce955686522a257853461 # v1.0.0
- name: Set up QEMU
if: env.USE_DEPOT != 'true'
uses: docker/setup-qemu-action@ce360397dd3f832beb865e1373c09c0e9f86d70a # v4.0.0
- name: Set up Docker Buildx
if: env.USE_DEPOT != 'true'
id: buildx
uses: docker/setup-buildx-action@4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd # v4.0.0
- name: Build docker/unoserver/Dockerfile
- name: Build docker/unoserver/Dockerfile (Depot)
if: env.USE_DEPOT == 'true'
uses: depot/build-push-action@5f3b3c2e5a00f0093de47f657aeaefcedff27d18 # v1.16.0
with:
project: ${{ vars.DEPOT_PROJECT_ID }}
context: .
file: ./docker/unoserver/Dockerfile
push: false
load: true
platforms: linux/amd64
tags: stirling-unoserver:pr-test
provenance: false
sbom: false
- name: Build docker/unoserver/Dockerfile (Docker fork fallback)
if: env.USE_DEPOT != 'true'
uses: docker/build-push-action@bcafcacb16a39f128d818304e6c9c0c18556b85f # v7.1.0
with:
builder: ${{ steps.buildx.outputs.name }}
+40 -12
View File
@@ -20,12 +20,22 @@ permissions:
contents: read
jobs:
pick:
uses: ./.github/workflows/_runner-pick.yml
deploy:
if: ${{ vars.CI_PROFILE != 'lite' }}
runs-on: ubuntu-latest
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
permissions:
contents: read
id-token: write
env:
USE_DEPOT: ${{ needs.pick.outputs.is_fork != 'true' }}
DEPOT_TOKEN: ${{ secrets.DEPOT_TOKEN }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -51,7 +61,12 @@ jobs:
MAVEN_PUBLIC_URL: ${{ secrets.MAVEN_PUBLIC_URL }}
DISABLE_ADDITIONAL_FEATURES: true
- name: Set up Depot CLI
if: env.USE_DEPOT == 'true'
uses: depot/setup-action@15c09a5f77a0840ad4bce955686522a257853461 # v1.0.0
- name: Set up Docker Buildx
if: env.USE_DEPOT != 'true'
uses: docker/setup-buildx-action@4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd # v4.0.0
- name: Get version number
@@ -66,7 +81,20 @@ jobs:
username: ${{ secrets.DOCKER_HUB_USERNAME }}
password: ${{ secrets.DOCKER_HUB_API }}
- name: Build and push test image
- name: Build and push test image (Depot)
if: env.USE_DEPOT == 'true'
uses: depot/build-push-action@5f3b3c2e5a00f0093de47f657aeaefcedff27d18 # v1.16.0
with:
project: ${{ vars.DEPOT_PROJECT_ID }}
context: .
file: ./docker/embedded/Dockerfile
push: true
tags: ${{ secrets.DOCKER_HUB_USERNAME }}/test:test-${{ github.sha }}
build-args: VERSION_TAG=${{ steps.versionNumber.outputs.versionNumber }}
platforms: linux/amd64
- name: Build and push test image (Docker fork fallback)
if: env.USE_DEPOT != 'true'
uses: docker/build-push-action@bcafcacb16a39f128d818304e6c9c0c18556b85f # v7.1.0
with:
context: .
@@ -125,13 +153,14 @@ jobs:
files-changed:
if: always()
name: detect what files changed
runs-on: ubuntu-latest
needs: pick
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
timeout-minutes: 3
outputs:
frontend: ${{ steps.changes.outputs.frontend }}
steps:
- name: Harden the runner (Audit all outbound calls)
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -145,12 +174,11 @@ jobs:
test:
if: needs.files-changed.outputs.frontend == 'true'
needs: [deploy, files-changed]
runs-on: ubuntu-latest
needs: [pick, deploy, files-changed]
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
@@ -180,13 +208,13 @@ jobs:
FORCE_COLOR: "3"
cleanup:
needs: [deploy, test]
runs-on: ubuntu-latest
needs: [pick, deploy, test]
runs-on: ${{ needs.pick.outputs.is_fork == 'true' && 'ubuntu-latest' || 'depot-ubuntu-24.04-4' }}
if: always()
steps:
- name: Harden Runner
uses: step-security/harden-runner@a5ad31d6a139d249332a2605b85202e8c0b78450 # v2.19.1
uses: step-security/harden-runner@ab7a9404c0f3da075243ca237b5fac12c98deaa5 # v2.19.3
with:
egress-policy: audit
+4
View File
@@ -158,6 +158,8 @@ common/build
proprietary/build
stirling-pdf/build
frontend/src-tauri/provisioner/target
frontend/src-tauri/target
frontend/src-tauri/runtime/
# Byte-compiled / optimized / DLL files
__pycache__/
@@ -265,6 +267,8 @@ docs/type3/signatures/
# Type3 sample PDFs (development only)
**/type3/samples/
**/application-dev-local.properties
# Claude
.claude/
+3 -3
View File
@@ -1,6 +1,6 @@
repos:
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.15.11
rev: v0.15.14
hooks:
- id: ruff
args:
@@ -20,7 +20,7 @@ repos:
- --skip="./.*,*.csv,*.json,*.ambr"
- --quiet-level=2
files: \.(html|css|js|py|md)$
exclude: (.vscode|.devcontainer|app/core/src/main/resources|app/proprietary/src/main/resources|frontend/public/vendor|Dockerfile|.*/pdfjs.*|.*/thirdParty.*|bootstrap.*|.*\.min\..*|.*diff\.js)
exclude: (.vscode|.devcontainer|app/core/src/main/resources|app/proprietary/src/main/resources|frontend/editor/public/vendor|Dockerfile|.*/pdfjs.*|.*/thirdParty.*|bootstrap.*|.*\.min\..*|.*diff\.js)
- repo: https://github.com/gitleaks/gitleaks
rev: v8.30.0
hooks:
@@ -38,7 +38,7 @@ repos:
rev: v0.24.4
hooks:
- id: toml-sort-fix
files: frontend/public/locales/.*\.toml$
files: frontend/editor/public/locales/.*\.toml$
args: ['--in-place', '--all', '--ignore-case']
# - repo: https://github.com/thibaudcolas/pre-commit-stylelint
# rev: v16.21.1
+85 -15
View File
@@ -1,7 +1,19 @@
version: '3'
# All gradle invocations go through `bash gradlew` (the sh wrapper) so they work
# uniformly on Linux/macOS (native bash) and Windows-with-Git-Bash.
# Gradle invocation strategy:
# - Linux/macOS: `./gradlew` runs the POSIX shell wrapper natively.
# - Windows: `cmd /c ".\gradlew.bat ..."` invokes the .bat wrapper through
# cmd.exe so it inherits the user's Windows-side `JAVA_HOME`
# and PATH. Routing through `bash gradlew` on Windows ends up
# under WSL or Git-Bash, neither of which inherits
# Adoptium/Temurin's default Windows-only Java env - gradlew
# then errors with "JAVA_HOME is not set and no 'java' command
# could be found".
#
# The entire `.\gradlew.bat ...` payload is double-quoted so
# the leading `.\` survives mvdan/sh's POSIX backslash
# stripping; cmd.exe also requires `.\` (not bare `gradlew.bat`)
# because modern Windows excludes cwd from cmd's search path.
tasks:
dev:
@@ -13,43 +25,86 @@ tasks:
env:
SERVER_PORT: '{{.PORT}}'
cmds:
- '{{if .AIENGINE_URL}}AIENGINE_URL={{.AIENGINE_URL}} AIENGINE_ENABLED=true {{end}}bash gradlew :stirling-pdf:bootRun'
- cmd: '{{if .AIENGINE_URL}}AIENGINE_URL={{.AIENGINE_URL}} AIENGINE_ENABLED=true AIENGINE_TIMEOUTSECONDS=120 {{end}}cmd /c ".\gradlew.bat :stirling-pdf:bootRun"'
platforms: [windows]
- cmd: '{{if .AIENGINE_URL}}AIENGINE_URL={{.AIENGINE_URL}} AIENGINE_ENABLED=true AIENGINE_TIMEOUTSECONDS=120 {{end}}./gradlew :stirling-pdf:bootRun'
platforms: [linux, darwin]
dev:bundled:
desc: "Clean + bootRun with frontend bundled into the backend (single :8080 server)"
ignore_error: true
cmds:
- bash gradlew clean bootRun -PbuildWithFrontend=true
- cmd: cmd /c ".\gradlew.bat clean bootRun -PbuildWithFrontend=true"
platforms: [windows]
- cmd: ./gradlew clean bootRun -PbuildWithFrontend=true
platforms: [linux, darwin]
dev:saas:
desc: "Start backend in SaaS flavor against Supabase (loads .env.saas.local)"
# `dotenv:` reads from the root Taskfile's directory (".") because this
# subtaskfile is included with `dir: .`. Drop the file at the repo root.
dotenv: ['.env.saas.local']
ignore_error: true
vars:
PORT: '{{.PORT | default "8080"}}'
# Override to "" to run the pure `saas` profile against your own SAAS_DB_*.
PROFILES: '{{.PROFILES | default "dev"}}'
env:
SERVER_PORT: '{{.PORT}}'
STIRLING_FLAVOR: saas
cmds:
- cmd: cmd /c ".\gradlew.bat :stirling-pdf:bootRun {{if .PROFILES}}--args=\"--spring.profiles.include={{.PROFILES}}\"{{end}}"
platforms: [windows]
- cmd: ./gradlew :stirling-pdf:bootRun {{if .PROFILES}}--args='--spring.profiles.include={{.PROFILES}}'{{end}}
platforms: [linux, darwin]
build:
desc: "Full backend build"
cmds:
- bash gradlew clean build
- cmd: cmd /c ".\gradlew.bat clean build"
platforms: [windows]
- cmd: ./gradlew clean build
platforms: [linux, darwin]
build:fast:
desc: "Build without tests"
cmds:
- bash gradlew clean build -x test
- cmd: cmd /c ".\gradlew.bat clean build -x test"
platforms: [windows]
- cmd: ./gradlew clean build -x test
platforms: [linux, darwin]
build:ci:
desc: "Build for CI (formatting checked separately)"
cmds:
- bash gradlew build -PnoSpotless
- cmd: cmd /c ".\gradlew.bat build -PnoSpotless"
platforms: [windows]
- cmd: ./gradlew build -PnoSpotless
platforms: [linux, darwin]
test:
desc: "Run backend tests"
cmds:
- bash gradlew test
- cmd: cmd /c ".\gradlew.bat test"
platforms: [windows]
- cmd: ./gradlew test
platforms: [linux, darwin]
format:
desc: "Auto-fix code formatting"
cmds:
- bash gradlew spotlessApply
- cmd: cmd /c ".\gradlew.bat spotlessApply"
platforms: [windows]
- cmd: ./gradlew spotlessApply
platforms: [linux, darwin]
format:check:
desc: "Check code formatting"
cmds:
- bash gradlew spotlessCheck
- cmd: cmd /c ".\gradlew.bat spotlessCheck"
platforms: [windows]
- cmd: ./gradlew spotlessCheck
platforms: [linux, darwin]
fix:
desc: "Auto-fix backend"
@@ -59,7 +114,10 @@ tasks:
swagger:
desc: "Generate OpenAPI docs"
cmds:
- bash gradlew :stirling-pdf:copySwaggerDoc
- cmd: cmd /c ".\gradlew.bat :stirling-pdf:copySwaggerDoc"
platforms: [windows]
- cmd: ./gradlew :stirling-pdf:copySwaggerDoc
platforms: [linux, darwin]
sources:
- app/core/src/main/java/**/*.java
- app/proprietary/src/main/java/**/*.java
@@ -77,19 +135,31 @@ tasks:
desc: "Print project version"
silent: true
cmds:
- bash gradlew printVersion --quiet | tail -1
- cmd: cmd /c ".\gradlew.bat printVersion --quiet" | tail -1
platforms: [windows]
- cmd: ./gradlew printVersion --quiet | tail -1
platforms: [linux, darwin]
licenses:check:
desc: "Check dependency licenses"
cmds:
- bash gradlew checkLicense --no-parallel
- cmd: cmd /c ".\gradlew.bat checkLicense --no-parallel"
platforms: [windows]
- cmd: ./gradlew checkLicense --no-parallel
platforms: [linux, darwin]
licenses:generate:
desc: "Check and generate dependency license report"
cmds:
- bash gradlew checkLicense generateLicenseReport --no-parallel
- cmd: cmd /c ".\gradlew.bat checkLicense generateLicenseReport --no-parallel"
platforms: [windows]
- cmd: ./gradlew checkLicense generateLicenseReport --no-parallel
platforms: [linux, darwin]
clean:
desc: "Clean build artifacts"
cmds:
- bash gradlew clean
- cmd: cmd /c ".\gradlew.bat clean"
platforms: [windows]
- cmd: ./gradlew clean
platforms: [linux, darwin]
+39 -11
View File
@@ -3,6 +3,22 @@ version: '3'
vars:
JLINK_MODULES: "java.base,java.compiler,java.desktop,java.instrument,java.logging,java.management,java.naming,java.net.http,java.prefs,java.rmi,java.scripting,java.security.jgss,java.security.sasl,java.sql,java.transaction.xa,java.xml,java.xml.crypto,jdk.crypto.ec,jdk.crypto.cryptoki,jdk.unsupported"
# Override via JPDFIUM_PLATFORMS env (csv of platform keys, or 'all').
JPDFIUM_PLATFORMS:
sh: |
if [ -n "${JPDFIUM_PLATFORMS:-}" ]; then
echo "$JPDFIUM_PLATFORMS"
else
case "{{OS}}-{{ARCH}}" in
darwin-arm64) echo "darwin-arm64";;
darwin-amd64) echo "darwin-x64";;
linux-amd64) echo "linux-x64";;
linux-arm64) echo "linux-arm64";;
windows-amd64) echo "windows-x64";;
*) echo "all";;
esac
fi
tasks:
prepare:
desc: "Prepare desktop build dependencies"
@@ -15,6 +31,7 @@ tasks:
provisioner:
desc: "Build installer provisioner"
platforms: [windows]
dir: editor
cmds:
- node scripts/build-provisioner.mjs
@@ -22,41 +39,48 @@ tasks:
desc: "Start Tauri desktop dev mode"
deps: [prepare]
ignore_error: true
dir: editor
cmds:
- npx tauri dev --no-watch
build:
desc: "Build Tauri desktop app (production)"
deps: [prepare]
dir: editor
cmds:
- npx tauri build
build:dev:
desc: "Build Tauri desktop app (dev, no bundling)"
deps: [prepare]
dir: editor
cmds:
- npx tauri build --no-bundle
build:dev:mac:
desc: "Build Tauri desktop .app bundle (macOS)"
deps: [prepare]
dir: editor
cmds:
- npx tauri build --bundles app
build:dev:windows:
desc: "Build Tauri desktop NSIS installer (Windows)"
deps: [prepare]
dir: editor
cmds:
- npx tauri build --bundles nsis
build:dev:linux:
desc: "Build Tauri desktop AppImage (Linux)"
deps: [prepare]
dir: editor
cmds:
- npx tauri build --bundles appimage
clean:
desc: "Clean Tauri/Cargo build artifacts"
dir: editor
cmds:
- task: jlink:clean
- cd src-tauri && cargo clean
@@ -71,27 +95,29 @@ tasks:
deps: [jlink:jar, jlink:runtime]
jlink:jar:
desc: "Build backend JAR for Tauri bundling"
desc: "Build backend JAR for Tauri bundling (host-OS natives only by default)"
run: once
dir: ..
env:
DISABLE_ADDITIONAL_FEATURES: "true"
cmds:
- cmd: cmd /c gradlew.bat bootJar --no-daemon
- echo "Building bootJar with JPDFium natives for {{.JPDFIUM_PLATFORMS}}"
- cmd: cmd /c gradlew.bat bootJar --no-daemon -PjpdfiumPlatforms={{.JPDFIUM_PLATFORMS}}
platforms: [windows]
- cmd: ./gradlew bootJar --no-daemon
- cmd: ./gradlew bootJar --no-daemon -PjpdfiumPlatforms={{.JPDFIUM_PLATFORMS}}
platforms: [linux, darwin]
- mkdir -p frontend/src-tauri/libs
- cp app/core/build/libs/stirling-pdf-*.jar frontend/src-tauri/libs/
- mkdir -p frontend/editor/src-tauri/libs
- cp app/core/build/libs/stirling-pdf-*.jar frontend/editor/src-tauri/libs/
status:
- test -f frontend/src-tauri/libs/stirling-pdf-*.jar
- test -f frontend/editor/src-tauri/libs/stirling-pdf-*.jar
jlink:runtime:
desc: "Create custom JRE with jlink"
deps: [jlink:jar]
dir: editor/src-tauri
cmds:
- rm -rf src-tauri/runtime/jre
- mkdir -p src-tauri/runtime
- rm -rf runtime/jre
- mkdir -p runtime
- >-
jlink
--add-modules {{.JLINK_MODULES}}
@@ -99,14 +125,15 @@ tasks:
--compress=zip-6
--no-header-files
--no-man-pages
--output src-tauri/runtime/jre
--output runtime/jre
status:
- test -d src-tauri/runtime/jre
- test -d editor/src-tauri/runtime/jre
jlink:clean:
desc: "Remove JLink runtime and bundled JARs"
dir: editor/src-tauri
cmds:
- rm -rf src-tauri/libs src-tauri/runtime
- rm -rf libs runtime
# macOS-only. Replaces jlink:runtime's single-arch JRE with a universal
# (arm64 + x86_64) one for the universal Tauri shell. Runs the x86_64
@@ -117,6 +144,7 @@ tasks:
desc: "Create universal (arm64+x86_64) JRE for the macOS Tauri build"
deps: [jlink:jar]
platforms: [darwin]
dir: editor
env:
JLINK_MODULES: "{{.JLINK_MODULES}}"
OUTPUT_DIR: src-tauri/runtime/jre
+5 -5
View File
@@ -3,14 +3,14 @@ version: '3'
tasks:
install:
desc: "Install Playwright browsers"
dir: frontend
dir: frontend/editor
deps: [ ':frontend:install' ]
cmds:
- npx playwright install {{.CLI_ARGS}} --with-deps
stubbed:
desc: "Run stubbed E2E tests"
dir: frontend
dir: frontend/editor
deps: [ ':frontend:prepare' ]
cmds:
- npx playwright test --project=stubbed {{.CLI_ARGS}}
@@ -72,7 +72,7 @@ tasks:
live:runner:
internal: true
deps: [ ':frontend:prepare' ]
dir: frontend
dir: frontend/editor
vars:
BASE_DIR: '{{.ROOT_DIR}}/.test-state/playwright'
cmds:
@@ -141,14 +141,14 @@ tasks:
up first with:
task e2e:oauth:up
task e2e:saml:up
dir: frontend
dir: frontend/editor
deps: [ ':frontend:prepare' ]
cmds:
- npx playwright test --project=enterprise {{.CLI_ARGS}}
cross-browser:
desc: "Run stubbed E2E tests on Chromium, Firefox, and WebKit"
dir: frontend
dir: frontend/editor
deps: [ ':frontend:prepare' ]
cmds:
- npx playwright test --project=stubbed --project=stubbed-firefox --project=stubbed-webkit {{.CLI_ARGS}}
+2 -1
View File
@@ -86,8 +86,9 @@ tasks:
fix:
desc: "Auto-fix lint + format"
cmds:
- task: format # Can auto-fix some things that `lint:fix` can't like line length violations
- task: lint:fix
- task: format
- task: format # Ensure that after lint fixing that the code is still formatted correctly
check:
desc: "Full engine quality gate"
+43 -23
View File
@@ -1,5 +1,10 @@
version: '3'
# Tasks operate from the workspace root (frontend/). Editor commands pass
# `editor` as the vite project root (positional after `build` / before the
# mode flag) or use `--project editor/...` for tsc — so the editor lives
# under frontend/editor/ without each task needing a cd.
tasks:
install:
desc: "Install dependencies"
@@ -21,19 +26,19 @@ tasks:
vars:
MODE: '{{.MODE | default ""}}'
cmds:
- npx tsx scripts/setup-env.ts{{if .MODE}} --{{.MODE}}{{end}}
- npx tsx editor/scripts/setup-env.mts{{if .MODE}} --{{.MODE}}{{end}}
sources:
- scripts/setup-env.ts
- editor/scripts/setup-env.mts
generates:
- .env.local
- .env{{if .MODE}}.{{.MODE}}{{end}}.local
- editor/.env.local
- editor/.env{{if .MODE}}.{{.MODE}}{{end}}.local
prepare:icons:
internal: true
run: once
deps: [install]
cmds:
- node scripts/generate-icons.js
- node editor/scripts/generate-icons.js
prepare:
desc: "Set up dev environment"
@@ -60,7 +65,7 @@ tasks:
env:
BACKEND_URL: '{{.BACKEND_URL}}'
cmds:
- npx vite --mode {{.MODE}} --port {{.PORT}}{{if .OPEN}} --open{{end}}
- npx vite editor --mode {{.MODE}} --port {{.PORT}}{{if .OPEN}} --open{{end}}
dev:
desc: "Start frontend dev server"
@@ -115,19 +120,19 @@ tasks:
desc: "Production build (default mode)"
deps: [prepare]
cmds:
- npx vite build
- npx vite build editor
build:core:
desc: "Build for core mode"
deps: [prepare]
cmds:
- npx vite build --mode core
- npx vite build editor --mode core
build:proprietary:
desc: "Build for proprietary mode"
deps: [prepare]
cmds:
- npx vite build --mode proprietary
- npx vite build editor --mode proprietary
build:saas:
desc: "Build for SaaS mode"
@@ -135,7 +140,7 @@ tasks:
- task: prepare
vars: { MODE: saas }
cmds:
- npx vite build --mode saas
- npx vite build editor --mode saas
build:desktop:
desc: "Build for desktop mode"
@@ -143,13 +148,13 @@ tasks:
- task: prepare
vars: { MODE: desktop }
cmds:
- npx vite build --mode desktop
- npx vite build editor --mode desktop
build:prototypes:
desc: "Build for prototypes mode"
deps: [prepare]
cmds:
- npx vite build --mode prototypes
- npx vite build editor --mode prototypes
# ============================================================
# Code quality
@@ -160,7 +165,7 @@ tasks:
deps: [install]
cmds:
- npx eslint --max-warnings=0
- npx dpdm src --circular --no-warning --no-tree --exit-code circular:1
- npx dpdm editor/src --circular --no-warning --no-tree --exit-code circular:1
lint:fix:
desc: "Auto-fix lint issues"
@@ -195,13 +200,13 @@ tasks:
desc: "Typecheck core build variant"
deps: [prepare]
cmds:
- npx tsc --noEmit --project src/core/tsconfig.json
- npx tsc --noEmit --project editor/src/core/tsconfig.json
typecheck:proprietary:
desc: "Typecheck proprietary build variant"
deps: [prepare]
cmds:
- npx tsc --noEmit --project src/proprietary/tsconfig.json
- npx tsc --noEmit --project editor/src/proprietary/tsconfig.json
typecheck:saas:
desc: "Typecheck SaaS build variant"
@@ -209,7 +214,7 @@ tasks:
- task: prepare
vars: { MODE: saas }
cmds:
- npx tsc --noEmit --project src/saas/tsconfig.json
- npx tsc --noEmit --project editor/src/saas/tsconfig.json
typecheck:desktop:
desc: "Typecheck desktop build variant"
@@ -217,19 +222,19 @@ tasks:
- task: prepare
vars: { MODE: desktop }
cmds:
- npx tsc --noEmit --project src/desktop/tsconfig.json
- npx tsc --noEmit --project editor/src/desktop/tsconfig.json
typecheck:scripts:
desc: "Typecheck scripts"
deps: [prepare]
cmds:
- npx tsc --noEmit --project scripts/tsconfig.json
- npx tsc --noEmit --project editor/scripts/tsconfig.json
typecheck:prototypes:
desc: "Typecheck prototypes build variant"
deps: [prepare]
cmds:
- npx tsc --noEmit --project src/prototypes/tsconfig.json
- npx tsc --noEmit --project editor/src/prototypes/tsconfig.json
typecheck:all:
desc: "Typecheck all build variants"
@@ -252,6 +257,7 @@ tasks:
- task: lint
- task: format:check
- task: test
- task: theme:check-unused-vars
check:all:
desc: "Full CI quality gate"
@@ -270,19 +276,19 @@ tasks:
desc: "Run tests"
deps: [install]
cmds:
- npx vitest run
- npx vitest run --root editor
test:watch:
desc: "Run tests in watch mode"
deps: [install]
cmds:
- npx vitest --watch
- npx vitest --watch --root editor
test:coverage:
desc: "Run tests with coverage"
deps: [install]
cmds:
- npx vitest --coverage
- npx vitest --coverage --root editor
# ============================================================
# Code Generation
@@ -292,4 +298,18 @@ tasks:
desc: "Generate frontend license report"
deps: [install]
cmds:
- node scripts/generate-licenses.js
- node editor/scripts/generate-licenses.js
theme:clean-unused-vars:
desc: "Remove unused CSS variables from theme.css"
deps: [install]
vars:
REMOVE: '{{.REMOVE | default "false"}}'
cmds:
- node editor/scripts/clean-unused-theme-vars.js{{if eq .REMOVE "true"}} --remove{{end}} {{.CLI_ARGS}}
theme:check-unused-vars:
desc: "Fail if theme.css contains unused CSS variables"
deps: [install]
cmds:
- node editor/scripts/clean-unused-theme-vars.js --check
+6 -6
View File
@@ -10,14 +10,14 @@ When adding tools, follow this systematic approach using the established pattern
Create these files in the correct directories:
```
frontend/src/hooks/tools/[toolName]/
frontend/editor/src/hooks/tools/[toolName]/
├── use[ToolName]Parameters.ts # Parameter definitions and validation
└── use[ToolName]Operation.ts # Tool operation logic using useToolOperation
frontend/src/components/tools/[toolName]/
frontend/editor/src/components/tools/[toolName]/
└── [ToolName]Settings.tsx # Settings UI component (if needed)
frontend/src/tools/
frontend/editor/src/tools/
└── [ToolName].tsx # Main tool component
```
@@ -128,7 +128,7 @@ export default [ToolName] as ToolComponent;
## 3. Register Tool in System
Update these files to register your new tool:
**Tool Registry** (`frontend/src/data/useTranslatedToolRegistry.tsx`):
**Tool Registry** (`frontend/editor/src/data/useTranslatedToolRegistry.tsx`):
1. Add imports at the top:
```typescript
import [ToolName] from "../tools/[ToolName]";
@@ -155,7 +155,7 @@ import [ToolName]Settings from "../components/tools/[toolName]/[ToolName]Setting
## 4. Add Tooltips (Optional but Recommended)
Create user-friendly tooltips to help non-technical users understand your tool. **Use simple, clear language - avoid technical jargon:**
**Tooltip Hook** (`frontend/src/components/tooltips/use[ToolName]Tips.ts`):
**Tooltip Hook** (`frontend/editor/src/components/tooltips/use[ToolName]Tips.ts`):
```typescript
import { useTranslation } from 'react-i18next';
import { TooltipContent } from '../../types/tips';
@@ -202,7 +202,7 @@ const [ToolName] = (props: BaseToolProps) => {
## 5. Add Translations
Update translation files. **Important: Only update `en-GB` files** - other languages are handled separately.
**File to update:** `frontend/public/locales/en-GB/translation.toml`
**File to update:** `frontend/editor/public/locales/en-GB/translation.toml`
**Required Translation Keys**:
```toml
+49 -26
View File
@@ -139,9 +139,9 @@ The project structure is defined in `engine/pyproject.toml`. Any new dependencie
#### Environment Variables
- All `VITE_*` variables must be declared in the appropriate committed env file:
- `frontend/.env` — core, proprietary, and shared vars
- `frontend/.env.saas` — SaaS-only vars (layered on top of `.env` in SaaS mode)
- `frontend/.env.desktop` — desktop (Tauri)-only vars (layered on top of `.env` in desktop mode)
- `frontend/editor/.env` — core, proprietary, and shared vars
- `frontend/editor/.env.saas` — SaaS-only vars (layered on top of `.env` in SaaS mode)
- `frontend/editor/.env.desktop` — desktop (Tauri)-only vars (layered on top of `.env` in desktop mode)
- These files are committed to Git and must not contain private keys
- Local overrides (API keys, machine-specific settings) go in uncommitted sibling `.env.local` / `.env.saas.local` / `.env.desktop.local` files — Vite automatically layers them on top
- Never use `|| 'hardcoded-fallback'` inline — put defaults in the committed env files
@@ -152,7 +152,7 @@ The project structure is defined in `engine/pyproject.toml`. Any new dependencie
#### Import Paths - CRITICAL
**ALWAYS use `@app/*` for imports.** Do not use `@core/*` or `@proprietary/*` unless explicitly wrapping/extending a lower layer implementation.
For a broader explanation of the frontend layering and override architecture, see [frontend/DeveloperGuide.md](frontend/DeveloperGuide.md).
For a broader explanation of the frontend layering and override architecture, see [frontend/editor/DeveloperGuide.md](frontend/editor/DeveloperGuide.md).
```typescript
// ✅ CORRECT - Use @app/* for all imports
@@ -183,26 +183,26 @@ Use this pattern for desktop-specific or proprietary-specific features WITHOUT r
**Example - Desktop-specific footer:**
```typescript
// core/components/rightRail/RightRailFooterExtensions.tsx (stub)
interface RightRailFooterExtensionsProps {
// core/components/workbenchBar/WorkbenchBarFooterExtensions.tsx (stub)
interface WorkbenchBarFooterExtensionsProps {
className?: string;
}
export function RightRailFooterExtensions(_props: RightRailFooterExtensionsProps) {
export function WorkbenchBarFooterExtensions(_props: WorkbenchBarFooterExtensionsProps) {
return null; // Stub - does nothing in web builds
}
```
```tsx
// desktop/components/rightRail/RightRailFooterExtensions.tsx (real implementation)
// desktop/components/workbenchBar/WorkbenchBarFooterExtensions.tsx (real implementation)
import { Box } from '@mantine/core';
import { BackendHealthIndicator } from '@app/components/BackendHealthIndicator';
interface RightRailFooterExtensionsProps {
interface WorkbenchBarFooterExtensionsProps {
className?: string;
}
export function RightRailFooterExtensions({ className }: RightRailFooterExtensionsProps) {
export function WorkbenchBarFooterExtensions({ className }: WorkbenchBarFooterExtensionsProps) {
return (
<Box className={className}>
<BackendHealthIndicator />
@@ -212,15 +212,15 @@ export function RightRailFooterExtensions({ className }: RightRailFooterExtensio
```
```tsx
// core/components/shared/RightRail.tsx (usage - works in ALL builds)
import { RightRailFooterExtensions } from '@app/components/rightRail/RightRailFooterExtensions';
// core/components/shared/WorkbenchBar.tsx (usage - works in ALL builds)
import { WorkbenchBarFooterExtensions } from '@app/components/workbenchBar/WorkbenchBarFooterExtensions';
export function RightRail() {
export function WorkbenchBar() {
return (
<div>
{/* In web builds: renders nothing (stub returns null) */}
{/* In desktop builds: renders BackendHealthIndicator */}
<RightRailFooterExtensions className="right-rail-footer" />
<WorkbenchBarFooterExtensions className="workbench-bar-footer" />
</div>
);
}
@@ -243,7 +243,7 @@ Frontend designed for **stateful document processing**:
- No file reloading between tools - performance critical for large PDFs (up to 100GB+)
#### FileContext - Central State Management
**Location**: `frontend/src/core/contexts/FileContext.tsx`
**Location**: `frontend/editor/src/core/contexts/FileContext.tsx`
- **Active files**: Currently loaded PDFs and their variants
- **Tool navigation**: Current mode (viewer/pageEditor/fileEditor/toolName)
- **Memory management**: PDF document cleanup, blob URL lifecycle, Web Worker management
@@ -268,7 +268,7 @@ Without cleanup: browser crashes with memory leaks.
**Architecture**: Modular hook-based system with clear separation of concerns:
- **useToolOperation** (`frontend/src/core/hooks/tools/shared/useToolOperation.ts`): Main orchestrator hook
- **useToolOperation** (`frontend/editor/src/core/hooks/tools/shared/useToolOperation.ts`): Main orchestrator hook
- Coordinates all tool operations with consistent interface
- Integrates with FileContext for operation tracking
- Handles validation, error handling, and UI state management
@@ -356,7 +356,7 @@ return useToolOperation({
### Frontend Directory Structure
The frontend is organized with a clear separation of concerns:
- **`frontend/src/core/`**: Main application code (shared, production-ready components)
- **`frontend/editor/src/core/`**: Main application code (shared, production-ready components)
- **`core/components/`**: React components organized by feature
- `core/components/tools/`: Individual PDF tool implementations
- `core/components/viewer/`: PDF viewer components
@@ -374,17 +374,17 @@ The frontend is organized with a clear separation of concerns:
- **`core/data/`**: Static data (tool taxonomy, etc.)
- **`core/services/`**: Business logic services (PDF processing, storage, etc.)
- **`frontend/src/desktop/`**: Desktop-specific (Tauri) code
- **`frontend/src/proprietary/`**: Proprietary/licensed features
- **`frontend/src-tauri/`**: Tauri (Rust) native desktop application code
- **`frontend/public/`**: Static assets served directly
- **`frontend/editor/src/desktop/`**: Desktop-specific (Tauri) code
- **`frontend/editor/src/proprietary/`**: Proprietary/licensed features
- **`frontend/editor/src-tauri/`**: Tauri (Rust) native desktop application code
- **`frontend/editor/public/`**: Static assets served directly
- `public/locales/`: Translation JSON files
### Component Architecture
- **Static Assets**: CSS, JS, and resources in `src/main/resources/static/` (legacy) + `frontend/public/` (modern)
- **Static Assets**: CSS, JS, and resources in `src/main/resources/static/` (legacy) + `frontend/editor/public/` (modern)
- **Internationalization**:
- Backend: `messages_*.properties` files
- Frontend: JSON files in `frontend/public/locales/` (converted from .properties)
- Frontend: JSON files in `frontend/editor/public/locales/` (converted from .properties)
- Conversion Script: `scripts/convert_properties_to_json.py`
### Configuration Modes
@@ -409,7 +409,7 @@ The frontend is organized with a clear separation of concerns:
4. **Code Style**: Spotless enforces Google Java Format automatically (`task backend:format`)
5. **Translations**:
- Backend: Use helper scripts in `/scripts` for multi-language updates
- Frontend: Update JSON files in `frontend/public/locales/` or use conversion script
- Frontend: Update JSON files in `frontend/editor/public/locales/` or use conversion script
6. **Documentation**: API docs auto-generated and available at `/swagger-ui/index.html`
## Frontend Architecture Status
@@ -427,11 +427,11 @@ The frontend is organized with a clear separation of concerns:
## Translation Rules
- **CRITICAL**: Always update translations in `en-GB` only, never `en-US`
- Translation files are located in `frontend/public/locales/`
- Translation files are located in `frontend/editor/public/locales/`
## Important Notes
- **Java Version**: Minimum JDK 21, supports and recommends JDK 25
- **Java Version**: Requires JDK 25.
- **Lombok**: Used extensively - ensure IDE plugin is installed
- **File Persistence**:
- **Backend**: Designed to be stateless - files are processed in memory/temp locations only
@@ -458,3 +458,26 @@ The frontend is organized with a clear separation of concerns:
- Confirm approach before making structural changes
- Request guidance on preferences (cross-platform vs specific tools, etc.)
- Verify understanding of requirements before proceeding
## Stack reality check (don't trust LLM training data) <!-- bleeding-edge-stack-note -->
This codebase is on bleeding-edge versions of its core JVM stack: **Spring Boot 4.0.6**,
**Jackson 3 (`tools.jackson`)**, **JDK 21/25 source/target with JDK 25 toolchain**.
All three are *post*-2024 releases and your training corpus is overwhelmingly Spring Boot 2/3 and
Jackson 2 patterns — those patterns will compile, run differently, or hallucinate APIs that no
longer exist.
Before writing or editing Spring / Jackson / JDK code:
1. Open an existing module in `app/core/` or `app/common/` and grep for the actual imports being
used — `import tools.jackson...` not `import com.fasterxml.jackson...`, and the new
`org.springframework.boot` 4.x package layout.
2. If you're not sure whether an API exists in this stack version, **check the source on disk
first** (the dependency JARs are downloaded under `~/.gradle/caches/modules-2/`).
3. Do not silently downgrade a Spring Boot 4 pattern to a Spring Boot 3 equivalent. If something
doesn't work, surface it to the human — don't guess.
Same goes for Jackson 3's API surface (renamed `ObjectMapper` builder methods, new
`tools.jackson.databind` namespace) and JDK 25 preview features. Ground your code in this repo's
actual imports, not what worked three years ago.
+24 -22
View File
@@ -11,7 +11,7 @@ This guide focuses on developing for Stirling 2.0, including both the React fron
**Stirling 2.0** is built using:
**Backend:**
- Spring Boot (Java 21+, JDK 25 recommended)
- Spring Boot (requires JDK 25)
- PDFBox for core PDF operations
- LibreOffice for document conversions
- qpdf for PDF optimization
@@ -45,7 +45,7 @@ This guide focuses on developing for Stirling 2.0, including both the React fron
- [Task](https://taskfile.dev/installation/) — unified command runner (recommended)
- Docker
- Git
- Java JDK 21 or later (JDK 25 recommended)
- Java JDK 25
- Node.js 18+ and npm (required for frontend development)
- Gradle 7.0 or later (Included within the repo)
- [uv](https://docs.astral.sh/uv/) — Python package manager (required for engine development)
@@ -61,7 +61,7 @@ This guide focuses on developing for Stirling 2.0, including both the React fron
cd Stirling-PDF
```
2. Install Docker and JDK 21 (or JDK 25 recommended) if not already installed.
2. Install Docker and JDK 25 if not already installed.
3. Install a recommended Java IDE such as Eclipse, IntelliJ, or VSCode
1. Only VSCode
@@ -128,24 +128,26 @@ See [the frontend README](frontend/README.md#tauri) for detailed build instructi
Stirling-PDF/
├── .github/ # GitHub-specific files (workflows, issue templates)
├── configs/ # Configuration files used by stirling at runtime (generated at runtime)
├── frontend/ # React SPA frontend (Stirling 2.0)
│ ├── src/
│ │ ├── components/ # React components
│ │ ├── tools/ # Tool-specific React components
│ │ ├── hooks/ # Custom React hooks
│ │ ├── services/ # API and utility services
│ │ ├── types/ # TypeScript type definitions
│ │ └── utils/ # Utility functions
│ ├── src-tauri/ # Tauri desktop app configuration
│ │ ├── src/ # Rust backend code
│ │ ├── libs/ # JAR files (generated by build scripts)
│ │ ├── runtime/ # Bundled JRE (generated by build scripts)
│ │ ├── Cargo.toml # Rust dependencies
│ │ └── tauri.conf.json # Tauri configuration
│ ├── public/
│ │ └── locales/ # Internationalization files (JSON)
│ ├── package.json # Frontend dependencies
│ └── vite.config.ts # Vite configuration
├── frontend/ # Frontend workspace (Stirling 2.0)
│ ├── editor/ # PDF editor app (the original React SPA)
│ │ ├── src/
│ │ │ ├── components/ # React components
│ │ │ ├── tools/ # Tool-specific React components
│ │ │ ├── hooks/ # Custom React hooks
│ │ │ ├── services/ # API and utility services
│ │ │ ├── types/ # TypeScript type definitions
│ │ │ └── utils/ # Utility functions
│ │ ├── src-tauri/ # Tauri desktop app configuration
│ │ │ ├── src/ # Rust backend code
│ │ │ ├── libs/ # JAR files (generated by build scripts)
│ │ │ ├── runtime/ # Bundled JRE (generated by build scripts)
│ │ │ ├── Cargo.toml # Rust dependencies
│ │ │ └── tauri.conf.json # Tauri configuration
│ │ ├── public/
│ │ │ └── locales/ # Internationalization files (JSON)
│ │ └── vite.config.ts # Vite configuration
│ ├── package.json # Shared workspace dependencies
│ └── eslint.config.mjs # Shared lint config
├── customFiles/ # Custom static files and templates (generated at runtime used to replace existing files)
├── docs/ # Documentation files
├── exampleYmlFiles/ # Example YAML configuration files
@@ -445,7 +447,7 @@ For Stirling 2.0, new features are built as React components:
1. **Create the React Component:**
```typescript
// frontend/src/tools/NewTool.tsx
// frontend/editor/src/tools/NewTool.tsx
import { useState } from 'react';
import { Button, FileInput, Container } from '@mantine/core';
+2
View File
@@ -6,6 +6,8 @@ Portions of this software are licensed as follows:
* All content that resides under the "app/proprietary/" directory of this repository,
if that directory exists, is licensed under the license defined in "app/proprietary/LICENSE".
* All content that resides under the "app/saas/" directory of this repository,
if that directory exists, is licensed under the license defined in "app/saas/LICENSE".
* All content that resides under the "engine/" directory of this repository,
if that directory exists, is licensed under the license defined in "engine/LICENSE".
* All content that resides under the "frontend/src/proprietary/" directory of this repository,
+4 -4
View File
@@ -3,7 +3,7 @@ version: '3'
output: prefixed
vars:
FIND_FREE_PORT_PS: powershell -NoProfile -File scripts\find-free-port.ps1 -Preferred
FIND_FREE_PORT_PS: powershell -NoProfile -File scripts/find-free-port.ps1
FIND_FREE_PORT_SH: bash scripts/find-free-port.sh
includes:
@@ -45,7 +45,7 @@ tasks:
desc: "Start backend + frontend concurrently on free ports"
vars:
PORTS:
sh: '{{if eq OS "windows"}}{{.FIND_FREE_PORT_PS}} 8080,5173{{else}}{{.FIND_FREE_PORT_SH}} 8080 5173{{end}}'
sh: '{{if eq OS "windows"}}{{.FIND_FREE_PORT_PS}} 8080 5173{{else}}{{.FIND_FREE_PORT_SH}} 8080 5173{{end}}'
BACKEND_PORT: '{{index (splitList "\n" .PORTS) 0}}'
FRONTEND_PORT: '{{index (splitList "\n" .PORTS) 1}}'
deps:
@@ -62,7 +62,7 @@ tasks:
desc: "Start backend + frontend + engine concurrently on free ports"
vars:
PORTS:
sh: '{{if eq OS "windows"}}{{.FIND_FREE_PORT_PS}} 8080,5173,5001{{else}}{{.FIND_FREE_PORT_SH}} 8080 5173 5001{{end}}'
sh: '{{if eq OS "windows"}}{{.FIND_FREE_PORT_PS}} 8080 5173 5001{{else}}{{.FIND_FREE_PORT_SH}} 8080 5173 5001{{end}}'
BACKEND_PORT: '{{index (splitList "\n" .PORTS) 0}}'
FRONTEND_PORT: '{{index (splitList "\n" .PORTS) 1}}'
ENGINE_PORT: '{{index (splitList "\n" .PORTS) 2}}'
@@ -74,7 +74,7 @@ tasks:
vars:
PORT: '{{.BACKEND_PORT}}'
AIENGINE_URL: 'http://localhost:{{.ENGINE_PORT}}'
- task: frontend:dev:prototypes
- task: frontend:dev
vars:
PORT: '{{.FRONTEND_PORT}}'
BACKEND_URL: 'http://localhost:{{.BACKEND_PORT}}'
+1 -1
View File
@@ -86,7 +86,7 @@ If you're using Tauri's built-in updater feature:
## Configuration Files
### 1. Tauri Configuration (frontend/src-tauri/tauri.conf.json)
### 1. Tauri Configuration (frontend/editor/src-tauri/tauri.conf.json)
The Windows signing configuration is already set up:
+29
View File
@@ -51,4 +51,33 @@ dependencies {
api 'org.simplejavamail:outlook-module:8.12.6' // MSG file support
api 'jakarta.mail:jakarta.mail-api:2.1.5'
runtimeOnly 'org.eclipse.angus:angus-mail:2.0.5'
// Tabula table extraction — used by the shared PDF parser and directly by downstream modules.
// api-scoped so downstream modules (core, proprietary) retain it on their compile classpath.
api ('technology.tabula:tabula:1.0.5') {
exclude group: 'org.slf4j', module: 'slf4j-simple'
exclude group: 'org.bouncycastle', module: 'bcprov-jdk15on'
exclude group: 'com.google.code.gson', module: 'gson'
}
api 'com.stirling:jpdfium:1.0.1'
// -PjpdfiumPlatforms=all|<csv of linux-x64,linux-arm64,darwin-x64,darwin-arm64,windows-x64>
def jpdfiumPlatformsProp = (project.findProperty('jpdfiumPlatforms') ?: 'all').toString().trim()
def jpdfiumAllPlatforms = ['linux-x64', 'linux-arm64', 'darwin-x64', 'darwin-arm64', 'windows-x64']
def jpdfiumPlatforms = jpdfiumPlatformsProp == 'all'
? jpdfiumAllPlatforms
: jpdfiumPlatformsProp.split(',').collect { it.trim() }.findAll { it }
def jpdfiumInvalid = jpdfiumPlatforms.findAll { !jpdfiumAllPlatforms.contains(it) }
if (jpdfiumInvalid) {
throw new GradleException("Unknown jpdfiumPlatforms value(s): ${jpdfiumInvalid.join(', ')}. " +
"Valid: ${jpdfiumAllPlatforms.join(', ')} or 'all'.")
}
logger.lifecycle("JPDFium native platforms: ${jpdfiumPlatforms.join(', ')}")
jpdfiumPlatforms.each { platform ->
runtimeOnly "com.stirling:jpdfium-natives-${platform}:1.0.1"
}
// ArchUnit: enforces module dependency direction (see ArchitectureTest)
testImplementation 'com.tngtech.archunit:archunit-junit5:1.4.2'
}
@@ -0,0 +1,73 @@
package stirling.software.SPDF.pdf.parser;
import static stirling.software.SPDF.pdf.parser.PdfModels.*;
import java.io.IOException;
import java.util.List;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.springframework.context.annotation.Primary;
import org.springframework.stereotype.Service;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
/**
* Chains table parsers in priority order: Tabula lattice → Tabula stream → {@link
* LineAlignmentTableParser}. The first parser returning a result above {@link
* #TABULA_CONFIDENCE_THRESHOLD} wins; results from different parsers are never mixed on one page.
*/
@Service
@Primary
@RequiredArgsConstructor
@Slf4j
public class CompositeTableParser implements TableParser {
/** Min Tabula confidence to accept results; below this LineAlignment is tried instead. */
static final float TABULA_CONFIDENCE_THRESHOLD = 0.5f;
private final TabulaTableParser tabulaParser;
private final LineAlignmentTableParser lineAlignmentParser;
@Override
public List<TableFragment> parse(PDDocument document, RawPage rawPage) throws IOException {
// Step 1: Tabula lattice mode (ruled/bordered tables).
List<TableFragment> latticeResults = filterConfident(tabulaParser.parse(document, rawPage));
if (!latticeResults.isEmpty()) {
log.debug(
"Page {}: using Tabula lattice ({} table(s))",
rawPage.pageNumber(),
latticeResults.size());
return latticeResults;
}
// Step 2: Tabula stream mode (borderless/whitespace-delimited tables).
// parseStream is not on the TableParser interface — this intentionally couples to the
// concrete TabulaTableParser since stream mode is a Tabula-specific concept.
List<TableFragment> streamResults =
filterConfident(tabulaParser.parseStream(document, rawPage));
if (!streamResults.isEmpty()) {
log.debug(
"Page {}: using Tabula stream ({} table(s))",
rawPage.pageNumber(),
streamResults.size());
return streamResults;
}
// Step 3: Geometry-based line-alignment fallback.
List<TableFragment> lineResults = lineAlignmentParser.parse(document, rawPage);
if (!lineResults.isEmpty()) {
log.debug(
"Page {}: using LineAlignment ({} table(s))",
rawPage.pageNumber(),
lineResults.size());
return lineResults;
}
return List.of();
}
private List<TableFragment> filterConfident(List<TableFragment> tables) {
return tables.stream().filter(t -> t.confidence() >= TABULA_CONFIDENCE_THRESHOLD).toList();
}
}
@@ -0,0 +1,528 @@
package stirling.software.SPDF.pdf.parser;
import static stirling.software.SPDF.pdf.parser.PdfModels.*;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collections;
import java.util.Comparator;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Optional;
import java.util.TreeMap;
import java.util.regex.Pattern;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.springframework.stereotype.Service;
import lombok.extern.slf4j.Slf4j;
/**
* Fallback {@link TableParser} for borderless financial tables using text geometry.
*
* <p>Identifies "anchor lines" (≥2 numeric tokens), builds a column grid from their right-edge
* positions, groups vertically proximate anchor lines into table candidates, then scores each group
* on column consistency and anchor density (confidence ceiling 0.85).
*/
@Service
@Slf4j
public class LineAlignmentTableParser implements TableParser {
/** Width in points of each column position bucket. */
static final float COLUMN_BUCKET_PT = 5f;
/** Tolerance in buckets when matching a token's right-edge to a confirmed column position. */
private static final int COLUMN_MATCH_BUCKETS = 2;
/** Maximum gap (as a multiple of modal line spacing) before splitting a group. */
private static final float MAX_GAP_FACTOR = 2.5f;
/** Minimum anchor rows (numeric-heavy) to form a valid table. */
static final int MIN_TABLE_ROWS = 3;
/** Minimum confirmed column positions to form a valid table. */
static final int MIN_COLUMNS = 2;
/**
* Min fraction of anchor lines a column must appear on to be confirmed (permissive for N/A
* rows).
*/
private static final double COLUMN_MIN_FREQUENCY = 0.40;
/**
* Matches financial numeric tokens: integers, decimals, parenthetical negatives, currency,
* percent, nil dashes.
*/
private static final Pattern NUMERIC =
Pattern.compile("^[\\(\\-\\$£€¥]?\\d[\\d,\\.]*[\\)%]?$|^[-–—]$");
/**
* Lines within this y-distance are merged into one row (restores rows split by LineBuilder's
* column-gap logic).
*/
static final float ROW_MERGE_TOLERANCE_PT = 2f;
// ── public API ───────────────────────────────────────────────────────────────────────────────
@Override
public List<TableFragment> parse(PDDocument document, RawPage rawPage) throws IOException {
List<RawLine> lines = rawPage.lines();
if (lines.size() < MIN_TABLE_ROWS) return List.of();
float modalSpacing = computeModalSpacing(lines);
List<TokenizedLine> tokenized =
mergeCoincidentLines(lines.stream().map(this::tokenize).toList());
List<TokenizedLine> anchors = tokenized.stream().filter(TokenizedLine::isAnchor).toList();
if (anchors.size() < MIN_TABLE_ROWS) return List.of();
List<Float> columnGrid = buildColumnGrid(anchors);
if (columnGrid.size() < MIN_COLUMNS) {
log.debug(
"Page {}: LineAlignment — fewer than {} confirmed columns, skipping",
rawPage.pageNumber(),
MIN_COLUMNS);
return List.of();
}
List<List<TokenizedLine>> groups = groupRows(tokenized, columnGrid, modalSpacing);
List<TableFragment> results = new ArrayList<>();
for (int i = 0; i < groups.size(); i++) {
buildFragment(groups.get(i), columnGrid, rawPage.pageNumber(), i)
.ifPresent(results::add);
}
log.debug(
"Page {}: LineAlignment detected {} table(s) ({} anchor lines, {} columns)",
rawPage.pageNumber(),
results.size(),
anchors.size(),
columnGrid.size());
return results;
}
// ── coincident-line merging ──────────────────────────────────────────────────────────────────
/**
* Merges tokenised lines sharing the same y-position into one row, rejoining label/value halves
* split by LineBuilder.
*/
List<TokenizedLine> mergeCoincidentLines(List<TokenizedLine> tokenized) {
if (tokenized.size() < 2) return tokenized;
List<TokenizedLine> result = new ArrayList<>();
int i = 0;
while (i < tokenized.size()) {
float baseY = tokenized.get(i).line().bounds().y();
int j = i + 1;
while (j < tokenized.size()
&& Math.abs(tokenized.get(j).line().bounds().y() - baseY)
<= ROW_MERGE_TOLERANCE_PT) {
j++;
}
if (j == i + 1) {
result.add(tokenized.get(i));
} else {
result.add(mergeGroup(tokenized.subList(i, j)));
}
i = j;
}
return result;
}
private TokenizedLine mergeGroup(List<TokenizedLine> group) {
List<TextFragment> mergedFragments =
group.stream()
.flatMap(tl -> tl.line().fragments().stream())
.sorted(Comparator.comparingDouble(f -> f.bounds().x()))
.toList();
Bounds mergedBounds =
group.stream()
.map(tl -> tl.line().bounds())
.reduce(Bounds::merge)
.orElse(group.get(0).line().bounds());
RawLine mergedLine =
new RawLine(
group.get(0).line().lineId(),
mergedFragments,
mergedBounds,
group.get(0).line().pageNumber());
return tokenize(mergedLine);
}
// ── tokenisation ─────────────────────────────────────────────────────────────────────────────
/**
* Splits fragments into word-level tokens; x-positions are estimated linearly within each
* fragment.
*/
TokenizedLine tokenize(RawLine line) {
List<LineToken> tokens = new ArrayList<>();
for (TextFragment frag : line.fragments()) {
tokens.addAll(tokensFromFragment(frag));
}
List<LineToken> numeric = tokens.stream().filter(LineToken::numeric).toList();
return new TokenizedLine(line, tokens, numeric);
}
private List<LineToken> tokensFromFragment(TextFragment frag) {
String raw = frag.text();
if (raw == null || raw.isBlank()) return List.of();
float fragX = frag.bounds().x();
float fragWidth = frag.bounds().width();
int rawLen = raw.length();
List<LineToken> result = new ArrayList<>();
int offset = 0;
for (String part : raw.split("\\s+")) {
if (part.isEmpty()) {
offset++;
continue;
}
int idx = raw.indexOf(part, offset);
if (idx < 0) idx = offset;
float tokenX = rawLen > 0 ? fragX + ((float) idx / rawLen) * fragWidth : fragX;
float tokenRight =
rawLen > 0
? fragX + ((float) (idx + part.length()) / rawLen) * fragWidth
: fragX + fragWidth;
result.add(new LineToken(part, tokenX, tokenRight, NUMERIC.matcher(part).matches()));
offset = idx + part.length();
}
return result;
}
// ── column grid ──────────────────────────────────────────────────────────────────────────────
/**
* Returns confirmed column right-edge positions — those appearing on ≥ {@value
* #COLUMN_MIN_FREQUENCY} × N anchor lines.
*/
private List<Float> buildColumnGrid(List<TokenizedLine> anchors) {
// bucket → set of line indices that contributed a numeric token to that bucket
Map<Integer, List<Integer>> bucketLines = new HashMap<>();
for (int i = 0; i < anchors.size(); i++) {
for (LineToken t : anchors.get(i).numeric()) {
int bucket = bucket(t.right());
bucketLines.computeIfAbsent(bucket, k -> new ArrayList<>()).add(i);
}
}
int minHits =
Math.max(MIN_TABLE_ROWS, (int) Math.ceil(anchors.size() * COLUMN_MIN_FREQUENCY));
// Confirmed buckets → average right-edge for that bucket
TreeMap<Integer, Float> confirmed = new TreeMap<>();
for (Map.Entry<Integer, List<Integer>> entry : bucketLines.entrySet()) {
// Count distinct lines
long distinctLines = entry.getValue().stream().distinct().count();
if (distinctLines >= minHits) {
double avg =
entry.getValue().stream()
.distinct() // weight each line equally regardless of token count
.mapToDouble(
lineIdx ->
avgRightEdgeForBucket(
anchors, lineIdx, entry.getKey()))
.average()
.orElse(entry.getKey() * (double) COLUMN_BUCKET_PT);
confirmed.put(entry.getKey(), (float) avg);
}
}
return new ArrayList<>(confirmed.values()); // already sorted by bucket (left to right)
}
/**
* Returns the average right-edge position of tokens in {@code line} whose bucket matches {@code
* targetBucket}, falling back to the bucket's nominal centre when no tokens match.
*/
private double avgRightEdgeForBucket(
List<TokenizedLine> anchors, int lineIdx, int targetBucket) {
return anchors.get(lineIdx).numeric().stream()
.filter(t -> bucket(t.right()) == targetBucket)
.mapToDouble(LineToken::right)
.average()
.orElse(targetBucket * (double) COLUMN_BUCKET_PT);
}
// ── grouping ─────────────────────────────────────────────────────────────────────────────────
/**
* Groups anchor lines into table candidates, including adjacent label rows; a gap &gt;
* MAX_GAP_FACTOR × modal spacing splits groups.
*/
private List<List<TokenizedLine>> groupRows(
List<TokenizedLine> all, List<Float> columnGrid, float modalSpacing) {
float maxGap = modalSpacing > 0 ? modalSpacing * MAX_GAP_FACTOR : 30f;
List<List<TokenizedLine>> groups = new ArrayList<>();
List<TokenizedLine> current = new ArrayList<>();
for (int i = 0; i < all.size(); i++) {
TokenizedLine tl = all.get(i);
boolean fits = tl.isAnchor() && matchesGrid(tl, columnGrid);
if (current.isEmpty()) {
if (fits) current.add(tl);
continue;
}
float gap =
tl.line().bounds().y()
- current.get(current.size() - 1).line().bounds().bottom();
if (gap > maxGap) {
groups.add(current);
current = new ArrayList<>();
if (fits) current.add(tl);
continue;
}
if (fits) {
current.add(tl);
} else if (!tl.line().text().isBlank()) {
// Include non-anchor lines (labels) only if they have text and are within
// proximity.
current.add(tl);
}
}
if (!current.isEmpty()) groups.add(current);
return groups.stream().filter(g -> hasEnoughAnchorRows(g, columnGrid)).toList();
}
private boolean hasEnoughAnchorRows(List<TokenizedLine> group, List<Float> columnGrid) {
return group.stream().filter(r -> r.isAnchor() && matchesGrid(r, columnGrid)).count()
>= MIN_TABLE_ROWS;
}
/** A line "matches" the grid when ≥ 60 % of its numeric tokens land in confirmed columns. */
private boolean matchesGrid(TokenizedLine tl, List<Float> columnGrid) {
if (tl.numeric().isEmpty()) return false;
long matches =
tl.numeric().stream()
.filter(t -> nearestColumnIndex(t.right(), columnGrid) >= 0)
.count();
return (double) matches / tl.numeric().size() >= 0.60;
}
private boolean hasInconsistentColumnMatch(TokenizedLine tl, List<Float> columnGrid) {
if (tl.numeric().isEmpty()) return false;
long hits =
tl.numeric().stream()
.filter(t -> nearestColumnIndex(t.right(), columnGrid) >= 0)
.count();
return (double) hits / tl.numeric().size() < 0.60;
}
// ── fragment assembly ────────────────────────────────────────────────────────────────────────
private Optional<TableFragment> buildFragment(
List<TokenizedLine> group, List<Float> columnGrid, int pageNumber, int tableIndex) {
long anchorCount =
group.stream().filter(r -> r.isAnchor() && matchesGrid(r, columnGrid)).count();
if (anchorCount < MIN_TABLE_ROWS) return Optional.empty();
List<String> warnings = new ArrayList<>();
List<List<String>> rawRows = new ArrayList<>();
List<TableRow> rows = new ArrayList<>();
for (int rowIdx = 0; rowIdx < group.size(); rowIdx++) {
TokenizedLine tl = group.get(rowIdx);
List<String> rawRow = buildRawRow(tl, columnGrid);
rawRows.add(Collections.unmodifiableList(rawRow));
rows.add(buildTableRow(rowIdx, tl, rawRow, columnGrid));
}
// Column count = 1 label column + confirmed numeric columns
int colCount = columnGrid.size() + 1;
Bounds bounds = computeGroupBounds(group);
float confidence = computeConfidence(group, columnGrid, warnings);
return Optional.of(
new TableFragment(
"tbl-la-p" + pageNumber + "-" + tableIndex,
pageNumber,
bounds,
List.of(),
Collections.unmodifiableList(rows),
Collections.unmodifiableList(rawRows),
colCount,
confidence,
Collections.unmodifiableList(warnings),
null));
}
/**
* Builds a raw row as a list of strings: index 0 = label text, indices 1..N = column values.
*/
private List<String> buildRawRow(TokenizedLine tl, List<Float> columnGrid) {
String[] cells = new String[columnGrid.size() + 1];
Arrays.fill(cells, "");
// Separate label tokens (those not landing in any confirmed column) from column tokens.
List<String> labelParts = new ArrayList<>();
for (LineToken token : tl.all()) {
int col = nearestColumnIndex(token.right(), columnGrid);
if (col >= 0 && token.numeric()) {
int cellIdx = col + 1;
cells[cellIdx] =
cells[cellIdx].isEmpty()
? token.text()
: cells[cellIdx] + " " + token.text();
} else {
labelParts.add(token.text());
}
}
cells[0] = String.join(" ", labelParts).trim();
return Arrays.asList(cells);
}
private TableRow buildTableRow(
int rowIdx, TokenizedLine tl, List<String> rawRow, List<Float> columnGrid) {
List<TableCell> cells = new ArrayList<>(rawRow.size());
// Label cell: use the line's full bounds as an approximation.
cells.add(TableCell.of(0, rawRow.get(0), tl.line().bounds()));
for (int col = 0; col < columnGrid.size(); col++) {
String text = col + 1 < rawRow.size() ? rawRow.get(col + 1) : "";
float right = columnGrid.get(col);
float left = col > 0 ? columnGrid.get(col - 1) : right - 50f;
Bounds cellBounds =
new Bounds(
left,
tl.line().bounds().y(),
right - left,
tl.line().bounds().height());
cells.add(TableCell.of(col + 1, text, cellBounds));
}
return new TableRow(rowIdx, Collections.unmodifiableList(cells));
}
// ── confidence scoring ───────────────────────────────────────────────────────────────────────
/**
* Heuristic score in [0.0, 0.85] (ceiling keeps results below Tabula lattice which starts at
* 1.0). Base 0.70; +0.05/col beyond 2 (max +0.10); +0.05 at ≥5 anchors, +0.05 at ≥8; −0.15 if
* &gt;30 % of anchors have inconsistent columns; −0.10 if non-anchors outnumber anchors.
*/
private float computeConfidence(
List<TokenizedLine> group, List<Float> columnGrid, List<String> warnings) {
float score = 0.70f;
long anchorCount =
group.stream().filter(r -> r.isAnchor() && matchesGrid(r, columnGrid)).count();
long totalRows = group.size();
// More columns
int extraCols = Math.min(columnGrid.size() - MIN_COLUMNS, 2);
score += extraCols * 0.05f;
// More anchor rows
if (anchorCount >= 5) score += 0.05f;
if (anchorCount >= 8) score += 0.05f;
// Inconsistent column matching
long inconsistent =
group.stream()
.filter(TokenizedLine::isAnchor)
.filter(tl -> hasInconsistentColumnMatch(tl, columnGrid))
.count();
if (inconsistent > anchorCount * 0.30) {
score -= 0.15f;
warnings.add(
"Column match inconsistent on "
+ inconsistent
+ "/"
+ anchorCount
+ " anchor rows");
}
// Label-heavy
long nonAnchor = totalRows - anchorCount;
if (nonAnchor > anchorCount) {
score -= 0.10f;
warnings.add(
"Non-anchor rows ("
+ nonAnchor
+ ") outnumber anchor rows ("
+ anchorCount
+ ")");
}
return Math.max(0f, Math.min(0.85f, score));
}
// ── utility ──────────────────────────────────────────────────────────────────────────────────
/**
* Returns the grid index nearest to {@code rightEdge}, or -1 if none is within {@value
* #COLUMN_MATCH_BUCKETS} buckets.
*/
private int nearestColumnIndex(float rightEdge, List<Float> grid) {
int nearest = -1;
float minDist = COLUMN_MATCH_BUCKETS * COLUMN_BUCKET_PT + 1f;
for (int i = 0; i < grid.size(); i++) {
float dist = Math.abs(rightEdge - grid.get(i));
if (dist < minDist) {
minDist = dist;
nearest = i;
}
}
return nearest;
}
private Bounds computeGroupBounds(List<TokenizedLine> group) {
return group.stream()
.map(tl -> tl.line().bounds())
.reduce(Bounds::merge)
.orElse(new Bounds(0, 0, 0, 0));
}
/** Modal gap between consecutive line edges, used to calibrate the group-split threshold. */
private float computeModalSpacing(List<RawLine> lines) {
if (lines.size() < 2) return 0f;
Map<Float, Long> freq = new HashMap<>();
for (int i = 1; i < lines.size(); i++) {
float gap = lines.get(i).bounds().y() - lines.get(i - 1).bounds().bottom();
if (gap > 0) freq.merge(Math.round(gap / 2f) * 2f, 1L, Long::sum);
}
return freq.entrySet().stream()
.max(Map.Entry.comparingByValue())
.map(Map.Entry::getKey)
.orElse(0f);
}
private static int bucket(float x) {
return Math.round(x / COLUMN_BUCKET_PT);
}
// ── private data types ───────────────────────────────────────────────────────────────────────
/** A word-level token with an approximate right-edge x-position. */
record LineToken(String text, float x, float right, boolean numeric) {}
/** A {@link RawLine} with tokens pre-computed; an "anchor" has ≥ 2 numeric tokens. */
record TokenizedLine(RawLine line, List<LineToken> all, List<LineToken> numeric) {
boolean isAnchor() {
return numeric.size() >= 2;
}
}
}
@@ -0,0 +1,139 @@
package stirling.software.SPDF.pdf.parser;
import static stirling.software.SPDF.pdf.parser.PdfModels.*;
import java.util.ArrayList;
import java.util.Comparator;
import java.util.List;
import org.springframework.stereotype.Service;
import lombok.extern.slf4j.Slf4j;
/**
* Groups {@link TextFragment} objects into visual {@link RawLine}s using baseline proximity.
*
* <p>Fragments are on the same line when their baselines are within a font-size-derived tolerance.
* A new line starts whenever the horizontal gap exceeds an adaptive column-gap threshold ({@code
* max(effectiveWidth * COLUMN_GAP_RATIO, COLUMN_GAP_MIN_PT)}), splitting two-column text.
*/
@Service
@Slf4j
public class LineBuilder {
/** Baseline tolerance as a fraction of font size; 0.5 keeps mixed-size text on one line. */
private static final float BASELINE_TOLERANCE_FACTOR = 0.5f;
/** Absolute minimum tolerance so tiny font sizes don't collapse multi-line content. */
private static final float MIN_BASELINE_TOLERANCE = 2f;
/**
* Column-gap threshold as a fraction of page width; 0.10 clears tab stops but stays below
* two-column gutters.
*/
static final float COLUMN_GAP_RATIO = 0.10f;
/** Floor for the column-gap threshold so narrow pages don't over-split lines. */
static final float COLUMN_GAP_MIN_PT = 40f;
public List<RawLine> build(List<TextFragment> fragments, int pageNumber) {
if (fragments.isEmpty()) return List.of();
float effectiveWidth = inferEffectiveWidth(fragments);
float columnGapThreshold = Math.max(effectiveWidth * COLUMN_GAP_RATIO, COLUMN_GAP_MIN_PT);
log.debug(
"LineBuilder page {}: effectiveWidth={:.1f}pt, columnGapThreshold={:.1f}pt",
pageNumber,
effectiveWidth,
columnGapThreshold);
// Sort top-to-bottom first, then left-to-right within the same baseline band.
List<TextFragment> sorted =
fragments.stream()
.sorted(
Comparator.comparingDouble(TextFragment::baseline)
.thenComparingDouble(f -> f.bounds().x()))
.toList();
List<List<TextFragment>> groups = groupByBaseline(sorted, columnGapThreshold);
List<RawLine> lines = new ArrayList<>(groups.size());
for (int i = 0; i < groups.size(); i++) {
List<TextFragment> group =
groups.get(i).stream()
.sorted(Comparator.comparingDouble(f -> f.bounds().x()))
.toList();
Bounds lineBounds =
group.stream()
.map(TextFragment::bounds)
.reduce(Bounds::merge)
.orElse(new Bounds(0, 0, 0, 0));
lines.add(new RawLine("ln-p" + pageNumber + "-" + i, group, lineBounds, pageNumber));
}
return lines;
}
private List<List<TextFragment>> groupByBaseline(
List<TextFragment> sorted, float columnGapThreshold) {
List<List<TextFragment>> groups = new ArrayList<>();
List<TextFragment> current = new ArrayList<>();
float currentBaseline = Float.NaN;
for (TextFragment fragment : sorted) {
if (current.isEmpty()) {
current.add(fragment);
currentBaseline = fragment.baseline();
continue;
}
float maxFontSize =
Math.max(
fragment.fontSize(),
(float)
current.stream()
.mapToDouble(TextFragment::fontSize)
.max()
.orElse(0));
float tolerance =
Math.max(maxFontSize * BASELINE_TOLERANCE_FACTOR, MIN_BASELINE_TOLERANCE);
boolean sameBaseline = Math.abs(fragment.baseline() - currentBaseline) <= tolerance;
boolean columnGap = sameBaseline && hasColumnGap(fragment, current, columnGapThreshold);
if (sameBaseline && !columnGap) {
current.add(fragment);
// Anchor to the weighted mean baseline so long lines stay stable.
currentBaseline =
(currentBaseline * (current.size() - 1) + fragment.baseline())
/ current.size();
} else {
groups.add(current);
current = new ArrayList<>();
current.add(fragment);
currentBaseline = fragment.baseline();
}
}
if (!current.isEmpty()) groups.add(current);
return groups;
}
/**
* True when the gap from the rightmost fragment in {@code group} to {@code next} exceeds {@code
* threshold}.
*/
private static boolean hasColumnGap(
TextFragment next, List<TextFragment> group, float threshold) {
float lastRight = group.get(group.size() - 1).bounds().right();
return next.bounds().x() - lastRight > threshold;
}
/** Infers effective page width from the rightmost fragment right-edge plus a 10 % margin. */
private static float inferEffectiveWidth(List<TextFragment> fragments) {
double maxRight =
fragments.stream().mapToDouble(f -> f.bounds().right()).max().orElse(500.0);
return (float) maxRight * 1.10f;
}
}
@@ -0,0 +1,79 @@
package stirling.software.SPDF.pdf.parser;
import static stirling.software.SPDF.pdf.parser.PdfModels.*;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.apache.pdfbox.pdmodel.PDPage;
import org.apache.pdfbox.pdmodel.common.PDRectangle;
import org.springframework.stereotype.Service;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
/**
* Runs the per-page ingestion pipeline: {@link WordExtractingStripper} → {@link LineBuilder} →
* {@link TableParser}, producing a {@link PdfModels.ParsedPage} per page. The caller owns the
* {@link PDDocument} lifecycle.
*/
@Service
@RequiredArgsConstructor
@Slf4j
public class PdfIngester {
private final LineBuilder lineBuilder;
private final TableParser tableParser;
public List<ParsedPage> parse(PDDocument document) throws IOException {
return parse(document, document.getNumberOfPages());
}
public List<ParsedPage> parse(PDDocument document, int maxPages) throws IOException {
int pageCount = Math.min(document.getNumberOfPages(), maxPages);
List<ParsedPage> pages = new ArrayList<>(pageCount);
long fragmentsMs = 0;
long tablesMs = 0;
long t0 = System.currentTimeMillis();
for (int p = 1; p <= pageCount; p++) {
long ft = System.currentTimeMillis();
List<TextFragment> fragments = extractFragments(document, p);
fragmentsMs += System.currentTimeMillis() - ft;
PDPage page = document.getPage(p - 1);
PDRectangle mediaBox = page.getMediaBox();
List<RawLine> lines = lineBuilder.build(fragments, p);
RawPage rawPage = new RawPage(p, mediaBox.getWidth(), mediaBox.getHeight(), lines);
long tt = System.currentTimeMillis();
List<TableFragment> tables = tableParser.parse(document, rawPage);
tablesMs += System.currentTimeMillis() - tt;
log.debug(
"Page {}: {} fragments → {} lines, {} table(s)",
p,
fragments.size(),
lines.size(),
tables.size());
pages.add(new ParsedPage(p, mediaBox.getWidth(), mediaBox.getHeight(), tables, lines));
}
log.info(
"[timing] parse pages={} total={}ms fragments={}ms tables={}ms",
pageCount,
System.currentTimeMillis() - t0,
fragmentsMs,
tablesMs);
return pages;
}
private List<TextFragment> extractFragments(PDDocument document, int pageNumber)
throws IOException {
WordExtractingStripper stripper = new WordExtractingStripper(pageNumber);
stripper.getText(document);
return stripper.getFragments();
}
}
@@ -0,0 +1,148 @@
package stirling.software.SPDF.pdf.parser;
import java.io.IOException;
import java.util.List;
import java.util.Map;
import java.util.stream.Collectors;
import org.apache.pdfbox.pdmodel.PDDocument;
/**
* All PDF parser model types and the table-parser contract in one place.
*
* <p>Import as {@code import static stirling.software.SPDF.pdf.parser.PdfModels.*;} to use all
* nested types without qualification.
*/
public final class PdfModels {
private PdfModels() {}
// ── Geometry ──────────────────────────────────────────────────────────────
public record Bounds(float x, float y, float width, float height) {
public float right() {
return x + width;
}
public float bottom() {
return y + height;
}
public static Bounds merge(Bounds a, Bounds b) {
float x = Math.min(a.x, b.x);
float y = Math.min(a.y, b.y);
return new Bounds(
x, y, Math.max(a.right(), b.right()) - x, Math.max(a.bottom(), b.bottom()) - y);
}
}
// ── Text fragments and lines ──────────────────────────────────────────────
/**
* A contiguous run of text from a single PDF content-stream string operation. {@code baseline}
* is preserved separately from bounds for line-grouping — characters of different sizes on the
* same visual line share a baseline but differ in top Y.
*/
public record TextFragment(
String fragmentId,
String text,
Bounds bounds,
float baseline,
float fontSize,
String fontName,
boolean bold) {}
public record RawLine(
String lineId, List<TextFragment> fragments, Bounds bounds, int pageNumber) {
public String text() {
if (fragments.isEmpty()) return "";
StringBuilder sb = new StringBuilder();
TextFragment prev = null;
for (TextFragment f : fragments) {
if (prev != null) {
float gap = f.bounds().x() - prev.bounds().right();
float avgCharWidth = prev.bounds().width() / Math.max(prev.text().length(), 1);
if (gap > avgCharWidth * 0.5f) sb.append(' ');
}
sb.append(f.text());
prev = f;
}
return sb.toString();
}
public float dominantFontSize() {
return fragments.stream()
.collect(Collectors.groupingBy(TextFragment::fontSize, Collectors.counting()))
.entrySet()
.stream()
.max(Map.Entry.comparingByValue())
.map(Map.Entry::getKey)
.orElse(0f);
}
public boolean hasBold() {
return fragments.stream().anyMatch(TextFragment::bold);
}
}
public record RawPage(int pageNumber, float widthPt, float heightPt, List<RawLine> lines) {}
// ── Table model ───────────────────────────────────────────────────────────
/**
* A single cell within a {@link TableRow}. {@code colSpan} and {@code rowSpan} are always 1 in
* v1 — span detection is deferred but the fields keep the schema stable.
*/
public record TableCell(int colIndex, String text, Bounds bounds, int colSpan, int rowSpan) {
public static TableCell of(int colIndex, String text, Bounds bounds) {
return new TableCell(colIndex, text, bounds, 1, 1);
}
}
public record TableRow(int rowIndex, List<TableCell> cells) {}
/**
* A table as it appears on a single page. {@code headers} is empty in v1 — all rows are in
* {@code rows}. {@code rawRows} preserves exact Tabula text output for diagnostics. {@code
* confidence} is a heuristic score in [0.0, 1.0]. {@code continuedFromPage} is null in v1.
*/
public record TableFragment(
String tableId,
int pageNumber,
Bounds bounds,
List<TableRow> headers,
List<TableRow> rows,
List<List<String>> rawRows,
int columnCount,
float confidence,
List<String> warnings,
Integer continuedFromPage) {}
// ── Page output ───────────────────────────────────────────────────────────
public record ParsedPage(
int pageNumber,
float widthPt,
float heightPt,
List<TableFragment> tables,
List<RawLine> layoutLines) {}
// ── Parser contract ───────────────────────────────────────────────────────
/**
* Extracts tables from a single page of a PDF. The caller owns the document lifecycle;
* implementations must not close it.
*/
public interface TableParser {
/**
* @param document open PDF; must not be closed by the implementation
* @param rawPage page metadata and lines for the page to process
* @return zero or more table fragments found on the page, never null
*/
List<TableFragment> parse(PDDocument document, RawPage rawPage) throws IOException;
}
}
@@ -0,0 +1,256 @@
package stirling.software.SPDF.pdf.parser;
import static stirling.software.SPDF.pdf.parser.PdfModels.*;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Collection;
import java.util.Collections;
import java.util.List;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.springframework.stereotype.Service;
import lombok.extern.slf4j.Slf4j;
import technology.tabula.ObjectExtractor;
import technology.tabula.Page;
import technology.tabula.RectangularTextContainer;
import technology.tabula.Table;
import technology.tabula.extractors.BasicExtractionAlgorithm;
import technology.tabula.extractors.ExtractionAlgorithm;
import technology.tabula.extractors.SpreadsheetExtractionAlgorithm;
/**
* Primary {@link TableParser} implementation using Tabula's lattice-mode extraction.
*
* <h3>Algorithm</h3>
*
* Uses {@link SpreadsheetExtractionAlgorithm} (lattice mode), which detects tables from ruled lines
* (horizontal and vertical PDF path operators). This is reliable for tables with visible borders.
* Borderless or whitespace-delimited tables will not be detected — that requires stream mode, which
* is deferred to a later iteration.
*
* <h3>Coordinate system</h3>
*
* Tabula normalises page coordinates so that (0,0) is the top-left corner and Y increases downward,
* matching the {@link Bounds} convention used throughout this parser. No coordinate transformation
* is needed when mapping Tabula cell bounds to {@link Bounds}.
*
* <h3>Known limitations</h3>
*
* <ul>
* <li>Borderless tables are not detected (use stream mode, deferred).
* <li>Colspan and rowspan are not detected; all cells report colSpan=1, rowSpan=1.
* <li>Header rows are not identified; all rows appear in {@code rows}, headers is empty.
* <li>Cross-page table linking is not performed; each page is independent.
* <li>Rotated tables (90°/270° pages) may produce incorrect bounds.
* </ul>
*/
@Service
@Slf4j
public class TabulaTableParser implements TableParser {
/** Lattice mode — reliable for tables with visible ruled borders. */
@Override
public List<TableFragment> parse(PDDocument document, RawPage rawPage) throws IOException {
return parseWithAlgorithm(
document, rawPage, new SpreadsheetExtractionAlgorithm(), "lattice");
}
/**
* Convenience overload for callers that only have a page number, not a full {@link RawPage}.
*/
public List<TableFragment> parse(PDDocument document, int pageNumber) throws IOException {
return parse(document, new RawPage(pageNumber, 0f, 0f, List.of()));
}
/** Stream mode — whitespace-based column detection for borderless tables. */
public List<TableFragment> parseStream(PDDocument document, RawPage rawPage)
throws IOException {
return parseWithAlgorithm(document, rawPage, new BasicExtractionAlgorithm(), "stream");
}
private List<TableFragment> parseWithAlgorithm(
PDDocument document, RawPage rawPage, ExtractionAlgorithm algorithm, String modeName)
throws IOException {
int pageNumber = rawPage.pageNumber();
List<Table> tabulaTables;
try {
// Do NOT use try-with-resources: ObjectExtractor.close() closes the underlying
// PDDocument, which we don't own. The extractor holds no resources of its own.
ObjectExtractor extractor = new ObjectExtractor(document);
Page page = extractor.extract(pageNumber);
tabulaTables = new ArrayList<>(algorithm.extract(page));
} catch (Exception e) {
log.warn(
"Tabula {} extraction failed on page {}: {}",
modeName,
pageNumber,
e.getMessage());
return List.of();
}
if (tabulaTables.isEmpty()) {
log.debug("Page {}: no tables detected by Tabula ({})", pageNumber, modeName);
return List.of();
}
log.debug(
"Page {}: Tabula ({}) detected {} table(s)",
pageNumber,
modeName,
tabulaTables.size());
List<TableFragment> fragments = new ArrayList<>(tabulaTables.size());
for (int i = 0; i < tabulaTables.size(); i++) {
fragments.add(toFragment(tabulaTables.get(i), pageNumber, i));
}
return fragments;
}
// ── private helpers ──────────────────────────────────────────────────────────────────────────
private TableFragment toFragment(Table table, int pageNumber, int tableIndex) {
List<List<RectangularTextContainer>> tabulaRows = table.getRows();
List<String> warnings = new ArrayList<>();
List<List<String>> rawRows = buildRawRows(tabulaRows);
int colCount = inferColumnCount(rawRows, warnings);
List<TableRow> rows = buildRows(tabulaRows, colCount, warnings);
float confidence = computeConfidence(rawRows, colCount, warnings);
Bounds bounds = tableBounds(table);
String tableId = "tbl-p" + pageNumber + "-" + tableIndex;
if (!warnings.isEmpty()) {
log.warn("Page {}, table {}: {}", pageNumber, tableIndex, warnings);
}
return new TableFragment(
tableId,
pageNumber,
bounds,
List.of(), // headers: deferred to v2
rows,
rawRows,
colCount,
confidence,
Collections.unmodifiableList(warnings),
null); // continuedFromPage: deferred to v2
}
private List<List<String>> buildRawRows(List<List<RectangularTextContainer>> tabulaRows) {
List<List<String>> rawRows = new ArrayList<>(tabulaRows.size());
for (List<RectangularTextContainer> tabulaRow : tabulaRows) {
List<String> cells = new ArrayList<>(tabulaRow.size());
for (RectangularTextContainer cell : tabulaRow) {
cells.add(normaliseText(cell.getText()));
}
rawRows.add(Collections.unmodifiableList(cells));
}
return rawRows;
}
private List<TableRow> buildRows(
List<List<RectangularTextContainer>> tabulaRows, int colCount, List<String> warnings) {
List<TableRow> rows = new ArrayList<>(tabulaRows.size());
for (int rowIdx = 0; rowIdx < tabulaRows.size(); rowIdx++) {
List<RectangularTextContainer> tabulaRow = tabulaRows.get(rowIdx);
List<TableCell> cells = new ArrayList<>(tabulaRow.size());
for (int colIdx = 0; colIdx < tabulaRow.size(); colIdx++) {
RectangularTextContainer c = tabulaRow.get(colIdx);
Bounds cellBounds =
new Bounds(
(float) c.getX(),
(float) c.getY(),
(float) c.getWidth(),
(float) c.getHeight());
cells.add(TableCell.of(colIdx, normaliseText(c.getText()), cellBounds));
}
if (tabulaRow.size() != colCount) {
warnings.add(
"Row "
+ rowIdx
+ " has "
+ tabulaRow.size()
+ " cells; expected "
+ colCount);
}
rows.add(new TableRow(rowIdx, Collections.unmodifiableList(cells)));
}
return rows;
}
/**
* The canonical column count for a table is the size of the widest row. Tabula can produce
* uneven rows when a cell's ruling lines are partially missing.
*/
private int inferColumnCount(List<List<String>> rawRows, List<String> warnings) {
if (rawRows.isEmpty()) return 0;
int max = rawRows.stream().mapToInt(List::size).max().orElse(0);
int mode =
rawRows.stream()
.collect(
java.util.stream.Collectors.groupingBy(
List::size, java.util.stream.Collectors.counting()))
.entrySet()
.stream()
.max(java.util.Map.Entry.comparingByValue())
.map(java.util.Map.Entry::getKey)
.orElse(0);
if (max != mode) {
warnings.add("Inconsistent column count: modal=" + mode + " max=" + max);
}
return mode > 0 ? mode : max;
}
/**
* Heuristic confidence score in [0.0, 1.0].
*
* <ul>
* <li>Starts at 1.0.
* <li>-0.3 if only one column (single-column tables are usually not real tables).
* <li>-0.1 per row with an inconsistent column count, capped at -0.4.
* <li>-0.3 if the empty-cell ratio across all cells exceeds 80%.
* </ul>
*/
private float computeConfidence(
List<List<String>> rawRows, int colCount, List<String> warnings) {
if (rawRows.isEmpty() || colCount == 0) return 0f;
float score = 1.0f;
if (colCount == 1) score -= 0.3f;
long inconsistentRows = rawRows.stream().filter(r -> r.size() != colCount).count();
score -= Math.min(inconsistentRows * 0.1f, 0.4f);
long totalCells = rawRows.stream().mapToLong(List::size).sum();
long emptyCells =
rawRows.stream().flatMap(Collection::stream).filter(String::isBlank).count();
if (totalCells > 0 && (float) emptyCells / totalCells > 0.8f) {
score -= 0.3f;
}
return Math.max(0f, Math.min(1f, score));
}
private Bounds tableBounds(Table table) {
return new Bounds(
(float) table.getX(),
(float) table.getY(),
(float) table.getWidth(),
(float) table.getHeight());
}
private String normaliseText(String raw) {
if (raw == null) return "";
// Tabula wraps multi-line cell content with \r\n — collapse to a single space.
return raw.replace("\r\n", " ").replace("\n", " ").replace("\r", " ").trim();
}
}
@@ -0,0 +1,113 @@
package stirling.software.SPDF.pdf.parser;
import static stirling.software.SPDF.pdf.parser.PdfModels.*;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import org.apache.pdfbox.pdmodel.PDPage;
import org.apache.pdfbox.pdmodel.font.PDFont;
import org.apache.pdfbox.text.PDFTextStripper;
import org.apache.pdfbox.text.TextPosition;
/**
* Extends {@link PDFTextStripper} to capture per-fragment geometry and font metadata.
*
* <p>Overrides {@link #writeString} to split each content-stream string into word-level {@link
* TextFragment}s with bounding boxes, baseline, font name, and bold flag. Coordinates are in
* PDFTextStripper space: (0,0) top-left, Y increases downward, {@code getY()} is the baseline.
*/
class WordExtractingStripper extends PDFTextStripper {
private final int targetPage;
private final List<TextFragment> fragments = new ArrayList<>();
private int fragmentIndex = 0;
WordExtractingStripper(int pageNumber) throws IOException {
this.targetPage = pageNumber;
setStartPage(pageNumber);
setEndPage(pageNumber);
setSortByPosition(true);
}
@Override
protected void startPage(PDPage page) throws IOException {
super.startPage(page);
fragments.clear();
fragmentIndex = 0;
}
@Override
protected void writeString(String text, List<TextPosition> textPositions) throws IOException {
if (text == null || text.isBlank()) return;
// Fast path: no whitespace → emit one fragment (most financial PDFs have each
// number as its own string operation, so this is the common case).
if (text.indexOf(' ') < 0) {
emitFragment(text, textPositions);
return;
}
// Per-word splitting requires 1:1 text-char to TextPosition correspondence.
// Fall back to one fragment when sizes differ (ligatures, encoding edge cases).
if (textPositions.size() != text.length()) {
emitFragment(text, textPositions);
return;
}
// Emit one TextFragment per whitespace-delimited word with accurate per-word bounds.
int start = 0;
for (int i = 0; i <= text.length(); i++) {
if (i == text.length() || text.charAt(i) == ' ') {
if (start < i) {
emitFragment(text.substring(start, i), textPositions.subList(start, i));
}
start = i + 1;
}
}
}
private void emitFragment(String text, List<TextPosition> positions) {
if (positions.isEmpty()) return;
float minX = Float.MAX_VALUE;
float minY = Float.MAX_VALUE;
float maxRight = -Float.MAX_VALUE;
float maxBaseline = -Float.MAX_VALUE;
TextPosition first = null;
for (TextPosition tp : positions) {
if (tp == null) continue;
if (first == null) first = tp;
float x = tp.getX();
// getY() is the baseline; top of character = getY() - getHeight().
float top = tp.getY() - tp.getHeight();
float right = x + tp.getWidth();
float baseline = tp.getY();
minX = Math.min(minX, x);
minY = Math.min(minY, top);
maxRight = Math.max(maxRight, right);
maxBaseline = Math.max(maxBaseline, baseline);
}
if (first == null) return;
PDFont font = first.getFont();
String fontName = font != null ? font.getName() : "";
boolean bold = fontName != null && fontName.toLowerCase().contains("bold");
// getHeight() gives the rendered glyph height, which is the most reliable visual size.
float fontSize = first.getHeight();
Bounds bounds = new Bounds(minX, minY, maxRight - minX, maxBaseline - minY);
String id = "tf-p" + targetPage + "-" + fragmentIndex++;
fragments.add(new TextFragment(id, text, bounds, maxBaseline, fontSize, fontName, bold));
}
List<TextFragment> getFragments() {
return Collections.unmodifiableList(fragments);
}
}
@@ -75,8 +75,8 @@ public @interface AutoJobPostMapping {
boolean queueable() default false;
/**
* Relative resource weight (1–100) used by the scheduler to prioritise / throttle jobs. Values
* below 1 are clamped to&nbsp;1, values above 100 to&nbsp;100.
* Relative resource weight (1-100). See {@link
* stirling.software.common.enumeration.ResourceWeight} for the standard tiers.
*/
int resourceWeight() default 50;
int resourceWeight() default 1;
}
@@ -0,0 +1,22 @@
package stirling.software.common.enumeration;
/**
* Standard resource-weight tiers for {@link
* stirling.software.common.annotations.AutoJobPostMapping#resourceWeight()}.
*/
public final class ResourceWeight {
/** Lightweight: rotate, page numbers, extract pages, permissions, etc. */
public static final int SMALL_WEIGHT = 1;
/** Medium: merge, split, multi-tool batch. */
public static final int MEDIUM_WEIGHT = 3;
/** Heavy: compress, OCR (small), conversions, raster. */
public static final int LARGE_WEIGHT = 5;
/** Extra heavy: OCR on large files, full-document re-render, AI-assisted edits. */
public static final int XLARGE_WEIGHT = 10;
private ResourceWeight() {}
}
@@ -97,7 +97,14 @@ public class ApplicationProperties {
EncodedResource encodedResource = new EncodedResource(resource);
PropertySource<?> propertySource =
new YamlPropertySourceFactory().createPropertySource(null, encodedResource);
environment.getPropertySources().addFirst(propertySource);
boolean saasActive = Arrays.asList(environment.getActiveProfiles()).contains("saas");
if (saasActive) {
// Saas-pinned values in application-saas.properties must beat settings.yml.
environment.getPropertySources().addLast(propertySource);
} else {
environment.getPropertySources().addFirst(propertySource);
}
log.debug("Loaded properties: {}", propertySource.getSource());
@@ -14,7 +14,6 @@ import lombok.NoArgsConstructor;
@Data
@NoArgsConstructor
@EqualsAndHashCode
@Schema(description = "PDF file input - either upload a file or provide a server-side file ID")
public class PDFFile {
@Schema(
@@ -0,0 +1,26 @@
package stirling.software.common.model.enumeration;
import lombok.Getter;
import lombok.RequiredArgsConstructor;
/** Team invitation status */
@Getter
@RequiredArgsConstructor
public enum InvitationStatus {
PENDING("PENDING"),
ACCEPTED("ACCEPTED"),
REJECTED("REJECTED"),
CANCELLED("CANCELLED"),
EXPIRED("EXPIRED");
private final String statusName;
public static InvitationStatus fromString(String statusName) {
for (InvitationStatus status : InvitationStatus.values()) {
if (status.getStatusName().equalsIgnoreCase(statusName)) {
return status;
}
}
throw new IllegalArgumentException("No InvitationStatus defined for name: " + statusName);
}
}
@@ -16,6 +16,9 @@ public enum Role {
// Unlimited access
USER("ROLE_USER", Integer.MAX_VALUE, Integer.MAX_VALUE, "adminUserSettings.user"),
// Paid tier; set by Stripe webhooks under saas mode.
PRO_USER("ROLE_PRO_USER", Integer.MAX_VALUE, Integer.MAX_VALUE, "adminUserSettings.proUser"),
// 40 API calls Per Day, 40 web calls
LIMITED_API_USER("ROLE_LIMITED_API_USER", 40, 40, "adminUserSettings.apiUser"),
@@ -0,0 +1,26 @@
package stirling.software.common.model.enumeration;
import lombok.Getter;
import lombok.RequiredArgsConstructor;
/**
* Team membership roles LEADER: Can invite/remove members, manage settings, view usage, manage
* billing MEMBER: Regular team member with standard access
*/
@Getter
@RequiredArgsConstructor
public enum TeamRole {
LEADER("LEADER"),
MEMBER("MEMBER");
private final String roleName;
public static TeamRole fromString(String roleName) {
for (TeamRole role : TeamRole.values()) {
if (role.getRoleName().equalsIgnoreCase(roleName)) {
return role;
}
}
throw new IllegalArgumentException("No TeamRole defined for name: " + roleName);
}
}
@@ -2289,6 +2289,85 @@ public class FormUtils {
acroForm.getFields().add(field);
}
/** Drops AcroForm fields whose widgets are no longer on any page of {@code document}. */
public void pruneOrphanedFormFields(PDDocument document) {
if (document == null) {
return;
}
PDDocumentCatalog catalog = document.getDocumentCatalog();
if (catalog == null) {
return;
}
PDAcroForm form = catalog.getAcroForm(null);
if (form == null) {
return;
}
List<PDField> fields = form.getFields();
if (fields.isEmpty()) {
return;
}
Set<COSDictionary> liveWidgets = collectLiveWidgetDictionaries(document);
List<PDField> kept = pruneFieldList(fields, liveWidgets);
if (kept.isEmpty()) {
catalog.setAcroForm(null);
} else if (kept.size() != fields.size()) {
form.setFields(kept);
}
}
private Set<COSDictionary> collectLiveWidgetDictionaries(PDDocument document) {
Set<COSDictionary> live = new HashSet<>();
int pageCount = document.getNumberOfPages();
for (int i = 0; i < pageCount; i++) {
try {
for (PDAnnotation annotation : document.getPage(i).getAnnotations()) {
if (annotation instanceof PDAnnotationWidget) {
live.add(annotation.getCOSObject());
}
}
} catch (IOException e) {
log.debug("Failed reading page {} annotations: {}", i, e.getMessage());
}
}
return live;
}
private List<PDField> pruneFieldList(List<PDField> fields, Set<COSDictionary> liveWidgets) {
List<PDField> kept = new ArrayList<>(fields.size());
for (PDField field : fields) {
if (field instanceof PDNonTerminalField nonTerminal) {
List<PDField> children = nonTerminal.getChildren();
List<PDField> remaining = pruneFieldList(children, liveWidgets);
if (remaining.isEmpty()) {
continue;
}
if (remaining.size() != children.size()) {
nonTerminal.setChildren(remaining);
}
kept.add(nonTerminal);
} else if (field instanceof PDTerminalField terminal) {
List<PDAnnotationWidget> widgets = terminal.getWidgets();
List<PDAnnotationWidget> liveOnes = new ArrayList<>(widgets.size());
for (PDAnnotationWidget widget : widgets) {
if (liveWidgets.contains(widget.getCOSObject())) {
liveOnes.add(widget);
}
}
if (liveOnes.isEmpty()) {
continue;
}
if (liveOnes.size() != widgets.size()) {
terminal.setWidgets(liveOnes);
}
kept.add(terminal);
} else {
kept.add(field);
}
}
return kept;
}
// Delegation methods to GeneralFormCopyUtils for form field transformation
public boolean hasAnyRotatedPage(PDDocument document) {
return stirling.software.common.util.GeneralFormCopyUtils.hasAnyRotatedPage(document);
@@ -172,7 +172,7 @@ public class RequestUriUtils {
"/api/v1/ui-data/footer-info") // Public footer configuration
|| trimmedUri.startsWith("/api/v1/invite/validate")
|| trimmedUri.startsWith("/api/v1/invite/accept")
// Health Endoints
// Health Endpoints
|| trimmedUri.startsWith("/actuator/health")
|| trimmedUri.startsWith("/health")
|| trimmedUri.startsWith("/healthz")
@@ -0,0 +1,153 @@
package stirling.software.SPDF.pdf.parser;
import static org.assertj.core.api.Assertions.assertThat;
import static stirling.software.SPDF.pdf.parser.PdfModels.*;
import java.util.List;
import org.junit.jupiter.api.Test;
/**
* Unit tests for {@link LineAlignmentTableParser}, focused on the coincident-line merge logic and
* column-grid construction.
*/
class LineAlignmentTableParserTest {
private final LineAlignmentTableParser parser = new LineAlignmentTableParser();
// ── mergeCoincidentLines ─────────────────────────────────────────────────────────────────────
@Test
void mergeCoincidentLines_singleLine_unchanged() {
var lines = List.of(tokenized(rawLine(10f, 100f, "Revenue")));
assertThat(parser.mergeCoincidentLines(lines)).hasSize(1);
}
@Test
void mergeCoincidentLines_distinctYLines_unchanged() {
// Two lines at different y positions — must NOT be merged.
var lines =
List.of(
tokenized(rawLine(10f, 100f, "Revenue")),
tokenized(rawLine(10f, 115f, "Cost")));
assertThat(parser.mergeCoincidentLines(lines)).hasSize(2);
}
@Test
void mergeCoincidentLines_sameY_merged() {
// Simulates a financial-table row split by LineBuilder at the column gap:
// label fragment at x=72 → "Revenue"
// value fragment at x=350 → "1,234"
// Both have y=100. After merge they should form one TokenizedLine.
var label = rawLine(72f, 100f, "Revenue");
var value = rawLine(350f, 100f, "1,234");
var merged = parser.mergeCoincidentLines(List.of(tokenized(label), tokenized(value)));
assertThat(merged).hasSize(1);
// The merged line should contain tokens from both halves.
var tokens = merged.get(0).all();
assertThat(tokens.stream().map(t -> t.text()).toList())
.containsExactlyInAnyOrder("Revenue", "1,234");
}
@Test
void mergeCoincidentLines_sameY_mergedLineHasCorrectBounds() {
var label = rawLine(72f, 100f, "Revenue"); // 7 chars × 6pt = 42pt wide → right = 114
var value = rawLine(350f, 100f, "1,234"); // 5 chars × 6pt = 30pt wide → right = 380
var merged = parser.mergeCoincidentLines(List.of(tokenized(label), tokenized(value)));
var bounds = merged.get(0).line().bounds();
assertThat(bounds.x()).isEqualTo(72f);
assertThat(bounds.right()).isEqualTo(380f);
}
@Test
void mergeCoincidentLines_withinTolerance_merged() {
// Lines 1.5pt apart (within ROW_MERGE_TOLERANCE_PT = 2pt) should merge.
var a = rawLine(10f, 100.0f, "Alpha");
var b = rawLine(200f, 101.5f, "99");
var merged = parser.mergeCoincidentLines(List.of(tokenized(a), tokenized(b)));
assertThat(merged).hasSize(1);
}
@Test
void mergeCoincidentLines_beyondTolerance_notMerged() {
// Lines 3pt apart (beyond ROW_MERGE_TOLERANCE_PT = 2pt) should NOT merge.
var a = rawLine(10f, 100.0f, "Alpha");
var b = rawLine(200f, 103.0f, "99");
var merged = parser.mergeCoincidentLines(List.of(tokenized(a), tokenized(b)));
assertThat(merged).hasSize(2);
}
@Test
void mergeCoincidentLines_threeCoincident_allMerged() {
// Three fragments at the same y (e.g. wide financial table with two value columns).
var a = rawLine(72f, 100f, "Revenue");
var b = rawLine(300f, 100f, "1,234");
var c = rawLine(400f, 100f, "5,678");
var merged = parser.mergeCoincidentLines(List.of(tokenized(a), tokenized(b), tokenized(c)));
assertThat(merged).hasSize(1);
assertThat(merged.get(0).all()).hasSize(3);
}
@Test
void mergeCoincidentLines_coincidentPairFollowedByDistinctLine_twoGroups() {
var a = rawLine(72f, 100f, "Revenue");
var b = rawLine(350f, 100f, "1,234"); // same y as a → merges with a
var c = rawLine(10f, 115f, "Expenses"); // different y → stays separate
var merged = parser.mergeCoincidentLines(List.of(tokenized(a), tokenized(b), tokenized(c)));
assertThat(merged).hasSize(2);
}
@Test
void mergeCoincidentLines_numericAnchorStatus_correctAfterMerge() {
// After merging, the combined line should be an anchor (≥2 numeric tokens).
// "Revenue" alone → not an anchor. "1,234 567" alone → anchor.
// Merged → anchor with at least 2 numerics.
var label = rawLine(72f, 100f, "Revenue");
var values = rawLineMultiWord(350f, 100f, "1,234", 30f, "567", 30f);
var merged = parser.mergeCoincidentLines(List.of(tokenized(label), tokenized(values)));
assertThat(merged).hasSize(1);
assertThat(merged.get(0).isAnchor()).isTrue();
}
// ── helpers ──────────────────────────────────────────────────────────────────────────────────
/** Creates a RawLine with a single TextFragment of the given text at the given position. */
private static RawLine rawLine(float x, float y, String text) {
float width = text.length() * 6f; // ~6pt per char — rough but consistent
float height = 12f;
Bounds bounds = new Bounds(x, y, width, height);
TextFragment fragment =
new TextFragment("tf-test", text, bounds, y + height, 11f, "Helvetica", false);
return new RawLine("ln-test", List.of(fragment), bounds, 1);
}
/**
* Creates a RawLine with two TextFragments representing two words separated by a small gap.
* Used to simulate a values-only line with multiple numeric tokens.
*/
private static RawLine rawLineMultiWord(
float x, float y, String word1, float w1, String word2, float w2) {
float height = 12f;
Bounds b1 = new Bounds(x, y, w1, height);
Bounds b2 = new Bounds(x + w1 + 5f, y, w2, height);
TextFragment f1 = new TextFragment("tf-1", word1, b1, y + height, 11f, "Helvetica", false);
TextFragment f2 = new TextFragment("tf-2", word2, b2, y + height, 11f, "Helvetica", false);
Bounds lineBounds = new Bounds(x, y, x + w1 + 5f + w2 - x, height);
return new RawLine("ln-test", List.of(f1, f2), lineBounds, 1);
}
/** Tokenises a RawLine via the parser's own tokenise logic (package-private access). */
private LineAlignmentTableParser.TokenizedLine tokenized(RawLine line) {
return parser.tokenize(line);
}
}
@@ -0,0 +1,59 @@
package stirling.software.common.architecture;
import static com.tngtech.archunit.lang.syntax.ArchRuleDefinition.noClasses;
import org.junit.jupiter.api.Test;
import com.tngtech.archunit.core.domain.JavaClasses;
import com.tngtech.archunit.core.importer.ClassFileImporter;
import com.tngtech.archunit.core.importer.ImportOption;
import com.tngtech.archunit.lang.ArchRule;
/**
* Module dependency-direction guardrails. Allowed direction: {@code saas → proprietary → common}
* and {@code stirling-pdf → proprietary → common}.
*/
class ArchitectureTest {
private static final JavaClasses commonClasses =
new ClassFileImporter()
.withImportOption(ImportOption.Predefined.DO_NOT_INCLUDE_JARS)
.withImportOption(ImportOption.Predefined.DO_NOT_INCLUDE_TESTS)
.importPackages("stirling.software.common");
@Test
void commonDoesNotDependOnCore() {
ArchRule rule =
noClasses()
.that()
.resideInAPackage("stirling.software.common..")
.should()
.dependOnClassesThat()
.resideInAPackage("stirling.software.SPDF..");
rule.check(commonClasses);
}
@Test
void commonDoesNotDependOnProprietary() {
ArchRule rule =
noClasses()
.that()
.resideInAPackage("stirling.software.common..")
.should()
.dependOnClassesThat()
.resideInAPackage("stirling.software.proprietary..");
rule.check(commonClasses);
}
@Test
void commonDoesNotDependOnSaas() {
ArchRule rule =
noClasses()
.that()
.resideInAPackage("stirling.software.common..")
.should()
.dependOnClassesThat()
.resideInAPackage("stirling.software.saas..");
rule.check(commonClasses);
}
}
@@ -0,0 +1,32 @@
package stirling.software.common.jpdfium;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertTrue;
import java.io.IOException;
import java.io.InputStream;
import java.nio.file.Files;
import java.nio.file.Path;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.io.TempDir;
import stirling.software.jpdfium.PdfDocument;
class JPDFiumSmokeTest {
@Test
void opensExamplePdfAndReadsPageCount(@TempDir Path tmp) throws IOException {
Path pdf = tmp.resolve("example.pdf");
try (InputStream in = getClass().getResourceAsStream("/example.pdf")) {
assertNotNull(in, "example.pdf must exist under src/test/resources");
Files.copy(in, pdf);
}
try (PdfDocument doc = PdfDocument.open(pdf)) {
assertTrue(
doc.pageCount() >= 1,
"PdfDocument should report at least one page for example.pdf");
}
}
}
@@ -0,0 +1,229 @@
package stirling.software.common.util;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertNull;
import static org.junit.jupiter.api.Assertions.assertTrue;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import org.apache.pdfbox.Loader;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.apache.pdfbox.pdmodel.PDPage;
import org.apache.pdfbox.pdmodel.PDResources;
import org.apache.pdfbox.pdmodel.common.PDRectangle;
import org.apache.pdfbox.pdmodel.interactive.annotation.PDAnnotationWidget;
import org.apache.pdfbox.pdmodel.interactive.form.PDAcroForm;
import org.apache.pdfbox.pdmodel.interactive.form.PDField;
import org.apache.pdfbox.pdmodel.interactive.form.PDNonTerminalField;
import org.apache.pdfbox.pdmodel.interactive.form.PDTextField;
import org.junit.jupiter.api.Test;
class FormUtilsPruneOrphanedFieldsTest {
@Test
void noAcroFormIsNoOp() throws IOException {
try (PDDocument document = new PDDocument()) {
document.addPage(new PDPage(PDRectangle.A4));
FormUtils.pruneOrphanedFormFields(document);
assertNull(document.getDocumentCatalog().getAcroForm(null));
}
}
@Test
void dropsFieldsWhoseWidgetsAreAllOnRemovedPages() throws IOException {
byte[] pdfBytes = buildPdfWithFieldPerPage(3);
try (PDDocument document = Loader.loadPDF(pdfBytes)) {
document.removePage(2);
document.removePage(0);
FormUtils.pruneOrphanedFormFields(document);
PDAcroForm form = document.getDocumentCatalog().getAcroForm(null);
assertNotNull(form);
List<String> remainingNames = new ArrayList<>();
for (PDField field : form.getFields()) {
remainingNames.add(field.getPartialName());
}
assertEquals(List.of("field_1"), remainingNames);
}
}
@Test
void dropsAcroFormEntirelyWhenNoFieldsSurvive() throws IOException {
byte[] pdfBytes = buildPdfWithFieldPerPage(2);
try (PDDocument document = Loader.loadPDF(pdfBytes)) {
document.removePage(1);
document.removePage(0);
document.addPage(new PDPage(PDRectangle.A4));
FormUtils.pruneOrphanedFormFields(document);
assertNull(document.getDocumentCatalog().getAcroForm(null));
}
}
@Test
void keepsLiveWidgetsAndDropsOrphanWidgetsFromMultiWidgetField() throws IOException {
byte[] pdfBytes = buildPdfWithMultiWidgetField();
try (PDDocument document = Loader.loadPDF(pdfBytes)) {
document.removePage(0);
FormUtils.pruneOrphanedFormFields(document);
PDAcroForm form = document.getDocumentCatalog().getAcroForm(null);
assertNotNull(form);
assertEquals(1, form.getFields().size());
PDField field = form.getFields().get(0);
assertEquals("multi", field.getPartialName());
assertEquals(2, field.getWidgets().size(), "two widgets remain after one is dropped");
}
}
@Test
void survivesRoundTripWithoutOrphanPagesInOutput() throws IOException {
byte[] pdfBytes = buildPdfWithFieldPerPage(3);
byte[] writtenBytes;
try (PDDocument document = Loader.loadPDF(pdfBytes)) {
document.removePage(2);
document.removePage(1);
FormUtils.pruneOrphanedFormFields(document);
try (ByteArrayOutputStream out = new ByteArrayOutputStream()) {
document.save(out);
writtenBytes = out.toByteArray();
}
}
try (PDDocument reloaded = Loader.loadPDF(writtenBytes)) {
assertEquals(1, reloaded.getNumberOfPages());
PDAcroForm form = reloaded.getDocumentCatalog().getAcroForm(null);
assertNotNull(form);
assertEquals(1, form.getFields().size());
assertEquals("field_0", form.getFields().get(0).getPartialName());
}
}
@Test
void prunesNestedNonTerminalFields() throws IOException {
byte[] pdfBytes = buildPdfWithNestedFields();
try (PDDocument document = Loader.loadPDF(pdfBytes)) {
document.removePage(1);
FormUtils.pruneOrphanedFormFields(document);
PDAcroForm form = document.getDocumentCatalog().getAcroForm(null);
assertNotNull(form);
assertEquals(1, form.getFields().size());
PDField group = form.getFields().get(0);
assertEquals("group", group.getPartialName());
assertTrue(group instanceof PDNonTerminalField);
PDNonTerminalField nonTerminal = (PDNonTerminalField) group;
assertEquals(1, nonTerminal.getChildren().size());
assertEquals("kept", nonTerminal.getChildren().get(0).getPartialName());
}
}
private static byte[] buildPdfWithFieldPerPage(int pageCount) throws IOException {
try (PDDocument document = new PDDocument()) {
PDAcroForm acroForm = new PDAcroForm(document);
acroForm.setDefaultResources(new PDResources());
document.getDocumentCatalog().setAcroForm(acroForm);
for (int i = 0; i < pageCount; i++) {
PDPage page = new PDPage(PDRectangle.A4);
document.addPage(page);
PDTextField field = new PDTextField(acroForm);
field.setPartialName("field_" + i);
PDAnnotationWidget widget = new PDAnnotationWidget();
widget.setRectangle(new PDRectangle(50, 50, 100, 20));
widget.setPage(page);
field.setWidgets(List.of(widget));
acroForm.getFields().add(field);
page.getAnnotations().add(widget);
}
try (ByteArrayOutputStream out = new ByteArrayOutputStream()) {
document.save(out);
return out.toByteArray();
}
}
}
private static byte[] buildPdfWithMultiWidgetField() throws IOException {
try (PDDocument document = new PDDocument()) {
PDAcroForm acroForm = new PDAcroForm(document);
acroForm.setDefaultResources(new PDResources());
document.getDocumentCatalog().setAcroForm(acroForm);
List<PDAnnotationWidget> widgets = new ArrayList<>();
for (int i = 0; i < 3; i++) {
PDPage page = new PDPage(PDRectangle.A4);
document.addPage(page);
PDAnnotationWidget widget = new PDAnnotationWidget();
widget.setRectangle(new PDRectangle(50, 50, 100, 20));
widget.setPage(page);
page.getAnnotations().add(widget);
widgets.add(widget);
}
PDTextField field = new PDTextField(acroForm);
field.setPartialName("multi");
field.setWidgets(widgets);
acroForm.getFields().add(field);
try (ByteArrayOutputStream out = new ByteArrayOutputStream()) {
document.save(out);
return out.toByteArray();
}
}
}
private static byte[] buildPdfWithNestedFields() throws IOException {
try (PDDocument document = new PDDocument()) {
PDAcroForm acroForm = new PDAcroForm(document);
acroForm.setDefaultResources(new PDResources());
document.getDocumentCatalog().setAcroForm(acroForm);
PDPage pageA = new PDPage(PDRectangle.A4);
PDPage pageB = new PDPage(PDRectangle.A4);
document.addPage(pageA);
document.addPage(pageB);
PDNonTerminalField group = new PDNonTerminalField(acroForm);
group.setPartialName("group");
PDTextField kept = new PDTextField(acroForm);
kept.setPartialName("kept");
PDAnnotationWidget keptWidget = new PDAnnotationWidget();
keptWidget.setRectangle(new PDRectangle(50, 50, 100, 20));
keptWidget.setPage(pageA);
kept.setWidgets(List.of(keptWidget));
pageA.getAnnotations().add(keptWidget);
PDTextField dropped = new PDTextField(acroForm);
dropped.setPartialName("dropped");
PDAnnotationWidget droppedWidget = new PDAnnotationWidget();
droppedWidget.setRectangle(new PDRectangle(50, 100, 100, 20));
droppedWidget.setPage(pageB);
dropped.setWidgets(List.of(droppedWidget));
pageB.getAnnotations().add(droppedWidget);
group.setChildren(List.of(kept, dropped));
acroForm.getFields().add(group);
try (ByteArrayOutputStream out = new ByteArrayOutputStream()) {
document.save(out);
return out.toByteArray();
}
}
}
}
+50 -22
View File
@@ -39,12 +39,14 @@ spotless {
}
dependencies {
if (System.getenv('DISABLE_ADDITIONAL_FEATURES') != 'true'
|| (project.hasProperty('DISABLE_ADDITIONAL_FEATURES')
&& System.getProperty('DISABLE_ADDITIONAL_FEATURES') != 'true')) {
if (!gradle.ext.disableAdditional) {
implementation project(':proprietary')
}
if (gradle.ext.enableSaas) {
implementation project(':saas')
}
implementation project(':common')
implementation 'org.springframework.boot:spring-boot-starter-jetty'
implementation 'org.eclipse.jetty.http2:jetty-http2-server'
@@ -87,12 +89,6 @@ dependencies {
implementation 'com.sun.xml.bind:jaxb-core:4.0.7'
implementation 'org.apache.poi:poi-ooxml:5.5.1'
// https://mvnrepository.com/artifact/technology.tabula/tabula
implementation ('technology.tabula:tabula:1.0.5') {
exclude group: 'org.slf4j', module: 'slf4j-simple'
exclude group: 'org.bouncycastle', module: 'bcprov-jdk15on'
exclude group: 'com.google.code.gson', module: 'gson'
}
// CVE-2022-25647: Explicit gson 2.13.2 to prevent unsafe deserialization (tabula would pull 2.8.7)
implementation 'com.google.code.gson:gson:2.13.2'
implementation 'org.apache.pdfbox:jbig2-imageio:3.0.4'
@@ -166,7 +162,8 @@ bootJar {
manifest {
attributes(
'Implementation-Title': 'Stirling-PDF',
'Implementation-Version': project.version
'Implementation-Version': project.version,
'Enable-Native-Access': 'ALL-UNNAMED'
)
}
}
@@ -179,8 +176,33 @@ springBoot {
// Frontend build tasks - only enabled with -PbuildWithFrontend=true
def buildWithFrontend = project.hasProperty('buildWithFrontend') && project.property('buildWithFrontend') == 'true'
def buildPrototypes = project.hasProperty('prototypesMode') && project.property('prototypesMode') == 'true'
// Workspace root holds package.json and node_modules (shared across editor /
// future portal). Editor-specific paths (src, public, dist, tauri) live one
// level deeper under frontend/editor/.
// Vite mode: -PprototypesMode > -PfrontendMode > enableSaas > disableAdditional > proprietary.
def frontendModeOverride = project.findProperty('frontendMode')?.toString()?.toLowerCase()
def frontendMode
if (buildPrototypes) {
frontendMode = 'prototypes'
} else if (frontendModeOverride) {
frontendMode = frontendModeOverride
} else if (gradle.ext.enableSaas) {
frontendMode = 'saas'
} else if (gradle.ext.disableAdditional) {
frontendMode = 'core'
} else {
frontendMode = 'proprietary'
}
def frontendBuildTask = "frontend:build:${frontendMode}"
// Workspace root holds package.json and node_modules (shared across editor /
// future portal). Editor-specific paths (src, public, dist, tauri) live one
// level deeper under frontend/editor/. When the portal lands as an embedded
// app, add a sibling frontendPortalDir / frontendPortalDistDir alongside.
def frontendDir = file('../../frontend')
def frontendDistDir = file('../../frontend/dist')
def frontendEditorDir = file('../../frontend/editor')
def frontendEditorDistDir = file('../../frontend/editor/dist')
def resourcesStaticDir = file('src/main/resources/static')
def generatedFrontendPaths = [
'assets',
@@ -245,13 +267,13 @@ tasks.register('npmBuild', Exec) {
doNotTrackState("Frontend build depends on untracked npmInstall task")
enabled = buildWithFrontend
group = 'frontend'
description = 'Build frontend application'
description = 'Build editor frontend application'
workingDir file('../..')
commandLine = buildPrototypes ? ['task', 'frontend:build:prototypes'] : ['task', 'frontend:build']
inputs.dir(new File(frontendDir, 'src'))
inputs.dir(new File(frontendDir, 'public'))
commandLine = ['task', frontendBuildTask]
inputs.dir(new File(frontendEditorDir, 'src'))
inputs.dir(new File(frontendEditorDir, 'public'))
inputs.file(new File(frontendDir, 'package.json'))
outputs.dir(frontendDistDir)
outputs.dir(frontendEditorDistDir)
// Show live output
standardOutput = System.out
@@ -262,16 +284,17 @@ tasks.register('npmBuild', Exec) {
environment 'VITE_API_BASE_URL', '/'
doFirst {
println "Building frontend application for production (VITE_API_BASE_URL=/)"
println "Building editor frontend application for production (mode=${frontendMode}, VITE_API_BASE_URL=/)"
}
}
tasks.register('copyFrontendAssets', Copy) {
enabled = buildWithFrontend
group = 'frontend'
description = 'Copy frontend build to static resources'
description = 'Copy editor frontend build to static resources'
dependsOn npmBuild
from(frontendDistDir) {
dependsOn cleanFrontendAssets
from(frontendEditorDistDir) {
// Exclude files that conflict with backend static resources
exclude 'robots.txt' // Backend already has this
exclude 'favicon.ico' // Backend already has this
@@ -279,7 +302,7 @@ tasks.register('copyFrontendAssets', Copy) {
into resourcesStaticDir
duplicatesStrategy = DuplicatesStrategy.INCLUDE // Let frontend overwrite when needed
doFirst {
println "Copying frontend build from ${frontendDistDir} to ${resourcesStaticDir}..."
println "Copying frontend build from ${frontendEditorDistDir} to ${resourcesStaticDir}..."
println "Backend static resources will be preserved"
}
doLast {
@@ -311,7 +334,7 @@ tasks.named('copyFrontendAssets').configure {
}
if (buildWithFrontend) {
println "Frontend build enabled - JAR will include React frontend"
println "Editor frontend build enabled - JAR will include React frontend (mode=${frontendMode})"
processResources.dependsOn copyFrontendAssets
} else {
println "Frontend build disabled - JAR will be backend-only with API landing page"
@@ -320,4 +343,9 @@ if (buildWithFrontend) {
}
bootJar.dependsOn ':common:jar'
bootJar.dependsOn ':proprietary:jar'
if (!gradle.ext.disableAdditional) {
bootJar.dependsOn ':proprietary:jar'
}
if (gradle.ext.enableSaas) {
bootJar.dependsOn ':saas:jar'
}
@@ -35,7 +35,8 @@ import stirling.software.common.model.ApplicationProperties;
scanBasePackages = {
"stirling.software.SPDF",
"stirling.software.common",
"stirling.software.proprietary"
"stirling.software.proprietary",
"stirling.software.saas"
})
public class SPDFApplication {
@@ -205,15 +206,22 @@ public class SPDFApplication {
}
}
// 2. Detect if SecurityConfiguration is present on classpath
if (isClassPresent(
"stirling.software.proprietary.security.configuration.SecurityConfiguration")) {
// 2. Detect classpath shape and pick the matching profile chain.
boolean hasSaas = isClassPresent("stirling.software.saas.security.SupabaseSecurityConfig");
boolean hasSecurity =
isClassPresent(
"stirling.software.proprietary.security.configuration.SecurityConfiguration");
if (hasSaas) {
log.info("SaaS features in jar");
return new String[] {"security", "saas"};
}
if (hasSecurity) {
log.info("Additional features in jar");
return new String[] {"security"};
} else {
log.info("Without additional features in jar");
return new String[] {"default"};
}
log.info("Without additional features in jar");
return new String[] {"default"};
}
private static boolean isClassPresent(String className) {
@@ -23,6 +23,7 @@ import lombok.RequiredArgsConstructor;
import stirling.software.SPDF.config.swagger.JsonDataResponse;
import stirling.software.common.annotations.AutoJobPostMapping;
import stirling.software.common.annotations.api.AnalysisApi;
import stirling.software.common.enumeration.ResourceWeight;
import stirling.software.common.model.api.PDFFile;
import stirling.software.common.service.CustomPDFDocumentFactory;
@@ -32,7 +33,10 @@ public class AnalysisController {
private final CustomPDFDocumentFactory pdfDocumentFactory;
@AutoJobPostMapping(value = "/page-count", consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
@AutoJobPostMapping(
value = "/page-count",
consumes = MediaType.MULTIPART_FORM_DATA_VALUE,
resourceWeight = ResourceWeight.SMALL_WEIGHT)
@JsonDataResponse
@Operation(
summary = "Get PDF page count",
@@ -43,7 +47,10 @@ public class AnalysisController {
}
}
@AutoJobPostMapping(value = "/basic-info", consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
@AutoJobPostMapping(
value = "/basic-info",
consumes = MediaType.MULTIPART_FORM_DATA_VALUE,
resourceWeight = ResourceWeight.SMALL_WEIGHT)
@JsonDataResponse
@Operation(
summary = "Get basic PDF information",
@@ -60,7 +67,8 @@ public class AnalysisController {
@AutoJobPostMapping(
value = "/document-properties",
consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
consumes = MediaType.MULTIPART_FORM_DATA_VALUE,
resourceWeight = ResourceWeight.SMALL_WEIGHT)
@JsonDataResponse
@Operation(
summary = "Get PDF document properties",
@@ -90,7 +98,10 @@ public class AnalysisController {
}
}
@AutoJobPostMapping(value = "/page-dimensions", consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
@AutoJobPostMapping(
value = "/page-dimensions",
consumes = MediaType.MULTIPART_FORM_DATA_VALUE,
resourceWeight = ResourceWeight.SMALL_WEIGHT)
@JsonDataResponse
@Operation(
summary = "Get page dimensions for all pages",
@@ -110,7 +121,10 @@ public class AnalysisController {
}
}
@AutoJobPostMapping(value = "/form-fields", consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
@AutoJobPostMapping(
value = "/form-fields",
consumes = MediaType.MULTIPART_FORM_DATA_VALUE,
resourceWeight = ResourceWeight.SMALL_WEIGHT)
@JsonDataResponse
@Operation(
summary = "Get form field information",
@@ -134,7 +148,10 @@ public class AnalysisController {
}
}
@AutoJobPostMapping(value = "/annotation-info", consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
@AutoJobPostMapping(
value = "/annotation-info",
consumes = MediaType.MULTIPART_FORM_DATA_VALUE,
resourceWeight = ResourceWeight.SMALL_WEIGHT)
@JsonDataResponse
@Operation(
summary = "Get annotation information",
@@ -159,7 +176,10 @@ public class AnalysisController {
}
}
@AutoJobPostMapping(value = "/font-info", consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
@AutoJobPostMapping(
value = "/font-info",
consumes = MediaType.MULTIPART_FORM_DATA_VALUE,
resourceWeight = ResourceWeight.SMALL_WEIGHT)
@JsonDataResponse
@Operation(
summary = "Get font information",
@@ -185,7 +205,10 @@ public class AnalysisController {
}
}
@AutoJobPostMapping(value = "/security-info", consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
@AutoJobPostMapping(
value = "/security-info",
consumes = MediaType.MULTIPART_FORM_DATA_VALUE,
resourceWeight = ResourceWeight.SMALL_WEIGHT)
@JsonDataResponse
@Operation(
summary = "Get security information",
@@ -28,6 +28,7 @@ import lombok.RequiredArgsConstructor;
import stirling.software.SPDF.model.api.general.BookletImpositionRequest;
import stirling.software.common.annotations.AutoJobPostMapping;
import stirling.software.common.enumeration.ResourceWeight;
import stirling.software.common.service.CustomPDFDocumentFactory;
import stirling.software.common.util.GeneralUtils;
import stirling.software.common.util.TempFileManager;
@@ -44,7 +45,8 @@ public class BookletImpositionController {
@AutoJobPostMapping(
value = "/booklet-imposition",
consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
consumes = MediaType.MULTIPART_FORM_DATA_VALUE,
resourceWeight = ResourceWeight.MEDIUM_WEIGHT)
@Operation(
summary = "Create a booklet with proper page imposition",
description =
@@ -26,6 +26,7 @@ import stirling.software.SPDF.config.EndpointConfiguration;
import stirling.software.SPDF.model.api.general.CropPdfForm;
import stirling.software.common.annotations.AutoJobPostMapping;
import stirling.software.common.annotations.api.GeneralApi;
import stirling.software.common.enumeration.ResourceWeight;
import stirling.software.common.service.CustomPDFDocumentFactory;
import stirling.software.common.util.ExceptionUtils;
import stirling.software.common.util.GeneralUtils;
@@ -126,7 +127,10 @@ public class CropController {
return endpointConfiguration.isGroupEnabled("Ghostscript");
}
@AutoJobPostMapping(value = "/crop", consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
@AutoJobPostMapping(
value = "/crop",
consumes = MediaType.MULTIPART_FORM_DATA_VALUE,
resourceWeight = ResourceWeight.SMALL_WEIGHT)
@Operation(
summary = "Crops a PDF document",
description =
@@ -25,6 +25,7 @@ import lombok.extern.slf4j.Slf4j;
import stirling.software.SPDF.model.api.EditTableOfContentsRequest;
import stirling.software.common.annotations.AutoJobPostMapping;
import stirling.software.common.annotations.api.GeneralApi;
import stirling.software.common.enumeration.ResourceWeight;
import stirling.software.common.service.CustomPDFDocumentFactory;
import stirling.software.common.util.GeneralUtils;
import stirling.software.common.util.TempFileManager;
@@ -44,7 +45,8 @@ public class EditTableOfContentsController {
@AutoJobPostMapping(
value = "/extract-bookmarks",
consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
consumes = MediaType.MULTIPART_FORM_DATA_VALUE,
resourceWeight = ResourceWeight.SMALL_WEIGHT)
@Operation(
summary = "Extract PDF Bookmarks",
description = "Extracts bookmarks/table of contents from a PDF document as JSON.")
@@ -147,7 +149,8 @@ public class EditTableOfContentsController {
@AutoJobPostMapping(
value = "/edit-table-of-contents",
consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
consumes = MediaType.MULTIPART_FORM_DATA_VALUE,
resourceWeight = ResourceWeight.SMALL_WEIGHT)
@Operation(
summary = "Edit Table of Contents",
description = "Add or edit bookmarks/table of contents in a PDF document.")
@@ -3,13 +3,14 @@ package stirling.software.SPDF.controller.api;
import java.io.File;
import java.io.IOException;
import java.io.InputStream;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Comparator;
import java.util.List;
import java.util.regex.Pattern;
import org.apache.pdfbox.multipdf.PDFMergerUtility;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.apache.pdfbox.pdmodel.PDDocumentCatalog;
import org.apache.pdfbox.pdmodel.PDDocumentInformation;
@@ -39,6 +40,7 @@ import stirling.software.SPDF.config.swagger.StandardPdfResponse;
import stirling.software.SPDF.model.api.general.MergePdfsRequest;
import stirling.software.common.annotations.AutoJobPostMapping;
import stirling.software.common.annotations.api.GeneralApi;
import stirling.software.common.enumeration.ResourceWeight;
import stirling.software.common.service.CustomPDFDocumentFactory;
import stirling.software.common.util.ExceptionUtils;
import stirling.software.common.util.GeneralUtils;
@@ -46,6 +48,11 @@ import stirling.software.common.util.PdfErrorUtils;
import stirling.software.common.util.TempFile;
import stirling.software.common.util.TempFileManager;
import stirling.software.common.util.WebResponseUtils;
import stirling.software.jpdfium.PdfDocument;
import stirling.software.jpdfium.PdfMerge;
import stirling.software.jpdfium.doc.Bookmark;
import stirling.software.jpdfium.doc.PdfBookmarkEditor;
import stirling.software.jpdfium.doc.PdfBookmarkEditor.BookmarkTree;
@GeneralApi
@Slf4j
@@ -56,7 +63,6 @@ public class MergeController {
private final CustomPDFDocumentFactory pdfDocumentFactory;
private final TempFileManager tempFileManager;
// Merges a list of PDDocument objects into a single PDDocument
public PDDocument mergeDocuments(List<PDDocument> documents) throws IOException {
PDDocument mergedDoc = pdfDocumentFactory.createNewDocument();
boolean success = false;
@@ -75,11 +81,8 @@ public class MergeController {
}
}
// Re-order files to match the explicit order provided by the front-end.
// fileOrder is newline-delimited original filenames in the desired order.
private static MultipartFile[] reorderFilesByProvidedOrder(
MultipartFile[] files, String fileOrder) {
// Split by various line endings and trim each entry
String[] desired =
stirling.software.common.util.RegexPatternUtils.getInstance()
.getNewlineSplitPattern()
@@ -106,7 +109,6 @@ public class MergeController {
return ordered.toArray(new MultipartFile[0]);
}
// Returns a comparator for sorting MultipartFile arrays based on the given sort type
private Comparator<MultipartFile> getSortComparator(String sortType) {
return switch (sortType) {
case "byFileName" ->
@@ -154,18 +156,16 @@ public class MergeController {
return 0;
}
};
case "orderProvided" -> (file1, file2) -> 0; // Default is the order provided
default -> (file1, file2) -> 0; // Default is the order provided
case "orderProvided" -> (file1, file2) -> 0;
default -> (file1, file2) -> 0;
};
}
// Parse client file IDs from JSON string
private String[] parseClientFileIds(String clientFileIds) {
if (clientFileIds == null || clientFileIds.trim().isEmpty()) {
return new String[0];
}
try {
// Simple JSON array parsing - remove brackets and split by comma
String trimmed = clientFileIds.trim();
if (trimmed.startsWith("[") && trimmed.endsWith("]")) {
String inside = trimmed.substring(1, trimmed.length() - 1).trim();
@@ -185,39 +185,29 @@ public class MergeController {
return new String[0];
}
// Adds a table of contents to the merged document using filenames as chapter titles
private void addTableOfContents(PDDocument mergedDocument, MultipartFile[] files) {
// Create the document outline
PDDocumentOutline outline = new PDDocumentOutline();
mergedDocument.getDocumentCatalog().setDocumentOutline(outline);
int pageIndex = 0; // Current page index in the merged document
// Iterate through the original files
int pageIndex = 0;
for (MultipartFile file : files) {
// Get the filename without extension to use as bookmark title
String filename = file.getOriginalFilename();
String title = GeneralUtils.removeExtension(filename);
// Create an outline item for this file
PDOutlineItem item = new PDOutlineItem();
item.setTitle(title);
// Set the destination to the first page of this file in the merged document
if (pageIndex < mergedDocument.getNumberOfPages()) {
PDPage page = mergedDocument.getPage(pageIndex);
item.setDestination(page);
}
// Add the item to the outline
outline.addLast(item);
// Increment page index for the next file
try (PDDocument doc = pdfDocumentFactory.load(file)) {
pageIndex += doc.getNumberOfPages();
} catch (IOException e) {
ExceptionUtils.logException("document loading for TOC generation", e);
pageIndex++; // Increment by at least one if we can't determine page count
pageIndex++;
}
}
}
@@ -235,7 +225,6 @@ public class MergeController {
}
}
// Fallback to XMP metadata if Info dates are missing
PDMetadata metadata = doc.getDocumentCatalog().getMetadata();
if (metadata != null) {
try (InputStream is = metadata.createInputStream()) {
@@ -271,7 +260,10 @@ public class MergeController {
return -1;
}
@AutoJobPostMapping(consumes = MediaType.MULTIPART_FORM_DATA_VALUE, value = "/merge-pdfs")
@AutoJobPostMapping(
consumes = MediaType.MULTIPART_FORM_DATA_VALUE,
value = "/merge-pdfs",
resourceWeight = ResourceWeight.MEDIUM_WEIGHT)
@StandardPdfResponse
@Operation(
summary = "Merge multiple PDF files into one",
@@ -283,7 +275,7 @@ public class MergeController {
@ModelAttribute MergePdfsRequest request,
@RequestParam(value = "fileOrder", required = false) String fileOrder)
throws IOException {
List<File> filesToDelete = new ArrayList<>(); // List of temporary files to delete
List<File> filesToDelete = new ArrayList<>();
TempFile outputTempFile = null;
boolean removeCertSign = Boolean.TRUE.equals(request.getRemoveCertSign());
@@ -294,48 +286,35 @@ public class MergeController {
files = new MultipartFile[0];
}
// If front-end provided explicit visible order, honor it and override backend sorting
if (fileOrder != null && !fileOrder.isBlank()) {
log.info("Reordering files based on fileOrder parameter");
files = reorderFilesByProvidedOrder(files, fileOrder);
} else {
log.info("Sorting files based on sortType: {}", request.getSortType());
Arrays.sort(
files,
getSortComparator(
request.getSortType())); // Sort files based on requested sort type
Arrays.sort(files, getSortComparator(request.getSortType()));
}
try (TempFile mt = new TempFile(tempFileManager, ".pdf")) {
PDFMergerUtility mergerUtility = new PDFMergerUtility();
long totalSize = 0;
List<Path> inputPaths = new ArrayList<>(files.length);
List<Integer> invalidIndexes = new ArrayList<>();
for (int index = 0; index < files.length; index++) {
MultipartFile multipartFile = files[index];
totalSize += multipartFile.getSize();
File tempFile =
tempFileManager.convertMultipartFileToFile(
multipartFile); // Convert MultipartFile to File
filesToDelete.add(tempFile); // Add temp file to the list for later deletion
File tempFile = tempFileManager.convertMultipartFileToFile(multipartFile);
filesToDelete.add(tempFile);
inputPaths.add(tempFile.toPath());
// Pre-validate each PDF so we can report which one(s) are broken
// Use the original MultipartFile to avoid deleting the tempFile during validation
try (PDDocument ignored = pdfDocumentFactory.load(multipartFile)) {
// OK
} catch (IOException e) {
try (PdfDocument ignored = PdfDocument.open(tempFile.toPath())) {
} catch (Exception e) {
ExceptionUtils.logException("PDF pre-validate", e);
invalidIndexes.add(index);
}
mergerUtility.addSource(tempFile); // Add source file to the merger utility
}
mergerUtility.setDestinationFileName(mt.getFile().getAbsolutePath());
int[] pageCounts;
try {
mergerUtility.mergeDocuments(
pdfDocumentFactory.getStreamCacheFunction(
totalSize)); // Merge the documents
pageCounts =
mergeWithJpdfium(inputPaths, files, generateToc, mt.getFile().toPath());
} catch (IOException e) {
ExceptionUtils.logException("PDF merge", e);
if (PdfErrorUtils.isCorruptedPdfError(e)) {
@@ -344,10 +323,26 @@ public class MergeController {
throw e;
}
// Load the merged PDF document and operate on it inside try-with-resources
try (PDDocument mergedDocument = pdfDocumentFactory.load(mt.getFile())) {
// Remove signatures if removeCertSign is true
if (removeCertSign) {
boolean sigFlattenNeeded = false;
if (removeCertSign) {
try (PdfDocument check = PdfDocument.open(mt.getFile().toPath())) {
sigFlattenNeeded = !check.signatures().isEmpty();
} catch (Exception e) {
log.debug(
"JPDFium signature pre-check failed; falling back to PDFBox flatten:"
+ " {}",
e.getMessage());
sigFlattenNeeded = true;
}
if (!sigFlattenNeeded) {
log.info(
"removeCertSign requested but merged document has no signature"
+ " fields; skipping PDFBox flatten pass");
}
}
if (sigFlattenNeeded) {
try (PDDocument mergedDocument = pdfDocumentFactory.load(mt.getFile())) {
PDDocumentCatalog catalog = mergedDocument.getDocumentCatalog();
PDAcroForm acroForm = catalog.getAcroForm();
if (acroForm != null) {
@@ -355,24 +350,26 @@ public class MergeController {
acroForm.getFields().stream()
.filter(PDSignatureField.class::isInstance)
.toList();
if (!fieldsToRemove.isEmpty()) {
acroForm.flatten(
fieldsToRemove,
false); // Flatten the fields, effectively removing them
acroForm.flatten(fieldsToRemove, false);
}
}
outputTempFile = new TempFile(tempFileManager, ".pdf");
try {
mergedDocument.save(outputTempFile.getFile());
} catch (Exception e) {
outputTempFile.close();
outputTempFile = null;
throw e;
}
}
// Add table of contents if generateToc is true
if (generateToc && files.length > 0) {
addTableOfContents(mergedDocument, files);
}
// Save the modified document to a temporary file
} else {
outputTempFile = new TempFile(tempFileManager, ".pdf");
try {
mergedDocument.save(outputTempFile.getFile());
Files.copy(
mt.getFile().toPath(),
outputTempFile.getFile().toPath(),
java.nio.file.StandardCopyOption.REPLACE_EXISTING);
} catch (Exception e) {
outputTempFile.close();
outputTempFile = null;
@@ -391,7 +388,7 @@ public class MergeController {
throw ex;
} finally {
for (File file : filesToDelete) {
tempFileManager.deleteTempFile(file); // Delete temporary files
tempFileManager.deleteTempFile(file);
}
}
@@ -401,4 +398,113 @@ public class MergeController {
return WebResponseUtils.pdfFileToWebResponse(outputTempFile, mergedFileName);
}
private int[] mergeWithJpdfium(
List<Path> inputPaths, MultipartFile[] files, boolean generateToc, Path outputPath)
throws IOException {
if (inputPaths.isEmpty()) {
try (PdfDocument empty = PdfDocument.open(new byte[0])) {
empty.save(outputPath);
} catch (Exception ignored) {
Files.write(outputPath, new byte[0]);
}
return new int[0];
}
List<PdfDocument> docs = new ArrayList<>(inputPaths.size());
int[] pageCounts = new int[inputPaths.size()];
int[] pageOffsets = new int[inputPaths.size()];
List<List<Bookmark>> sourceBookmarks = new ArrayList<>(inputPaths.size());
int runningOffset = 0;
try {
for (int i = 0; i < inputPaths.size(); i++) {
Path p = inputPaths.get(i);
PdfDocument doc = PdfDocument.open(p);
docs.add(doc);
pageCounts[i] = doc.pageCount();
pageOffsets[i] = runningOffset;
sourceBookmarks.add(doc.bookmarks());
runningOffset += pageCounts[i];
}
BookmarkTree combinedTree =
buildCombinedBookmarkTree(files, pageOffsets, sourceBookmarks, generateToc);
try (PdfDocument merged = PdfMerge.merge(docs)) {
if (combinedTree.entries().isEmpty()) {
merged.save(outputPath);
} else {
PdfBookmarkEditor.setBookmarks(merged, combinedTree, outputPath);
}
}
} catch (RuntimeException e) {
throw new IOException("JPDFium merge failed", e);
} finally {
for (PdfDocument doc : docs) {
try {
doc.close();
} catch (Exception ignored) {
}
}
}
return pageCounts;
}
private BookmarkTree buildCombinedBookmarkTree(
MultipartFile[] files,
int[] pageOffsets,
List<List<Bookmark>> sourceBookmarks,
boolean generateToc) {
BookmarkTree.Builder builder = BookmarkTree.builder();
if (generateToc) {
for (int i = 0; i < files.length; i++) {
String filename = files[i].getOriginalFilename();
String title = GeneralUtils.removeExtension(filename);
if (title == null || title.isBlank()) {
title = "Document " + (i + 1);
}
builder.add(title, pageOffsets[i]);
}
}
for (int i = 0; i < sourceBookmarks.size(); i++) {
int offset = pageOffsets[i];
for (Bookmark bm : sourceBookmarks.get(i)) {
addBookmarkFlat(builder, bm, offset);
}
}
return builder.build();
}
private void addBookmarkFlat(BookmarkTree.Builder builder, Bookmark root, int offset) {
final int maxNodes = 100_000;
java.util.Deque<Bookmark> stack = new java.util.ArrayDeque<>();
java.util.Set<Bookmark> visited =
java.util.Collections.newSetFromMap(new java.util.IdentityHashMap<>());
stack.push(root);
int processed = 0;
while (!stack.isEmpty() && processed < maxNodes) {
Bookmark bm = stack.pop();
if (!visited.add(bm)) {
continue;
}
processed++;
if (bm.isInternal() && bm.title() != null) {
builder.add(bm.title(), offset + bm.pageIndex());
}
if (bm.hasChildren()) {
List<Bookmark> children = bm.children();
for (int i = children.size() - 1; i >= 0; i--) {
stack.push(children.get(i));
}
}
}
if (processed >= maxNodes) {
log.warn(
"Source bookmark traversal hit {}-node cap; remaining bookmarks dropped",
maxNodes);
}
}
}
@@ -24,6 +24,7 @@ import lombok.extern.slf4j.Slf4j;
import stirling.software.SPDF.model.api.general.MergeMultiplePagesRequest;
import stirling.software.common.annotations.AutoJobPostMapping;
import stirling.software.common.annotations.api.GeneralApi;
import stirling.software.common.enumeration.ResourceWeight;
import stirling.software.common.service.CustomPDFDocumentFactory;
import stirling.software.common.util.ExceptionUtils;
import stirling.software.common.util.GeneralFormCopyUtils;
@@ -41,7 +42,8 @@ public class MultiPageLayoutController {
@AutoJobPostMapping(
value = "/multi-page-layout",
consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
consumes = MediaType.MULTIPART_FORM_DATA_VALUE,
resourceWeight = ResourceWeight.MEDIUM_WEIGHT)
@Operation(
summary = "Merge multiple pages of a PDF document into a single page",
description =
@@ -25,6 +25,7 @@ import stirling.software.SPDF.config.swagger.StandardPdfResponse;
import stirling.software.SPDF.model.api.general.OverlayPdfsRequest;
import stirling.software.common.annotations.AutoJobPostMapping;
import stirling.software.common.annotations.api.GeneralApi;
import stirling.software.common.enumeration.ResourceWeight;
import stirling.software.common.service.CustomPDFDocumentFactory;
import stirling.software.common.util.ExceptionUtils;
import stirling.software.common.util.GeneralUtils;
@@ -39,7 +40,10 @@ public class PdfOverlayController {
private final CustomPDFDocumentFactory pdfDocumentFactory;
private final TempFileManager tempFileManager;
@AutoJobPostMapping(value = "/overlay-pdfs", consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
@AutoJobPostMapping(
value = "/overlay-pdfs",
consumes = MediaType.MULTIPART_FORM_DATA_VALUE,
resourceWeight = ResourceWeight.MEDIUM_WEIGHT)
@StandardPdfResponse
@Operation(
summary = "Overlay PDF files in various modes",
@@ -28,6 +28,7 @@ import stirling.software.SPDF.config.swagger.MultiFileResponse;
import stirling.software.SPDF.model.api.general.PosterPdfRequest;
import stirling.software.common.annotations.AutoJobPostMapping;
import stirling.software.common.annotations.api.GeneralApi;
import stirling.software.common.enumeration.ResourceWeight;
import stirling.software.common.service.CustomPDFDocumentFactory;
import stirling.software.common.util.ExceptionUtils;
import stirling.software.common.util.GeneralUtils;
@@ -43,7 +44,10 @@ public class PosterPdfController {
private final CustomPDFDocumentFactory pdfDocumentFactory;
private final TempFileManager tempFileManager;
@AutoJobPostMapping(value = "/split-for-poster-print", consumes = "multipart/form-data")
@AutoJobPostMapping(
value = "/split-for-poster-print",
consumes = "multipart/form-data",
resourceWeight = ResourceWeight.MEDIUM_WEIGHT)
@MultiFileResponse
@Operation(
summary = "Split large PDF pages into smaller printable chunks",
@@ -6,8 +6,11 @@ import java.util.Collections;
import java.util.List;
import java.util.Locale;
import org.apache.pdfbox.cos.COSName;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.apache.pdfbox.pdmodel.PDDocumentCatalog;
import org.apache.pdfbox.pdmodel.PDPage;
import org.apache.pdfbox.pdmodel.interactive.form.PDAcroForm;
import org.springframework.core.io.Resource;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
@@ -25,8 +28,10 @@ import stirling.software.SPDF.model.api.PDFWithPageNums;
import stirling.software.SPDF.model.api.general.RearrangePagesRequest;
import stirling.software.common.annotations.AutoJobPostMapping;
import stirling.software.common.annotations.api.GeneralApi;
import stirling.software.common.enumeration.ResourceWeight;
import stirling.software.common.service.CustomPDFDocumentFactory;
import stirling.software.common.util.ExceptionUtils;
import stirling.software.common.util.FormUtils;
import stirling.software.common.util.GeneralUtils;
import stirling.software.common.util.TempFileManager;
import stirling.software.common.util.WebResponseUtils;
@@ -39,7 +44,10 @@ public class RearrangePagesPDFController {
private final CustomPDFDocumentFactory pdfDocumentFactory;
private final TempFileManager tempFileManager;
@AutoJobPostMapping(consumes = MediaType.MULTIPART_FORM_DATA_VALUE, value = "/remove-pages")
@AutoJobPostMapping(
consumes = MediaType.MULTIPART_FORM_DATA_VALUE,
value = "/remove-pages",
resourceWeight = ResourceWeight.SMALL_WEIGHT)
@StandardPdfResponse
@Operation(
summary = "Remove pages from a PDF file",
@@ -67,6 +75,7 @@ public class RearrangePagesPDFController {
int pageIndex = pagesToRemove.get(i);
document.removePage(pageIndex);
}
FormUtils.pruneOrphanedFormFields(document);
return WebResponseUtils.pdfDocToWebResponse(
document,
GeneralUtils.generateFilename(
@@ -219,7 +228,10 @@ public class RearrangePagesPDFController {
}
}
@AutoJobPostMapping(consumes = MediaType.MULTIPART_FORM_DATA_VALUE, value = "/rearrange-pages")
@AutoJobPostMapping(
consumes = MediaType.MULTIPART_FORM_DATA_VALUE,
value = "/rearrange-pages",
resourceWeight = ResourceWeight.SMALL_WEIGHT)
@StandardPdfResponse
@Operation(
summary = "Rearrange pages in a PDF file",
@@ -265,6 +277,17 @@ public class RearrangePagesPDFController {
rearrangedDocument.addPage(page);
}
PDDocumentCatalog sourceCatalog = document.getDocumentCatalog();
if (sourceCatalog != null) {
PDAcroForm sourceForm = sourceCatalog.getAcroForm(null);
if (sourceForm != null) {
rearrangedDocument
.getDocumentCatalog()
.getCOSObject()
.setItem(COSName.ACRO_FORM, sourceForm.getCOSObject());
}
}
return WebResponseUtils.pdfDocToWebResponse(
rearrangedDocument,
GeneralUtils.generateFilename(
@@ -19,6 +19,7 @@ import stirling.software.SPDF.config.swagger.StandardPdfResponse;
import stirling.software.SPDF.model.api.general.RotatePDFRequest;
import stirling.software.common.annotations.AutoJobPostMapping;
import stirling.software.common.annotations.api.GeneralApi;
import stirling.software.common.enumeration.ResourceWeight;
import stirling.software.common.service.CustomPDFDocumentFactory;
import stirling.software.common.util.ExceptionUtils;
import stirling.software.common.util.GeneralUtils;
@@ -32,7 +33,10 @@ public class RotationController {
private final CustomPDFDocumentFactory pdfDocumentFactory;
private final TempFileManager tempFileManager;
@AutoJobPostMapping(consumes = MediaType.MULTIPART_FORM_DATA_VALUE, value = "/rotate-pdf")
@AutoJobPostMapping(
consumes = MediaType.MULTIPART_FORM_DATA_VALUE,
value = "/rotate-pdf",
resourceWeight = ResourceWeight.SMALL_WEIGHT)
@StandardPdfResponse
@Operation(
summary = "Rotate a PDF file",
@@ -25,6 +25,7 @@ import lombok.extern.slf4j.Slf4j;
import stirling.software.SPDF.model.api.general.ScalePagesRequest;
import stirling.software.common.annotations.AutoJobPostMapping;
import stirling.software.common.annotations.api.GeneralApi;
import stirling.software.common.enumeration.ResourceWeight;
import stirling.software.common.service.CustomPDFDocumentFactory;
import stirling.software.common.util.ExceptionUtils;
import stirling.software.common.util.GeneralUtils;
@@ -114,7 +115,10 @@ public class ScalePagesController {
return sizeMap;
}
@AutoJobPostMapping(value = "/scale-pages", consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
@AutoJobPostMapping(
value = "/scale-pages",
consumes = MediaType.MULTIPART_FORM_DATA_VALUE,
resourceWeight = ResourceWeight.SMALL_WEIGHT)
@Operation(
summary = "Change the size of a PDF page/document",
description =
@@ -16,6 +16,7 @@ import stirling.software.SPDF.config.EndpointConfiguration;
import stirling.software.common.annotations.AutoJobPostMapping;
import stirling.software.common.annotations.api.SettingsApi;
import stirling.software.common.configuration.InstallationPathConfig;
import stirling.software.common.enumeration.ResourceWeight;
import stirling.software.common.model.ApplicationProperties;
import stirling.software.common.util.GeneralUtils;
@@ -27,7 +28,9 @@ public class SettingsController {
private final ApplicationProperties applicationProperties;
private final EndpointConfiguration endpointConfiguration;
@AutoJobPostMapping("/update-enable-analytics")
@AutoJobPostMapping(
value = "/update-enable-analytics",
resourceWeight = ResourceWeight.SMALL_WEIGHT)
@Hidden
public ResponseEntity<Map<String, Object>> updateApiKey(@RequestParam Boolean enabled)
throws IOException {
@@ -1,9 +1,14 @@
package stirling.software.SPDF.controller.api;
import java.io.File;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.StandardCopyOption;
import java.util.ArrayList;
import java.util.HashSet;
import java.util.List;
import java.util.Set;
import java.util.stream.Collectors;
import java.util.zip.ZipEntry;
import java.util.zip.ZipOutputStream;
@@ -24,12 +29,15 @@ import stirling.software.SPDF.config.swagger.MultiFileResponse;
import stirling.software.SPDF.model.api.SplitPagesRequest;
import stirling.software.common.annotations.AutoJobPostMapping;
import stirling.software.common.annotations.api.GeneralApi;
import stirling.software.common.enumeration.ResourceWeight;
import stirling.software.common.service.CustomPDFDocumentFactory;
import stirling.software.common.util.ExceptionUtils;
import stirling.software.common.util.FormUtils;
import stirling.software.common.util.GeneralUtils;
import stirling.software.common.util.TempFile;
import stirling.software.common.util.TempFileManager;
import stirling.software.common.util.WebResponseUtils;
import stirling.software.jpdfium.PdfDocument;
import stirling.software.jpdfium.PdfSplit;
@GeneralApi
@Slf4j
@@ -39,7 +47,10 @@ public class SplitPDFController {
private final CustomPDFDocumentFactory pdfDocumentFactory;
private final TempFileManager tempFileManager;
@AutoJobPostMapping(consumes = MediaType.MULTIPART_FORM_DATA_VALUE, value = "/split-pages")
@AutoJobPostMapping(
consumes = MediaType.MULTIPART_FORM_DATA_VALUE,
value = "/split-pages",
resourceWeight = ResourceWeight.MEDIUM_WEIGHT)
@MultiFileResponse
@Operation(
summary = "Split a PDF file into separate documents",
@@ -54,9 +65,21 @@ public class SplitPDFController {
MultipartFile file = request.getFileInput();
TempFile outputTempFile = new TempFile(tempFileManager, ".zip");
try {
try (PDDocument document = pdfDocumentFactory.load(file)) {
int totalPages = document.getNumberOfPages();
List<Integer> pageNumbers = request.getPageNumbersList(document, false);
try (TempFile sourceTempFile = new TempFile(tempFileManager, ".pdf")) {
Files.copy(
file.getInputStream(),
sourceTempFile.getPath(),
StandardCopyOption.REPLACE_EXISTING);
int totalPages;
List<Integer> pageNumbers;
boolean hasForm;
try (PDDocument document =
pdfDocumentFactory.load(sourceTempFile.getFile(), true)) {
totalPages = document.getNumberOfPages();
pageNumbers = request.getPageNumbersList(document, false);
hasForm = document.getDocumentCatalog().getAcroForm(null) != null;
}
if (!pageNumbers.contains(totalPages - 1)) {
pageNumbers = new ArrayList<>(pageNumbers);
pageNumbers.add(totalPages - 1);
@@ -69,33 +92,20 @@ public class SplitPDFController {
String baseFilename = GeneralUtils.removeExtension(file.getOriginalFilename());
try (ZipOutputStream zipOut =
new ZipOutputStream(Files.newOutputStream(outputTempFile.getPath()))) {
int previousPageNumber = 0;
for (int splitIndex = 0; splitIndex < pageNumbers.size(); splitIndex++) {
int splitPoint = pageNumbers.get(splitIndex);
try (PDDocument splitDocument =
pdfDocumentFactory.createNewDocumentBasedOnOldDocument(document)) {
for (int i = previousPageNumber; i <= splitPoint; i++) {
splitDocument.addPage(document.getPage(i));
log.debug("Adding page {} to split document", i);
}
previousPageNumber = splitPoint + 1;
String fileName = baseFilename + "_" + (splitIndex + 1) + ".pdf";
zipOut.putNextEntry(new ZipEntry(fileName));
splitDocument.save(zipOut);
zipOut.closeEntry();
log.debug("Wrote split document {} to zip file", fileName);
} catch (Exception e) {
ExceptionUtils.logException("document splitting and saving", e);
throw e;
}
if (hasForm) {
// JPDFium's FPDF_ImportPagesByIndex drops the AcroForm dictionary, which
// breaks Fill Forms downstream. Fall back to the PDFBox load-and-remove
// path so the AcroForm (with only fields whose widgets remain on kept
// pages) is preserved.
writeSplitsViaPdfBox(
sourceTempFile.getFile(), pageNumbers, baseFilename, zipOut);
} else {
writeSplitsViaJpdfium(
sourceTempFile.getFile(), pageNumbers, baseFilename, zipOut);
}
}
}
log.debug(
"Successfully created zip file with split documents: {}",
outputTempFile.getPath().toString());
String zipFilename =
GeneralUtils.generateFilename(file.getOriginalFilename(), "_split.zip");
return WebResponseUtils.zipFileToWebResponse(outputTempFile, zipFilename);
@@ -104,4 +114,61 @@ public class SplitPDFController {
throw e;
}
}
private void writeSplitsViaJpdfium(
File source, List<Integer> pageNumbers, String baseFilename, ZipOutputStream zipOut)
throws IOException {
try (PdfDocument sourceDoc = PdfDocument.open(source.toPath())) {
int previousPageNumber = 0;
for (int splitIndex = 0; splitIndex < pageNumbers.size(); splitIndex++) {
int splitPoint = pageNumbers.get(splitIndex);
try (TempFile splitTemp = new TempFile(tempFileManager, ".pdf")) {
try (PdfDocument splitDoc =
PdfSplit.extractPageRange(sourceDoc, previousPageNumber, splitPoint)) {
splitDoc.save(splitTemp.getPath());
}
writeEntry(zipOut, baseFilename, splitIndex + 1, splitTemp.getPath());
}
previousPageNumber = splitPoint + 1;
}
}
}
private void writeSplitsViaPdfBox(
File source, List<Integer> pageNumbers, String baseFilename, ZipOutputStream zipOut)
throws IOException {
int previousPageNumber = 0;
for (int splitIndex = 0; splitIndex < pageNumbers.size(); splitIndex++) {
int splitPoint = pageNumbers.get(splitIndex);
Set<Integer> keep = new HashSet<>();
for (int i = previousPageNumber; i <= splitPoint; i++) {
keep.add(i);
}
previousPageNumber = splitPoint + 1;
try (PDDocument splitDoc = pdfDocumentFactory.load(source)) {
for (int p = splitDoc.getNumberOfPages() - 1; p >= 0; p--) {
if (!keep.contains(p)) {
splitDoc.removePage(p);
}
}
FormUtils.pruneOrphanedFormFields(splitDoc);
writeEntry(zipOut, baseFilename, splitIndex + 1, splitDoc);
}
}
}
private void writeEntry(ZipOutputStream zipOut, String baseFilename, int index, Path pdfPath)
throws IOException {
zipOut.putNextEntry(new ZipEntry(baseFilename + "_" + index + ".pdf"));
Files.copy(pdfPath, zipOut);
zipOut.closeEntry();
}
private void writeEntry(ZipOutputStream zipOut, String baseFilename, int index, PDDocument doc)
throws IOException {
zipOut.putNextEntry(new ZipEntry(baseFilename + "_" + index + ".pdf"));
doc.save(zipOut);
zipOut.closeEntry();
}
}
@@ -1,6 +1,10 @@
package stirling.software.SPDF.controller.api;
import java.io.File;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.StandardCopyOption;
import java.util.ArrayList;
import java.util.List;
import java.util.Locale;
@@ -8,9 +12,7 @@ import java.util.zip.ZipEntry;
import java.util.zip.ZipOutputStream;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.apache.pdfbox.pdmodel.PDPage;
import org.apache.pdfbox.pdmodel.interactive.documentnavigation.outline.PDDocumentOutline;
import org.apache.pdfbox.pdmodel.interactive.documentnavigation.outline.PDOutlineItem;
import org.apache.pdfbox.pdmodel.interactive.form.PDAcroForm;
import org.springframework.core.io.Resource;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
@@ -30,14 +32,18 @@ import stirling.software.SPDF.config.swagger.MultiFileResponse;
import stirling.software.SPDF.model.api.SplitPdfByChaptersRequest;
import stirling.software.common.annotations.AutoJobPostMapping;
import stirling.software.common.annotations.api.GeneralApi;
import stirling.software.common.enumeration.ResourceWeight;
import stirling.software.common.model.PdfMetadata;
import stirling.software.common.service.CustomPDFDocumentFactory;
import stirling.software.common.service.PdfMetadataService;
import stirling.software.common.util.ExceptionUtils;
import stirling.software.common.util.FormUtils;
import stirling.software.common.util.GeneralUtils;
import stirling.software.common.util.TempFile;
import stirling.software.common.util.TempFileManager;
import stirling.software.common.util.WebResponseUtils;
import stirling.software.jpdfium.PdfDocument;
import stirling.software.jpdfium.PdfSplit;
@GeneralApi
@Slf4j
@@ -50,77 +56,42 @@ public class SplitPdfByChaptersController {
private final TempFileManager tempFileManager;
private static List<Bookmark> extractOutlineItems(
PDDocument sourceDocument,
PDOutlineItem current,
List<Bookmark> bookmarks,
PDOutlineItem nextParent,
private static void collectBookmarks(
List<stirling.software.jpdfium.doc.Bookmark> source,
List<Bookmark> out,
int level,
int maxLevel)
throws Exception {
while (current != null) {
String currentTitle = current.getTitle().replace("/", "");
int firstPage =
sourceDocument.getPages().indexOf(current.findDestinationPage(sourceDocument));
PDOutlineItem child = current.getFirstChild();
PDOutlineItem nextSibling = current.getNextSibling();
int endPage;
if (child != null && level < maxLevel) {
endPage =
sourceDocument
.getPages()
.indexOf(child.findDestinationPage(sourceDocument));
} else if (nextSibling != null) {
endPage =
sourceDocument
.getPages()
.indexOf(nextSibling.findDestinationPage(sourceDocument));
} else if (nextParent != null) {
endPage =
sourceDocument
.getPages()
.indexOf(nextParent.findDestinationPage(sourceDocument));
} else {
endPage = -2;
/*
happens when we have something like this:
Outline Item 2
Outline Item 2.1
Outline Item 2.1.1
Outline Item 2.2
Outline 2.2.1
Outline 2.2.2 <--- this item neither has an immediate next parent nor an immediate next sibling
Outline Item 3
*/
int maxLevel) {
for (stirling.software.jpdfium.doc.Bookmark bm : source) {
if (!bm.isInternal()) {
continue;
}
if (!bookmarks.isEmpty()
&& bookmarks.get(bookmarks.size() - 1).getEndPage() == -2
&& firstPage
>= bookmarks
.get(bookmarks.size() - 1)
.getStartPage()) { // for handling the above-mentioned case
Bookmark previousBookmark = bookmarks.get(bookmarks.size() - 1);
previousBookmark.setEndPage(firstPage);
String title = bm.title() == null ? "" : bm.title().replace("/", "");
int firstPage = Math.max(0, bm.pageIndex());
out.add(new Bookmark(title, firstPage, -2));
if (bm.hasChildren() && level < maxLevel) {
collectBookmarks(bm.children(), out, level + 1, maxLevel);
}
bookmarks.add(new Bookmark(currentTitle, firstPage, endPage));
// Recursively process children
if (child != null && level < maxLevel) {
extractOutlineItems(
sourceDocument, child, bookmarks, nextSibling, level + 1, maxLevel);
}
current = nextSibling;
}
return bookmarks;
}
private static void assignEndPages(List<Bookmark> bookmarks, int totalPages) {
for (int i = 0; i < bookmarks.size(); i++) {
Bookmark current = bookmarks.get(i);
int next = -1;
for (int j = i + 1; j < bookmarks.size(); j++) {
if (bookmarks.get(j).getStartPage() >= current.getStartPage()) {
next = bookmarks.get(j).getStartPage();
break;
}
}
current.setEndPage(next == -1 ? totalPages : next);
}
}
@AutoJobPostMapping(
value = "/split-pdf-by-chapters",
consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
consumes = MediaType.MULTIPART_FORM_DATA_VALUE,
resourceWeight = ResourceWeight.MEDIUM_WEIGHT)
@MultiFileResponse
@Operation(
summary = "Split PDFs by Chapters",
@@ -132,46 +103,41 @@ public class SplitPdfByChaptersController {
MultipartFile file = request.getFileInput();
boolean includeMetadata = Boolean.TRUE.equals(request.getIncludeMetadata());
Integer bookmarkLevel =
request.getBookmarkLevel(); // levels start from 0 (top most bookmarks)
Integer bookmarkLevel = request.getBookmarkLevel();
if (bookmarkLevel < 0) {
throw ExceptionUtils.createIllegalArgumentException(
"error.invalidArgument", "Invalid argument: {0}", "bookmark level");
}
try (PDDocument sourceDocument = pdfDocumentFactory.load(file)) {
PDDocumentOutline outline = sourceDocument.getDocumentCatalog().getDocumentOutline();
try (TempFile sourceTempFile = new TempFile(tempFileManager, ".pdf")) {
Files.copy(
file.getInputStream(),
sourceTempFile.getPath(),
StandardCopyOption.REPLACE_EXISTING);
if (outline == null) {
log.warn("No outline found for {}", file.getOriginalFilename());
throw ExceptionUtils.createIllegalArgumentException(
"error.pdfBookmarksNotFound", "No PDF bookmarks/outline found in document");
}
List<Bookmark> bookmarks = new ArrayList<>();
try {
bookmarks =
extractOutlineItems(
sourceDocument,
outline.getFirstChild(),
bookmarks,
outline.getFirstChild().getNextSibling(),
0,
bookmarkLevel);
// to handle last page edge case
bookmarks.get(bookmarks.size() - 1).setEndPage(sourceDocument.getNumberOfPages());
} catch (Exception e) {
ExceptionUtils.logException("outline extraction", e);
throw e;
int totalPages;
try (PdfDocument sourceDocument = PdfDocument.open(sourceTempFile.getPath())) {
totalPages = sourceDocument.pageCount();
List<stirling.software.jpdfium.doc.Bookmark> roots = sourceDocument.bookmarks();
if (roots == null || roots.isEmpty()) {
log.warn("No outline found for {}", file.getOriginalFilename());
throw ExceptionUtils.createIllegalArgumentException(
"error.pdfBookmarksNotFound",
"No PDF bookmarks/outline found in document");
}
collectBookmarks(roots, bookmarks, 0, bookmarkLevel);
if (bookmarks.isEmpty()) {
log.warn("No outline found for {}", file.getOriginalFilename());
throw ExceptionUtils.createIllegalArgumentException(
"error.pdfBookmarksNotFound",
"No PDF bookmarks/outline found in document");
}
assignEndPages(bookmarks, totalPages);
}
boolean allowDuplicates = Boolean.TRUE.equals(request.getAllowDuplicates());
if (!allowDuplicates) {
/*
duplicates are generated when multiple bookmarks correspond to the same page,
if the user doesn't want duplicates mergeBookmarksThatCorrespondToSamePage() method will merge the titles of all
the bookmarks that correspond to the same page, and treat them as a single bookmark
*/
bookmarks = mergeBookmarksThatCorrespondToSamePage(bookmarks);
}
for (Bookmark bookmark : bookmarks) {
@@ -182,7 +148,23 @@ public class SplitPdfByChaptersController {
bookmark.getEndPage());
}
TempFile zipTempFile = createZipFile(sourceDocument, bookmarks, includeMetadata);
PdfMetadata metadata = null;
boolean hasForm = false;
if (includeMetadata) {
try (PDDocument metaDoc = pdfDocumentFactory.load(sourceTempFile.getFile())) {
metadata = pdfMetadataService.extractMetadataFromPdf(metaDoc);
PDAcroForm acroForm = metaDoc.getDocumentCatalog().getAcroForm(null);
hasForm = acroForm != null;
}
} else {
try (PDDocument acroDoc = pdfDocumentFactory.load(sourceTempFile.getFile(), true)) {
hasForm = acroDoc.getDocumentCatalog().getAcroForm(null) != null;
}
}
TempFile zipTempFile =
createZipFile(
sourceTempFile.getFile(), bookmarks, metadata, totalPages, hasForm);
String filename = GeneralUtils.generateFilename(file.getOriginalFilename(), "");
return WebResponseUtils.zipFileToWebResponse(zipTempFile, filename + ".zip");
}
@@ -214,43 +196,40 @@ public class SplitPdfByChaptersController {
}
private TempFile createZipFile(
PDDocument sourceDocument, List<Bookmark> bookmarks, boolean includeMetadata)
File sourceFile,
List<Bookmark> bookmarks,
PdfMetadata metadata,
int totalPages,
boolean hasForm)
throws Exception {
PdfMetadata metadata =
includeMetadata ? pdfMetadataService.extractMetadataFromPdf(sourceDocument) : null;
String fileNumberFormatter = "%0" + (Integer.toString(bookmarks.size()).length()) + "d ";
TempFile zipTempFile = new TempFile(tempFileManager, ".zip");
try {
try (ZipOutputStream zipOut =
new ZipOutputStream(Files.newOutputStream(zipTempFile.getPath()))) {
try (ZipOutputStream zipOut =
new ZipOutputStream(Files.newOutputStream(zipTempFile.getPath()))) {
if (hasForm) {
// JPDFium's FPDF_ImportPagesByIndex drops the AcroForm dictionary. For form
// PDFs, do the per-chapter extract via PDFBox so form fields survive the split.
for (int i = 0; i < bookmarks.size(); i++) {
Bookmark bookmark = bookmarks.get(i);
try (PDDocument splitDocument = new PDDocument()) {
boolean isSinglePage = (bookmark.getStartPage() == bookmark.getEndPage());
for (int pg = bookmark.getStartPage();
pg < bookmark.getEndPage() + (isSinglePage ? 1 : 0);
pg++) {
PDPage page = sourceDocument.getPage(pg);
splitDocument.addPage(page);
log.debug("Adding page {} to split document", pg);
}
if (includeMetadata) {
pdfMetadataService.setMetadataToPdf(splitDocument, metadata);
}
// split files will be named as "[FILE_NUMBER] [BOOKMARK_TITLE].pdf"
String fileName =
String.format(Locale.ROOT, fileNumberFormatter, i)
+ bookmark.getTitle()
+ ".pdf";
zipOut.putNextEntry(new ZipEntry(fileName));
splitDocument.save(zipOut);
zipOut.closeEntry();
log.debug("Wrote split document {} to zip file", fileName);
} catch (Exception e) {
ExceptionUtils.logException("document splitting and saving", e);
throw e;
writeChapterViaPdfBox(
sourceFile,
bookmarks.get(i),
i,
fileNumberFormatter,
metadata,
zipOut,
totalPages);
}
} else {
try (PdfDocument sourceDocument = PdfDocument.open(sourceFile.toPath())) {
for (int i = 0; i < bookmarks.size(); i++) {
writeChapterViaJpdfium(
sourceDocument,
bookmarks.get(i),
i,
fileNumberFormatter,
metadata,
zipOut,
totalPages);
}
}
}
@@ -263,6 +242,97 @@ public class SplitPdfByChaptersController {
throw e;
}
}
private void writeChapterViaJpdfium(
PdfDocument sourceDocument,
Bookmark bookmark,
int index,
String fileNumberFormatter,
PdfMetadata metadata,
ZipOutputStream zipOut,
int totalPages)
throws Exception {
int[] range = clampRange(bookmark, totalPages);
int from = range[0];
int to = range[1];
try (TempFile splitTemp = new TempFile(tempFileManager, ".pdf")) {
try (PdfDocument splitDoc = PdfSplit.extractPageRange(sourceDocument, from, to)) {
splitDoc.save(splitTemp.getPath());
}
Path finalPath = splitTemp.getPath();
TempFile metaTemp = null;
try {
if (metadata != null) {
metaTemp = new TempFile(tempFileManager, ".pdf");
try (PDDocument doc = pdfDocumentFactory.load(splitTemp.getFile())) {
pdfMetadataService.setMetadataToPdf(doc, metadata);
doc.save(metaTemp.getFile());
}
finalPath = metaTemp.getPath();
}
writeZipEntry(zipOut, fileNumberFormatter, index, bookmark.getTitle(), finalPath);
} finally {
if (metaTemp != null) {
metaTemp.close();
}
}
}
}
private void writeChapterViaPdfBox(
File sourceFile,
Bookmark bookmark,
int index,
String fileNumberFormatter,
PdfMetadata metadata,
ZipOutputStream zipOut,
int totalPages)
throws Exception {
int[] range = clampRange(bookmark, totalPages);
int from = range[0];
int to = range[1];
try (PDDocument doc = pdfDocumentFactory.load(sourceFile)) {
for (int p = doc.getNumberOfPages() - 1; p >= 0; p--) {
if (p < from || p > to) {
doc.removePage(p);
}
}
FormUtils.pruneOrphanedFormFields(doc);
if (metadata != null) {
pdfMetadataService.setMetadataToPdf(doc, metadata);
}
String fileName =
String.format(Locale.ROOT, fileNumberFormatter, index)
+ bookmark.getTitle()
+ ".pdf";
zipOut.putNextEntry(new ZipEntry(fileName));
doc.save(zipOut);
zipOut.closeEntry();
log.debug("Wrote split document {} to zip file", fileName);
}
}
private void writeZipEntry(
ZipOutputStream zipOut,
String fileNumberFormatter,
int index,
String title,
Path pdfPath)
throws IOException {
String fileName = String.format(Locale.ROOT, fileNumberFormatter, index) + title + ".pdf";
zipOut.putNextEntry(new ZipEntry(fileName));
Files.copy(pdfPath, zipOut);
zipOut.closeEntry();
log.debug("Wrote split document {} to zip file", fileName);
}
private static int[] clampRange(Bookmark bookmark, int totalPages) {
boolean isSinglePage = bookmark.getStartPage() == bookmark.getEndPage();
int from = Math.min(Math.max(0, bookmark.getStartPage()), totalPages - 1);
int rawEnd = isSinglePage ? bookmark.getEndPage() : bookmark.getEndPage() - 1;
int to = Math.min(Math.max(from, rawEnd), totalPages - 1);
return new int[] {from, to};
}
}
@Data
@@ -33,6 +33,7 @@ import stirling.software.SPDF.model.SplitTypes;
import stirling.software.SPDF.model.api.SplitPdfBySectionsRequest;
import stirling.software.common.annotations.AutoJobPostMapping;
import stirling.software.common.annotations.api.GeneralApi;
import stirling.software.common.enumeration.ResourceWeight;
import stirling.software.common.service.CustomPDFDocumentFactory;
import stirling.software.common.util.ExceptionUtils;
import stirling.software.common.util.GeneralUtils;
@@ -50,7 +51,8 @@ public class SplitPdfBySectionsController {
@AutoJobPostMapping(
consumes = MediaType.MULTIPART_FORM_DATA_VALUE,
value = "/split-pdf-by-sections")
value = "/split-pdf-by-sections",
resourceWeight = ResourceWeight.MEDIUM_WEIGHT)
@MultiFileResponse
@Operation(
summary = "Split PDF pages into smaller sections",
@@ -1,13 +1,18 @@
package stirling.software.SPDF.controller.api;
import java.io.ByteArrayOutputStream;
import java.io.File;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.StandardCopyOption;
import java.util.ArrayList;
import java.util.HashSet;
import java.util.List;
import java.util.Set;
import java.util.zip.ZipEntry;
import java.util.zip.ZipOutputStream;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.apache.pdfbox.pdmodel.PDPage;
import org.springframework.core.io.Resource;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
@@ -16,7 +21,6 @@ import org.springframework.web.multipart.MultipartFile;
import io.swagger.v3.oas.annotations.Operation;
import lombok.Getter;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
@@ -24,12 +28,16 @@ import stirling.software.SPDF.config.swagger.MultiFileResponse;
import stirling.software.SPDF.model.api.general.SplitPdfBySizeOrCountRequest;
import stirling.software.common.annotations.AutoJobPostMapping;
import stirling.software.common.annotations.api.GeneralApi;
import stirling.software.common.enumeration.ResourceWeight;
import stirling.software.common.service.CustomPDFDocumentFactory;
import stirling.software.common.util.ExceptionUtils;
import stirling.software.common.util.FormUtils;
import stirling.software.common.util.GeneralUtils;
import stirling.software.common.util.TempFile;
import stirling.software.common.util.TempFileManager;
import stirling.software.common.util.WebResponseUtils;
import stirling.software.jpdfium.PdfDocument;
import stirling.software.jpdfium.PdfSplit;
@GeneralApi
@Slf4j
@@ -41,7 +49,8 @@ public class SplitPdfBySizeController {
@AutoJobPostMapping(
value = "/split-by-size-or-count",
consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
consumes = MediaType.MULTIPART_FORM_DATA_VALUE,
resourceWeight = ResourceWeight.MEDIUM_WEIGHT)
@MultiFileResponse
@Operation(
summary = "Auto split PDF pages into separate documents based on size or count",
@@ -54,52 +63,44 @@ public class SplitPdfBySizeController {
public ResponseEntity<Resource> autoSplitPdf(
@ModelAttribute SplitPdfBySizeOrCountRequest request) throws Exception {
log.debug("Starting PDF split process with request: {}", request);
MultipartFile file = request.getFileInput();
String filename = GeneralUtils.generateFilename(file.getOriginalFilename(), "");
log.debug("Base filename for output: {}", filename);
TempFile zipTempFile = new TempFile(tempFileManager, ".zip");
try {
log.debug("Created temporary managed zip file: {}", zipTempFile.getPath());
log.debug("Creating ZIP output stream");
try (ZipOutputStream zipOut =
new ZipOutputStream(Files.newOutputStream(zipTempFile.getPath()));
PDDocument sourceDocument = pdfDocumentFactory.load(file)) {
log.debug(
"Successfully loaded PDF with {} pages", sourceDocument.getNumberOfPages());
try (TempFile sourceTempFile = new TempFile(tempFileManager, ".pdf");
ZipOutputStream zipOut =
new ZipOutputStream(Files.newOutputStream(zipTempFile.getPath()))) {
Files.copy(
file.getInputStream(),
sourceTempFile.getPath(),
StandardCopyOption.REPLACE_EXISTING);
int type = request.getSplitType();
String value = request.getSplitValue();
log.debug("Split type: {}, Split value: {}", type, value);
if (type == 0) {
log.debug("Processing split by size");
long maxBytes = GeneralUtils.convertSizeToBytes(value);
log.debug("Max bytes per document: {}", maxBytes);
handleSplitBySize(sourceDocument, maxBytes, zipOut, filename);
} else if (type == 1) {
log.debug("Processing split by page count");
int pageCount = Integer.parseInt(value);
log.debug("Pages per document: {}", pageCount);
handleSplitByPageCount(sourceDocument, pageCount, zipOut, filename);
} else if (type == 2) {
log.debug("Processing split by document count");
int documentCount = Integer.parseInt(value);
log.debug("Total number of documents: {}", documentCount);
handleSplitByDocCount(sourceDocument, documentCount, zipOut, filename);
} else {
log.error("Invalid split type: {}", type);
throw ExceptionUtils.createIllegalArgumentException(
"error.invalidArgument",
"Invalid argument: {0}",
"split type: " + type);
boolean hasForm;
try (PDDocument acroDoc = pdfDocumentFactory.load(sourceTempFile.getFile(), true)) {
hasForm = acroDoc.getDocumentCatalog().getAcroForm(null) != null;
}
try (PdfDocument sourceDocument = PdfDocument.open(sourceTempFile.getPath())) {
List<int[]> ranges = computeRanges(request, sourceDocument);
int fileIndex = 1;
for (int[] range : ranges) {
if (range.length == 0) {
continue;
}
writeRange(
sourceDocument,
sourceTempFile.getFile(),
range,
zipOut,
filename,
fileIndex++,
hasForm);
}
}
log.debug("PDF splitting completed successfully");
}
log.debug("Returning streaming response for zip file");
return WebResponseUtils.zipFileToWebResponse(zipTempFile, filename + ".zip");
} catch (Exception e) {
ExceptionUtils.logException("PDF splitting process", e);
@@ -108,387 +109,207 @@ public class SplitPdfBySizeController {
}
}
private void handleSplitBySize(
PDDocument sourceDocument, long maxBytes, ZipOutputStream zipOut, String baseFilename)
private List<int[]> computeRanges(SplitPdfBySizeOrCountRequest request, PdfDocument sourceDoc)
throws IOException {
log.debug("Starting handleSplitBySize with maxBytes={}", maxBytes);
int type = request.getSplitType();
String value = request.getSplitValue();
if (type == 0) {
return computeSizeRanges(sourceDoc, GeneralUtils.convertSizeToBytes(value));
} else if (type == 1) {
return computePageCountRanges(sourceDoc, Integer.parseInt(value));
} else if (type == 2) {
return computeDocCountRanges(sourceDoc, Integer.parseInt(value));
}
throw ExceptionUtils.createIllegalArgumentException(
"error.invalidArgument", "Invalid argument: {0}", "split type: " + type);
}
@Getter
class DocHolder implements AutoCloseable {
private PDDocument doc;
public DocHolder(PDDocument doc) {
this.doc = doc;
}
public void setDoc(PDDocument doc) {
if (this.doc != null) {
try {
this.doc.close();
} catch (IOException e) {
log.error("Error closing document", e);
}
}
this.doc = doc;
}
@Override
public void close() throws IOException {
if (doc != null) {
doc.close();
}
private void writeRange(
PdfDocument sourceDoc,
File sourceFile,
int[] range,
ZipOutputStream zipOut,
String baseFilename,
int fileIndex,
boolean hasForm)
throws IOException {
if (hasForm) {
// JPDFium's FPDF_ImportPagesByIndex drops the AcroForm dictionary, breaking form
// fields downstream. For form-bearing PDFs, do the extract via PDFBox so the
// AcroForm survives (pruneOrphanedFormFields removes references to dropped pages).
writeRangeViaPdfBox(sourceFile, range, zipOut, baseFilename, fileIndex);
} else {
try (TempFile splitTemp = new TempFile(tempFileManager, ".pdf")) {
extractRangeToFile(sourceDoc, range, splitTemp.getPath());
writeEntry(zipOut, baseFilename, fileIndex, splitTemp.getPath());
}
}
}
int fileIndex = 1;
try (DocHolder holder =
new DocHolder(
pdfDocumentFactory.createNewDocumentBasedOnOldDocument(sourceDocument))) {
int totalPages = sourceDocument.getNumberOfPages();
int pageAdded = 0;
private void writeRangeViaPdfBox(
File sourceFile,
int[] range,
ZipOutputStream zipOut,
String baseFilename,
int fileIndex)
throws IOException {
Set<Integer> keep = new HashSet<>();
for (int p : range) {
keep.add(p);
}
try (PDDocument doc = pdfDocumentFactory.load(sourceFile)) {
for (int i = doc.getNumberOfPages() - 1; i >= 0; i--) {
if (!keep.contains(i)) {
doc.removePage(i);
}
}
FormUtils.pruneOrphanedFormFields(doc);
zipOut.putNextEntry(new ZipEntry(baseFilename + "_" + fileIndex + ".pdf"));
doc.save(zipOut);
zipOut.closeEntry();
}
}
// Smart size check frequency - check more often with larger documents
int baseCheckFrequency = 5;
private void extractRangeToFile(PdfDocument sourceDoc, int[] range, Path outputPath)
throws IOException {
int from = range[0];
int to = range[range.length - 1];
try (PdfDocument split = PdfSplit.extractPageRange(sourceDoc, from, to)) {
split.save(outputPath);
}
}
private void writeEntry(
ZipOutputStream zipOut, String baseFilename, int fileIndex, Path pdfPath)
throws IOException {
zipOut.putNextEntry(new ZipEntry(baseFilename + "_" + fileIndex + ".pdf"));
Files.copy(pdfPath, zipOut);
zipOut.closeEntry();
}
/** Returns contiguous page-index ranges fitting within {@code maxBytes}. */
private List<int[]> computeSizeRanges(PdfDocument sourceDoc, long maxBytes) throws IOException {
List<int[]> ranges = new ArrayList<>();
int totalPages = sourceDoc.pageCount();
int baseCheckFrequency = 5;
int rangeStart = 0;
int rangeEnd = -1;
try (TempFile probe = new TempFile(tempFileManager, ".pdf")) {
File probeFile = probe.getFile();
for (int pageIndex = 0; pageIndex < totalPages; pageIndex++) {
PDPage page = sourceDocument.getPage(pageIndex);
log.debug("Processing page {} of {}", pageIndex + 1, totalPages);
// Add the page to current document
PDPage newPage = new PDPage(page.getCOSObject());
holder.getDoc().addPage(newPage);
pageAdded++;
// Dynamic size checking based on document size and page count
rangeEnd = pageIndex;
int pageAdded = rangeEnd - rangeStart + 1;
boolean shouldCheckSize =
(pageAdded % baseCheckFrequency == 0)
|| (pageIndex == totalPages - 1)
|| (pageAdded >= 20); // Always check after 20 pages
if (shouldCheckSize) {
log.debug("Performing size check after {} pages", pageAdded);
long actualSize;
try (ByteArrayOutputStream checkSizeStream = new ByteArrayOutputStream()) {
holder.getDoc().save(checkSizeStream);
actualSize = checkSizeStream.size();
}
log.debug(
"Current document size: {} bytes (max: {} bytes)",
actualSize,
maxBytes);
if (actualSize > maxBytes) {
// We exceeded the limit - remove the last page and save
if (holder.getDoc().getNumberOfPages() > 1) {
holder.getDoc().removePage(holder.getDoc().getNumberOfPages() - 1);
pageIndex--; // Process this page again in the next document
log.debug("Size limit exceeded - removed last page");
}
log.debug(
"Saving document with {} pages as part {}",
holder.getDoc().getNumberOfPages(),
fileIndex);
saveDocumentToZip(holder.getDoc(), zipOut, baseFilename, fileIndex++);
holder.setDoc(new PDDocument());
pageAdded = 0;
} else if (pageIndex < totalPages - 1) {
// We're under the limit, calculate if we might fit more pages
// Try to predict how many more similar pages might fit
if (actualSize < maxBytes * 0.75 && pageAdded > 0) {
// Rather than using a ratio, look ahead to test actual upcoming pages
int pagesToLookAhead = Math.min(5, totalPages - pageIndex - 1);
if (pagesToLookAhead > 0) {
log.debug(
"Testing {} upcoming pages for potential addition",
pagesToLookAhead);
// Create a temp document with current pages + look-ahead pages
try (PDDocument testDoc = new PDDocument()) {
// First copy existing pages
for (int i = 0; i < holder.getDoc().getNumberOfPages(); i++) {
testDoc.addPage(
new PDPage(
holder.getDoc().getPage(i).getCOSObject()));
}
// Try adding look-ahead pages one by one
int extraPagesAdded = 0;
for (int i = 0; i < pagesToLookAhead; i++) {
int testPageIndex = pageIndex + 1 + i;
PDPage testPage = sourceDocument.getPage(testPageIndex);
testDoc.addPage(new PDPage(testPage.getCOSObject()));
// Check if we're still under size
long testSize;
try (ByteArrayOutputStream testStream =
new ByteArrayOutputStream()) {
testDoc.save(testStream);
testSize = testStream.size();
}
if (testSize <= maxBytes) {
extraPagesAdded++;
log.debug(
"Test: Can add page {} (size would be {})",
testPageIndex + 1,
testSize);
} else {
log.debug(
"Test: Cannot add page {} (size would be {})",
testPageIndex + 1,
testSize);
break;
}
}
// Add the pages we verified would fit
if (extraPagesAdded > 0) {
log.debug(
"Adding {} verified pages ahead", extraPagesAdded);
for (int i = 0; i < extraPagesAdded; i++) {
int extraPageIndex = pageIndex + 1 + i;
PDPage extraPage =
sourceDocument.getPage(extraPageIndex);
holder.getDoc()
.addPage(new PDPage(extraPage.getCOSObject()));
}
pageIndex += extraPagesAdded;
pageAdded += extraPagesAdded;
}
}
}
}
}
|| (pageAdded >= 20);
if (!shouldCheckSize) {
continue;
}
}
long actualSize = saveRange(sourceDoc, rangeStart, rangeEnd, probeFile);
// Save final document if it has any pages
if (holder.getDoc() != null && holder.getDoc().getNumberOfPages() > 0) {
log.debug(
"Saving final document with {} pages as part {}",
holder.getDoc().getNumberOfPages(),
fileIndex);
saveDocumentToZip(holder.getDoc(), zipOut, baseFilename, fileIndex++);
holder.setDoc(null);
}
}
log.debug("Completed handleSplitBySize with {} document parts created", fileIndex - 1);
}
private void handleSplitByPageCount(
PDDocument sourceDocument, int pageCount, ZipOutputStream zipOut, String baseFilename)
throws IOException {
log.debug("Starting handleSplitByPageCount with pageCount={}", pageCount);
int currentPageCount = 0;
PDDocument currentDoc = null;
int fileIndex = 1;
try {
log.debug("Creating initial output document");
try {
currentDoc = pdfDocumentFactory.createNewDocumentBasedOnOldDocument(sourceDocument);
log.debug("Successfully created initial output document");
} catch (Exception e) {
ExceptionUtils.logException("initial output document creation", e);
throw ExceptionUtils.createFileProcessingException("split", e);
}
int pageIndex = 0;
int totalPages = sourceDocument.getNumberOfPages();
log.debug("Processing {} pages", totalPages);
try {
for (PDPage page : sourceDocument.getPages()) {
pageIndex++;
log.debug("Processing page {} of {}", pageIndex, totalPages);
try {
log.debug("Adding page {} to current document", pageIndex);
currentDoc.addPage(page);
log.debug("Successfully added page {} to current document", pageIndex);
} catch (Exception e) {
log.error("Error adding page {} to current document", pageIndex, e);
throw ExceptionUtils.createFileProcessingException("split", e);
if (actualSize > maxBytes) {
if (pageAdded > 1) {
rangeEnd = pageIndex - 1;
pageIndex--;
}
currentPageCount++;
log.debug("Current page count: {}/{}", currentPageCount, pageCount);
if (currentPageCount == pageCount) {
log.debug(
"Reached target page count ({}), saving current document as part {}",
pageCount,
fileIndex);
try {
saveDocumentToZip(currentDoc, zipOut, baseFilename, fileIndex++);
currentDoc = null; // Document is closed by saveDocumentToZip
log.debug("Successfully saved document part {}", fileIndex - 1);
} catch (Exception e) {
log.error("Error saving document part {}", fileIndex - 1, e);
throw e;
}
try {
log.debug("Creating new document for next part");
currentDoc = new PDDocument();
log.debug("Successfully created new document");
} catch (Exception e) {
log.error("Error creating new document for next part", e);
throw ExceptionUtils.createFileProcessingException("split", e);
}
currentPageCount = 0;
log.debug("Reset current page count to 0");
}
}
} catch (Exception e) {
log.error("Error iterating through pages", e);
throw ExceptionUtils.createFileProcessingException("split", e);
}
// Add the last document if it contains any pages
try {
if (currentDoc != null && currentDoc.getPages().getCount() != 0) {
log.debug(
"Saving final document with {} pages as part {}",
currentDoc.getPages().getCount(),
fileIndex);
try {
saveDocumentToZip(currentDoc, zipOut, baseFilename, fileIndex++);
currentDoc = null; // Document is closed by saveDocumentToZip
log.debug("Successfully saved final document part {}", fileIndex - 1);
} catch (Exception e) {
log.error("Error saving final document part {}", fileIndex - 1, e);
throw e;
}
} else {
log.debug("Final document has no pages, skipping");
}
} catch (Exception e) {
log.error("Error checking or saving final document", e);
throw ExceptionUtils.createFileProcessingException("split", e);
}
} finally {
if (currentDoc != null) {
try {
log.debug("Closing remaining document");
currentDoc.close();
log.debug("Successfully closed remaining document");
} catch (Exception e) {
log.error("Error closing remaining document", e);
ranges.add(buildRange(rangeStart, rangeEnd));
rangeStart = rangeEnd + 1;
rangeEnd = rangeStart - 1;
} else if (pageIndex < totalPages - 1 && actualSize < maxBytes * 0.75) {
int extra =
lookAheadFit(
sourceDoc,
rangeStart,
pageIndex,
maxBytes,
totalPages,
probeFile);
pageIndex += extra;
rangeEnd = pageIndex;
}
}
}
log.debug("Completed handleSplitByPageCount with {} document parts created", fileIndex - 1);
if (rangeEnd >= rangeStart) {
ranges.add(buildRange(rangeStart, rangeEnd));
}
return ranges;
}
private void handleSplitByDocCount(
PDDocument sourceDocument,
int documentCount,
ZipOutputStream zipOut,
String baseFilename)
private long saveRange(PdfDocument sourceDoc, int from, int to, File output)
throws IOException {
log.debug("Starting handleSplitByDocCount with documentCount={}", documentCount);
int totalPageCount = sourceDocument.getNumberOfPages();
log.debug("Total pages in source document: {}", totalPageCount);
try (PdfDocument split = PdfSplit.extractPageRange(sourceDoc, from, to)) {
split.save(output.toPath());
}
return output.length();
}
int pagesPerDocument = totalPageCount / documentCount;
int extraPages = totalPageCount % documentCount;
log.debug("Pages per document: {}, Extra pages: {}", pagesPerDocument, extraPages);
private int lookAheadFit(
PdfDocument sourceDoc,
int rangeStart,
int currentEnd,
long maxBytes,
int totalPages,
File probeFile)
throws IOException {
int pagesToLookAhead = Math.min(5, totalPages - currentEnd - 1);
int extra = 0;
for (int i = 0; i < pagesToLookAhead; i++) {
int trialEnd = currentEnd + 1 + i;
long size = saveRange(sourceDoc, rangeStart, trialEnd, probeFile);
if (size > maxBytes) {
break;
}
extra++;
}
return extra;
}
int currentPageIndex = 0;
int fileIndex = 1;
private List<int[]> computePageCountRanges(PdfDocument sourceDoc, int pageCount) {
if (pageCount <= 0) {
throw ExceptionUtils.createIllegalArgumentException(
"error.invalidArgument", "Invalid argument: {0}", "page count: " + pageCount);
}
int totalPages = sourceDoc.pageCount();
List<int[]> ranges = new ArrayList<>();
int start = 0;
while (start < totalPages) {
int end = Math.min(start + pageCount - 1, totalPages - 1);
ranges.add(buildRange(start, end));
start = end + 1;
}
return ranges;
}
private List<int[]> computeDocCountRanges(PdfDocument sourceDoc, int documentCount) {
if (documentCount <= 0) {
throw ExceptionUtils.createIllegalArgumentException(
"error.invalidArgument",
"Invalid argument: {0}",
"document count: " + documentCount);
}
int totalPages = sourceDoc.pageCount();
int pagesPerDocument = totalPages / documentCount;
int extraPages = totalPages % documentCount;
List<int[]> ranges = new ArrayList<>();
int cursor = 0;
for (int i = 0; i < documentCount; i++) {
log.debug("Creating document {} of {}", i + 1, documentCount);
PDDocument currentDoc = null;
try {
currentDoc = pdfDocumentFactory.createNewDocumentBasedOnOldDocument(sourceDocument);
log.debug("Successfully created document {} of {}", i + 1, documentCount);
int pagesToAdd = pagesPerDocument + (i < extraPages ? 1 : 0);
log.debug("Adding {} pages to document {}", pagesToAdd, i + 1);
for (int j = 0; j < pagesToAdd; j++) {
try {
log.debug(
"Adding page {} (index {}) to document {}",
j + 1,
currentPageIndex,
i + 1);
currentDoc.addPage(sourceDocument.getPage(currentPageIndex));
log.debug("Successfully added page {} to document {}", j + 1, i + 1);
currentPageIndex++;
} catch (Exception e) {
log.error("Error adding page {} to document {}", j + 1, i + 1, e);
throw ExceptionUtils.createFileProcessingException("split", e);
}
}
try {
log.debug("Saving document {} with {} pages", i + 1, pagesToAdd);
saveDocumentToZip(currentDoc, zipOut, baseFilename, fileIndex++);
// saveDocumentToZip closes the document
currentDoc = null;
log.debug("Successfully saved document {}", i + 1);
} catch (Exception e) {
log.error("Error saving document {}", i + 1, e);
throw e;
}
} catch (Exception e) {
log.error("Error creating document {} of {}", i + 1, documentCount, e);
throw ExceptionUtils.createFileProcessingException("split", e);
} finally {
if (currentDoc != null) {
try {
currentDoc.close();
} catch (IOException e) {
log.error("Error closing document {} of {}", i + 1, documentCount, e);
}
}
int pagesToAdd = pagesPerDocument + (i < extraPages ? 1 : 0);
if (pagesToAdd == 0) {
continue;
}
int end = cursor + pagesToAdd - 1;
ranges.add(buildRange(cursor, end));
cursor = end + 1;
}
log.debug("Completed handleSplitByDocCount with {} documents created", documentCount);
return ranges;
}
private void saveDocumentToZip(
PDDocument document, ZipOutputStream zipOut, String baseFilename, int index)
throws IOException {
log.debug("Starting saveDocumentToZip for document part {}", index);
try (ByteArrayOutputStream outStream = new ByteArrayOutputStream()) {
try (PDDocument doc = document) {
log.debug("Saving document part {} to byte array", index);
doc.save(outStream);
log.debug(
"Successfully saved document part {} ({} bytes)", index, outStream.size());
} catch (Exception e) {
log.error("Error saving document part {} to byte array", index, e);
throw ExceptionUtils.createFileProcessingException("split", e);
}
try {
// Create a new zip entry
String entryName = baseFilename + "_" + index + ".pdf";
log.debug("Creating ZIP entry: {}", entryName);
ZipEntry zipEntry = new ZipEntry(entryName);
zipOut.putNextEntry(zipEntry);
byte[] bytes = outStream.toByteArray();
log.debug("Writing {} bytes to ZIP entry", bytes.length);
zipOut.write(bytes);
log.debug("Closing ZIP entry");
zipOut.closeEntry();
log.debug("Successfully added document part {} to ZIP", index);
} catch (Exception e) {
log.error("Error adding document part {} to ZIP", index, e);
throw ExceptionUtils.createFileProcessingException("split", e);
}
private static int[] buildRange(int start, int end) {
int[] range = new int[end - start + 1];
for (int i = 0; i < range.length; i++) {
range[i] = start + i;
}
return range;
}
}
@@ -20,6 +20,7 @@ import lombok.RequiredArgsConstructor;
import stirling.software.SPDF.config.swagger.StandardPdfResponse;
import stirling.software.common.annotations.AutoJobPostMapping;
import stirling.software.common.annotations.api.GeneralApi;
import stirling.software.common.enumeration.ResourceWeight;
import stirling.software.common.model.api.PDFFile;
import stirling.software.common.service.CustomPDFDocumentFactory;
import stirling.software.common.util.GeneralUtils;
@@ -35,7 +36,8 @@ public class ToSinglePageController {
@AutoJobPostMapping(
consumes = MediaType.MULTIPART_FORM_DATA_VALUE,
value = "/pdf-to-single-page")
value = "/pdf-to-single-page",
resourceWeight = ResourceWeight.MEDIUM_WEIGHT)
@StandardPdfResponse
@Operation(
summary = "Convert a multi-page PDF into a single long page PDF",
@@ -28,6 +28,7 @@ import stirling.software.SPDF.config.EndpointConfiguration;
import stirling.software.SPDF.model.api.converters.ConvertEbookToPdfRequest;
import stirling.software.common.annotations.AutoJobPostMapping;
import stirling.software.common.annotations.api.ConvertApi;
import stirling.software.common.enumeration.ResourceWeight;
import stirling.software.common.service.CustomPDFDocumentFactory;
import stirling.software.common.util.GeneralUtils;
import stirling.software.common.util.ProcessExecutor;
@@ -56,7 +57,10 @@ public class ConvertEbookToPDFController {
return endpointConfiguration.isGroupEnabled("Ghostscript");
}
@AutoJobPostMapping(consumes = MediaType.MULTIPART_FORM_DATA_VALUE, value = "/ebook/pdf")
@AutoJobPostMapping(
consumes = MediaType.MULTIPART_FORM_DATA_VALUE,
value = "/ebook/pdf",
resourceWeight = ResourceWeight.LARGE_WEIGHT)
@Operation(
summary = "Convert an eBook file to PDF",
description =
@@ -25,6 +25,7 @@ import stirling.software.SPDF.config.swagger.StandardPdfResponse;
import stirling.software.common.annotations.AutoJobPostMapping;
import stirling.software.common.annotations.api.ConvertApi;
import stirling.software.common.configuration.RuntimePathConfig;
import stirling.software.common.enumeration.ResourceWeight;
import stirling.software.common.model.api.converters.EmlToPdfRequest;
import stirling.software.common.service.CustomPDFDocumentFactory;
import stirling.software.common.util.CustomHtmlSanitizer;
@@ -43,7 +44,10 @@ public class ConvertEmlToPDF {
private final TempFileManager tempFileManager;
private final CustomHtmlSanitizer customHtmlSanitizer;
@AutoJobPostMapping(consumes = MediaType.MULTIPART_FORM_DATA_VALUE, value = "/eml/pdf")
@AutoJobPostMapping(
consumes = MediaType.MULTIPART_FORM_DATA_VALUE,
value = "/eml/pdf",
resourceWeight = ResourceWeight.LARGE_WEIGHT)
@StandardPdfResponse
@Operation(
summary = "Convert EML/MSG to PDF",
@@ -17,6 +17,7 @@ import stirling.software.SPDF.config.swagger.StandardPdfResponse;
import stirling.software.common.annotations.AutoJobPostMapping;
import stirling.software.common.annotations.api.ConvertApi;
import stirling.software.common.configuration.RuntimePathConfig;
import stirling.software.common.enumeration.ResourceWeight;
import stirling.software.common.model.api.converters.HTMLToPdfRequest;
import stirling.software.common.service.CustomPDFDocumentFactory;
import stirling.software.common.util.*;
@@ -33,7 +34,10 @@ public class ConvertHtmlToPDF {
private final CustomHtmlSanitizer customHtmlSanitizer;
@AutoJobPostMapping(consumes = MediaType.MULTIPART_FORM_DATA_VALUE, value = "/html/pdf")
@AutoJobPostMapping(
consumes = MediaType.MULTIPART_FORM_DATA_VALUE,
value = "/html/pdf",
resourceWeight = ResourceWeight.LARGE_WEIGHT)
@StandardPdfResponse
@Operation(
summary = "Convert an HTML or ZIP (containing HTML and CSS) to PDF",
@@ -40,6 +40,7 @@ import stirling.software.SPDF.model.api.converters.ConvertToImageRequest;
import stirling.software.SPDF.model.api.converters.ConvertToPdfRequest;
import stirling.software.common.annotations.AutoJobPostMapping;
import stirling.software.common.annotations.api.ConvertApi;
import stirling.software.common.enumeration.ResourceWeight;
import stirling.software.common.service.CustomPDFDocumentFactory;
import stirling.software.common.util.CbrUtils;
import stirling.software.common.util.CbzUtils;
@@ -72,7 +73,10 @@ public class ConvertImgPDFController {
return endpointConfiguration.isGroupEnabled("Ghostscript");
}
@AutoJobPostMapping(consumes = MediaType.MULTIPART_FORM_DATA_VALUE, value = "/pdf/img")
@AutoJobPostMapping(
consumes = MediaType.MULTIPART_FORM_DATA_VALUE,
value = "/pdf/img",
resourceWeight = ResourceWeight.MEDIUM_WEIGHT)
@MultiFileResponse
@Operation(
summary = "Convert PDF to image(s)",
@@ -239,7 +243,10 @@ public class ConvertImgPDFController {
}
}
@AutoJobPostMapping(consumes = MediaType.MULTIPART_FORM_DATA_VALUE, value = "/img/pdf")
@AutoJobPostMapping(
consumes = MediaType.MULTIPART_FORM_DATA_VALUE,
value = "/img/pdf",
resourceWeight = ResourceWeight.LARGE_WEIGHT)
@StandardPdfResponse
@Operation(
summary = "Convert images to a PDF file",
@@ -26,6 +26,7 @@ import stirling.software.SPDF.config.swagger.StandardPdfResponse;
import stirling.software.common.annotations.AutoJobPostMapping;
import stirling.software.common.annotations.api.ConvertApi;
import stirling.software.common.configuration.RuntimePathConfig;
import stirling.software.common.enumeration.ResourceWeight;
import stirling.software.common.model.api.GeneralFile;
import stirling.software.common.service.CustomPDFDocumentFactory;
import stirling.software.common.util.*;
@@ -41,7 +42,10 @@ public class ConvertMarkdownToPdf {
private final CustomHtmlSanitizer customHtmlSanitizer;
@AutoJobPostMapping(consumes = MediaType.MULTIPART_FORM_DATA_VALUE, value = "/markdown/pdf")
@AutoJobPostMapping(
consumes = MediaType.MULTIPART_FORM_DATA_VALUE,
value = "/markdown/pdf",
resourceWeight = ResourceWeight.LARGE_WEIGHT)
@StandardPdfResponse
@Operation(
summary = "Convert a Markdown file to PDF",
@@ -29,6 +29,7 @@ import stirling.software.SPDF.config.EndpointConfiguration;
import stirling.software.common.annotations.AutoJobPostMapping;
import stirling.software.common.annotations.api.ConvertApi;
import stirling.software.common.configuration.RuntimePathConfig;
import stirling.software.common.enumeration.ResourceWeight;
import stirling.software.common.model.api.GeneralFile;
import stirling.software.common.service.CustomPDFDocumentFactory;
import stirling.software.common.util.CustomHtmlSanitizer;
@@ -200,7 +201,10 @@ public class ConvertOfficeController {
.matches();
}
@AutoJobPostMapping(consumes = MediaType.MULTIPART_FORM_DATA_VALUE, value = "/file/pdf")
@AutoJobPostMapping(
consumes = MediaType.MULTIPART_FORM_DATA_VALUE,
value = "/file/pdf",
resourceWeight = ResourceWeight.LARGE_WEIGHT)
@Operation(
summary = "Convert a file to a PDF using LibreOffice",
description =

Some files were not shown because too many files have changed in this diff Show More