Compare commits

...
Author SHA1 Message Date
James Brunton 14aaf64934 Fix posthog import 2026-03-09 10:35:26 +00:00
Brian BanerjeeandGitHub 81596f0299 Limit PostHog cookie to Stirling PDF's subdomain only (#5882) 2026-03-08 21:03:10 +00:00
Reece BrowneandGitHub 63d38e382d Chore/v2/transforms as root (#5868)
Any task that changes file type or produces more/fewer files than the
input are now consumed as root files not incremented versions of the
input.
2026-03-06 13:46:40 +00:00
Anthony StirlingandGitHub a57e336675 Add searchable settings selector to AppConfigModal and improve nav behavior (#5873) 2026-03-06 11:11:17 +00:00
Anthony StirlingandGitHub 456106195e translations and version bump (#5878) 2026-03-06 11:09:18 +00:00
Anthony StirlingandGitHub 7c1eb4183b stop enabling english (#5874) 2026-03-06 10:09:51 +00:00
7d640e9ce6 option to hide google drive and add settings (#5863)
Co-authored-by: EthanHealy01 <80844253+EthanHealy01@users.noreply.github.com>
2026-03-06 10:09:33 +00:00
ConnorYohandGitHub cafcee6c99 Add the production billing portal link for static plan page (#5860) 2026-03-06 10:08:38 +00:00
ConnorYohandGitHub 7fdd100abf Fix signatures not showing (#5872) 2026-03-06 00:43:29 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
086b55b0bb build(deps): bump pypdf from 6.7.4 to 6.7.5 in /testing/cucumber in the pip group across 1 directory (#5853)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-06 00:15:46 +00:00
c77242d943 fix: merge pdf pipeline validation (#5799)
Co-authored-by: RenzoMXD <RenzoMXD@users.noreply.github.com>
2026-03-06 00:14:30 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
30b0924d6b build(deps): bump digicert/ssm-code-signing from 1.2.0 to 1.2.1 (#5692)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-06 00:13:59 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2d6f206c36 build(deps): bump actions/setup-node from 4.4.0 to 6.2.0 (#5691)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-06 00:13:35 +00:00
Anthony StirlingandGitHub 6c83da6417 Audit fixes and improvements (#5835) 2026-03-05 22:00:44 +00:00
Anthony StirlingandGitHub 879ffc066f tauri notifications (#5875) 2026-03-05 18:30:20 +00:00
Anthony StirlingandGitHub 0f7ee5c5b0 settings menu reworks (#5864) 2026-03-05 16:20:20 +00:00
Anthony StirlingandGitHub ba2d10a75b Persist Tauri window state between launches (#5871) 2026-03-05 16:18:25 +00:00
ConnorYohandGitHub 98835ce7b5 Don't build mac if you don't have the secrets (#5861)
Don't build mac if signing secrets unnavailable. 

No point in trying to build without signing as you cannot install it on
a mac without signature.
2026-03-04 15:59:42 +00:00
stirlingbot[bot]GitHubstirlingbot[bot] <195170888+stirlingbot[bot]@users.noreply.github.com>
2f2ced321a 🤖 format everything with pre-commit by stirlingbot (#5775)
Co-authored-by: stirlingbot[bot] <195170888+stirlingbot[bot]@users.noreply.github.com>
2026-03-03 21:51:44 +00:00
Anthony StirlingandGitHub bfe655fecb Bump version from 2.5.3 to 2.6.0 2026-03-03 21:46:31 +00:00
Balázs SzücsandGitHub 9ac260ee92 feat(aot): add aot-diagnostics.sh for AOT cache diagnostics and validation (#5848)
# Description of Changes



This pull request makes significant improvements to the Docker build
process for the embedded Stirling-PDF image, focusing on build
efficiency, runtime optimization, and maintainability. Key changes
include upgrading major tool versions, introducing optional stripping of
Calibre's WebEngine to reduce image size, consolidating ImageMagick
layers, and refining the Python environment build process. The runtime
image is now leaner, with clearer separation between build and runtime
dependencies, and improved caching for faster builds and pulls.

**Build and Dependency Management Improvements**
* Upgraded Calibre to version `9.4.0` and added support for the
`TARGETPLATFORM` build argument for multi-platform builds.
* Added an optional `CALIBRE_STRIP_WEBENGINE` build argument to strip
Chromium/WebEngine from Calibre, saving ~80 MB when PDF output via
Calibre is not needed.
* Consolidated ImageMagick outputs into a single staging directory
(`/magick-export`) to reduce Docker layers and improve caching
efficiency.
* Refactored Python virtual environment build: now built in a dedicated
stage with pre-built wheels and copied into the runtime image,
eliminating the need for build tools and pip installs at runtime.

**Runtime Image Optimization**
* Reduced installed system packages to only what is needed at runtime;
Python build tools and dev packages are no longer included.
* Cleaned up unnecessary runtime files, including removal of build-only
Python artifacts and system headers, for a smaller and more secure
image.

**Layer and Copy Optimization**
* Switched to `COPY --link` for all major external tool layers and
application files, enabling independent layer caching and parallel pulls
for faster builds.

**Runtime Configuration and Health**
* Improved runtime directory structure and permissions, added persistent
cache directories for Project Leyden AOT, and wrote the version tag to
`/etc/stirling_version` for easier script access.
* Updated the healthcheck to wait longer for startup and increased
timeout/retries for more robust readiness detection.

<!--
Please provide a summary of the changes, including:

- What was changed
- Why the change was made
- Any challenges encountered

Closes #(issue_number)
-->

---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/DeveloperGuide.md#6-testing)
for more details.

---------

Signed-off-by: Balázs Szücs <bszucs1209@gmail.com>
2026-03-03 19:06:46 +00:00
Anthony StirlingandGitHub 1b68a513a9 tauri jdk25 and docs (#5814) 2026-03-03 13:49:33 +00:00
93d7919c4c Fix split tooltips (#5847)
Co-authored-by: EthanHealy01 <80844253+EthanHealy01@users.noreply.github.com>
2026-03-03 13:49:05 +00:00
ConnorYohandGitHub 3e4c984fcc Add check for ghostscript before plowing on with removeDataOutsideCrop (#5845) 2026-03-03 12:52:28 +00:00
ConnorYohandGitHub c4c43593e6 fallback for /api/v1/config/endpoints-availability (#5842) 2026-03-02 22:03:23 +00:00
fd1b7abc83 refactor(merge,split,json): adopt streaming approach and standardize types, address gradle warnings (#5803)
Co-authored-by: Anthony Stirling <77850077+Frooodle@users.noreply.github.com>
Co-authored-by: Balázs <balazs@heim-041-30.jkh.uni-linz.ac.at>
2026-03-02 21:55:07 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>Anthony Stirling
0c46f77179 build(deps): bump com.sun.xml.bind:jaxb-core from 2.3.0.1 to 4.0.6 (#5365)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Anthony Stirling <77850077+Frooodle@users.noreply.github.com>
2026-03-02 16:16:05 +00:00
ConnorYohandGitHub afda066579 Frontend and Desktop audit fixes (#5840) 2026-03-02 15:44:05 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
e9148437f6 build(deps-dev): bump stylelint from 16.26.1 to 17.4.0 in /devTools (#5822)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-02 15:23:26 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
cb835bcce1 build(deps): bump ajv from 8.17.1 to 8.18.0 in /devTools in the npm_and_yarn group across 1 directory (#5774)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-02 15:17:53 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
161bfef7da build(deps): bump actions/ai-inference from 2.0.5 to 2.0.7 (#5831)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-02 15:14:07 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
690eceb548 build(deps): bump github/codeql-action from 4.31.10 to 4.32.4 (#5833)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-02 15:13:52 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
b53c236234 build(deps): bump pypdf from 6.6.2 to 6.7.4 in /testing/cucumber (#5825)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-02 15:12:30 +00:00
Anthony StirlingandGitHub abb8b1f721 Fix health status checks falling under mettric flag (#5821) 2026-03-02 13:56:51 +00:00
012bd1af92 hardening (#5807)
Co-authored-by: Claude Haiku 4.5 <noreply@anthropic.com>
2026-03-02 13:56:39 +00:00
Anthony StirlingandGitHub 8b25db37ad fix split cuased by defaultParameters breaking dynamic endpoint tools (#5838) 2026-03-02 13:55:58 +00:00
Balázs SzücsandGitHub 48dd4154e9 feat(conversion): switch PDF input engine to pdftohtml for improved performance and reduced dependencies (#5820) 2026-03-02 13:55:42 +00:00
StepSecurity BotandGitHub cfe040485b [StepSecurity] Apply security best practices (#5830) 2026-03-01 17:16:03 +00:00
Balázs SzücsandGitHub c15ff1e832 fix(aot): use Spring Boot exploded layer format for aot cache (#5811) 2026-03-01 15:46:02 +00:00
Balázs SzücsandGitHub c244edf8b7 feat(annotation): add moveAnnotation API for efficient repositioning of annotations, and bump embed to 2.7.0 (#5809) 2026-03-01 15:45:44 +00:00
Anthony StirlingandGitHub 13d7ee7496 skip certs (#5819)
# Description of Changes

<!--
Please provide a summary of the changes, including:

- What was changed
- Why the change was made
- Any challenges encountered

Closes #(issue_number)
-->

---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### Translations (if applicable)

- [ ] I ran
[`scripts/counter_translation.py`](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/docs/counter_translation.md)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/DeveloperGuide.md#6-testing)
for more details.
2026-02-27 17:31:40 +00:00
Anthony StirlingandGitHub 930d7a0df8 open-saml bumps (#5805) 2026-02-27 15:03:12 +00:00
Anthony StirlingandGitHub 6a1597bb8d ci: provide default desktop env vars in tauri GitHub Actions builds (#5815) 2026-02-27 10:57:50 +00:00
aikido-autofix[bot]GitHubaikido-autofix[bot] <119856028+aikido-autofix[bot]@users.noreply.github.com>
d98ff194e4 [Aikido] AI Fix for 3rd party Github Actions should be pinned (#5817)
Co-authored-by: aikido-autofix[bot] <119856028+aikido-autofix[bot]@users.noreply.github.com>
2026-02-27 10:42:51 +00:00
242 changed files with 35378 additions and 6156 deletions
+5
View File
@@ -68,3 +68,8 @@ updates:
schedule:
interval: "weekly"
rebase-strategy: "auto"
- package-ecosystem: cargo
directory: /frontend/src-tauri/provisioner
schedule:
interval: daily
+1 -1
View File
@@ -87,7 +87,7 @@ jobs:
- name: AI PR Title Analysis
if: steps.actor.outputs.is_repo_dev == 'true'
id: ai-title-analysis
uses: actions/ai-inference@a6101c89c6feaecc585efdd8d461f18bb7896f20 # v2.0.5
uses: actions/ai-inference@e09e65981758de8b2fdab13c2bfb7c7d5493b0b6 # v2.0.7
with:
model: openai/gpt-4o
system-prompt-file: ".github/config/system-prompt.txt"
+8 -8
View File
@@ -69,7 +69,7 @@ jobs:
distribution: "temurin"
- name: Cache Gradle dependency artifacts
uses: actions/cache@v4
uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
with:
path: |
~/.gradle/wrapper
@@ -147,14 +147,14 @@ jobs:
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Set up JDK 21
- name: Set up JDK 25
uses: actions/setup-java@be666c2fcd27ec809703dec50e508c2fdc7f6654 # v5.2.0
with:
java-version: "25"
distribution: "temurin"
- name: Cache Gradle dependency artifacts
uses: actions/cache@v4
uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
with:
path: |
~/.gradle/wrapper
@@ -194,7 +194,7 @@ jobs:
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Set up Node.js
uses: actions/setup-node@395ad3262231945c25e8478fd5baf05154b1d79f # v6.1.0
uses: actions/setup-node@53b83947a5a98c8d113130e565377fae1a50d02f # v6.3.0
with:
node-version: "22"
cache: "npm"
@@ -236,7 +236,7 @@ jobs:
distribution: "temurin"
- name: Cache Gradle dependency artifacts
uses: actions/cache@v4
uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
with:
path: |
~/.gradle/wrapper
@@ -309,7 +309,7 @@ jobs:
distribution: "temurin"
- name: Cache Gradle dependency artifacts
uses: actions/cache@v4
uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
with:
path: |
~/.gradle/wrapper
@@ -328,7 +328,7 @@ jobs:
# Expose ACTIONS_RUNTIME_TOKEN / ACTIONS_RESULTS_URL for docker buildx type=gha cache backend.
- name: Expose GitHub runtime for Buildx cache
uses: crazy-max/ghaction-github-runtime@v3
uses: crazy-max/ghaction-github-runtime@3cb05d89e1f492524af3d41a1c98c83bc3025124 # v3.1.0
- name: Install Docker Compose
run: |
@@ -416,7 +416,7 @@ jobs:
distribution: "temurin"
- name: Cache Gradle dependency artifacts
uses: actions/cache@v4
uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
with:
path: |
~/.gradle/wrapper
@@ -77,7 +77,7 @@ jobs:
persist-credentials: false
- name: Set up Node.js
uses: actions/setup-node@395ad3262231945c25e8478fd5baf05154b1d79f # v6.1.0
uses: actions/setup-node@53b83947a5a98c8d113130e565377fae1a50d02f # v6.3.0
with:
node-version: "22"
cache: "npm"
+7 -6
View File
@@ -49,7 +49,7 @@ jobs:
distribution: "temurin"
- name: Cache Gradle dependencies
uses: actions/cache@v4
uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
with:
path: |
~/.gradle/caches
@@ -138,7 +138,7 @@ jobs:
- name: Setup Node.js
if: matrix.variant.build_frontend == true
uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
uses: actions/setup-node@53b83947a5a98c8d113130e565377fae1a50d02f # v6.3.0
with:
node-version: 22
cache: "npm"
@@ -193,7 +193,7 @@ jobs:
sudo apt-get install -y libgtk-3-dev libwebkit2gtk-4.0-dev libwebkit2gtk-4.1-dev libappindicator3-dev librsvg2-dev patchelf libjavascriptcoregtk-4.0-dev libsoup2.4-dev libjavascriptcoregtk-4.1-dev libsoup-3.0-dev
- name: Setup Node.js
uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
uses: actions/setup-node@53b83947a5a98c8d113130e565377fae1a50d02f # v6.3.0
with:
node-version: 22
cache: "npm"
@@ -299,7 +299,7 @@ jobs:
- name: Setup DigiCert KeyLocker
id: digicert-setup
if: ${{ matrix.platform == 'windows-latest' && env.SM_API_KEY != '' && (github.event_name == 'release' || github.ref == 'refs/heads/V2-master') }}
uses: digicert/ssm-code-signing@af7f4b07694120e514fe5b19f0598af3c20256e3 # v1.2.0
uses: digicert/ssm-code-signing@1d820463733701cf1484c7eb5d7d24a15ca2c454 # v1.2.1
env:
SM_API_KEY: ${{ secrets.SM_API_KEY }}
SM_CLIENT_CERT_FILE_B64: ${{ secrets.SM_CLIENT_CERT_FILE_B64 }}
@@ -419,8 +419,9 @@ jobs:
APPIMAGETOOL_SIGN_PASSPHRASE: ${{ secrets.APPIMAGETOOL_SIGN_PASSPHRASE }}
TAURI_SIGNING_PRIVATE_KEY: ${{ secrets.TAURI_SIGNING_PRIVATE_KEY }}
TAURI_SIGNING_PRIVATE_KEY_PASSWORD: ${{ secrets.TAURI_SIGNING_PRIVATE_KEY_PASSWORD }}
VITE_SUPABASE_PUBLISHABLE_DEFAULT_KEY: ${{ secrets.VITE_SUPABASE_PUBLISHABLE_DEFAULT_KEY }}
VITE_SAAS_SERVER_URL: ${{ secrets.VITE_SAAS_SERVER_URL }}
VITE_SUPABASE_PUBLISHABLE_DEFAULT_KEY: ${{ secrets.VITE_SUPABASE_PUBLISHABLE_DEFAULT_KEY || 'sb_publishable_UHz2SVRF5mvdrPHWkRteyA_yNlZTkYb' }}
VITE_SAAS_SERVER_URL: ${{ secrets.VITE_SAAS_SERVER_URL || 'https://app.stirlingpdf.com' }}
VITE_SAAS_BACKEND_API_URL: ${{ secrets.VITE_SAAS_BACKEND_API_URL || 'https://api.stirlingpdf.com' }}
# Only enable Windows signing in Tauri when on release or V2-master
SIGN: ${{ (github.event_name == 'release' || github.ref == 'refs/heads/V2-master') && (env.SM_API_KEY == '' && env.WINDOWS_CERTIFICATE != '') && '1' || '0' }}
CI: true
+2 -2
View File
@@ -59,10 +59,10 @@ jobs:
pre-commit run trailing-whitespace --all-files -c .pre-commit-config.yaml
continue-on-error: true
- name: Set up JDK 21
- name: Set up JDK 25
uses: actions/setup-java@be666c2fcd27ec809703dec50e508c2fdc7f6654 # v5.2.0
with:
java-version: 21
java-version: "25"
distribution: "temurin"
- name: Setup Gradle
+1 -1
View File
@@ -46,7 +46,7 @@ jobs:
distribution: "temurin"
- name: Cache Gradle dependencies
uses: actions/cache@v4
uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
with:
path: |
~/.gradle/caches
+1 -1
View File
@@ -75,6 +75,6 @@ jobs:
# Upload the results to GitHub's code scanning dashboard.
- name: "Upload to code-scanning"
uses: github/codeql-action/upload-sarif@cdefb33c0f6224e58673d9004f47f7cb3e328b89 # v3.29.5
uses: github/codeql-action/upload-sarif@89a39a4e59826350b863aa6b6252a07ad50cf83e # v3.29.5
with:
sarif_file: results.sarif
+42 -26
View File
@@ -14,7 +14,7 @@ on:
- macos
- linux
pull_request:
branches: [main, V2-tauri-windows]
branches: [main]
types: [opened, reopened, synchronize, ready_for_review]
paths:
- "frontend/src-tauri/**"
@@ -43,27 +43,42 @@ jobs:
- name: Determine build matrix
id: set-matrix
env:
APPLE_CERTIFICATE: ${{ secrets.APPLE_CERTIFICATE }}
run: |
WINDOWS='{"platform":"windows-latest","args":"--target x86_64-pc-windows-msvc","name":"windows-x86_64"}'
MACOS_ARM='{"platform":"macos-15","args":"--target aarch64-apple-darwin","name":"macos-aarch64"}'
MACOS_INTEL='{"platform":"macos-15-intel","args":"--target x86_64-apple-darwin","name":"macos-x86_64"}'
LINUX='{"platform":"ubuntu-22.04","args":"","name":"linux-x86_64"}'
# Resolve requested platform (non-dispatch events always build all)
if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
case "${{ github.event.inputs.platform }}" in
"windows")
echo 'matrix={"include":[{"platform":"windows-latest","args":"--target x86_64-pc-windows-msvc","name":"windows-x86_64"}]}' >> $GITHUB_OUTPUT
;;
"macos")
echo 'matrix={"include":[{"platform":"macos-15","args":"--target aarch64-apple-darwin","name":"macos-aarch64"},{"platform":"macos-15-intel","args":"--target x86_64-apple-darwin","name":"macos-x86_64"}]}' >> $GITHUB_OUTPUT
;;
"linux")
echo 'matrix={"include":[{"platform":"ubuntu-22.04","args":"","name":"linux-x86_64"}]}' >> $GITHUB_OUTPUT
;;
*)
echo 'matrix={"include":[{"platform":"windows-latest","args":"--target x86_64-pc-windows-msvc","name":"windows-x86_64"},{"platform":"macos-15","args":"--target aarch64-apple-darwin","name":"macos-aarch64"},{"platform":"macos-15-intel","args":"--target x86_64-apple-darwin","name":"macos-x86_64"},{"platform":"ubuntu-22.04","args":"","name":"linux-x86_64"}]}' >> $GITHUB_OUTPUT
;;
esac
PLATFORM="${{ github.event.inputs.platform }}"
else
# For PR/push events, build all platforms
echo 'matrix={"include":[{"platform":"windows-latest","args":"--target x86_64-pc-windows-msvc","name":"windows-x86_64"},{"platform":"macos-15","args":"--target aarch64-apple-darwin","name":"macos-aarch64"},{"platform":"macos-15-intel","args":"--target x86_64-apple-darwin","name":"macos-x86_64"},{"platform":"ubuntu-22.04","args":"","name":"linux-x86_64"}]}' >> $GITHUB_OUTPUT
PLATFORM="all"
fi
# Build candidate list
case "$PLATFORM" in
windows) ENTRIES=("$WINDOWS") ;;
macos) ENTRIES=("$MACOS_ARM" "$MACOS_INTEL") ;;
linux) ENTRIES=("$LINUX") ;;
*) ENTRIES=("$WINDOWS" "$MACOS_ARM" "$MACOS_INTEL" "$LINUX") ;;
esac
# Drop macOS entries when Apple certificate secret is unavailable
if [ -z "$APPLE_CERTIFICATE" ]; then
echo "⚠️ APPLE_CERTIFICATE secret not available - skipping macOS builds"
FILTERED=()
for entry in "${ENTRIES[@]}"; do
[[ "$entry" != *'"macos'* ]] && FILTERED+=("$entry")
done
ENTRIES=("${FILTERED[@]}")
fi
JOINED=$(IFS=','; echo "${ENTRIES[*]}")
echo "matrix={\"include\":[$JOINED]}" >> $GITHUB_OUTPUT
build:
needs: determine-matrix
strategy:
@@ -73,6 +88,7 @@ jobs:
env:
SM_API_KEY: ${{ secrets.SM_API_KEY }}
WINDOWS_CERTIFICATE: ${{ secrets.WINDOWS_CERTIFICATE }}
APPLE_CERTIFICATE: ${{ secrets.APPLE_CERTIFICATE }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@20cf305ff2072d973412fa9b1e3a4f227bda3c76 # v2.14.0
@@ -89,7 +105,7 @@ jobs:
sudo apt-get install -y libgtk-3-dev libwebkit2gtk-4.0-dev libwebkit2gtk-4.1-dev libappindicator3-dev librsvg2-dev patchelf libjavascriptcoregtk-4.0-dev libsoup2.4-dev libjavascriptcoregtk-4.1-dev libsoup-3.0-dev
- name: Setup Node.js
uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
uses: actions/setup-node@53b83947a5a98c8d113130e565377fae1a50d02f # v6.3.0
with:
node-version: 22
cache: "npm"
@@ -101,10 +117,10 @@ jobs:
toolchain: stable
targets: ${{ (matrix.platform == 'macos-15' || matrix.platform == 'macos-15-intel') && 'aarch64-apple-darwin,x86_64-apple-darwin' || '' }}
- name: Set up JDK 21
- name: Set up JDK 25
uses: actions/setup-java@be666c2fcd27ec809703dec50e508c2fdc7f6654 # v5.2.0
with:
java-version: "21"
java-version: "25"
distribution: "temurin"
- name: Setup Gradle
@@ -196,7 +212,7 @@ jobs:
- name: Setup DigiCert KeyLocker
id: digicert-setup
if: ${{ matrix.platform == 'windows-latest' && env.SM_API_KEY != '' && github.ref == 'refs/heads/main' }}
uses: digicert/ssm-code-signing@af7f4b07694120e514fe5b19f0598af3c20256e3 # v1.2.0
uses: digicert/ssm-code-signing@1d820463733701cf1484c7eb5d7d24a15ca2c454 # v1.2.1
env:
SM_API_KEY: ${{ secrets.SM_API_KEY }}
SM_CLIENT_CERT_FILE_B64: ${{ secrets.SM_CLIENT_CERT_FILE_B64 }}
@@ -271,7 +287,7 @@ jobs:
}
- name: Import Apple Developer Certificate
if: matrix.platform == 'macos-15' || matrix.platform == 'macos-15-intel'
if: (matrix.platform == 'macos-15' || matrix.platform == 'macos-15-intel') && env.APPLE_CERTIFICATE != ''
env:
APPLE_CERTIFICATE: ${{ secrets.APPLE_CERTIFICATE }}
APPLE_CERTIFICATE_PASSWORD: ${{ secrets.APPLE_CERTIFICATE_PASSWORD }}
@@ -292,7 +308,7 @@ jobs:
rm certificate.p12
- name: Verify Certificate
if: matrix.platform == 'macos-15' || matrix.platform == 'macos-15-intel'
if: (matrix.platform == 'macos-15' || matrix.platform == 'macos-15-intel') && env.APPLE_CERTIFICATE != ''
run: |
echo "Verifying Apple Developer Certificate..."
KEYCHAIN_PATH=$RUNNER_TEMP/app-signing.keychain-db
@@ -327,9 +343,9 @@ jobs:
APPIMAGETOOL_SIGN_PASSPHRASE: ${{ secrets.APPIMAGETOOL_SIGN_PASSPHRASE }}
TAURI_SIGNING_PRIVATE_KEY: ${{ secrets.TAURI_SIGNING_PRIVATE_KEY }}
TAURI_SIGNING_PRIVATE_KEY_PASSWORD: ${{ secrets.TAURI_SIGNING_PRIVATE_KEY_PASSWORD }}
VITE_SUPABASE_PUBLISHABLE_DEFAULT_KEY: ${{ secrets.VITE_SUPABASE_PUBLISHABLE_DEFAULT_KEY }}
VITE_SAAS_SERVER_URL: ${{ secrets.VITE_SAAS_SERVER_URL }}
VITE_SAAS_BACKEND_API_URL: ${{ secrets.VITE_SAAS_BACKEND_API_URL }}
VITE_SUPABASE_PUBLISHABLE_DEFAULT_KEY: ${{ secrets.VITE_SUPABASE_PUBLISHABLE_DEFAULT_KEY || 'sb_publishable_UHz2SVRF5mvdrPHWkRteyA_yNlZTkYb' }}
VITE_SAAS_SERVER_URL: ${{ secrets.VITE_SAAS_SERVER_URL || 'https://app.stirlingpdf.com' }}
VITE_SAAS_BACKEND_API_URL: ${{ secrets.VITE_SAAS_BACKEND_API_URL || 'https://api.stirlingpdf.com' }}
# Only enable Windows signing in Tauri when on main
SIGN: ${{ github.ref == 'refs/heads/main' && (env.SM_API_KEY == '' && env.WINDOWS_CERTIFICATE != '') && '1' || '0' }}
CI: true
+3 -3
View File
@@ -32,10 +32,10 @@ jobs:
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Set up JDK 21
- name: Set up JDK 25
uses: actions/setup-java@be666c2fcd27ec809703dec50e508c2fdc7f6654 # v5.2.0
with:
java-version: "21"
java-version: "25"
distribution: "temurin"
- name: Setup Gradle
@@ -157,7 +157,7 @@ jobs:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Set up Node
uses: actions/setup-node@395ad3262231945c25e8478fd5baf05154b1d79f # v6.1.0
uses: actions/setup-node@53b83947a5a98c8d113130e565377fae1a50d02f # v6.3.0
with:
cache: "npm"
cache-dependency-path: frontend/package-lock.json
+1 -1
View File
@@ -308,7 +308,7 @@ The frontend is organized with a clear separation of concerns:
## Important Notes
- **Java Version**: Minimum JDK 17, supports and recommends JDK 21
- **Java Version**: Minimum JDK 21, supports and recommends JDK 25
- **Lombok**: Used extensively - ensure IDE plugin is installed
- **File Persistence**:
- **Backend**: Designed to be stateless - files are processed in memory/temp locations only
+3 -3
View File
@@ -11,7 +11,7 @@ This guide focuses on developing for Stirling 2.0, including both the React fron
**Stirling 2.0** is built using:
**Backend:**
- Spring Boot (Java 17+, JDK 21 recommended)
- Spring Boot (Java 21+, JDK 25 recommended)
- PDFBox for core PDF operations
- LibreOffice for document conversions
- qpdf for PDF optimization
@@ -44,7 +44,7 @@ This guide focuses on developing for Stirling 2.0, including both the React fron
- Docker
- Git
- Java JDK 17 or later (JDK 21 recommended)
- Java JDK 21 or later (JDK 25 recommended)
- Node.js 18+ and npm (required for frontend development)
- Gradle 7.0 or later (Included within the repo)
- Rust and Cargo (required for Tauri desktop app development)
@@ -59,7 +59,7 @@ This guide focuses on developing for Stirling 2.0, including both the React fron
cd Stirling-PDF
```
2. Install Docker and JDK17 if not already installed.
2. Install Docker and JDK 21 (or JDK 25 recommended) if not already installed.
3. Install a recommended Java IDE such as Eclipse, IntelliJ, or VSCode
1. Only VSCode
+2 -1
View File
@@ -5,10 +5,10 @@ bootRun {
spotless {
java {
target 'src/**/java/**/*.java'
targetExclude 'src/main/java/org/apache/**'
googleJavaFormat(googleJavaFormatVersion).aosp().reorderImports(false)
importOrder("java", "javax", "org", "com", "net", "io", "jakarta", "lombok", "me", "stirling")
toggleOffOn()
trimTrailingWhitespace()
leadingTabsToSpaces()
endWithNewline()
@@ -27,6 +27,7 @@ spotless {
}
}
dependencies {
api 'com.google.guava:guava:33.4.8-jre'
api 'org.springframework.boot:spring-boot-starter-webmvc'
api 'org.springframework.boot:spring-boot-starter-aspectj'
api 'com.googlecode.owasp-java-html-sanitizer:owasp-java-html-sanitizer:20260102.1'
@@ -103,8 +103,9 @@ public class EndpointConfiguration {
// Rule 2: Functional-group override - check if endpoint belongs to any disabled functional
// group
for (String group : endpointGroups.keySet()) {
if (disabledGroups.contains(group) && endpointGroups.get(group).contains(endpoint)) {
for (Map.Entry<String, Set<String>> entry : endpointGroups.entrySet()) {
String group = entry.getKey();
if (disabledGroups.contains(group) && entry.getValue().contains(endpoint)) {
// Skip tool groups (qpdf, OCRmyPDF, Ghostscript, LibreOffice, etc.)
if (!isToolGroup(group)) {
log.debug(
@@ -131,10 +132,11 @@ public class EndpointConfiguration {
// Rule 4: Single-dependency check - if no alternatives defined, check if endpoint belongs
// to any disabled tool groups
for (String group : endpointGroups.keySet()) {
for (Map.Entry<String, Set<String>> entry : endpointGroups.entrySet()) {
String group = entry.getKey();
if (isToolGroup(group)
&& disabledGroups.contains(group)
&& endpointGroups.get(group).contains(endpoint)) {
&& entry.getValue().contains(endpoint)) {
log.debug(
"isEndpointEnabled('{}') -> false (single tool group '{}' disabled, no alternatives)",
original,
@@ -645,8 +647,9 @@ public class EndpointConfiguration {
}
// Check if endpoint belongs to any disabled functional group
for (String group : endpointGroups.keySet()) {
if (disabledGroups.contains(group) && endpointGroups.get(group).contains(endpoint)) {
for (Map.Entry<String, Set<String>> entry : endpointGroups.entrySet()) {
String group = entry.getKey();
if (disabledGroups.contains(group) && entry.getValue().contains(endpoint)) {
if (!isToolGroup(group)) {
return false;
}
@@ -2,12 +2,14 @@ package stirling.software.common.aop;
import java.io.IOException;
import java.time.Duration;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.TimeUnit;
import java.util.HashMap;
import java.util.Map;
import java.util.concurrent.atomic.AtomicReference;
import java.util.function.Supplier;
import org.aspectj.lang.ProceedingJoinPoint;
import org.aspectj.lang.annotation.*;
import org.slf4j.MDC;
import org.springframework.core.annotation.Order;
import org.springframework.stereotype.Component;
import org.springframework.web.multipart.MultipartFile;
@@ -26,7 +28,7 @@ import stirling.software.common.service.JobExecutorService;
@Component
@RequiredArgsConstructor
@Slf4j
@Order(0) // Highest precedence - executes before audit aspects
@Order(20) // Lower precedence - executes AFTER audit aspects populate MDC
public class AutoJobAspect {
private static final Duration RETRY_BASE_DELAY = Duration.ofMillis(100);
@@ -70,26 +72,29 @@ public class AutoJobAspect {
// No retries needed, simple execution
return jobExecutorService.runJobGeneric(
async,
() -> {
try {
// Note: Progress tracking is handled in TaskManager/JobExecutorService
// The trackProgress flag controls whether detailed progress is stored
// for REST API queries, not WebSocket notifications
return joinPoint.proceed(args);
} catch (Throwable ex) {
log.error(
"AutoJobAspect caught exception during job execution: {}",
ex.getMessage(),
ex);
// Rethrow RuntimeException as-is to preserve exception type
if (ex instanceof RuntimeException) {
throw (RuntimeException) ex;
}
// Wrap checked exceptions - GlobalExceptionHandler will unwrap
// BaseAppException
throw new RuntimeException(ex);
}
},
wrapWithMDC(
() -> {
try {
// Note: Progress tracking is handled in
// TaskManager/JobExecutorService
// The trackProgress flag controls whether detailed progress is
// stored
// for REST API queries, not WebSocket notifications
return joinPoint.proceed(args);
} catch (Throwable ex) {
log.error(
"AutoJobAspect caught exception during job execution: {}",
ex.getMessage(),
ex);
// Rethrow RuntimeException as-is to preserve exception type
if (ex instanceof RuntimeException) {
throw (RuntimeException) ex;
}
// Wrap checked exceptions - GlobalExceptionHandler will unwrap
// BaseAppException
throw new RuntimeException(ex);
}
}),
timeout,
queueable,
resourceWeight);
@@ -123,114 +128,108 @@ public class AutoJobAspect {
return jobExecutorService.runJobGeneric(
async,
() -> {
// Use iterative approach instead of recursion to avoid stack overflow
Throwable lastException = null;
wrapWithMDC(
() -> {
// Use iterative approach instead of recursion to avoid stack overflow
Throwable lastException = null;
// Attempt counter starts at 1 for first try
for (int currentAttempt = 1; currentAttempt <= maxRetries; currentAttempt++) {
try {
if (trackProgress && async) {
// Get jobId for progress tracking in TaskManager
// This enables REST API progress queries, not WebSocket
if (jobIdRef.get() == null) {
jobIdRef.set(getJobIdFromContext());
}
String jobId = jobIdRef.get();
if (jobId != null) {
log.debug(
"Tracking progress for job {} (attempt {}/{})",
jobId,
// Attempt counter starts at 1 for first try
for (int currentAttempt = 1;
currentAttempt <= maxRetries;
currentAttempt++) {
try {
if (trackProgress && async) {
// Get jobId for progress tracking in TaskManager
// This enables REST API progress queries, not WebSocket
if (jobIdRef.get() == null) {
jobIdRef.set(getJobIdFromContext());
}
String jobId = jobIdRef.get();
if (jobId != null) {
log.debug(
"Tracking progress for job {} (attempt {}/{})",
jobId,
currentAttempt,
maxRetries);
// Progress is tracked in TaskManager for REST API
// access
// No WebSocket notifications sent here
}
}
// Attempt to execute the operation
return joinPoint.proceed(args);
} catch (Throwable ex) {
lastException = ex;
log.error(
"AutoJobAspect caught exception during job execution (attempt"
+ " {}/{}): {}",
currentAttempt,
maxRetries);
// Progress is tracked in TaskManager for REST API access
// No WebSocket notifications sent here
}
}
maxRetries,
ex.getMessage(),
ex);
// Attempt to execute the operation
return joinPoint.proceed(args);
// Check if we should retry
if (currentAttempt < maxRetries) {
log.info(
"Retrying operation, attempt {}/{}",
currentAttempt + 1,
maxRetries);
} catch (Throwable ex) {
lastException = ex;
log.error(
"AutoJobAspect caught exception during job execution (attempt"
+ " {}/{}): {}",
currentAttempt,
maxRetries,
ex.getMessage(),
ex);
if (trackProgress && async) {
String jobId = jobIdRef.get();
if (jobId != null) {
log.debug(
"Recording retry attempt for job {} in TaskManager",
jobId);
// Retry info is tracked in TaskManager for REST API
// access
}
}
// Check if we should retry
if (currentAttempt < maxRetries) {
log.info(
"Retrying operation, attempt {}/{}",
currentAttempt + 1,
maxRetries);
// Use sleep for retry delay
// For sync jobs, both sleep and async are blocking at this
// point
// For async jobs, the delay occurs in the executor thread
long delayMs = RETRY_BASE_DELAY.toMillis() * currentAttempt;
if (trackProgress && async) {
String jobId = jobIdRef.get();
if (jobId != null) {
log.debug(
"Recording retry attempt for job {} in TaskManager",
jobId);
// Retry info is tracked in TaskManager for REST API access
try {
Thread.sleep(delayMs);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
log.debug(
"Retry delay interrupted for attempt {}/{}",
currentAttempt,
maxRetries);
break;
}
} else {
// No more retries, we'll throw the exception after the loop
break;
}
}
// Use non-blocking delay for all retry attempts to avoid blocking
// threads
// For sync jobs this avoids starving the tomcat thread pool under
// load
long delayMs = RETRY_BASE_DELAY.toMillis() * currentAttempt;
// Execute the retry after a delay through the JobExecutorService
// rather than blocking the current thread with sleep
CompletableFuture<Object> delayedRetry = new CompletableFuture<>();
// Use a delayed executor for non-blocking delay
CompletableFuture.delayedExecutor(delayMs, TimeUnit.MILLISECONDS)
.execute(
() -> {
// Continue the retry loop in the next iteration
// We can't return from here directly since
// we're in a Runnable
delayedRetry.complete(null);
});
// Wait for the delay to complete before continuing
try {
delayedRetry.join();
} catch (Exception e) {
Thread.currentThread().interrupt();
break;
}
} else {
// No more retries, we'll throw the exception after the loop
break;
}
}
}
// If we get here, all retries failed
if (lastException != null) {
// Rethrow RuntimeException as-is to preserve exception type
if (lastException instanceof RuntimeException) {
throw (RuntimeException) lastException;
}
// Wrap checked exceptions - GlobalExceptionHandler will unwrap
// BaseAppException
throw new RuntimeException(
"Job failed after "
+ maxRetries
+ " attempts: "
+ lastException.getMessage(),
lastException);
}
// If we get here, all retries failed
if (lastException != null) {
// Rethrow RuntimeException as-is to preserve exception type
if (lastException instanceof RuntimeException) {
throw (RuntimeException) lastException;
}
// Wrap checked exceptions - GlobalExceptionHandler will unwrap
// BaseAppException
throw new RuntimeException(
"Job failed after "
+ maxRetries
+ " attempts: "
+ lastException.getMessage(),
lastException);
}
// This should never happen if lastException is properly tracked
throw new RuntimeException("Job failed but no exception was recorded");
},
// This should never happen if lastException is properly tracked
throw new RuntimeException("Job failed but no exception was recorded");
}),
timeout,
queueable,
resourceWeight);
@@ -299,4 +298,32 @@ public class AutoJobAspect {
return null;
}
}
/**
* Wraps a supplier to propagate MDC context to background threads. Captures MDC on request
* thread and restores it in the background thread. Ensures proper cleanup to prevent context
* leakage across jobs in thread pools.
*/
private <T> Supplier<T> wrapWithMDC(Supplier<T> supplier) {
final Map<String, String> captured = MDC.getCopyOfContextMap();
return () -> {
final Map<String, String> previous = MDC.getCopyOfContextMap();
try {
// Set the captured context (or clear if none was captured)
if (captured != null) {
MDC.setContextMap(new HashMap<>(captured));
} else {
MDC.clear();
}
return supplier.get();
} finally {
// Restore previous state (or clear if there was none)
if (previous != null) {
MDC.setContextMap(previous);
} else {
MDC.clear();
}
}
};
}
}
@@ -365,11 +365,11 @@ public class ApplicationProperties {
}
public boolean isSettingsValid() {
return !ValidationUtils.isStringEmpty(this.getIssuer())
&& !ValidationUtils.isStringEmpty(this.getClientId())
&& !ValidationUtils.isStringEmpty(this.getClientSecret())
&& !ValidationUtils.isCollectionEmpty(this.getScopes())
&& !ValidationUtils.isStringEmpty(this.getUseAsUsername());
return !ValidationUtils.isStringEmpty(this.issuer)
&& !ValidationUtils.isStringEmpty(this.clientId)
&& !ValidationUtils.isStringEmpty(this.clientSecret)
&& !ValidationUtils.isCollectionEmpty(this.scopes)
&& !ValidationUtils.isStringEmpty(this.useAsUsername);
}
@Data
@@ -575,19 +575,17 @@ public class ApplicationProperties {
}
public boolean isAnalyticsEnabled() {
return this.getEnableAnalytics() != null && this.getEnableAnalytics();
return this.enableAnalytics != null && this.enableAnalytics;
}
public boolean isPosthogEnabled() {
// Treat null as enabled when analytics is enabled
return this.isAnalyticsEnabled()
&& (this.getEnablePosthog() == null || this.getEnablePosthog());
return this.isAnalyticsEnabled() && (this.enablePosthog == null || this.enablePosthog);
}
public boolean isScarfEnabled() {
// Treat null as enabled when analytics is enabled
return this.isAnalyticsEnabled()
&& (this.getEnableScarf() == null || this.getEnableScarf());
return this.isAnalyticsEnabled() && (this.enableScarf == null || this.enableScarf);
}
}
@@ -698,7 +696,8 @@ public class ApplicationProperties {
@Override
public String toString() {
return """
return
"""
Driver {
driverName='%s'
}
@@ -714,6 +713,7 @@ public class ApplicationProperties {
private String logoStyle = "classic"; // Options: "classic" (default) or "modern"
private boolean defaultHideUnavailableTools = false;
private boolean defaultHideUnavailableConversions = false;
private HideDisabledTools hideDisabledTools = new HideDisabledTools();
public String getAppNameNavbar() {
return appNameNavbar != null && !appNameNavbar.trim().isEmpty() ? appNameNavbar : null;
@@ -726,6 +726,12 @@ public class ApplicationProperties {
}
return "classic"; // default
}
@Data
public static class HideDisabledTools {
private boolean googleDrive = false;
private boolean mobileQRScanner = false;
}
}
@Data
@@ -913,6 +919,15 @@ public class ApplicationProperties {
private boolean ssoAutoLogin;
private boolean database;
private CustomMetadata customMetadata = new CustomMetadata();
private GoogleDrive googleDrive = new GoogleDrive();
@Data
public static class GoogleDrive {
private boolean enabled = false;
private String clientId = "";
private String apiKey = "";
private String appId = "";
}
@Data
public static class CustomMetadata {
@@ -962,6 +977,12 @@ public class ApplicationProperties {
private boolean enabled = true;
private int level = 2; // 0=OFF, 1=BASIC, 2=STANDARD, 3=VERBOSE
private int retentionDays = 90;
private boolean captureFileHash =
false; // Capture SHA-256 hash of files (increases processing time)
private boolean capturePdfAuthor =
false; // Capture PDF author metadata (increases processing time)
private boolean captureOperationResults =
false; // Capture operation return values (not recommended, high volume)
}
@Data
@@ -42,7 +42,7 @@ public enum Role {
// Using the fromString method to get the Role enum based on the roleId
Role role = fromString(roleId);
// Return the roleName of the found Role enum
return role.getRoleName();
return role.roleName;
}
// Method to retrieve all role IDs and role names
@@ -50,14 +50,14 @@ public enum Role {
// Using LinkedHashMap to preserve order
Map<String, String> roleDetails = new LinkedHashMap<>();
for (Role role : Role.values()) {
roleDetails.put(role.getRoleId(), role.getRoleName());
roleDetails.put(role.roleId, role.roleName);
}
return roleDetails;
}
public static Role fromString(String roleId) {
for (Role role : Role.values()) {
if (role.getRoleId().equalsIgnoreCase(roleId)) {
if (role.roleId.equalsIgnoreCase(roleId)) {
return role;
}
}
@@ -117,13 +117,13 @@ public class Provider {
+ ", clientName="
+ getClientName()
+ ", clientId="
+ getClientId()
+ clientId
+ ", clientSecret="
+ (getClientSecret() != null && !getClientSecret().isEmpty() ? "*****" : "NULL")
+ (clientSecret != null && !clientSecret.isEmpty() ? "*****" : "NULL")
+ ", scopes="
+ getScopes()
+ ", useAsUsername="
+ getUseAsUsername()
+ useAsUsername
+ "]";
}
}
@@ -1,6 +1,8 @@
package stirling.software.common.service;
import java.io.BufferedInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.UUID;
@@ -21,6 +23,9 @@ import lombok.extern.slf4j.Slf4j;
@Slf4j
public class FileStorage {
/** Holds the result of a stream-to-disk store operation: the file ID and the bytes written. */
public record StoredFile(String fileId, long size) {}
@Value("${stirling.tempDir:/tmp/stirling-files}")
private String tempDirPath;
@@ -104,6 +109,40 @@ public class FileStorage {
return Files.readAllBytes(filePath);
}
/**
* Retrieve a file by its ID as a streaming InputStream. The caller is responsible for closing
* the returned stream.
*
* @param fileId The ID of the file to retrieve
* @return A buffered InputStream for the file
* @throws IOException If the file doesn't exist or can't be read
*/
public InputStream retrieveInputStream(String fileId) throws IOException {
Path filePath = getFilePath(fileId);
// Let Files.newInputStream throw NoSuchFileException naturally — avoids TOCTOU race
// between exists-check and open when another thread may delete concurrently.
return new BufferedInputStream(Files.newInputStream(filePath));
}
/**
* Store data from an InputStream as a file and return its unique ID and byte count. Streams
* directly to disk without buffering the entire content in heap.
*
* @param inputStream The input stream to read from
* @param originalName The original name of the file (unused, kept for API symmetry)
* @return A {@link StoredFile} containing the file ID and the number of bytes written
* @throws IOException If there is an error storing the file
*/
public StoredFile storeInputStream(InputStream inputStream, String originalName)
throws IOException {
String fileId = generateFileId();
Path filePath = getFilePath(fileId);
Files.createDirectories(filePath.getParent());
long size = Files.copy(inputStream, filePath);
log.debug("Stored input stream with ID: {}", fileId);
return new StoredFile(fileId, size);
}
/**
* Delete a file by its ID
*
@@ -316,7 +316,7 @@ public class JobExecutorService {
filename =
disposition.substring(
disposition.indexOf("filename=") + 9,
disposition.lastIndexOf("\""));
disposition.lastIndexOf('"'));
}
}
@@ -129,7 +129,7 @@ public class MobileScannerService {
FILE_EXTENSION_PATTERN.matcher(safeFilename).replaceFirst("");
String ext =
safeFilename.contains(".")
? safeFilename.substring(safeFilename.lastIndexOf("."))
? safeFilename.substring(safeFilename.lastIndexOf('.'))
: "";
safeFilename = nameWithoutExt + "-" + counter + ext;
filePath = sessionDir.resolve(safeFilename).normalize().toAbsolutePath();
@@ -169,7 +169,10 @@ public class PdfMetadataService {
.getAuthor();
if (userService != null) {
author = author.replace("username", userService.getCurrentUsername());
String username = userService.getCurrentUsername();
if (username != null) {
author = author.replace("username", username);
}
}
}
pdf.getDocumentInformation().setAuthor(author);
@@ -1,8 +1,8 @@
package stirling.software.common.service;
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.BufferedInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.time.LocalDateTime;
import java.time.temporal.ChronoUnit;
import java.util.ArrayList;
@@ -19,7 +19,6 @@ import java.util.zip.ZipInputStream;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.http.MediaType;
import org.springframework.stereotype.Service;
import org.springframework.web.multipart.MultipartFile;
import io.github.pixee.security.ZipSecurity;
@@ -363,39 +362,29 @@ public class TaskManager {
String zipFileId, String originalZipFileName) throws IOException {
List<ResultFile> extractedFiles = new ArrayList<>();
MultipartFile zipFile = fileStorage.retrieveFile(zipFileId);
try (ZipInputStream zipIn =
ZipSecurity.createHardenedInputStream(
new ByteArrayInputStream(zipFile.getBytes()))) {
try (InputStream fileStream = fileStorage.retrieveInputStream(zipFileId);
ZipInputStream zipIn =
ZipSecurity.createHardenedInputStream(
new BufferedInputStream(fileStream))) {
ZipEntry entry;
while ((entry = zipIn.getNextEntry()) != null) {
if (!entry.isDirectory()) {
// Use buffered reading for memory safety
ByteArrayOutputStream out = new ByteArrayOutputStream();
byte[] buffer = new byte[4096];
int bytesRead;
while ((bytesRead = zipIn.read(buffer)) != -1) {
out.write(buffer, 0, bytesRead);
}
byte[] fileContent = out.toByteArray();
String contentType = determineContentType(entry.getName());
String individualFileId = fileStorage.storeBytes(fileContent, entry.getName());
// storeInputStream returns the fileId and byte count — no extra stat needed
FileStorage.StoredFile stored =
fileStorage.storeInputStream(zipIn, entry.getName());
ResultFile resultFile =
ResultFile.builder()
.fileId(individualFileId)
.fileId(stored.fileId())
.fileName(entry.getName())
.contentType(contentType)
.fileSize(fileContent.length)
.fileSize(stored.size())
.build();
extractedFiles.add(resultFile);
log.debug(
"Extracted file: {} (size: {} bytes)",
entry.getName(),
fileContent.length);
"Extracted file: {} (size: {} bytes)", entry.getName(), stored.size());
}
zipIn.closeEntry();
}
@@ -467,4 +456,24 @@ public class TaskManager {
}
return null;
}
/**
* Find the job key that owns a given file ID.
*
* @param fileId file identifier to look up
* @return scoped job key if found, otherwise null
*/
public String findJobKeyByFileId(String fileId) {
for (Map.Entry<String, JobResult> entry : jobResults.entrySet()) {
JobResult jobResult = entry.getValue();
if (jobResult.hasFiles()) {
for (ResultFile resultFile : jobResult.getAllResultFiles()) {
if (fileId.equals(resultFile.getFileId())) {
return entry.getKey();
}
}
}
}
return null;
}
}
@@ -4,6 +4,7 @@ import java.io.BufferedInputStream;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.nio.file.Files;
import java.util.ArrayList;
import java.util.Comparator;
import java.util.Enumeration;
@@ -30,15 +31,7 @@ import stirling.software.common.service.CustomPDFDocumentFactory;
@UtilityClass
public class CbzUtils {
public byte[] convertCbzToPdf(
MultipartFile cbzFile,
CustomPDFDocumentFactory pdfDocumentFactory,
TempFileManager tempFileManager)
throws IOException {
return convertCbzToPdf(cbzFile, pdfDocumentFactory, tempFileManager, false);
}
public byte[] convertCbzToPdf(
public TempFile convertCbzToPdf(
MultipartFile cbzFile,
CustomPDFDocumentFactory pdfDocumentFactory,
TempFileManager tempFileManager,
@@ -64,70 +57,90 @@ public class CbzUtils {
try (PDDocument document = pdfDocumentFactory.createNewDocument();
ZipFile zipFile = new ZipFile(tempFile.getFile())) {
// Pass 1: collect sorted image names (cheap just strings, no image data)
List<String> sortedImageNames = new ArrayList<>();
Enumeration<? extends ZipEntry> entries = zipFile.entries();
List<ImageEntryData> imageEntries = new ArrayList<>();
while (entries.hasMoreElements()) {
ZipEntry entry = entries.nextElement();
if (!entry.isDirectory() && isImageFile(entry.getName())) {
try (InputStream is = zipFile.getInputStream(entry)) {
ByteArrayOutputStream baos = new ByteArrayOutputStream();
is.transferTo(baos);
imageEntries.add(
new ImageEntryData(entry.getName(), baos.toByteArray()));
} catch (IOException e) {
log.warn("Error reading image {}: {}", entry.getName(), e.getMessage());
}
sortedImageNames.add(entry.getName());
}
}
sortedImageNames.sort(new NaturalOrderComparator());
imageEntries.sort(
Comparator.comparing(ImageEntryData::name, new NaturalOrderComparator()));
if (imageEntries.isEmpty()) {
if (sortedImageNames.isEmpty()) {
throw ExceptionUtils.createCbzNoImagesException();
}
for (ImageEntryData imageEntry : imageEntries) {
try {
PDImageXObject pdImage =
PDImageXObject.createFromByteArray(
document, imageEntry.data(), imageEntry.name());
PDPage page =
new PDPage(
new PDRectangle(pdImage.getWidth(), pdImage.getHeight()));
document.addPage(page);
try (PDPageContentStream contentStream =
new PDPageContentStream(
document,
page,
PDPageContentStream.AppendMode.OVERWRITE,
true,
true)) {
contentStream.drawImage(pdImage, 0, 0);
// Pass 2: load ONE image at a time peak memory = max(single image)
for (String imageName : sortedImageNames) {
ZipEntry entry = zipFile.getEntry(imageName);
try (InputStream is = zipFile.getInputStream(entry)) {
ByteArrayOutputStream imgBaos = new ByteArrayOutputStream();
is.transferTo(imgBaos);
byte[] imageBytes = imgBaos.toByteArray();
try {
PDImageXObject pdImage =
PDImageXObject.createFromByteArray(
document, imageBytes, imageName);
PDPage page =
new PDPage(
new PDRectangle(
pdImage.getWidth(), pdImage.getHeight()));
document.addPage(page);
try (PDPageContentStream contentStream =
new PDPageContentStream(
document,
page,
PDPageContentStream.AppendMode.OVERWRITE,
true,
true)) {
contentStream.drawImage(pdImage, 0, 0);
}
} catch (IOException e) {
log.warn("Error processing image {}: {}", imageName, e.getMessage());
}
// imageBytes eligible for GC after each iteration
} catch (IOException e) {
log.warn(
"Error processing image {}: {}", imageEntry.name(), e.getMessage());
log.warn("Error reading image {}: {}", imageName, e.getMessage());
}
}
if (document.getNumberOfPages() == 0) {
throw ExceptionUtils.createCbzCorruptedImagesException();
}
ByteArrayOutputStream baos = new ByteArrayOutputStream();
document.save(baos);
byte[] pdfBytes = baos.toByteArray();
// Apply Ghostscript optimization if requested
if (optimizeForEbook) {
try {
return GeneralUtils.optimizePdfWithGhostscript(pdfBytes);
} catch (IOException e) {
log.warn("Ghostscript optimization failed, returning unoptimized PDF", e);
// Write to TempFile (not BAOS)
TempFile pdfTempFile = new TempFile(tempFileManager, ".pdf");
try {
document.save(pdfTempFile.getFile());
if (optimizeForEbook) {
try {
byte[] pdfBytes = Files.readAllBytes(pdfTempFile.getPath());
byte[] optimized = GeneralUtils.optimizePdfWithGhostscript(pdfBytes);
pdfTempFile.close();
TempFile optimizedFile = new TempFile(tempFileManager, ".pdf");
try {
Files.write(optimizedFile.getPath(), optimized);
return optimizedFile;
} catch (Exception e) {
optimizedFile.close();
throw e;
}
} catch (IOException e) {
log.warn(
"Ghostscript optimization failed, returning unoptimized PDF",
e);
}
}
}
return pdfBytes;
return pdfTempFile;
} catch (Exception e) {
pdfTempFile.close();
throw e;
}
}
}
}
@@ -175,8 +188,6 @@ public class CbzUtils {
return RegexPatternUtils.getInstance().getImageFilePattern().matcher(filename).matches();
}
private record ImageEntryData(String name, byte[] data) {}
private class NaturalOrderComparator implements Comparator<String> {
@Override
public int compare(String s1, String s2) {
@@ -44,9 +44,7 @@ public class FileToPdf {
sanitizeHtmlContent(
new String(fileBytes, StandardCharsets.UTF_8),
customHtmlSanitizer);
Files.write(
tempInputFile.getPath(),
sanitizedHtml.getBytes(StandardCharsets.UTF_8));
Files.writeString(tempInputFile.getPath(), sanitizedHtml);
} else if (fileName.toLowerCase(Locale.ROOT).endsWith(".zip")) {
Files.write(tempInputFile.getPath(), fileBytes);
sanitizeHtmlFilesInZip(
@@ -115,8 +113,7 @@ public class FileToPdf {
new String(zipIn.readAllBytes(), StandardCharsets.UTF_8);
String sanitizedContent =
sanitizeHtmlContent(content, customHtmlSanitizer);
Files.write(
filePath, sanitizedContent.getBytes(StandardCharsets.UTF_8));
Files.writeString(filePath, sanitizedContent);
} else {
Files.copy(zipIn, filePath);
}
@@ -17,6 +17,7 @@ import java.util.Map;
import java.util.Objects;
import java.util.Optional;
import java.util.Set;
import java.util.regex.Pattern;
import org.apache.pdfbox.cos.COSArray;
import org.apache.pdfbox.cos.COSBase;
@@ -74,6 +75,10 @@ public class FormUtils {
*/
private static final float SAME_LINE_THRESHOLD_PT = 10.0f;
private static final Pattern HEX_UUID_PATTERN =
Pattern.compile("^[0-9a-fA-F]{8}[0-9a-fA-F]{24,}$");
private static final Pattern WHITESPACE_PATTERN = Pattern.compile("\\s+");
/**
* Returns a normalized logical type string for the supplied PDFBox field instance. Centralized
* so all callers share identical mapping logic.
@@ -1357,7 +1362,7 @@ public class FormUtils {
if (da != null && !da.isBlank()) {
// Standard DA looks like: /Helv 12 Tf 0 g
// We want the number before 'Tf'
String[] tokens = da.split("\\s+");
String[] tokens = WHITESPACE_PATTERN.split(da);
for (int i = 0; i < tokens.length; i++) {
if ("Tf".equals(tokens[i]) && i > 0) {
try {
@@ -1457,9 +1462,8 @@ public class FormUtils {
// Detect UUID-like hex strings (e.g. "cdc47b7041524571 7b2d93017fe77bf7")
// Standard UUIDs are 32 hex characters; require at least that to avoid
// false positives on short hex-like field names.
String nospaces = simplified.replaceAll("\\s+", "");
if (nospaces.length() >= 32 && nospaces.matches("^[0-9a-fA-F]{8}[0-9a-fA-F]{24,}$"))
return true;
String nospaces = WHITESPACE_PATTERN.matcher(simplified).replaceAll("");
if (nospaces.length() >= 32 && HEX_UUID_PATTERN.matcher(nospaces).matches()) return true;
return patterns.getGenericFieldNamePattern().matcher(simplified).matches()
|| patterns.getSimpleFormFieldPattern().matcher(simplified).matches()
@@ -873,6 +873,36 @@ public class GeneralUtils {
settingsYaml.saveOverride(settingsPath);
}
/**
* Updates multiple settings in a single transaction. This ensures that nested settings (e.g.,
* oauth2.client.google.*) don't lose sibling values when partial updates are made.
*
* <p>Instead of multiple read-update-write cycles (which could cause race conditions), this
* method loads the YAML once, applies all updates, and saves once.
*
* @param settingsMap Map of dotted-notation keys to values to update
* @throws IOException if file read/write fails
*/
public void updateSettingsTransactional(Map<String, Object> settingsMap) throws IOException {
if (settingsMap == null || settingsMap.isEmpty()) {
return;
}
Path settingsPath = Paths.get(InstallationPathConfig.getSettingsPath());
YamlHelper settingsYaml = new YamlHelper(settingsPath);
// Apply all updates to the same YamlHelper instance
for (Map.Entry<String, Object> entry : settingsMap.entrySet()) {
String key = entry.getKey();
Object value = entry.getValue();
String[] keyArray = key.split("\\.");
settingsYaml.updateValue(Arrays.asList(keyArray), value);
}
// Save only once after all updates are applied
settingsYaml.saveOverride(settingsPath);
}
/*
* Machine fingerprint generation with better error logging and fallbacks.
*
@@ -10,6 +10,7 @@ import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.Objects;
import java.util.regex.Pattern;
import java.util.zip.ZipEntry;
import java.util.zip.ZipOutputStream;
@@ -33,6 +34,8 @@ import stirling.software.common.util.ProcessExecutor.ProcessExecutorResult;
@Slf4j
public class PDFToFile {
private static final Pattern PATTERN =
Pattern.compile("(!\\[.*?\\])\\((?!images/)([^/)][^)]*?)\\)");
private final TempFileManager tempFileManager;
private final RuntimePathConfig runtimePathConfig;
@@ -163,7 +166,7 @@ public class PDFToFile {
private String updateImageReferences(String markdown) {
// Match markdown image syntax: ![alt text](image.png)
// Only update if the path doesn't already start with images/
return markdown.replaceAll("(!\\[.*?\\])\\((?!images/)([^/)][^)]*?)\\)", "$1(images/$2)");
return PATTERN.matcher(markdown).replaceAll("$1(images/$2)");
}
public ResponseEntity<byte[]> processPdfToHtml(MultipartFile inputFile)
@@ -1,8 +1,8 @@
package stirling.software.common.util;
import java.awt.image.BufferedImage;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.nio.file.Files;
import java.util.Locale;
import java.util.zip.ZipEntry;
import java.util.zip.ZipOutputStream;
@@ -22,8 +22,11 @@ import stirling.software.common.service.CustomPDFDocumentFactory;
@Slf4j
public class PdfToCbzUtils {
public static byte[] convertPdfToCbz(
MultipartFile pdfFile, int dpi, CustomPDFDocumentFactory pdfDocumentFactory)
public static TempFile convertPdfToCbz(
MultipartFile pdfFile,
int dpi,
CustomPDFDocumentFactory pdfDocumentFactory,
TempFileManager tempFileManager)
throws IOException {
validatePdfFile(pdfFile);
@@ -33,7 +36,7 @@ public class PdfToCbzUtils {
throw ExceptionUtils.createPdfNoPages();
}
return createCbzFromPdf(document, dpi);
return createCbzFromPdf(document, dpi, tempFileManager);
}
}
@@ -53,46 +56,49 @@ public class PdfToCbzUtils {
}
}
private static byte[] createCbzFromPdf(PDDocument document, int dpi) throws IOException {
private static TempFile createCbzFromPdf(
PDDocument document, int dpi, TempFileManager tempFileManager) throws IOException {
PDFRenderer pdfRenderer = new PDFRenderer(document);
pdfRenderer.setSubsamplingAllowed(true); // Enable subsampling to reduce memory usage
try (ByteArrayOutputStream cbzOutputStream = new ByteArrayOutputStream();
ZipOutputStream zipOut = new ZipOutputStream(cbzOutputStream)) {
TempFile cbzTempFile = new TempFile(tempFileManager, ".cbz");
try {
try (ZipOutputStream zipOut =
new ZipOutputStream(Files.newOutputStream(cbzTempFile.getPath()))) {
int totalPages = document.getNumberOfPages();
int totalPages = document.getNumberOfPages();
for (int pageIndex = 0; pageIndex < totalPages; pageIndex++) {
final int currentPage = pageIndex;
try {
BufferedImage image =
ExceptionUtils.handleOomRendering(
currentPage + 1,
dpi,
() ->
pdfRenderer.renderImageWithDPI(
currentPage, dpi, ImageType.RGB));
for (int pageIndex = 0; pageIndex < totalPages; pageIndex++) {
final int currentPage = pageIndex;
try {
BufferedImage image =
ExceptionUtils.handleOomRendering(
currentPage + 1,
dpi,
() ->
pdfRenderer.renderImageWithDPI(
currentPage, dpi, ImageType.RGB));
String imageFilename =
String.format(Locale.ROOT, "page_%03d.png", currentPage + 1);
ZipEntry zipEntry = new ZipEntry(imageFilename);
zipOut.putNextEntry(zipEntry);
String imageFilename =
String.format(Locale.ROOT, "page_%03d.png", currentPage + 1);
zipOut.putNextEntry(new ZipEntry(imageFilename));
ImageIO.write(image, "PNG", zipOut);
zipOut.closeEntry();
ImageIO.write(image, "PNG", zipOut);
zipOut.closeEntry();
} catch (ExceptionUtils.OutOfMemoryDpiException e) {
// Re-throw OOM exceptions without wrapping
throw e;
} catch (IOException e) {
// Wrap other IOExceptions with context
throw ExceptionUtils.createFileProcessingException(
"CBZ creation for page " + (currentPage + 1), e);
} catch (ExceptionUtils.OutOfMemoryDpiException e) {
// Re-throw OOM exceptions without wrapping
throw e;
} catch (IOException e) {
// Wrap other IOExceptions with context
throw ExceptionUtils.createFileProcessingException(
"CBZ creation for page " + (currentPage + 1), e);
}
}
}
zipOut.finish();
return cbzOutputStream.toByteArray();
return cbzTempFile;
} catch (Exception e) {
cbzTempFile.close();
throw e;
}
}
@@ -219,58 +219,31 @@ public class PdfUtils {
int maxWidth = 0;
int totalHeight = 0;
BufferedImage pdfSizeImage = null;
int pdfSizeImageIndex = -1;
// Using a map to store the rendered dimensions of each page size
// to avoid rendering the same page sizes multiple times
// Using a map to store the calculated dimensions of each page size
HashMap<PdfRenderSettingsKey, PdfImageDimensionValue> pageSizes =
new HashMap<>();
for (int i = 0; i < pageCount; ++i) {
final int pageIndex = i;
PDPage page = document.getPage(i);
PDRectangle mediaBox = page.getMediaBox();
PDRectangle cropBox = page.getCropBox();
int rotation = page.getRotation();
PdfRenderSettingsKey settings =
new PdfRenderSettingsKey(
mediaBox.getWidth(), mediaBox.getHeight(), rotation);
cropBox.getWidth(), cropBox.getHeight(), rotation);
PdfImageDimensionValue dimension = pageSizes.get(settings);
if (dimension == null) {
// Render the image to get the dimensions
try {
// Validate dimensions before rendering
ExceptionUtils.validateRenderingDimensions(
page, pageIndex + 1, DPI);
pdfSizeImage =
ExceptionUtils.handleOomRendering(
pageIndex + 1,
DPI,
() ->
pdfRenderer.renderImageWithDPI(
pageIndex, DPI, colorType));
} catch (IllegalArgumentException e) {
if (e.getMessage() != null
&& e.getMessage()
.contains("Maximum size of image exceeded")) {
throw ExceptionUtils.createIllegalArgumentException(
"error.pageTooBigExceedsArray",
"PDF page {0} is too large to render at {1} DPI. The"
+ " resulting image would exceed Java's maximum"
+ " array size. Please try a lower DPI value"
+ " (recommended: 150 or less).",
i + 1,
DPI);
}
throw e;
float scale = DPI / 72f;
int widthPx = (int) Math.max(Math.floor(cropBox.getWidth() * scale), 1);
int heightPx =
(int) Math.max(Math.floor(cropBox.getHeight() * scale), 1);
if (rotation == 90 || rotation == 270) {
int tmp = widthPx;
widthPx = heightPx;
heightPx = tmp;
}
pdfSizeImageIndex = i;
dimension =
new PdfImageDimensionValue(
pdfSizeImage.getWidth(), pdfSizeImage.getHeight());
dimension = new PdfImageDimensionValue(widthPx, heightPx);
pageSizes.put(settings, dimension);
if (pdfSizeImage.getWidth() > maxWidth) {
maxWidth = pdfSizeImage.getWidth();
if (widthPx > maxWidth) {
maxWidth = widthPx;
}
}
totalHeight += dimension.height();
@@ -284,40 +257,32 @@ public class PdfUtils {
int currentHeight = 0;
BufferedImage pageImage;
// Check if the first image is the last rendered image
boolean firstImageAlreadyRendered = pdfSizeImageIndex == 0;
for (int i = 0; i < pageCount; ++i) {
final int pageIndex = i;
if (firstImageAlreadyRendered && i == 0) {
pageImage = pdfSizeImage;
} else {
try {
// Validate dimensions before rendering
ExceptionUtils.validateRenderingDimensions(
document.getPage(pageIndex), pageIndex + 1, DPI);
try {
// Validate dimensions before rendering
ExceptionUtils.validateRenderingDimensions(
document.getPage(pageIndex), pageIndex + 1, DPI);
pageImage =
ExceptionUtils.handleOomRendering(
pageIndex + 1,
DPI,
() ->
pdfRenderer.renderImageWithDPI(
pageIndex, DPI, colorType));
} catch (IllegalArgumentException e) {
if (e.getMessage() != null
&& e.getMessage()
.contains("Maximum size of image exceeded")) {
throw ExceptionUtils.createIllegalArgumentException(
"error.pageTooBigForDpi",
"PDF page {0} is too large to render at {1} DPI. Please"
+ " try a lower DPI value (recommended: 150 or"
+ " less).",
i + 1,
DPI);
}
throw e;
pageImage =
ExceptionUtils.handleOomRendering(
pageIndex + 1,
DPI,
() ->
pdfRenderer.renderImageWithDPI(
pageIndex, DPI, colorType));
} catch (IllegalArgumentException e) {
if (e.getMessage() != null
&& e.getMessage().contains("Maximum size of image exceeded")) {
throw ExceptionUtils.createIllegalArgumentException(
"error.pageTooBigForDpi",
"PDF page {0} is too large to render at {1} DPI. Please"
+ " try a lower DPI value (recommended: 150 or"
+ " less).",
i + 1,
DPI);
}
throw e;
}
// Calculate the x-coordinate to center the image
@@ -191,7 +191,7 @@ public class YamlHelper {
mappingNode.getValue().clear();
mappingNode.getValue().addAll(updatedTuples);
}
setNewNode(node);
updatedRootNode = node;
return updated;
}
@@ -19,7 +19,7 @@ public abstract class ReplaceAndInvertColorStrategy extends PDFFile {
public ReplaceAndInvertColorStrategy(MultipartFile file, ReplaceAndInvert replaceAndInvert) {
setFileInput(file);
setReplaceAndInvert(replaceAndInvert);
this.replaceAndInvert = replaceAndInvert;
}
public abstract InputStreamResource replace() throws IOException;
+5 -4
View File
@@ -12,11 +12,10 @@ configurations {
spotless {
java {
target 'src/**/java/**/*.java'
targetExclude 'src/main/resources/static/**'
targetExclude 'src/main/resources/static/**', 'src/main/java/org/apache/**'
googleJavaFormat(googleJavaFormatVersion).aosp().reorderImports(false)
importOrder("java", "javax", "org", "com", "net", "io", "jakarta", "lombok", "me", "stirling")
toggleOffOn()
trimTrailingWhitespace()
leadingTabsToSpaces()
endWithNewline()
@@ -82,7 +81,7 @@ dependencies {
// veraPDF still uses javax.xml.bind, not the new jakarta namespace
implementation 'javax.xml.bind:jaxb-api:2.3.1'
implementation 'com.sun.xml.bind:jaxb-impl:2.3.9'
implementation 'com.sun.xml.bind:jaxb-core:2.3.0.1'
implementation 'com.sun.xml.bind:jaxb-core:4.0.6'
implementation 'org.apache.poi:poi-ooxml:5.5.1'
// https://mvnrepository.com/artifact/technology.tabula/tabula
@@ -91,7 +90,7 @@ dependencies {
exclude group: 'org.bouncycastle', module: 'bcprov-jdk15on'
exclude group: 'com.google.code.gson', module: 'gson'
}
// CVE-2022-25647: Explicit gson 2.8.9 to prevent unsafe deserialization (tabula would pull 2.8.7)
// CVE-2022-25647: Explicit gson 2.13.2 to prevent unsafe deserialization (tabula would pull 2.8.7)
implementation 'com.google.code.gson:gson:2.13.2'
implementation 'org.apache.pdfbox:jbig2-imageio:3.0.4'
implementation 'com.opencsv:opencsv:5.12.0' // https://mvnrepository.com/artifact/com.opencsv/opencsv
@@ -100,6 +99,8 @@ dependencies {
// Batik only bridge module needed (transitively pulls anim, gvt, util, css, dom, svg-dom)
// Replaces batik-all which included unused codec, svggen, transcoder, script modules
implementation 'org.apache.xmlgraphics:batik-bridge:1.19'
// Required by TwelveMonkeys imageio-batik SPI (SVGImageReaderSpi) during ImageIO init
runtimeOnly 'org.apache.xmlgraphics:batik-transcoder:1.19'
// PDFBox Graphics2D bridge for Batik SVG to PDF conversion
implementation 'de.rototor.pdfbox:graphics2d:3.0.5'
@@ -88,10 +88,10 @@ public abstract class CreateSignatureBase implements SignatureInterface {
Certificate cert = null;
while (cert == null && aliases.hasMoreElements()) {
alias = aliases.nextElement();
setPrivateKey((PrivateKey) keystore.getKey(alias, pin));
privateKey = (PrivateKey) keystore.getKey(alias, pin);
Certificate[] certChain = keystore.getCertificateChain(alias);
if (certChain != null) {
setCertificateChain(certChain);
certificateChain = certChain;
cert = certChain[0];
if (cert instanceof X509Certificate) {
// avoid expired certificate
@@ -134,7 +134,7 @@ public class SPDFApplication {
baseUrlStatic = normalizeBackendUrl(backendUrl, serverPort);
contextPathStatic = contextPath;
serverPortStatic = serverPort;
String url = buildFullUrl(baseUrlStatic, getStaticPort(), contextPathStatic);
String url = buildFullUrl(baseUrlStatic, serverPortStatic, contextPathStatic);
// Log Tauri mode information
if (Boolean.parseBoolean(System.getProperty("STIRLING_PDF_TAURI_MODE", "false"))) {
@@ -188,7 +188,7 @@ public class SPDFApplication {
private static void printStartupLogs() {
log.info("Stirling-PDF Started.");
String url = buildFullUrl(baseUrlStatic, getStaticPort(), contextPathStatic);
String url = buildFullUrl(baseUrlStatic, serverPortStatic, contextPathStatic);
log.info("Navigate to {}", url);
}
@@ -104,7 +104,7 @@ public class EndpointInspector implements ApplicationListener<ContextRefreshedEv
String infoString = mappingInfo.toString();
if (infoString.contains("{")) {
String patternsSection =
infoString.substring(infoString.indexOf("{") + 1, infoString.indexOf("}"));
infoString.substring(infoString.indexOf('{') + 1, infoString.indexOf('}'));
for (String pattern : patternsSection.split(",")) {
pattern = pattern.trim();
@@ -24,6 +24,7 @@ import io.swagger.v3.oas.annotations.Operation;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import stirling.software.SPDF.config.EndpointConfiguration;
import stirling.software.SPDF.model.api.general.CropPdfForm;
import stirling.software.common.annotations.AutoJobPostMapping;
import stirling.software.common.annotations.api.GeneralApi;
@@ -118,6 +119,12 @@ public class CropController {
return r >= threshold && g >= threshold && b >= threshold;
}
private final EndpointConfiguration endpointConfiguration;
private boolean isGhostscriptEnabled() {
return endpointConfiguration.isGroupEnabled("Ghostscript");
}
@AutoJobPostMapping(value = "/crop", consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
@Operation(
summary = "Crops a PDF document",
@@ -137,7 +144,7 @@ public class CropController {
"Crop coordinates (x, y, width, height) are required when auto-crop is not enabled");
}
if (request.isRemoveDataOutsideCrop()) {
if (request.isRemoveDataOutsideCrop() && isGhostscriptEnabled()) {
return cropWithGhostscript(request);
} else {
return cropWithPDFBox(request);
@@ -1,7 +1,7 @@
package stirling.software.SPDF.controller.api;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.nio.file.Files;
import java.util.HashMap;
import java.util.Map;
import java.util.zip.ZipEntry;
@@ -14,10 +14,10 @@ import org.apache.pdfbox.pdmodel.PDPageContentStream;
import org.apache.pdfbox.pdmodel.common.PDRectangle;
import org.apache.pdfbox.pdmodel.graphics.form.PDFormXObject;
import org.apache.pdfbox.util.Matrix;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.ModelAttribute;
import org.springframework.web.multipart.MultipartFile;
import org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody;
import io.swagger.v3.oas.annotations.Operation;
@@ -31,6 +31,8 @@ import stirling.software.common.annotations.api.GeneralApi;
import stirling.software.common.service.CustomPDFDocumentFactory;
import stirling.software.common.util.ExceptionUtils;
import stirling.software.common.util.GeneralUtils;
import stirling.software.common.util.TempFile;
import stirling.software.common.util.TempFileManager;
import stirling.software.common.util.WebResponseUtils;
@GeneralApi
@@ -39,10 +41,9 @@ import stirling.software.common.util.WebResponseUtils;
public class PosterPdfController {
private final CustomPDFDocumentFactory pdfDocumentFactory;
private final TempFileManager tempFileManager;
@AutoJobPostMapping(
value = "/split-for-poster-print",
consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
@AutoJobPostMapping(value = "/split-for-poster-print", consumes = "multipart/form-data")
@MultiFileResponse
@Operation(
summary = "Split large PDF pages into smaller printable chunks",
@@ -51,7 +52,7 @@ public class PosterPdfController {
+ "suitable for printing on standard paper sizes (e.g., A4, Letter). "
+ "Divides each page into a grid of smaller pages using Apache PDFBox. "
+ "Input: PDF Output: ZIP-PDF Type: SISO")
public ResponseEntity<byte[]> posterPdf(@ModelAttribute PosterPdfRequest request)
public ResponseEntity<StreamingResponseBody> posterPdf(@ModelAttribute PosterPdfRequest request)
throws Exception {
log.debug("Starting PDF poster split process with request: {}", request);
@@ -60,184 +61,187 @@ public class PosterPdfController {
String filename = GeneralUtils.generateFilename(file.getOriginalFilename(), "");
log.debug("Base filename for output: {}", filename);
try (PDDocument sourceDocument = pdfDocumentFactory.load(file);
PDDocument outputDocument =
pdfDocumentFactory.createNewDocumentBasedOnOldDocument(sourceDocument);
ByteArrayOutputStream pdfOutputStream = new ByteArrayOutputStream();
ByteArrayOutputStream zipOutputStream = new ByteArrayOutputStream()) {
TempFile zipTempFile = new TempFile(tempFileManager, ".zip");
try {
try (PDDocument sourceDocument = pdfDocumentFactory.load(file);
PDDocument outputDocument =
pdfDocumentFactory.createNewDocumentBasedOnOldDocument(sourceDocument);
TempFile pdfTempFile = new TempFile(tempFileManager, ".pdf")) {
// Get target page size
PDRectangle targetPageSize = getTargetPageSize(request.getPageSize());
log.debug(
"Target page size: {} ({}x{})",
request.getPageSize(),
targetPageSize.getWidth(),
targetPageSize.getHeight());
// Get target page size
PDRectangle targetPageSize = getTargetPageSize(request.getPageSize());
log.debug(
"Target page size: {} ({}x{})",
request.getPageSize(),
targetPageSize.getWidth(),
targetPageSize.getHeight());
// Create LayerUtility for importing pages as forms
LayerUtility layerUtility = new LayerUtility(outputDocument);
// Create LayerUtility for importing pages as forms
LayerUtility layerUtility = new LayerUtility(outputDocument);
int totalPages = sourceDocument.getNumberOfPages();
int xFactor = request.getXFactor();
int yFactor = request.getYFactor();
boolean rightToLeft = request.isRightToLeft();
log.debug(
"Processing {} pages with grid {}x{}, RTL={}",
totalPages,
xFactor,
yFactor,
rightToLeft);
// Process each page
for (int pageIndex = 0; pageIndex < totalPages; pageIndex++) {
PDPage sourcePage = sourceDocument.getPage(pageIndex);
// Get both MediaBox and CropBox
PDRectangle mediaBox = sourcePage.getMediaBox();
PDRectangle cropBox = sourcePage.getCropBox();
// If no CropBox is set, use MediaBox
if (cropBox == null) {
cropBox = mediaBox;
}
// Save original boxes for restoration
PDRectangle originalMediaBox = sourcePage.getMediaBox();
PDRectangle originalCropBox = sourcePage.getCropBox();
// Normalize the page: set MediaBox to CropBox
// This ensures the form's coordinate space starts at (0, 0)
// instead of having an offset from the original MediaBox
sourcePage.setMediaBox(cropBox);
sourcePage.setCropBox(cropBox);
// Handle page rotation
int rotation = sourcePage.getRotation();
float sourceWidth = cropBox.getWidth();
float sourceHeight = cropBox.getHeight();
// Swap dimensions if rotated 90 or 270 degrees
if (rotation == 90 || rotation == 270) {
float temp = sourceWidth;
sourceWidth = sourceHeight;
sourceHeight = temp;
}
int totalPages = sourceDocument.getNumberOfPages();
int xFactor = request.getXFactor();
int yFactor = request.getYFactor();
boolean rightToLeft = request.isRightToLeft();
log.debug(
"Page {}: Normalized to CropBox dimensions {}x{}, rotation {}",
pageIndex,
sourceWidth,
sourceHeight,
rotation);
"Processing {} pages with grid {}x{}, RTL={}",
totalPages,
xFactor,
yFactor,
rightToLeft);
// Import source page as form (now with normalized coordinate space)
PDFormXObject form = layerUtility.importPageAsForm(sourceDocument, pageIndex);
// Process each page
for (int pageIndex = 0; pageIndex < totalPages; pageIndex++) {
PDPage sourcePage = sourceDocument.getPage(pageIndex);
// Restore original boxes
sourcePage.setMediaBox(originalMediaBox);
sourcePage.setCropBox(originalCropBox);
// Get both MediaBox and CropBox
PDRectangle mediaBox = sourcePage.getMediaBox();
PDRectangle cropBox = sourcePage.getCropBox();
// Calculate cell dimensions in source page coordinates
float cellWidth = sourceWidth / xFactor;
float cellHeight = sourceHeight / yFactor;
// If no CropBox is set, use MediaBox
if (cropBox == null) {
cropBox = mediaBox;
}
// Create grid cells (rows × columns)
for (int row = 0; row < yFactor; row++) {
for (int col = 0; col < xFactor; col++) {
// Apply RTL ordering for columns if enabled
int actualCol = rightToLeft ? (xFactor - 1 - col) : col;
// Save original boxes for restoration
PDRectangle originalMediaBox = sourcePage.getMediaBox();
PDRectangle originalCropBox = sourcePage.getCropBox();
// Calculate crop rectangle in source coordinates
// PDF coordinates start at bottom-left
float cropX = actualCol * cellWidth;
// For Y: invert so row 0 shows TOP (following SplitPdfBySectionsController
// pattern)
float cropY = (yFactor - 1 - row) * cellHeight;
// Normalize the page: set MediaBox to CropBox
// This ensures the form's coordinate space starts at (0, 0)
// instead of having an offset from the original MediaBox
sourcePage.setMediaBox(cropBox);
sourcePage.setCropBox(cropBox);
// Create new output page with target size
PDPage outputPage = new PDPage(targetPageSize);
outputDocument.addPage(outputPage);
// Handle page rotation
int rotation = sourcePage.getRotation();
float sourceWidth = cropBox.getWidth();
float sourceHeight = cropBox.getHeight();
try (PDPageContentStream contentStream =
new PDPageContentStream(
outputDocument,
outputPage,
PDPageContentStream.AppendMode.APPEND,
true,
true)) {
// Swap dimensions if rotated 90 or 270 degrees
if (rotation == 90 || rotation == 270) {
float temp = sourceWidth;
sourceWidth = sourceHeight;
sourceHeight = temp;
}
// Calculate uniform scale to fit cell into target page
// Scale UP if cell is smaller than target, scale DOWN if larger
float scaleX = targetPageSize.getWidth() / cellWidth;
float scaleY = targetPageSize.getHeight() / cellHeight;
float scale = Math.min(scaleX, scaleY);
log.debug(
"Page {}: Normalized to CropBox dimensions {}x{}, rotation {}",
pageIndex,
sourceWidth,
sourceHeight,
rotation);
// Center the scaled content on the target page
float scaledCellWidth = cellWidth * scale;
float scaledCellHeight = cellHeight * scale;
float offsetX = (targetPageSize.getWidth() - scaledCellWidth) / 2;
float offsetY = (targetPageSize.getHeight() - scaledCellHeight) / 2;
// Import source page as form (now with normalized coordinate space)
PDFormXObject form = layerUtility.importPageAsForm(sourceDocument, pageIndex);
// Apply transformations
contentStream.saveGraphicsState();
// Restore original boxes
sourcePage.setMediaBox(originalMediaBox);
sourcePage.setCropBox(originalCropBox);
// Translate to center position
contentStream.transform(Matrix.getTranslateInstance(offsetX, offsetY));
// Calculate cell dimensions in source page coordinates
float cellWidth = sourceWidth / xFactor;
float cellHeight = sourceHeight / yFactor;
// Scale uniformly
contentStream.transform(Matrix.getScaleInstance(scale, scale));
// Create grid cells (rows × columns)
for (int row = 0; row < yFactor; row++) {
for (int col = 0; col < xFactor; col++) {
// Apply RTL ordering for columns if enabled
int actualCol = rightToLeft ? (xFactor - 1 - col) : col;
// Translate to show only the desired grid cell
// IMPORTANT: The PDFormXObject's BBox already matches the CropBox
// (including its offset), so we only need to translate by cropX/cropY
// relative to the CropBox origin, NOT the MediaBox origin
contentStream.transform(Matrix.getTranslateInstance(-cropX, -cropY));
// Calculate crop rectangle in source coordinates
// PDF coordinates start at bottom-left
float cropX = actualCol * cellWidth;
// For Y: invert so row 0 shows TOP (following
// SplitPdfBySectionsController
// pattern)
float cropY = (yFactor - 1 - row) * cellHeight;
// Draw the form
contentStream.drawForm(form);
// Create new output page with target size
PDPage outputPage = new PDPage(targetPageSize);
outputDocument.addPage(outputPage);
contentStream.restoreGraphicsState();
try (PDPageContentStream contentStream =
new PDPageContentStream(
outputDocument,
outputPage,
PDPageContentStream.AppendMode.APPEND,
true,
true)) {
// Calculate uniform scale to fit cell into target page
// Scale UP if cell is smaller than target, scale DOWN if larger
float scaleX = targetPageSize.getWidth() / cellWidth;
float scaleY = targetPageSize.getHeight() / cellHeight;
float scale = Math.min(scaleX, scaleY);
// Center the scaled content on the target page
float scaledCellWidth = cellWidth * scale;
float scaledCellHeight = cellHeight * scale;
float offsetX = (targetPageSize.getWidth() - scaledCellWidth) / 2;
float offsetY = (targetPageSize.getHeight() - scaledCellHeight) / 2;
// Apply transformations
contentStream.saveGraphicsState();
// Translate to center position
contentStream.transform(
Matrix.getTranslateInstance(offsetX, offsetY));
// Scale uniformly
contentStream.transform(Matrix.getScaleInstance(scale, scale));
// Translate to show only the desired grid cell
// IMPORTANT: The PDFormXObject's BBox already matches the CropBox
// (including its offset), so we only need to translate by
// cropX/cropY
// relative to the CropBox origin, NOT the MediaBox origin
contentStream.transform(
Matrix.getTranslateInstance(-cropX, -cropY));
// Draw the form
contentStream.drawForm(form);
contentStream.restoreGraphicsState();
}
log.trace(
"Created output page for grid cell [{},{}] of page {}: cropX={}, cropY={}, translate=({}, {})",
row,
actualCol,
pageIndex,
cropX,
cropY,
-cropX,
-cropY);
}
log.trace(
"Created output page for grid cell [{},{}] of page {}: cropX={}, cropY={}, translate=({}, {})",
row,
actualCol,
pageIndex,
cropX,
cropY,
-cropX,
-cropY);
}
}
// Save output PDF to intermediate TempFile
outputDocument.save(pdfTempFile.getFile());
log.debug("Generated output PDF with {} pages", outputDocument.getNumberOfPages());
// Create ZIP from the PDF TempFile, streaming directly to zip TempFile
try (ZipOutputStream zipOut =
new ZipOutputStream(Files.newOutputStream(zipTempFile.getPath()))) {
zipOut.putNextEntry(new ZipEntry(filename + "_poster.pdf"));
Files.copy(pdfTempFile.getPath(), zipOut);
zipOut.closeEntry();
}
// pdfTempFile auto-closed and deleted here (end of inner try-with-resources)
}
// Save output PDF
outputDocument.save(pdfOutputStream);
byte[] pdfData = pdfOutputStream.toByteArray();
log.debug(
"Generated output PDF with {} pages ({} bytes)",
outputDocument.getNumberOfPages(),
pdfData.length);
// Create ZIP file with the result
try (ZipOutputStream zipOut = new ZipOutputStream(zipOutputStream)) {
ZipEntry zipEntry = new ZipEntry(filename + "_poster.pdf");
zipOut.putNextEntry(zipEntry);
zipOut.write(pdfData);
zipOut.closeEntry();
}
byte[] zipData = zipOutputStream.toByteArray();
log.debug("Successfully created ZIP with {} bytes", zipData.length);
return WebResponseUtils.bytesToWebResponse(
zipData, filename + "_poster.zip", MediaType.APPLICATION_OCTET_STREAM);
log.debug("Successfully created ZIP");
return WebResponseUtils.zipFileToWebResponse(zipTempFile, filename + "_poster.zip");
} catch (IOException e) {
ExceptionUtils.logException("PDF poster split process", e);
zipTempFile.close();
throw e;
} catch (Exception e) {
zipTempFile.close();
throw e;
}
}
@@ -1,6 +1,5 @@
package stirling.software.SPDF.controller.api;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.nio.file.Files;
import java.util.ArrayList;
@@ -10,11 +9,11 @@ import java.util.zip.ZipEntry;
import java.util.zip.ZipOutputStream;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.apache.pdfbox.pdmodel.PDPage;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.ModelAttribute;
import org.springframework.web.multipart.MultipartFile;
import org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody;
import io.swagger.v3.oas.annotations.Operation;
@@ -49,83 +48,60 @@ public class SplitPDFController {
+ " specified page numbers or ranges. Users can specify pages using"
+ " individual numbers, ranges, or 'all' for every page. Input:PDF"
+ " Output:PDF Type:SIMO")
public ResponseEntity<byte[]> splitPdf(@ModelAttribute PDFWithPageNums request)
public ResponseEntity<StreamingResponseBody> splitPdf(@ModelAttribute PDFWithPageNums request)
throws IOException {
MultipartFile file = request.getFileInput();
try (TempFile outputTempFile = new TempFile(tempFileManager, ".zip");
PDDocument document = pdfDocumentFactory.load(file)) {
List<ByteArrayOutputStream> splitDocumentsBoas = new ArrayList<>();
int totalPages = document.getNumberOfPages();
List<Integer> pageNumbers = request.getPageNumbersList(document, false);
if (!pageNumbers.contains(totalPages - 1)) {
// Create a mutable ArrayList so we can add to it
pageNumbers = new ArrayList<>(pageNumbers);
pageNumbers.add(totalPages - 1);
}
log.debug(
"Splitting PDF into pages: {}",
pageNumbers.stream().map(String::valueOf).collect(Collectors.joining(",")));
splitDocumentsBoas = new ArrayList<>(pageNumbers.size());
int previousPageNumber = 0;
for (int splitPoint : pageNumbers) {
try (PDDocument splitDocument =
pdfDocumentFactory.createNewDocumentBasedOnOldDocument(document);
ByteArrayOutputStream baos = new ByteArrayOutputStream()) {
for (int i = previousPageNumber; i <= splitPoint; i++) {
PDPage page = document.getPage(i);
splitDocument.addPage(page);
log.debug("Adding page {} to split document", i);
}
previousPageNumber = splitPoint + 1;
// Transfer metadata to split pdf
// PdfMetadataService.setMetadataToPdf(splitDocument, metadata);
splitDocument.save(baos);
splitDocumentsBoas.add(baos);
} catch (Exception e) {
ExceptionUtils.logException("document splitting and saving", e);
throw e;
TempFile outputTempFile = new TempFile(tempFileManager, ".zip");
try {
try (PDDocument document = pdfDocumentFactory.load(file)) {
int totalPages = document.getNumberOfPages();
List<Integer> pageNumbers = request.getPageNumbersList(document, false);
if (!pageNumbers.contains(totalPages - 1)) {
pageNumbers = new ArrayList<>(pageNumbers);
pageNumbers.add(totalPages - 1);
}
}
String baseFilename = GeneralUtils.removeExtension(file.getOriginalFilename());
log.debug(
"Splitting PDF into pages: {}",
pageNumbers.stream().map(String::valueOf).collect(Collectors.joining(",")));
try (ZipOutputStream zipOut =
new ZipOutputStream(Files.newOutputStream(outputTempFile.getPath()))) {
int splitDocumentsSize = splitDocumentsBoas.size();
for (int i = 0; i < splitDocumentsSize; i++) {
StringBuilder sb = new StringBuilder(baseFilename.length() + 10);
sb.append(baseFilename).append('_').append(i + 1).append(".pdf");
String fileName = sb.toString();
String baseFilename = GeneralUtils.removeExtension(file.getOriginalFilename());
try (ZipOutputStream zipOut =
new ZipOutputStream(Files.newOutputStream(outputTempFile.getPath()))) {
int previousPageNumber = 0;
for (int splitIndex = 0; splitIndex < pageNumbers.size(); splitIndex++) {
int splitPoint = pageNumbers.get(splitIndex);
try (PDDocument splitDocument =
pdfDocumentFactory.createNewDocumentBasedOnOldDocument(document)) {
for (int i = previousPageNumber; i <= splitPoint; i++) {
splitDocument.addPage(document.getPage(i));
log.debug("Adding page {} to split document", i);
}
previousPageNumber = splitPoint + 1;
ByteArrayOutputStream baos = splitDocumentsBoas.get(i);
byte[] pdf = baos.toByteArray();
ZipEntry pdfEntry = new ZipEntry(fileName);
zipOut.putNextEntry(pdfEntry);
zipOut.write(pdf);
zipOut.closeEntry();
log.debug("Wrote split document {} to zip file", fileName);
String fileName = baseFilename + "_" + (splitIndex + 1) + ".pdf";
zipOut.putNextEntry(new ZipEntry(fileName));
splitDocument.save(zipOut);
zipOut.closeEntry();
log.debug("Wrote split document {} to zip file", fileName);
} catch (Exception e) {
ExceptionUtils.logException("document splitting and saving", e);
throw e;
}
}
}
}
log.debug(
"Successfully created zip file with split documents: {}",
outputTempFile.getPath().toString());
byte[] data = Files.readAllBytes(outputTempFile.getPath());
String zipFilename =
GeneralUtils.generateFilename(file.getOriginalFilename(), "_split.zip");
return WebResponseUtils.bytesToWebResponse(
data, zipFilename, MediaType.APPLICATION_OCTET_STREAM);
return WebResponseUtils.zipFileToWebResponse(outputTempFile, zipFilename);
} catch (Exception e) {
outputTempFile.close();
throw e;
}
}
}
@@ -1,8 +1,6 @@
package stirling.software.SPDF.controller.api;
import java.io.ByteArrayOutputStream;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.ArrayList;
import java.util.List;
import java.util.Locale;
@@ -17,6 +15,7 @@ import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.ModelAttribute;
import org.springframework.web.multipart.MultipartFile;
import org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody;
import io.swagger.v3.oas.annotations.Operation;
@@ -36,6 +35,8 @@ import stirling.software.common.service.CustomPDFDocumentFactory;
import stirling.software.common.service.PdfMetadataService;
import stirling.software.common.util.ExceptionUtils;
import stirling.software.common.util.GeneralUtils;
import stirling.software.common.util.TempFile;
import stirling.software.common.util.TempFileManager;
import stirling.software.common.util.WebResponseUtils;
@GeneralApi
@@ -47,6 +48,8 @@ public class SplitPdfByChaptersController {
private final CustomPDFDocumentFactory pdfDocumentFactory;
private final TempFileManager tempFileManager;
private static List<Bookmark> extractOutlineItems(
PDDocument sourceDocument,
PDOutlineItem current,
@@ -122,22 +125,19 @@ public class SplitPdfByChaptersController {
@Operation(
summary = "Split PDFs by Chapters",
description = "Splits a PDF into chapters and returns a ZIP file.")
public ResponseEntity<byte[]> splitPdf(@ModelAttribute SplitPdfByChaptersRequest request)
throws Exception {
public ResponseEntity<StreamingResponseBody> splitPdf(
@ModelAttribute SplitPdfByChaptersRequest request) throws Exception {
MultipartFile file = request.getFileInput();
PDDocument sourceDocument = null;
Path zipFile = null;
try {
boolean includeMetadata = Boolean.TRUE.equals(request.getIncludeMetadata());
Integer bookmarkLevel =
request.getBookmarkLevel(); // levels start from 0 (top most bookmarks)
if (bookmarkLevel < 0) {
throw ExceptionUtils.createIllegalArgumentException(
"error.invalidArgument", "Invalid argument: {0}", "bookmark level");
}
sourceDocument = pdfDocumentFactory.load(file);
boolean includeMetadata = Boolean.TRUE.equals(request.getIncludeMetadata());
Integer bookmarkLevel =
request.getBookmarkLevel(); // levels start from 0 (top most bookmarks)
if (bookmarkLevel < 0) {
throw ExceptionUtils.createIllegalArgumentException(
"error.invalidArgument", "Invalid argument: {0}", "bookmark level");
}
try (PDDocument sourceDocument = pdfDocumentFactory.load(file)) {
PDDocumentOutline outline = sourceDocument.getDocumentCatalog().getDocumentOutline();
if (outline == null) {
@@ -157,12 +157,10 @@ public class SplitPdfByChaptersController {
bookmarkLevel);
// to handle last page edge case
bookmarks.get(bookmarks.size() - 1).setEndPage(sourceDocument.getNumberOfPages());
Bookmark lastBookmark = bookmarks.get(bookmarks.size() - 1);
} catch (Exception e) {
ExceptionUtils.logException("outline extraction", e);
return ResponseEntity.internalServerError()
.body("Unable to extract outline items".getBytes());
throw e;
}
boolean allowDuplicates = Boolean.TRUE.equals(request.getAllowDuplicates());
@@ -181,29 +179,10 @@ public class SplitPdfByChaptersController {
bookmark.getStartPage(),
bookmark.getEndPage());
}
List<ByteArrayOutputStream> splitDocumentsBoas =
getSplitDocumentsBoas(sourceDocument, bookmarks, includeMetadata);
zipFile = createZipFile(bookmarks, splitDocumentsBoas);
byte[] data = Files.readAllBytes(zipFile);
Files.deleteIfExists(zipFile);
TempFile zipTempFile = createZipFile(sourceDocument, bookmarks, includeMetadata);
String filename = GeneralUtils.generateFilename(file.getOriginalFilename(), "");
sourceDocument.close();
return WebResponseUtils.bytesToWebResponse(
data, filename + ".zip", MediaType.APPLICATION_OCTET_STREAM);
} finally {
try {
if (sourceDocument != null) {
sourceDocument.close();
}
if (zipFile != null) {
Files.deleteIfExists(zipFile);
}
} catch (Exception e) {
log.error("Error while cleaning up resources", e);
}
return WebResponseUtils.zipFileToWebResponse(zipTempFile, filename + ".zip");
}
}
@@ -232,72 +211,55 @@ public class SplitPdfByChaptersController {
return bookmarks;
}
private Path createZipFile(
List<Bookmark> bookmarks, List<ByteArrayOutputStream> splitDocumentsBoas)
throws Exception {
Path zipFile = Files.createTempFile("split_documents", ".zip");
String fileNumberFormatter = "%0" + (Integer.toString(bookmarks.size()).length()) + "d ";
try (ZipOutputStream zipOut = new ZipOutputStream(Files.newOutputStream(zipFile))) {
for (int i = 0; i < splitDocumentsBoas.size(); i++) {
// split files will be named as "[FILE_NUMBER] [BOOKMARK_TITLE].pdf"
String fileName =
String.format(Locale.ROOT, fileNumberFormatter, i)
+ bookmarks.get(i).getTitle()
+ ".pdf";
ByteArrayOutputStream baos = splitDocumentsBoas.get(i);
byte[] pdf = baos.toByteArray();
ZipEntry pdfEntry = new ZipEntry(fileName);
zipOut.putNextEntry(pdfEntry);
zipOut.write(pdf);
zipOut.closeEntry();
log.debug("Wrote split document {} to zip file", fileName);
}
} catch (Exception e) {
log.error("Failed writing to zip", e);
throw e;
}
log.info("Successfully created zip file with split documents: {}", zipFile);
return zipFile;
}
public List<ByteArrayOutputStream> getSplitDocumentsBoas(
private TempFile createZipFile(
PDDocument sourceDocument, List<Bookmark> bookmarks, boolean includeMetadata)
throws Exception {
List<ByteArrayOutputStream> splitDocumentsBoas = new ArrayList<>();
PdfMetadata metadata = null;
if (includeMetadata) {
metadata = pdfMetadataService.extractMetadataFromPdf(sourceDocument);
}
for (Bookmark bookmark : bookmarks) {
try (PDDocument splitDocument = new PDDocument()) {
boolean isSinglePage = (bookmark.getStartPage() == bookmark.getEndPage());
PdfMetadata metadata =
includeMetadata ? pdfMetadataService.extractMetadataFromPdf(sourceDocument) : null;
String fileNumberFormatter = "%0" + (Integer.toString(bookmarks.size()).length()) + "d ";
TempFile zipTempFile = new TempFile(tempFileManager, ".zip");
try {
try (ZipOutputStream zipOut =
new ZipOutputStream(Files.newOutputStream(zipTempFile.getPath()))) {
for (int i = 0; i < bookmarks.size(); i++) {
Bookmark bookmark = bookmarks.get(i);
try (PDDocument splitDocument = new PDDocument()) {
boolean isSinglePage = (bookmark.getStartPage() == bookmark.getEndPage());
for (int i = bookmark.getStartPage();
i < bookmark.getEndPage() + (isSinglePage ? 1 : 0);
i++) {
PDPage page = sourceDocument.getPage(i);
splitDocument.addPage(page);
log.debug("Adding page {} to split document", i);
for (int pg = bookmark.getStartPage();
pg < bookmark.getEndPage() + (isSinglePage ? 1 : 0);
pg++) {
PDPage page = sourceDocument.getPage(pg);
splitDocument.addPage(page);
log.debug("Adding page {} to split document", pg);
}
if (includeMetadata) {
pdfMetadataService.setMetadataToPdf(splitDocument, metadata);
}
// split files will be named as "[FILE_NUMBER] [BOOKMARK_TITLE].pdf"
String fileName =
String.format(Locale.ROOT, fileNumberFormatter, i)
+ bookmark.getTitle()
+ ".pdf";
zipOut.putNextEntry(new ZipEntry(fileName));
splitDocument.save(zipOut);
zipOut.closeEntry();
log.debug("Wrote split document {} to zip file", fileName);
} catch (Exception e) {
ExceptionUtils.logException("document splitting and saving", e);
throw e;
}
}
ByteArrayOutputStream baos = new ByteArrayOutputStream();
if (includeMetadata) {
pdfMetadataService.setMetadataToPdf(splitDocument, metadata);
}
splitDocument.save(baos);
splitDocumentsBoas.add(baos);
} catch (Exception e) {
ExceptionUtils.logException("document splitting and saving", e);
throw e;
}
log.info(
"Successfully created zip file with split documents: {}",
zipTempFile.getPath());
return zipTempFile;
} catch (Exception e) {
zipTempFile.close();
throw e;
}
return splitDocumentsBoas;
}
}
@@ -3,7 +3,6 @@ package stirling.software.SPDF.controller.api;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.zip.ZipEntry;
import java.util.zip.ZipOutputStream;
@@ -13,9 +12,11 @@ import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.ModelAttribute;
import org.springframework.web.multipart.MultipartFile;
import org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody;
import io.swagger.v3.oas.annotations.Operation;
import lombok.Getter;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
@@ -50,8 +51,8 @@ public class SplitPdfBySizeController {
+ " if 10MB and each page is 1MB and you enter 2MB then 5 docs each 2MB"
+ " (rounded so that it accepts 1.9MB but not 2.1MB) Input:PDF"
+ " Output:ZIP-PDF Type:SISO")
public ResponseEntity<byte[]> autoSplitPdf(@ModelAttribute SplitPdfBySizeOrCountRequest request)
throws Exception {
public ResponseEntity<StreamingResponseBody> autoSplitPdf(
@ModelAttribute SplitPdfBySizeOrCountRequest request) throws Exception {
log.debug("Starting PDF split process with request: {}", request);
MultipartFile file = request.getFileInput();
@@ -59,63 +60,51 @@ public class SplitPdfBySizeController {
String filename = GeneralUtils.generateFilename(file.getOriginalFilename(), "");
log.debug("Base filename for output: {}", filename);
try (TempFile zipTempFile = new TempFile(tempFileManager, ".zip")) {
Path managedZipPath = zipTempFile.getPath();
log.debug("Created temporary managed zip file: {}", managedZipPath);
try {
log.debug("Reading input file bytes");
byte[] pdfBytes = file.getBytes();
log.debug("Successfully read {} bytes from input file", pdfBytes.length);
TempFile zipTempFile = new TempFile(tempFileManager, ".zip");
try {
log.debug("Created temporary managed zip file: {}", zipTempFile.getPath());
log.debug("Creating ZIP output stream");
try (ZipOutputStream zipOut =
new ZipOutputStream(Files.newOutputStream(zipTempFile.getPath()));
PDDocument sourceDocument = pdfDocumentFactory.load(file)) {
log.debug(
"Successfully loaded PDF with {} pages", sourceDocument.getNumberOfPages());
log.debug("Creating ZIP output stream");
try (ZipOutputStream zipOut =
new ZipOutputStream(Files.newOutputStream(managedZipPath))) {
log.debug("Loading PDF document");
try (PDDocument sourceDocument = pdfDocumentFactory.load(pdfBytes)) {
log.debug(
"Successfully loaded PDF with {} pages",
sourceDocument.getNumberOfPages());
int type = request.getSplitType();
String value = request.getSplitValue();
log.debug("Split type: {}, Split value: {}", type, value);
int type = request.getSplitType();
String value = request.getSplitValue();
log.debug("Split type: {}, Split value: {}", type, value);
if (type == 0) {
log.debug("Processing split by size");
long maxBytes = GeneralUtils.convertSizeToBytes(value);
log.debug("Max bytes per document: {}", maxBytes);
handleSplitBySize(sourceDocument, maxBytes, zipOut, filename);
} else if (type == 1) {
log.debug("Processing split by page count");
int pageCount = Integer.parseInt(value);
log.debug("Pages per document: {}", pageCount);
handleSplitByPageCount(sourceDocument, pageCount, zipOut, filename);
} else if (type == 2) {
log.debug("Processing split by document count");
int documentCount = Integer.parseInt(value);
log.debug("Total number of documents: {}", documentCount);
handleSplitByDocCount(sourceDocument, documentCount, zipOut, filename);
} else {
log.error("Invalid split type: {}", type);
throw ExceptionUtils.createIllegalArgumentException(
"error.invalidArgument",
"Invalid argument: {0}",
"split type: " + type);
}
log.debug("PDF splitting completed successfully");
}
if (type == 0) {
log.debug("Processing split by size");
long maxBytes = GeneralUtils.convertSizeToBytes(value);
log.debug("Max bytes per document: {}", maxBytes);
handleSplitBySize(sourceDocument, maxBytes, zipOut, filename);
} else if (type == 1) {
log.debug("Processing split by page count");
int pageCount = Integer.parseInt(value);
log.debug("Pages per document: {}", pageCount);
handleSplitByPageCount(sourceDocument, pageCount, zipOut, filename);
} else if (type == 2) {
log.debug("Processing split by document count");
int documentCount = Integer.parseInt(value);
log.debug("Total number of documents: {}", documentCount);
handleSplitByDocCount(sourceDocument, documentCount, zipOut, filename);
} else {
log.error("Invalid split type: {}", type);
throw ExceptionUtils.createIllegalArgumentException(
"error.invalidArgument",
"Invalid argument: {0}",
"split type: " + type);
}
byte[] data = Files.readAllBytes(managedZipPath);
log.debug("Successfully read {} bytes from ZIP file", data.length);
log.debug("Returning response with {} bytes of data", data.length);
return WebResponseUtils.bytesToWebResponse(
data, filename + ".zip", MediaType.APPLICATION_OCTET_STREAM);
} catch (Exception e) {
ExceptionUtils.logException("PDF splitting process", e);
throw e; // Re-throw to ensure proper error response
log.debug("PDF splitting completed successfully");
}
log.debug("Returning streaming response for zip file");
return WebResponseUtils.zipFileToWebResponse(zipTempFile, filename + ".zip");
} catch (Exception e) {
ExceptionUtils.logException("PDF splitting process", e);
zipTempFile.close();
throw e;
}
}
@@ -124,6 +113,7 @@ public class SplitPdfBySizeController {
throws IOException {
log.debug("Starting handleSplitBySize with maxBytes={}", maxBytes);
@Getter
class DocHolder implements AutoCloseable {
private PDDocument doc;
@@ -131,10 +121,6 @@ public class SplitPdfBySizeController {
this.doc = doc;
}
public PDDocument getDoc() {
return doc;
}
public void setDoc(PDDocument doc) {
if (this.doc != null) {
try {
@@ -22,6 +22,7 @@ import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.ModelAttribute;
import org.springframework.web.multipart.MultipartFile;
import org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody;
import io.swagger.v3.oas.annotations.Operation;
@@ -51,6 +52,7 @@ import stirling.software.common.util.PdfUtils;
import stirling.software.common.util.ProcessExecutor;
import stirling.software.common.util.ProcessExecutor.ProcessExecutorResult;
import stirling.software.common.util.RegexPatternUtils;
import stirling.software.common.util.TempFile;
import stirling.software.common.util.TempFileManager;
import stirling.software.common.util.WebResponseUtils;
@@ -78,7 +80,7 @@ public class ConvertImgPDFController {
"This endpoint converts a PDF file to image(s) with the specified image format,"
+ " color type, and DPI. Users can choose to get a single image or multiple"
+ " images. Input:PDF Output:Image Type:SI-Conditional")
public ResponseEntity<byte[]> convertToImage(@ModelAttribute ConvertToImageRequest request)
public ResponseEntity<?> convertToImage(@ModelAttribute ConvertToImageRequest request)
throws Exception {
MultipartFile file = request.getFileInput();
String imageFormat = request.getImageFormat();
@@ -180,28 +182,33 @@ public class ConvertImgPDFController {
"No WebP files were created. " + resultProcess.getMessages());
}
byte[] bodyBytes = new byte[0];
if (webpFiles.size() == 1) {
// Return the single WebP file directly
Path webpFilePath = webpFiles.get(0);
bodyBytes = Files.readAllBytes(webpFilePath);
byte[] webpBytes = Files.readAllBytes(webpFilePath);
Files.deleteIfExists(tempFile);
tempFile = null;
FileUtils.deleteDirectory(tempOutputDir.toFile());
tempOutputDir = null;
String docName = filename + "." + imageFormat;
MediaType mediaType = MediaType.parseMediaType(getMediaType(imageFormat));
return WebResponseUtils.bytesToWebResponse(webpBytes, docName, mediaType);
} else {
// Create a ZIP file containing all WebP images
try (ByteArrayOutputStream zipOutputStream = new ByteArrayOutputStream();
ZipOutputStream zos = new ZipOutputStream(zipOutputStream)) {
ByteArrayOutputStream zipBAOS = new ByteArrayOutputStream();
try (ZipOutputStream zos = new ZipOutputStream(zipBAOS)) {
for (Path webpFile : webpFiles) {
zos.putNextEntry(new ZipEntry(webpFile.getFileName().toString()));
Files.copy(webpFile, zos);
zos.closeEntry();
}
bodyBytes = zipOutputStream.toByteArray();
}
Files.deleteIfExists(tempFile);
tempFile = null;
FileUtils.deleteDirectory(tempOutputDir.toFile());
tempOutputDir = null;
String zipFilename = filename + "_convertedToImages.zip";
return WebResponseUtils.bytesToWebResponse(
zipBAOS.toByteArray(), zipFilename, MediaType.APPLICATION_OCTET_STREAM);
}
// Clean up the temporary files
Files.deleteIfExists(tempFile);
if (tempOutputDir != null) FileUtils.deleteDirectory(tempOutputDir.toFile());
result = bodyBytes;
}
if (singleImage) {
@@ -267,8 +274,8 @@ public class ConvertImgPDFController {
description =
"This endpoint converts a CBZ (ZIP) comic book archive to a PDF file. "
+ "Input:CBZ Output:PDF Type:SISO")
public ResponseEntity<?> convertCbzToPdf(@ModelAttribute ConvertCbzToPdfRequest request)
throws IOException {
public ResponseEntity<StreamingResponseBody> convertCbzToPdf(
@ModelAttribute ConvertCbzToPdfRequest request) throws IOException {
MultipartFile file = request.getFileInput();
boolean optimizeForEbook = request.isOptimizeForEbook();
@@ -278,13 +285,13 @@ public class ConvertImgPDFController {
optimizeForEbook = false;
}
byte[] pdfBytes =
TempFile pdfFile =
CbzUtils.convertCbzToPdf(
file, pdfDocumentFactory, tempFileManager, optimizeForEbook);
String filename = createConvertedFilename(file.getOriginalFilename(), "_converted.pdf");
return WebResponseUtils.bytesToWebResponse(pdfBytes, filename);
return WebResponseUtils.pdfFileToWebResponse(pdfFile, filename);
}
@AutoJobPostMapping(consumes = MediaType.MULTIPART_FORM_DATA_VALUE, value = "/pdf/cbz")
@@ -293,8 +300,8 @@ public class ConvertImgPDFController {
description =
"This endpoint converts a PDF file to a CBZ (ZIP) comic book archive. "
+ "Input:PDF Output:CBZ Type:SISO")
public ResponseEntity<?> convertPdfToCbz(@ModelAttribute ConvertPdfToCbzRequest request)
throws IOException {
public ResponseEntity<StreamingResponseBody> convertPdfToCbz(
@ModelAttribute ConvertPdfToCbzRequest request) throws IOException {
MultipartFile file = request.getFileInput();
int dpi = request.getDpi();
@@ -302,12 +309,12 @@ public class ConvertImgPDFController {
dpi = 300;
}
byte[] cbzBytes = PdfToCbzUtils.convertPdfToCbz(file, dpi, pdfDocumentFactory);
TempFile cbzFile =
PdfToCbzUtils.convertPdfToCbz(file, dpi, pdfDocumentFactory, tempFileManager);
String filename = createConvertedFilename(file.getOriginalFilename(), "_converted.cbz");
return WebResponseUtils.bytesToWebResponse(
cbzBytes, filename, MediaType.APPLICATION_OCTET_STREAM);
return WebResponseUtils.zipFileToWebResponse(cbzFile, filename);
}
@AutoJobPostMapping(consumes = MediaType.MULTIPART_FORM_DATA_VALUE, value = "/cbr/pdf")
@@ -54,6 +54,12 @@ public class ConvertPDFToEpubController {
command.add(inputPath.toString());
command.add(outputPath.toString());
// Use pdftohtml engine (poppler) for PDF input instead of calibre's Qt-based engine.
// This avoids the Qt WebEngine dependency for PDF parsing and uses the lighter
// poppler-utils pdftohtml binary which is already available in the container.
command.add("--pdf-engine");
command.add("pdftohtml");
// Golden defaults
command.add("--enable-heuristics");
command.add("--insert-blank-line");
@@ -39,6 +39,8 @@ import stirling.software.common.util.WebResponseUtils;
public class ConvertPdfJsonController {
private static final Pattern FILE_EXTENSION_PATTERN = Pattern.compile("[.][^.]+$");
private static final Pattern WHITESPACE_PATTERN = Pattern.compile("[\\r\\n\\t]+");
private static final Pattern NON_PRINTABLE_PATTERN = Pattern.compile("[^\\x20-\\x7E]");
private final PdfJsonConversionService pdfJsonConversionService;
@Autowired(required = false)
@@ -259,7 +261,10 @@ public class ConvertPdfJsonController {
if (length > 0) {
int start = Math.max(0, length - 64);
tail = new String(jsonBytes, start, length - start, StandardCharsets.UTF_8);
tail = tail.replaceAll("[\\r\\n\\t]+", " ").replaceAll("[^\\x20-\\x7E]", "?");
tail =
NON_PRINTABLE_PATTERN
.matcher(WHITESPACE_PATTERN.matcher(tail).replaceAll(" "))
.replaceAll("?");
}
log.debug(
"Returning {} JSON response ({} bytes, endsWithJson={}, tail='{}')",
@@ -421,9 +426,9 @@ public class ConvertPdfJsonController {
private String truncateForLog(String value) {
int max = 64;
if (value.length() <= max) {
return value.replaceAll("[\\r\\n\\t]+", " ");
return WHITESPACE_PATTERN.matcher(value).replaceAll(" ");
}
return value.substring(0, max).replaceAll("[\\r\\n\\t]+", " ") + "...";
return WHITESPACE_PATTERN.matcher(value.substring(0, max)).replaceAll(" ") + "...";
}
/**
@@ -159,6 +159,24 @@ public class ConfigController {
"defaultHideUnavailableConversions",
applicationProperties.getUi().isDefaultHideUnavailableConversions());
// Hide disabled tools settings
configData.put(
"hideDisabledToolsGoogleDrive",
applicationProperties.getUi().getHideDisabledTools().isGoogleDrive());
configData.put(
"hideDisabledToolsMobileQRScanner",
applicationProperties.getUi().getHideDisabledTools().isMobileQRScanner());
// Google Drive backend settings (only if enabled)
ApplicationProperties.Premium.ProFeatures.GoogleDrive googleDrive =
applicationProperties.getPremium().getProFeatures().getGoogleDrive();
if (googleDrive.isEnabled()) {
configData.put("googleDriveEnabled", true);
configData.put("googleDriveClientId", googleDrive.getClientId());
configData.put("googleDriveApiKey", googleDrive.getApiKey());
configData.put("googleDriveAppId", googleDrive.getAppId());
}
// Security settings
// enableLogin requires both the config flag AND proprietary features to be loaded
// If userService is null, proprietary module isn't loaded
@@ -67,7 +67,7 @@ public class ExtractImageScansController {
MultipartFile inputFile = request.getFileInput();
String fileName = inputFile.getOriginalFilename();
String extension = fileName.substring(fileName.lastIndexOf(".") + 1);
String extension = fileName.substring(fileName.lastIndexOf('.') + 1);
List<String> images = new ArrayList<>();
@@ -5,6 +5,7 @@ import java.awt.image.BufferedImage;
import java.awt.image.RenderedImage;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.nio.file.Files;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import java.util.Arrays;
@@ -28,6 +29,7 @@ import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.ModelAttribute;
import org.springframework.web.multipart.MultipartFile;
import org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody;
import io.swagger.v3.oas.annotations.Operation;
@@ -42,6 +44,8 @@ import stirling.software.common.service.CustomPDFDocumentFactory;
import stirling.software.common.util.ExceptionUtils;
import stirling.software.common.util.GeneralUtils;
import stirling.software.common.util.ImageProcessingUtils;
import stirling.software.common.util.TempFile;
import stirling.software.common.util.TempFileManager;
import stirling.software.common.util.WebResponseUtils;
@MiscApi
@@ -50,6 +54,7 @@ import stirling.software.common.util.WebResponseUtils;
public class ExtractImagesController {
private final CustomPDFDocumentFactory pdfDocumentFactory;
private final TempFileManager tempFileManager;
@AutoJobPostMapping(consumes = MediaType.MULTIPART_FORM_DATA_VALUE, value = "/extract-images")
@MultiFileResponse
@@ -59,111 +64,115 @@ public class ExtractImagesController {
"This endpoint extracts images from a given PDF file and returns them in a zip"
+ " file. Users can specify the output image format. Input:PDF"
+ " Output:IMAGE/ZIP Type:SIMO")
public ResponseEntity<byte[]> extractImages(@ModelAttribute PDFExtractImagesRequest request)
public ResponseEntity<StreamingResponseBody> extractImages(
@ModelAttribute PDFExtractImagesRequest request)
throws IOException, InterruptedException, ExecutionException {
MultipartFile file = request.getFileInput();
String format = request.getFormat();
boolean allowDuplicates = Boolean.TRUE.equals(request.getAllowDuplicates());
PDDocument document = pdfDocumentFactory.load(file);
// Determine if multithreading should be used based on PDF size or number of pages
boolean useMultithreading = shouldUseMultithreading(file, document);
// Create ByteArrayOutputStream to write zip file to byte array
ByteArrayOutputStream baos = new ByteArrayOutputStream();
// Create ZipOutputStream to create zip file
ZipOutputStream zos = new ZipOutputStream(baos);
// Set compression level
zos.setLevel(Deflater.BEST_COMPRESSION);
String filename = GeneralUtils.removeExtension(file.getOriginalFilename());
Set<byte[]> processedImages = new HashSet<>();
if (useMultithreading) {
// Virtual thread executor lightweight threads ideal for I/O-bound image extraction
ExecutorService executor = Executors.newVirtualThreadPerTaskExecutor();
Set<Future<Void>> futures = new HashSet<>();
TempFile zipTempFile = new TempFile(tempFileManager, ".zip");
try (ZipOutputStream zos =
new ZipOutputStream(Files.newOutputStream(zipTempFile.getPath()));
PDDocument document = pdfDocumentFactory.load(file)) {
// Safely iterate over each page, handling corrupt PDFs where page count might be wrong
try {
int pageCount = document.getPages().getCount();
log.debug("Document reports {} pages", pageCount);
// Set compression level
zos.setLevel(Deflater.BEST_COMPRESSION);
int consecutiveFailures = 0;
// Determine if multithreading should be used based on PDF size or number of pages
boolean useMultithreading = shouldUseMultithreading(file, document);
for (int pgNum = 0; pgNum < pageCount; pgNum++) {
try {
PDPage page = document.getPage(pgNum);
consecutiveFailures = 0; // Reset on success
final int currentPageNum = pgNum + 1; // Convert to 1-based page numbering
Future<Void> future =
executor.submit(
() -> {
try {
// Call the image extraction method for each page
extractImagesFromPage(
page,
format,
filename,
currentPageNum,
processedImages,
zos,
allowDuplicates);
} catch (Exception e) {
// Log the error and continue processing other pages
ExceptionUtils.logException(
"image extraction from page "
+ currentPageNum,
e);
}
if (useMultithreading) {
ExecutorService executor = Executors.newVirtualThreadPerTaskExecutor();
Set<Future<Void>> futures = new HashSet<>();
return null; // Callable requires a return type
});
try {
int pageCount = document.getPages().getCount();
log.debug("Document reports {} pages", pageCount);
// Add the Future object to the list to track completion
futures.add(future);
} catch (Exception e) {
consecutiveFailures++;
ExceptionUtils.logException("page access for page " + (pgNum + 1), e);
int consecutiveFailures = 0;
if (consecutiveFailures >= 3) {
log.warn("Stopping page iteration after 3 consecutive failures");
break;
for (int pgNum = 0; pgNum < pageCount; pgNum++) {
try {
PDPage page = document.getPage(pgNum);
consecutiveFailures = 0; // Reset on success
final int currentPageNum =
pgNum + 1; // Convert to 1-based page numbering
Future<Void> future =
executor.submit(
() -> {
try {
// Call the image extraction method for each
// page
extractImagesFromPage(
page,
format,
filename,
currentPageNum,
processedImages,
zos,
allowDuplicates);
} catch (Exception e) {
// Log the error and continue processing other
// pages
ExceptionUtils.logException(
"image extraction from page "
+ currentPageNum,
e);
}
return null; // Callable requires a return type
});
// Add the Future object to the list to track completion
futures.add(future);
} catch (Exception e) {
consecutiveFailures++;
ExceptionUtils.logException("page access for page " + (pgNum + 1), e);
if (consecutiveFailures >= 3) {
log.warn("Stopping page iteration after 3 consecutive failures");
break;
}
}
}
} catch (Exception e) {
ExceptionUtils.logException("page count determination", e);
throw e;
}
} catch (Exception e) {
ExceptionUtils.logException("page count determination", e);
throw e;
}
// Wait for all tasks to complete
for (Future<Void> future : futures) {
future.get();
}
// Wait for all tasks to complete
for (Future<Void> future : futures) {
future.get();
}
// Close executor service
executor.shutdown();
} else {
// Single-threaded extraction
for (int pgNum = 0; pgNum < document.getPages().getCount(); pgNum++) {
PDPage page = document.getPage(pgNum);
extractImagesFromPage(
page, format, filename, pgNum + 1, processedImages, zos, allowDuplicates);
// Close executor service
executor.shutdown();
} else {
// Single-threaded extraction
for (int pgNum = 0; pgNum < document.getPages().getCount(); pgNum++) {
PDPage page = document.getPage(pgNum);
extractImagesFromPage(
page,
format,
filename,
pgNum + 1,
processedImages,
zos,
allowDuplicates);
}
}
// document and zos closed by try-with-resources
} catch (Exception e) {
zipTempFile.close();
throw e;
}
// Close PDDocument and ZipOutputStream
document.close();
zos.close();
// Create ByteArrayResource from byte array
byte[] zipContents = baos.toByteArray();
return WebResponseUtils.baosToWebResponse(
baos, filename + "_extracted-images.zip", MediaType.APPLICATION_OCTET_STREAM);
return WebResponseUtils.zipFileToWebResponse(
zipTempFile, filename + "_extracted-images.zip");
}
private boolean shouldUseMultithreading(MultipartFile file, PDDocument document) {
@@ -214,13 +223,17 @@ public class ExtractImagesController {
// Convert to standard RGB colorspace if needed
BufferedImage bufferedImage = convertToRGB(renderedImage, format);
// Write image to zip file
// Encode image outside the lock to allow parallel encoding across threads
String imageName = filename + "_page_" + pageNum + "_" + count++ + "." + format;
ByteArrayOutputStream imageBaos = new ByteArrayOutputStream();
ImageIO.write(bufferedImage, format, imageBaos);
byte[] imageData = imageBaos.toByteArray();
// Write encoded bytes to zip under lock (ZipOutputStream requires
// serialization)
synchronized (zos) {
zos.putNextEntry(new ZipEntry(imageName));
ByteArrayOutputStream imageBaos = new ByteArrayOutputStream();
ImageIO.write(bufferedImage, format, imageBaos);
zos.write(imageBaos.toByteArray());
zos.write(imageData);
zos.closeEntry();
}
}
@@ -94,7 +94,7 @@ public class OCRController {
throws IOException, InterruptedException {
MultipartFile inputFile = request.getFileInput();
List<String> selectedLanguages = request.getLanguages();
Boolean sidecar = request.isSidecar();
boolean sidecar = request.isSidecar();
Boolean deskew = request.isDeskew();
Boolean clean = request.isClean();
Boolean cleanFinal = request.isCleanFinal();
@@ -233,7 +233,7 @@ public class StampController {
};
ClassPathResource classPathResource = new ClassPathResource(resourceDir);
String fileExtension = resourceDir.substring(resourceDir.lastIndexOf("."));
String fileExtension = resourceDir.substring(resourceDir.lastIndexOf('.'));
// Use TempFile with try-with-resources for automatic cleanup
try (TempFile tempFileWrapper = new TempFile(tempFileManager, fileExtension)) {
@@ -1,7 +1,7 @@
package stirling.software.SPDF.controller.api.pipeline;
import java.io.ByteArrayOutputStream;
import java.io.InputStream;
import java.nio.file.Files;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
@@ -13,6 +13,7 @@ import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.ModelAttribute;
import org.springframework.web.multipart.MultipartFile;
import org.springframework.web.servlet.mvc.method.annotation.StreamingResponseBody;
import io.swagger.v3.oas.annotations.Operation;
@@ -28,6 +29,8 @@ import stirling.software.common.annotations.AutoJobPostMapping;
import stirling.software.common.annotations.api.PipelineApi;
import stirling.software.common.service.PostHogService;
import stirling.software.common.util.GeneralUtils;
import stirling.software.common.util.TempFile;
import stirling.software.common.util.TempFileManager;
import stirling.software.common.util.WebResponseUtils;
import tools.jackson.core.JacksonException;
@@ -45,6 +48,8 @@ public class PipelineController {
private final PostHogService postHogService;
private final TempFileManager tempFileManager;
@AutoJobPostMapping(value = "/handleData", consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
@MultiFileResponse
@Operation(
@@ -53,8 +58,8 @@ public class PipelineController {
"This endpoint processes multiple PDF files through a configurable pipeline of operations. "
+ "Users provide files and a JSON configuration defining the sequence of operations to perform. "
+ "Input:PDF Output:PDF/ZIP Type:MIMO")
public ResponseEntity<byte[]> handleData(@ModelAttribute HandleDataRequest request)
throws DatabindException, JacksonException {
public ResponseEntity<StreamingResponseBody> handleData(
@ModelAttribute HandleDataRequest request) throws DatabindException, JacksonException {
MultipartFile[] files = request.getFileInput();
String jsonString = request.getJson();
if (files == null) {
@@ -80,51 +85,56 @@ public class PipelineController {
PipelineResult result = processor.runPipelineAgainstFiles(inputFiles, config);
List<Resource> outputFiles = result.getOutputFiles();
if (outputFiles != null && outputFiles.size() == 1) {
// If there is only one file, return it directly
// If there is only one file, return it directly stream without int-overflow
Resource singleFile = outputFiles.get(0);
InputStream is = singleFile.getInputStream();
byte[] bytes = new byte[(int) singleFile.contentLength()];
is.read(bytes);
is.close();
log.info("Returning single file response...");
return WebResponseUtils.bytesToWebResponse(
bytes, singleFile.getFilename(), MediaType.APPLICATION_OCTET_STREAM);
TempFile singleTempFile = new TempFile(tempFileManager, ".out");
try {
try (InputStream is = singleFile.getInputStream()) {
is.transferTo(Files.newOutputStream(singleTempFile.getPath()));
}
log.info("Returning single file response...");
return WebResponseUtils.fileToWebResponse(
singleTempFile,
singleFile.getFilename(),
MediaType.APPLICATION_OCTET_STREAM);
} catch (Exception e) {
singleTempFile.close();
throw e;
}
} else if (outputFiles == null) {
return null;
}
// Create a ByteArrayOutputStream to hold the zip
ByteArrayOutputStream baos = new ByteArrayOutputStream();
ZipOutputStream zipOut = new ZipOutputStream(baos);
// A map to keep track of filenames and their counts
Map<String, Integer> filenameCount = new HashMap<>();
// Loop through each file and add it to the zip
for (Resource file : outputFiles) {
String originalFilename = file.getFilename();
String filename = originalFilename;
// Check if the filename already exists, and modify it if necessary
if (filenameCount.containsKey(originalFilename)) {
int count = filenameCount.get(originalFilename);
assert originalFilename != null;
filename = GeneralUtils.generateFilename(originalFilename, "(" + count + ")");
filenameCount.put(originalFilename, count + 1);
} else {
filenameCount.put(originalFilename, 1);
// Multiple files: stream into a zip TempFile
TempFile zipTempFile = new TempFile(tempFileManager, ".zip");
try {
Map<String, Integer> filenameCount = new HashMap<>();
try (ZipOutputStream zipOut =
new ZipOutputStream(Files.newOutputStream(zipTempFile.getPath()))) {
for (Resource file : outputFiles) {
String originalFilename = file.getFilename();
String filename = originalFilename;
if (filenameCount.containsKey(originalFilename)) {
int count = filenameCount.get(originalFilename);
filename =
GeneralUtils.generateFilename(
originalFilename, "(" + count + ")");
filenameCount.put(originalFilename, count + 1);
} else {
filenameCount.put(originalFilename, 1);
}
zipOut.putNextEntry(new ZipEntry(filename));
try (InputStream is = file.getInputStream()) {
is.transferTo(zipOut);
}
zipOut.closeEntry();
}
}
ZipEntry zipEntry = new ZipEntry(filename);
zipOut.putNextEntry(zipEntry);
// Read the file into a byte array
InputStream is = file.getInputStream();
byte[] bytes = new byte[(int) file.contentLength()];
is.read(bytes);
// Write the bytes of the file to the zip
zipOut.write(bytes, 0, bytes.length);
zipOut.closeEntry();
is.close();
log.info("Returning zipped file response...");
return WebResponseUtils.zipFileToWebResponse(zipTempFile, "output.zip");
} catch (Exception e) {
zipTempFile.close();
throw e;
}
zipOut.close();
log.info("Returning zipped file response...");
return WebResponseUtils.baosToWebResponse(
baos, "output.zip", MediaType.APPLICATION_OCTET_STREAM);
} catch (Exception e) {
log.error("Error handling data: ", e);
return null;
@@ -254,7 +254,7 @@ public class PipelineDirectoryProcessor {
String extension =
filename.contains(".")
? filename.substring(
filename.lastIndexOf(".")
filename.lastIndexOf('.')
+ 1)
.toLowerCase(Locale.ROOT)
: "";
@@ -67,7 +67,7 @@ public class PipelineProcessor {
public static String removeTrailingNaming(String filename) {
// Splitting filename into name and extension
int dotIndex = filename.lastIndexOf(".");
int dotIndex = filename.lastIndexOf('.');
if (dotIndex == -1) {
// No extension found
return filename;
@@ -75,7 +75,7 @@ public class PipelineProcessor {
String name = filename.substring(0, dotIndex);
String extension = filename.substring(dotIndex);
// Finding the last underscore
int underscoreIndex = name.lastIndexOf("_");
int underscoreIndex = name.lastIndexOf('_');
if (underscoreIndex == -1) {
// No underscore found
return filename;
@@ -173,7 +173,7 @@ public class PipelineProcessor {
String providedExtension = "no extension";
if (filename != null && filename.contains(".")) {
providedExtension =
filename.substring(filename.lastIndexOf("."))
filename.substring(filename.lastIndexOf('.'))
.toLowerCase(Locale.ROOT);
}
@@ -248,7 +248,7 @@ public class PipelineProcessor {
String filename = file.getFilename();
if (filename != null && filename.contains(".")) {
return filename.substring(
filename.lastIndexOf("."))
filename.lastIndexOf('.'))
.toLowerCase(Locale.ROOT);
}
return "no extension";
@@ -450,7 +450,21 @@ public class PipelineProcessor {
return isZip(data, null);
}
private static final int MAX_UNZIP_DEPTH = 10;
private List<Resource> unzip(Resource data, PipelineResult result) throws IOException {
return unzip(data, result, 0);
}
private List<Resource> unzip(Resource data, PipelineResult result, int depth)
throws IOException {
if (depth > MAX_UNZIP_DEPTH) {
log.warn(
"ZIP nesting depth {} exceeds limit {}, treating as file",
depth,
MAX_UNZIP_DEPTH);
return List.of(data);
}
log.info("Unzipping data of length: {}", data.contentLength());
List<Resource> unzippedFiles = new ArrayList<>();
try (InputStream bais = data.getInputStream();
@@ -481,7 +495,7 @@ public class PipelineProcessor {
// If the unzipped file is a zip file, unzip it
if (isZip(fileResource, filename)) {
log.info("File {} is a zip file. Unzipping...", filename);
unzippedFiles.addAll(unzip(fileResource, result));
unzippedFiles.addAll(unzip(fileResource, result, depth + 1));
} else {
unzippedFiles.add(fileResource);
}
@@ -188,7 +188,7 @@ public class WatermarkController {
};
ClassPathResource classPathResource = new ClassPathResource(resourceDir);
String fileExtension = resourceDir.substring(resourceDir.lastIndexOf("."));
String fileExtension = resourceDir.substring(resourceDir.lastIndexOf('.'));
File tempFile = Files.createTempFile("NotoSansFont", fileExtension).toFile();
try (InputStream is = classPathResource.getInputStream();
FileOutputStream os = new FileOutputStream(tempFile)) {
@@ -51,9 +51,19 @@ public class MetricsController {
description =
"This endpoint returns the status of the application and its version number.")
public ResponseEntity<?> getStatus() {
if (!metricsEnabled) {
return ResponseEntity.status(HttpStatus.FORBIDDEN).body("This endpoint is disabled.");
}
return getApplicationStatus();
}
@GetMapping("/health")
@Operation(
summary = "Application health check",
description =
"This endpoint returns the health status of the application and its version number. Mirrors /api/v1/info/status.")
public ResponseEntity<?> getHealth() {
return getApplicationStatus();
}
private ResponseEntity<?> getApplicationStatus() {
Map<String, String> status = new HashMap<>();
status.put("status", "UP");
String version = getClass().getPackage().getImplementationVersion();
@@ -3,6 +3,7 @@ package stirling.software.SPDF.exception;
import java.io.IOException;
import java.net.URI;
import java.time.Instant;
import java.util.LinkedHashMap;
import java.util.List;
import org.springframework.context.MessageSource;
@@ -23,8 +24,6 @@ import org.springframework.web.multipart.MaxUploadSizeExceededException;
import org.springframework.web.multipart.support.MissingServletRequestPartException;
import org.springframework.web.servlet.NoHandlerFoundException;
import com.fasterxml.jackson.databind.ObjectMapper;
import jakarta.servlet.http.HttpServletRequest;
import jakarta.servlet.http.HttpServletResponse;
@@ -35,6 +34,8 @@ import stirling.software.common.util.ExceptionUtils;
import stirling.software.common.util.ExceptionUtils.*;
import stirling.software.common.util.RegexPatternUtils;
import tools.jackson.databind.ObjectMapper;
/**
* Returns RFC 7807 Problem Details for HTTP APIs, ensuring consistent error responses across the
* application.
@@ -870,7 +871,7 @@ public class GlobalExceptionHandler {
// Use ObjectMapper to properly escape JSON values and prevent XSS
ObjectMapper mapper = new ObjectMapper();
java.util.Map<String, Object> errorMap = new java.util.LinkedHashMap<>();
java.util.Map<String, Object> errorMap = new LinkedHashMap<>();
errorMap.put("type", "about:blank");
errorMap.put("title", "Not Acceptable");
errorMap.put("status", 406);
@@ -25,8 +25,9 @@ public class ApiEndpoint {
}
public boolean areParametersValid(Map<String, Object> providedParams) {
for (String requiredParam : parameters.keySet()) {
if (!providedParams.containsKey(requiredParam)) {
for (Map.Entry<String, JsonNode> entry : parameters.entrySet()) {
boolean isRequired = entry.getValue().path("required").asBoolean(false);
if (isRequired && !providedParams.containsKey(entry.getKey())) {
return false;
}
}
@@ -1,7 +1,5 @@
package stirling.software.SPDF.model.json;
import java.util.List;
import com.fasterxml.jackson.annotation.JsonInclude;
import lombok.AllArgsConstructor;
@@ -27,13 +25,13 @@ public class PdfJsonAnnotation {
private String contents;
/** Annotation rectangle [x1, y1, x2, y2] */
private List<Float> rect;
private float[] rect;
/** Annotation appearance characteristics */
private String appearanceState;
/** Color components (e.g., [r, g, b] for RGB) */
private List<Float> color;
private float[] color;
/** Annotation flags (print, hidden, etc.) */
private Integer flags;
@@ -14,7 +14,7 @@ import lombok.NoArgsConstructor;
@Builder
@NoArgsConstructor
@AllArgsConstructor
@JsonInclude(JsonInclude.Include.NON_NULL)
@JsonInclude(JsonInclude.Include.NON_DEFAULT)
public class PdfJsonDocument {
private PdfJsonMetadata metadata;
@@ -23,7 +23,7 @@ public class PdfJsonDocument {
private String xmpMetadata;
/** Indicates that images should be loaded lazily via API rather than embedded in the JSON. */
private Boolean lazyImages;
private boolean lazyImages;
@Builder.Default private List<PdfJsonFont> fonts = new ArrayList<>();
@@ -1,7 +1,5 @@
package stirling.software.SPDF.model.json;
import java.util.List;
import com.fasterxml.jackson.annotation.JsonInclude;
import lombok.AllArgsConstructor;
@@ -65,5 +63,5 @@ public class PdfJsonFontConversionCandidate {
private String diagnostics;
/** Known unicode/codepoint coverage derived from the conversion strategy. */
private List<Integer> glyphCoverage;
private int[] glyphCoverage;
}
@@ -45,13 +45,13 @@ public class PdfJsonFormField {
private Integer pageNumber;
/** Field rectangle [x1, y1, x2, y2] on the page */
private List<Float> rect;
private float[] rect;
/** For choice fields: list of options */
private List<String> options;
/** For choice fields: selected indices */
private List<Integer> selectedIndices;
private int[] selectedIndices;
/** For button fields: whether it's checked */
private Boolean checked;
@@ -1,8 +1,5 @@
package stirling.software.SPDF.model.json;
import java.util.ArrayList;
import java.util.List;
import com.fasterxml.jackson.annotation.JsonInclude;
import lombok.AllArgsConstructor;
@@ -30,7 +27,7 @@ public class PdfJsonImageElement {
private Float right;
private Float top;
private Float bottom;
@Builder.Default private List<Float> transform = new ArrayList<>();
private float[] transform;
private Integer zOrder;
private String imageData;
private String imageFormat;
@@ -11,10 +11,10 @@ import lombok.NoArgsConstructor;
@Builder
@NoArgsConstructor
@AllArgsConstructor
@JsonInclude(JsonInclude.Include.NON_NULL)
@JsonInclude(JsonInclude.Include.NON_DEFAULT)
public class PdfJsonPageDimension {
private Integer pageNumber;
private Float width;
private Float height;
private Integer rotation;
private int pageNumber;
private float width;
private float height;
private int rotation;
}
@@ -1,7 +1,5 @@
package stirling.software.SPDF.model.json;
import java.util.List;
import com.fasterxml.jackson.annotation.JsonInclude;
import lombok.AllArgsConstructor;
@@ -17,5 +15,5 @@ import lombok.NoArgsConstructor;
public class PdfJsonTextColor {
private String colorSpace;
private List<Float> components;
private float[] components;
}
@@ -1,7 +1,5 @@
package stirling.software.SPDF.model.json;
import java.util.List;
import com.fasterxml.jackson.annotation.JsonInclude;
import lombok.AllArgsConstructor;
@@ -32,10 +30,10 @@ public class PdfJsonTextElement {
private Float y;
private Float width;
private Float height;
private List<Float> textMatrix;
private float[] textMatrix;
private PdfJsonTextColor fillColor;
private PdfJsonTextColor strokeColor;
private Integer renderingMode;
private Boolean fallbackUsed;
private List<Integer> charCodes;
private int[] charCodes;
}
@@ -42,9 +42,10 @@ public class LanguageService {
languageCode -> {
Set<String> allowedLanguages =
new HashSet<>(applicationProperties.getUi().getLanguages());
// Empty list means all languages are allowed (no filtering)
// Non-empty list acts as a strict whitelist
return allowedLanguages.isEmpty()
|| allowedLanguages.contains(languageCode)
|| "en_GB".equals(languageCode);
|| allowedLanguages.contains(languageCode);
})
.collect(Collectors.toSet());
@@ -273,11 +273,11 @@ public class PdfJsonConversionService {
// Get job ID from request context if running in async mode
String contextJobId = getJobIdFromRequest();
boolean isRealJobId = (contextJobId != null && !contextJobId.isEmpty());
boolean useLazyImages = (contextJobId != null && !contextJobId.isEmpty());
// Generate synthetic jobId for synchronous conversions to prevent cache collisions
final String jobId;
if (!isRealJobId) {
if (!useLazyImages) {
jobId = "pdf2json:" + java.util.UUID.randomUUID().toString();
log.debug("Generated synthetic jobId for synchronous conversion: {}", jobId);
} else {
@@ -302,7 +302,7 @@ public class PdfJsonConversionService {
: "");
progressCallback.accept(p);
}
: isRealJobId
: useLazyImages
? (p) -> {
log.debug(
"Progress: [{}%] {} - {}{}",
@@ -369,7 +369,6 @@ public class PdfJsonConversionService {
int totalPages = document.getNumberOfPages();
// Always enable lazy mode for real async jobs so cache is available regardless of
// page count. Synchronous calls with synthetic jobId still do full extraction.
boolean useLazyImages = isRealJobId;
Map<COSBase, FontModelCacheEntry> fontCache = new IdentityHashMap<>();
Map<COSBase, EncodedImage> imageCache = new IdentityHashMap<>();
log.debug(
@@ -377,7 +376,7 @@ public class PdfJsonConversionService {
totalPages,
useLazyImages ? "lazy image" : "standard",
jobId,
isRealJobId);
useLazyImages);
Map<String, PdfJsonFont> fonts = new LinkedHashMap<>();
Map<Integer, List<PdfJsonTextElement>> textByPage = new LinkedHashMap<>();
Map<Integer, Map<PDFont, String>> pageFontResources = new HashMap<>();
@@ -431,7 +430,7 @@ public class PdfJsonConversionService {
progress.accept(
PdfJsonConversionProgress.of(
80, "annotations", "Collecting annotations and form fields"));
boolean includeAnnotationRawData = !(lightweight && isRealJobId);
boolean includeAnnotationRawData = !(lightweight && useLazyImages);
Map<Integer, List<PdfJsonAnnotation>> annotationsByPage =
collectAnnotations(
document, totalPages, progress, includeAnnotationRawData);
@@ -463,8 +462,8 @@ public class PdfJsonConversionService {
textByPage,
imagesByPage,
annotationsByPage,
lightweight && isRealJobId));
if (lightweight && isRealJobId) {
lightweight && useLazyImages));
if (lightweight && useLazyImages) {
// Lightweight async editor flow does not use form fields and this payload can
// be
// very large due nested raw dictionaries.
@@ -474,12 +473,11 @@ public class PdfJsonConversionService {
}
// Only cache for real async jobIds, not synthetic synchronous ones
if (useLazyImages && isRealJobId) {
if (useLazyImages) {
log.debug(
"Creating cache for jobId: {} (useLazyImages={}, isRealJobId={})",
"Creating cache for jobId: {} (useLazyImages={})",
jobId,
useLazyImages,
isRealJobId);
useLazyImages);
PdfJsonDocumentMetadata docMetadata = new PdfJsonDocumentMetadata();
docMetadata.setMetadata(pdfJson.getMetadata());
docMetadata.setXmpMetadata(pdfJson.getXmpMetadata());
@@ -529,16 +527,15 @@ public class PdfJsonConversionService {
scheduleDocumentCleanup(jobId);
} else {
log.warn(
"Skipping cache creation: useLazyImages={}, isRealJobId={}, jobId={}",
"Skipping cache creation: useLazyImages={}, jobId={}",
useLazyImages,
isRealJobId,
jobId);
}
if (lightweight) {
applyLightweightTransformations(pdfJson);
}
if (lightweight && isRealJobId) {
if (lightweight && useLazyImages) {
stripFontProgramPayloads(responseFonts);
stripFontCosStreamData(responseFonts);
}
@@ -2503,11 +2500,12 @@ public class PdfJsonConversionService {
PDRectangle rect = annotation.getRectangle();
if (rect != null) {
ann.setRect(
List.of(
rect.getLowerLeftX(),
rect.getLowerLeftY(),
rect.getUpperRightX(),
rect.getUpperRightY()));
new float[] {
rect.getLowerLeftX(),
rect.getLowerLeftY(),
rect.getUpperRightX(),
rect.getUpperRightY()
});
}
COSName appearanceState = annotation.getAppearanceState();
@@ -2516,12 +2514,7 @@ public class PdfJsonConversionService {
}
if (annotation.getColor() != null) {
float[] colorComponents = annotation.getColor().getComponents();
List<Float> colorList = new ArrayList<>(colorComponents.length);
for (float c : colorComponents) {
colorList.add(c);
}
ann.setColor(colorList);
ann.setColor(annotation.getColor().getComponents());
}
COSDictionary annotDict = annotation.getCOSObject();
@@ -2636,11 +2629,12 @@ public class PdfJsonConversionService {
PDRectangle rect = widget.getRectangle();
if (rect != null) {
formField.setRect(
List.of(
rect.getLowerLeftX(),
rect.getLowerLeftY(),
rect.getUpperRightX(),
rect.getUpperRightY()));
new float[] {
rect.getLowerLeftX(),
rect.getLowerLeftY(),
rect.getUpperRightX(),
rect.getUpperRightY()
});
}
}
}
@@ -3351,7 +3345,7 @@ public class PdfJsonConversionService {
PDFont currentFont = baseFont;
String currentFontId = baseFontId;
List<Integer> elementCodes = element.getCharCodes();
int[] elementCodes = element.getCharCodes();
int codeIndex = 0;
boolean rawType3CodesUsed = false;
int rawType3GlyphCount = 0;
@@ -3363,8 +3357,8 @@ public class PdfJsonConversionService {
PDFont targetFont = baseFont;
String targetFontId = baseFontId;
Integer rawCode = null;
if (elementCodes != null && codeIndex < elementCodes.size()) {
rawCode = elementCodes.get(codeIndex);
if (elementCodes != null && codeIndex < elementCodes.length) {
rawCode = elementCodes[codeIndex];
}
codeIndex++;
@@ -3667,10 +3661,7 @@ public class PdfJsonConversionService {
if (color == null || color.getComponents() == null) {
return;
}
float[] components = new float[color.getComponents().size()];
for (int i = 0; i < components.length; i++) {
components[i] = color.getComponents().get(i);
}
float[] components = color.getComponents();
String space = color.getColorSpace();
if (space == null) {
// Infer color space from component count
@@ -4287,8 +4278,11 @@ public class PdfJsonConversionService {
List<Integer> combinedCodes = new ArrayList<>();
for (PdfJsonTextElement element : elements) {
builder.append(Objects.toString(element.getText(), ""));
if (element.getCharCodes() != null && !element.getCharCodes().isEmpty()) {
combinedCodes.addAll(element.getCharCodes());
int[] codes = element.getCharCodes();
if (codes != null && codes.length > 0) {
for (int code : codes) {
combinedCodes.add(code);
}
}
}
return new MergedText(builder.toString(), combinedCodes.isEmpty() ? null : combinedCodes);
@@ -4354,9 +4348,9 @@ public class PdfJsonConversionService {
}
private int countGlyphs(PdfJsonTextElement element) {
List<Integer> codes = element.getCharCodes();
if (codes != null && !codes.isEmpty()) {
return codes.size();
int[] codes = element.getCharCodes();
if (codes != null && codes.length > 0) {
return codes.length;
}
String text = element.getText();
if (text != null && !text.isEmpty()) {
@@ -4920,15 +4914,15 @@ public class PdfJsonConversionService {
private void applyTextMatrix(PDPageContentStream contentStream, PdfJsonTextElement element)
throws IOException {
List<Float> matrix = element.getTextMatrix();
if (matrix != null && matrix.size() == 6) {
float[] matrix = element.getTextMatrix();
if (matrix != null && matrix.length == 6) {
float fontScale = resolveFontMatrixSize(element);
float a = matrix.get(0);
float b = matrix.get(1);
float c = matrix.get(2);
float d = matrix.get(3);
float e = matrix.get(4);
float f = matrix.get(5);
float a = matrix[0];
float b = matrix[1];
float c = matrix[2];
float d = matrix[3];
float e = matrix[4];
float f = matrix[5];
if (fontScale != 0f) {
a /= fontScale;
@@ -4950,12 +4944,12 @@ public class PdfJsonConversionService {
if (fromElement != null && fromElement > 0f) {
return fromElement;
}
List<Float> matrix = element.getTextMatrix();
if (matrix != null && matrix.size() >= 4) {
float a = matrix.get(0);
float b = matrix.get(1);
float c = matrix.get(2);
float d = matrix.get(3);
float[] matrix = element.getTextMatrix();
if (matrix != null && matrix.length >= 4) {
float a = matrix[0];
float b = matrix[1];
float c = matrix[2];
float d = matrix[3];
float verticalScale = (float) Math.hypot(b, d);
if (verticalScale > 0f) {
return verticalScale;
@@ -5048,7 +5042,7 @@ public class PdfJsonConversionService {
}
Matrix ctm = getGraphicsState().getCurrentTransformationMatrix();
Bounds bounds = computeBounds(ctm);
List<Float> matrixValues = toMatrixValues(ctm);
float[] matrixValues = toMatrixValues(ctm);
PdfJsonImageElement element =
PdfJsonImageElement.builder()
@@ -5222,15 +5216,15 @@ public class PdfJsonConversionService {
private record EncodedImage(String base64, String format) {}
private List<Float> toMatrixValues(Matrix matrix) {
List<Float> values = new ArrayList<>(6);
values.add(matrix.getValue(0, 0));
values.add(matrix.getValue(0, 1));
values.add(matrix.getValue(1, 0));
values.add(matrix.getValue(1, 1));
values.add(matrix.getValue(2, 0));
values.add(matrix.getValue(2, 1));
return values;
private float[] toMatrixValues(Matrix matrix) {
return new float[] {
matrix.getValue(0, 0),
matrix.getValue(0, 1),
matrix.getValue(1, 0),
matrix.getValue(1, 1),
matrix.getValue(2, 0),
matrix.getValue(2, 1)
};
}
private EncodedImage encodeImage(PDImage image) {
@@ -5341,16 +5335,16 @@ public class PdfJsonConversionService {
cache.put(cacheKey, image);
}
List<Float> transform = element.getTransform();
if (transform != null && transform.size() == 6) {
float[] transform = element.getTransform();
if (transform != null && transform.length == 6) {
Matrix matrix =
new Matrix(
safeFloat(transform.get(0), 1f),
safeFloat(transform.get(1), 0f),
safeFloat(transform.get(2), 0f),
safeFloat(transform.get(3), 1f),
safeFloat(transform.get(4), 0f),
safeFloat(transform.get(5), 0f));
safeFloat(transform[0], 1f),
safeFloat(transform[1], 0f),
safeFloat(transform[2], 0f),
safeFloat(transform[3], 1f),
safeFloat(transform[4], 0f),
safeFloat(transform[5], 0f));
contentStream.drawImage(image, matrix);
return;
}
@@ -5516,14 +5510,17 @@ public class PdfJsonConversionService {
if (pdfont instanceof PDType3Font) {
int[] codes = position.getCharacterCodes();
if (codes != null && codes.length > 0) {
List<Integer> codeList = new ArrayList<>(codes.length);
int count = 0;
for (int code : codes) {
if (code >= 0) {
codeList.add(code);
}
if (code >= 0) count++;
}
if (!codeList.isEmpty()) {
element.setCharCodes(codeList);
if (count > 0) {
int[] filtered = new int[count];
int idx = 0;
for (int code : codes) {
if (code >= 0) filtered[idx++] = code;
}
element.setCharCodes(filtered);
}
}
}
@@ -5552,11 +5549,11 @@ public class PdfJsonConversionService {
return;
}
List<Float> matrix = element.getTextMatrix();
float[] matrix = element.getTextMatrix();
if (matrix != null) {
if (matrix.isEmpty()) {
if (matrix.length == 0) {
element.setTextMatrix(null);
} else if (matrix.size() == 6) {
} else if (matrix.length == 6) {
element.setX(null);
element.setY(null);
}
@@ -5597,29 +5594,29 @@ public class PdfJsonConversionService {
if (color == null || color.getComponents() == null) {
return true;
}
List<Float> components = color.getComponents();
if (components.isEmpty()) {
float[] components = color.getComponents();
if (components.length == 0) {
return true;
}
String space = color.getColorSpace();
if (space == null || "DeviceRGB".equals(space)) {
if (components.size() < 3) {
if (components.length < 3) {
return false;
}
return Math.abs(components.get(0)) < FLOAT_EPSILON
&& Math.abs(components.get(1)) < FLOAT_EPSILON
&& Math.abs(components.get(2)) < FLOAT_EPSILON;
return Math.abs(components[0]) < FLOAT_EPSILON
&& Math.abs(components[1]) < FLOAT_EPSILON
&& Math.abs(components[2]) < FLOAT_EPSILON;
}
if ("DeviceGray".equals(space)) {
return Math.abs(components.get(0)) < FLOAT_EPSILON;
return Math.abs(components[0]) < FLOAT_EPSILON;
}
return false;
}
private Float baselineFrom(PdfJsonTextElement element) {
List<Float> matrix = element.getTextMatrix();
if (matrix != null && matrix.size() >= 6) {
return matrix.get(5);
float[] matrix = element.getTextMatrix();
if (matrix != null && matrix.length >= 6) {
return matrix[5];
}
return element.getY();
}
@@ -5648,11 +5645,12 @@ public class PdfJsonConversionService {
private final float orientationC;
private final float orientationD;
private final Float baseline;
private final List<Float> baseMatrix;
private final float[] baseMatrix;
private final float startXCoord;
private final float startYCoord;
private final StringBuilder textBuilder = new StringBuilder();
private final List<Integer> charCodeBuffer = new ArrayList<>();
private int[] charCodeBuf = new int[16];
private int charCodeLen = 0;
private float totalWidth;
private float maxHeight;
private float endXCoord;
@@ -5660,17 +5658,15 @@ public class PdfJsonConversionService {
TextRunAccumulator(PdfJsonTextElement element, TextPosition position) {
this.baseElement = element;
this.styleKey = buildStyleKey(element);
this.baseMatrix =
element.getTextMatrix() != null
? new ArrayList<>(element.getTextMatrix())
: null;
if (baseMatrix != null && baseMatrix.size() >= 6) {
orientationA = baseMatrix.get(0);
orientationB = baseMatrix.get(1);
orientationC = baseMatrix.get(2);
orientationD = baseMatrix.get(3);
startXCoord = baseMatrix.get(4);
startYCoord = baseMatrix.get(5);
float[] tm = element.getTextMatrix();
this.baseMatrix = tm != null ? tm.clone() : null;
if (baseMatrix != null && baseMatrix.length >= 6) {
orientationA = baseMatrix[0];
orientationB = baseMatrix[1];
orientationC = baseMatrix[2];
orientationD = baseMatrix[3];
startXCoord = baseMatrix[4];
startYCoord = baseMatrix[5];
} else {
orientationA = 1f;
orientationB = 0f;
@@ -5684,25 +5680,23 @@ public class PdfJsonConversionService {
this.maxHeight = element.getHeight() != null ? element.getHeight() : 0f;
this.endXCoord = position.getXDirAdj() + position.getWidthDirAdj();
this.textBuilder.append(element.getText());
if (element.getCharCodes() != null) {
charCodeBuffer.addAll(element.getCharCodes());
}
appendCharCodes(element.getCharCodes());
}
boolean canAppend(PdfJsonTextElement element, TextPosition position) {
if (!styleKey.equals(buildStyleKey(element))) {
return false;
}
List<Float> matrix = element.getTextMatrix();
float[] matrix = element.getTextMatrix();
float a = 1f;
float b = 0f;
float c = 0f;
float d = 1f;
if (matrix != null && matrix.size() >= 4) {
a = matrix.get(0);
b = matrix.get(1);
c = matrix.get(2);
d = matrix.get(3);
if (matrix != null && matrix.length >= 4) {
a = matrix[0];
b = matrix[1];
c = matrix[2];
d = matrix[3];
}
if (Math.abs(a - orientationA) > ORIENTATION_TOLERANCE
|| Math.abs(b - orientationB) > ORIENTATION_TOLERANCE
@@ -5734,9 +5728,19 @@ public class PdfJsonConversionService {
maxHeight = height;
}
endXCoord = position.getXDirAdj() + position.getWidthDirAdj();
if (element.getCharCodes() != null) {
charCodeBuffer.addAll(element.getCharCodes());
appendCharCodes(element.getCharCodes());
}
private void appendCharCodes(int[] codes) {
if (codes == null) return;
int needed = charCodeLen + codes.length;
if (needed > charCodeBuf.length) {
charCodeBuf =
java.util.Arrays.copyOf(
charCodeBuf, Math.max(needed, charCodeBuf.length * 2));
}
System.arraycopy(codes, 0, charCodeBuf, charCodeLen, codes.length);
charCodeLen += codes.length;
}
PdfJsonTextElement build() {
@@ -5748,22 +5752,21 @@ public class PdfJsonConversionService {
}
result.setWidth(totalWidth);
result.setHeight(maxHeight);
if (baseMatrix != null && baseMatrix.size() == 6) {
List<Float> matrix = new ArrayList<>(baseMatrix);
matrix.set(0, orientationA);
matrix.set(1, orientationB);
matrix.set(2, orientationC);
matrix.set(3, orientationD);
matrix.set(4, startXCoord);
matrix.set(5, startYCoord);
if (baseMatrix != null && baseMatrix.length == 6) {
float[] matrix =
new float[] {
orientationA, orientationB,
orientationC, orientationD,
startXCoord, startYCoord
};
result.setTextMatrix(matrix);
result.setX(null);
result.setY(null);
}
if (charCodeBuffer.isEmpty()) {
if (charCodeLen == 0) {
result.setCharCodes(null);
} else {
result.setCharCodes(new ArrayList<>(charCodeBuffer));
result.setCharCodes(java.util.Arrays.copyOf(charCodeBuf, charCodeLen));
}
compactTextElement(result);
return result;
@@ -5784,29 +5787,25 @@ public class PdfJsonConversionService {
Integer renderingMode,
Float spaceWidth) {}
private List<Float> extractMatrix(TextPosition position) {
float[] values = new float[6];
values[0] = position.getTextMatrix().getValue(0, 0);
values[1] = position.getTextMatrix().getValue(0, 1);
values[2] = position.getTextMatrix().getValue(1, 0);
values[3] = position.getTextMatrix().getValue(1, 1);
values[4] = position.getTextMatrix().getValue(2, 0);
values[5] = position.getTextMatrix().getValue(2, 1);
List<Float> matrix = new ArrayList<>(6);
for (float value : values) {
matrix.add(value);
}
return matrix;
private float[] extractMatrix(TextPosition position) {
return new float[] {
position.getTextMatrix().getValue(0, 0),
position.getTextMatrix().getValue(0, 1),
position.getTextMatrix().getValue(1, 0),
position.getTextMatrix().getValue(1, 1),
position.getTextMatrix().getValue(2, 0),
position.getTextMatrix().getValue(2, 1)
};
}
private Float computeFontMatrixSize(List<Float> matrix) {
if (matrix == null || matrix.size() < 4) {
private Float computeFontMatrixSize(float[] matrix) {
if (matrix == null || matrix.length < 4) {
return null;
}
float a = matrix.get(0);
float b = matrix.get(1);
float c = matrix.get(2);
float d = matrix.get(3);
float a = matrix[0];
float b = matrix[1];
float c = matrix[2];
float d = matrix[3];
float scaleX = (float) Math.hypot(a, c);
float scaleY = (float) Math.hypot(b, d);
float scale = Math.max(scaleX, scaleY);
@@ -5850,11 +5849,10 @@ public class PdfJsonConversionService {
colorSpaceName,
ex.getMessage());
}
List<Float> values = new ArrayList<>(effective.length);
for (float component : effective) {
values.add(component);
}
return PdfJsonTextColor.builder().colorSpace(colorSpaceName).components(values).build();
return PdfJsonTextColor.builder()
.colorSpace(colorSpaceName)
.components(effective)
.build();
}
private String sanitizeForLog(String value) {
@@ -6241,11 +6239,12 @@ public class PdfJsonConversionService {
PDRectangle rect = annotation.getRectangle();
if (rect != null) {
ann.setRect(
List.of(
rect.getLowerLeftX(),
rect.getLowerLeftY(),
rect.getUpperRightX(),
rect.getUpperRightY()));
new float[] {
rect.getLowerLeftX(),
rect.getLowerLeftY(),
rect.getUpperRightX(),
rect.getUpperRightY()
});
}
COSName appearanceState = annotation.getAppearanceState();
@@ -6254,12 +6253,7 @@ public class PdfJsonConversionService {
}
if (annotation.getColor() != null) {
float[] colorComponents = annotation.getColor().getComponents();
List<Float> colorList = new ArrayList<>(colorComponents.length);
for (float c : colorComponents) {
colorList.add(c);
}
ann.setColor(colorList);
ann.setColor(annotation.getColor().getComponents());
}
COSDictionary annotDict = annotation.getCOSObject();
@@ -30,7 +30,7 @@ public class SharedSignatureService {
private static final Pattern FILENAME_VALIDATION_PATTERN = Pattern.compile("^[a-zA-Z0-9_.-]+$");
private final String SIGNATURE_BASE_PATH;
private final String ALL_USERS_FOLDER = "ALL_USERS";
private static final String ALL_USERS_FOLDER = "ALL_USERS";
private final ObjectMapper objectMapper;
public SharedSignatureService(ObjectMapper objectMapper) {
@@ -159,12 +159,12 @@ public class SharedSignatureService {
String dataUrl = request.getDataUrl();
if (dataUrl != null && dataUrl.startsWith("data:image/")) {
// Extract base64 data
String base64Data = dataUrl.substring(dataUrl.indexOf(",") + 1);
String base64Data = dataUrl.substring(dataUrl.indexOf(',') + 1);
byte[] imageBytes = Base64.getDecoder().decode(base64Data);
// Determine and validate file extension from data URL
String mimeType = dataUrl.substring(dataUrl.indexOf(":") + 1, dataUrl.indexOf(";"));
String rawExtension = mimeType.substring(mimeType.indexOf("/") + 1);
String mimeType = dataUrl.substring(dataUrl.indexOf(':') + 1, dataUrl.indexOf(';'));
String rawExtension = mimeType.substring(mimeType.indexOf('/') + 1);
String extension = validateAndNormalizeExtension(rawExtension);
// Save image file only
@@ -134,16 +134,16 @@ public class PdfJsonImageService {
cache.put(cacheKey, image);
}
List<Float> transform = element.getTransform();
if (transform != null && transform.size() == 6) {
float[] transform = element.getTransform();
if (transform != null && transform.length == 6) {
Matrix matrix =
new Matrix(
safeFloat(transform.get(0), 1f),
safeFloat(transform.get(1), 0f),
safeFloat(transform.get(2), 0f),
safeFloat(transform.get(3), 1f),
safeFloat(transform.get(4), 0f),
safeFloat(transform.get(5), 0f));
safeFloat(transform[0], 1f),
safeFloat(transform[1], 0f),
safeFloat(transform[2], 0f),
safeFloat(transform[3], 1f),
safeFloat(transform[4], 0f),
safeFloat(transform[5], 0f));
contentStream.drawImage(image, matrix);
return;
}
@@ -269,18 +269,18 @@ public class PdfJsonImageService {
return 0f;
}
private List<Float> toMatrixValues(Matrix matrix) {
List<Float> values = new ArrayList<>(6);
values.add(matrix.getValue(0, 0));
values.add(matrix.getValue(0, 1));
values.add(matrix.getValue(1, 0));
values.add(matrix.getValue(1, 1));
values.add(matrix.getValue(2, 0));
values.add(matrix.getValue(2, 1));
return values;
private float[] toMatrixValues(Matrix matrix) {
return new float[] {
matrix.getValue(0, 0),
matrix.getValue(0, 1),
matrix.getValue(1, 0),
matrix.getValue(1, 1),
matrix.getValue(2, 0),
matrix.getValue(2, 1)
};
}
private float safeFloat(Float value, float defaultValue) {
private static float safeFloat(Float value, float defaultValue) {
if (value == null || Float.isNaN(value) || Float.isInfinite(value)) {
return defaultValue;
}
@@ -324,7 +324,7 @@ public class PdfJsonImageService {
}
Matrix ctm = getGraphicsState().getCurrentTransformationMatrix();
Bounds bounds = computeBounds(ctm);
List<Float> matrixValues = toMatrixValues(ctm);
float[] matrixValues = toMatrixValues(ctm);
PdfJsonImageElement element =
PdfJsonImageElement.builder()
@@ -110,7 +110,12 @@ public class Type3LibraryStrategy implements Type3ConversionStrategy {
.webProgramFormat(toFormat(entry.getWebProgram()))
.pdfProgram(toBase64(entry.getPdfProgram()))
.pdfProgramFormat(toFormat(entry.getPdfProgram()))
.glyphCoverage(entry.getGlyphCoverage())
.glyphCoverage(
entry.getGlyphCoverage() != null
? entry.getGlyphCoverage().stream()
.mapToInt(Integer::intValue)
.toArray()
: null)
.message(message)
.build();
}
@@ -262,17 +262,30 @@ public class JobController {
@Operation(summary = "Get file metadata")
public ResponseEntity<?> getFileMetadata(@PathVariable("fileId") String fileId) {
try {
// Verify file exists
if (!fileStorage.fileExists(fileId)) {
String jobKey = taskManager.findJobKeyByFileId(fileId);
if (jobKey == null) {
return ResponseEntity.notFound().build();
}
if (!validateJobAccess(jobKey)) {
log.warn("Unauthorized attempt to access file metadata: {}", fileId);
return ResponseEntity.status(403)
.body(Map.of("message", "You are not authorized to access this file"));
}
// Find the file metadata from any job that contains this file
ResultFile resultFile = taskManager.findResultFileByFileId(fileId);
if (resultFile != null) {
return ResponseEntity.ok(resultFile);
} else {
}
if (!isSecurityEnabled()) {
// Backwards compatibility when ownership service is unavailable
if (!fileStorage.fileExists(fileId)) {
return ResponseEntity.notFound().build();
}
// File exists but no metadata found, get basic info efficiently
long fileSize = fileStorage.getFileSize(fileId);
return ResponseEntity.ok(
@@ -286,6 +299,8 @@ public class JobController {
"fileSize",
fileSize));
}
return ResponseEntity.notFound().build();
} catch (Exception e) {
log.error("Error retrieving file metadata {}: {}", fileId, e.getMessage(), e);
return ResponseEntity.internalServerError()
@@ -303,11 +318,17 @@ public class JobController {
@Operation(summary = "Download a file")
public ResponseEntity<?> downloadFile(@PathVariable("fileId") String fileId) {
try {
// Verify file exists
if (!fileStorage.fileExists(fileId)) {
String jobKey = taskManager.findJobKeyByFileId(fileId);
if (jobKey == null) {
return ResponseEntity.notFound().build();
}
if (!validateJobAccess(jobKey)) {
log.warn("Unauthorized attempt to download file: {}", fileId);
return ResponseEntity.status(403)
.body(Map.of("message", "You are not authorized to access this file"));
}
// Retrieve file content
byte[] fileContent = fileStorage.retrieveBytes(fileId);
@@ -327,11 +348,14 @@ public class JobController {
.body(fileContent);
} catch (Exception e) {
log.error("Error retrieving file {}: {}", fileId, e.getMessage(), e);
return ResponseEntity.internalServerError()
.body("Error retrieving file: " + e.getMessage());
return ResponseEntity.internalServerError().body("Error retrieving file");
}
}
private boolean isSecurityEnabled() {
return jobOwnershipService != null;
}
/**
* Create Content-Disposition header with UTF-8 filename support
*
@@ -1,4 +1,9 @@
multipart.enabled=true
# Jackson 3 defaults FAIL_ON_NULL_FOR_PRIMITIVES to true (was false in Jackson 2).
# Restore Jackson 2 behaviour so absent/null JSON fields map to Java primitive defaults.
spring.jackson.deserialization.fail-on-null-for-primitives=false
logging.level.org.springframework=WARN
logging.level.org.springframework.security=WARN
logging.level.org.hibernate=WARN
@@ -25,14 +30,16 @@ server.compression.enabled=true
server.compression.min-response-size=1024
server.compression.mime-types=application/json,application/xml,text/html,text/plain,text/css,application/javascript
server.error.path=/error
server.error.whitelabel.enabled=false
server.error.include-stacktrace=always
server.error.include-exception=true
server.error.include-message=always
spring.web.error.path=/error
spring.web.error.whitelabel.enabled=false
spring.web.error.include-stacktrace=always
spring.web.error.include-exception=true
spring.web.error.include-message=always
# Enable RFC 7807 Problem Details for HTTP APIs
spring.mvc.problemdetails.enabled=true
# Disable Spring's built-in ProblemDetailsExceptionHandler (@Order(0)) so that
# GlobalExceptionHandler runs first and provides detailed, logged error responses.
# GlobalExceptionHandler already produces RFC 7807 ProblemDetail objects.
spring.mvc.problemdetails.enabled=false
#logging.level.org.springframework.web=DEBUG
#logging.level.org.springframework=DEBUG
@@ -96,11 +96,19 @@ premium:
author: username
creator: Stirling-PDF
producer: Stirling-PDF
googleDrive:
enabled: false # Enable Google Drive file picker integration
clientId: "" # Google OAuth 2.0 client ID (obtain from Google Cloud Console)
apiKey: "" # Google API key for Google Picker API (obtain from Google Cloud Console)
appId: "" # Google Drive app ID
enterpriseFeatures:
audit:
enabled: true # Enable audit logging
level: 2 # Audit logging level: 0=OFF, 1=BASIC, 2=STANDARD, 3=VERBOSE
retentionDays: 90 # Number of days to retain audit logs
enabled: true # Enable audit logging for security and compliance tracking
level: 2 # Audit logging level: 0=OFF, 1=BASIC (compress/split/merge/etc and settings), 2=STANDARD (BASIC + user actions, excludes polling), 3=VERBOSE (everything including polling).
retentionDays: 90 # Number of days to retain audit logs (0 or negative = infinite retention)
captureFileHash: false # Capture SHA-256 hash of uploaded/processed files. Warning: adds 50-200ms per file depending on size. Only enabled independently of audit level.
capturePdfAuthor: false # Capture author metadata from PDF documents. Warning: requires PDF parsing which increases processing time. Only enabled independently of audit level.
captureOperationResults: false # Capture operation return values and responses in audit log. Warning: not recommended, significantly increases log volume and disk usage. Use only for debugging.
databaseNotifications:
backups:
successful: false # set to 'true' to enable email notifications for successful database backups
@@ -173,7 +181,7 @@ system:
corsAllowedOrigins: [] # List of allowed origins for CORS (e.g. ['http://localhost:5173', 'https://app.example.com']). Leave empty to disable CORS. For local development with frontend on port 5173, add 'http://localhost:5173'
backendUrl: "" # Backend base URL for SAML/OAuth/API callbacks (e.g. 'http://localhost:8080' for dev, 'https://api.example.com' for production). REQUIRED for SSO authentication to work correctly. This is where your IdP will send SAML responses and OAuth callbacks. Leave empty to default to 'http://localhost:8080' in development.
frontendUrl: "" # Frontend URL for invite email links (e.g. 'https://app.example.com'). Optional - if not set, will use backendUrl. This is the URL users click in invite emails.
enableMobileScanner: false # Enable mobile phone QR code upload feature. Requires frontendUrl to be configured.
enableMobileScanner: true # Enable mobile phone QR code upload feature. Requires frontendUrl to be configured.
mobileScannerSettings:
convertToPdf: true # Automatically convert uploaded images to PDF format. If false, images are kept as-is.
imageResolution: full # Image resolution for mobile uploads: 'full' (original size) or 'reduced' (max 1200px on longest side). Only applies when convertToPdf is true.
@@ -232,9 +240,12 @@ system:
ui:
appNameNavbar: "" # name displayed on the navigation bar
logoStyle: classic # Options: 'classic' (default - classic S icon) or 'modern' (minimalist logo)
languages: [] # If empty, all languages are enabled. To display only German and Polish ["de_DE", "pl_PL"]. British English is always enabled.
languages: [] # If empty, all languages are enabled. To restrict to specific languages, use a whitelist like ["de_DE", "pl_PL", "sv_SE"]. Empty list or not restricting any languages will enable all available languages.
defaultHideUnavailableTools: false # Default user preference: hide disabled tools instead of greying them out
defaultHideUnavailableConversions: false # Default user preference: hide disabled conversion options instead of greying them out
hideDisabledTools:
googleDrive: false # Hide Google Drive button when not enabled
mobileQRScanner: false # Hide mobile QR scanner button when not enabled
endpoints:
toRemove: [] # list endpoints to disable (e.g. ['img-to-pdf', 'remove-pages'])
@@ -25,6 +25,7 @@ import org.springframework.http.HttpStatus;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.mock.web.MockMultipartFile;
import org.springframework.web.multipart.MultipartFile;
import stirling.software.SPDF.model.api.general.SplitPdfBySizeOrCountRequest;
import stirling.software.common.service.CustomPDFDocumentFactory;
@@ -69,16 +70,15 @@ class SplitPdfBySizeControllerTest {
request.setSplitType(1); // Page count
request.setSplitValue("2");
when(pdfDocumentFactory.load(any(byte[].class)))
.thenAnswer(inv -> Loader.loadPDF((byte[]) inv.getArgument(0)));
when(pdfDocumentFactory.load(any(MultipartFile.class)))
.thenAnswer(inv -> Loader.loadPDF(((MultipartFile) inv.getArgument(0)).getBytes()));
when(pdfDocumentFactory.createNewDocumentBasedOnOldDocument(any(PDDocument.class)))
.thenAnswer(inv -> new PDDocument());
ResponseEntity<byte[]> response = controller.autoSplitPdf(request);
ResponseEntity<?> response = controller.autoSplitPdf(request);
assertThat(response.getStatusCode()).isEqualTo(HttpStatus.OK);
assertThat(response.getBody()).isNotEmpty();
assertThat(response.getHeaders().getContentType())
.isEqualTo(MediaType.APPLICATION_OCTET_STREAM);
}
@@ -104,15 +104,19 @@ class SplitPdfBySizeControllerTest {
request.setSplitType(2); // Document count
request.setSplitValue("3"); // Split into 3 docs (2 pages each)
when(pdfDocumentFactory.load(any(byte[].class)))
.thenAnswer(inv -> Loader.loadPDF((byte[]) inv.getArgument(0)));
when(pdfDocumentFactory.load(any(org.springframework.web.multipart.MultipartFile.class)))
.thenAnswer(
inv ->
Loader.loadPDF(
((org.springframework.web.multipart.MultipartFile)
inv.getArgument(0))
.getBytes()));
when(pdfDocumentFactory.createNewDocumentBasedOnOldDocument(any(PDDocument.class)))
.thenAnswer(inv -> new PDDocument());
ResponseEntity<byte[]> response = controller.autoSplitPdf(request);
ResponseEntity<?> response = controller.autoSplitPdf(request);
assertThat(response.getStatusCode()).isEqualTo(HttpStatus.OK);
assertThat(response.getBody()).isNotEmpty();
}
}
@@ -113,10 +113,12 @@ class ConvertPDFToEpubControllerTest {
ResponseEntity<byte[]> response = controller.convertPdfToEpub(request);
List<String> command = commandCaptor.getValue();
assertEquals(11, command.size());
assertEquals(13, command.size());
assertEquals("ebook-convert", command.get(0));
assertEquals(expectedInput.toString(), command.get(1));
assertEquals(expectedOutput.toString(), command.get(2));
assertTrue(command.contains("--pdf-engine"));
assertTrue(command.contains("pdftohtml"));
assertTrue(command.contains("--enable-heuristics"));
assertTrue(command.contains("--insert-blank-line"));
assertTrue(command.contains("--filter-css"));
@@ -206,11 +208,13 @@ class ConvertPDFToEpubControllerTest {
assertTrue(command.stream().noneMatch(arg -> "--chapter".equals(arg)));
assertTrue(command.contains("--output-profile"));
assertTrue(command.contains(TargetDevice.KINDLE_EINK_TEXT.getCalibreProfile()));
assertTrue(command.contains("--pdf-engine"));
assertTrue(command.contains("pdftohtml"));
assertTrue(command.contains("--filter-css"));
assertTrue(
command.contains(
"font-family,color,background-color,margin-left,margin-right"));
assertTrue(command.size() >= 9);
assertTrue(command.size() >= 11);
assertEquals(EPUB_MEDIA_TYPE, response.getHeaders().getContentType());
assertEquals(
@@ -289,6 +293,8 @@ class ConvertPDFToEpubControllerTest {
assertEquals("ebook-convert", command.get(0));
assertEquals(expectedInput.toString(), command.get(1));
assertEquals(expectedOutput.toString(), command.get(2));
assertTrue(command.contains("--pdf-engine"));
assertTrue(command.contains("pdftohtml"));
assertTrue(command.contains("--enable-heuristics"));
assertTrue(command.contains("--insert-blank-line"));
assertTrue(command.contains("--filter-css"));
@@ -13,10 +13,12 @@ import org.springframework.mock.web.MockMultipartFile;
import stirling.software.common.service.CustomPDFDocumentFactory;
import stirling.software.common.util.PdfToCbzUtils;
import stirling.software.common.util.TempFileManager;
public class PdfToCbzUtilsTest {
@Mock private CustomPDFDocumentFactory pdfDocumentFactory;
@Mock private TempFileManager tempFileManager;
@BeforeEach
public void setUp() {
@@ -42,7 +44,9 @@ public class PdfToCbzUtilsTest {
IllegalArgumentException exception =
Assertions.assertThrows(
IllegalArgumentException.class,
() -> PdfToCbzUtils.convertPdfToCbz(null, 300, pdfDocumentFactory));
() ->
PdfToCbzUtils.convertPdfToCbz(
null, 300, pdfDocumentFactory, tempFileManager));
Assertions.assertEquals("File cannot be null or empty", exception.getMessage());
}
@@ -54,7 +58,9 @@ public class PdfToCbzUtilsTest {
IllegalArgumentException exception =
Assertions.assertThrows(
IllegalArgumentException.class,
() -> PdfToCbzUtils.convertPdfToCbz(emptyFile, 300, pdfDocumentFactory));
() ->
PdfToCbzUtils.convertPdfToCbz(
emptyFile, 300, pdfDocumentFactory, tempFileManager));
Assertions.assertEquals("File cannot be null or empty", exception.getMessage());
}
@@ -66,7 +72,9 @@ public class PdfToCbzUtilsTest {
IllegalArgumentException exception =
Assertions.assertThrows(
IllegalArgumentException.class,
() -> PdfToCbzUtils.convertPdfToCbz(nonPdfFile, 300, pdfDocumentFactory));
() ->
PdfToCbzUtils.convertPdfToCbz(
nonPdfFile, 300, pdfDocumentFactory, tempFileManager));
Assertions.assertEquals("File must be in PDF format", exception.getMessage());
}
@@ -84,7 +92,9 @@ public class PdfToCbzUtilsTest {
// structure
Assertions.assertThrows(
Exception.class,
() -> PdfToCbzUtils.convertPdfToCbz(pdfFile, 300, pdfDocumentFactory));
() ->
PdfToCbzUtils.convertPdfToCbz(
pdfFile, 300, pdfDocumentFactory, tempFileManager));
// Verify that load was called
Mockito.verify(pdfDocumentFactory).load(pdfFile);
@@ -18,6 +18,11 @@ class ApiEndpointTest {
private final ObjectMapper mapper = JsonMapper.builder().build();
private JsonNode postNodeWithParams(String description, String... names) {
return postNodeWithParams(description, true, names);
}
private JsonNode postNodeWithParams(
String description, boolean required, String... names) {
ObjectNode post = mapper.createObjectNode();
post.put("description", description);
ArrayNode params = mapper.createArrayNode();
@@ -26,6 +31,7 @@ class ApiEndpointTest {
if (n != null) {
p.put("name", n);
}
p.put("required", required);
params.add(p);
}
post.set("parameters", params);
@@ -92,6 +98,75 @@ class ApiEndpointTest {
assertTrue(endpoint.areParametersValid(Map.of("", 42)));
}
@Test
void optional_parameters_can_be_omitted() {
JsonNode post = postNodeWithParams("desc", false, "fileOrder");
ApiEndpoint endpoint = new ApiEndpoint("merge", post);
assertTrue(
endpoint.areParametersValid(Map.of()),
"Should be valid when optional param is omitted");
}
@Test
void mixed_required_and_optional_validates_only_required() {
ObjectNode post = mapper.createObjectNode();
post.put("description", "merge pdfs");
ArrayNode params = mapper.createArrayNode();
ObjectNode required = mapper.createObjectNode();
required.put("name", "sortType");
required.put("required", true);
params.add(required);
ObjectNode optional = mapper.createObjectNode();
optional.put("name", "fileOrder");
optional.put("required", false);
params.add(optional);
post.set("parameters", params);
ApiEndpoint endpoint = new ApiEndpoint("/api/v1/general/merge-pdfs", post);
Map<String, Object> provided = new HashMap<>();
provided.put("sortType", "byFileName");
assertTrue(
endpoint.areParametersValid(provided),
"Should pass when required params present and optional omitted");
provided.put("fileOrder", "0,1,2");
assertTrue(
endpoint.areParametersValid(provided),
"Should also pass when optional param is provided");
}
@Test
void missing_required_param_with_optional_present_still_fails() {
ObjectNode post = mapper.createObjectNode();
post.put("description", "desc");
ArrayNode params = mapper.createArrayNode();
ObjectNode required = mapper.createObjectNode();
required.put("name", "file");
required.put("required", true);
params.add(required);
ObjectNode optional = mapper.createObjectNode();
optional.put("name", "fileOrder");
optional.put("required", false);
params.add(optional);
post.set("parameters", params);
ApiEndpoint endpoint = new ApiEndpoint("x", post);
Map<String, Object> provided = new HashMap<>();
provided.put("fileOrder", "0,1");
assertFalse(
endpoint.areParametersValid(provided),
"Should fail when required param is missing even if optional is present");
}
@Test
void toString_contains_name_and_parameter_names() {
JsonNode post = postNodeWithParams("desc", "file", "mode");
@@ -74,15 +74,37 @@ class ApiDocServiceTest {
@Test
void isValidOperationChecksRequiredParameters() throws Exception {
String json =
"{\"description\": \"desc\", \"parameters\": [{\"name\":\"param1\"}, {\"name\":\"param2\"}]}";
"{\"description\": \"desc\", \"parameters\": [{\"name\":\"param1\", \"required\": true}, {\"name\":\"param2\", \"required\": true}]}";
JsonNode postNode = mapper.readTree(json);
ApiEndpoint endpoint = new ApiEndpoint("/op", postNode);
setApiDocumentation(Map.of("/op", endpoint));
setApiDocsJsonRootNode();
// All required params provided - valid
assertTrue(apiDocService.isValidOperation("/op", Map.of("param1", "a", "param2", "b")));
// Missing required param2 - invalid
assertFalse(apiDocService.isValidOperation("/op", Map.of("param1", "a")));
// Missing required param1 - invalid
assertFalse(apiDocService.isValidOperation("/op", Map.of("param2", "b")));
}
@Test
void isValidOperationAllowsOptionalParameters() throws Exception {
String json =
"{\"description\": \"desc\", \"parameters\": [{\"name\":\"param1\", \"required\": false}, {\"name\":\"param2\", \"required\": false}]}";
JsonNode postNode = mapper.readTree(json);
ApiEndpoint endpoint = new ApiEndpoint("/op", postNode);
setApiDocumentation(Map.of("/op", endpoint));
setApiDocsJsonRootNode();
// All optional params provided - valid
assertTrue(apiDocService.isValidOperation("/op", Map.of("param1", "a", "param2", "b")));
// Only one optional param provided - valid
assertTrue(apiDocService.isValidOperation("/op", Map.of("param1", "a")));
// No optional params provided - valid
assertTrue(apiDocService.isValidOperation("/op", Map.of()));
}
@Test
@@ -93,7 +93,7 @@ class LanguageServiceBasicTest {
// Configure the test service
((LanguageServiceForTest) languageService).setMockResources(mockResources);
// Allow only specific languages (en_GB is always included)
// Allow only specific languages - strict whitelist
when(applicationProperties.getUi().getLanguages())
.thenReturn(Arrays.asList("en_US", "fr_FR"));
@@ -103,7 +103,7 @@ class LanguageServiceBasicTest {
// Verify filtering by restrictions
assertTrue(supportedLanguages.contains("en_US"), "Allowed language should be included");
assertTrue(supportedLanguages.contains("fr_FR"), "Allowed language should be included");
assertTrue(supportedLanguages.contains("en_GB"), "en_GB should always be included");
assertFalse(supportedLanguages.contains("en_GB"), "en_GB should NOT be included when not in whitelist");
assertFalse(supportedLanguages.contains("de_DE"), "Restricted language should be excluded");
}
@@ -69,15 +69,15 @@ class LanguageServiceTest {
// Setup
Set<String> expectedLanguages =
new HashSet<>(Arrays.asList("en_US", "fr_FR", "de_DE", "en_GB"));
Set<String> allowedLanguages = new HashSet<>(Arrays.asList("en_US", "fr_FR", "en_GB"));
Set<String> allowedLanguages = new HashSet<>(Arrays.asList("en_US", "fr_FR"));
// Mock the resource resolver response
Resource[] mockResources = createMockResources(expectedLanguages);
((LanguageServiceForTest) languageService).setMockResources(mockResources);
// Set language restrictions in properties
// Set language restrictions in properties - strict whitelist only
when(applicationProperties.getUi().getLanguages())
.thenReturn(Arrays.asList("en_US", "fr_FR")); // en_GB is always allowed
.thenReturn(Arrays.asList("en_US", "fr_FR"));
// Test
Set<String> supportedLanguages = languageService.getSupportedLanguages();
@@ -86,8 +86,9 @@ class LanguageServiceTest {
assertEquals(
allowedLanguages,
supportedLanguages,
"Should return only allowed languages, plus en_GB which is always allowed");
assertTrue(supportedLanguages.contains("en_GB"), "en_GB should always be included");
"Should return only whitelisted languages");
assertFalse(supportedLanguages.contains("en_GB"), "en_GB should NOT be included when not in whitelist");
assertFalse(supportedLanguages.contains("de_DE"), "de_DE should NOT be included when not in whitelist");
}
@Test
@@ -27,7 +27,7 @@ import stirling.software.common.service.UserServiceInterface;
class PdfMetadataServiceBasicTest {
private PdfMetadataService pdfMetadataService;
private final String STIRLING_PDF_LABEL = "Stirling PDF";
private static final String STIRLING_PDF_LABEL = "Stirling PDF";
@BeforeEach
void setUp() {
@@ -33,7 +33,7 @@ class PdfMetadataServiceTest {
@Mock private ApplicationProperties applicationProperties;
@Mock private UserServiceInterface userService;
private PdfMetadataService pdfMetadataService;
private final String STIRLING_PDF_LABEL = "Stirling PDF";
private static final String STIRLING_PDF_LABEL = "Stirling PDF";
@BeforeEach
void setUp() {
@@ -28,8 +28,8 @@ class SignatureServiceTest {
private SharedSignatureService signatureService;
private Path personalSignatureFolder;
private Path sharedSignatureFolder;
private final String ALL_USERS_FOLDER = "ALL_USERS";
private final String TEST_USER = "testUser";
private static final String ALL_USERS_FOLDER = "ALL_USERS";
private static final String TEST_USER = "testUser";
@BeforeEach
void setUp() throws IOException {
@@ -15,11 +15,13 @@ import org.springframework.http.HttpStatus;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.mock.web.MockHttpSession;
import org.springframework.test.util.ReflectionTestUtils;
import jakarta.servlet.http.HttpServletRequest;
import stirling.software.common.model.job.JobResult;
import stirling.software.common.service.FileStorage;
import stirling.software.common.service.JobOwnershipService;
import stirling.software.common.service.JobQueue;
import stirling.software.common.service.TaskManager;
@@ -33,6 +35,8 @@ class JobControllerTest {
@Mock private HttpServletRequest request;
@Mock private JobOwnershipService jobOwnershipService;
private MockHttpSession session;
@InjectMocks private JobController controller;
@@ -404,4 +408,32 @@ class JobControllerTest {
verify(taskManager).setError(jobId, "Job was cancelled by user");
}
@Test
void testDownloadFile_ForbiddenWhenFileOwnedByAnotherUser() throws Exception {
String fileId = "file-id";
ReflectionTestUtils.setField(controller, "jobOwnershipService", jobOwnershipService);
when(taskManager.findJobKeyByFileId(fileId)).thenReturn("other-user:job-id");
when(jobOwnershipService.validateJobAccess("other-user:job-id")).thenReturn(false);
ResponseEntity<?> response = controller.downloadFile(fileId);
assertEquals(HttpStatus.FORBIDDEN, response.getStatusCode());
verify(fileStorage, never()).retrieveBytes(eq(fileId));
}
@Test
void testGetFileMetadata_ForbiddenWhenFileOwnedByAnotherUser() throws Exception {
String fileId = "file-id";
ReflectionTestUtils.setField(controller, "jobOwnershipService", jobOwnershipService);
when(taskManager.findJobKeyByFileId(fileId)).thenReturn("other-user:job-id");
when(jobOwnershipService.validateJobAccess("other-user:job-id")).thenReturn(false);
ResponseEntity<?> response = controller.getFileMetadata(fileId);
assertEquals(HttpStatus.FORBIDDEN, response.getStatusCode());
verify(fileStorage, never()).getFileSize(eq(fileId));
}
}
+5 -6
View File
@@ -1,5 +1,6 @@
repositories {
maven { url = "https://build.shibboleth.net/maven/releases" }
maven { url = "https://repository.jboss.org/" }
}
ext {
@@ -13,10 +14,10 @@ bootRun {
spotless {
java {
target 'src/**/java/**/*.java'
targetExclude 'src/main/java/org/apache/**'
googleJavaFormat(googleJavaFormatVersion).aosp().reorderImports(false)
importOrder("java", "javax", "org", "com", "net", "io", "jakarta", "lombok", "me", "stirling")
toggleOffOn()
trimTrailingWhitespace()
leadingTabsToSpaces()
endWithNewline()
@@ -36,6 +37,7 @@ spotless {
}
dependencies {
implementation project(':common')
api 'com.google.guava:guava:33.4.8-jre'
api 'org.springframework:spring-jdbc'
api 'org.springframework:spring-webmvc'
@@ -63,12 +65,9 @@ dependencies {
runtimeOnly "io.jsonwebtoken:jjwt-jackson:$jwtVersion"
runtimeOnly 'com.h2database:h2:2.3.232' // Don't upgrade h2database
runtimeOnly 'org.postgresql:postgresql:42.7.10'
constraints {
implementation "org.opensaml:opensaml-core:$openSamlVersion"
implementation "org.opensaml:opensaml-saml-api:$openSamlVersion"
implementation "org.opensaml:opensaml-saml-impl:$openSamlVersion"
implementation('com.coveo:saml-client:5.0.0') {
exclude group: 'org.opensaml', module: 'opensaml-core'
}
implementation 'com.coveo:saml-client:5.0.0'
}
tasks.register('prepareKotlinBuildScriptModel') {}
@@ -8,6 +8,7 @@ import org.aspectj.lang.ProceedingJoinPoint;
import org.aspectj.lang.annotation.Around;
import org.aspectj.lang.annotation.Aspect;
import org.aspectj.lang.reflect.MethodSignature;
import org.slf4j.MDC;
import org.springframework.stereotype.Component;
import org.springframework.web.context.request.RequestContextHolder;
import org.springframework.web.context.request.ServletRequestAttributes;
@@ -41,33 +42,67 @@ public class AuditAspect {
// Fast path: use unified check to determine if we should audit
// This avoids all data collection if auditing is disabled
if (!AuditUtils.shouldAudit(method, auditConfig)) {
if (!auditService.shouldAudit(method, auditConfig)) {
return joinPoint.proceed();
}
// EARLY CAPTURE: Try to get from MDC first (propagated from background threads)
// If not found, capture from SecurityContext on request thread
String capturedPrincipal = MDC.get("auditPrincipal");
if (capturedPrincipal == null) {
// Fallback: Capture from SecurityContext if running in request thread
capturedPrincipal = auditService.captureCurrentPrincipal();
}
String capturedOrigin = MDC.get("auditOrigin");
if (capturedOrigin == null) {
// Fallback: Capture from SecurityContext if running in request thread
capturedOrigin = auditService.captureCurrentOrigin();
}
ServletRequestAttributes attrs =
(ServletRequestAttributes) RequestContextHolder.getRequestAttributes();
HttpServletRequest req = attrs != null ? attrs.getRequest() : null;
String capturedIp = MDC.get("auditIp");
if (capturedIp == null) {
// Fallback: Try to extract from request if available
capturedIp = auditService.extractClientIp(req);
}
// Only create the map once we know we'll use it
Map<String, Object> auditData =
AuditUtils.createBaseAuditData(joinPoint, auditedAnnotation.level());
auditService.createBaseAuditData(joinPoint, auditedAnnotation.level());
// Add HTTP information if we're in a web context
ServletRequestAttributes attrs =
(ServletRequestAttributes) RequestContextHolder.getRequestAttributes();
if (attrs != null) {
HttpServletRequest req = attrs.getRequest();
String path = req.getRequestURI();
String httpMethod = req.getMethod();
AuditUtils.addHttpData(auditData, httpMethod, path, auditedAnnotation.level());
AuditUtils.addFileData(auditData, joinPoint, auditedAnnotation.level());
auditService.addHttpData(auditData, httpMethod, path, auditedAnnotation.level());
auditService.addFileData(auditData, joinPoint, auditedAnnotation.level());
// File operation details logged at DEBUG level for verification
if (auditData.containsKey("files") || auditData.containsKey("filename")) {
log.debug(
"@Audited method file operation - Principal: {}, Origin: {}, IP: {}, Method: {}, Path: {}, Files: {}",
capturedPrincipal,
capturedOrigin,
capturedIp,
httpMethod,
path,
auditData.getOrDefault("files", auditData.getOrDefault("filename", "N/A")));
}
if (auditData.containsKey("fileHash") || auditData.containsKey("hash")) {
log.debug(
"@Audited file hash captured - Hash: {}, Document: {}",
auditData.getOrDefault("fileHash", auditData.getOrDefault("hash", "N/A")),
auditData.getOrDefault("filename", "N/A"));
}
}
// Add arguments if requested and if at VERBOSE level, or if specifically requested
boolean includeArgs =
auditedAnnotation.includeArgs()
&& (auditedAnnotation.level() == AuditLevel.VERBOSE
|| auditConfig.getAuditLevel() == AuditLevel.VERBOSE);
if (includeArgs) {
AuditUtils.addMethodArguments(auditData, joinPoint, AuditLevel.VERBOSE);
// Add method arguments if requested (captured at all audit levels for operational context)
if (auditedAnnotation.includeArgs()) {
auditService.addMethodArguments(auditData, joinPoint, auditedAnnotation.level());
}
// Record start time for latency calculation
@@ -80,15 +115,14 @@ public class AuditAspect {
// Add success status
auditData.put("status", "success");
// Add result if requested and if at VERBOSE level
// Add result only if requested in annotation AND operation result capture is enabled
boolean includeResult =
auditedAnnotation.includeResult()
&& (auditedAnnotation.level() == AuditLevel.VERBOSE
|| auditConfig.getAuditLevel() == AuditLevel.VERBOSE);
&& auditService.shouldCaptureOperationResults();
if (includeResult && result != null) {
// Use safe string conversion with size limiting
auditData.put("result", AuditUtils.safeToString(result, 1000));
auditData.put("result", auditService.safeToString(result, 1000));
}
return result;
@@ -105,20 +139,19 @@ public class AuditAspect {
// methods
HttpServletResponse resp = attrs != null ? attrs.getResponse() : null;
boolean isHttpRequest = attrs != null;
AuditUtils.addTimingData(
auditService.addTimingData(
auditData, startTime, resp, auditedAnnotation.level(), isHttpRequest);
// Resolve the event type based on annotation and context
String httpMethod = null;
String path = null;
if (attrs != null) {
HttpServletRequest req = attrs.getRequest();
httpMethod = req.getMethod();
path = req.getRequestURI();
}
AuditEventType eventType =
AuditUtils.resolveEventType(
auditService.resolveEventType(
method,
joinPoint.getTarget().getClass(),
path,
@@ -128,11 +161,23 @@ public class AuditAspect {
// Check if we should use string type instead
String typeString = auditedAnnotation.typeString();
if (eventType == AuditEventType.HTTP_REQUEST && StringUtils.isNotEmpty(typeString)) {
// Use the string type (for backward compatibility)
auditService.audit(typeString, auditData, auditedAnnotation.level());
// Use the string type with early-captured values
auditService.audit(
capturedPrincipal,
capturedOrigin,
capturedIp,
typeString,
auditData,
auditedAnnotation.level());
} else {
// Use the enum type (preferred)
auditService.audit(eventType, auditData, auditedAnnotation.level());
// Use the enum type with early-captured values
auditService.audit(
capturedPrincipal,
capturedOrigin,
capturedIp,
eventType,
auditData,
auditedAnnotation.level());
}
}
}
@@ -22,6 +22,9 @@ public enum AuditEventType {
// PDF operations - STANDARD level
PDF_PROCESS("PDF processing operation"),
// UI data requests - STANDARD level
UI_DATA("UI data request"),
// HTTP requests - STANDARD level
HTTP_REQUEST("HTTP request");
@@ -49,7 +52,7 @@ public enum AuditEventType {
// If the exact enum name doesn't match, try finding a similar one
for (AuditEventType eventType : values()) {
if (eventType.name().equalsIgnoreCase(type)
|| eventType.getDescription().equalsIgnoreCase(type)) {
|| eventType.description.equalsIgnoreCase(type)) {
return eventType;
}
}
@@ -12,21 +12,26 @@ public enum AuditLevel {
OFF(0),
/**
* BASIC - Minimal audit logging (level 1) Includes: - Authentication events (login, logout,
* failed logins) - Password changes - User/role changes - System configuration changes
* BASIC - File modifications only (level 1) Tracks: PDF file operations like compress, split,
* merge, etc., and settings changes. Captures: Operation status (success/failure), method
* parameters, timing. Ideal for: Compliance tracking of file modifications with minimal log
* volume.
*/
BASIC(1),
/**
* STANDARD - Standard audit logging (level 2) Includes everything in BASIC plus: - All HTTP
* requests (basic info: URL, method, status) - File operations (upload, download, process) -
* PDF operations (view, edit, etc.) - User operations
* STANDARD - File operations and user actions (level 2) Tracks: Everything in BASIC plus user
* actions like login/logout, account changes, and general GET requests. Excludes continuous
* polling calls (e.g., auth/me, app-config, health, metrics endpoints). Ideal for: General
* audit trail with reasonable log volume for most deployments.
*/
STANDARD(2),
/**
* VERBOSE - Detailed audit logging (level 3) Includes everything in STANDARD plus: - Request
* headers and parameters - Method parameters - Operation results - Detailed timing information
* VERBOSE - Everything including polling (level 3) Tracks: Everything in STANDARD plus
* continuous polling calls and all GET requests. Captures: Detailed timing information. Note:
* Operation results (return values) are controlled by separate captureOperationResults flag.
* Warning: High log volume and performance impact.
*/
VERBOSE(3);
@@ -1,427 +0,0 @@
package stirling.software.proprietary.audit;
import java.lang.reflect.Method;
import java.time.Instant;
import java.util.Arrays;
import java.util.HashMap;
import java.util.List;
import java.util.Locale;
import java.util.Map;
import java.util.stream.Collectors;
import java.util.stream.IntStream;
import org.apache.commons.lang3.StringUtils;
import org.aspectj.lang.ProceedingJoinPoint;
import org.aspectj.lang.reflect.MethodSignature;
import org.slf4j.MDC;
import org.springframework.http.MediaType;
import org.springframework.security.core.Authentication;
import org.springframework.security.core.context.SecurityContextHolder;
import org.springframework.web.context.request.RequestContextHolder;
import org.springframework.web.context.request.ServletRequestAttributes;
import org.springframework.web.multipart.MultipartFile;
import jakarta.servlet.http.HttpServletRequest;
import jakarta.servlet.http.HttpServletResponse;
import lombok.extern.slf4j.Slf4j;
import stirling.software.common.util.RegexPatternUtils;
import stirling.software.common.util.RequestUriUtils;
import stirling.software.proprietary.config.AuditConfigurationProperties;
/**
* Shared utilities for audit aspects to ensure consistent behavior across different audit
* mechanisms.
*/
@Slf4j
public class AuditUtils {
/**
* Create a standard audit data map with common attributes based on the current audit level
*
* @param joinPoint The AspectJ join point
* @param auditLevel The current audit level
* @return A map with standard audit data
*/
public static Map<String, Object> createBaseAuditData(
ProceedingJoinPoint joinPoint, AuditLevel auditLevel) {
Map<String, Object> data = new HashMap<>();
// Common data for all levels
data.put("timestamp", Instant.now().toString());
// Add principal if available
Authentication auth = SecurityContextHolder.getContext().getAuthentication();
if (auth != null && auth.getName() != null) {
data.put("principal", auth.getName());
} else {
data.put("principal", "system");
}
// Add class name and method name only at VERBOSE level
if (auditLevel.includes(AuditLevel.VERBOSE)) {
data.put("className", joinPoint.getTarget().getClass().getName());
data.put(
"methodName",
((MethodSignature) joinPoint.getSignature()).getMethod().getName());
}
return data;
}
/**
* Add HTTP-specific information to the audit data if available
*
* @param data The existing audit data map
* @param httpMethod The HTTP method (GET, POST, etc.)
* @param path The request path
* @param auditLevel The current audit level
*/
public static void addHttpData(
Map<String, Object> data, String httpMethod, String path, AuditLevel auditLevel) {
if (httpMethod == null || path == null) {
return; // Skip if we don't have basic HTTP info
}
// BASIC level HTTP data
data.put("httpMethod", httpMethod);
data.put("path", path);
// Get request attributes safely
ServletRequestAttributes attrs =
(ServletRequestAttributes) RequestContextHolder.getRequestAttributes();
if (attrs == null) {
return; // No request context available
}
HttpServletRequest req = attrs.getRequest();
if (req == null) {
return; // No request available
}
// STANDARD level HTTP data
if (auditLevel.includes(AuditLevel.STANDARD)) {
data.put("clientIp", req.getRemoteAddr());
data.put(
"sessionId",
req.getSession(false) != null ? req.getSession(false).getId() : null);
data.put("requestId", MDC.get("requestId"));
// Form data for POST/PUT/PATCH
if (("POST".equalsIgnoreCase(httpMethod)
|| "PUT".equalsIgnoreCase(httpMethod)
|| "PATCH".equalsIgnoreCase(httpMethod))
&& req.getContentType() != null) {
String contentType = req.getContentType();
if (contentType.contains(MediaType.APPLICATION_FORM_URLENCODED_VALUE)
|| contentType.contains(MediaType.MULTIPART_FORM_DATA_VALUE)) {
Map<String, String[]> params = new HashMap<>(req.getParameterMap());
// Remove CSRF token from logged parameters
params.remove("_csrf");
if (!params.isEmpty()) {
data.put("formParams", params);
}
}
}
}
}
/**
* Add file information to the audit data if available
*
* @param data The existing audit data map
* @param joinPoint The AspectJ join point
* @param auditLevel The current audit level
*/
public static void addFileData(
Map<String, Object> data, ProceedingJoinPoint joinPoint, AuditLevel auditLevel) {
if (auditLevel.includes(AuditLevel.STANDARD)) {
List<MultipartFile> files =
Arrays.stream(joinPoint.getArgs())
.filter(a -> a instanceof MultipartFile)
.map(a -> (MultipartFile) a)
.collect(Collectors.toList());
if (!files.isEmpty()) {
List<Map<String, Object>> fileInfos =
files.stream()
.map(
f -> {
Map<String, Object> m = new HashMap<>();
m.put("name", f.getOriginalFilename());
m.put("size", f.getSize());
m.put("type", f.getContentType());
return m;
})
.collect(Collectors.toList());
data.put("files", fileInfos);
}
}
}
/**
* Add method arguments to the audit data
*
* @param data The existing audit data map
* @param joinPoint The AspectJ join point
* @param auditLevel The current audit level
*/
public static void addMethodArguments(
Map<String, Object> data, ProceedingJoinPoint joinPoint, AuditLevel auditLevel) {
if (auditLevel.includes(AuditLevel.VERBOSE)) {
MethodSignature sig = (MethodSignature) joinPoint.getSignature();
String[] names = sig.getParameterNames();
Object[] vals = joinPoint.getArgs();
if (names != null && vals != null) {
IntStream.range(0, names.length)
.forEach(
i -> {
if (vals[i] != null) {
// Convert objects to safe string representation
data.put("arg_" + names[i], safeToString(vals[i], 500));
} else {
data.put("arg_" + names[i], null);
}
});
}
}
}
/**
* Safely convert an object to string with size limiting
*
* @param obj The object to convert
* @param maxLength Maximum length of the resulting string
* @return A safe string representation, truncated if needed
*/
public static String safeToString(Object obj, int maxLength) {
if (obj == null) {
return "null";
}
String result;
try {
// Handle common types directly to avoid toString() overhead
if (obj instanceof String) {
result = (String) obj;
} else if (obj instanceof Number || obj instanceof Boolean) {
result = obj.toString();
} else if (obj instanceof byte[]) {
result = "[binary data length=" + ((byte[]) obj).length + "]";
} else {
// For complex objects, use toString but handle exceptions
result = obj.toString();
}
// Truncate if necessary
if (result != null && result.length() > maxLength) {
return StringUtils.truncate(result, maxLength - 3) + "...";
}
return result;
} catch (Exception e) {
// If toString() fails, return the class name
return "[" + obj.getClass().getName() + " - toString() failed]";
}
}
/**
* Determine if a method should be audited based on config and annotation
*
* @param method The method to check
* @param auditConfig The audit configuration
* @return true if the method should be audited
*/
public static boolean shouldAudit(Method method, AuditConfigurationProperties auditConfig) {
// First check if audit is globally enabled - fast path
if (!auditConfig.isEnabled()) {
return false;
}
// Check for annotation override
Audited auditedAnnotation = method.getAnnotation(Audited.class);
AuditLevel requiredLevel =
(auditedAnnotation != null) ? auditedAnnotation.level() : AuditLevel.BASIC;
// Check if the required level is enabled
return auditConfig.getAuditLevel().includes(requiredLevel);
}
/**
* Add timing and response status data to the audit record
*
* @param data The audit data to add to
* @param startTime The start time in milliseconds
* @param response The HTTP response (may be null for non-HTTP methods)
* @param level The current audit level
* @param isHttpRequest Whether this is an HTTP request (controller) or a regular method call
*/
public static void addTimingData(
Map<String, Object> data,
long startTime,
HttpServletResponse response,
AuditLevel level,
boolean isHttpRequest) {
if (level.includes(AuditLevel.STANDARD)) {
// For HTTP requests, let ControllerAuditAspect handle timing separately
// For non-HTTP methods, add execution time here
if (!isHttpRequest) {
data.put("latencyMs", System.currentTimeMillis() - startTime);
}
// Add HTTP status code if available
if (response != null) {
try {
data.put("statusCode", response.getStatus());
} catch (Exception e) {
// Ignore - response might be in an inconsistent state
}
}
}
}
/**
* Resolve the event type to use for auditing, considering annotations and context
*
* @param method The method being audited
* @param controller The controller class
* @param path The request path (may be null for non-HTTP methods)
* @param httpMethod The HTTP method (may be null for non-HTTP methods)
* @param annotation The @Audited annotation (may be null)
* @return The resolved event type (never null)
*/
public static AuditEventType resolveEventType(
Method method,
Class<?> controller,
String path,
String httpMethod,
Audited annotation) {
// First check if we have an explicit annotation
if (annotation != null && annotation.type() != AuditEventType.HTTP_REQUEST) {
return annotation.type();
}
// For HTTP methods, infer based on controller and path
if (httpMethod != null && path != null) {
String cls = controller.getSimpleName().toLowerCase(Locale.ROOT);
String pkg = controller.getPackage().getName().toLowerCase(Locale.ROOT);
if ("GET".equals(httpMethod)) return AuditEventType.HTTP_REQUEST;
if (cls.contains("user")
|| cls.contains("auth")
|| pkg.contains("auth")
|| path.startsWith("/user")
|| path.startsWith("/login")) {
return AuditEventType.USER_PROFILE_UPDATE;
} else if (cls.contains("admin")
|| path.startsWith("/admin")
|| path.startsWith("/settings")) {
return AuditEventType.SETTINGS_CHANGED;
} else if (cls.contains("file")
|| path.startsWith("/file")
|| RegexPatternUtils.getInstance()
.getUploadDownloadPathPattern()
.matcher(path)
.matches()) {
return AuditEventType.FILE_OPERATION;
}
}
// Default for non-HTTP methods or when no specific match
return AuditEventType.PDF_PROCESS;
}
/**
* Determine the appropriate audit level to use
*
* @param method The method to check
* @param defaultLevel The default level to use if no annotation present
* @param auditConfig The audit configuration
* @return The audit level to use
*/
public static AuditLevel getEffectiveAuditLevel(
Method method, AuditLevel defaultLevel, AuditConfigurationProperties auditConfig) {
Audited auditedAnnotation = method.getAnnotation(Audited.class);
if (auditedAnnotation != null) {
// Method has @Audited - use its level
return auditedAnnotation.level();
}
// Use default level (typically from global config)
return defaultLevel;
}
/**
* Determine the appropriate audit event type to use
*
* @param method The method being audited
* @param controller The controller class
* @param path The request path
* @param httpMethod The HTTP method
* @return The determined audit event type
*/
public static AuditEventType determineAuditEventType(
Method method, Class<?> controller, String path, String httpMethod) {
// First check for explicit annotation
Audited auditedAnnotation = method.getAnnotation(Audited.class);
if (auditedAnnotation != null) {
return auditedAnnotation.type();
}
// Otherwise infer from controller and path
String cls = controller.getSimpleName().toLowerCase(Locale.ROOT);
String pkg = controller.getPackage().getName().toLowerCase(Locale.ROOT);
if ("GET".equals(httpMethod)) return AuditEventType.HTTP_REQUEST;
if (cls.contains("user")
|| cls.contains("auth")
|| pkg.contains("auth")
|| path.startsWith("/user")
|| path.startsWith("/login")) {
return AuditEventType.USER_PROFILE_UPDATE;
} else if (cls.contains("admin")
|| path.startsWith("/admin")
|| path.startsWith("/settings")) {
return AuditEventType.SETTINGS_CHANGED;
} else if (cls.contains("file")
|| path.startsWith("/file")
|| RegexPatternUtils.getInstance()
.getUploadDownloadPathPattern()
.matcher(path)
.matches()) {
return AuditEventType.FILE_OPERATION;
} else {
return AuditEventType.PDF_PROCESS;
}
}
/**
* Get the current HTTP request if available
*
* @return The current request or null if not in a request context
*/
public static HttpServletRequest getCurrentRequest() {
ServletRequestAttributes attrs =
(ServletRequestAttributes) RequestContextHolder.getRequestAttributes();
return attrs != null ? attrs.getRequest() : null;
}
/**
* Check if a GET request is for a static resource
*
* @param request The HTTP request
* @return true if this is a static resource request
*/
public static boolean isStaticResourceRequest(HttpServletRequest request) {
return request != null
&& !RequestUriUtils.isTrackableResource(
request.getContextPath(), request.getRequestURI());
}
}
@@ -9,6 +9,7 @@ import org.aspectj.lang.ProceedingJoinPoint;
import org.aspectj.lang.annotation.Around;
import org.aspectj.lang.annotation.Aspect;
import org.aspectj.lang.reflect.MethodSignature;
import org.slf4j.MDC;
import org.springframework.stereotype.Component;
import org.springframework.web.bind.annotation.DeleteMapping;
import org.springframework.web.bind.annotation.GetMapping;
@@ -37,7 +38,7 @@ import stirling.software.proprietary.service.AuditService;
@Slf4j
@RequiredArgsConstructor
@org.springframework.core.annotation.Order(
10) // Lower precedence (higher number) - executes after AutoJobAspect
0) // Highest precedence - runs BEFORE AutoJobAspect to populate MDC
public class ControllerAuditAspect {
private final AuditService auditService;
@@ -92,7 +93,7 @@ public class ControllerAuditAspect {
// Fast path: check if auditing is enabled before doing any work
// This avoids all data collection if auditing is disabled
if (!AuditUtils.shouldAudit(method, auditConfig)) {
if (!auditService.shouldAudit(method, auditConfig)) {
return joinPoint.proceed();
}
@@ -110,8 +111,14 @@ public class ControllerAuditAspect {
// Skip static GET resources
if ("GET".equals(httpMethod)) {
HttpServletRequest maybe = AuditUtils.getCurrentRequest();
if (maybe != null && AuditUtils.isStaticResourceRequest(maybe)) {
HttpServletRequest maybe = auditService.getCurrentRequest();
if (maybe != null && auditService.isStaticResourceRequest(maybe)) {
return joinPoint.proceed();
}
// Skip polling calls at STANDARD level (exclude from audit log noise)
if (maybe != null
&& auditService.isPollingCall(maybe)
&& auditConfig.getAuditLevel() == AuditLevel.STANDARD) {
return joinPoint.proceed();
}
}
@@ -121,75 +128,143 @@ public class ControllerAuditAspect {
HttpServletRequest req = attrs != null ? attrs.getRequest() : null;
HttpServletResponse resp = attrs != null ? attrs.getResponse() : null;
long start = System.currentTimeMillis();
String previousPrincipal = MDC.get("auditPrincipal");
String previousOrigin = MDC.get("auditOrigin");
String previousIp = MDC.get("auditIp");
// Use AuditUtils to create the base audit data
Map<String, Object> data = AuditUtils.createBaseAuditData(joinPoint, level);
// Add HTTP-specific information
AuditUtils.addHttpData(data, httpMethod, path, level);
// Add file information if present
AuditUtils.addFileData(data, joinPoint, level);
// Add method arguments if at VERBOSE level
if (level.includes(AuditLevel.VERBOSE)) {
AuditUtils.addMethodArguments(data, joinPoint, level);
// EARLY CAPTURE: Capture from SecurityContext on request thread, store in MDC for async
// propagation
// MDC.put is necessary for background threads to inherit audit context
String capturedPrincipal = previousPrincipal;
if (capturedPrincipal == null) {
capturedPrincipal = auditService.captureCurrentPrincipal();
MDC.put("auditPrincipal", capturedPrincipal);
}
Object result = null;
String capturedOrigin = previousOrigin;
if (capturedOrigin == null) {
capturedOrigin = auditService.captureCurrentOrigin();
MDC.put("auditOrigin", capturedOrigin);
}
String capturedIp = previousIp;
if (capturedIp == null && req != null) {
capturedIp = auditService.extractClientIp(req);
if (capturedIp != null) {
MDC.put("auditIp", capturedIp);
}
}
try {
result = joinPoint.proceed();
data.put("outcome", "success");
} catch (Throwable ex) {
data.put("outcome", "failure");
data.put("errorType", ex.getClass().getSimpleName());
data.put("errorMessage", ex.getMessage());
throw ex;
} finally {
// Handle timing directly for HTTP requests
if (level.includes(AuditLevel.STANDARD)) {
data.put("latencyMs", System.currentTimeMillis() - start);
if (resp != null) data.put("statusCode", resp.getStatus());
}
// Call AuditUtils but with isHttpRequest=true to skip additional timing
AuditUtils.addTimingData(data, start, resp, level, true);
// Add result for VERBOSE level
if (level.includes(AuditLevel.VERBOSE) && result != null) {
// Use safe string conversion with size limiting
data.put("result", AuditUtils.safeToString(result, 1000));
}
// Resolve the event type using the unified method
AuditEventType eventType =
AuditUtils.resolveEventType(
method,
joinPoint.getTarget().getClass(),
path,
httpMethod,
auditedAnnotation);
// Check if we should use string type instead (for backward compatibility)
// Avoid duplicate events for controller methods explicitly annotated with @Audited.
// @Audited methods are audited by AuditAspect.
if (auditedAnnotation != null) {
String typeString = auditedAnnotation.typeString();
if (eventType == AuditEventType.HTTP_REQUEST
&& StringUtils.isNotEmpty(typeString)) {
auditService.audit(typeString, data, level);
return result;
return joinPoint.proceed();
}
long start = System.currentTimeMillis();
// Use auditService to create the base audit data
Map<String, Object> data = auditService.createBaseAuditData(joinPoint, level);
// Add HTTP-specific information
auditService.addHttpData(data, httpMethod, path, level);
// Add file information if present
auditService.addFileData(data, joinPoint, level);
// Add method arguments if at VERBOSE level
if (level.includes(AuditLevel.VERBOSE)) {
auditService.addMethodArguments(data, joinPoint, level);
}
Object result = null;
try {
result = joinPoint.proceed();
data.put("outcome", "success");
} catch (Throwable ex) {
data.put("outcome", "failure");
data.put("errorType", ex.getClass().getSimpleName());
data.put("errorMessage", ex.getMessage());
throw ex;
} finally {
// Handle timing directly for HTTP requests
if (level.includes(AuditLevel.STANDARD)) {
data.put("latencyMs", System.currentTimeMillis() - start);
if (resp != null) data.put("statusCode", resp.getStatus());
}
// Call auditService but with isHttpRequest=true to skip additional timing
auditService.addTimingData(data, start, resp, level, true);
// Resolve the event type using the unified method
AuditEventType eventType =
auditService.resolveEventType(
method,
joinPoint.getTarget().getClass(),
path,
httpMethod,
auditedAnnotation);
// Add result only if operation result capture is explicitly enabled
// Skip result for UI_DATA events to avoid storing large response bodies
if (auditService.shouldCaptureOperationResults()
&& result != null
&& eventType != AuditEventType.UI_DATA) {
// Use safe string conversion with size limiting
data.put("result", auditService.safeToString(result, 1000));
}
// Check if we should use string type instead (for backward compatibility)
if (auditedAnnotation != null) {
String typeString = auditedAnnotation.typeString();
if (eventType == AuditEventType.HTTP_REQUEST
&& StringUtils.isNotEmpty(typeString)) {
auditService.audit(
capturedPrincipal,
capturedOrigin,
capturedIp,
typeString,
data,
level);
} else {
// Use the enum type with early-captured values
auditService.audit(
capturedPrincipal,
capturedOrigin,
capturedIp,
eventType,
data,
level);
}
} else {
// Use the enum type with early-captured values
auditService.audit(
capturedPrincipal, capturedOrigin, capturedIp, eventType, data, level);
}
}
// Use the enum type
auditService.audit(eventType, data, level);
return result;
} finally {
restoreMdcValue("auditPrincipal", previousPrincipal);
restoreMdcValue("auditOrigin", previousOrigin);
restoreMdcValue("auditIp", previousIp);
}
return result;
}
// Using AuditUtils.determineAuditEventType instead
private String getRequestPath(Method method, String httpMethod) {
// Prefer actual request URI over annotation patterns (which may contain regex)
ServletRequestAttributes attrs =
(ServletRequestAttributes) RequestContextHolder.getRequestAttributes();
if (attrs != null) {
HttpServletRequest request = attrs.getRequest();
if (request != null) {
return request.getRequestURI();
}
}
// Fallback: reconstruct from annotations when not in web context
String base = "";
RequestMapping cm = method.getDeclaringClass().getAnnotation(RequestMapping.class);
if (cm != null && cm.value().length > 0) base = cm.value()[0];
@@ -211,5 +286,13 @@ public class ControllerAuditAspect {
return base + mp;
}
private void restoreMdcValue(String key, String previousValue) {
if (previousValue != null) {
MDC.put(key, previousValue);
} else {
MDC.remove(key);
}
}
// Using AuditUtils.getCurrentRequest instead
}
@@ -23,6 +23,9 @@ public class AuditConfigurationProperties {
private final boolean enabled;
private final int level;
private final int retentionDays;
private final boolean captureFileHash;
private final boolean capturePdfAuthor;
private final boolean captureOperationResults;
public AuditConfigurationProperties(ApplicationProperties applicationProperties) {
ApplicationProperties.Premium.EnterpriseFeatures.Audit auditConfig =
@@ -37,11 +40,19 @@ public class AuditConfigurationProperties {
// Retention days (0 means infinite)
this.retentionDays = auditConfig.getRetentionDays();
// Metadata and detail capture flags
this.captureFileHash = auditConfig.isCaptureFileHash();
this.capturePdfAuthor = auditConfig.isCapturePdfAuthor();
this.captureOperationResults = auditConfig.isCaptureOperationResults();
log.debug(
"Initialized audit configuration: enabled={}, level={}, retentionDays={} (0=infinite)",
"Initialized audit configuration: enabled={}, level={}, retentionDays={} (0=infinite), fileHash={}, pdfAuthor={}, operationResults={}",
this.enabled,
this.level,
this.retentionDays);
this.retentionDays,
this.captureFileHash,
this.capturePdfAuthor,
this.captureOperationResults);
}
/**
@@ -1,5 +1,6 @@
package stirling.software.proprietary.controller.api;
import java.nio.charset.StandardCharsets;
import java.time.Instant;
import java.time.LocalDate;
import java.time.LocalDateTime;
@@ -18,6 +19,7 @@ import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.security.access.prepost.PreAuthorize;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestParam;
import lombok.RequiredArgsConstructor;
@@ -44,12 +46,13 @@ public class AuditRestController {
private final ObjectMapper objectMapper;
/**
* Get audit events with pagination and filters. Maps to frontend's getEvents() call.
* Get audit events with pagination and filters. Maps to frontend's getEvents() call. Supports
* both single values and multi-select arrays for eventType and username.
*
* @param page Page number (0-indexed)
* @param pageSize Number of items per page
* @param eventType Filter by event type
* @param username Filter by username (principal)
* @param eventType Filter by event type(s) - can be single value or array
* @param username Filter by username(s) - can be single value or array
* @param startDate Filter start date
* @param endDate Filter end date
* @return Paginated audit events response
@@ -58,8 +61,8 @@ public class AuditRestController {
public ResponseEntity<AuditEventsResponse> getAuditEvents(
@RequestParam(value = "page", defaultValue = "0") int page,
@RequestParam(value = "pageSize", defaultValue = "30") int pageSize,
@RequestParam(value = "eventType", required = false) String eventType,
@RequestParam(value = "username", required = false) String username,
@RequestParam(value = "eventType", required = false) String[] eventTypes,
@RequestParam(value = "username", required = false) String[] usernames,
@RequestParam(value = "startDate", required = false)
@DateTimeFormat(iso = DateTimeFormat.ISO.DATE)
LocalDate startDate,
@@ -70,33 +73,45 @@ public class AuditRestController {
Pageable pageable = PageRequest.of(page, pageSize, Sort.by("timestamp").descending());
Page<PersistentAuditEvent> events;
// Convert arrays to lists
List<String> eventTypeList =
(eventTypes != null && eventTypes.length > 0) ? Arrays.asList(eventTypes) : null;
List<String> usernameList =
(usernames != null && usernames.length > 0) ? Arrays.asList(usernames) : null;
Instant startInstant = null;
Instant endInstant = null;
if (startDate != null && endDate != null) {
startInstant = startDate.atStartOfDay(ZoneId.systemDefault()).toInstant();
endInstant = endDate.plusDays(1).atStartOfDay(ZoneId.systemDefault()).toInstant();
}
// Apply filters based on provided parameters
if (eventType != null && username != null && startDate != null && endDate != null) {
Instant start = startDate.atStartOfDay(ZoneId.systemDefault()).toInstant();
Instant end = endDate.plusDays(1).atStartOfDay(ZoneId.systemDefault()).toInstant();
if (eventTypeList != null
&& usernameList != null
&& startInstant != null
&& endInstant != null) {
events =
auditRepository.findByPrincipalAndTypeAndTimestampBetween(
username, eventType, start, end, pageable);
} else if (eventType != null && username != null) {
events = auditRepository.findByPrincipalAndType(username, eventType, pageable);
} else if (eventType != null && startDate != null && endDate != null) {
Instant start = startDate.atStartOfDay(ZoneId.systemDefault()).toInstant();
Instant end = endDate.plusDays(1).atStartOfDay(ZoneId.systemDefault()).toInstant();
events = auditRepository.findByTypeAndTimestampBetween(eventType, start, end, pageable);
} else if (username != null && startDate != null && endDate != null) {
Instant start = startDate.atStartOfDay(ZoneId.systemDefault()).toInstant();
Instant end = endDate.plusDays(1).atStartOfDay(ZoneId.systemDefault()).toInstant();
auditRepository.findByTypeInAndPrincipalInAndTimestampBetween(
eventTypeList, usernameList, startInstant, endInstant, pageable);
} else if (eventTypeList != null && usernameList != null) {
events =
auditRepository.findByPrincipalAndTimestampBetween(
username, start, end, pageable);
} else if (startDate != null && endDate != null) {
Instant start = startDate.atStartOfDay(ZoneId.systemDefault()).toInstant();
Instant end = endDate.plusDays(1).atStartOfDay(ZoneId.systemDefault()).toInstant();
events = auditRepository.findByTimestampBetween(start, end, pageable);
} else if (eventType != null) {
events = auditRepository.findByType(eventType, pageable);
} else if (username != null) {
events = auditRepository.findByPrincipal(username, pageable);
auditRepository.findByTypeInAndPrincipalIn(
eventTypeList, usernameList, pageable);
} else if (eventTypeList != null && startInstant != null && endInstant != null) {
events =
auditRepository.findByTypeInAndTimestampBetween(
eventTypeList, startInstant, endInstant, pageable);
} else if (usernameList != null && startInstant != null && endInstant != null) {
events =
auditRepository.findByPrincipalInAndTimestampBetween(
usernameList, startInstant, endInstant, pageable);
} else if (startInstant != null && endInstant != null) {
events = auditRepository.findByTimestampBetween(startInstant, endInstant, pageable);
} else if (eventTypeList != null) {
events = auditRepository.findByTypeIn(eventTypeList, pageable);
} else if (usernameList != null) {
events = auditRepository.findByPrincipalIn(usernameList, pageable);
} else {
events = auditRepository.findAll(pageable);
}
@@ -258,11 +273,253 @@ public class AuditRestController {
}
/**
* Export audit data in CSV or JSON format. Maps to frontend's exportData() call.
* Get audit statistics for KPI dashboard. Includes success rates, latency metrics, and top
* items.
*
* @param period Time period for statistics (day/week/month)
* @return Audit statistics data for dashboard KPI cards and enhanced charts
*/
@GetMapping("/audit-stats")
public ResponseEntity<AuditStatsData> getAuditStats(
@RequestParam(value = "period", defaultValue = "week") String period) {
// Calculate days based on period
int days;
switch (period.toLowerCase()) {
case "day":
days = 1;
break;
case "month":
days = 30;
break;
case "week":
default:
days = 7;
break;
}
// Get events from the specified period and previous period
Instant now = Instant.now();
Instant start = now.minus(java.time.Duration.ofDays(days));
Instant prevStart = start.minus(java.time.Duration.ofDays(days));
List<PersistentAuditEvent> currentEvents = auditRepository.findByTimestampAfter(start);
List<PersistentAuditEvent> prevEvents =
auditRepository.findAllByTimestampBetweenForExport(prevStart, start);
// Compute metrics for current period
AuditMetrics currentMetrics = computeMetrics(currentEvents);
AuditMetrics prevMetrics = computeMetrics(prevEvents);
// Get hourly distribution using DB aggregation
List<Object[]> hourlyData = auditRepository.histogramByHourBetween(start, now);
Map<String, Long> hourlyDistribution = new TreeMap<>();
for (int h = 0; h < 24; h++) {
hourlyDistribution.put(String.format("%02d", h), 0L);
}
for (Object[] row : hourlyData) {
int hour = ((Number) row[0]).intValue();
long count = ((Number) row[1]).longValue();
hourlyDistribution.put(String.format("%02d", hour), count);
}
return ResponseEntity.ok(
AuditStatsData.builder()
.totalEvents(currentMetrics.totalEvents)
.prevTotalEvents(prevMetrics.totalEvents)
.uniqueUsers(currentMetrics.uniqueUsers)
.prevUniqueUsers(prevMetrics.uniqueUsers)
.successRate(currentMetrics.successRate)
.prevSuccessRate(prevMetrics.successRate)
.avgLatencyMs(currentMetrics.avgLatencyMs)
.prevAvgLatencyMs(prevMetrics.avgLatencyMs)
.errorCount(currentMetrics.errorCount)
.topEventType(currentMetrics.topEventType)
.topUser(currentMetrics.topUser)
.eventsByType(currentMetrics.eventsByType)
.eventsByUser(currentMetrics.eventsByUser)
.topTools(currentMetrics.topTools)
.hourlyDistribution(hourlyDistribution)
.build());
}
/** Compute metrics from a list of audit events. */
private AuditMetrics computeMetrics(List<PersistentAuditEvent> events) {
if (events.isEmpty()) {
return AuditMetrics.builder().build();
}
// Count by type
Map<String, Long> eventsByType =
events.stream()
.collect(
Collectors.groupingBy(
PersistentAuditEvent::getType, Collectors.counting()));
// Count by principal (user)
Map<String, Long> eventsByUser =
events.stream()
.collect(
Collectors.groupingBy(
PersistentAuditEvent::getPrincipal, Collectors.counting()));
// Parse JSON data once for success rate, latency, tool extraction, and error counting
long successCount = 0;
long failureCount = 0;
long errorCount = 0;
long totalLatencyMs = 0;
long latencyCount = 0;
Map<String, Long> topTools = new HashMap<>();
for (PersistentAuditEvent event : events) {
if (event.getData() != null) {
try {
@SuppressWarnings("unchecked")
Map<String, Object> data = objectMapper.readValue(event.getData(), Map.class);
// Track success/failure (safe type conversion)
// Check both "status" (current) and "outcome" (legacy) for compatibility
Object statusObj = data.get("status");
if (statusObj == null) {
statusObj = data.get("outcome");
}
String status = null;
if (statusObj instanceof String) {
status = (String) statusObj;
} else if (statusObj != null) {
status = String.valueOf(statusObj);
}
if ("success".equals(status)) {
successCount++;
} else if ("failure".equals(status)) {
failureCount++;
errorCount++;
} else {
// Check statusCode for error counting (when status is not explicit failure)
Object statusCode = data.get("statusCode");
if (statusCode != null) {
try {
int statusCodeVal;
if (statusCode instanceof Number) {
statusCodeVal = ((Number) statusCode).intValue();
} else if (statusCode instanceof String) {
statusCodeVal = Integer.parseInt((String) statusCode);
} else {
statusCodeVal = 0;
}
if (statusCodeVal >= 400) {
errorCount++;
}
} catch (NumberFormatException e) {
log.trace("Failed to parse statusCode value: {}", statusCode);
}
}
}
// Track latency (safe conversion to handle strings/numbers)
Object latency = data.get("latencyMs");
if (latency != null) {
try {
long latencyVal;
if (latency instanceof Number) {
latencyVal = ((Number) latency).longValue();
} else if (latency instanceof String) {
latencyVal = Long.parseLong((String) latency);
} else {
latencyVal = 0;
}
totalLatencyMs += latencyVal;
latencyCount++;
} catch (NumberFormatException e) {
log.trace("Failed to parse latency value: {}", latency);
}
}
// Extract tool from path (safe type conversion)
Object pathObj = data.get("path");
String path = null;
if (pathObj instanceof String) {
path = (String) pathObj;
} else if (pathObj != null) {
path = String.valueOf(pathObj);
}
if (path != null && !path.isEmpty()) {
String[] parts = path.split("/");
if (parts.length > 0) {
String tool = parts[parts.length - 1];
if (!tool.isEmpty()) {
topTools.put(tool, topTools.getOrDefault(tool, 0L) + 1);
}
}
}
} catch (JacksonException e) {
log.trace("Failed to parse audit event data: {}", event.getData());
}
}
}
// Calculate success rate
double successRate = 0;
long totalWithOutcome = successCount + failureCount;
if (totalWithOutcome > 0) {
successRate = (successCount * 100.0) / totalWithOutcome;
}
// Calculate average latency
double avgLatencyMs = 0;
if (latencyCount > 0) {
avgLatencyMs = totalLatencyMs / (double) latencyCount;
}
// Get top event type
String topEventType =
eventsByType.entrySet().stream()
.max((e1, e2) -> Long.compare(e1.getValue(), e2.getValue()))
.map(Map.Entry::getKey)
.orElse("");
// Get top user
String topUser =
eventsByUser.entrySet().stream()
.max((e1, e2) -> Long.compare(e1.getValue(), e2.getValue()))
.map(Map.Entry::getKey)
.orElse("");
// Sort and limit top tools to 10
Map<String, Long> topToolsSorted =
topTools.entrySet().stream()
.sorted((e1, e2) -> Long.compare(e2.getValue(), e1.getValue()))
.limit(10)
.collect(
Collectors.toMap(
Map.Entry::getKey,
Map.Entry::getValue,
(e1, e2) -> e1,
LinkedHashMap::new));
return AuditMetrics.builder()
.totalEvents(events.size())
.uniqueUsers((int) eventsByUser.size())
.successRate(successRate)
.avgLatencyMs(avgLatencyMs)
.errorCount(errorCount)
.topEventType(topEventType)
.topUser(topUser)
.eventsByType(eventsByType)
.eventsByUser(eventsByUser)
.topTools(topToolsSorted)
.build();
}
/**
* Export audit data in CSV or JSON format. Maps to frontend's exportData() call. Supports both
* single values and multi-select arrays for eventType and username.
*
* @param format Export format (csv or json)
* @param eventType Filter by event type
* @param username Filter by username
* @param fields Comma-separated list of fields to include (e.g.,
* "date,username,tool,documentName,author,fileHash")
* @param eventTypes Filter by event type(s) - can be single value or array
* @param usernames Filter by username(s) - can be single value or array
* @param startDate Filter start date
* @param endDate Filter end date
* @return File download response
@@ -270,8 +527,9 @@ public class AuditRestController {
@GetMapping("/audit-export")
public ResponseEntity<byte[]> exportAuditData(
@RequestParam(value = "format", defaultValue = "csv") String format,
@RequestParam(value = "eventType", required = false) String eventType,
@RequestParam(value = "username", required = false) String username,
@RequestParam(value = "fields", required = false) String fields,
@RequestParam(value = "eventType", required = false) String[] eventTypes,
@RequestParam(value = "username", required = false) String[] usernames,
@RequestParam(value = "startDate", required = false)
@DateTimeFormat(iso = DateTimeFormat.ISO.DATE)
LocalDate startDate,
@@ -282,34 +540,44 @@ public class AuditRestController {
// Get data with same filtering as getAuditEvents
List<PersistentAuditEvent> events;
if (eventType != null && username != null && startDate != null && endDate != null) {
Instant start = startDate.atStartOfDay(ZoneId.systemDefault()).toInstant();
Instant end = endDate.plusDays(1).atStartOfDay(ZoneId.systemDefault()).toInstant();
// Convert arrays to lists
List<String> eventTypeList =
(eventTypes != null && eventTypes.length > 0) ? Arrays.asList(eventTypes) : null;
List<String> usernameList =
(usernames != null && usernames.length > 0) ? Arrays.asList(usernames) : null;
Instant startInstant = null;
Instant endInstant = null;
if (startDate != null && endDate != null) {
startInstant = startDate.atStartOfDay(ZoneId.systemDefault()).toInstant();
endInstant = endDate.plusDays(1).atStartOfDay(ZoneId.systemDefault()).toInstant();
}
if (eventTypeList != null
&& usernameList != null
&& startInstant != null
&& endInstant != null) {
events =
auditRepository.findAllByPrincipalAndTypeAndTimestampBetweenForExport(
username, eventType, start, end);
} else if (eventType != null && username != null) {
events = auditRepository.findAllByPrincipalAndTypeForExport(username, eventType);
} else if (eventType != null && startDate != null && endDate != null) {
Instant start = startDate.atStartOfDay(ZoneId.systemDefault()).toInstant();
Instant end = endDate.plusDays(1).atStartOfDay(ZoneId.systemDefault()).toInstant();
auditRepository.findByTypeInAndPrincipalInAndTimestampBetweenForExport(
eventTypeList, usernameList, startInstant, endInstant);
} else if (eventTypeList != null && usernameList != null) {
events =
auditRepository.findAllByTypeAndTimestampBetweenForExport(
eventType, start, end);
} else if (username != null && startDate != null && endDate != null) {
Instant start = startDate.atStartOfDay(ZoneId.systemDefault()).toInstant();
Instant end = endDate.plusDays(1).atStartOfDay(ZoneId.systemDefault()).toInstant();
auditRepository.findByTypeInAndPrincipalInForExport(
eventTypeList, usernameList);
} else if (eventTypeList != null && startInstant != null && endInstant != null) {
events =
auditRepository.findAllByPrincipalAndTimestampBetweenForExport(
username, start, end);
} else if (startDate != null && endDate != null) {
Instant start = startDate.atStartOfDay(ZoneId.systemDefault()).toInstant();
Instant end = endDate.plusDays(1).atStartOfDay(ZoneId.systemDefault()).toInstant();
events = auditRepository.findAllByTimestampBetweenForExport(start, end);
} else if (eventType != null) {
events = auditRepository.findByTypeForExport(eventType);
} else if (username != null) {
events = auditRepository.findAllByPrincipalForExport(username);
auditRepository.findByTypeInAndTimestampBetweenForExport(
eventTypeList, startInstant, endInstant);
} else if (usernameList != null && startInstant != null && endInstant != null) {
events =
auditRepository.findByPrincipalInAndTimestampBetweenForExport(
usernameList, startInstant, endInstant);
} else if (startInstant != null && endInstant != null) {
events = auditRepository.findAllByTimestampBetweenForExport(startInstant, endInstant);
} else if (eventTypeList != null) {
events = auditRepository.findByTypeInForExport(eventTypeList);
} else if (usernameList != null) {
events = auditRepository.findByPrincipalInForExport(usernameList);
} else {
events = auditRepository.findAll();
}
@@ -318,7 +586,7 @@ public class AuditRestController {
if ("json".equalsIgnoreCase(format)) {
return exportAsJson(events);
} else {
return exportAsCsv(events);
return exportAsCsv(events, fields);
}
}
@@ -338,17 +606,89 @@ public class AuditRestController {
}
}
// Extract IP address (check both clientIp and __ipAddress for async/audited events)
String ipAddress = "";
Object ipObj = details.get("clientIp");
if (ipObj != null) {
ipAddress = String.valueOf(ipObj);
} else {
ipObj = details.get("__ipAddress");
if (ipObj != null) {
ipAddress = String.valueOf(ipObj);
}
}
return AuditEventDto.builder()
.id(String.valueOf(event.getId()))
.timestamp(event.getTimestamp().toString())
.eventType(event.getType())
.username(event.getPrincipal())
.ipAddress((String) details.getOrDefault("ipAddress", "")) // Extract if available
.ipAddress(ipAddress)
.details(details)
.build();
}
private ResponseEntity<byte[]> exportAsCsv(List<PersistentAuditEvent> events) {
private ResponseEntity<byte[]> exportAsCsv(List<PersistentAuditEvent> events, String fields) {
// Parse selected fields (comma-separated:
// date,username,tool,documentName,author,fileHash,ipAddress,etc)
Set<String> selectedFields = new HashSet<>();
if (fields != null && !fields.trim().isEmpty()) {
String[] fieldArray = fields.split(",");
for (String field : fieldArray) {
selectedFields.add(field.trim().toLowerCase());
}
}
// If no fields specified, use default technical export
if (selectedFields.isEmpty()) {
return exportAsDefaultCsv(events);
}
StringBuilder csv = new StringBuilder();
// Build header based on selected fields
List<String> headerOrder = new ArrayList<>();
if (selectedFields.contains("date")) headerOrder.add("date");
if (selectedFields.contains("username")) headerOrder.add("username");
if (selectedFields.contains("ipaddress")) headerOrder.add("ipaddress");
if (selectedFields.contains("tool")) headerOrder.add("tool");
if (selectedFields.contains("documentname")) headerOrder.add("documentname");
if (selectedFields.contains("outcome")) headerOrder.add("outcome");
if (selectedFields.contains("author")) headerOrder.add("author");
if (selectedFields.contains("filehash")) headerOrder.add("filehash");
if (selectedFields.contains("operationresults")) headerOrder.add("operationresults");
if (selectedFields.contains("eventtype")) headerOrder.add("eventtype");
// Write header
for (int i = 0; i < headerOrder.size(); i++) {
csv.append(capitalizeHeader(headerOrder.get(i)));
if (i < headerOrder.size() - 1) csv.append(",");
}
csv.append("\n");
DateTimeFormatter formatter = DateTimeFormatter.ISO_INSTANT;
// Write data rows
for (PersistentAuditEvent event : events) {
Map<String, String> rowData = extractEventData(event, formatter);
for (int i = 0; i < headerOrder.size(); i++) {
csv.append(escapeCSV(rowData.getOrDefault(headerOrder.get(i), "")));
if (i < headerOrder.size() - 1) csv.append(",");
}
csv.append("\n");
}
byte[] csvBytes = csv.toString().getBytes(StandardCharsets.UTF_8);
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.parseMediaType("text/csv;charset=UTF-8"));
headers.setContentDispositionFormData(
"attachment", "audit_export_" + System.currentTimeMillis() + ".csv");
return ResponseEntity.ok().headers(headers).body(csvBytes);
}
private ResponseEntity<byte[]> exportAsDefaultCsv(List<PersistentAuditEvent> events) {
StringBuilder csv = new StringBuilder();
csv.append("ID,Principal,Type,Timestamp,Data\n");
@@ -362,15 +702,102 @@ public class AuditRestController {
csv.append(escapeCSV(event.getData())).append("\n");
}
byte[] csvBytes = csv.toString().getBytes();
byte[] csvBytes = csv.toString().getBytes(StandardCharsets.UTF_8);
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.APPLICATION_OCTET_STREAM);
headers.setContentType(MediaType.parseMediaType("text/csv;charset=UTF-8"));
headers.setContentDispositionFormData("attachment", "audit_export.csv");
return ResponseEntity.ok().headers(headers).body(csvBytes);
}
private Map<String, String> extractEventData(
PersistentAuditEvent event, DateTimeFormatter formatter) {
Map<String, String> data = new HashMap<>();
data.put("date", formatter.format(event.getTimestamp()));
data.put("username", event.getPrincipal());
data.put("eventtype", event.getType());
data.put("ipaddress", "");
data.put("tool", "");
data.put("documentname", "");
data.put("outcome", "");
data.put("author", "");
data.put("filehash", "");
data.put("operationresults", "");
if (event.getData() != null) {
try {
@SuppressWarnings("unchecked")
Map<String, Object> eventData = objectMapper.readValue(event.getData(), Map.class);
// Extract IP address (check both clientIp and __ipAddress)
String ipAddress = "";
if (eventData.containsKey("clientIp")) {
ipAddress = String.valueOf(eventData.getOrDefault("clientIp", ""));
} else if (eventData.containsKey("__ipAddress")) {
ipAddress = String.valueOf(eventData.getOrDefault("__ipAddress", ""));
}
if (!ipAddress.isEmpty()) {
data.put("ipaddress", ipAddress);
}
// Extract outcome (success/failure), supporting legacy "status" key
if (eventData.containsKey("outcome")) {
data.put("outcome", String.valueOf(eventData.getOrDefault("outcome", "")));
} else if (eventData.containsKey("status")) {
data.put("outcome", String.valueOf(eventData.getOrDefault("status", "")));
}
// Extract operation result if present
if (eventData.containsKey("result")) {
data.put(
"operationresults",
String.valueOf(eventData.getOrDefault("result", "")));
}
// Extract tool from path
if (eventData.containsKey("path")) {
String path = (String) eventData.get("path");
if (path != null) {
String[] parts = path.split("/");
data.put("tool", parts.length > 0 ? parts[parts.length - 1] : "");
}
}
// Extract file information
@SuppressWarnings("unchecked")
List<Map<String, Object>> files =
(List<Map<String, Object>>) eventData.get("files");
if (files != null && !files.isEmpty()) {
Map<String, Object> firstFile = files.get(0);
data.put("documentname", String.valueOf(firstFile.getOrDefault("name", "")));
data.put("author", String.valueOf(firstFile.getOrDefault("pdfAuthor", "")));
data.put("filehash", String.valueOf(firstFile.getOrDefault("fileHash", "")));
}
} catch (Exception e) {
log.trace("Failed to parse audit event data: {}", event.getData());
}
}
return data;
}
private String capitalizeHeader(String field) {
return switch (field.toLowerCase()) {
case "date" -> "Date";
case "username" -> "Username";
case "ipaddress" -> "IP Address";
case "tool" -> "Tool";
case "documentname" -> "Document Name";
case "outcome" -> "Outcome";
case "author" -> "Author";
case "filehash" -> "File Hash";
case "operationresults" -> "Operation Results";
case "eventtype" -> "Event Type";
default -> field;
};
}
private ResponseEntity<byte[]> exportAsJson(List<PersistentAuditEvent> events) {
try {
byte[] jsonBytes = objectMapper.writeValueAsBytes(events);
@@ -431,4 +858,60 @@ public class AuditRestController {
private List<String> labels;
private List<Integer> values;
}
@lombok.Data
@lombok.Builder
public static class AuditStatsData {
private long totalEvents;
private long prevTotalEvents;
private int uniqueUsers;
private int prevUniqueUsers;
private double successRate;
private double prevSuccessRate;
private double avgLatencyMs;
private double prevAvgLatencyMs;
private long errorCount;
private String topEventType;
private String topUser;
private Map<String, Long> eventsByType;
private Map<String, Long> eventsByUser;
private Map<String, Long> topTools;
private Map<String, Long> hourlyDistribution;
}
@lombok.Data
@lombok.Builder
public static class AuditMetrics {
private long totalEvents;
private int uniqueUsers;
private double successRate;
private double avgLatencyMs;
private long errorCount;
private String topEventType;
private String topUser;
private Map<String, Long> eventsByType;
private Map<String, Long> eventsByUser;
private Map<String, Long> topTools;
}
/**
* Clear all audit data from the database. This is an irreversible operation. Requires ADMIN
* role.
*
* @return Success response
*/
@PostMapping("/audit-clear-all")
public ResponseEntity<?> clearAllAuditData() {
try {
// Delete all audit events
auditRepository.deleteAll();
log.warn("All audit data has been cleared by admin user");
return ResponseEntity.ok()
.body(Map.of("message", "All audit data has been cleared successfully"));
} catch (Exception e) {
log.error("Error clearing audit data", e);
return ResponseEntity.internalServerError()
.body("Failed to clear audit data: " + e.getMessage());
}
}
}
@@ -43,6 +43,7 @@ import stirling.software.proprietary.security.database.repository.UserRepository
import stirling.software.proprietary.security.model.Authority;
import stirling.software.proprietary.security.model.SessionEntity;
import stirling.software.proprietary.security.model.User;
import stirling.software.proprietary.security.model.dto.AdminUserSummary;
import stirling.software.proprietary.security.repository.TeamRepository;
import stirling.software.proprietary.security.saml2.CustomSaml2AuthenticatedPrincipal;
import stirling.software.proprietary.security.service.DatabaseService;
@@ -126,6 +127,13 @@ public class ProprietaryUIDataController {
data.setRetentionDays(auditConfig.getRetentionDays());
data.setAuditLevels(AuditLevel.values());
data.setAuditEventTypes(AuditEventType.values());
// Metadata capture settings (independent flags)
data.setCaptureFileHash(auditConfig.isCaptureFileHash());
data.setCapturePdfAuthor(auditConfig.isCapturePdfAuthor());
data.setCaptureOperationResults(auditConfig.isCaptureOperationResults());
// pdfMetadataEnabled: true if any metadata flag is enabled (file hash or PDF author)
data.setPdfMetadataEnabled(
auditConfig.isCaptureFileHash() || auditConfig.isCapturePdfAuthor());
return ResponseEntity.ok(data);
}
@@ -357,8 +365,12 @@ public class ProprietaryUIDataController {
int licenseMaxUsers = licenseSettingsService.getSettings().getLicenseMaxUsers();
boolean premiumEnabled = applicationProperties.getPremium().isEnabled();
// Convert User entities to AdminUserSummary DTOs to exclude sensitive fields
List<AdminUserSummary> userSummaries =
sortedUsers.stream().map(this::convertUserToSummary).toList();
AdminSettingsData data = new AdminSettingsData();
data.setUsers(sortedUsers);
data.setUsers(userSummaries);
data.setCurrentUsername(authentication.getName());
data.setRoleDetails(roleDetails);
data.setUserSessions(userSessions);
@@ -518,6 +530,34 @@ public class ProprietaryUIDataController {
return ResponseEntity.ok(data);
}
/**
* Convert User entity to AdminUserSummary DTO, excluding sensitive fields like password and
* apiKey.
*/
private AdminUserSummary convertUserToSummary(User user) {
AdminUserSummary summary = new AdminUserSummary();
summary.setId(user.getId());
summary.setUsername(user.getUsername());
summary.setEmail(user.getUsername()); // Use username as email for consistency
summary.setRoleName(user.getRoleName());
summary.setRolesAsString(user.getRolesAsString());
summary.setEnabled(user.isEnabled());
summary.setIsFirstLogin(user.isFirstLogin());
summary.setAuthenticationType(user.getAuthenticationType());
summary.setCreatedAt(user.getCreatedAt());
summary.setUpdatedAt(user.getUpdatedAt());
// Map team if present
if (user.getTeam() != null) {
AdminUserSummary.TeamSummary teamSummary = new AdminUserSummary.TeamSummary();
teamSummary.setId(user.getTeam().getId());
teamSummary.setName(user.getTeam().getName());
summary.setTeam(teamSummary);
}
return summary;
}
// Data classes
@Data
public static class AuditDashboardData {
@@ -527,6 +567,10 @@ public class ProprietaryUIDataController {
private int retentionDays;
private AuditLevel[] auditLevels;
private AuditEventType[] auditEventTypes;
private boolean pdfMetadataEnabled;
private boolean captureFileHash;
private boolean capturePdfAuthor;
private boolean captureOperationResults;
}
@Data
@@ -544,7 +588,7 @@ public class ProprietaryUIDataController {
@Data
public static class AdminSettingsData {
private List<User> users;
private List<AdminUserSummary> users;
private String currentUsername;
private Map<String, String> roleDetails;
private Map<String, Boolean> userSessions;
@@ -1,5 +1,7 @@
package stirling.software.proprietary.controller.api;
import java.time.Duration;
import java.time.Instant;
import java.util.*;
import java.util.stream.Collectors;
@@ -32,34 +34,32 @@ public class UsageRestController {
private final ObjectMapper objectMapper;
/**
* Get endpoint statistics derived from audit events. This endpoint analyzes HTTP_REQUEST audit
* events to generate usage statistics.
* Get endpoint statistics derived from audit events. This endpoint analyzes audit events
* filtered by type to generate usage statistics.
*
* @param limit Optional limit on number of endpoints to return
* @param dataType Type of data to include: "all" (default), "api" (API endpoints excluding
* auth), or "ui" (non-API endpoints)
* @param dataType Type of data to include: "all" (default), "api" (operational endpoints), or
* "ui" (UI data endpoints)
* @param days Lookback window in days (default 30, clamped to 1-365)
* @return Endpoint statistics response
*/
@GetMapping("/usage-endpoint-statistics")
public ResponseEntity<EndpointStatisticsResponse> getEndpointStatistics(
@RequestParam(value = "limit", required = false) Integer limit,
@RequestParam(value = "dataType", defaultValue = "all") String dataType) {
@RequestParam(value = "dataType", defaultValue = "all") String dataType,
@RequestParam(value = "days", defaultValue = "30") Integer days) {
// Get all HTTP_REQUEST audit events
List<PersistentAuditEvent> httpEvents =
auditRepository.findByTypeForExport(AuditEventType.HTTP_REQUEST.name());
int lookbackDays = Math.max(1, Math.min(days, 365));
// Get audit events filtered by type
List<PersistentAuditEvent> events = getEventsByDataType(dataType, lookbackDays);
// Count visits per endpoint
Map<String, Long> endpointCounts = new HashMap<>();
for (PersistentAuditEvent event : httpEvents) {
for (PersistentAuditEvent event : events) {
String endpoint = extractEndpointFromAuditData(event.getData());
if (endpoint != null) {
// Apply data type filter
if (!shouldIncludeEndpoint(endpoint, dataType)) {
continue;
}
endpointCounts.merge(endpoint, 1L, Long::sum);
}
}
@@ -168,52 +168,28 @@ public class UsageRestController {
}
/**
* Determine if an endpoint should be included based on the data type filter.
* Get audit events filtered by data type. UI = UI_DATA events. API = everything except UI_DATA.
*
* @param endpoint The endpoint path to check
* @param dataType The filter type: "all", "api", or "ui"
* @return true if the endpoint should be included, false otherwise
* @param dataType "all", "api" (not UI_DATA), or "ui" (UI_DATA only)
* @param days lookback window in days
* @return List of audit events matching the data type filter
*/
private boolean shouldIncludeEndpoint(String endpoint, String dataType) {
private List<PersistentAuditEvent> getEventsByDataType(String dataType, int days) {
Instant start = Instant.now().minus(Duration.ofDays(days));
if ("all".equalsIgnoreCase(dataType)) {
return true;
}
boolean isApiEndpoint = isApiEndpoint(endpoint);
if ("api".equalsIgnoreCase(dataType)) {
return isApiEndpoint;
return auditRepository.findByTimestampAfter(start);
} else if ("ui".equalsIgnoreCase(dataType)) {
return !isApiEndpoint;
// UI data endpoints only
return auditRepository.findByTypeAndTimestampAfterForExport(
AuditEventType.UI_DATA.name(), start);
} else if ("api".equalsIgnoreCase(dataType)) {
// API = everything except UI_DATA (queried at DB level, not filtered in-memory)
return auditRepository.findAllExceptTypeAndTimestampAfterForExport(
AuditEventType.UI_DATA.name(), start);
}
// Default to including all if unrecognized type
return true;
}
/**
* Check if an endpoint is an API endpoint. API endpoints match /api/v1/* pattern but exclude
* /api/v1/auth/* paths.
*
* @param endpoint The endpoint path to check
* @return true if this is an API endpoint (excluding auth endpoints), false otherwise
*/
private boolean isApiEndpoint(String endpoint) {
if (endpoint == null) {
return false;
}
// Check if it starts with /api/v1/
if (!endpoint.startsWith("/api/v1/")) {
return false;
}
// Exclude auth endpoints
if (endpoint.startsWith("/api/v1/auth/")) {
return false;
}
return true;
return new ArrayList<>();
}
// DTOs for response formatting
@@ -68,6 +68,10 @@ public interface PersistentAuditEventRepository extends JpaRepository<Persistent
@Query("SELECT e FROM PersistentAuditEvent e WHERE e.type = :type")
List<PersistentAuditEvent> findByTypeForExport(@Param("type") String type);
@Query("SELECT e FROM PersistentAuditEvent e WHERE e.type = :type AND e.timestamp > :startDate")
List<PersistentAuditEvent> findByTypeAndTimestampAfterForExport(
@Param("type") String type, @Param("startDate") Instant startDate);
@Query("SELECT e FROM PersistentAuditEvent e WHERE e.timestamp BETWEEN :startDate AND :endDate")
List<PersistentAuditEvent> findAllByTimestampBetweenForExport(
@Param("startDate") Instant startDate, @Param("endDate") Instant endDate);
@@ -171,4 +175,88 @@ public interface PersistentAuditEventRepository extends JpaRepository<Persistent
Optional<PersistentAuditEvent> findTopByPrincipalOrderByTimestampDesc(String principal);
Optional<PersistentAuditEvent> findTopByTypeOrderByTimestampDesc(String type);
// Multi-value queries for filtering by multiple types and/or principals
@Query("SELECT e FROM PersistentAuditEvent e WHERE e.type IN :types")
Page<PersistentAuditEvent> findByTypeIn(@Param("types") List<String> types, Pageable pageable);
@Query("SELECT e FROM PersistentAuditEvent e WHERE e.principal IN :principals")
Page<PersistentAuditEvent> findByPrincipalIn(
@Param("principals") List<String> principals, Pageable pageable);
@Query(
"SELECT e FROM PersistentAuditEvent e WHERE e.type IN :types AND e.timestamp BETWEEN :startDate AND :endDate")
Page<PersistentAuditEvent> findByTypeInAndTimestampBetween(
@Param("types") List<String> types,
@Param("startDate") Instant startDate,
@Param("endDate") Instant endDate,
Pageable pageable);
@Query(
"SELECT e FROM PersistentAuditEvent e WHERE e.principal IN :principals AND e.timestamp BETWEEN :startDate AND :endDate")
Page<PersistentAuditEvent> findByPrincipalInAndTimestampBetween(
@Param("principals") List<String> principals,
@Param("startDate") Instant startDate,
@Param("endDate") Instant endDate,
Pageable pageable);
@Query(
"SELECT e FROM PersistentAuditEvent e WHERE e.type IN :types AND e.principal IN :principals")
Page<PersistentAuditEvent> findByTypeInAndPrincipalIn(
@Param("types") List<String> types,
@Param("principals") List<String> principals,
Pageable pageable);
@Query(
"SELECT e FROM PersistentAuditEvent e WHERE e.type IN :types AND e.principal IN :principals AND e.timestamp BETWEEN :startDate AND :endDate")
Page<PersistentAuditEvent> findByTypeInAndPrincipalInAndTimestampBetween(
@Param("types") List<String> types,
@Param("principals") List<String> principals,
@Param("startDate") Instant startDate,
@Param("endDate") Instant endDate,
Pageable pageable);
// Export versions (non-paged)
@Query("SELECT e FROM PersistentAuditEvent e WHERE e.type IN :types")
List<PersistentAuditEvent> findByTypeInForExport(@Param("types") List<String> types);
@Query("SELECT e FROM PersistentAuditEvent e WHERE e.principal IN :principals")
List<PersistentAuditEvent> findByPrincipalInForExport(
@Param("principals") List<String> principals);
@Query(
"SELECT e FROM PersistentAuditEvent e WHERE e.type IN :types AND e.timestamp BETWEEN :startDate AND :endDate")
List<PersistentAuditEvent> findByTypeInAndTimestampBetweenForExport(
@Param("types") List<String> types,
@Param("startDate") Instant startDate,
@Param("endDate") Instant endDate);
@Query(
"SELECT e FROM PersistentAuditEvent e WHERE e.principal IN :principals AND e.timestamp BETWEEN :startDate AND :endDate")
List<PersistentAuditEvent> findByPrincipalInAndTimestampBetweenForExport(
@Param("principals") List<String> principals,
@Param("startDate") Instant startDate,
@Param("endDate") Instant endDate);
@Query(
"SELECT e FROM PersistentAuditEvent e WHERE e.type IN :types AND e.principal IN :principals")
List<PersistentAuditEvent> findByTypeInAndPrincipalInForExport(
@Param("types") List<String> types, @Param("principals") List<String> principals);
@Query(
"SELECT e FROM PersistentAuditEvent e WHERE e.type IN :types AND e.principal IN :principals AND e.timestamp BETWEEN :startDate AND :endDate")
List<PersistentAuditEvent> findByTypeInAndPrincipalInAndTimestampBetweenForExport(
@Param("types") List<String> types,
@Param("principals") List<String> principals,
@Param("startDate") Instant startDate,
@Param("endDate") Instant endDate);
// Query events excluding a specific type (used for analytics where we want to exclude UI_DATA)
@Query("SELECT e FROM PersistentAuditEvent e WHERE e.type != :excludeType")
List<PersistentAuditEvent> findAllExceptTypeForExport(@Param("excludeType") String excludeType);
@Query(
"SELECT e FROM PersistentAuditEvent e WHERE e.type != :excludeType AND e.timestamp > :startDate")
List<PersistentAuditEvent> findAllExceptTypeAndTimestampAfterForExport(
@Param("excludeType") String excludeType, @Param("startDate") Instant startDate);
}
@@ -172,7 +172,7 @@ public class AdminSettingsController {
.body(Map.of("error", "No settings provided to update"));
}
int updatedCount = 0;
// Validate all settings first before applying any changes
for (Map.Entry<String, Object> entry : settings.entrySet()) {
String key = entry.getKey();
Object value = entry.getValue();
@@ -192,14 +192,18 @@ public class AdminSettingsController {
return ResponseEntity.badRequest()
.body(Map.of("error", HtmlUtils.htmlEscape(validationError)));
}
}
// Apply all updates in a single transaction (load once, update all, save once)
// This ensures nested settings like oauth2.client.* don't lose sibling values
GeneralUtils.updateSettingsTransactional(settings);
// Track all as pending changes
for (Map.Entry<String, Object> entry : settings.entrySet()) {
String key = entry.getKey();
Object value = entry.getValue();
log.info("Admin updating setting: {} = {}", key, value);
GeneralUtils.saveKeyToSettings(key, value);
// Track this as a pending change
pendingChanges.put(key, value);
updatedCount++;
pendingChanges.put(key, value != null ? value : "");
}
return ResponseEntity.ok(
@@ -208,7 +212,7 @@ public class AdminSettingsController {
String.format(
"Successfully updated %d setting(s). Changes will take effect on"
+ " application restart.",
updatedCount)));
settings.size())));
} catch (IOException e) {
log.error("Failed to save settings to file: {}", e.getMessage(), e);
@@ -268,6 +272,9 @@ public class AdminSettingsController {
}
}
// Mask sensitive fields before returning to frontend
sectionMap = maskSensitiveFields(sectionMap);
log.debug(
"Admin requested settings section: {} (includePending={})",
sectionName,
@@ -404,6 +411,13 @@ public class AdminSettingsController {
return ResponseEntity.badRequest()
.body("Setting key not found: " + HtmlUtils.htmlEscape(key));
}
// Mask sensitive values before returning
String keyName = key.contains(".") ? key.substring(key.lastIndexOf(".") + 1) : key;
if (isSensitiveFieldWithPath(keyName, key)) {
value = createMaskedValue(value);
}
log.debug("Admin requested setting: {}", key);
return ResponseEntity.ok(new SettingValueResponse(key, value));
} catch (IllegalArgumentException e) {
@@ -441,6 +455,20 @@ public class AdminSettingsController {
}
Object value = request.getValue();
// Prevent saving masked values for sensitive fields to avoid data loss
if ("********".equals(value)) {
String keyName = key.contains(".") ? key.substring(key.lastIndexOf(".") + 1) : key;
if (isSensitiveFieldWithPath(keyName, key)) {
log.warn(
"Admin attempted to save masked value for sensitive field: {}. This operation is blocked to prevent data loss.",
key);
return ResponseEntity.badRequest()
.body(
"Cannot save masked values for sensitive settings. Please provide the actual value.");
}
}
log.info("Admin updating single setting: {} = {}", key, value);
GeneralUtils.saveKeyToSettings(key, value);
@@ -123,7 +123,7 @@ public class AuthController {
log.warn("Invalid password for user: {} from IP: {}", username, ip);
loginAttemptService.loginFailed(username);
return ResponseEntity.status(HttpStatus.UNAUTHORIZED)
.body(Map.of("error", "Invalid credentials"));
.body(Map.of("error", "Invalid username or password"));
}
if (!user.isEnabled()) {
@@ -191,6 +191,20 @@ public class DatabaseController {
if (fileName == null || fileName.isEmpty()) {
throw new IllegalArgumentException("File must not be null or empty");
}
// Validate that file is a legitimate backup file
// Only allow files matching the backup naming pattern
if (!fileName.startsWith("backup_") || !fileName.endsWith(".sql")) {
log.warn("Attempted download of non-backup file: {}", fileName);
return ResponseEntity.status(HttpStatus.BAD_REQUEST)
.body(
java.util.Map.of(
"error",
"invalidFileName",
"message",
"Only backup files are allowed"));
}
try {
Path filePath = databaseService.getBackupFilePath(fileName);
InputStreamResource resource = new InputStreamResource(Files.newInputStream(filePath));
@@ -358,27 +358,23 @@ public class InviteLinkController {
Optional<InviteToken> inviteOpt = inviteTokenRepository.findByToken(token);
if (inviteOpt.isEmpty()) {
return ResponseEntity.status(HttpStatus.NOT_FOUND)
.body(Map.of("error", "Invalid invite link"));
return invalidInviteResponse();
}
InviteToken invite = inviteOpt.get();
if (invite.isUsed()) {
return ResponseEntity.status(HttpStatus.GONE)
.body(Map.of("error", "This invite link has already been used"));
return invalidInviteResponse();
}
if (invite.isExpired()) {
return ResponseEntity.status(HttpStatus.GONE)
.body(Map.of("error", "This invite link has expired"));
return invalidInviteResponse();
}
// Check if user already exists (only if email is pre-set)
if (invite.getEmail() != null
&& userService.usernameExistsIgnoreCase(invite.getEmail())) {
return ResponseEntity.status(HttpStatus.CONFLICT)
.body(Map.of("error", "User already exists"));
return invalidInviteResponse();
}
Map<String, Object> response = new HashMap<>();
@@ -391,8 +387,7 @@ public class InviteLinkController {
} catch (Exception e) {
log.error("Failed to validate invite token: {}", e.getMessage(), e);
return ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR)
.body(Map.of("error", "Failed to validate invite link"));
return invalidInviteResponse();
}
}
@@ -419,20 +414,17 @@ public class InviteLinkController {
Optional<InviteToken> inviteOpt = inviteTokenRepository.findByToken(token);
if (inviteOpt.isEmpty()) {
return ResponseEntity.status(HttpStatus.NOT_FOUND)
.body(Map.of("error", "Invalid invite link"));
return invalidInviteResponse();
}
InviteToken invite = inviteOpt.get();
if (invite.isUsed()) {
return ResponseEntity.status(HttpStatus.GONE)
.body(Map.of("error", "This invite link has already been used"));
return invalidInviteResponse();
}
if (invite.isExpired()) {
return ResponseEntity.status(HttpStatus.GONE)
.body(Map.of("error", "This invite link has expired"));
return invalidInviteResponse();
}
// Determine the email to use
@@ -455,8 +447,7 @@ public class InviteLinkController {
// Check if user already exists
if (userService.usernameExistsIgnoreCase(effectiveEmail)) {
return ResponseEntity.status(HttpStatus.CONFLICT)
.body(Map.of("error", "User already exists"));
return invalidInviteResponse();
}
// Create the user account
@@ -484,7 +475,12 @@ public class InviteLinkController {
} catch (Exception e) {
log.error("Failed to accept invite: {}", e.getMessage(), e);
return ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR)
.body(Map.of("error", "Failed to create account: " + e.getMessage()));
.body(Map.of("error", "Failed to create account"));
}
}
private ResponseEntity<Map<String, String>> invalidInviteResponse() {
return ResponseEntity.status(HttpStatus.NOT_FOUND)
.body(Map.of("error", "Invalid invite link"));
}
}
@@ -942,7 +942,7 @@ public class UserController {
public ResponseEntity<?> completeInitialSetup() {
try {
String username = userService.getCurrentUsername();
if (username == null) {
if (username == null || "anonymousUser".equalsIgnoreCase(username)) {
return ResponseEntity.status(HttpStatus.UNAUTHORIZED)
.body("User not authenticated");
}
@@ -46,9 +46,11 @@ public class User implements UserDetails, Serializable {
private String username;
@Column(name = "password")
@JsonIgnore
private String password;
@Column(name = "apiKey")
@JsonIgnore
private String apiKey;
@Column(name = "enabled")

Some files were not shown because too many files have changed in this diff Show More