Compare commits

..
Author SHA1 Message Date
Anthony Stirling 24162d060b util for repair and compress as example 2025-08-01 23:34:49 +01:00
24a9104ebf Fix GB language file (#4087)
Language file fix

Co-authored-by: Connor Yoh <connor@stirlingpdf.com>
2025-08-01 20:14:05 +01:00
stirlingbot[bot]GitHubstirlingbot[bot] <195170888+stirlingbot[bot]@users.noreply.github.com>
03f86fc59c Update Frontend 3rd Party Licenses (#4074)
Auto-generated by stirlingbot[bot]

This PR updates the frontend license report based on changes to
package.json dependencies.

Signed-off-by: stirlingbot[bot] <stirlingbot[bot]@users.noreply.github.com>
Co-authored-by: stirlingbot[bot] <195170888+stirlingbot[bot]@users.noreply.github.com>
2025-08-01 17:22:30 +01:00
9c9acbfb5b V2: Convert Tool (#3828)
🔄 Dynamic Processing Strategies

- Adaptive routing: Same tool uses different backend endpoints based on
file analysis
- Combined vs separate processing: Intelligently chooses between merge
operations and individual file processing
- Cross-format workflows: Enable complex conversions like "mixed files →
PDF" that other tools can't handle

  ⚙️ Format-Specific Intelligence

  Each conversion type gets tailored options:
  - HTML/ZIP → PDF: Zoom controls (0.1-3.0 increments) with live preview
  - Email → PDF: Attachment handling, size limits, recipient control
  - PDF → PDF/A: Digital signature detection with warnings
  - Images → PDF: Smart combining vs individual file options

 File Architecture

  Core Implementation:
  ├── Convert.tsx                     # Main stepped workflow UI
├── ConvertSettings.tsx # Centralized settings with smart detection
├── GroupedFormatDropdown.tsx # Enhanced format selector with grouping
├── useConvertParameters.ts # Smart detection & parameter management
  ├── useConvertOperation.ts         # Multi-strategy processing logic
  └── Settings Components:
      ├── ConvertFromWebSettings.tsx      # HTML zoom controls
      ├── ConvertFromEmailSettings.tsx    # Email attachment options
├── ConvertToPdfaSettings.tsx # PDF/A with signature detection
      ├── ConvertFromImageSettings.tsx    # Image PDF options
      └── ConvertToImageSettings.tsx      # PDF to image options

 Utility Layer

  Utils & Services:
├── convertUtils.ts # Format detection & endpoint routing
  ├── fileResponseUtils.ts          # Generic API response handling
└── setupTests.ts # Enhanced test environment with crypto mocks

  Testing & Quality

  Comprehensive Test Coverage

  Test Suite:
├── useConvertParameters.test.ts # Parameter logic & smart detection
  ├── useConvertParametersAutoDetection.test.ts  # File type analysis
├── ConvertIntegration.test.tsx # End-to-end conversion workflows
  ├── ConvertSmartDetectionIntegration.test.tsx  # Mixed file scenarios
  ├── ConvertE2E.spec.ts                     # Playwright browser tests
├── convertUtils.test.ts # Utility function validation
  └── fileResponseUtils.test.ts              # API response handling

  Advanced Test Features

  - Crypto API mocking: Proper test environment for file hashing
  - File.arrayBuffer() polyfills: Complete browser API simulation
  - Multi-file scenario testing: Complex batch processing validation
- CI/CD integration: Vitest runs in GitHub Actions with proper artifacts

---------

Co-authored-by: Connor Yoh <connor@stirlingpdf.com>
Co-authored-by: Anthony Stirling <77850077+Frooodle@users.noreply.github.com>
2025-08-01 16:08:04 +01:00
8881f19b03 Tools/ocr/v2 (#4055)
# Description of Changes

- Added the OCR tool 
- Added language mappings file to map selected browser language -> OCR
language and OCR language codes -> english display values. TODO: Use the
translation function to translate the languages rather than mapping them
to english be default
- Added chevron icons to tool step to show expand and collapsed state
more visibly
- Added a re-usable dropdown picker with a footer component

---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/DeveloperGuide.md#6-testing)
for more details.

---------

Co-authored-by: Anthony Stirling <77850077+Frooodle@users.noreply.github.com>
2025-08-01 14:22:19 +01:00
Anthony StirlingandGitHub 8802daf67f Support for SSL testing (#4070)
# Description of Changes

<!--
Please provide a summary of the changes, including:

- What was changed
- Why the change was made
- Any challenges encountered

Closes #(issue_number)
-->

---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/DeveloperGuide.md#6-testing)
for more details.
2025-08-01 10:26:20 +01:00
LudyandGitHub 64a471c3a9 fix(workflows): correct YAML indentation in frontend license update workflow (#4056)
…orkflow

# Description of Changes

- Fixed invalid YAML syntax caused by inconsistent indentation inside a
multi-line JavaScript template string in
`.github/workflows/frontend-licenses-update.yml`.
- The incorrect indentation broke the GitHub Actions runner with a
syntax error on line 131.
- This change ensures that the entire `commentBody = \`...\`;` block is
treated as a valid JavaScript string inside the `run:` shell context.


---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/DeveloperGuide.md#6-testing)
for more details.
2025-07-30 22:38:03 +01:00
Anthony StirlingandGitHub fc89ee2c78 Update docker-compose.fat.yml 2025-07-30 11:19:28 +01:00
Anthony StirlingandGitHub a6df5e4246 Update docker-compose.yml 2025-07-30 11:19:13 +01:00
Anthony StirlingandGitHub 989052b099 Update deploy-on-v2-commit.yml 2025-07-29 16:00:08 +01:00
Anthony StirlingandGitHub dbe84f7e96 Update deploy-on-v2-commit.yml 2025-07-29 15:50:36 +01:00
Anthony StirlingandGitHub 8989bdb211 Revert "V2 Update PR-Auto-Deploy-V2.yml to use common tag" (#4047)
Reverts Stirling-Tools/Stirling-PDF#4046
2025-07-29 15:49:27 +01:00
Anthony StirlingandGitHub 1bd166f6fb V2 Update PR-Auto-Deploy-V2.yml to use common tag (#4046)
# Description of Changes

<!--
Please provide a summary of the changes, including:

- What was changed
- Why the change was made
- Any challenges encountered

Closes #(issue_number)
-->

---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/DeveloperGuide.md#6-testing)
for more details.
2025-07-29 15:38:37 +01:00
75bd154a82 V2 async requests (#4043)
# Description of Changes

<!--
Please provide a summary of the changes, including:

- What was changed
- Why the change was made
- Any challenges encountered

Closes #(issue_number)
-->

---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/DeveloperGuide.md#6-testing)
for more details.

---------

Co-authored-by: a <a>
2025-07-29 13:44:17 +01:00
Anthony StirlingandGitHub 1ce813bcd8 Endpoint UI data for V2 (#4044)
# Description of Changes
This pull request introduces several enhancements and new features,
primarily focused on improving API documentation support, expanding
backend functionality, and adding new frontend tools. Key changes
include the integration of Swagger API documentation, the addition of a
new `UIDataController` for backend data handling, and updates to the
frontend to include a Swagger UI tool.

### Backend Enhancements:
* **Swagger API Documentation Integration**:
- Added support for dynamically configuring Swagger servers using the
`SWAGGER_SERVER_URL` environment variable in `OpenApiConfig`
(`[app/core/src/main/java/stirling/software/SPDF/config/OpenApiConfig.javaR54-L63](diffhunk://#diff-6080fb3dc14efc430c9de1bf9fa4996a23deebc14230dde7788949b2c49cca68R54-L63)`).
- Imported necessary Swagger dependencies
(`[app/core/src/main/java/stirling/software/SPDF/config/OpenApiConfig.javaR13](diffhunk://#diff-6080fb3dc14efc430c9de1bf9fa4996a23deebc14230dde7788949b2c49cca68R13)`).
- Updated `nginx.conf` to proxy Swagger-related requests to the backend
(`[docker/frontend/nginx.confR55-R92](diffhunk://#diff-6d35fafb4405bd052c6d5e48bd946bcef7c77552a74e1b902de45e85eee09aceR55-R92)`).

* **New `UIDataController`**:
- Introduced a new controller (`UIDataController`) to serve React UI
data, including endpoints for home data, licenses, pipeline
configurations, signature data, and OCR data
(`[app/core/src/main/java/stirling/software/SPDF/controller/api/UIDataController.javaR1-R301](diffhunk://#diff-3e7063d4e921c7b9e6eedfcad0e535ba3eff68476dcff5e6f28b00c388cff646R1-R301)`).

* **Endpoint Handling**:
- Modified `ConfigController` to include explicit parameter naming for
better clarity in API requests
(`[app/core/src/main/java/stirling/software/SPDF/controller/api/misc/ConfigController.javaL113-R120](diffhunk://#diff-43d19d45ae547fd79090596c06d58cb0eb7f722ed43eb4da59f9dec39f6def6eL113-R120)`).

### Frontend Enhancements:
* **Swagger UI Tool**:
- Added a new tool definition (`swagger`) in `useToolManagement.tsx`,
with an icon and lazy-loaded component
(`[frontend/src/hooks/useToolManagement.tsxR30-R38](diffhunk://#diff-57f8a6b3e75ecaec10ad445b01afe8fccc376af6f8ad4d693c68cf98e8863273R30-R38)`).
- Implemented the `SwaggerUI` component to open the Swagger
documentation in a new tab
(`[frontend/src/tools/SwaggerUI.tsxR1-R18](diffhunk://#diff-ca9bdf83c5d611a5edff10255103d7939895635b33a258dd77db6571da6c4600R1-R18)`).

* **Localization Updates**:
- Updated English (US and GB) translation files to include
Swagger-related strings
(`[[1]](diffhunk://#diff-14c707e28788a3a84ed5293ff6689be73d4bca00e155beaf090f9b37c978babbR578-R581)`,
`[[2]](diffhunk://#diff-14c707e28788a3a84ed5293ff6689be73d4bca00e155beaf090f9b37c978babbR1528-R1533)`,
`[[3]](diffhunk://#diff-e4d543afa388d9eb8a423e45dfebb91641e3558d00848d70b285ebb91c40b249R578-R581)`,
`[[4]](diffhunk://#diff-e4d543afa388d9eb8a423e45dfebb91641e3558d00848d70b285ebb91c40b249R1528-R1533)`).

### Workflow Updates:
* **Environment Variable Additions**:
- Added `SWAGGER_SERVER_URL` to the `PR-Auto-Deploy-V2.yml` and
`deploy-on-v2-commit.yml` workflows for configuring Swagger server URLs
(`[[1]](diffhunk://#diff-931fcb06ba030420d7044dde06465ad55b4e769a9bd374dcd6a0c76f79a5e30eR320)`,
`[[2]](diffhunk://#diff-f8b6ec3c0af9cd2d8dffef6f3def2be6357fe596a606850ca7f5d799e1349069R150)`).
<!--
Please provide a summary of the changes, including:

- What was changed
- Why the change was made
- Any challenges encountered

Closes #(issue_number)
-->

---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/DeveloperGuide.md#6-testing)
for more details.
2025-07-29 13:11:53 +01:00
Anthony StirlingandGitHub 3b82602744 Deploy on v2 commit (#4033)
# Description of Changes

<!--
Please provide a summary of the changes, including:

- What was changed
- Why the change was made
- Any challenges encountered

Closes #(issue_number)
-->

---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/DeveloperGuide.md#6-testing)
for more details.
2025-07-25 14:40:58 +01:00
EthanHealy01andGitHub a9a41b3877 V2: Design Navbar (#4017)
# Description of Changes

- Changed the navbar styling to include all the icons that were on our
figma design
- Added a new link to our index.html to include the MUI symbols. 
- I chose to keep the automate and read icons the same as they were
already because I feel as though they make more sense

```
<link rel="stylesheet" href="https://fonts.googleapis.com/css2?family=Material+Symbols+Outlined:opsz,wght,FILL,GRAD@24,700,0,0" />
```
---

figma vs app dark
<img width="422" height="1038" alt="Screenshot 2025-07-21 at 5 44 19 PM"
src="https://github.com/user-attachments/assets/15d5583f-ce3c-418f-81c6-6e6022f5f4d0"
/>

figma vs app light
<img width="244" height="926" alt="Screenshot 2025-07-21 at 5 57 27 PM"
src="https://github.com/user-attachments/assets/c855d02b-20ee-4ccf-af1f-a3c1a4c2a154"
/>



## Checklist

### General

- [x] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/DeveloperGuide.md)
(if applicable)
- [x] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [x] I have performed a self-review of my own code
- [x] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### UI Changes (if applicable)

- [x] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [x] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/DeveloperGuide.md#6-testing)
for more details.
2025-07-25 12:27:30 +01:00
EthanHealy01andGitHub 5f7e578ff8 change logos, favicon and browser tab title (#4032)
# Description of Changes

- changed favicon and other logos in public folder
- changes the title title tag in the index.html to "Stirling PDF"

---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/DeveloperGuide.md#6-testing)
for more details.
2025-07-25 12:14:37 +01:00
Reece BrowneandGitHub e4d480c7b3 Feature/v2/multiselect (#4024)
# Description of Changes
This pull request introduces significant updates to the file selection
logic, tool rendering, and file context management in the frontend
codebase. The changes aim to improve modularity, enhance
maintainability, and streamline the handling of file-related operations.
Key updates include the introduction of a new `FileSelectionContext`,
refactoring of file selection logic, and updates to tool management and
rendering.

### File Selection Context and Logic Refactor:
* Added a new `FileSelectionContext` to centralize file selection state
and provide utility hooks for managing selected files, selection limits,
and tool mode. (`frontend/src/contexts/FileSelectionContext.tsx`,
[frontend/src/contexts/FileSelectionContext.tsxR1-R77](diffhunk://#diff-bda35f1aaa5eafa0a0dc48e0b1270d862f6da360ba1241234e891f0ca8907327R1-R77))
* Replaced local file selection logic in `FileEditor` with context-based
logic, improving consistency and reducing duplication.
(`frontend/src/components/fileEditor/FileEditor.tsx`,
[[1]](diffhunk://#diff-481d0a2d8a1714d34d21181db63a020b08dfccfbfa80bf47ac9af382dff25310R63-R70)
[[2]](diffhunk://#diff-481d0a2d8a1714d34d21181db63a020b08dfccfbfa80bf47ac9af382dff25310R404-R438)

### Tool Management and Rendering:
* Refactored `ToolRenderer` to use a `Suspense` fallback for lazy-loaded
tools, improving user experience during tool loading.
(`frontend/src/components/tools/ToolRenderer.tsx`,
[frontend/src/components/tools/ToolRenderer.tsxL32-L64](diffhunk://#diff-2083701113aa92cd1f5ce1b4b52cc233858e31ed7bcf39c5bfb1bcc34e99b6a9L32-L64))
* Simplified `ToolPicker` by reusing the `ToolRegistry` type, reducing
redundancy. (`frontend/src/components/tools/ToolPicker.tsx`,
[frontend/src/components/tools/ToolPicker.tsxL4-R4](diffhunk://#diff-e47deca9132018344c159925f1264794acdd57f4b65e582eb9b2a4ea69ec126dL4-R4))

### File Context Enhancements:
* Introduced a utility function `getFileId` for consistent file ID
extraction, replacing repetitive inline logic.
(`frontend/src/contexts/FileContext.tsx`,
[[1]](diffhunk://#diff-95b3d103fa434f81fdae55f2ea14eda705f0def45a0f2c5754f81de6f2fd93bcR25)
[[2]](diffhunk://#diff-95b3d103fa434f81fdae55f2ea14eda705f0def45a0f2c5754f81de6f2fd93bcL101-R102)
* Updated `FileContextProvider` to use more specific types for PDF
documents, enhancing type safety.
(`frontend/src/contexts/FileContext.tsx`,
[[1]](diffhunk://#diff-95b3d103fa434f81fdae55f2ea14eda705f0def45a0f2c5754f81de6f2fd93bcL350-R351)
[[2]](diffhunk://#diff-95b3d103fa434f81fdae55f2ea14eda705f0def45a0f2c5754f81de6f2fd93bcL384-R385)

### Compression Tool Enhancements:
* Added blob URL cleanup logic to the compression hook to prevent memory
leaks. (`frontend/src/hooks/tools/compress/useCompressOperation.ts`,
[frontend/src/hooks/tools/compress/useCompressOperation.tsR58-L66](diffhunk://#diff-d7815fea0e89989511ae1786f7031cba492b9f2db39b7ade92d9736d1bd4b673R58-L66))
* Adjusted file ID generation in the compression operation to handle
multiple files more effectively.
(`frontend/src/hooks/tools/compress/useCompressOperation.ts`,
[frontend/src/hooks/tools/compress/useCompressOperation.tsL90-R102](diffhunk://#diff-d7815fea0e89989511ae1786f7031cba492b9f2db39b7ade92d9736d1bd4b673L90-R102))

---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/DeveloperGuide.md#6-testing)
for more details.
2025-07-25 09:37:52 +01:00
4e5f595951 Feature/V2/Compress (#3982)
# Description of Changes

- Added the compression tool
- Slightly adjusted the padding on split, compress and the parent
sidebar component so things aren't so smushed
- Future work will be to allow multi-select on files and to add further
styling to the toolstep / sidebar component

---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/DeveloperGuide.md#6-testing)
for more details.

---------

Co-authored-by: Ethan <ethan@MacBook-Pro.local>
Co-authored-by: Anthony Stirling <77850077+Frooodle@users.noreply.github.com>
2025-07-22 11:01:28 +01:00
Anthony StirlingandGitHub 088466b742 Merge pull request #4016 from Stirling-Tools/V2-rebase
V2 rebase
2025-07-21 20:11:54 +01:00
Anthony StirlingandGitHub fd072cccba Update build.yml 2025-07-21 19:19:41 +01:00
Anthony StirlingandGitHub 67d3f28f53 Update build.yml 2025-07-21 18:52:45 +01:00
Anthony StirlingandGitHub cdb3ffff78 Update build.yml 2025-07-21 16:45:26 +01:00
Anthony Stirling 9adb02fcfb Merge remote-tracking branch 'origin/main' into V2 2025-07-21 15:37:59 +01:00
Anthony StirlingandGitHub b8d582a1e3 Update PR-Auto to include ethan 2025-07-20 21:45:43 +01:00
Anthony StirlingandGitHub 0549c5b191 testing and docker replacements (#3968)
This PR restructures testing scripts and Docker configurations to use centralized compose files, introduces new Docker Compose variants with integrated frontend services, and updates related CI workflows.

Migrate test scripts to reference testing/compose files and streamline test flows with forced rebuilds and direct curl checks.
Add ultra-lite, security, and security-with-login compose files under testing/compose, each defining both backend and frontend services.
Rename and adjust frontend imports and update CI workflows to build and validate the frontend separately.
2025-07-18 14:19:36 +01:00
Anthony StirlingandGitHub 0742364a03 V2 frontend license checker (#3944)
# Added scripts for checking the licenses of dependencies similar to the backend app
2025-07-18 13:50:40 +01:00
Anthony StirlingandGitHub 9af09c6ac2 Merge pull request #3965 from Stirling-Tools/V2-gha
V2 testing and docker replacements
2025-07-17 14:29:09 +01:00
Anthony Stirling 117d906be3 test new doc 2025-07-17 00:40:30 +01:00
Anthony Stirling 33549c2d67 fix for deploy 2025-07-17 00:34:08 +01:00
Anthony Stirling a35956d927 V2 2025-07-16 23:42:38 +01:00
Anthony Stirling 3fd52ed564 testing and docker replacements 2025-07-16 23:09:26 +01:00
Anthony Stirling 2f977ff250 Merge branch 'main' into V2 2025-07-16 21:25:37 +01:00
922bbc9076 Stirling 2.0 (#3928)
# Description of Changes

<!--

File context for managing files between tools and views
Optimisation for large files
Updated Split to work with new file system and match Matts stepped
design closer

-->

---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/DeveloperGuide.md#6-testing)
for more details.

---------

Co-authored-by: Anthony Stirling <77850077+Frooodle@users.noreply.github.com>
2025-07-16 17:53:50 +01:00
Anthony Stirling 584e2ecee7 PR extra 2025-07-16 15:20:02 +01:00
Anthony Stirling 80e425926e PR auto deploy 2025-07-16 15:14:03 +01:00
Anthony StirlingandGitHub 58d53c7a6b test (#3963)
# Description of Changes

<!--
Please provide a summary of the changes, including:

- What was changed
- Why the change was made
- Any challenges encountered

Closes #(issue_number)
-->

---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/DeveloperGuide.md#6-testing)
for more details.
2025-07-16 14:49:34 +01:00
Anthony Stirling 89cda87490 re-delete files 2025-07-11 15:18:03 +01:00
Anthony Stirling af8e065e87 Merge remote-tracking branch 'origin/main' into feature/react-overhaul 2025-07-11 15:14:11 +01:00
21210850ec V2 docker support react version2 (#3930)
# Description of Changes

<!--
Please provide a summary of the changes, including:

- What was changed
- Why the change was made
- Any challenges encountered

Closes #(issue_number)
-->

---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/DeveloperGuide.md#6-testing)
for more details.

---------

Co-authored-by: a <a>
2025-07-11 14:16:34 +01:00
ConnorYohandGitHub 50d745225c Merge pull request #3833 from Stirling-Tools/feature/react-backend-config-access
Feature/react backend config access
2025-06-30 09:28:36 +01:00
Connor Yoh f446fcd50b Remove excess hidden, premium key and redundant code 2025-06-27 18:29:04 +01:00
ConnorYohandGitHub 618485576d Merge pull request #3834 from Stirling-Tools/pixeebot/feature/react-backend-config-access
Hardening suggestions for Stirling-PDF / feature/react-backend-config-access
2025-06-27 18:25:27 +01:00
pixeebot[bot]andGitHub e75213a206 Switch order of literals to prevent NullPointerException 2025-06-27 17:17:11 +00:00
Connor Yoh 248a14c571 Added config controller to be used by react frontend 2025-06-27 18:00:35 +01:00
Connor Yoh 62fb5025a9 Whitespace fix 2025-06-27 17:00:25 +01:00
ConnorYohandGitHub 3444aa6525 Merge pull request #3832 from Stirling-Tools/react-prop-merges
React prop merges
2025-06-27 16:20:20 +01:00
Connor Yoh 5786421616 Merge branch 'main' into react-prop-merges 2025-06-27 16:15:11 +01:00
Connor Yoh 0090d9f483 Remove accidental introduction of propriatary folder structure 2025-06-27 16:07:03 +01:00
ConnorYohandGitHub 29916d85b1 Merge pull request #3811 from reecebrowne/Stirling-2.0
Stirling 2.0
2025-06-25 16:07:22 +01:00
Reece ffe5b9577b Page editor fixes and improvements and restructuring 2025-06-24 23:31:21 +01:00
Reece 2f90220b7b File handling fixes 2025-06-24 20:25:03 +01:00
ConnorYohandGitHub bfc679edc5 Merge pull request #3663 from reecebrowne/Stirling-2.0
Stirling 2.0
2025-06-24 15:04:28 +01:00
Reece 27dd11ba8a Replace old components with moved versions from Stirling-2.0 2025-06-24 12:55:15 +01:00
Reece a51960b199 Resolve conflicts using files from Stirling-2.0 2025-06-24 12:51:39 +01:00
Reece 16e56e3208 Resolve merge conflicts by using versions from Stirling-2.0 2025-06-24 12:20:31 +01:00
Reece 25e9db2570 Clean up 2025-06-20 23:00:26 +01:00
Reece cbc5616a39 Translations for file management 2025-06-20 21:46:37 +01:00
Reece 215bb86a8e Bug fixing, file management changes 2025-06-20 20:22:18 +01:00
Reece 3ebf75ae6f File editor groundwork and refactor work 2025-06-20 17:51:24 +01:00
Reece 9c410865f9 Big refactor 2025-06-19 22:41:05 +01:00
Reece 868969192b Refactor 2025-06-19 19:47:44 +01:00
Reece d981968e0f Page editor redesign 2025-06-18 18:12:15 +01:00
Reece ae508730c3 Added quick access bar consolodated sidebars 2025-06-18 14:16:39 +01:00
Reece bf6a74210f Remove repeated tailwind imports causing issues 2025-06-18 13:36:24 +01:00
Reece e7995a0256 Page editor updates 2025-06-16 19:57:50 +01:00
Reece 7fc850b138 Update Page editor styling 2025-06-16 15:11:00 +01:00
Reece ac3da9b7c2 Developer guide updates and claude.md 2025-06-14 22:03:46 +01:00
Reece 5298cb1607 Ground work for page editor 2025-06-10 11:19:54 +01:00
Reece 41efd80dd5 RAINBOW MODE!!!! 2025-06-09 18:38:06 +01:00
Reece 342345cbb9 Fix viewer positioning error 2025-06-09 18:37:47 +01:00
Reece 6bf41a4cbe Remove Claude.md from gitignore 2025-06-09 10:53:32 +01:00
Reece 1e0949ff2b Css fixes, Tailwind forward 2025-06-08 13:45:45 +01:00
Reece b1584151ba Css overhaul 2025-06-06 17:20:06 +01:00
Reece 0c76bf2c8b Homepage refactor 2025-06-05 21:59:18 +01:00
Reece BrowneandGitHub b5d84db3c8 Stirling 2.0 (#3645)
# Description of Changes

Please provide a summary of the changes, including:

Vite fixes
Indexxdb

Closes #(issue_number)

---

## Checklist

### General

- [ ] I have read the [Contribution
Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md)
- [ ] I have read the [Stirling-PDF Developer
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md)
(if applicable)
- [ ] I have read the [How to add new languages to
Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/HowToAddNewLanguage.md)
(if applicable)
- [ ] I have performed a self-review of my own code
- [ ] My changes generate no new warnings

### Documentation

- [ ] I have updated relevant docs on [Stirling-PDF's doc
repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/)
(if functionality has heavily changed)
- [ ] I have read the section [Add New Translation
Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/HowToAddNewLanguage.md#add-new-translation-tags)
(for new translation tags only)

### UI Changes (if applicable)

- [ ] Screenshots or videos demonstrating the UI changes are attached
(e.g., as comments or direct attachments in the PR)

### Testing (if applicable)

- [ ] I have tested my changes locally. Refer to the [Testing
Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#6-testing)
for more details.
2025-06-05 11:12:39 +01:00
Reece 7dd79aa2a1 Fix url params and fix vite 2025-06-04 23:02:53 +01:00
Reece 61eac67dc1 Claude.md in gitignore 2025-06-04 19:06:07 +01:00
Reece 16f150a203 IndexxedDb refactored 2025-06-04 19:04:56 +01:00
Reece a3c4f1a305 IndexedDb integration 2025-06-04 19:04:18 +01:00
Reece 0650a6a877 Remove License file 2025-06-03 13:19:16 +01:00
Reece 09f05ac8d0 React translations 2025-05-29 17:26:32 +01:00
Reece 5f862f55d9 Restore theme toggle 2025-05-28 22:18:19 +01:00
Reece 15761ae743 Remove CRA and use Vite instead 2025-05-28 21:43:02 +01:00
Reece d216811317 - Viewer overhaul
-Dark mode toggle
-URL params improvements
-app.js set up fix
- UI clean up
2025-05-27 19:28:03 +01:00
Reece 41c82b15da TSX rewrite and query strings initial set up 2025-05-21 21:47:44 +01:00
Reece 5f7a4e1664 Merge branch 'Stirling-2.0' of https://github.com/reecebrowne/Stirling-PDF into Stirling-2.0 2025-05-19 13:14:58 +01:00
Reece Browne b5e7995554 File management groundwork 2025-05-15 23:12:39 +01:00
Reece Browne f789533c83 Mantine overhaul 2025-05-15 20:07:33 +01:00
Reece Browne d669964975 Basic homepage with compress and split 2025-05-13 23:32:54 +01:00
Reece Browne b567f4b110 Initial react set up 2025-05-09 20:01:09 +01:00
459 changed files with 168448 additions and 6725 deletions
+5 -1
View File
@@ -5,7 +5,11 @@
"Bash(mkdir:*)",
"Bash(./gradlew:*)",
"Bash(grep:*)",
"Bash(cat:*)"
"Bash(cat:*)",
"Bash(find:*)",
"Bash(npm test)",
"Bash(npm test:*)",
"Bash(ls:*)"
],
"deny": []
}
+1 -12
View File
@@ -49,7 +49,7 @@
"java.configuration.updateBuildConfiguration": "interactive",
"java.format.enabled": true,
"java.format.settings.profile": "GoogleStyle",
"java.format.settings.google.version": "1.28.0",
"java.format.settings.google.version": "1.26.0",
"java.format.settings.google.extra": "--aosp --skip-sorting-imports --skip-javadoc-formatting",
"java.saveActions.cleanup": true,
"java.cleanup.actions": [
@@ -79,17 +79,9 @@
".venv*/",
".vscode/",
"bin/",
"app/core/bin/",
"app/common/bin/",
"app/proprietary/bin/",
"build/",
"app/core/build/",
"app/common/build/",
"app/proprietary/build/",
"configs/",
"app/core/configs/",
"customFiles/",
"app/core/customFiles/",
"docs/",
"exampleYmlFiles",
"gradle/",
@@ -101,9 +93,6 @@
".git-blame-ignore-revs",
".gitattributes",
".gitignore",
"app/core/.gitignore",
"app/common/.gitignore",
"app/proprietary/.gitignore",
".pre-commit-config.yaml"
],
"java.signatureHelp.enabled": true,
+7 -1
View File
@@ -31,12 +31,18 @@ indent_size = 2
# CSS files typically use an indent size of 2 spaces for better readability and alignment with community standards.
indent_size = 2
[*.{yml,yaml}]
[*.yaml]
# YAML files use an indent size of 2 spaces to maintain consistency with common YAML formatting practices.
indent_size = 2
insert_final_newline = false
trim_trailing_whitespace = false
[*.yml]
# YML files follow the same conventions as YAML files, using an indent size of 2 spaces.
indent_size = 2
insert_final_newline = false
trim_trailing_whitespace = false
[*.json]
# JSON files use an indent size of 2 spaces, which is the standard for JSON formatting.
indent_size = 2
+2 -21
View File
@@ -1,21 +1,2 @@
# All PRs must be approved by Frooodle or Ludy87
* @Frooodle @Ludy87 @jbrunton96 @ConnorYoh
# Backend
/app/** @DarioGii
#V1 frontend
/app/core/src/main/resources/static/** @reecebrowne @ConnorYoh @EthanHealy01 @jbrunton96
/app/core/src/main/resources/templates/** @reecebrowne @ConnorYoh @EthanHealy01 @jbrunton96
#V2 frontend
/frontend/** @reecebrowne @ConnorYoh @EthanHealy01 @jbrunton96
#V2 docker
/docker/backend/** @Frooodle @Ludy87 @DarioGii
/docker/frontend/** @reecebrowne @ConnorYoh @EthanHealy01 @jbrunton96
/docker/compose/** @reecebrowne @ConnorYoh @EthanHealy01 @DarioGii @jbrunton96
#GHA (All users)
/.github/** @reecebrowne @ConnorYoh @EthanHealy01 @DarioGii @jbrunton96
# All PRs to V1 must be approved by Frooodle
* @Frooodle @reecebrowne @Ludy87 @DarioGii @ConnorYoh @EthanHealy01
+1 -1
View File
@@ -26,4 +26,4 @@ project: &project
- gradlew
- gradlew.bat
- launch4jConfig.xml
- settings.gradle
- settings.gradle
-1
View File
@@ -78,7 +78,6 @@ labels:
- 'app/core/src/main/resources/banner.txt'
- 'app/core/src/main/resources/static/python/png_to_webp.py'
- 'app/core/src/main/resources/static/python/split_photos.py'
- 'app/core/src/main/resources/static/pipeline/defaultWebUIConfigs/**'
- 'application.properties'
- label: 'Security'
-5
View File
@@ -42,7 +42,6 @@
- name: "Front End"
color: "BBD2F1"
description: "Issues or pull requests related to front-end development"
from_name: "frontend"
- name: "github-actions"
description: "Pull requests that update GitHub Actions code"
color: "999999"
@@ -78,7 +77,6 @@
- name: "Translation"
color: "9FABF9"
from_name: "translation"
description: "Issues or pull requests related to translation"
- name: "upstream"
color: "DEDEDE"
- name: "v2"
@@ -180,6 +178,3 @@
- name: "pr-deployed"
color: "00FF00"
description: "Pull request has been deployed to a test environment"
- name: "codex"
color: "ededed"
description: "chatgpt AI generated code"
+503
View File
@@ -0,0 +1,503 @@
name: Auto PR V2 Deployment
on:
pull_request:
types: [opened, synchronize, reopened, closed]
permissions:
contents: read
issues: write
pull-requests: write
jobs:
check-pr:
if: github.event.action != 'closed'
runs-on: ubuntu-latest
outputs:
should_deploy: ${{ steps.check-conditions.outputs.should_deploy }}
pr_number: ${{ github.event.number }}
pr_repository: ${{ steps.get-pr-info.outputs.repository }}
pr_ref: ${{ steps.get-pr-info.outputs.ref }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@002fdce3c6a235733a90a27c80493a3241e56863 # v2.12.1
with:
egress-policy: audit
- name: Check deployment conditions
id: check-conditions
env:
PR_TITLE: ${{ github.event.pull_request.title }}
PR_AUTHOR: ${{ github.event.pull_request.user.login }}
PR_BRANCH: ${{ github.event.pull_request.head.ref }}
run: |
echo "PR Title: $PR_TITLE"
echo "PR Author: $PR_AUTHOR"
echo "PR Branch: $PR_BRANCH"
echo "PR Base Branch: ${{ github.event.pull_request.base.ref }}"
# Define authorized users
authorized_users=(
"Frooodle"
"sf298"
"Ludy87"
"LaserKaspar"
"sbplat"
"reecebrowne"
"DarioGii"
"ConnorYoh"
"EthanHealy01"
)
# Check if author is in the authorized list
is_authorized=false
for user in "${authorized_users[@]}"; do
if [[ "$PR_AUTHOR" == "$user" ]]; then
is_authorized=true
break
fi
done
# If PR is targeting V2 and user is authorized, deploy unconditionally
PR_BASE_BRANCH="${{ github.event.pull_request.base.ref }}"
if [[ "$PR_BASE_BRANCH" == "V2" && "$is_authorized" == "true" ]]; then
echo "✅ Deployment forced: PR targets V2 and author is authorized."
echo "should_deploy=true" >> $GITHUB_OUTPUT
exit 0
fi
# Otherwise, continue with original keyword checks
has_v2_keyword=false
if [[ "$PR_TITLE" =~ [Vv]2 ]] || [[ "$PR_TITLE" =~ [Vv]ersion.?2 ]] || [[ "$PR_TITLE" =~ [Vv]ersion.?[Tt]wo ]]; then
has_v2_keyword=true
fi
has_branch_keyword=false
if [[ "$PR_BRANCH" =~ [Vv]2 ]] || [[ "$PR_BRANCH" =~ [Rr]eact ]]; then
has_branch_keyword=true
fi
if [[ "$is_authorized" == "true" && ( "$has_v2_keyword" == "true" || "$has_branch_keyword" == "true" ) ]]; then
echo "✅ Deployment conditions met"
echo "should_deploy=true" >> $GITHUB_OUTPUT
else
echo "❌ Deployment conditions not met"
echo " - Authorized user: $is_authorized"
echo " - Has V2 keyword in title: $has_v2_keyword"
echo " - Has V2/React keyword in branch: $has_branch_keyword"
echo "should_deploy=false" >> $GITHUB_OUTPUT
fi
- name: Get PR repository and ref
id: get-pr-info
if: steps.check-conditions.outputs.should_deploy == 'true'
run: |
# For forks, use the full repository name, for internal PRs use the current repo
if [[ "${{ github.event.pull_request.head.repo.fork }}" == "true" ]]; then
repository="${{ github.event.pull_request.head.repo.full_name }}"
else
repository="${{ github.repository }}"
fi
echo "repository=$repository" >> $GITHUB_OUTPUT
echo "ref=${{ github.event.pull_request.head.ref }}" >> $GITHUB_OUTPUT
deploy-v2-pr:
needs: check-pr
runs-on: ubuntu-latest
if: needs.check-pr.outputs.should_deploy == 'true'
# Concurrency control - only one deployment per PR at a time
concurrency:
group: v2-deploy-pr-${{ needs.check-pr.outputs.pr_number }}
cancel-in-progress: true
permissions:
contents: read
issues: write
pull-requests: write
steps:
- name: Harden Runner
uses: step-security/harden-runner@002fdce3c6a235733a90a27c80493a3241e56863 # v2.12.1
with:
egress-policy: audit
- name: Checkout main repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
repository: ${{ github.repository }}
ref: main
- name: Setup GitHub App Bot
if: github.actor != 'dependabot[bot]'
id: setup-bot
uses: ./.github/actions/setup-bot
continue-on-error: true
with:
app-id: ${{ secrets.GH_APP_ID }}
private-key: ${{ secrets.GH_APP_PRIVATE_KEY }}
- name: Add deployment started comment
id: deployment-started
uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea # v7.0.1
with:
github-token: ${{ steps.setup-bot.outputs.token }}
script: |
const { owner, repo } = context.repo;
const prNumber = ${{ needs.check-pr.outputs.pr_number }};
// Delete previous V2 deployment comments to avoid clutter
const { data: comments } = await github.rest.issues.listComments({
owner,
repo,
issue_number: prNumber,
per_page: 100
});
const v2Comments = comments.filter(comment =>
comment.body.includes('🚀 **Auto-deploying V2 version**') ||
comment.body.includes('## 🚀 V2 Auto-Deployment Complete!') ||
comment.body.includes('❌ **V2 Auto-deployment failed**')
);
for (const comment of v2Comments) {
console.log(`Deleting old V2 comment: ${comment.id}`);
await github.rest.issues.deleteComment({
owner,
repo,
comment_id: comment.id
});
}
// Create new deployment started comment
const { data: newComment } = await github.rest.issues.createComment({
owner,
repo,
issue_number: prNumber,
body: `🚀 **Auto-deploying V2 version** for PR #${prNumber}...\n\n_This is an automated deployment triggered by V2/version2 keywords in the PR title or V2/React keywords in the branch name._\n\n⚠️ **Note:** If new commits are pushed during deployment, this build will be cancelled and replaced with the latest version.`
});
return newComment.id;
- name: Checkout PR
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
repository: ${{ needs.check-pr.outputs.pr_repository }}
ref: ${{ needs.check-pr.outputs.pr_ref }}
token: ${{ secrets.GITHUB_TOKEN }}
fetch-depth: 0 # Fetch full history for commit hash detection
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1
- name: Get version number
id: versionNumber
run: |
VERSION=$(grep "^version =" build.gradle | awk -F'"' '{print $2}')
echo "versionNumber=$VERSION" >> $GITHUB_OUTPUT
- name: Login to Docker Hub
uses: docker/login-action@74a5d142397b4f367a81961eba4e8cd7edddf772 # v3.4.0
with:
username: ${{ secrets.DOCKER_HUB_USERNAME }}
password: ${{ secrets.DOCKER_HUB_API }}
- name: Get commit hashes for frontend and backend
id: commit-hashes
run: |
# Get last commit that touched the frontend folder, docker/frontend, or docker/compose
FRONTEND_HASH=$(git log -1 --format="%H" -- frontend/ docker/frontend/ docker/compose/ 2>/dev/null || echo "")
if [ -z "$FRONTEND_HASH" ]; then
FRONTEND_HASH="no-frontend-changes"
fi
# Get last commit that touched backend code, docker/backend, or docker/compose
BACKEND_HASH=$(git log -1 --format="%H" -- app/ docker/backend/ docker/compose/ 2>/dev/null || echo "")
if [ -z "$BACKEND_HASH" ]; then
BACKEND_HASH="no-backend-changes"
fi
echo "Frontend hash: $FRONTEND_HASH"
echo "Backend hash: $BACKEND_HASH"
echo "frontend_hash=$FRONTEND_HASH" >> $GITHUB_OUTPUT
echo "backend_hash=$BACKEND_HASH" >> $GITHUB_OUTPUT
# Short hashes for tags
if [ "$FRONTEND_HASH" = "no-frontend-changes" ]; then
echo "frontend_short=no-frontend" >> $GITHUB_OUTPUT
else
echo "frontend_short=${FRONTEND_HASH:0:8}" >> $GITHUB_OUTPUT
fi
if [ "$BACKEND_HASH" = "no-backend-changes" ]; then
echo "backend_short=no-backend" >> $GITHUB_OUTPUT
else
echo "backend_short=${BACKEND_HASH:0:8}" >> $GITHUB_OUTPUT
fi
- name: Check if frontend image exists
id: check-frontend
run: |
if docker manifest inspect ${{ secrets.DOCKER_HUB_USERNAME }}/test:v2-frontend-${{ steps.commit-hashes.outputs.frontend_short }} >/dev/null 2>&1; then
echo "exists=true" >> $GITHUB_OUTPUT
echo "Frontend image already exists, skipping build"
else
echo "exists=false" >> $GITHUB_OUTPUT
echo "Frontend image needs to be built"
fi
- name: Check if backend image exists
id: check-backend
run: |
if docker manifest inspect ${{ secrets.DOCKER_HUB_USERNAME }}/test:v2-backend-${{ steps.commit-hashes.outputs.backend_short }} >/dev/null 2>&1; then
echo "exists=true" >> $GITHUB_OUTPUT
echo "Backend image already exists, skipping build"
else
echo "exists=false" >> $GITHUB_OUTPUT
echo "Backend image needs to be built"
fi
- name: Build and push V2 frontend image
if: steps.check-frontend.outputs.exists == 'false'
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
with:
context: .
file: ./docker/frontend/Dockerfile
push: true
tags: ${{ secrets.DOCKER_HUB_USERNAME }}/test:v2-frontend-${{ steps.commit-hashes.outputs.frontend_short }}
build-args: VERSION_TAG=v2-alpha
platforms: linux/amd64
- name: Build and push V2 backend image
if: steps.check-backend.outputs.exists == 'false'
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
with:
context: .
file: ./docker/backend/Dockerfile
push: true
tags: ${{ secrets.DOCKER_HUB_USERNAME }}/test:v2-backend-${{ steps.commit-hashes.outputs.backend_short }}
build-args: VERSION_TAG=v2-alpha
platforms: linux/amd64
- name: Set up SSH
run: |
mkdir -p ~/.ssh/
echo "${{ secrets.VPS_SSH_KEY }}" > ../private.key
sudo chmod 600 ../private.key
- name: Deploy V2 to VPS
id: deploy
run: |
# Use same port strategy as regular PRs - just the PR number
V2_PORT=${{ needs.check-pr.outputs.pr_number }}
BACKEND_PORT=$((V2_PORT + 10000)) # Backend on higher port to avoid conflicts
# Create docker-compose for V2 with separate frontend and backend
cat > docker-compose.yml << EOF
version: '3.3'
services:
stirling-pdf-v2-backend:
container_name: stirling-pdf-v2-backend-pr-${{ needs.check-pr.outputs.pr_number }}
image: ${{ secrets.DOCKER_HUB_USERNAME }}/test:v2-backend-${{ steps.commit-hashes.outputs.backend_short }}
ports:
- "${BACKEND_PORT}:8080" # Backend API port
volumes:
- /stirling/V2-PR-${{ needs.check-pr.outputs.pr_number }}/data:/usr/share/tessdata:rw
- /stirling/V2-PR-${{ needs.check-pr.outputs.pr_number }}/config:/configs:rw
- /stirling/V2-PR-${{ needs.check-pr.outputs.pr_number }}/logs:/logs:rw
environment:
DISABLE_ADDITIONAL_FEATURES: "true"
SECURITY_ENABLELOGIN: "false"
SYSTEM_DEFAULTLOCALE: en-GB
UI_APPNAME: "Stirling-PDF V2 PR#${{ needs.check-pr.outputs.pr_number }}"
UI_HOMEDESCRIPTION: "V2 PR#${{ needs.check-pr.outputs.pr_number }} - Frontend/Backend Split Architecture"
UI_APPNAMENAVBAR: "V2 PR#${{ needs.check-pr.outputs.pr_number }}"
SYSTEM_MAXFILESIZE: "100"
METRICS_ENABLED: "true"
SYSTEM_GOOGLEVISIBILITY: "false"
SWAGGER_SERVER_URL: "http://${{ secrets.VPS_HOST }}:${V2_PORT}"
restart: on-failure:5
stirling-pdf-v2-frontend:
container_name: stirling-pdf-v2-frontend-pr-${{ needs.check-pr.outputs.pr_number }}
image: ${{ secrets.DOCKER_HUB_USERNAME }}/test:v2-frontend-${{ steps.commit-hashes.outputs.frontend_short }}
ports:
- "${V2_PORT}:80" # Frontend port (same as regular PRs)
environment:
VITE_API_BASE_URL: "http://${{ secrets.VPS_HOST }}:${BACKEND_PORT}"
depends_on:
- stirling-pdf-v2-backend
restart: on-failure:5
EOF
# Deploy to VPS
scp -i ../private.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null docker-compose.yml ${{ secrets.VPS_USERNAME }}@${{ secrets.VPS_HOST }}:/tmp/docker-compose-v2.yml
ssh -i ../private.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -T ${{ secrets.VPS_USERNAME }}@${{ secrets.VPS_HOST }} << ENDSSH
# Create V2 PR-specific directories
mkdir -p /stirling/V2-PR-${{ needs.check-pr.outputs.pr_number }}/{data,config,logs}
# Move docker-compose file to correct location
mv /tmp/docker-compose-v2.yml /stirling/V2-PR-${{ needs.check-pr.outputs.pr_number }}/docker-compose.yml
# Stop any existing container and clean up
cd /stirling/V2-PR-${{ needs.check-pr.outputs.pr_number }}
docker-compose down --remove-orphans 2>/dev/null || true
# Start the new container
docker-compose pull
docker-compose up -d
# Clean up unused Docker resources to save space
docker system prune -af --volumes
# Clean up old backend/frontend images (older than 2 weeks)
docker image prune -af --filter "until=336h" --filter "label!=keep=true"
ENDSSH
# Set port for output
echo "v2_port=${V2_PORT}" >> $GITHUB_OUTPUT
- name: Post V2 deployment URL to PR
if: success()
uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea # v7.0.1
with:
github-token: ${{ steps.setup-bot.outputs.token }}
script: |
const { owner, repo } = context.repo;
const prNumber = ${{ needs.check-pr.outputs.pr_number }};
const v2Port = ${{ steps.deploy.outputs.v2_port }};
// Delete the "deploying..." comment since we're posting the final result
const deploymentStartedId = ${{ steps.deployment-started.outputs.result }};
if (deploymentStartedId) {
console.log(`Deleting deployment started comment: ${deploymentStartedId}`);
try {
await github.rest.issues.deleteComment({
owner,
repo,
comment_id: deploymentStartedId
});
} catch (error) {
console.log(`Could not delete deployment started comment: ${error.message}`);
}
}
const deploymentUrl = `http://${{ secrets.VPS_HOST }}:${v2Port}`;
const httpsUrl = `https://${v2Port}.ssl.stirlingpdf.cloud`;
const commentBody = `## 🚀 V2 Auto-Deployment Complete!\n\n` +
`Your V2 PR with the new frontend/backend split architecture has been deployed!\n\n` +
`🔗 **Direct Test URL (non-SSL)** [${deploymentUrl}](${deploymentUrl})\n\n` +
`🔐 **Secure HTTPS URL**: [${httpsUrl}](${httpsUrl})\n\n` +
`_This deployment will be automatically cleaned up when the PR is closed._\n\n` +
`🔄 **Auto-deployed** because PR title or branch name contains V2/version2/React keywords.`;
await github.rest.issues.createComment({
owner,
repo,
issue_number: prNumber,
body: commentBody
});
cleanup-v2-deployment:
if: github.event.action == 'closed'
runs-on: ubuntu-latest
permissions:
contents: read
issues: write
pull-requests: write
steps:
- name: Harden Runner
uses: step-security/harden-runner@6c439dc8bdf85cadbbce9ed30d1c7b959517bc49 # v2.12.2
with:
egress-policy: audit
- name: Checkout repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Setup GitHub App Bot
if: github.actor != 'dependabot[bot]'
id: setup-bot
uses: ./.github/actions/setup-bot
continue-on-error: true
with:
app-id: ${{ secrets.GH_APP_ID }}
private-key: ${{ secrets.GH_APP_PRIVATE_KEY }}
- name: Clean up V2 deployment comments
uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea # v7.0.1
with:
github-token: ${{ steps.setup-bot.outputs.token }}
script: |
const { owner, repo } = context.repo;
const prNumber = ${{ github.event.pull_request.number }};
// Find and delete V2 deployment comments
const { data: comments } = await github.rest.issues.listComments({
owner,
repo,
issue_number: prNumber
});
const v2Comments = comments.filter(c =>
c.body?.includes("## 🚀 V2 Auto-Deployment Complete!") &&
c.user?.type === "Bot"
);
for (const comment of v2Comments) {
await github.rest.issues.deleteComment({
owner,
repo,
comment_id: comment.id
});
console.log(`Deleted V2 deployment comment (ID: ${comment.id})`);
}
- name: Set up SSH
run: |
mkdir -p ~/.ssh/
echo "${{ secrets.VPS_SSH_KEY }}" > ../private.key
sudo chmod 600 ../private.key
- name: Cleanup V2 deployment
run: |
ssh -i ../private.key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -T ${{ secrets.VPS_USERNAME }}@${{ secrets.VPS_HOST }} << 'ENDSSH'
if [ -d "/stirling/V2-PR-${{ github.event.pull_request.number }}" ]; then
echo "Found V2 PR directory, proceeding with cleanup..."
# Stop and remove V2 containers
cd /stirling/V2-PR-${{ github.event.pull_request.number }}
docker-compose down || true
# Go back to root before removal
cd /
# Remove V2 PR-specific directories
rm -rf /stirling/V2-PR-${{ github.event.pull_request.number }}
# Clean up V2 containers by name (in case compose cleanup missed them)
docker rm -f stirling-pdf-v2-frontend-pr-${{ github.event.pull_request.number }} || true
docker rm -f stirling-pdf-v2-backend-pr-${{ github.event.pull_request.number }} || true
echo "V2 cleanup completed"
else
echo "V2 PR directory not found, nothing to clean up"
fi
# Clean up old unused images (older than 2 weeks) but keep recent ones for reuse
docker image prune -af --filter "until=336h" --filter "label!=keep=true"
# Note: We don't remove the commit-based images since they can be reused across PRs
# Only remove PR-specific containers and directories
ENDSSH
- name: Cleanup temporary files
if: always()
run: |
rm -f ../private.key
continue-on-error: true
@@ -41,7 +41,7 @@ jobs:
enable_enterprise: ${{ steps.check-pro-flag.outputs.enable_enterprise }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@ec9f2d5744a09debf3a187a3f4f675c53b671911 # v2.13.0
uses: step-security/harden-runner@6c439dc8bdf85cadbbce9ed30d1c7b959517bc49 # v2.12.2
with:
egress-policy: audit
@@ -152,7 +152,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ec9f2d5744a09debf3a187a3f4f675c53b671911 # v2.13.0
uses: step-security/harden-runner@6c439dc8bdf85cadbbce9ed30d1c7b959517bc49 # v2.12.2
with:
egress-policy: audit
@@ -196,7 +196,7 @@ jobs:
uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1
- name: Login to Docker Hub
uses: docker/login-action@184bdaa0721073962dff0199f1fb9940f07167d1 # v3.5.0
uses: docker/login-action@74a5d142397b4f367a81961eba4e8cd7edddf772 # v3.4.0
with:
username: ${{ secrets.DOCKER_HUB_USERNAME }}
password: ${{ secrets.DOCKER_HUB_API }}
+1 -1
View File
@@ -21,7 +21,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ec9f2d5744a09debf3a187a3f4f675c53b671911 # v2.13.0
uses: step-security/harden-runner@6c439dc8bdf85cadbbce9ed30d1c7b959517bc49 # v2.12.2
with:
egress-policy: audit
+2 -2
View File
@@ -19,7 +19,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ec9f2d5744a09debf3a187a3f4f675c53b671911 # v2.13.0
uses: step-security/harden-runner@6c439dc8bdf85cadbbce9ed30d1c7b959517bc49 # v2.12.2
with:
egress-policy: audit
@@ -87,7 +87,7 @@ jobs:
- name: AI PR Title Analysis
if: steps.actor.outputs.is_repo_dev == 'true'
id: ai-title-analysis
uses: actions/ai-inference@0cbed4a10641c75090de5968e66d70eb4660f751 # v1.2.7
uses: actions/ai-inference@d645f067d89ee1d5d736a5990e327e504d1c5a4a # v1.1.0
with:
model: openai/gpt-4o
system-prompt-file: ".github/config/system-prompt.txt"
+1 -1
View File
@@ -13,7 +13,7 @@ jobs:
pull-requests: write
steps:
- name: Harden Runner
uses: step-security/harden-runner@ec9f2d5744a09debf3a187a3f4f675c53b671911 # v2.13.0
uses: step-security/harden-runner@6c439dc8bdf85cadbbce9ed30d1c7b959517bc49 # v2.12.2
with:
egress-policy: audit
+56 -143
View File
@@ -1,23 +1,9 @@
name: Build and Test Workflow
on:
workflow_dispatch:
# push:
# branches: ["main"]
pull_request:
branches: ["main"]
# cancel in-progress jobs if a new job is triggered
# This is useful to avoid running multiple builds for the same branch if a new commit is pushed
# or a pull request is updated.
# It helps to save resources and time by ensuring that only the latest commit is built and tested
# This is particularly useful for long-running jobs that may take a while to complete.
# The `group` is set to a combination of the workflow name, event name, and branch name.
# This ensures that jobs are grouped by the workflow and branch, allowing for cancellation of
# in-progress jobs when a new commit is pushed to the same branch or a new pull request is opened.
concurrency:
group: ${{ github.workflow }}-${{ github.event_name }}-${{ github.event.pull_request.number || github.ref_name || github.ref }}
cancel-in-progress: true
branches: ["main", "V2", "V2-gha"]
workflow_dispatch:
permissions:
contents: read
@@ -27,60 +13,50 @@ jobs:
name: detect what files changed
runs-on: ubuntu-latest
timeout-minutes: 3
# Map a step output to a job output
outputs:
build: ${{ steps.changes.outputs.build }}
app: ${{ steps.changes.outputs.app }}
project: ${{ steps.changes.outputs.project }}
openapi: ${{ steps.changes.outputs.openapi }}
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- uses: actions/checkout@v2
- name: Check for file changes
uses: dorny/paths-filter@de90cc6fb38fc0963ad72b210f1f284cd68cea36 # v3.0.2
uses: dorny/paths-filter@v3.0.2
id: changes
with:
filters: ".github/config/.files.yaml"
filters: .github/config/.files.yaml
build:
runs-on: ubuntu-latest
permissions:
actions: read
security-events: write
strategy:
fail-fast: false
matrix:
jdk-version: [17, 21]
spring-security: [true, false]
steps:
- name: Harden Runner
uses: step-security/harden-runner@ec9f2d5744a09debf3a187a3f4f675c53b671911 # v2.13.0
uses: step-security/harden-runner@v2.12.2
with:
egress-policy: audit
- name: Checkout repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
uses: actions/checkout@v4.2.2
- name: Set up JDK ${{ matrix.jdk-version }}
uses: actions/setup-java@c5195efecf7bdfc987ee8bae7a71cb8b11521c00 # v4.7.1
uses: actions/setup-java@v4.7.1
with:
java-version: ${{ matrix.jdk-version }}
distribution: "temurin"
- name: Setup Gradle
uses: gradle/actions/setup-gradle@ac638b010cf58a27ee6c972d7336334ccaf61c96 # v4.4.1
uses: gradle/actions/setup-gradle@v4.4.1
with:
gradle-version: 8.14
- name: Build with Gradle and spring security ${{ matrix.spring-security }}
run: ./gradlew clean build
env:
DISABLE_ADDITIONAL_FEATURES: ${{ matrix.spring-security }}
- name: Check Test Reports Exist
id: check-reports
if: always()
run: |
declare -a dirs=(
@@ -91,98 +67,101 @@ jobs:
"app/proprietary/build/reports/tests/"
"app/proprietary/build/test-results/"
)
missing_reports=()
for dir in "${dirs[@]}"; do
if [ ! -d "$dir" ]; then
missing_reports+=("$dir")
echo "Missing $dir"
exit 1
fi
done
if [ ${#missing_reports[@]} -gt 0 ]; then
echo "ERROR: The following required test report directories are missing:"
printf '%s\n' "${missing_reports[@]}"
exit 1
fi
echo "All required test report directories are present"
- name: Upload Test Reports
if: always()
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
uses: actions/upload-artifact@v4.6.2
with:
name: test-reports-jdk-${{ matrix.jdk-version }}-spring-security-${{ matrix.spring-security }}
path: |
app/core/build/reports/tests/
app/core/build/test-results/
app/core/build/reports/problems/
app/common/build/reports/tests/
app/common/build/test-results/
app/common/build/reports/problems/
app/proprietary/build/reports/tests/
app/proprietary/build/test-results/
app/proprietary/build/reports/problems/
app/**/build/reports/tests/
app/**/build/test-results/
app/**/build/reports/problems/
build/reports/problems/
retention-days: 3
if-no-files-found: warn
check-generateOpenApiDocs:
if: needs.files-changed.outputs.openapi == 'true'
needs: [files-changed, build]
needs: [files-changed]
runs-on: ubuntu-latest
steps:
- name: Harden Runner
uses: step-security/harden-runner@ec9f2d5744a09debf3a187a3f4f675c53b671911 # v2.13.0
uses: step-security/harden-runner@v2.12.2
with:
egress-policy: audit
- name: Checkout repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- uses: actions/checkout@v4.2.2
- name: Set up JDK 17
uses: actions/setup-java@c5195efecf7bdfc987ee8bae7a71cb8b11521c00 # v4.7.1
uses: actions/setup-java@v4.7.1
with:
java-version: "17"
distribution: "temurin"
- name: Setup Gradle
uses: gradle/actions/setup-gradle@ac638b010cf58a27ee6c972d7336334ccaf61c96 # v4.4.1
- uses: gradle/actions/setup-gradle@v4.4.1
- name: Generate OpenAPI documentation
run: ./gradlew :stirling-pdf:generateOpenApiDocs
- name: Upload OpenAPI Documentation
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
uses: actions/upload-artifact@v4.6.2
with:
name: openapi-docs
path: ./SwaggerDoc.json
frontend-validation:
runs-on: ubuntu-latest
steps:
- name: Harden Runner
uses: step-security/harden-runner@v2.12.2
with:
egress-policy: audit
- name: Checkout repository
uses: actions/checkout@v4.2.2
- name: Set up Node.js
uses: actions/setup-node@v4.1.0
with:
node-version: '20'
cache: 'npm'
cache-dependency-path: frontend/package-lock.json
- name: Install frontend dependencies
run: cd frontend && npm ci
- name: Build frontend
run: cd frontend && npm run build
- name: Run frontend tests
run: cd frontend && npm run test -- --run
- name: Upload frontend build artifacts
uses: actions/upload-artifact@v4.6.2
with:
name: frontend-build
path: frontend/dist/
retention-days: 3
check-licence:
if: needs.files-changed.outputs.build == 'true'
needs: [files-changed, build]
runs-on: ubuntu-latest
steps:
- name: Harden Runner
uses: step-security/harden-runner@ec9f2d5744a09debf3a187a3f4f675c53b671911 # v2.13.0
uses: step-security/harden-runner@v2.12.2
with:
egress-policy: audit
- name: Checkout repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
uses: actions/checkout@v4.2.2
- name: Set up JDK 17
uses: actions/setup-java@c5195efecf7bdfc987ee8bae7a71cb8b11521c00 # v4.7.1
uses: actions/setup-java@v4.7.1
with:
java-version: "17"
distribution: "temurin"
- name: check the licenses for compatibility
run: ./gradlew clean checkLicense
- name: FAILED - check the licenses for compatibility
if: failure()
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
uses: actions/upload-artifact@v4.6.2
with:
name: dependencies-without-allowed-license.json
path: |
build/reports/dependency-license/dependencies-without-allowed-license.json
path: build/reports/dependency-license/dependencies-without-allowed-license.json
retention-days: 3
docker-compose-tests:
@@ -206,7 +185,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ec9f2d5744a09debf3a187a3f4f675c53b671911 # v2.13.0
uses: step-security/harden-runner@6c439dc8bdf85cadbbce9ed30d1c7b959517bc49 # v2.12.2
with:
egress-policy: audit
@@ -244,69 +223,3 @@ jobs:
chmod +x ./testing/test.sh
chmod +x ./testing/test_disabledEndpoints.sh
./testing/test.sh
test-build-docker-images:
if: github.event_name == 'pull_request' && needs.files-changed.outputs.project == 'true'
needs: [files-changed, build, check-generateOpenApiDocs, check-licence]
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
docker-rev: ["Dockerfile", "Dockerfile.ultra-lite", "Dockerfile.fat"]
steps:
- name: Harden Runner
uses: step-security/harden-runner@ec9f2d5744a09debf3a187a3f4f675c53b671911 # v2.13.0
with:
egress-policy: audit
- name: Checkout Repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Set up JDK 17
uses: actions/setup-java@c5195efecf7bdfc987ee8bae7a71cb8b11521c00 # v4.7.1
with:
java-version: "17"
distribution: "temurin"
- name: Set up Gradle
uses: gradle/actions/setup-gradle@ac638b010cf58a27ee6c972d7336334ccaf61c96 # v4.4.1
with:
gradle-version: 8.14
- name: Build application
run: ./gradlew clean build
env:
DISABLE_ADDITIONAL_FEATURES: true
STIRLING_PDF_DESKTOP_UI: false
- name: Set up QEMU
uses: docker/setup-qemu-action@29109295f81e9208d7d86ff1c6c12d2833863392 # v3.6.0
- name: Set up Docker Buildx
id: buildx
uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1
- name: Build ${{ matrix.docker-rev }}
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
with:
builder: ${{ steps.buildx.outputs.name }}
context: .
file: ./${{ matrix.docker-rev }}
push: false
cache-from: type=gha
cache-to: type=gha,mode=max
platforms: linux/amd64,linux/arm64/v8
provenance: true
sbom: true
- name: Upload Reports
if: always()
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: reports-docker-${{ matrix.docker-rev }}
path: |
build/reports/tests/
build/test-results/
build/reports/problems/
retention-days: 3
if-no-files-found: warn
+3 -15
View File
@@ -6,18 +6,6 @@ on:
paths:
- "app/core/src/main/resources/messages_*.properties"
# cancel in-progress jobs if a new job is triggered
# This is useful to avoid running multiple builds for the same branch if a new commit is pushed
# or a pull request is updated.
# It helps to save resources and time by ensuring that only the latest commit is built and tested
# This is particularly useful for long-running jobs that may take a while to complete.
# The `group` is set to a combination of the workflow name, event name, and branch name.
# This ensures that jobs are grouped by the workflow and branch, allowing for cancellation of
# in-progress jobs when a new commit is pushed to the same branch or a new pull request is opened.
concurrency:
group: ${{ github.workflow }}-${{ github.event_name }}-${{ github.event.pull_request.number || github.ref_name || github.ref }}
cancel-in-progress: true
permissions:
contents: read # Allow read access to repository content
@@ -30,7 +18,7 @@ jobs:
pull-requests: write # Allow writing to pull requests
steps:
- name: Harden Runner
uses: step-security/harden-runner@ec9f2d5744a09debf3a187a3f4f675c53b671911 # v2.13.0
uses: step-security/harden-runner@6c439dc8bdf85cadbbce9ed30d1c7b959517bc49 # v2.12.2
with:
egress-policy: audit
@@ -127,7 +115,7 @@ jobs:
// Filter for relevant files based on the PR changes
const changedFiles = files
.filter(file =>
.filter(file =>
file.status !== "removed" &&
/^app\/core\/src\/main\/resources\/messages_[a-zA-Z_]{2}_[a-zA-Z_]{2,7}\.properties$/.test(file.filename)
)
@@ -289,4 +277,4 @@ jobs:
rm -rf pr-branch
rm -f pr-branch-messages_en_GB.properties main-branch-messages_en_GB.properties changed_files.txt result.txt
echo "Cleanup complete."
continue-on-error: true # Ensure cleanup runs even if previous steps fail
continue-on-error: true # Ensure cleanup runs even if previous steps fail
+1 -1
View File
@@ -17,7 +17,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Harden Runner
uses: step-security/harden-runner@ec9f2d5744a09debf3a187a3f4f675c53b671911 # v2.13.0
uses: step-security/harden-runner@6c439dc8bdf85cadbbce9ed30d1c7b959517bc49 # v2.12.2
with:
egress-policy: audit
+188
View File
@@ -0,0 +1,188 @@
name: Auto V2 Deploy on Push
on:
push:
branches:
- V2
- deploy-on-v2-commit
permissions:
contents: read
jobs:
deploy-v2-on-push:
runs-on: ubuntu-latest
concurrency:
group: deploy-v2-push-V2
cancel-in-progress: true
steps:
- name: Harden Runner
uses: step-security/harden-runner@002fdce3c6a235733a90a27c80493a3241e56863 # v2.12.1
with:
egress-policy: audit
- name: Checkout code
uses: actions/checkout@v4
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Get commit hashes for frontend and backend
id: commit-hashes
run: |
# Get last commit that touched the frontend folder, docker/frontend, or docker/compose
FRONTEND_HASH=$(git log -1 --format="%H" -- frontend/ docker/frontend/ docker/compose/ 2>/dev/null || echo "")
if [ -z "$FRONTEND_HASH" ]; then
FRONTEND_HASH="no-frontend-changes"
fi
# Get last commit that touched backend code, docker/backend, or docker/compose
BACKEND_HASH=$(git log -1 --format="%H" -- app/ docker/backend/ docker/compose/ 2>/dev/null || echo "")
if [ -z "$BACKEND_HASH" ]; then
BACKEND_HASH="no-backend-changes"
fi
echo "Frontend hash: $FRONTEND_HASH"
echo "Backend hash: $BACKEND_HASH"
echo "frontend_hash=$FRONTEND_HASH" >> $GITHUB_OUTPUT
echo "backend_hash=$BACKEND_HASH" >> $GITHUB_OUTPUT
# Short hashes for tags
if [ "$FRONTEND_HASH" = "no-frontend-changes" ]; then
echo "frontend_short=no-frontend" >> $GITHUB_OUTPUT
else
echo "frontend_short=${FRONTEND_HASH:0:8}" >> $GITHUB_OUTPUT
fi
if [ "$BACKEND_HASH" = "no-backend-changes" ]; then
echo "backend_short=no-backend" >> $GITHUB_OUTPUT
else
echo "backend_short=${BACKEND_HASH:0:8}" >> $GITHUB_OUTPUT
fi
- name: Check if frontend image exists
id: check-frontend
run: |
if docker manifest inspect ${{ secrets.DOCKER_HUB_USERNAME }}/test:v2-frontend-${{ steps.commit-hashes.outputs.frontend_short }} >/dev/null 2>&1; then
echo "exists=true" >> $GITHUB_OUTPUT
echo "Frontend image already exists, skipping build"
else
echo "exists=false" >> $GITHUB_OUTPUT
echo "Frontend image needs to be built"
fi
- name: Check if backend image exists
id: check-backend
run: |
if docker manifest inspect ${{ secrets.DOCKER_HUB_USERNAME }}/test:v2-backend-${{ steps.commit-hashes.outputs.backend_short }} >/dev/null 2>&1; then
echo "exists=true" >> $GITHUB_OUTPUT
echo "Backend image already exists, skipping build"
else
echo "exists=false" >> $GITHUB_OUTPUT
echo "Backend image needs to be built"
fi
- name: Login to Docker Hub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKER_HUB_USERNAME }}
password: ${{ secrets.DOCKER_HUB_API }}
- name: Build and push frontend image
if: steps.check-frontend.outputs.exists == 'false'
uses: docker/build-push-action@v6
with:
context: .
file: ./docker/frontend/Dockerfile
push: true
tags: |
${{ secrets.DOCKER_HUB_USERNAME }}/test:v2-frontend-${{ steps.commit-hashes.outputs.frontend_short }}
${{ secrets.DOCKER_HUB_USERNAME }}/test:v2-frontend-latest
build-args: VERSION_TAG=v2-alpha
platforms: linux/amd64
- name: Build and push backend image
if: steps.check-backend.outputs.exists == 'false'
uses: docker/build-push-action@v6
with:
context: .
file: ./docker/backend/Dockerfile
push: true
tags: |
${{ secrets.DOCKER_HUB_USERNAME }}/test:v2-backend-${{ steps.commit-hashes.outputs.backend_short }}
${{ secrets.DOCKER_HUB_USERNAME }}/test:v2-backend-latest
build-args: VERSION_TAG=v2-alpha
platforms: linux/amd64
- name: Set up SSH
run: |
mkdir -p ~/.ssh/
echo "${{ secrets.VPS_SSH_KEY }}" > ../private.key
chmod 600 ../private.key
- name: Deploy to VPS on port 3000
run: |
export UNIQUE_NAME=docker-compose-v2-$GITHUB_RUN_ID.yml
cat > $UNIQUE_NAME << EOF
version: '3.3'
services:
backend:
container_name: stirling-v2-backend
image: ${{ secrets.DOCKER_HUB_USERNAME }}/test:v2-backend-${{ steps.commit-hashes.outputs.backend_short }}
ports:
- "13000:8080"
volumes:
- /stirling/V2/data:/usr/share/tessdata:rw
- /stirling/V2/config:/configs:rw
- /stirling/V2/logs:/logs:rw
environment:
DISABLE_ADDITIONAL_FEATURES: "true"
SECURITY_ENABLELOGIN: "false"
SYSTEM_DEFAULTLOCALE: en-GB
UI_APPNAME: "Stirling-PDF V2"
UI_HOMEDESCRIPTION: "V2 Frontend/Backend Split"
UI_APPNAMENAVBAR: "V2 Deployment"
SYSTEM_MAXFILESIZE: "100"
METRICS_ENABLED: "true"
SYSTEM_GOOGLEVISIBILITY: "false"
SWAGGER_SERVER_URL: "http://${{ secrets.VPS_HOST }}:3000"
restart: on-failure:5
frontend:
container_name: stirling-v2-frontend
image: ${{ secrets.DOCKER_HUB_USERNAME }}/test:v2-frontend-${{ steps.commit-hashes.outputs.frontend_short }}
ports:
- "3000:80"
environment:
VITE_API_BASE_URL: "http://${{ secrets.VPS_HOST }}:13000"
depends_on:
- backend
restart: on-failure:5
EOF
# Copy to remote with unique name
scp -i ../private.key -o StrictHostKeyChecking=no $UNIQUE_NAME ${{ secrets.VPS_USERNAME }}@${{ secrets.VPS_HOST }}:/tmp/$UNIQUE_NAME
# SSH and rename/move atomically to avoid interference
ssh -i ../private.key -o StrictHostKeyChecking=no ${{ secrets.VPS_USERNAME }}@${{ secrets.VPS_HOST }} << ENDSSH
mkdir -p /stirling/V2/{data,config,logs}
mv /tmp/$UNIQUE_NAME /stirling/V2/docker-compose.yml
cd /stirling/V2
docker-compose down || true
docker-compose pull
docker-compose up -d
docker system prune -af --volumes
docker image prune -af --filter "until=336h" --filter "label!=keep=true"
ENDSSH
- name: Cleanup temporary files
if: always()
run: |
rm -f ../private.key
@@ -0,0 +1,217 @@
name: Frontend License Report Workflow
on:
push:
branches:
- V2
paths:
- "frontend/package.json"
- "frontend/package-lock.json"
- "frontend/scripts/generate-licenses.js"
pull_request:
branches:
- V2
paths:
- "frontend/package.json"
- "frontend/package-lock.json"
- "frontend/scripts/generate-licenses.js"
permissions:
contents: read
jobs:
generate-frontend-license-report:
runs-on: ubuntu-latest
permissions:
contents: write
pull-requests: write
repository-projects: write # Required for enabling automerge
steps:
- name: Harden Runner
uses: step-security/harden-runner@6c439dc8bdf85cadbbce9ed30d1c7b959517bc49 # v2.12.2
with:
egress-policy: audit
- name: Check out code
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
fetch-depth: 0
- name: Setup GitHub App Bot
id: setup-bot
uses: ./.github/actions/setup-bot
with:
app-id: ${{ secrets.GH_APP_ID }}
private-key: ${{ secrets.GH_APP_PRIVATE_KEY }}
- name: Set up Node.js
uses: actions/setup-node@39370e3970a6d050c480ffad4ff0ed4d3fdee5af # v4.1.0
with:
node-version: '18'
cache: 'npm'
cache-dependency-path: frontend/package-lock.json
- name: Install frontend dependencies
working-directory: frontend
run: npm ci
- name: Generate frontend license report
working-directory: frontend
run: npm run generate-licenses
- name: Check for license warnings
run: |
if [ -f "frontend/src/assets/license-warnings.json" ]; then
echo "LICENSE_WARNINGS_EXIST=true" >> $GITHUB_ENV
else
echo "LICENSE_WARNINGS_EXIST=false" >> $GITHUB_ENV
fi
# PR Event: Check licenses and comment on PR
- name: Delete previous license check comments
if: github.event_name == 'pull_request'
uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea # v7.0.1
with:
github-token: ${{ steps.setup-bot.outputs.token }}
script: |
const { owner, repo } = context.repo;
const prNumber = context.issue.number;
// Get all comments on the PR
const { data: comments } = await github.rest.issues.listComments({
owner,
repo,
issue_number: prNumber,
per_page: 100
});
// Filter for license check comments
const licenseComments = comments.filter(comment =>
comment.body.includes('## ✅ Frontend License Check Passed') ||
comment.body.includes('## ❌ Frontend License Check Failed')
);
// Delete old license check comments
for (const comment of licenseComments) {
console.log(`Deleting old license check comment: ${comment.id}`);
await github.rest.issues.deleteComment({
owner,
repo,
comment_id: comment.id
});
}
- name: Comment on PR - License Check Results
if: github.event_name == 'pull_request'
uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea # v7.0.1
with:
github-token: ${{ steps.setup-bot.outputs.token }}
script: |
const { owner, repo } = context.repo;
const prNumber = context.issue.number;
const hasWarnings = process.env.LICENSE_WARNINGS_EXIST === 'true';
let commentBody;
if (hasWarnings) {
// Read warnings file to get specific issues
const fs = require('fs');
let warningDetails = '';
try {
const warnings = JSON.parse(fs.readFileSync('frontend/src/assets/license-warnings.json', 'utf8'));
warningDetails = warnings.warnings.map(w => `- ${w.message}`).join('\n');
} catch (e) {
warningDetails = 'Unable to read warning details';
}
commentBody = `## ❌ Frontend License Check Failed
The frontend license check has detected compatibility warnings that require review:
${warningDetails}
**Action Required:** Please review these licenses to ensure they are acceptable for your use case before merging.
_This check will fail the PR until license issues are resolved._`;
} else {
commentBody = `## ✅ Frontend License Check Passed
All frontend licenses have been validated and no compatibility warnings were detected.
The frontend license report has been updated successfully.`;
}
await github.rest.issues.createComment({
owner,
repo,
issue_number: prNumber,
body: commentBody
});
- name: Fail workflow if license warnings exist (PR only)
if: github.event_name == 'pull_request' && env.LICENSE_WARNINGS_EXIST == 'true'
run: |
echo "❌ License warnings detected. Failing the workflow."
exit 1
# Push Event: Commit license files and create PR
- name: Commit changes (Push only)
if: github.event_name == 'push'
run: |
git add frontend/src/assets/3rdPartyLicenses.json
# Note: Do NOT commit license-warnings.json - it's only for PR review
git diff --staged --quiet || echo "CHANGES_DETECTED=true" >> $GITHUB_ENV
- name: Prepare PR body (Push only)
if: github.event_name == 'push'
run: |
PR_BODY="Auto-generated by ${{ steps.setup-bot.outputs.app-slug }}[bot]
This PR updates the frontend license report based on changes to package.json dependencies."
if [ "${{ env.LICENSE_WARNINGS_EXIST }}" = "true" ]; then
PR_BODY="$PR_BODY
## ⚠️ License Compatibility Warnings
The following licenses may require review for corporate compatibility:
$(cat frontend/src/assets/license-warnings.json | jq -r '.warnings[].message')
Please review these licenses to ensure they are acceptable for your use case."
fi
echo "PR_BODY<<EOF" >> $GITHUB_ENV
echo "$PR_BODY" >> $GITHUB_ENV
echo "EOF" >> $GITHUB_ENV
- name: Create Pull Request (Push only)
id: cpr
if: github.event_name == 'push' && env.CHANGES_DETECTED == 'true'
uses: peter-evans/create-pull-request@271a8d0340265f705b14b6d32b9829c1cb33d45e # v7.0.8
with:
token: ${{ steps.setup-bot.outputs.token }}
commit-message: "Update Frontend 3rd Party Licenses"
committer: ${{ steps.setup-bot.outputs.committer }}
author: ${{ steps.setup-bot.outputs.committer }}
signoff: true
branch: update-frontend-3rd-party-licenses
base: V2
title: "Update Frontend 3rd Party Licenses"
body: ${{ env.PR_BODY }}
labels: Licenses,github-actions,frontend
draft: false
delete-branch: true
sign-commits: true
- name: Enable Pull Request Automerge (Push only)
if: github.event_name == 'push' && steps.cpr.outputs.pull-request-operation == 'created' && env.LICENSE_WARNINGS_EXIST == 'false'
run: gh pr merge --squash --auto "${{ steps.cpr.outputs.pull-request-number }}"
env:
GH_TOKEN: ${{ steps.setup-bot.outputs.token }}
- name: Add review required label (Push only)
if: github.event_name == 'push' && steps.cpr.outputs.pull-request-operation == 'created' && env.LICENSE_WARNINGS_EXIST == 'true'
run: gh pr edit "${{ steps.cpr.outputs.pull-request-number }}" --add-label "license-review-required"
env:
GH_TOKEN: ${{ steps.setup-bot.outputs.token }}
+1 -13
View File
@@ -7,18 +7,6 @@ on:
paths:
- "build.gradle"
# cancel in-progress jobs if a new job is triggered
# This is useful to avoid running multiple builds for the same branch if a new commit is pushed
# or a pull request is updated.
# It helps to save resources and time by ensuring that only the latest commit is built and tested
# This is particularly useful for long-running jobs that may take a while to complete.
# The `group` is set to a combination of the workflow name, event name, and branch name.
# This ensures that jobs are grouped by the workflow and branch, allowing for cancellation of
# in-progress jobs when a new commit is pushed to the same branch or a new pull request is opened.
concurrency:
group: ${{ github.workflow }}-${{ github.event_name }}-${{ github.ref_name || github.ref }}
cancel-in-progress: true
permissions:
contents: read
@@ -31,7 +19,7 @@ jobs:
repository-projects: write # Required for enabling automerge
steps:
- name: Harden Runner
uses: step-security/harden-runner@ec9f2d5744a09debf3a187a3f4f675c53b671911 # v2.13.0
uses: step-security/harden-runner@6c439dc8bdf85cadbbce9ed30d1c7b959517bc49 # v2.12.2
with:
egress-policy: audit
+1 -1
View File
@@ -15,7 +15,7 @@ jobs:
issues: write
steps:
- name: Harden Runner
uses: step-security/harden-runner@ec9f2d5744a09debf3a187a3f4f675c53b671911 # v2.13.0
uses: step-security/harden-runner@6c439dc8bdf85cadbbce9ed30d1c7b959517bc49 # v2.12.2
with:
egress-policy: audit
+7 -7
View File
@@ -21,7 +21,7 @@ jobs:
versionMac: ${{ steps.versionNumberMac.outputs.versionNumberMac }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@ec9f2d5744a09debf3a187a3f4f675c53b671911 # v2.13.0
uses: step-security/harden-runner@6c439dc8bdf85cadbbce9ed30d1c7b959517bc49 # v2.12.2
with:
egress-policy: audit
@@ -60,7 +60,7 @@ jobs:
file_suffix: ""
steps:
- name: Harden Runner
uses: step-security/harden-runner@ec9f2d5744a09debf3a187a3f4f675c53b671911 # v2.13.0
uses: step-security/harden-runner@6c439dc8bdf85cadbbce9ed30d1c7b959517bc49 # v2.12.2
with:
egress-policy: audit
@@ -110,7 +110,7 @@ jobs:
file_suffix: ""
steps:
- name: Harden Runner
uses: step-security/harden-runner@ec9f2d5744a09debf3a187a3f4f675c53b671911 # v2.13.0
uses: step-security/harden-runner@6c439dc8bdf85cadbbce9ed30d1c7b959517bc49 # v2.12.2
with:
egress-policy: audit
@@ -148,7 +148,7 @@ jobs:
contents: write
steps:
- name: Harden Runner
uses: step-security/harden-runner@ec9f2d5744a09debf3a187a3f4f675c53b671911 # v2.13.0
uses: step-security/harden-runner@6c439dc8bdf85cadbbce9ed30d1c7b959517bc49 # v2.12.2
with:
egress-policy: audit
@@ -238,7 +238,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Harden Runner
uses: step-security/harden-runner@ec9f2d5744a09debf3a187a3f4f675c53b671911 # v2.13.0
uses: step-security/harden-runner@6c439dc8bdf85cadbbce9ed30d1c7b959517bc49 # v2.12.2
with:
egress-policy: audit
@@ -252,7 +252,7 @@ jobs:
- name: Install Cosign
if: matrix.os == 'windows-latest'
uses: sigstore/cosign-installer@d58896d6a1865668819e1d91763c7751a165e159 # v3.9.2
uses: sigstore/cosign-installer@398d4b0eeef1380460a10c8013a76f728fb906ac # v3.9.1
- name: Generate key pair
if: matrix.os == 'windows-latest'
@@ -301,7 +301,7 @@ jobs:
contents: write
steps:
- name: Harden Runner
uses: step-security/harden-runner@ec9f2d5744a09debf3a187a3f4f675c53b671911 # v2.13.0
uses: step-security/harden-runner@6c439dc8bdf85cadbbce9ed30d1c7b959517bc49 # v2.12.2
with:
egress-policy: audit
+3 -13
View File
@@ -2,9 +2,8 @@ name: Pre-commit
on:
workflow_dispatch:
push:
branches:
- main
schedule:
- cron: "0 0 * * 1"
permissions:
contents: read
@@ -17,7 +16,7 @@ jobs:
pull-requests: write
steps:
- name: Harden Runner
uses: step-security/harden-runner@ec9f2d5744a09debf3a187a3f4f675c53b671911 # v2.13.0
uses: step-security/harden-runner@6c439dc8bdf85cadbbce9ed30d1c7b959517bc49 # v2.12.2
with:
egress-policy: audit
@@ -47,15 +46,6 @@ jobs:
- run: pre-commit run --all-files -c .pre-commit-config.yaml
continue-on-error: true
- name: Set up JDK
uses: actions/setup-java@c5195efecf7bdfc987ee8bae7a71cb8b11521c00 # v4.7.1
with:
java-version: 17
distribution: "temurin"
- name: Build with Gradle
run: ./gradlew clean build
- name: git add
run: |
git add .
+7 -19
View File
@@ -7,18 +7,6 @@ on:
- master
- main
# cancel in-progress jobs if a new job is triggered
# This is useful to avoid running multiple builds for the same branch if a new commit is pushed
# or a pull request is updated.
# It helps to save resources and time by ensuring that only the latest commit is built and tested
# This is particularly useful for long-running jobs that may take a while to complete.
# The `group` is set to a combination of the workflow name, event name, and branch name.
# This ensures that jobs are grouped by the workflow and branch, allowing for cancellation of
# in-progress jobs when a new commit is pushed to the same branch or a new pull request is opened.
concurrency:
group: ${{ github.workflow }}-${{ github.event_name }}-${{ github.ref_name || github.ref }}
cancel-in-progress: true
permissions:
contents: read
@@ -30,7 +18,7 @@ jobs:
id-token: write
steps:
- name: Harden Runner
uses: step-security/harden-runner@ec9f2d5744a09debf3a187a3f4f675c53b671911 # v2.13.0
uses: step-security/harden-runner@6c439dc8bdf85cadbbce9ed30d1c7b959517bc49 # v2.12.2
with:
egress-policy: audit
@@ -54,7 +42,7 @@ jobs:
- name: Install cosign
if: github.ref == 'refs/heads/master'
uses: sigstore/cosign-installer@d58896d6a1865668819e1d91763c7751a165e159 # v3.9.2
uses: sigstore/cosign-installer@398d4b0eeef1380460a10c8013a76f728fb906ac # v3.9.1
with:
cosign-release: "v2.4.1"
@@ -67,13 +55,13 @@ jobs:
run: echo "versionNumber=$(./gradlew printVersion --quiet | tail -1)" >> $GITHUB_OUTPUT
- name: Login to Docker Hub
uses: docker/login-action@184bdaa0721073962dff0199f1fb9940f07167d1 # v3.5.0
uses: docker/login-action@74a5d142397b4f367a81961eba4e8cd7edddf772 # v3.4.0
with:
username: ${{ secrets.DOCKER_HUB_USERNAME }}
password: ${{ secrets.DOCKER_HUB_API }}
- name: Login to GitHub Container Registry
uses: docker/login-action@184bdaa0721073962dff0199f1fb9940f07167d1 # v3.5.0
uses: docker/login-action@74a5d142397b4f367a81961eba4e8cd7edddf772 # v3.4.0
with:
registry: ghcr.io
username: ${{ github.actor }}
@@ -88,7 +76,7 @@ jobs:
- name: Generate tags
id: meta
uses: docker/metadata-action@c1e51972afc2121e065aed6d45c65596fe445f3f # v5.8.0
uses: docker/metadata-action@902fa8ec7d6ecbf8d84d538b9b233a880e428804 # v5.7.0
if: github.ref != 'refs/heads/main'
with:
images: |
@@ -134,7 +122,7 @@ jobs:
- name: Generate tags ultra-lite
id: meta2
uses: docker/metadata-action@c1e51972afc2121e065aed6d45c65596fe445f3f # v5.8.0
uses: docker/metadata-action@902fa8ec7d6ecbf8d84d538b9b233a880e428804 # v5.7.0
if: github.ref != 'refs/heads/main'
with:
images: |
@@ -165,7 +153,7 @@ jobs:
- name: Generate tags fat
id: meta3
uses: docker/metadata-action@c1e51972afc2121e065aed6d45c65596fe445f3f # v5.8.0
uses: docker/metadata-action@902fa8ec7d6ecbf8d84d538b9b233a880e428804 # v5.7.0
with:
images: |
${{ secrets.DOCKER_HUB_USERNAME }}/s-pdf
+4 -4
View File
@@ -23,7 +23,7 @@ jobs:
version: ${{ steps.versionNumber.outputs.versionNumber }}
steps:
- name: Harden Runner
uses: step-security/harden-runner@ec9f2d5744a09debf3a187a3f4f675c53b671911 # v2.13.0
uses: step-security/harden-runner@6c439dc8bdf85cadbbce9ed30d1c7b959517bc49 # v2.12.2
with:
egress-policy: audit
@@ -83,7 +83,7 @@ jobs:
file_suffix: ""
steps:
- name: Harden Runner
uses: step-security/harden-runner@ec9f2d5744a09debf3a187a3f4f675c53b671911 # v2.13.0
uses: step-security/harden-runner@6c439dc8bdf85cadbbce9ed30d1c7b959517bc49 # v2.12.2
with:
egress-policy: audit
@@ -95,7 +95,7 @@ jobs:
run: ls -R
- name: Install Cosign
uses: sigstore/cosign-installer@d58896d6a1865668819e1d91763c7751a165e159 # v3.9.2
uses: sigstore/cosign-installer@398d4b0eeef1380460a10c8013a76f728fb906ac # v3.9.1
- name: Generate key pair
run: cosign generate-key-pair
@@ -161,7 +161,7 @@ jobs:
file_suffix: ""
steps:
- name: Harden Runner
uses: step-security/harden-runner@ec9f2d5744a09debf3a187a3f4f675c53b671911 # v2.13.0
uses: step-security/harden-runner@6c439dc8bdf85cadbbce9ed30d1c7b959517bc49 # v2.12.2
with:
egress-policy: audit
+2 -2
View File
@@ -34,7 +34,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ec9f2d5744a09debf3a187a3f4f675c53b671911 # v2.13.0
uses: step-security/harden-runner@6c439dc8bdf85cadbbce9ed30d1c7b959517bc49 # v2.12.2
with:
egress-policy: audit
@@ -74,6 +74,6 @@ jobs:
# Upload the results to GitHub's code scanning dashboard.
- name: "Upload to code-scanning"
uses: github/codeql-action/upload-sarif@51f77329afa6477de8c49fc9c7046c15b9a4e79d # v3.29.5
uses: github/codeql-action/upload-sarif@181d5eefc20863364f96762470ba6f862bdef56b # v3.29.2
with:
sarif_file: results.sarif
+1 -13
View File
@@ -9,18 +9,6 @@ on:
- main
workflow_dispatch:
# cancel in-progress jobs if a new job is triggered
# This is useful to avoid running multiple builds for the same branch if a new commit is pushed
# or a pull request is updated.
# It helps to save resources and time by ensuring that only the latest commit is built and tested
# This is particularly useful for long-running jobs that may take a while to complete.
# The `group` is set to a combination of the workflow name, event name, and branch name.
# This ensures that jobs are grouped by the workflow and branch, allowing for cancellation of
# in-progress jobs when a new commit is pushed to the same branch or a new pull request is opened.
concurrency:
group: ${{ github.workflow }}-${{ github.event_name }}-${{ github.event.pull_request.number || github.ref_name || github.ref }}
cancel-in-progress: true
permissions:
pull-requests: read
actions: read
@@ -30,7 +18,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Harden Runner
uses: step-security/harden-runner@ec9f2d5744a09debf3a187a3f4f675c53b671911 # v2.13.0
uses: step-security/harden-runner@6c439dc8bdf85cadbbce9ed30d1c7b959517bc49 # v2.12.2
with:
egress-policy: audit
+1 -1
View File
@@ -16,7 +16,7 @@ jobs:
pull-requests: write
steps:
- name: Harden Runner
uses: step-security/harden-runner@ec9f2d5744a09debf3a187a3f4f675c53b671911 # v2.13.0
uses: step-security/harden-runner@6c439dc8bdf85cadbbce9ed30d1c7b959517bc49 # v2.12.2
with:
egress-policy: audit
+1 -13
View File
@@ -6,18 +6,6 @@ on:
branches:
- master
# cancel in-progress jobs if a new job is triggered
# This is useful to avoid running multiple builds for the same branch if a new commit is pushed
# or a pull request is updated.
# It helps to save resources and time by ensuring that only the latest commit is built and tested
# This is particularly useful for long-running jobs that may take a while to complete.
# The `group` is set to a combination of the workflow name, event name, and branch name.
# This ensures that jobs are grouped by the workflow and branch, allowing for cancellation of
# in-progress jobs when a new commit is pushed to the same branch or a new pull request is opened.
concurrency:
group: ${{ github.workflow }}-${{ github.event_name }}-${{ github.ref_name || github.ref }}
cancel-in-progress: true
permissions:
contents: read
@@ -26,7 +14,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Harden Runner
uses: step-security/harden-runner@ec9f2d5744a09debf3a187a3f4f675c53b671911 # v2.13.0
uses: step-security/harden-runner@6c439dc8bdf85cadbbce9ed30d1c7b959517bc49 # v2.12.2
with:
egress-policy: audit
+1 -13
View File
@@ -12,18 +12,6 @@ on:
- "app/core/src/main/resources/static/3rdPartyLicenses.json"
- "scripts/ignore_translation.toml"
# cancel in-progress jobs if a new job is triggered
# This is useful to avoid running multiple builds for the same branch if a new commit is pushed
# or a pull request is updated.
# It helps to save resources and time by ensuring that only the latest commit is built and tested
# This is particularly useful for long-running jobs that may take a while to complete.
# The `group` is set to a combination of the workflow name, event name, and branch name.
# This ensures that jobs are grouped by the workflow and branch, allowing for cancellation of
# in-progress jobs when a new commit is pushed to the same branch or a new pull request is opened.
concurrency:
group: ${{ github.workflow }}-${{ github.event_name }}-${{ github.ref_name || github.ref }}
cancel-in-progress: true
permissions:
contents: read
@@ -32,7 +20,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Harden Runner
uses: step-security/harden-runner@ec9f2d5744a09debf3a187a3f4f675c53b671911 # v2.13.0
uses: step-security/harden-runner@6c439dc8bdf85cadbbce9ed30d1c7b959517bc49 # v2.12.2
with:
egress-policy: audit
+4 -16
View File
@@ -4,18 +4,6 @@ on:
push:
branches: ["master", "UITest", "testdriver"]
# cancel in-progress jobs if a new job is triggered
# This is useful to avoid running multiple builds for the same branch if a new commit is pushed
# or a pull request is updated.
# It helps to save resources and time by ensuring that only the latest commit is built and tested
# This is particularly useful for long-running jobs that may take a while to complete.
# The `group` is set to a combination of the workflow name, event name, and branch name.
# This ensures that jobs are grouped by the workflow and branch, allowing for cancellation of
# in-progress jobs when a new commit is pushed to the same branch or a new pull request is opened.
concurrency:
group: ${{ github.workflow }}-${{ github.event_name }}-${{ github.ref_name || github.ref }}
cancel-in-progress: true
permissions:
contents: read
@@ -24,7 +12,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Harden Runner
uses: step-security/harden-runner@ec9f2d5744a09debf3a187a3f4f675c53b671911 # v2.13.0
uses: step-security/harden-runner@6c439dc8bdf85cadbbce9ed30d1c7b959517bc49 # v2.12.2
with:
egress-policy: audit
@@ -57,7 +45,7 @@ jobs:
echo "versionNumber=$VERSION" >> $GITHUB_OUTPUT
- name: Login to Docker Hub
uses: docker/login-action@184bdaa0721073962dff0199f1fb9940f07167d1 # v3.5.0
uses: docker/login-action@74a5d142397b4f367a81961eba4e8cd7edddf772 # v3.4.0
with:
username: ${{ secrets.DOCKER_HUB_USERNAME }}
password: ${{ secrets.DOCKER_HUB_API }}
@@ -122,7 +110,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ec9f2d5744a09debf3a187a3f4f675c53b671911 # v2.13.0
uses: step-security/harden-runner@6c439dc8bdf85cadbbce9ed30d1c7b959517bc49 # v2.12.2
with:
egress-policy: audit
@@ -156,7 +144,7 @@ jobs:
steps:
- name: Harden Runner
uses: step-security/harden-runner@ec9f2d5744a09debf3a187a3f4f675c53b671911 # v2.13.0
uses: step-security/harden-runner@6c439dc8bdf85cadbbce9ed30d1c7b959517bc49 # v2.12.2
with:
egress-policy: audit
+7 -4
View File
@@ -27,6 +27,7 @@ clientWebUI/
!cucumber/exampleFiles/
!cucumber/exampleFiles/example_html.zip
exampleYmlFiles/stirling/
/stirling/
/testing/file_snapshots
SwaggerDoc.json
@@ -124,10 +125,10 @@ SwaggerDoc.json
*.tar.gz
*.rar
*.db
build
app/core/build
app/common/build
app/proprietary/build
/build
/app/core/build
/app/common/build
/app/proprietary/build
common/build
proprietary/build
stirling-pdf/build
@@ -196,6 +197,8 @@ id_ed25519.pub
.pytest_cache
.ipynb_checkpoints
**/jcef-bundle/
# node_modules
+3 -3
View File
@@ -1,6 +1,6 @@
repos:
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.12.7
rev: v0.12.0
hooks:
- id: ruff
args:
@@ -22,7 +22,7 @@ repos:
files: \.(html|css|js|py|md)$
exclude: (.vscode|.devcontainer|app/core/src/main/resources|app/proprietary/src/main/resources|Dockerfile|.*/pdfjs.*|.*/thirdParty.*|bootstrap.*|.*\.min\..*|.*diff\.js)
- repo: https://github.com/gitleaks/gitleaks
rev: v8.28.0
rev: v8.27.2
hooks:
- id: gitleaks
- repo: https://github.com/pre-commit/pre-commit-hooks
@@ -43,4 +43,4 @@ repos:
# - stylelint-config-standard@38.0.0
# - "@stylistic/stylelint-plugin@3.1.3"
# files: \.(css)$
# args: [--fix]
# args: [--fix]
+1 -2
View File
@@ -2,7 +2,6 @@
"editor.wordSegmenterLocales": "",
"editor.guides.bracketPairs": "active",
"editor.guides.bracketPairsHorizontal": "active",
"editor.defaultFormatter": "EditorConfig.EditorConfig",
"cSpell.enabled": false,
"[feature]": {
"editor.defaultFormatter": "alexkrechik.cucumberautocomplete"
@@ -41,7 +40,7 @@
"java.configuration.updateBuildConfiguration": "interactive",
"java.format.enabled": true,
"java.format.settings.profile": "GoogleStyle",
"java.format.settings.google.version": "1.28.0",
"java.format.settings.google.version": "1.27.0",
"java.format.settings.google.extra": "--aosp --skip-sorting-imports --skip-javadoc-formatting",
// (DE) Aktiviert Kommentare im Java-Format.
// (EN) Enables comments in Java formatting.
+160
View File
@@ -0,0 +1,160 @@
# CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Common Development Commands
### Build and Test
- **Build project**: `./gradlew clean build`
- **Run locally**: `./gradlew bootRun`
- **Full test suite**: `./test.sh` (builds all Docker variants and runs comprehensive tests)
- **Code formatting**: `./gradlew spotlessApply` (runs automatically before compilation)
### Docker Development
- **Build ultra-lite**: `docker build -t stirlingtools/stirling-pdf:latest-ultra-lite -f ./Dockerfile.ultra-lite .`
- **Build standard**: `docker build -t stirlingtools/stirling-pdf:latest -f ./Dockerfile .`
- **Build fat version**: `docker build -t stirlingtools/stirling-pdf:latest-fat -f ./Dockerfile.fat .`
- **Example compose files**: Located in `exampleYmlFiles/` directory
### Security Mode Development
Set `DOCKER_ENABLE_SECURITY=true` environment variable to enable security features during development. This is required for testing the full version locally.
### Frontend Development
- **Frontend dev server**: `cd frontend && npm run dev` (requires backend on localhost:8080)
- **Tech Stack**: Vite + React + TypeScript + Mantine UI + TailwindCSS
- **Proxy Configuration**: Vite proxies `/api/*` calls to backend (localhost:8080)
- **Build Process**: DO NOT run build scripts manually - builds are handled by CI/CD pipelines
- **Package Installation**: DO NOT run npm install commands - package management handled separately
- **Deployment Options**:
- **Desktop App**: `npm run tauri-build` (native desktop application)
- **Web Server**: `npm run build` then serve dist/ folder
- **Development**: `npm run tauri-dev` for desktop dev mode
#### Multi-Tool Workflow Architecture
Frontend designed for **stateful document processing**:
- Users upload PDFs once, then chain tools (split → merge → compress → view)
- File state and processing results persist across tool switches
- No file reloading between tools - performance critical for large PDFs (up to 100GB+)
#### FileContext - Central State Management
**Location**: `src/contexts/FileContext.tsx`
- **Active files**: Currently loaded PDFs and their variants
- **Tool navigation**: Current mode (viewer/pageEditor/fileEditor/toolName)
- **Memory management**: PDF document cleanup, blob URL lifecycle, Web Worker management
- **IndexedDB persistence**: File storage with thumbnail caching
- **Preview system**: Tools can preview results (e.g., Split → Viewer → back to Split) without context pollution
**Critical**: All file operations go through FileContext. Don't bypass with direct file handling.
#### Processing Services
- **enhancedPDFProcessingService**: Background PDF parsing and manipulation
- **thumbnailGenerationService**: Web Worker-based with main-thread fallback
- **fileStorage**: IndexedDB with LRU cache management
#### Memory Management Strategy
**Why manual cleanup exists**: Large PDFs (up to 100GB+) through multiple tools accumulate:
- PDF.js documents that need explicit .destroy() calls
- Blob URLs from tool outputs that need revocation
- Web Workers that need termination
Without cleanup: browser crashes with memory leaks.
#### Tool Development
- **Pattern**: Follow `src/tools/Split.tsx` as reference implementation
- **File Access**: Tools receive `selectedFiles` prop (computed from activeFiles based on user selection)
- **File Selection**: Users select files in FileEditor (tool mode) → stored as IDs → computed to File objects for tools
- **Integration**: All files are part of FileContext ecosystem - automatic memory management and operation tracking
- **Parameters**: Tool parameter handling patterns still being standardized
- **Preview Integration**: Tools can implement preview functionality (see Split tool's thumbnail preview)
## Architecture Overview
### Project Structure
- **Backend**: Spring Boot application with Thymeleaf templating
- **Frontend**: React-based SPA in `/frontend` directory (Thymeleaf templates fully replaced)
- **File Storage**: IndexedDB for client-side file persistence and thumbnails
- **Internationalization**: JSON-based translations (converted from backend .properties)
- **PDF Processing**: PDFBox for core PDF operations, LibreOffice for conversions, PDF.js for client-side rendering
- **Security**: Spring Security with optional authentication (controlled by `DOCKER_ENABLE_SECURITY`)
- **Configuration**: YAML-based configuration with environment variable overrides
### Controller Architecture
- **API Controllers** (`src/main/java/.../controller/api/`): REST endpoints for PDF operations
- Organized by function: converters, security, misc, pipeline
- Follow pattern: `@RestController` + `@RequestMapping("/api/v1/...")`
- **Web Controllers** (`src/main/java/.../controller/web/`): Serve Thymeleaf templates
- Pattern: `@Controller` + return template names
### Key Components
- **SPDFApplication.java**: Main application class with desktop UI and browser launching logic
- **ConfigInitializer**: Handles runtime configuration and settings files
- **Pipeline System**: Automated PDF processing workflows via `PipelineController`
- **Security Layer**: Authentication, authorization, and user management (when enabled)
### Component Architecture
- **React Components**: Located in `frontend/src/components/` and `frontend/src/tools/`
- **Static Assets**: CSS, JS, and resources in `src/main/resources/static/` (legacy) + `frontend/public/` (modern)
- **Internationalization**:
- Backend: `messages_*.properties` files
- Frontend: JSON files in `frontend/public/locales/` (converted from .properties)
- Conversion Script: `scripts/convert_properties_to_json.py`
### Configuration Modes
- **Ultra-lite**: Basic PDF operations only
- **Standard**: Full feature set
- **Fat**: Pre-downloaded dependencies for air-gapped environments
- **Security Mode**: Adds authentication, user management, and enterprise features
### Testing Strategy
- **Integration Tests**: Cucumber tests in `testing/cucumber/`
- **Docker Testing**: `test.sh` validates all Docker variants
- **Manual Testing**: No unit tests currently - relies on UI and API testing
## Development Workflow
1. **Local Development**:
- Backend: `./gradlew bootRun` (runs on localhost:8080)
- Frontend: `cd frontend && npm run dev` (runs on localhost:5173, proxies to backend)
2. **Docker Testing**: Use `./test.sh` before submitting PRs
3. **Code Style**: Spotless enforces Google Java Format automatically
4. **Translations**:
- Backend: Use helper scripts in `/scripts` for multi-language updates
- Frontend: Update JSON files in `frontend/public/locales/` or use conversion script
5. **Documentation**: API docs auto-generated and available at `/swagger-ui/index.html`
## Frontend Architecture Status
- **Core Status**: React SPA architecture complete with multi-tool workflow support
- **State Management**: FileContext handles all file operations and tool navigation
- **File Processing**: Production-ready with memory management for large PDF workflows (up to 100GB+)
- **Tool Integration**: Standardized tool interface - see `src/tools/Split.tsx` as reference
- **Preview System**: Tool results can be previewed without polluting file context (Split tool example)
- **Performance**: Web Worker thumbnails, IndexedDB persistence, background processing
## Important Notes
- **Java Version**: Minimum JDK 17, supports and recommends JDK 21
- **Lombok**: Used extensively - ensure IDE plugin is installed
- **Desktop Mode**: Set `STIRLING_PDF_DESKTOP_UI=true` for desktop application mode
- **File Persistence**:
- **Backend**: Designed to be stateless - files are processed in memory/temp locations only
- **Frontend**: Uses IndexedDB for client-side file storage and caching (with thumbnails)
- **Security**: When `DOCKER_ENABLE_SECURITY=false`, security-related classes are excluded from compilation
- **FileContext**: All file operations MUST go through FileContext - never bypass with direct File handling
- **Memory Management**: Manual cleanup required for PDF.js documents and blob URLs - don't remove cleanup code
- **Tool Development**: New tools should follow Split tool pattern (`src/tools/Split.tsx`)
- **Performance Target**: Must handle PDFs up to 100GB+ without browser crashes
- **Preview System**: Tools can preview results without polluting main file context (see Split tool implementation)
## Communication Style
- Be direct and to the point
- No apologies or conversational filler
- Answer questions directly without preamble
- Explain reasoning concisely when asked
- Avoid unnecessary elaboration
## Decision Making
- Ask clarifying questions before making assumptions
- Stop and ask when uncertain about project-specific details
- Confirm approach before making structural changes
- Request guidance on preferences (cross-platform vs specific tools, etc.)
- Verify understanding of requirements before proceeding
+692
View File
@@ -0,0 +1,692 @@
# Stirling-PDF Developer Guide
## 1. Introduction
Stirling-PDF is a robust, locally hosted, web-based PDF manipulation tool. **Stirling 2.0** represents a complete frontend rewrite, replacing the legacy Thymeleaf-based UI with a modern React SPA (Single Page Application).
This guide focuses on developing for Stirling 2.0, including both the React frontend and Spring Boot backend development workflows.
## 2. Project Overview
**Stirling 2.0** is built using:
**Backend:**
- Spring Boot (Java 17+, JDK 21 recommended)
- PDFBox for core PDF operations
- LibreOffice for document conversions
- qpdf for PDF optimization
- Spring Security (optional, controlled by `DOCKER_ENABLE_SECURITY`)
- Lombok for reducing boilerplate code
**Frontend (React SPA):**
- React + TypeScript
- Vite for build tooling and development server
- Mantine UI component library
- TailwindCSS for styling
- PDF.js for client-side PDF rendering
- PDF-LIB.js for client-side PDF manipulation
- IndexedDB for client-side file storage and thumbnails
- i18next for internationalization
**Infrastructure:**
- Docker for containerization
- Gradle for build management
**Legacy (reference only during development):**
- Thymeleaf templates (being completely replaced in 2.0)
## 3. Development Environment Setup
### Prerequisites
- Docker
- Git
- Java JDK 17 or later (JDK 21 recommended)
- Node.js 18+ and npm (required for frontend development)
- Gradle 7.0 or later (Included within the repo)
### Setup Steps
1. Clone the repository:
```bash
git clone https://github.com/Stirling-Tools/Stirling-PDF.git
cd Stirling-PDF
```
2. Install Docker and JDK17 if not already installed.
3. Install a recommended Java IDE such as Eclipse, IntelliJ, or VSCode
1. Only VSCode
1. Open VS Code.
2. When prompted, install the recommended extensions.
3. Alternatively, open the command palette (`Ctrl + Shift + P` or `Cmd + Shift + P` on macOS) and run:
```sh
Extensions: Show Recommended Extensions
```
4. Install the required extensions from the list.
4. Lombok Setup
Stirling-PDF uses Lombok to reduce boilerplate code. Some IDEs, like Eclipse, don't support Lombok out of the box. To set up Lombok in your development environment:
Visit the [Lombok website](https://projectlombok.org/setup/) for installation instructions specific to your IDE.
5. Add environment variable
For local testing, you should generally be testing the full 'Security' version of Stirling PDF. To do this, you must add the environment flag DISABLE_ADDITIONAL_FEATURES=false to your system and/or IDE build/run step.
5. **Frontend Setup (Required for Stirling 2.0)**
Navigate to the frontend directory and install dependencies using npm.
## 4. Stirling 2.0 Development Workflow
### Frontend Development (React)
The frontend is a React SPA that runs independently during development:
1. **Start the backend**: Run the Spring Boot application (serves API endpoints on localhost:8080)
2. **Start the frontend dev server**: Navigate to the frontend directory and run the development server (serves UI on localhost:5173)
3. **Development flow**: The Vite dev server automatically proxies API calls to the backend
### File Storage Architecture
Stirling 2.0 uses client-side file storage:
- **IndexedDB**: Stores files locally in the browser with automatic thumbnail generation
- **PDF.js**: Handles client-side PDF rendering and processing
- **URL Parameters**: Support for deep linking and tool state persistence
### Legacy Code Reference
The existing Thymeleaf templates remain in the codebase during development as reference material but will be completely removed for the 2.0 release.
## 5. Project Structure
```bash
Stirling-PDF/
├── .github/ # GitHub-specific files (workflows, issue templates)
├── configs/ # Configuration files used by stirling at runtime (generated at runtime)
├── frontend/ # React SPA frontend (Stirling 2.0)
│ ├── src/
│ │ ├── components/ # React components
│ │ ├── tools/ # Tool-specific React components
│ │ ├── hooks/ # Custom React hooks
│ │ ├── services/ # API and utility services
│ │ ├── types/ # TypeScript type definitions
│ │ └── utils/ # Utility functions
│ ├── public/
│ │ └── locales/ # Internationalization files (JSON)
│ ├── package.json # Frontend dependencies
│ └── vite.config.ts # Vite configuration
├── customFiles/ # Custom static files and templates (generated at runtime used to replace existing files)
├── docs/ # Documentation files
├── exampleYmlFiles/ # Example YAML configuration files
├── images/ # Image assets
├── pipeline/ # Pipeline-related files (generated at runtime)
├── scripts/ # Utility scripts
├── src/ # Source code
│ ├── main/
│ │ ├── java/
│ │ │ └── stirling/
│ │ │ └── software/
│ │ │ └── SPDF/
│ │ │ ├── config/
│ │ │ ├── controller/
│ │ │ ├── model/
│ │ │ ├── repository/
│ │ │ ├── service/
│ │ │ └── utils/
│ │ └── resources/
│ │ ├── static/ # Legacy static assets (reference only)
│ │ │ ├── css/
│ │ │ ├── js/
│ │ │ └── pdfjs/
│ │ └── templates/ # Legacy Thymeleaf templates (reference only)
│ └── test/
├── testing/ # Cucumber and integration tests
│ └── cucumber/ # Cucumber test files
├── build.gradle # Gradle build configuration
├── Dockerfile # Main Dockerfile
├── Dockerfile.ultra-lite # Dockerfile for ultra-lite version
├── Dockerfile.fat # Dockerfile for fat version
├── docker-compose.yml # Docker Compose configuration
└── test.sh # Test script to deploy all docker versions and run cuke tests
```
## 6. Docker-based Development
Stirling-PDF offers several Docker versions:
- Full: All features included
- Ultra-Lite: Basic PDF operations only
- Fat: Includes additional libraries and fonts predownloaded
### Example Docker Compose Files
Stirling-PDF provides several example Docker Compose files in the `exampleYmlFiles` directory, such as:
- `docker-compose-latest.yml`: Latest version without login and security features
- `docker-compose-latest-security.yml`: Latest version with login and security features enabled
- `docker-compose-latest-fat-security.yml`: Fat version with login and security features enabled
These files provide pre-configured setups for different scenarios. For example, here's a snippet from `docker-compose-latest-security.yml`:
```yaml
services:
stirling-pdf:
container_name: Stirling-PDF-Security
image: docker.stirlingpdf.com/stirlingtools/stirling-pdf:latest
deploy:
resources:
limits:
memory: 4G
healthcheck:
test: ["CMD-SHELL", "curl -f http://localhost:8080/api/v1/info/status | grep -q 'UP' && curl -fL http://localhost:8080/ | grep -q 'Please sign in'"]
interval: 5s
timeout: 10s
retries: 16
ports:
- "8080:8080"
volumes:
- ./stirling/latest/data:/usr/share/tessdata:rw
- ./stirling/latest/config:/configs:rw
- ./stirling/latest/logs:/logs:rw
environment:
DISABLE_ADDITIONAL_FEATURES: "false"
SECURITY_ENABLELOGIN: "true"
PUID: 1002
PGID: 1002
UMASK: "022"
SYSTEM_DEFAULTLOCALE: en-US
UI_APPNAME: Stirling-PDF
UI_HOMEDESCRIPTION: Demo site for Stirling-PDF Latest with Security
UI_APPNAMENAVBAR: Stirling-PDF Latest
SYSTEM_MAXFILESIZE: "100"
METRICS_ENABLED: "true"
SYSTEM_GOOGLEVISIBILITY: "true"
SHOW_SURVEY: "true"
restart: on-failure:5
```
To use these example files, copy the desired file to your project root and rename it to `docker-compose.yml`, or specify the file explicitly when running Docker Compose:
```bash
docker-compose -f exampleYmlFiles/docker-compose-latest-security.yml up
```
### Building Docker Images
Stirling-PDF uses different Docker images for various configurations. The build process is controlled by environment variables and uses specific Dockerfile variants. Here's how to build the Docker images:
1. Set the security environment variable:
```bash
export DISABLE_ADDITIONAL_FEATURES=true # or false for to enable login and security features for builds
```
2. Build the project with Gradle:
```bash
./gradlew clean build
```
3. Build the Docker images:
For the latest version:
```bash
docker build --no-cache --pull --build-arg VERSION_TAG=alpha -t stirlingtools/stirling-pdf:latest -f ./Dockerfile .
```
For the ultra-lite version:
```bash
docker build --no-cache --pull --build-arg VERSION_TAG=alpha -t stirlingtools/stirling-pdf:latest-ultra-lite -f ./Dockerfile.ultra-lite .
```
For the fat version (with login and security features enabled):
```bash
export DISABLE_ADDITIONAL_FEATURES=false
docker build --no-cache --pull --build-arg VERSION_TAG=alpha -t stirlingtools/stirling-pdf:latest-fat -f ./Dockerfile.fat .
```
Note: The `--no-cache` and `--pull` flags ensure that the build process uses the latest base images and doesn't use cached layers, which is useful for testing and ensuring reproducible builds. however to improve build times these can often be removed depending on your usecase
## 7. Testing
### Comprehensive Testing Script
Stirling-PDF provides a `test.sh` script in the root directory. This script builds all versions of Stirling-PDF, checks that each version works, and runs Cucumber tests. It's recommended to run this script before submitting a final pull request.
To run the test script:
```bash
./test.sh
```
This script performs the following actions:
1. Builds all Docker images (full, ultra-lite, fat).
2. Runs each version to ensure it starts correctly.
3. Executes Cucumber tests against the main version and ensures feature compatibility. In the event these tests fail, your PR will not be merged.
Note: The `test.sh` script will run automatically when you raise a PR. However, it's recommended to run it locally first to save resources and catch any issues early.
### Full Testing with Docker
1. Build and run the Docker container per the above instructions:
2. Access the application at `http://localhost:8080` and manually test all features developed.
### Frontend Development Testing (Stirling 2.0)
For React frontend development:
1. Start the backend: Run the Spring Boot application to serve API endpoints on localhost:8080
2. Start the frontend dev server: Navigate to the frontend directory and run the development server on localhost:5173
3. The Vite dev server automatically proxies API calls to the backend
4. Test React components, UI interactions, and IndexedDB file operations using browser developer tools
### Local Testing (Java and UI Components)
For quick iterations and development of Java backend, JavaScript, and UI components, you can run and test Stirling-PDF locally without Docker. This approach allows you to work on and verify changes to:
- Java backend logic
- RESTful API endpoints
- JavaScript functionality
- User interface components and styling
- Thymeleaf templates
To run Stirling-PDF locally:
1. Compile and run the project using built-in IDE methods or by running:
```bash
./gradlew bootRun
```
2. Access the application at `http://localhost:8080` in your web browser.
3. Manually test the features you're working on through the UI.
4. For API changes, use tools like Postman or curl to test endpoints directly.
Important notes:
- Local testing doesn't include features that depend on external tools like qpdf, LibreOffice, or Python scripts.
- There are currently no automated unit tests. All testing is done manually through the UI or API calls. (You are welcome to add JUnits!)
- Always verify your changes in the full Docker environment before submitting pull requests, as some integrations and features will only work in the complete setup.
## 8. Contributing
1. Fork the repository on GitHub.
2. Create a new branch for your feature or bug fix.
3. Make your changes and commit them with clear, descriptive messages and ensure any documentation is updated related to your changes.
4. Test your changes thoroughly in the Docker environment.
5. Run the `test.sh` script to ensure all versions build correctly and pass the Cucumber tests:
```bash
./test.sh
```
6. Push your changes to your fork.
7. Submit a pull request to the main repository.
8. See additional [contributing guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md).
When you raise a PR:
- The `test.sh` script will run automatically against your PR.
- The PR checks will verify versioning and dependency updates.
- Documentation will be automatically updated for dependency changes.
- Security issues will be checked using Snyk and PixeeBot.
Address any issues that arise from these checks before finalizing your pull request.
## 9. API Documentation
API documentation is available at `/swagger-ui/index.html` when running the application. You can also view the latest API documentation [here](https://app.swaggerhub.com/apis-docs/Stirling-Tools/Stirling-PDF/).
## 10. Customization
Stirling-PDF can be customized through environment variables or a `settings.yml` file. Key customization options include:
- Application name and branding
- Security settings
- UI customization
- Endpoint management
When using Docker, pass environment variables using the `-e` flag or in your `docker-compose.yml` file.
Example:
```bash
docker run -p 8080:8080 -e APP_NAME="My PDF Tool" stirling-pdf:full
```
Refer to the main README for a full list of customization options.
## 11. Language Translations
For managing language translations that affect multiple files, Stirling-PDF provides a helper script:
```bash
/scripts/replace_translation_line.sh
```
This script helps you make consistent replacements across language files.
When contributing translations:
1. Use the helper script for multi-file changes.
2. Ensure all language files are updated consistently.
3. The PR checks will verify consistency in language file updates.
Remember to test your changes thoroughly to ensure they don't break any existing functionality.
## Code examples
### React Component Development (Stirling 2.0)
For Stirling 2.0, new features are built as React components instead of Thymeleaf templates:
#### Creating a New Tool Component
1. **Create the React Component:**
```typescript
// frontend/src/tools/NewTool.tsx
import { useState } from 'react';
import { Button, FileInput, Container } from '@mantine/core';
interface NewToolProps {
params: Record<string, any>;
updateParams: (updates: Record<string, any>) => void;
}
export default function NewTool({ params, updateParams }: NewToolProps) {
const [files, setFiles] = useState<File[]>([]);
const handleProcess = async () => {
// Process files using API or client-side logic
};
return (
<Container>
<FileInput
multiple
accept="application/pdf"
onChange={setFiles}
/>
<Button onClick={handleProcess}>Process</Button>
</Container>
);
}
```
2. **Add API Integration:**
```typescript
// Use existing API endpoints or create new ones
const response = await fetch('/api/v1/new-tool', {
method: 'POST',
body: formData
});
```
3. **Register in Tool Picker:**
Update the tool picker component to include the new tool with proper routing and URL parameter support.
### Legacy Reference: Overview of Thymeleaf
Thymeleaf is a server-side Java HTML template engine. It is used in Stirling-PDF to render dynamic web pages. Thymeleaf integrates heavily with Spring Boot.
### Thymeleaf overview
In Stirling-PDF, Thymeleaf is used to create HTML templates that are rendered on the server side. These templates are located in the `stirling-pdf/src/main/resources/templates` directory. Thymeleaf templates use a combination of HTML and special Thymeleaf attributes to dynamically generate content.
Some examples of this are:
```html
<th:block th:insert="~{fragments/navbar.html :: navbar}"></th:block>
```
or
```html
<th:block th:insert="~{fragments/footer.html :: footer}"></th:block>
```
Where it uses the `th:block`, `th:` indicating it's a special Thymeleaf element to be used server-side in generating the HTML, and block being the actual element type.
In this case, we are inserting the `navbar` entry within the `fragments/navbar.html` fragment into the `th:block` element.
They can be more complex, such as:
```html
<th:block th:insert="~{fragments/common :: head(title=#{pageExtracter.title}, header=#{pageExtracter.header})}"></th:block>
```
Which is the same as above but passes the parameters title and header into the fragment `common.html` to be used in its HTML generation.
Thymeleaf can also be used to loop through objects or pass things from the Java side into the HTML side.
```java
@GetMapping
public String newFeaturePage(Model model) {
model.addAttribute("exampleData", exampleData);
return "new-feature";
}
```
In the above example, if exampleData is a list of plain java objects of class Person and within it, you had id, name, age, etc. You can reference it like so
```html
<tbody>
<!-- Use th:each to iterate over the list -->
<tr th:each="person : ${exampleData}">
<td th:text="${person.id}"></td>
<td th:text="${person.name}"></td>
<td th:text="${person.age}"></td>
<td th:text="${person.email}"></td>
</tr>
</tbody>
```
This would generate n entries of tr for each person in exampleData
### Adding a New Feature to the Backend (API)
1. **Create a New Controller:**
- Create a new Java class in the `stirling-pdf/src/main/java/stirling/software/SPDF/controller/api` directory.
- Annotate the class with `@RestController` and `@RequestMapping` to define the API endpoint.
- Ensure to add API documentation annotations like `@Tag(name = "General", description = "General APIs")` and `@Operation(summary = "Crops a PDF document", description = "This operation takes an input PDF file and crops it according to the given coordinates. Input:PDF Output:PDF Type:SISO")`.
```java
package stirling.software.SPDF.controller.api;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.tags.Tag;
@RestController
@RequestMapping("/api/v1/new-feature")
@Tag(name = "General", description = "General APIs")
public class NewFeatureController {
@GetMapping
@Operation(summary = "New Feature", description = "This is a new feature endpoint.")
public String newFeature() {
return "NewFeatureResponse"; // This refers to the NewFeatureResponse.html template presenting the user with the generated html from that file when they navigate to /api/v1/new-feature
}
}
```
2. **Define the Service Layer:** (Not required but often useful)
- Create a new service class in the `stirling-pdf/src/main/java/stirling/software/SPDF/service` directory.
- Implement the business logic for the new feature.
```java
package stirling.software.SPDF.service;
import org.springframework.stereotype.Service;
@Service
public class NewFeatureService {
public String getNewFeatureData() {
// Implement business logic here
return "New Feature Data";
}
}
```
2b. **Integrate the Service with the Controller:**
- Autowire the service class in the controller and use it to handle the API request.
```java
package stirling.software.SPDF.controller.api;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import stirling.software.SPDF.service.NewFeatureService;
import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.tags.Tag;
@RestController
@RequestMapping("/api/v1/new-feature")
@Tag(name = "General", description = "General APIs")
public class NewFeatureController {
@Autowired
private NewFeatureService newFeatureService;
@GetMapping
@Operation(summary = "New Feature", description = "This is a new feature endpoint.")
public String newFeature() {
return newFeatureService.getNewFeatureData();
}
}
```
### Adding a New Feature to the Frontend (UI)
1. **Create a New Thymeleaf Template:**
- Create a new HTML file in the `stirling-pdf/src/main/resources/templates` directory.
- Use Thymeleaf attributes to dynamically generate content.
- Use `extract-page.html` as a base example for the HTML template, which is useful to ensure importing of the general layout, navbar, and footer.
```html
<!DOCTYPE html>
<html th:lang="${#locale.language}" th:dir="#{language.direction}" th:data-language="${#locale.toString()}" xmlns:th="https://www.thymeleaf.org">
<head>
<th:block th:insert="~{fragments/common :: head(title=#{newFeature.title}, header=#{newFeature.header})}"></th:block>
</head>
<body>
<div id="page-container">
<div id="content-wrap">
<th:block th:insert="~{fragments/navbar.html :: navbar}"></th:block>
<br><br>
<div class="container">
<div class="row justify-content-center">
<div class="col-md-6 bg-card">
<div class="tool-header">
<span class="material-symbols-rounded tool-header-icon organize">upload</span>
<span class="tool-header-text" th:text="#{newFeature.header}"></span>
</div>
<form th:action="@{'/api/v1/new-feature'}" method="post" enctype="multipart/form-data">
<div th:replace="~{fragments/common :: fileSelector(name='fileInput', multipleInputsForSingleRequest=false, accept='application/pdf')}"></div>
<input type="hidden" id="customMode" name="customMode" value="">
<div class="mb-3">
<label for="featureInput" th:text="#{newFeature.prompt}"></label>
<input type="text" class="form-control" id="featureInput" name="featureInput" th:placeholder="#{newFeature.placeholder}" required>
</div>
<button type="submit" id="submitBtn" class="btn btn-primary" th:text="#{newFeature.submit}"></button>
</form>
</div>
</div>
</div>
</div>
<th:block th:insert="~{fragments/footer.html :: footer}"></th:block>
</div>
</body>
</html>
```
2. **Create a New Controller for the UI:**
- Create a new Java class in the `stirling-pdf/src/main/java/stirling/software/SPDF/controller/ui` directory.
- Annotate the class with `@Controller` and `@RequestMapping` to define the UI endpoint.
```java
package stirling.software.SPDF.controller.ui;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Controller;
import org.springframework.ui.Model;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import stirling.software.SPDF.service.NewFeatureService;
@Controller
@RequestMapping("/new-feature")
public class NewFeatureUIController {
@Autowired
private NewFeatureService newFeatureService;
@GetMapping
public String newFeaturePage(Model model) {
model.addAttribute("newFeatureData", newFeatureService.getNewFeatureData());
return "new-feature";
}
}
```
3. **Update the Navigation Bar:**
- Add a link to the new feature page in the navigation bar.
- Update the `stirling-pdf/src/main/resources/templates/fragments/navbar.html` file.
```html
<li class="nav-item">
<a class="nav-link" th:href="@{'/new-feature'}">New Feature</a>
</li>
```
## Adding New Translations to Existing Language Files in Stirling-PDF
When adding a new feature or modifying existing ones in Stirling-PDF, you'll need to add new translation entries to the existing language files. Here's a step-by-step guide:
### 1. Locate Existing Language Files
Find the existing `messages.properties` files in the `stirling-pdf/src/main/resources` directory. You'll see files like:
- `messages.properties` (default, usually English)
- `messages_en_GB.properties`
- `messages_fr_FR.properties`
- `messages_de_DE.properties`
- etc.
### 2. Add New Translation Entries
Open each of these files and add your new translation entries. For example, if you're adding a new feature called "PDF Splitter",
Use descriptive, hierarchical keys (e.g., `feature.element.description`)
you might add:
```properties
pdfSplitter.title=PDF Splitter
pdfSplitter.description=Split your PDF into multiple documents
pdfSplitter.button.split=Split PDF
pdfSplitter.input.pages=Enter page numbers to split
```
Add these entries to the default GB language file and any others you wish, translating the values as appropriate for each language.
### 3. Use Translations in Thymeleaf Templates
In your Thymeleaf templates, use the `#{key}` syntax to reference the new translations:
```html
<h1 th:text="#{pdfSplitter.title}">PDF Splitter</h1>
<p th:text="#{pdfSplitter.description}">Split your PDF into multiple documents</p>
<input type="text" th:placeholder="#{pdfSplitter.input.pages}">
<button th:text="#{pdfSplitter.button.split}">Split PDF</button>
```
Remember, never hard-code text in your templates or Java code. Always use translation keys to ensure proper localization.
-60
View File
@@ -1,60 +0,0 @@
# dockerfile.dev
# Basisimage: Gradle mit JDK 17 (Debian-basiert)
FROM gradle:8.14-jdk17
# Als Root-Benutzer arbeiten, um benötigte Pakete zu installieren
USER root
# Set GRADLE_HOME und füge Gradle zum PATH hinzu
ENV GRADLE_HOME=/opt/gradle
ENV PATH="$GRADLE_HOME/bin:$PATH"
# Update und Installation zusätzlicher Pakete (Debian/Ubuntu-basiert)
RUN apt-get update && apt-get install -y \
sudo \
libreoffice \
poppler-utils \
qpdf \
# settings.yml | tessdataDir: /usr/share/tesseract-ocr/5/tessdata
tesseract-ocr \
tesseract-ocr-eng \
fonts-terminus fonts-dejavu fonts-font-awesome fonts-noto fonts-noto-core fonts-noto-cjk fonts-noto-extra fonts-liberation fonts-linuxlibertine fonts-urw-base35 \
python3-uno \
python3-venv \
# ss -tln
iproute2 \
&& apt-get clean && rm -rf /var/lib/apt/lists/*
# Setze die Environment Variable für setuptools
ENV SETUPTOOLS_USE_DISTUTILS=local \
STIRLING_TEMPFILES_DIRECTORY=/tmp/stirling-pdf \
TMPDIR=/tmp/stirling-pdf \
TEMP=/tmp/stirling-pdf \
TMP=/tmp/stirling-pdf
# Installation der benötigten Python-Pakete
RUN python3 -m venv --system-site-packages /opt/venv \
&& . /opt/venv/bin/activate \
&& pip install --no-cache-dir --upgrade pip setuptools \
&& pip install --no-cache-dir WeasyPrint pdf2image pillow unoserver opencv-python-headless pre-commit
# Füge den venv-Pfad zur globalen PATH-Variable hinzu, damit die Tools verfügbar sind
ENV PATH="/opt/venv/bin:$PATH"
COPY . /workspace
RUN mkdir -p /tmp/stirling-pdf \
&& fc-cache -f -v \
&& adduser --disabled-password --gecos '' devuser \
&& chown -R devuser:devuser /home/devuser /workspace /tmp/stirling-pdf
RUN echo "devuser ALL=(ALL) NOPASSWD:ALL" > /etc/sudoers.d/devuser \
&& chmod 0440 /etc/sudoers.d/devuser
# Setze das Arbeitsverzeichnis (wird später per Bind-Mount überschrieben)
WORKDIR /workspace
RUN chmod +x /workspace/.devcontainer/git-init.sh /workspace/.devcontainer/init-setup.sh
# Wechsel zum Nicht‑Root Benutzer
USER devuser
+2 -2
View File
@@ -128,7 +128,7 @@ Stirling-PDF currently supports 40 languages!
| English (English) (en_GB) | ![100%](https://geps.dev/progress/100) |
| English (US) (en_US) | ![100%](https://geps.dev/progress/100) |
| French (Français) (fr_FR) | ![91%](https://geps.dev/progress/91) |
| German (Deutsch) (de_DE) | ![99%](https://geps.dev/progress/99) |
| German (Deutsch) (de_DE) | ![100%](https://geps.dev/progress/100) |
| Greek (Ελληνικά) (el_GR) | ![69%](https://geps.dev/progress/69) |
| Hindi (हिंदी) (hi_IN) | ![68%](https://geps.dev/progress/68) |
| Hungarian (Magyar) (hu_HU) | ![99%](https://geps.dev/progress/99) |
@@ -152,7 +152,7 @@ Stirling-PDF currently supports 40 languages!
| Swedish (Svenska) (sv_SE) | ![67%](https://geps.dev/progress/67) |
| Thai (ไทย) (th_TH) | ![60%](https://geps.dev/progress/60) |
| Tibetan (བོད་ཡིག་) (bo_CN) | ![66%](https://geps.dev/progress/66) |
| Traditional Chinese (繁體中文) (zh_TW) | ![99%](https://geps.dev/progress/99) |
| Traditional Chinese (繁體中文) (zh_TW) | ![77%](https://geps.dev/progress/77) |
| Turkish (Türkçe) (tr_TR) | ![82%](https://geps.dev/progress/82) |
| Ukrainian (Українська) (uk_UA) | ![72%](https://geps.dev/progress/72) |
| Vietnamese (Tiếng Việt) (vi_VN) | ![58%](https://geps.dev/progress/58) |
+2 -14
View File
@@ -4,7 +4,7 @@ bootRun {
}
spotless {
java {
target 'src/**/java/**/*.java'
target sourceSets.main.allJava
googleJavaFormat(googleJavaFormatVersion).aosp().reorderImports(false)
importOrder("java", "javax", "org", "com", "net", "io", "jakarta", "lombok", "me", "stirling")
@@ -13,18 +13,6 @@ spotless {
leadingTabsToSpaces()
endWithNewline()
}
yaml {
target '**/*.yml', '**/*.yaml'
trimTrailingWhitespace()
leadingTabsToSpaces()
endWithNewline()
}
format 'gradle', {
target '**/gradle/*.gradle', '**/*.gradle'
trimTrailingWhitespace()
leadingTabsToSpaces()
endWithNewline()
}
}
dependencies {
api 'org.springframework.boot:spring-boot-starter-web'
@@ -41,5 +29,5 @@ dependencies {
api 'org.snakeyaml:snakeyaml-engine:2.10'
api "org.springdoc:springdoc-openapi-starter-webmvc-ui:2.8.9"
api 'jakarta.mail:jakarta.mail-api:2.1.3'
runtimeOnly 'org.eclipse.angus:angus-mail:2.0.4'
runtimeOnly 'org.eclipse.angus:angus-mail:2.0.3'
}
@@ -15,7 +15,6 @@ public class InstallationPathConfig {
private static final String CUSTOM_FILES_PATH;
private static final String CLIENT_WEBUI_PATH;
private static final String SCRIPTS_PATH;
private static final String PIPELINE_PATH;
// Config paths
private static final String SETTINGS_PATH;
@@ -34,7 +33,6 @@ public class InstallationPathConfig {
CONFIG_PATH = BASE_PATH + "configs" + File.separator;
CUSTOM_FILES_PATH = BASE_PATH + "customFiles" + File.separator;
CLIENT_WEBUI_PATH = BASE_PATH + "clientWebUI" + File.separator;
PIPELINE_PATH = BASE_PATH + "pipeline" + File.separator;
// Initialize config paths
SETTINGS_PATH = CONFIG_PATH + "settings.yml";
@@ -97,10 +95,6 @@ public class InstallationPathConfig {
return SCRIPTS_PATH;
}
public static String getPipelinePath() {
return PIPELINE_PATH;
}
public static String getSettingsPath() {
return SETTINGS_PATH;
}
@@ -25,9 +25,6 @@ import org.springframework.core.io.Resource;
import org.springframework.core.io.support.EncodedResource;
import org.springframework.stereotype.Component;
import com.fasterxml.jackson.annotation.JsonIgnore;
import com.fasterxml.jackson.annotation.JsonProperty;
import lombok.Data;
import lombok.Getter;
import lombok.Setter;
@@ -61,10 +58,7 @@ public class ApplicationProperties {
private Mail mail = new Mail();
private Premium premium = new Premium();
@JsonIgnore // Deprecated - completely hidden from JSON serialization
private EnterpriseEdition enterpriseEdition = new EnterpriseEdition();
private AutoPipeline autoPipeline = new AutoPipeline();
private ProcessExecutor processExecutor = new ProcessExecutor();
@@ -174,27 +168,14 @@ public class ApplicationProperties {
private Boolean autoCreateUser = false;
private Boolean blockRegistration = false;
private String registrationId = "stirling";
@ToString.Exclude
@JsonProperty("idpMetadataUri")
private String idpMetadataUri;
@ToString.Exclude private String idpMetadataUri;
private String idpSingleLogoutUrl;
private String idpSingleLoginUrl;
private String idpIssuer;
@JsonProperty("idpCert")
private String idpCert;
@ToString.Exclude private String privateKey;
@ToString.Exclude private String spCert;
@ToString.Exclude
@JsonProperty("privateKey")
private String privateKey;
@ToString.Exclude
@JsonProperty("spCert")
private String spCert;
@JsonIgnore
public InputStream getIdpMetadataUri() throws IOException {
if (idpMetadataUri.startsWith("classpath:")) {
return new ClassPathResource(idpMetadataUri.substring("classpath".length()))
@@ -211,7 +192,6 @@ public class ApplicationProperties {
}
}
@JsonIgnore
public Resource getSpCert() {
if (spCert == null) return null;
if (spCert.startsWith("classpath:")) {
@@ -221,7 +201,6 @@ public class ApplicationProperties {
}
}
@JsonIgnore
public Resource getIdpCert() {
if (idpCert == null) return null;
if (idpCert.startsWith("classpath:")) {
@@ -231,7 +210,6 @@ public class ApplicationProperties {
}
}
@JsonIgnore
public Resource getPrivateKey() {
if (privateKey.startsWith("classpath:")) {
return new ClassPathResource(privateKey.substring("classpath:".length()));
@@ -311,9 +289,7 @@ public class ApplicationProperties {
private Boolean enableAnalytics;
private Datasource datasource;
private Boolean disableSanitize;
private int maxDPI;
private Boolean enableUrlToPDF;
private Html html = new Html();
private CustomPaths customPaths = new CustomPaths();
private String fileUploadLimit;
private TempFileManagement tempFileManagement = new TempFileManagement();
@@ -344,12 +320,8 @@ public class ApplicationProperties {
@Data
public static class TempFileManagement {
@JsonProperty("baseTmpDir")
private String baseTmpDir = "";
@JsonProperty("libreofficeDir")
private String libreofficeDir = "";
private String systemTempDir = "";
private String prefix = "stirling-pdf-";
private long maxAgeHours = 24;
@@ -357,14 +329,12 @@ public class ApplicationProperties {
private boolean startupCleanup = true;
private boolean cleanupSystemTemp = false;
@JsonIgnore
public String getBaseTmpDir() {
return baseTmpDir != null && !baseTmpDir.isEmpty()
? baseTmpDir
: java.lang.System.getProperty("java.io.tmpdir") + "/stirling-pdf";
}
@JsonIgnore
public String getLibreofficeDir() {
return libreofficeDir != null && !libreofficeDir.isEmpty()
? libreofficeDir
@@ -372,25 +342,6 @@ public class ApplicationProperties {
}
}
@Data
public static class Html {
private UrlSecurity urlSecurity = new UrlSecurity();
@Data
public static class UrlSecurity {
private boolean enabled = true;
private String level = "MEDIUM"; // MAX, MEDIUM, OFF
private List<String> allowedDomains = new ArrayList<>();
private List<String> blockedDomains = new ArrayList<>();
private List<String> internalTlds =
Arrays.asList(".local", ".internal", ".corp", ".home");
private boolean blockPrivateNetworks = true;
private boolean blockLocalhost = true;
private boolean blockLinkLocal = true;
private boolean blockCloudMetadata = true;
}
}
@Data
public static class Datasource {
private boolean enableCustomDatabase;
@@ -640,24 +591,12 @@ public class ApplicationProperties {
@Data
public static class TimeoutMinutes {
@JsonProperty("libreOfficetimeoutMinutes")
private long libreOfficeTimeoutMinutes;
@JsonProperty("pdfToHtmltimeoutMinutes")
private long pdfToHtmlTimeoutMinutes;
@JsonProperty("pythonOpenCvtimeoutMinutes")
private long pythonOpenCvTimeoutMinutes;
@JsonProperty("weasyPrinttimeoutMinutes")
private long weasyPrintTimeoutMinutes;
@JsonProperty("installApptimeoutMinutes")
private long installAppTimeoutMinutes;
@JsonProperty("calibretimeoutMinutes")
private long calibreTimeoutMinutes;
private long tesseractTimeoutMinutes;
private long qpdfTimeoutMinutes;
private long ghostscriptTimeoutMinutes;
@@ -1,208 +0,0 @@
package stirling.software.common.service;
import java.net.InetAddress;
import java.net.URI;
import java.net.UnknownHostException;
import java.util.regex.Pattern;
import org.springframework.stereotype.Service;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import stirling.software.common.model.ApplicationProperties;
@Service
@RequiredArgsConstructor
@Slf4j
public class SsrfProtectionService {
private final ApplicationProperties applicationProperties;
private static final Pattern DATA_URL_PATTERN =
Pattern.compile("^data:.*", Pattern.CASE_INSENSITIVE);
private static final Pattern FRAGMENT_PATTERN = Pattern.compile("^#.*");
public enum SsrfProtectionLevel {
OFF, // No SSRF protection - allows all URLs
MEDIUM, // Block internal networks but allow external URLs
MAX // Block all external URLs - only data: and fragments
}
public boolean isUrlAllowed(String url) {
ApplicationProperties.Html.UrlSecurity config =
applicationProperties.getSystem().getHtml().getUrlSecurity();
if (!config.isEnabled()) {
return true;
}
if (url == null || url.trim().isEmpty()) {
return false;
}
String trimmedUrl = url.trim();
// Always allow data URLs and fragments
if (DATA_URL_PATTERN.matcher(trimmedUrl).matches()
|| FRAGMENT_PATTERN.matcher(trimmedUrl).matches()) {
return true;
}
SsrfProtectionLevel level = parseProtectionLevel(config.getLevel());
switch (level) {
case OFF:
return true;
case MAX:
return isMaxSecurityAllowed(trimmedUrl, config);
case MEDIUM:
return isMediumSecurityAllowed(trimmedUrl, config);
default:
return false;
}
}
private SsrfProtectionLevel parseProtectionLevel(String level) {
try {
return SsrfProtectionLevel.valueOf(level.toUpperCase());
} catch (IllegalArgumentException e) {
log.warn("Invalid SSRF protection level '{}', defaulting to MEDIUM", level);
return SsrfProtectionLevel.MEDIUM;
}
}
private boolean isMaxSecurityAllowed(
String url, ApplicationProperties.Html.UrlSecurity config) {
// MAX security: only allow explicitly whitelisted domains
try {
URI uri = new URI(url);
String host = uri.getHost();
if (host == null) {
return false;
}
return config.getAllowedDomains().contains(host.toLowerCase());
} catch (Exception e) {
log.debug("Failed to parse URL for MAX security check: {}", url, e);
return false;
}
}
private boolean isMediumSecurityAllowed(
String url, ApplicationProperties.Html.UrlSecurity config) {
try {
URI uri = new URI(url);
String host = uri.getHost();
if (host == null) {
return false;
}
String hostLower = host.toLowerCase();
// Check explicit blocked domains
if (config.getBlockedDomains().contains(hostLower)) {
log.debug("URL blocked by explicit domain blocklist: {}", url);
return false;
}
// Check internal TLD patterns
for (String tld : config.getInternalTlds()) {
if (hostLower.endsWith(tld.toLowerCase())) {
log.debug("URL blocked by internal TLD pattern '{}': {}", tld, url);
return false;
}
}
// If allowedDomains is specified, only allow those
if (!config.getAllowedDomains().isEmpty()) {
boolean isAllowed =
config.getAllowedDomains().stream()
.anyMatch(
domain ->
hostLower.equals(domain.toLowerCase())
|| hostLower.endsWith(
"." + domain.toLowerCase()));
if (!isAllowed) {
log.debug("URL not in allowed domains list: {}", url);
return false;
}
}
// Resolve hostname to IP address for network-based checks
try {
InetAddress address = InetAddress.getByName(host);
if (config.isBlockPrivateNetworks() && isPrivateAddress(address)) {
log.debug("URL blocked - private network address: {}", url);
return false;
}
if (config.isBlockLocalhost() && address.isLoopbackAddress()) {
log.debug("URL blocked - localhost address: {}", url);
return false;
}
if (config.isBlockLinkLocal() && address.isLinkLocalAddress()) {
log.debug("URL blocked - link-local address: {}", url);
return false;
}
if (config.isBlockCloudMetadata()
&& isCloudMetadataAddress(address.getHostAddress())) {
log.debug("URL blocked - cloud metadata endpoint: {}", url);
return false;
}
} catch (UnknownHostException e) {
log.debug("Failed to resolve hostname for SSRF check: {}", host, e);
return false;
}
return true;
} catch (Exception e) {
log.debug("Failed to parse URL for MEDIUM security check: {}", url, e);
return false;
}
}
private boolean isPrivateAddress(InetAddress address) {
return address.isSiteLocalAddress()
|| address.isAnyLocalAddress()
|| isPrivateIPv4Range(address.getHostAddress());
}
private boolean isPrivateIPv4Range(String ip) {
return ip.startsWith("10.")
|| ip.startsWith("192.168.")
|| (ip.startsWith("172.") && isInRange172(ip))
|| ip.startsWith("127.")
|| "0.0.0.0".equals(ip);
}
private boolean isInRange172(String ip) {
String[] parts = ip.split("\\.");
if (parts.length >= 2) {
try {
int secondOctet = Integer.parseInt(parts[1]);
return secondOctet >= 16 && secondOctet <= 31;
} catch (NumberFormatException e) {
return false;
}
}
return false;
}
private boolean isCloudMetadataAddress(String ip) {
// Cloud metadata endpoints for AWS, GCP, Azure, Oracle Cloud, and IBM Cloud
return ip.startsWith("169.254.169.254") // AWS/GCP/Azure
|| ip.startsWith("fd00:ec2::254") // AWS IPv6
|| ip.startsWith("169.254.169.253") // Oracle Cloud
|| ip.startsWith("169.254.169.250"); // IBM Cloud
}
}
@@ -1,71 +1,21 @@
package stirling.software.common.util;
import org.owasp.html.AttributePolicy;
import org.owasp.html.HtmlPolicyBuilder;
import org.owasp.html.PolicyFactory;
import org.owasp.html.Sanitizers;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;
import stirling.software.common.model.ApplicationProperties;
import stirling.software.common.service.SsrfProtectionService;
@Component
public class CustomHtmlSanitizer {
private final SsrfProtectionService ssrfProtectionService;
private final ApplicationProperties applicationProperties;
@Autowired
public CustomHtmlSanitizer(
SsrfProtectionService ssrfProtectionService,
ApplicationProperties applicationProperties) {
this.ssrfProtectionService = ssrfProtectionService;
this.applicationProperties = applicationProperties;
}
private final AttributePolicy SSRF_SAFE_URL_POLICY =
new AttributePolicy() {
@Override
public String apply(String elementName, String attributeName, String value) {
if (value == null || value.trim().isEmpty()) {
return null;
}
String trimmedValue = value.trim();
// Use the SSRF protection service to validate the URL
if (ssrfProtectionService != null
&& !ssrfProtectionService.isUrlAllowed(trimmedValue)) {
return null;
}
return trimmedValue;
}
};
private final PolicyFactory SSRF_SAFE_IMAGES_POLICY =
new HtmlPolicyBuilder()
.allowElements("img")
.allowAttributes("alt", "width", "height", "title")
.onElements("img")
.allowAttributes("src")
.matching(SSRF_SAFE_URL_POLICY)
.onElements("img")
.toFactory();
private final PolicyFactory POLICY =
private static final PolicyFactory POLICY =
Sanitizers.FORMATTING
.and(Sanitizers.BLOCKS)
.and(Sanitizers.STYLES)
.and(Sanitizers.LINKS)
.and(Sanitizers.TABLES)
.and(SSRF_SAFE_IMAGES_POLICY)
.and(Sanitizers.IMAGES)
.and(new HtmlPolicyBuilder().disallowElements("noscript").toFactory());
public String sanitize(String html) {
boolean disableSanitize =
Boolean.TRUE.equals(applicationProperties.getSystem().getDisableSanitize());
return disableSanitize ? html : POLICY.sanitize(html);
public static String sanitize(String html) {
String htmlAfter = POLICY.sanitize(html);
return htmlAfter;
}
}
@@ -1,652 +0,0 @@
package stirling.software.common.util;
import java.io.ByteArrayInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.lang.reflect.Constructor;
import java.lang.reflect.Method;
import java.nio.charset.StandardCharsets;
import java.util.ArrayList;
import java.util.Date;
import java.util.List;
import java.util.Properties;
import java.util.regex.Pattern;
import lombok.Data;
import lombok.experimental.UtilityClass;
import stirling.software.common.model.api.converters.EmlToPdfRequest;
@UtilityClass
public class EmlParser {
private static volatile Boolean jakartaMailAvailable = null;
private static volatile Method mimeUtilityDecodeTextMethod = null;
private static volatile boolean mimeUtilityChecked = false;
private static final Pattern MIME_ENCODED_PATTERN =
Pattern.compile("=\\?([^?]+)\\?([BbQq])\\?([^?]*)\\?=");
private static final String DISPOSITION_ATTACHMENT = "attachment";
private static final String TEXT_PLAIN = "text/plain";
private static final String TEXT_HTML = "text/html";
private static final String MULTIPART_PREFIX = "multipart/";
private static final String HEADER_CONTENT_TYPE = "content-type:";
private static final String HEADER_CONTENT_DISPOSITION = "content-disposition:";
private static final String HEADER_CONTENT_TRANSFER_ENCODING = "content-transfer-encoding:";
private static final String HEADER_CONTENT_ID = "Content-ID";
private static final String HEADER_SUBJECT = "Subject:";
private static final String HEADER_FROM = "From:";
private static final String HEADER_TO = "To:";
private static final String HEADER_CC = "Cc:";
private static final String HEADER_BCC = "Bcc:";
private static final String HEADER_DATE = "Date:";
private static synchronized boolean isJakartaMailAvailable() {
if (jakartaMailAvailable == null) {
try {
Class.forName("jakarta.mail.internet.MimeMessage");
Class.forName("jakarta.mail.Session");
Class.forName("jakarta.mail.internet.MimeUtility");
Class.forName("jakarta.mail.internet.MimePart");
Class.forName("jakarta.mail.internet.MimeMultipart");
Class.forName("jakarta.mail.Multipart");
Class.forName("jakarta.mail.Part");
jakartaMailAvailable = true;
} catch (ClassNotFoundException e) {
jakartaMailAvailable = false;
}
}
return jakartaMailAvailable;
}
public static EmailContent extractEmailContent(
byte[] emlBytes, EmlToPdfRequest request, CustomHtmlSanitizer customHtmlSanitizer)
throws IOException {
EmlProcessingUtils.validateEmlInput(emlBytes);
if (isJakartaMailAvailable()) {
return extractEmailContentAdvanced(emlBytes, request, customHtmlSanitizer);
} else {
return extractEmailContentBasic(emlBytes, request, customHtmlSanitizer);
}
}
private static EmailContent extractEmailContentBasic(
byte[] emlBytes, EmlToPdfRequest request, CustomHtmlSanitizer customHtmlSanitizer) {
String emlContent = new String(emlBytes, StandardCharsets.UTF_8);
EmailContent content = new EmailContent();
content.setSubject(extractBasicHeader(emlContent, HEADER_SUBJECT));
content.setFrom(extractBasicHeader(emlContent, HEADER_FROM));
content.setTo(extractBasicHeader(emlContent, HEADER_TO));
content.setCc(extractBasicHeader(emlContent, HEADER_CC));
content.setBcc(extractBasicHeader(emlContent, HEADER_BCC));
String dateStr = extractBasicHeader(emlContent, HEADER_DATE);
if (!dateStr.isEmpty()) {
content.setDateString(dateStr);
}
String htmlBody = extractHtmlBody(emlContent);
if (htmlBody != null) {
content.setHtmlBody(htmlBody);
} else {
String textBody = extractTextBody(emlContent);
content.setTextBody(textBody != null ? textBody : "Email content could not be parsed");
}
content.getAttachments().addAll(extractAttachmentsBasic(emlContent));
return content;
}
private static EmailContent extractEmailContentAdvanced(
byte[] emlBytes, EmlToPdfRequest request, CustomHtmlSanitizer customHtmlSanitizer) {
try {
Class<?> sessionClass = Class.forName("jakarta.mail.Session");
Class<?> mimeMessageClass = Class.forName("jakarta.mail.internet.MimeMessage");
Method getDefaultInstance =
sessionClass.getMethod("getDefaultInstance", Properties.class);
Object session = getDefaultInstance.invoke(null, new Properties());
Class<?>[] constructorArgs = new Class<?>[] {sessionClass, InputStream.class};
Constructor<?> mimeMessageConstructor =
mimeMessageClass.getConstructor(constructorArgs);
Object message =
mimeMessageConstructor.newInstance(session, new ByteArrayInputStream(emlBytes));
return extractFromMimeMessage(message, request, customHtmlSanitizer);
} catch (ReflectiveOperationException e) {
return extractEmailContentBasic(emlBytes, request, customHtmlSanitizer);
}
}
private static EmailContent extractFromMimeMessage(
Object message, EmlToPdfRequest request, CustomHtmlSanitizer customHtmlSanitizer) {
EmailContent content = new EmailContent();
try {
Class<?> messageClass = message.getClass();
Method getSubject = messageClass.getMethod("getSubject");
String subject = (String) getSubject.invoke(message);
content.setSubject(subject != null ? safeMimeDecode(subject) : "No Subject");
Method getFrom = messageClass.getMethod("getFrom");
Object[] fromAddresses = (Object[]) getFrom.invoke(message);
content.setFrom(buildAddressString(fromAddresses));
extractRecipients(message, messageClass, content);
Method getSentDate = messageClass.getMethod("getSentDate");
content.setDate((Date) getSentDate.invoke(message));
Method getContent = messageClass.getMethod("getContent");
Object messageContent = getContent.invoke(message);
processMessageContent(message, messageContent, content, request, customHtmlSanitizer);
} catch (ReflectiveOperationException | RuntimeException e) {
content.setSubject("Email Conversion");
content.setFrom("Unknown");
content.setTo("Unknown");
content.setCc("");
content.setBcc("");
content.setTextBody("Email content could not be parsed with advanced processing");
}
return content;
}
private static void extractRecipients(
Object message, Class<?> messageClass, EmailContent content) {
try {
Method getRecipients =
messageClass.getMethod(
"getRecipients", Class.forName("jakarta.mail.Message$RecipientType"));
Class<?> recipientTypeClass = Class.forName("jakarta.mail.Message$RecipientType");
Object toType = recipientTypeClass.getField("TO").get(null);
Object[] toRecipients = (Object[]) getRecipients.invoke(message, toType);
content.setTo(buildAddressString(toRecipients));
Object ccType = recipientTypeClass.getField("CC").get(null);
Object[] ccRecipients = (Object[]) getRecipients.invoke(message, ccType);
content.setCc(buildAddressString(ccRecipients));
Object bccType = recipientTypeClass.getField("BCC").get(null);
Object[] bccRecipients = (Object[]) getRecipients.invoke(message, bccType);
content.setBcc(buildAddressString(bccRecipients));
} catch (ReflectiveOperationException e) {
try {
Method getAllRecipients = messageClass.getMethod("getAllRecipients");
Object[] recipients = (Object[]) getAllRecipients.invoke(message);
content.setTo(buildAddressString(recipients));
content.setCc("");
content.setBcc("");
} catch (ReflectiveOperationException ex) {
content.setTo("");
content.setCc("");
content.setBcc("");
}
}
}
private static String buildAddressString(Object[] addresses) {
if (addresses == null || addresses.length == 0) {
return "";
}
StringBuilder builder = new StringBuilder();
for (int i = 0; i < addresses.length; i++) {
if (i > 0) builder.append(", ");
builder.append(safeMimeDecode(addresses[i].toString()));
}
return builder.toString();
}
private static void processMessageContent(
Object message,
Object messageContent,
EmailContent content,
EmlToPdfRequest request,
CustomHtmlSanitizer customHtmlSanitizer) {
try {
if (messageContent instanceof String stringContent) {
Method getContentType = message.getClass().getMethod("getContentType");
String contentType = (String) getContentType.invoke(message);
if (contentType != null && contentType.toLowerCase().contains(TEXT_HTML)) {
content.setHtmlBody(stringContent);
} else {
content.setTextBody(stringContent);
}
} else {
Class<?> multipartClass = Class.forName("jakarta.mail.Multipart");
if (multipartClass.isInstance(messageContent)) {
processMultipart(messageContent, content, request, customHtmlSanitizer, 0);
}
}
} catch (ReflectiveOperationException | ClassCastException e) {
content.setTextBody("Email content could not be parsed with advanced processing");
}
}
private static void processMultipart(
Object multipart,
EmailContent content,
EmlToPdfRequest request,
CustomHtmlSanitizer customHtmlSanitizer,
int depth) {
final int MAX_MULTIPART_DEPTH = 10;
if (depth > MAX_MULTIPART_DEPTH) {
content.setHtmlBody("<div class=\"error\">Maximum multipart depth exceeded</div>");
return;
}
try {
Class<?> multipartClass = multipart.getClass();
Method getCount = multipartClass.getMethod("getCount");
int count = (Integer) getCount.invoke(multipart);
Method getBodyPart = multipartClass.getMethod("getBodyPart", int.class);
for (int i = 0; i < count; i++) {
Object part = getBodyPart.invoke(multipart, i);
processPart(part, content, request, customHtmlSanitizer, depth + 1);
}
} catch (ReflectiveOperationException | ClassCastException e) {
content.setHtmlBody("<div class=\"error\">Error processing multipart content</div>");
}
}
private static void processPart(
Object part,
EmailContent content,
EmlToPdfRequest request,
CustomHtmlSanitizer customHtmlSanitizer,
int depth) {
try {
Class<?> partClass = part.getClass();
Method isMimeType = partClass.getMethod("isMimeType", String.class);
Method getContent = partClass.getMethod("getContent");
Method getDisposition = partClass.getMethod("getDisposition");
Method getFileName = partClass.getMethod("getFileName");
Method getContentType = partClass.getMethod("getContentType");
Method getHeader = partClass.getMethod("getHeader", String.class);
Object disposition = getDisposition.invoke(part);
String filename = (String) getFileName.invoke(part);
String contentType = (String) getContentType.invoke(part);
String normalizedDisposition =
disposition != null ? ((String) disposition).toLowerCase() : null;
if ((Boolean) isMimeType.invoke(part, TEXT_PLAIN) && normalizedDisposition == null) {
Object partContent = getContent.invoke(part);
if (partContent instanceof String stringContent) {
content.setTextBody(stringContent);
}
} else if ((Boolean) isMimeType.invoke(part, TEXT_HTML)
&& normalizedDisposition == null) {
Object partContent = getContent.invoke(part);
if (partContent instanceof String stringContent) {
String htmlBody =
customHtmlSanitizer != null
? customHtmlSanitizer.sanitize(stringContent)
: stringContent;
content.setHtmlBody(htmlBody);
}
} else if ((normalizedDisposition != null
&& normalizedDisposition.contains(DISPOSITION_ATTACHMENT))
|| (filename != null && !filename.trim().isEmpty())) {
processAttachment(
part, content, request, getHeader, getContent, filename, contentType);
} else if ((Boolean) isMimeType.invoke(part, "multipart/*")) {
Object multipartContent = getContent.invoke(part);
if (multipartContent != null) {
Class<?> multipartClass = Class.forName("jakarta.mail.Multipart");
if (multipartClass.isInstance(multipartContent)) {
processMultipart(
multipartContent, content, request, customHtmlSanitizer, depth + 1);
}
}
}
} catch (ReflectiveOperationException | RuntimeException e) {
// Continue processing other parts if one fails
}
}
private static void processAttachment(
Object part,
EmailContent content,
EmlToPdfRequest request,
Method getHeader,
Method getContent,
String filename,
String contentType) {
content.setAttachmentCount(content.getAttachmentCount() + 1);
if (filename != null && !filename.trim().isEmpty()) {
EmailAttachment attachment = new EmailAttachment();
attachment.setFilename(safeMimeDecode(filename));
attachment.setContentType(contentType);
try {
String[] contentIdHeaders = (String[]) getHeader.invoke(part, HEADER_CONTENT_ID);
if (contentIdHeaders != null) {
for (String contentIdHeader : contentIdHeaders) {
if (contentIdHeader != null && !contentIdHeader.trim().isEmpty()) {
attachment.setEmbedded(true);
String contentId = contentIdHeader.trim().replaceAll("[<>]", "");
attachment.setContentId(contentId);
break;
}
}
}
} catch (ReflectiveOperationException e) {
}
if ((request != null && request.isIncludeAttachments()) || attachment.isEmbedded()) {
extractAttachmentData(part, attachment, getContent, request);
}
content.getAttachments().add(attachment);
}
}
private static void extractAttachmentData(
Object part, EmailAttachment attachment, Method getContent, EmlToPdfRequest request) {
try {
Object attachmentContent = getContent.invoke(part);
byte[] attachmentData = null;
if (attachmentContent instanceof InputStream inputStream) {
try (InputStream stream = inputStream) {
attachmentData = stream.readAllBytes();
} catch (IOException e) {
if (attachment.isEmbedded()) {
attachmentData = new byte[0];
} else {
throw new RuntimeException(e);
}
}
} else if (attachmentContent instanceof byte[] byteArray) {
attachmentData = byteArray;
} else if (attachmentContent instanceof String stringContent) {
attachmentData = stringContent.getBytes(StandardCharsets.UTF_8);
}
if (attachmentData != null) {
long maxSizeMB = request != null ? request.getMaxAttachmentSizeMB() : 10L;
long maxSizeBytes = maxSizeMB * 1024 * 1024;
if (attachmentData.length <= maxSizeBytes || attachment.isEmbedded()) {
attachment.setData(attachmentData);
attachment.setSizeBytes(attachmentData.length);
} else {
attachment.setSizeBytes(attachmentData.length);
}
}
} catch (ReflectiveOperationException | RuntimeException e) {
// Continue without attachment data
}
}
private static String extractBasicHeader(String emlContent, String headerName) {
try {
String[] lines = emlContent.split("\r?\n");
for (int i = 0; i < lines.length; i++) {
String line = lines[i];
if (line.toLowerCase().startsWith(headerName.toLowerCase())) {
StringBuilder value =
new StringBuilder(line.substring(headerName.length()).trim());
for (int j = i + 1; j < lines.length; j++) {
if (lines[j].startsWith(" ") || lines[j].startsWith("\t")) {
value.append(" ").append(lines[j].trim());
} else {
break;
}
}
return safeMimeDecode(value.toString());
}
if (line.trim().isEmpty()) break;
}
} catch (RuntimeException e) {
// Ignore errors in header extraction
}
return "";
}
private static String extractHtmlBody(String emlContent) {
try {
String lowerContent = emlContent.toLowerCase();
int htmlStart = lowerContent.indexOf(HEADER_CONTENT_TYPE + " " + TEXT_HTML);
if (htmlStart == -1) return null;
int bodyStart = emlContent.indexOf("\r\n\r\n", htmlStart);
if (bodyStart == -1) bodyStart = emlContent.indexOf("\n\n", htmlStart);
if (bodyStart == -1) return null;
bodyStart += (emlContent.charAt(bodyStart + 1) == '\r') ? 4 : 2;
int bodyEnd = findPartEnd(emlContent, bodyStart);
return emlContent.substring(bodyStart, bodyEnd).trim();
} catch (Exception e) {
return null;
}
}
private static String extractTextBody(String emlContent) {
try {
String lowerContent = emlContent.toLowerCase();
int textStart = lowerContent.indexOf(HEADER_CONTENT_TYPE + " " + TEXT_PLAIN);
if (textStart == -1) {
int bodyStart = emlContent.indexOf("\r\n\r\n");
if (bodyStart == -1) bodyStart = emlContent.indexOf("\n\n");
if (bodyStart != -1) {
bodyStart += (emlContent.charAt(bodyStart + 1) == '\r') ? 4 : 2;
int bodyEnd = findPartEnd(emlContent, bodyStart);
return emlContent.substring(bodyStart, bodyEnd).trim();
}
return null;
}
int bodyStart = emlContent.indexOf("\r\n\r\n", textStart);
if (bodyStart == -1) bodyStart = emlContent.indexOf("\n\n", textStart);
if (bodyStart == -1) return null;
bodyStart += (emlContent.charAt(bodyStart + 1) == '\r') ? 4 : 2;
int bodyEnd = findPartEnd(emlContent, bodyStart);
return emlContent.substring(bodyStart, bodyEnd).trim();
} catch (RuntimeException e) {
return null;
}
}
private static int findPartEnd(String content, int start) {
String[] lines = content.substring(start).split("\r?\n");
StringBuilder result = new StringBuilder();
for (String line : lines) {
if (line.startsWith("--") && line.length() > 10) break;
result.append(line).append("\n");
}
return start + result.length();
}
private static List<EmailAttachment> extractAttachmentsBasic(String emlContent) {
List<EmailAttachment> attachments = new ArrayList<>();
try {
String[] lines = emlContent.split("\r?\n");
boolean inHeaders = true;
String currentContentType = "";
String currentDisposition = "";
String currentFilename = "";
String currentEncoding = "";
for (String line : lines) {
String lowerLine = line.toLowerCase().trim();
if (line.trim().isEmpty()) {
inHeaders = false;
if (isAttachment(currentDisposition, currentFilename, currentContentType)) {
EmailAttachment attachment = new EmailAttachment();
attachment.setFilename(currentFilename);
attachment.setContentType(currentContentType);
attachment.setTransferEncoding(currentEncoding);
attachments.add(attachment);
}
currentContentType = "";
currentDisposition = "";
currentFilename = "";
currentEncoding = "";
inHeaders = true;
continue;
}
if (!inHeaders) continue;
if (lowerLine.startsWith(HEADER_CONTENT_TYPE)) {
currentContentType = line.substring(HEADER_CONTENT_TYPE.length()).trim();
} else if (lowerLine.startsWith(HEADER_CONTENT_DISPOSITION)) {
currentDisposition = line.substring(HEADER_CONTENT_DISPOSITION.length()).trim();
currentFilename = extractFilenameFromDisposition(currentDisposition);
} else if (lowerLine.startsWith(HEADER_CONTENT_TRANSFER_ENCODING)) {
currentEncoding =
line.substring(HEADER_CONTENT_TRANSFER_ENCODING.length()).trim();
}
}
} catch (RuntimeException e) {
// Continue with empty list
}
return attachments;
}
private static boolean isAttachment(String disposition, String filename, String contentType) {
return (disposition.toLowerCase().contains(DISPOSITION_ATTACHMENT) && !filename.isEmpty())
|| (!filename.isEmpty() && !contentType.toLowerCase().startsWith("text/"))
|| (contentType.toLowerCase().contains("application/") && !filename.isEmpty());
}
private static String extractFilenameFromDisposition(String disposition) {
if (disposition == null || !disposition.contains("filename=")) {
return "";
}
// Handle filename*= (RFC 2231 encoded filename)
if (disposition.toLowerCase().contains("filename*=")) {
int filenameStarStart = disposition.toLowerCase().indexOf("filename*=") + 10;
int filenameStarEnd = disposition.indexOf(";", filenameStarStart);
if (filenameStarEnd == -1) filenameStarEnd = disposition.length();
String extendedFilename =
disposition.substring(filenameStarStart, filenameStarEnd).trim();
extendedFilename = extendedFilename.replaceAll("^\"|\"$", "");
if (extendedFilename.contains("'")) {
String[] parts = extendedFilename.split("'", 3);
if (parts.length == 3) {
return EmlProcessingUtils.decodeUrlEncoded(parts[2]);
}
}
}
// Handle regular filename=
int filenameStart = disposition.toLowerCase().indexOf("filename=") + 9;
int filenameEnd = disposition.indexOf(";", filenameStart);
if (filenameEnd == -1) filenameEnd = disposition.length();
String filename = disposition.substring(filenameStart, filenameEnd).trim();
filename = filename.replaceAll("^\"|\"$", "");
return safeMimeDecode(filename);
}
public static String safeMimeDecode(String headerValue) {
if (headerValue == null || headerValue.trim().isEmpty()) {
return "";
}
if (!mimeUtilityChecked) {
synchronized (EmlParser.class) {
if (!mimeUtilityChecked) {
initializeMimeUtilityDecoding();
}
}
}
if (mimeUtilityDecodeTextMethod != null) {
try {
return (String) mimeUtilityDecodeTextMethod.invoke(null, headerValue.trim());
} catch (ReflectiveOperationException | RuntimeException e) {
// Fall through to custom implementation
}
}
return EmlProcessingUtils.decodeMimeHeader(headerValue.trim());
}
private static void initializeMimeUtilityDecoding() {
try {
Class<?> mimeUtilityClass = Class.forName("jakarta.mail.internet.MimeUtility");
mimeUtilityDecodeTextMethod = mimeUtilityClass.getMethod("decodeText", String.class);
} catch (ClassNotFoundException | NoSuchMethodException e) {
mimeUtilityDecodeTextMethod = null;
}
mimeUtilityChecked = true;
}
@Data
public static class EmailContent {
private String subject;
private String from;
private String to;
private String cc;
private String bcc;
private Date date;
private String dateString; // For basic parsing fallback
private String htmlBody;
private String textBody;
private int attachmentCount;
private List<EmailAttachment> attachments = new ArrayList<>();
public void setHtmlBody(String htmlBody) {
this.htmlBody = htmlBody != null ? htmlBody.replaceAll("\r", "") : null;
}
public void setTextBody(String textBody) {
this.textBody = textBody != null ? textBody.replaceAll("\r", "") : null;
}
}
@Data
public static class EmailAttachment {
private String filename;
private String contentType;
private byte[] data;
private boolean embedded;
private String embeddedFilename;
private long sizeBytes;
private String contentId;
private String disposition;
private String transferEncoding;
public void setData(byte[] data) {
this.data = data;
if (data != null) {
this.sizeBytes = data.length;
}
}
}
}
@@ -1,601 +0,0 @@
package stirling.software.common.util;
import java.nio.charset.Charset;
import java.nio.charset.StandardCharsets;
import java.util.Base64;
import java.util.Locale;
import java.util.Map;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
import lombok.experimental.UtilityClass;
import stirling.software.common.model.api.converters.EmlToPdfRequest;
import stirling.software.common.model.api.converters.HTMLToPdfRequest;
@UtilityClass
public class EmlProcessingUtils {
// Style constants
private static final int DEFAULT_FONT_SIZE = 12;
private static final String DEFAULT_FONT_FAMILY = "Helvetica, sans-serif";
private static final float DEFAULT_LINE_HEIGHT = 1.4f;
private static final String DEFAULT_ZOOM = "1.0";
private static final String DEFAULT_TEXT_COLOR = "#202124";
private static final String DEFAULT_BACKGROUND_COLOR = "#ffffff";
private static final String DEFAULT_BORDER_COLOR = "#e8eaed";
private static final String ATTACHMENT_BACKGROUND_COLOR = "#f9f9f9";
private static final String ATTACHMENT_BORDER_COLOR = "#eeeeee";
private static final int EML_CHECK_LENGTH = 8192;
private static final int MIN_HEADER_COUNT_FOR_VALID_EML = 2;
// MIME type detection
private static final Map<String, String> EXTENSION_TO_MIME_TYPE =
Map.of(
".png", "image/png",
".jpg", "image/jpeg",
".jpeg", "image/jpeg",
".gif", "image/gif",
".bmp", "image/bmp",
".webp", "image/webp",
".svg", "image/svg+xml",
".ico", "image/x-icon",
".tiff", "image/tiff",
".tif", "image/tiff");
public static void validateEmlInput(byte[] emlBytes) {
if (emlBytes == null || emlBytes.length == 0) {
throw new IllegalArgumentException("EML file is empty or null");
}
if (isInvalidEmlFormat(emlBytes)) {
throw new IllegalArgumentException("Invalid EML file format");
}
}
private static boolean isInvalidEmlFormat(byte[] emlBytes) {
try {
int checkLength = Math.min(emlBytes.length, EML_CHECK_LENGTH);
String content;
try {
content = new String(emlBytes, 0, checkLength, StandardCharsets.UTF_8);
if (content.contains("\uFFFD")) {
content = new String(emlBytes, 0, checkLength, StandardCharsets.ISO_8859_1);
}
} catch (Exception e) {
content = new String(emlBytes, 0, checkLength, StandardCharsets.ISO_8859_1);
}
String lowerContent = content.toLowerCase(Locale.ROOT);
boolean hasFrom =
lowerContent.contains("from:") || lowerContent.contains("return-path:");
boolean hasSubject = lowerContent.contains("subject:");
boolean hasMessageId = lowerContent.contains("message-id:");
boolean hasDate = lowerContent.contains("date:");
boolean hasTo =
lowerContent.contains("to:")
|| lowerContent.contains("cc:")
|| lowerContent.contains("bcc:");
boolean hasMimeStructure =
lowerContent.contains("multipart/")
|| lowerContent.contains("text/plain")
|| lowerContent.contains("text/html")
|| lowerContent.contains("boundary=");
int headerCount = 0;
if (hasFrom) headerCount++;
if (hasSubject) headerCount++;
if (hasMessageId) headerCount++;
if (hasDate) headerCount++;
if (hasTo) headerCount++;
return headerCount < MIN_HEADER_COUNT_FOR_VALID_EML && !hasMimeStructure;
} catch (RuntimeException e) {
return false;
}
}
public static String generateEnhancedEmailHtml(
EmlParser.EmailContent content,
EmlToPdfRequest request,
CustomHtmlSanitizer customHtmlSanitizer) {
StringBuilder html = new StringBuilder();
html.append(
String.format(
"""
<!DOCTYPE html>
<html lang="en"><head><meta charset="UTF-8">
<title>%s</title>
<style>
""",
sanitizeText(content.getSubject(), customHtmlSanitizer)));
appendEnhancedStyles(html);
html.append(
"""
</style>
</head><body>
""");
html.append(
String.format(
"""
<div class="email-container">
<div class="email-header">
<h1>%s</h1>
<div class="email-meta">
<div><strong>From:</strong> %s</div>
<div><strong>To:</strong> %s</div>
""",
sanitizeText(content.getSubject(), customHtmlSanitizer),
sanitizeText(content.getFrom(), customHtmlSanitizer),
sanitizeText(content.getTo(), customHtmlSanitizer)));
if (content.getCc() != null && !content.getCc().trim().isEmpty()) {
html.append(
String.format(
"<div><strong>CC:</strong> %s</div>\n",
sanitizeText(content.getCc(), customHtmlSanitizer)));
}
if (content.getBcc() != null && !content.getBcc().trim().isEmpty()) {
html.append(
String.format(
"<div><strong>BCC:</strong> %s</div>\n",
sanitizeText(content.getBcc(), customHtmlSanitizer)));
}
if (content.getDate() != null) {
html.append(
String.format(
"<div><strong>Date:</strong> %s</div>\n",
PdfAttachmentHandler.formatEmailDate(content.getDate())));
} else if (content.getDateString() != null && !content.getDateString().trim().isEmpty()) {
html.append(
String.format(
"<div><strong>Date:</strong> %s</div>\n",
sanitizeText(content.getDateString(), customHtmlSanitizer)));
}
html.append("</div></div>\n");
html.append("<div class=\"email-body\">\n");
if (content.getHtmlBody() != null && !content.getHtmlBody().trim().isEmpty()) {
String processedHtml =
processEmailHtmlBody(content.getHtmlBody(), content, customHtmlSanitizer);
html.append(processedHtml);
} else if (content.getTextBody() != null && !content.getTextBody().trim().isEmpty()) {
html.append(
String.format(
"<div class=\"text-body\">%s</div>",
convertTextToHtml(content.getTextBody(), customHtmlSanitizer)));
} else {
html.append("<div class=\"no-content\"><p><em>No content available</em></p></div>");
}
html.append("</div>\n");
if (content.getAttachmentCount() > 0 || !content.getAttachments().isEmpty()) {
appendAttachmentsSection(html, content, request, customHtmlSanitizer);
}
html.append("</div>\n</body></html>");
return html.toString();
}
public static String processEmailHtmlBody(
String htmlBody,
EmlParser.EmailContent emailContent,
CustomHtmlSanitizer customHtmlSanitizer) {
if (htmlBody == null) return "";
String processed =
customHtmlSanitizer != null ? customHtmlSanitizer.sanitize(htmlBody) : htmlBody;
processed = processed.replaceAll("(?i)\\s*position\\s*:\\s*fixed[^;]*;?", "");
processed = processed.replaceAll("(?i)\\s*position\\s*:\\s*absolute[^;]*;?", "");
if (emailContent != null && !emailContent.getAttachments().isEmpty()) {
processed = PdfAttachmentHandler.processInlineImages(processed, emailContent);
}
return processed;
}
public static String convertTextToHtml(
String textBody, CustomHtmlSanitizer customHtmlSanitizer) {
if (textBody == null) return "";
String html =
customHtmlSanitizer != null
? customHtmlSanitizer.sanitize(textBody)
: escapeHtml(textBody);
html = html.replace("\r\n", "\n").replace("\r", "\n");
html = html.replace("\n", "<br>\n");
html =
html.replaceAll(
"(https?://[\\w\\-._~:/?#\\[\\]@!$&'()*+,;=%]+)",
"<a href=\"$1\" style=\"color: #1a73e8; text-decoration: underline;\">$1</a>");
html =
html.replaceAll(
"([a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,63})",
"<a href=\"mailto:$1\" style=\"color: #1a73e8; text-decoration: underline;\">$1</a>");
return html;
}
private static void appendEnhancedStyles(StringBuilder html) {
String css =
String.format(
"""
body {
font-family: %s;
font-size: %dpx;
line-height: %s;
color: %s;
margin: 0;
padding: 16px;
background-color: %s;
}
.email-container {
width: 100%%;
max-width: 100%%;
margin: 0 auto;
}
.email-header {
padding-bottom: 10px;
border-bottom: 1px solid %s;
margin-bottom: 10px;
}
.email-header h1 {
margin: 0 0 10px 0;
font-size: %dpx;
font-weight: bold;
}
.email-meta div {
margin-bottom: 2px;
font-size: %dpx;
}
.email-body {
word-wrap: break-word;
}
.attachment-section {
margin-top: 15px;
padding: 10px;
background-color: %s;
border: 1px solid %s;
border-radius: 3px;
}
.attachment-section h3 {
margin: 0 0 8px 0;
font-size: %dpx;
}
.attachment-item {
padding: 5px 0;
}
.attachment-icon {
margin-right: 5px;
}
.attachment-details, .attachment-type {
font-size: %dpx;
color: #555555;
}
.attachment-inclusion-note, .attachment-info-note {
margin-top: 8px;
padding: 6px;
font-size: %dpx;
border-radius: 3px;
}
.attachment-inclusion-note {
background-color: #e6ffed;
border: 1px solid #d4f7dc;
color: #006420;
}
.attachment-info-note {
background-color: #fff9e6;
border: 1px solid #fff0c2;
color: #664d00;
}
.attachment-link-container {
display: flex;
align-items: center;
padding: 8px;
background-color: #f8f9fa;
border: 1px solid #dee2e6;
border-radius: 4px;
margin: 4px 0;
}
.attachment-link-container:hover {
background-color: #e9ecef;
}
.attachment-note {
font-size: %dpx;
color: #6c757d;
font-style: italic;
margin-left: 8px;
}
.no-content {
padding: 20px;
text-align: center;
color: #666;
font-style: italic;
}
.text-body {
white-space: pre-wrap;
}
img {
max-width: 100%%;
height: auto;
display: block;
}
""",
DEFAULT_FONT_FAMILY,
DEFAULT_FONT_SIZE,
DEFAULT_LINE_HEIGHT,
DEFAULT_TEXT_COLOR,
DEFAULT_BACKGROUND_COLOR,
DEFAULT_BORDER_COLOR,
DEFAULT_FONT_SIZE + 4,
DEFAULT_FONT_SIZE - 1,
ATTACHMENT_BACKGROUND_COLOR,
ATTACHMENT_BORDER_COLOR,
DEFAULT_FONT_SIZE + 1,
DEFAULT_FONT_SIZE - 2,
DEFAULT_FONT_SIZE - 2,
DEFAULT_FONT_SIZE - 3);
html.append(css);
}
private static void appendAttachmentsSection(
StringBuilder html,
EmlParser.EmailContent content,
EmlToPdfRequest request,
CustomHtmlSanitizer customHtmlSanitizer) {
html.append("<div class=\"attachment-section\">\n");
int displayedAttachmentCount =
content.getAttachmentCount() > 0
? content.getAttachmentCount()
: content.getAttachments().size();
html.append("<h3>Attachments (").append(displayedAttachmentCount).append(")</h3>\n");
if (!content.getAttachments().isEmpty()) {
for (int i = 0; i < content.getAttachments().size(); i++) {
EmlParser.EmailAttachment attachment = content.getAttachments().get(i);
String embeddedFilename =
attachment.getFilename() != null
? attachment.getFilename()
: ("attachment_" + i);
attachment.setEmbeddedFilename(embeddedFilename);
String sizeStr = GeneralUtils.formatBytes(attachment.getSizeBytes());
String contentType =
attachment.getContentType() != null
&& !attachment.getContentType().isEmpty()
? ", " + escapeHtml(attachment.getContentType())
: "";
String attachmentId = "attachment_" + i;
html.append(
String.format(
"""
<div class="attachment-item" id="%s">
<span class="attachment-icon" data-filename="%s">@</span>
<span class="attachment-name">%s</span>
<span class="attachment-details">(%s%s)</span>
</div>
""",
attachmentId,
escapeHtml(embeddedFilename),
escapeHtml(EmlParser.safeMimeDecode(attachment.getFilename())),
sizeStr,
contentType));
}
}
if (request != null && request.isIncludeAttachments()) {
html.append(
"""
<div class="attachment-info-note">
<p><em>Attachments are embedded in the file.</em></p>
</div>
""");
} else {
html.append(
"""
<div class="attachment-info-note">
<p><em>Attachment information displayed - files not included in PDF.</em></p>
</div>
""");
}
html.append("</div>\n");
}
public static HTMLToPdfRequest createHtmlRequest(EmlToPdfRequest request) {
HTMLToPdfRequest htmlRequest = new HTMLToPdfRequest();
if (request != null) {
htmlRequest.setFileInput(request.getFileInput());
}
htmlRequest.setZoom(Float.parseFloat(DEFAULT_ZOOM));
return htmlRequest;
}
public static String detectMimeType(String filename, String existingMimeType) {
if (existingMimeType != null && !existingMimeType.isEmpty()) {
return existingMimeType;
}
if (filename != null) {
String lowerFilename = filename.toLowerCase();
for (Map.Entry<String, String> entry : EXTENSION_TO_MIME_TYPE.entrySet()) {
if (lowerFilename.endsWith(entry.getKey())) {
return entry.getValue();
}
}
}
return "image/png";
}
public static String decodeUrlEncoded(String encoded) {
try {
return java.net.URLDecoder.decode(encoded, StandardCharsets.UTF_8);
} catch (Exception e) {
return encoded; // Return original if decoding fails
}
}
public static String decodeMimeHeader(String encodedText) {
if (encodedText == null || encodedText.trim().isEmpty()) {
return encodedText;
}
try {
StringBuilder result = new StringBuilder();
Pattern concatenatedPattern =
Pattern.compile(
"(=\\?[^?]+\\?[BbQq]\\?[^?]*\\?=)(\\s*=\\?[^?]+\\?[BbQq]\\?[^?]*\\?=)+");
Matcher concatenatedMatcher = concatenatedPattern.matcher(encodedText);
String processedText =
concatenatedMatcher.replaceAll(
match -> match.group().replaceAll("\\s+(?==\\?)", ""));
Pattern mimePattern = Pattern.compile("=\\?([^?]+)\\?([BbQq])\\?([^?]*)\\?=");
Matcher matcher = mimePattern.matcher(processedText);
int lastEnd = 0;
while (matcher.find()) {
result.append(processedText, lastEnd, matcher.start());
String charset = matcher.group(1);
String encoding = matcher.group(2).toUpperCase();
String encodedValue = matcher.group(3);
try {
String decodedValue =
switch (encoding) {
case "B" -> {
String cleanBase64 = encodedValue.replaceAll("\\s", "");
byte[] decodedBytes = Base64.getDecoder().decode(cleanBase64);
Charset targetCharset;
try {
targetCharset = Charset.forName(charset);
} catch (Exception e) {
targetCharset = StandardCharsets.UTF_8;
}
yield new String(decodedBytes, targetCharset);
}
case "Q" -> decodeQuotedPrintable(encodedValue, charset);
default -> matcher.group(0); // Return original if unknown encoding
};
result.append(decodedValue);
} catch (RuntimeException e) {
result.append(matcher.group(0)); // Keep original on decode error
}
lastEnd = matcher.end();
}
result.append(processedText.substring(lastEnd));
return result.toString();
} catch (Exception e) {
return encodedText; // Return original on any parsing error
}
}
private static String decodeQuotedPrintable(String encodedText, String charset) {
StringBuilder result = new StringBuilder();
for (int i = 0; i < encodedText.length(); i++) {
char c = encodedText.charAt(i);
switch (c) {
case '=' -> {
if (i + 2 < encodedText.length()) {
String hex = encodedText.substring(i + 1, i + 3);
try {
int value = Integer.parseInt(hex, 16);
result.append((char) value);
i += 2;
} catch (NumberFormatException e) {
result.append(c);
}
} else if (i + 1 == encodedText.length()
|| (i + 2 == encodedText.length()
&& encodedText.charAt(i + 1) == '\n')) {
if (i + 1 < encodedText.length() && encodedText.charAt(i + 1) == '\n') {
i++; // Skip the newline too
}
} else {
result.append(c);
}
}
case '_' -> result.append(' '); // Space encoding in Q encoding
default -> result.append(c);
}
}
byte[] bytes = result.toString().getBytes(StandardCharsets.ISO_8859_1);
try {
Charset targetCharset = Charset.forName(charset);
return new String(bytes, targetCharset);
} catch (Exception e) {
try {
return new String(bytes, StandardCharsets.UTF_8);
} catch (Exception fallbackException) {
return new String(bytes, StandardCharsets.ISO_8859_1);
}
}
}
public static String escapeHtml(String text) {
if (text == null) return "";
return text.replace("&", "&amp;")
.replace("<", "&lt;")
.replace(">", "&gt;")
.replace("\"", "&quot;")
.replace("'", "&#39;");
}
public static String sanitizeText(String text, CustomHtmlSanitizer customHtmlSanitizer) {
if (customHtmlSanitizer != null) {
return customHtmlSanitizer.sanitize(text);
} else {
return escapeHtml(text);
}
}
public static String simplifyHtmlContent(String htmlContent) {
String simplified = htmlContent.replaceAll("(?i)<script[^>]*>.*?</script>", "");
simplified = simplified.replaceAll("(?i)<style[^>]*>.*?</style>", "");
return simplified;
}
}
File diff suppressed because it is too large Load Diff
@@ -26,8 +26,8 @@ public class FileToPdf {
HTMLToPdfRequest request,
byte[] fileBytes,
String fileName,
TempFileManager tempFileManager,
CustomHtmlSanitizer customHtmlSanitizer)
boolean disableSanitize,
TempFileManager tempFileManager)
throws IOException, InterruptedException {
try (TempFile tempOutputFile = new TempFile(tempFileManager, ".pdf")) {
@@ -39,15 +39,14 @@ public class FileToPdf {
if (fileName.toLowerCase().endsWith(".html")) {
String sanitizedHtml =
sanitizeHtmlContent(
new String(fileBytes, StandardCharsets.UTF_8),
customHtmlSanitizer);
new String(fileBytes, StandardCharsets.UTF_8), disableSanitize);
Files.write(
tempInputFile.getPath(),
sanitizedHtml.getBytes(StandardCharsets.UTF_8));
} else if (fileName.toLowerCase().endsWith(".zip")) {
Files.write(tempInputFile.getPath(), fileBytes);
sanitizeHtmlFilesInZip(
tempInputFile.getPath(), tempFileManager, customHtmlSanitizer);
tempInputFile.getPath(), disableSanitize, tempFileManager);
} else {
throw ExceptionUtils.createHtmlFileRequiredException();
}
@@ -79,15 +78,12 @@ public class FileToPdf {
} // tempOutputFile auto-closed
}
private static String sanitizeHtmlContent(
String htmlContent, CustomHtmlSanitizer customHtmlSanitizer) {
return customHtmlSanitizer.sanitize(htmlContent);
private static String sanitizeHtmlContent(String htmlContent, boolean disableSanitize) {
return (!disableSanitize) ? CustomHtmlSanitizer.sanitize(htmlContent) : htmlContent;
}
private static void sanitizeHtmlFilesInZip(
Path zipFilePath,
TempFileManager tempFileManager,
CustomHtmlSanitizer customHtmlSanitizer)
Path zipFilePath, boolean disableSanitize, TempFileManager tempFileManager)
throws IOException {
try (TempDirectory tempUnzippedDir = new TempDirectory(tempFileManager)) {
try (ZipInputStream zipIn =
@@ -103,8 +99,7 @@ public class FileToPdf {
|| entry.getName().toLowerCase().endsWith(".htm")) {
String content =
new String(zipIn.readAllBytes(), StandardCharsets.UTF_8);
String sanitizedContent =
sanitizeHtmlContent(content, customHtmlSanitizer);
String sanitizedContent = sanitizeHtmlContent(content, disableSanitize);
Files.write(
filePath, sanitizedContent.getBytes(StandardCharsets.UTF_8));
} else {
@@ -14,7 +14,6 @@ import java.util.Arrays;
import java.util.Enumeration;
import java.util.List;
import java.util.Locale;
import java.util.Set;
import java.util.UUID;
import org.springframework.core.io.ClassPathResource;
@@ -35,16 +34,8 @@ import stirling.software.common.configuration.InstallationPathConfig;
@Slf4j
public class GeneralUtils {
private static final Set<String> DEFAULT_VALID_SCRIPTS =
Set.of("png_to_webp.py", "split_photos.py");
private static final Set<String> DEFAULT_VALID_PIPELINE =
Set.of(
"OCR images.json",
"Prepare-pdfs-for-email.json",
"split-rotate-auto-rename.json");
private static final String DEFAULT_WEBUI_CONFIGS_DIR = "defaultWebUIConfigs";
private static final String PYTHON_SCRIPTS_DIR = "python";
private static final List<String> DEFAULT_VALID_SCRIPTS =
List.of("png_to_webp.py", "split_photos.py");
public static File convertMultipartFileToFile(MultipartFile multipartFile) throws IOException {
String customTempDir = System.getenv("STIRLING_TEMPFILES_DIRECTORY");
@@ -456,46 +447,7 @@ public class GeneralUtils {
}
/**
* Extracts the default pipeline configurations from the classpath to the installation path.
* Creates directories if needed and copies default JSON files.
*
* <p>Existing files will be overwritten atomically (when supported). In case of unsupported
* atomic moves, falls back to non-atomic replace.
*
* @throws IOException if an I/O error occurs during file operations
*/
public static void extractPipeline() throws IOException {
Path pipelineDir =
Paths.get(InstallationPathConfig.getPipelinePath(), DEFAULT_WEBUI_CONFIGS_DIR);
Files.createDirectories(pipelineDir);
for (String name : DEFAULT_VALID_PIPELINE) {
if (!Paths.get(name).getFileName().toString().equals(name)) {
log.error("Invalid pipeline file name: {}", name);
throw new IllegalArgumentException("Invalid pipeline file name: " + name);
}
Path target = pipelineDir.resolve(name);
ClassPathResource res =
new ClassPathResource(
"static/pipeline/" + DEFAULT_WEBUI_CONFIGS_DIR + "/" + name);
if (!res.exists()) {
log.error("Resource not found: {}", res.getPath());
throw new IOException("Resource not found: " + res.getPath());
}
copyResourceToFile(res, target);
}
}
/**
* Extracts the specified Python script from the classpath to the installation path. Validates
* name and copies file atomically when possible, overwriting existing.
*
* <p>Existing files will be overwritten atomically (when supported).
*
* @param scriptName the name of the script to extract
* @return the path to the extracted script
* @throws IllegalArgumentException if the script name is invalid or not allowed
* @throws IOException if an I/O error occurs
* Extracts a file from classpath:/static/python to a temporary directory and returns the path.
*/
public static Path extractScript(String scriptName) throws IOException {
// Validate input
@@ -506,71 +458,26 @@ public class GeneralUtils {
throw new IllegalArgumentException(
"scriptName must not contain path traversal characters");
}
if (!Paths.get(scriptName).getFileName().toString().equals(scriptName)) {
throw new IllegalArgumentException(
"scriptName must not contain path traversal characters");
}
if (!DEFAULT_VALID_SCRIPTS.contains(scriptName)) {
throw new IllegalArgumentException(
"scriptName must be either 'png_to_webp.py' or 'split_photos.py'");
}
Path scriptsDir = Paths.get(InstallationPathConfig.getScriptsPath(), PYTHON_SCRIPTS_DIR);
Path scriptsDir = Paths.get(InstallationPathConfig.getScriptsPath(), "python");
Files.createDirectories(scriptsDir);
Path target = scriptsDir.resolve(scriptName);
ClassPathResource res =
new ClassPathResource("static/" + PYTHON_SCRIPTS_DIR + "/" + scriptName);
if (!res.exists()) {
log.error("Resource not found: {}", res.getPath());
throw new IOException("Resource not found: " + res.getPath());
}
copyResourceToFile(res, target);
return target;
}
/**
* Copies a resource from the classpath to a specified target file.
*
* @param resource the ClassPathResource to copy
* @param target the target Path where the resource will be copied
* @throws IOException if an I/O error occurs during the copy operation
*/
private static void copyResourceToFile(ClassPathResource resource, Path target)
throws IOException {
Path dir = target.getParent();
Path tmp = Files.createTempFile(dir, target.getFileName().toString(), ".tmp");
try (InputStream in = resource.getInputStream()) {
Files.copy(in, tmp, StandardCopyOption.REPLACE_EXISTING);
try {
Files.move(tmp, target, StandardCopyOption.ATOMIC_MOVE);
} catch (AtomicMoveNotSupportedException e) {
log.warn(
"Atomic move not supported, falling back to non-atomic move for {}",
target,
e);
Files.move(tmp, target, StandardCopyOption.REPLACE_EXISTING);
}
} catch (FileAlreadyExistsException e) {
log.debug("File already exists at {}, attempting to replace it.", target);
Files.move(tmp, target, StandardCopyOption.REPLACE_EXISTING);
} catch (AccessDeniedException e) {
log.error("Access denied while attempting to copy resource to {}", target, e);
throw e;
} catch (FileSystemException e) {
log.error("File system error occurred while copying resource to {}", target, e);
throw e;
} catch (IOException e) {
log.error("Failed to copy resource to {}", target, e);
throw e;
} finally {
try {
Files.deleteIfExists(tmp);
Path scriptFile = scriptsDir.resolve(scriptName);
if (!Files.exists(scriptFile)) {
ClassPathResource resource = new ClassPathResource("static/python/" + scriptName);
try (InputStream in = resource.getInputStream()) {
Files.copy(in, scriptFile, StandardCopyOption.REPLACE_EXISTING);
} catch (IOException e) {
log.warn("Failed to delete temporary file {}", tmp, e);
log.error("Failed to extract Python script", e);
throw e;
}
}
return scriptFile;
}
public static boolean isVersionHigher(String currentVersion, String compareVersion) {
@@ -1,680 +0,0 @@
package stirling.software.common.util;
import static stirling.software.common.util.AttachmentUtils.setCatalogViewerPreferences;
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.text.SimpleDateFormat;
import java.util.ArrayList;
import java.util.Base64;
import java.util.Date;
import java.util.GregorianCalendar;
import java.util.HashMap;
import java.util.List;
import java.util.Locale;
import java.util.Map;
import java.util.Set;
import java.util.TimeZone;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.apache.pdfbox.pdmodel.PDDocumentCatalog;
import org.apache.pdfbox.pdmodel.PDDocumentNameDictionary;
import org.apache.pdfbox.pdmodel.PDEmbeddedFilesNameTreeNode;
import org.apache.pdfbox.pdmodel.PDPage;
import org.apache.pdfbox.pdmodel.PageMode;
import org.apache.pdfbox.pdmodel.common.PDRectangle;
import org.apache.pdfbox.pdmodel.common.filespecification.PDComplexFileSpecification;
import org.apache.pdfbox.pdmodel.common.filespecification.PDEmbeddedFile;
import org.apache.pdfbox.pdmodel.interactive.annotation.PDAnnotationFileAttachment;
import org.apache.pdfbox.pdmodel.interactive.annotation.PDAppearanceDictionary;
import org.apache.pdfbox.pdmodel.interactive.annotation.PDAppearanceStream;
import org.apache.pdfbox.text.PDFTextStripper;
import org.apache.pdfbox.text.TextPosition;
import org.jetbrains.annotations.NotNull;
import org.springframework.web.multipart.MultipartFile;
import lombok.Data;
import lombok.Getter;
import lombok.experimental.UtilityClass;
import stirling.software.common.service.CustomPDFDocumentFactory;
@UtilityClass
public class PdfAttachmentHandler {
// Note: This class is designed for EML attachments, not general PDF attachments.
private static final String ATTACHMENT_MARKER = "@";
private static final float ATTACHMENT_ICON_WIDTH = 12f;
private static final float ATTACHMENT_ICON_HEIGHT = 14f;
private static final float ANNOTATION_X_OFFSET = 2f;
private static final float ANNOTATION_Y_OFFSET = 10f;
public static byte[] attachFilesToPdf(
byte[] pdfBytes,
List<EmlParser.EmailAttachment> attachments,
CustomPDFDocumentFactory pdfDocumentFactory)
throws IOException {
if (attachments == null || attachments.isEmpty()) {
return pdfBytes;
}
try (PDDocument document = pdfDocumentFactory.load(pdfBytes);
ByteArrayOutputStream outputStream = new ByteArrayOutputStream()) {
List<MultipartFile> multipartAttachments = new ArrayList<>(attachments.size());
for (int i = 0; i < attachments.size(); i++) {
EmlParser.EmailAttachment attachment = attachments.get(i);
if (attachment.getData() != null && attachment.getData().length > 0) {
String embeddedFilename =
attachment.getFilename() != null
? attachment.getFilename()
: ("attachment_" + i);
attachment.setEmbeddedFilename(embeddedFilename);
multipartAttachments.add(createMultipartFile(attachment));
}
}
if (!multipartAttachments.isEmpty()) {
Map<Integer, String> indexToFilenameMap =
addAttachmentsToDocumentWithMapping(
document, multipartAttachments, attachments);
setCatalogViewerPreferences(document, PageMode.USE_ATTACHMENTS);
addAttachmentAnnotationsToDocumentWithMapping(
document, attachments, indexToFilenameMap);
}
document.save(outputStream);
return outputStream.toByteArray();
} catch (RuntimeException e) {
throw new IOException(
"Invalid PDF structure or processing error: " + e.getMessage(), e);
} catch (Exception e) {
throw new IOException("Error attaching files to PDF: " + e.getMessage(), e);
}
}
private static MultipartFile createMultipartFile(EmlParser.EmailAttachment attachment) {
return new MultipartFile() {
@Override
public @NotNull String getName() {
return "attachment";
}
@Override
public String getOriginalFilename() {
return attachment.getFilename() != null
? attachment.getFilename()
: "attachment_" + System.currentTimeMillis();
}
@Override
public String getContentType() {
return attachment.getContentType() != null
? attachment.getContentType()
: "application/octet-stream";
}
@Override
public boolean isEmpty() {
return attachment.getData() == null || attachment.getData().length == 0;
}
@Override
public long getSize() {
return attachment.getData() != null ? attachment.getData().length : 0;
}
@Override
public byte @NotNull [] getBytes() {
return attachment.getData() != null ? attachment.getData() : new byte[0];
}
@Override
public @NotNull InputStream getInputStream() {
byte[] data = attachment.getData();
return new ByteArrayInputStream(data != null ? data : new byte[0]);
}
@Override
public void transferTo(@NotNull File dest) throws IOException, IllegalStateException {
try (FileOutputStream fos = new FileOutputStream(dest)) {
byte[] data = attachment.getData();
if (data != null) {
fos.write(data);
}
}
}
};
}
private static String ensureUniqueFilename(String filename, Set<String> existingNames) {
if (!existingNames.contains(filename)) {
return filename;
}
String baseName;
String extension = "";
int lastDot = filename.lastIndexOf('.');
if (lastDot > 0) {
baseName = filename.substring(0, lastDot);
extension = filename.substring(lastDot);
} else {
baseName = filename;
}
int counter = 1;
String uniqueName;
do {
uniqueName = baseName + "_" + counter + extension;
counter++;
} while (existingNames.contains(uniqueName));
return uniqueName;
}
private static @NotNull PDRectangle calculateAnnotationRectangle(
PDPage page, float x, float y) {
PDRectangle cropBox = page.getCropBox();
// ISO 32000-1:2008 Section 8.3: PDF coordinate system transforms
int rotation = page.getRotation();
float pdfX = x;
float pdfY = cropBox.getHeight() - y;
switch (rotation) {
case 90 -> {
float temp = pdfX;
pdfX = pdfY;
pdfY = cropBox.getWidth() - temp;
}
case 180 -> {
pdfX = cropBox.getWidth() - pdfX;
pdfY = y;
}
case 270 -> {
float temp = pdfX;
pdfX = cropBox.getHeight() - pdfY;
pdfY = temp;
}
default -> {}
}
float iconHeight = ATTACHMENT_ICON_HEIGHT;
float paddingX = 2.0f;
float paddingY = 2.0f;
PDRectangle rect =
new PDRectangle(
pdfX + ANNOTATION_X_OFFSET + paddingX,
pdfY - iconHeight + ANNOTATION_Y_OFFSET + paddingY,
ATTACHMENT_ICON_WIDTH,
iconHeight);
PDRectangle mediaBox = page.getMediaBox();
if (rect.getLowerLeftX() < mediaBox.getLowerLeftX()
|| rect.getLowerLeftY() < mediaBox.getLowerLeftY()
|| rect.getUpperRightX() > mediaBox.getUpperRightX()
|| rect.getUpperRightY() > mediaBox.getUpperRightY()) {
float adjustedX =
Math.max(
mediaBox.getLowerLeftX(),
Math.min(
rect.getLowerLeftX(),
mediaBox.getUpperRightX() - rect.getWidth()));
float adjustedY =
Math.max(
mediaBox.getLowerLeftY(),
Math.min(
rect.getLowerLeftY(),
mediaBox.getUpperRightY() - rect.getHeight()));
rect = new PDRectangle(adjustedX, adjustedY, rect.getWidth(), rect.getHeight());
}
return rect;
}
public static String processInlineImages(
String htmlContent, EmlParser.EmailContent emailContent) {
if (htmlContent == null || emailContent == null) return htmlContent;
Map<String, EmlParser.EmailAttachment> contentIdMap = new HashMap<>();
for (EmlParser.EmailAttachment attachment : emailContent.getAttachments()) {
if (attachment.isEmbedded()
&& attachment.getContentId() != null
&& attachment.getData() != null) {
contentIdMap.put(attachment.getContentId(), attachment);
}
}
if (contentIdMap.isEmpty()) return htmlContent;
Pattern cidPattern =
Pattern.compile(
"(?i)<img[^>]*\\ssrc\\s*=\\s*['\"]cid:([^'\"]+)['\"][^>]*>",
Pattern.CASE_INSENSITIVE);
Matcher matcher = cidPattern.matcher(htmlContent);
StringBuilder result = new StringBuilder();
while (matcher.find()) {
String contentId = matcher.group(1);
EmlParser.EmailAttachment attachment = contentIdMap.get(contentId);
if (attachment != null && attachment.getData() != null) {
String mimeType =
EmlProcessingUtils.detectMimeType(
attachment.getFilename(), attachment.getContentType());
String base64Data = Base64.getEncoder().encodeToString(attachment.getData());
String dataUri = "data:" + mimeType + ";base64," + base64Data;
String replacement =
matcher.group(0).replaceFirst("cid:" + Pattern.quote(contentId), dataUri);
matcher.appendReplacement(result, Matcher.quoteReplacement(replacement));
} else {
matcher.appendReplacement(result, Matcher.quoteReplacement(matcher.group(0)));
}
}
matcher.appendTail(result);
return result.toString();
}
public static String formatEmailDate(Date date) {
if (date == null) return "";
SimpleDateFormat formatter =
new SimpleDateFormat("EEE, MMM d, yyyy 'at' h:mm a z", Locale.ENGLISH);
formatter.setTimeZone(TimeZone.getTimeZone("UTC"));
return formatter.format(date);
}
@Data
public static class MarkerPosition {
private int pageIndex;
private float x;
private float y;
private String character;
private String filename;
public MarkerPosition(int pageIndex, float x, float y, String character, String filename) {
this.pageIndex = pageIndex;
this.x = x;
this.y = y;
this.character = character;
this.filename = filename;
}
}
public static class AttachmentMarkerPositionFinder extends PDFTextStripper {
@Getter private final List<MarkerPosition> positions = new ArrayList<>();
private int currentPageIndex;
protected boolean sortByPosition;
private boolean isInAttachmentSection;
private boolean attachmentSectionFound;
private final StringBuilder currentText = new StringBuilder();
private static final Pattern ATTACHMENT_SECTION_PATTERN =
Pattern.compile("attachments\\s*\\(\\d+\\)", Pattern.CASE_INSENSITIVE);
private static final Pattern FILENAME_PATTERN =
Pattern.compile("@\\s*([^\\s\\(]+(?:\\.[a-zA-Z0-9]+)?)");
public AttachmentMarkerPositionFinder() {
super();
this.currentPageIndex = 0;
this.sortByPosition = false; // Disable sorting to preserve document order
this.isInAttachmentSection = false;
this.attachmentSectionFound = false;
}
@Override
public String getText(PDDocument document) throws IOException {
super.getText(document);
if (sortByPosition) {
positions.sort(
(a, b) -> {
int pageCompare = Integer.compare(a.getPageIndex(), b.getPageIndex());
if (pageCompare != 0) return pageCompare;
return Float.compare(
b.getY(), a.getY()); // Descending Y per PDF coordinate system
});
}
return ""; // Return empty string as we only need positions
}
@Override
protected void startPage(PDPage page) throws IOException {
super.startPage(page);
}
@Override
protected void endPage(PDPage page) throws IOException {
currentPageIndex++;
super.endPage(page);
}
@Override
protected void writeString(String string, List<TextPosition> textPositions)
throws IOException {
String lowerString = string.toLowerCase();
if (ATTACHMENT_SECTION_PATTERN.matcher(lowerString).find()) {
isInAttachmentSection = true;
attachmentSectionFound = true;
}
if (isInAttachmentSection
&& (lowerString.contains("</body>")
|| lowerString.contains("</html>")
|| (attachmentSectionFound
&& lowerString.trim().isEmpty()
&& string.length() > 50))) {
isInAttachmentSection = false;
}
if (isInAttachmentSection) {
currentText.append(string);
for (int i = 0; (i = string.indexOf(ATTACHMENT_MARKER, i)) != -1; i++) {
if (i < textPositions.size()) {
TextPosition textPosition = textPositions.get(i);
String filename = extractFilenameAfterMarker(string, i);
MarkerPosition position =
new MarkerPosition(
currentPageIndex,
textPosition.getXDirAdj(),
textPosition.getYDirAdj(),
ATTACHMENT_MARKER,
filename);
positions.add(position);
}
}
}
super.writeString(string, textPositions);
}
@Override
public void setSortByPosition(boolean sortByPosition) {
this.sortByPosition = sortByPosition;
}
private String extractFilenameAfterMarker(String text, int markerIndex) {
String afterMarker = text.substring(markerIndex + 1);
Matcher matcher = FILENAME_PATTERN.matcher("@" + afterMarker);
if (matcher.find()) {
return matcher.group(1);
}
String[] parts = afterMarker.split("[\\s\\(\\)]+");
for (String part : parts) {
part = part.trim();
if (part.length() > 3 && part.contains(".")) {
return part;
}
}
return null;
}
}
private static Map<Integer, String> addAttachmentsToDocumentWithMapping(
PDDocument document,
List<MultipartFile> attachments,
List<EmlParser.EmailAttachment> originalAttachments)
throws IOException {
PDDocumentCatalog catalog = document.getDocumentCatalog();
if (catalog == null) {
throw new IOException("PDF document catalog is not accessible");
}
PDDocumentNameDictionary documentNames = catalog.getNames();
if (documentNames == null) {
documentNames = new PDDocumentNameDictionary(catalog);
catalog.setNames(documentNames);
}
PDEmbeddedFilesNameTreeNode embeddedFilesTree = documentNames.getEmbeddedFiles();
if (embeddedFilesTree == null) {
embeddedFilesTree = new PDEmbeddedFilesNameTreeNode();
documentNames.setEmbeddedFiles(embeddedFilesTree);
}
Map<String, PDComplexFileSpecification> existingNames = embeddedFilesTree.getNames();
if (existingNames == null) {
existingNames = new HashMap<>();
}
Map<Integer, String> indexToFilenameMap = new HashMap<>();
for (int i = 0; i < attachments.size(); i++) {
MultipartFile attachment = attachments.get(i);
String filename = attachment.getOriginalFilename();
if (filename == null || filename.trim().isEmpty()) {
filename = "attachment_" + i;
}
String normalizedFilename =
isAscii(filename)
? filename
: java.text.Normalizer.normalize(
filename, java.text.Normalizer.Form.NFC);
String uniqueFilename =
ensureUniqueFilename(normalizedFilename, existingNames.keySet());
indexToFilenameMap.put(i, uniqueFilename);
PDEmbeddedFile embeddedFile = new PDEmbeddedFile(document, attachment.getInputStream());
embeddedFile.setSize((int) attachment.getSize());
GregorianCalendar currentTime = new GregorianCalendar();
embeddedFile.setCreationDate(currentTime);
embeddedFile.setModDate(currentTime);
String contentType = attachment.getContentType();
if (contentType != null && !contentType.trim().isEmpty()) {
embeddedFile.setSubtype(contentType);
}
PDComplexFileSpecification fileSpecification = new PDComplexFileSpecification();
fileSpecification.setFile(uniqueFilename);
fileSpecification.setFileUnicode(uniqueFilename);
fileSpecification.setEmbeddedFile(embeddedFile);
fileSpecification.setEmbeddedFileUnicode(embeddedFile);
existingNames.put(uniqueFilename, fileSpecification);
}
embeddedFilesTree.setNames(existingNames);
documentNames.setEmbeddedFiles(embeddedFilesTree);
catalog.setNames(documentNames);
return indexToFilenameMap;
}
private static void addAttachmentAnnotationsToDocumentWithMapping(
PDDocument document,
List<EmlParser.EmailAttachment> attachments,
Map<Integer, String> indexToFilenameMap)
throws IOException {
if (document.getNumberOfPages() == 0 || attachments == null || attachments.isEmpty()) {
return;
}
AttachmentMarkerPositionFinder finder = new AttachmentMarkerPositionFinder();
finder.setSortByPosition(false); // Keep document order to maintain pairing
finder.getText(document);
List<MarkerPosition> markerPositions = finder.getPositions();
int annotationsToAdd = Math.min(markerPositions.size(), attachments.size());
for (int i = 0; i < annotationsToAdd; i++) {
MarkerPosition position = markerPositions.get(i);
String filenameNearMarker = position.getFilename();
EmlParser.EmailAttachment matchingAttachment =
findAttachmentByFilename(attachments, filenameNearMarker);
if (matchingAttachment != null) {
String embeddedFilename =
findEmbeddedFilenameForAttachment(matchingAttachment, indexToFilenameMap);
if (embeddedFilename != null) {
PDPage page = document.getPage(position.getPageIndex());
addAttachmentAnnotationToPageWithMapping(
document,
page,
matchingAttachment,
embeddedFilename,
position.getX(),
position.getY(),
i);
} else {
// No embedded filename found for attachment
}
} else {
// No matching attachment found for filename near marker
}
}
}
private static EmlParser.EmailAttachment findAttachmentByFilename(
List<EmlParser.EmailAttachment> attachments, String targetFilename) {
if (targetFilename == null || targetFilename.trim().isEmpty()) {
return null;
}
String normalizedTarget = normalizeFilename(targetFilename);
// First try exact match
for (EmlParser.EmailAttachment attachment : attachments) {
if (attachment.getFilename() != null) {
String normalizedAttachment = normalizeFilename(attachment.getFilename());
if (normalizedAttachment.equals(normalizedTarget)) {
return attachment;
}
}
}
// Then try contains match
for (EmlParser.EmailAttachment attachment : attachments) {
if (attachment.getFilename() != null) {
String normalizedAttachment = normalizeFilename(attachment.getFilename());
if (normalizedAttachment.contains(normalizedTarget)
|| normalizedTarget.contains(normalizedAttachment)) {
return attachment;
}
}
}
return null;
}
private static String findEmbeddedFilenameForAttachment(
EmlParser.EmailAttachment attachment, Map<Integer, String> indexToFilenameMap) {
String attachmentFilename = attachment.getFilename();
if (attachmentFilename == null) {
return null;
}
for (Map.Entry<Integer, String> entry : indexToFilenameMap.entrySet()) {
String embeddedFilename = entry.getValue();
if (embeddedFilename != null
&& (embeddedFilename.equals(attachmentFilename)
|| embeddedFilename.contains(attachmentFilename)
|| attachmentFilename.contains(embeddedFilename))) {
return embeddedFilename;
}
}
return null;
}
private static String normalizeFilename(String filename) {
if (filename == null) return "";
return filename.toLowerCase()
.trim()
.replaceAll("\\s+", " ")
.replaceAll("[^a-zA-Z0-9._-]", "");
}
private static void addAttachmentAnnotationToPageWithMapping(
PDDocument document,
PDPage page,
EmlParser.EmailAttachment attachment,
String embeddedFilename,
float x,
float y,
int attachmentIndex)
throws IOException {
PDAnnotationFileAttachment fileAnnotation = new PDAnnotationFileAttachment();
PDRectangle rect = calculateAnnotationRectangle(page, x, y);
fileAnnotation.setRectangle(rect);
fileAnnotation.setPrinted(false);
fileAnnotation.setHidden(false);
fileAnnotation.setNoView(false);
fileAnnotation.setNoZoom(true);
fileAnnotation.setNoRotate(true);
try {
PDAppearanceDictionary appearance = new PDAppearanceDictionary();
PDAppearanceStream normalAppearance = new PDAppearanceStream(document);
normalAppearance.setBBox(new PDRectangle(0, 0, rect.getWidth(), rect.getHeight()));
appearance.setNormalAppearance(normalAppearance);
fileAnnotation.setAppearance(appearance);
} catch (RuntimeException e) {
fileAnnotation.setAppearance(null);
}
PDEmbeddedFilesNameTreeNode efTree =
document.getDocumentCatalog().getNames().getEmbeddedFiles();
if (efTree != null) {
Map<String, PDComplexFileSpecification> efMap = efTree.getNames();
if (efMap != null) {
PDComplexFileSpecification fileSpec = efMap.get(embeddedFilename);
if (fileSpec != null) {
fileAnnotation.setFile(fileSpec);
} else {
// Could not find embedded file
}
}
}
fileAnnotation.setContents(
"Attachment " + (attachmentIndex + 1) + ": " + attachment.getFilename());
fileAnnotation.setAnnotationName(
"EmbeddedFile_" + attachmentIndex + "_" + embeddedFilename);
page.getAnnotations().add(fileAnnotation);
}
private static boolean isAscii(String str) {
if (str == null) return true;
for (int i = 0; i < str.length(); i++) {
if (str.charAt(i) > 127) {
return false;
}
}
return true;
}
}
@@ -35,7 +35,6 @@ import io.github.pixee.security.Filenames;
import lombok.extern.slf4j.Slf4j;
import stirling.software.common.model.ApplicationProperties;
import stirling.software.common.service.CustomPDFDocumentFactory;
@Slf4j
@@ -146,18 +145,13 @@ public class PdfUtils {
throws IOException, Exception {
// Validate and limit DPI to prevent excessive memory usage
int maxSafeDpi = 500; // Default maximum safe DPI
ApplicationProperties properties =
ApplicationContextProvider.getBean(ApplicationProperties.class);
if (properties != null && properties.getSystem() != null) {
maxSafeDpi = properties.getSystem().getMaxDPI();
}
if (DPI > maxSafeDpi) {
final int MAX_SAFE_DPI = 500; // Maximum safe DPI to prevent memory issues
if (DPI > MAX_SAFE_DPI) {
throw ExceptionUtils.createIllegalArgumentException(
"error.dpiExceedsLimit",
"DPI value {0} exceeds maximum safe limit of {1}. High DPI values can cause memory issues and crashes. Please use a lower DPI value.",
DPI,
maxSafeDpi);
MAX_SAFE_DPI);
}
try (PDDocument document = pdfDocumentFactory.load(inputStream)) {
@@ -7,19 +7,24 @@ import static org.mockito.ArgumentMatchers.any;
import static org.mockito.ArgumentMatchers.anyBoolean;
import static org.mockito.ArgumentMatchers.anyInt;
import static org.mockito.ArgumentMatchers.anyLong;
import static org.mockito.ArgumentMatchers.anyString;
import static org.mockito.ArgumentMatchers.eq;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.times;
import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.when;
import org.junit.jupiter.api.BeforeEach;
import java.util.Arrays;
import java.util.function.Supplier;
import org.aspectj.lang.ProceedingJoinPoint;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.ArgumentCaptor;
import org.mockito.Captor;
import org.mockito.InjectMocks;
import org.mockito.Mock;
import org.mockito.junit.jupiter.MockitoExtension;
import org.springframework.http.ResponseEntity;
@@ -40,44 +45,62 @@ class AutoJobPostMappingIntegrationTest {
private AutoJobAspect autoJobAspect;
@Mock private JobExecutorService jobExecutorService;
@Mock
private JobExecutorService jobExecutorService;
@Mock private HttpServletRequest request;
@Mock
private HttpServletRequest request;
@Mock private FileOrUploadService fileOrUploadService;
@Mock
private FileOrUploadService fileOrUploadService;
@Mock private FileStorage fileStorage;
@Mock
private FileStorage fileStorage;
@Mock private ResourceMonitor resourceMonitor;
@Mock private JobQueue jobQueue;
@Mock
private ResourceMonitor resourceMonitor;
@Mock
private JobQueue jobQueue;
@BeforeEach
void setUp() {
autoJobAspect =
new AutoJobAspect(jobExecutorService, request, fileOrUploadService, fileStorage);
autoJobAspect = new AutoJobAspect(
jobExecutorService,
request,
fileOrUploadService,
fileStorage
);
}
@Mock private ProceedingJoinPoint joinPoint;
@Mock
private ProceedingJoinPoint joinPoint;
@Mock private AutoJobPostMapping autoJobPostMapping;
@Mock
private AutoJobPostMapping autoJobPostMapping;
@Captor private ArgumentCaptor<Supplier<Object>> workCaptor;
@Captor
private ArgumentCaptor<Supplier<Object>> workCaptor;
@Captor private ArgumentCaptor<Boolean> asyncCaptor;
@Captor
private ArgumentCaptor<Boolean> asyncCaptor;
@Captor private ArgumentCaptor<Long> timeoutCaptor;
@Captor
private ArgumentCaptor<Long> timeoutCaptor;
@Captor private ArgumentCaptor<Boolean> queueableCaptor;
@Captor
private ArgumentCaptor<Boolean> queueableCaptor;
@Captor private ArgumentCaptor<Integer> resourceWeightCaptor;
@Captor
private ArgumentCaptor<Integer> resourceWeightCaptor;
@Test
void shouldExecuteWithCustomParameters() throws Throwable {
// Given
PDFFile pdfFile = new PDFFile();
pdfFile.setFileId("test-file-id");
Object[] args = new Object[] {pdfFile};
Object[] args = new Object[] { pdfFile };
when(joinPoint.getArgs()).thenReturn(args);
when(request.getParameter("async")).thenReturn("true");
@@ -90,8 +113,9 @@ class AutoJobPostMappingIntegrationTest {
MultipartFile mockFile = mock(MultipartFile.class);
when(fileStorage.retrieveFile("test-file-id")).thenReturn(mockFile);
when(jobExecutorService.runJobGeneric(
anyBoolean(), any(Supplier.class), anyLong(), anyBoolean(), anyInt()))
anyBoolean(), any(Supplier.class), anyLong(), anyBoolean(), anyInt()))
.thenReturn(ResponseEntity.ok("success"));
// When
@@ -100,13 +124,12 @@ class AutoJobPostMappingIntegrationTest {
// Then
assertEquals(ResponseEntity.ok("success"), result);
verify(jobExecutorService)
.runJobGeneric(
asyncCaptor.capture(),
workCaptor.capture(),
timeoutCaptor.capture(),
queueableCaptor.capture(),
resourceWeightCaptor.capture());
verify(jobExecutorService).runJobGeneric(
asyncCaptor.capture(),
workCaptor.capture(),
timeoutCaptor.capture(),
queueableCaptor.capture(),
resourceWeightCaptor.capture());
assertTrue(asyncCaptor.getValue(), "Async should be true");
assertEquals(60000L, timeoutCaptor.getValue(), "Timeout should be 60000ms");
@@ -135,12 +158,11 @@ class AutoJobPostMappingIntegrationTest {
// Mock jobExecutorService to execute the work immediately
when(jobExecutorService.runJobGeneric(
anyBoolean(), any(Supplier.class), anyLong(), anyBoolean(), anyInt()))
.thenAnswer(
invocation -> {
Supplier<Object> work = invocation.getArgument(1);
return work.get();
});
anyBoolean(), any(Supplier.class), anyLong(), anyBoolean(), anyInt()))
.thenAnswer(invocation -> {
Supplier<Object> work = invocation.getArgument(1);
return work.get();
});
// When
Object result = autoJobAspect.wrapWithJobExecution(joinPoint, autoJobPostMapping);
@@ -157,7 +179,7 @@ class AutoJobPostMappingIntegrationTest {
// Given
PDFFile pdfFile = new PDFFile();
pdfFile.setFileInput(mock(MultipartFile.class));
Object[] args = new Object[] {pdfFile};
Object[] args = new Object[] { pdfFile };
when(joinPoint.getArgs()).thenReturn(args);
when(request.getParameter("async")).thenReturn("true");
@@ -168,16 +190,14 @@ class AutoJobPostMappingIntegrationTest {
// Mock job executor to return a successful response
when(jobExecutorService.runJobGeneric(
anyBoolean(), any(Supplier.class), anyLong(), anyBoolean(), anyInt()))
anyBoolean(), any(Supplier.class), anyLong(), anyBoolean(), anyInt()))
.thenReturn(ResponseEntity.ok("success"));
// When
autoJobAspect.wrapWithJobExecution(joinPoint, autoJobPostMapping);
// Then
assertEquals(
"stored-file-id",
pdfFile.getFileId(),
assertEquals("stored-file-id", pdfFile.getFileId(),
"FileId should be set to the stored file id");
assertNotNull(pdfFile.getFileInput(), "FileInput should be replaced with persistent file");
@@ -1,9 +1,10 @@
package stirling.software.common.service;
import static org.junit.jupiter.api.Assertions.*;
import static org.mockito.AdditionalAnswers.*;
import static org.mockito.Mockito.*;
import static org.mockito.AdditionalAnswers.*;
import java.io.ByteArrayInputStream;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
@@ -20,11 +21,14 @@ import org.springframework.web.multipart.MultipartFile;
class FileStorageTest {
@TempDir Path tempDir;
@TempDir
Path tempDir;
@Mock private FileOrUploadService fileOrUploadService;
@Mock
private FileOrUploadService fileOrUploadService;
@InjectMocks private FileStorage fileStorage;
@InjectMocks
private FileStorage fileStorage;
private MultipartFile mockFile;
@@ -46,14 +50,11 @@ class FileStorageTest {
when(mockFile.getBytes()).thenReturn(fileContent);
// Set up mock to handle transferTo by writing the file
doAnswer(
invocation -> {
java.io.File file = invocation.getArgument(0);
Files.write(file.toPath(), fileContent);
return null;
})
.when(mockFile)
.transferTo(any(java.io.File.class));
doAnswer(invocation -> {
java.io.File file = invocation.getArgument(0);
Files.write(file.toPath(), fileContent);
return null;
}).when(mockFile).transferTo(any(java.io.File.class));
// Act
String fileId = fileStorage.storeFile(mockFile);
@@ -89,7 +90,7 @@ class FileStorageTest {
MultipartFile expectedFile = mock(MultipartFile.class);
when(fileOrUploadService.toMockMultipartFile(eq(fileId), eq(fileContent)))
.thenReturn(expectedFile);
.thenReturn(expectedFile);
// Act
MultipartFile result = fileStorage.retrieveFile(fileId);
@@ -4,9 +4,14 @@ import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertTrue;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.ArgumentMatchers.anyInt;
import static org.mockito.ArgumentMatchers.anyLong;
import static org.mockito.ArgumentMatchers.anyString;
import static org.mockito.ArgumentMatchers.eq;
import static org.mockito.Mockito.doAnswer;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.never;
import static org.mockito.Mockito.times;
import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.when;
@@ -25,9 +30,11 @@ import org.mockito.Mockito;
import org.mockito.junit.jupiter.MockitoExtension;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.test.util.ReflectionTestUtils;
import jakarta.servlet.http.HttpServletRequest;
import stirling.software.common.model.job.JobProgress;
import stirling.software.common.model.job.JobResponse;
@ExtendWith(MockitoExtension.class)
@@ -35,31 +42,36 @@ class JobExecutorServiceTest {
private JobExecutorService jobExecutorService;
@Mock private TaskManager taskManager;
@Mock
private TaskManager taskManager;
@Mock private FileStorage fileStorage;
@Mock
private FileStorage fileStorage;
@Mock private HttpServletRequest request;
@Mock
private HttpServletRequest request;
@Mock private ResourceMonitor resourceMonitor;
@Mock
private ResourceMonitor resourceMonitor;
@Mock private JobQueue jobQueue;
@Mock
private JobQueue jobQueue;
@Captor private ArgumentCaptor<String> jobIdCaptor;
@Captor
private ArgumentCaptor<String> jobIdCaptor;
@BeforeEach
void setUp() {
// Initialize the service manually with all its dependencies
jobExecutorService =
new JobExecutorService(
taskManager,
fileStorage,
request,
resourceMonitor,
jobQueue,
30000L, // asyncRequestTimeoutMs
"30m" // sessionTimeout
);
jobExecutorService = new JobExecutorService(
taskManager,
fileStorage,
request,
resourceMonitor,
jobQueue,
30000L, // asyncRequestTimeoutMs
"30m" // sessionTimeout
);
}
@Test
@@ -97,13 +109,13 @@ class JobExecutorServiceTest {
verify(taskManager).createTask(jobIdCaptor.capture());
}
@Test
void shouldHandleSyncJobError() {
// Given
Supplier<Object> work =
() -> {
throw new RuntimeException("Test error");
};
Supplier<Object> work = () -> {
throw new RuntimeException("Test error");
};
// When
ResponseEntity<?> response = jobExecutorService.runJobGeneric(false, work);
@@ -129,7 +141,8 @@ class JobExecutorServiceTest {
when(jobQueue.queueJob(anyString(), eq(80), any(), anyLong())).thenReturn(future);
// When
ResponseEntity<?> response = jobExecutorService.runJobGeneric(true, work, 5000, true, 80);
ResponseEntity<?> response = jobExecutorService.runJobGeneric(
true, work, 5000, true, 80);
// Then
assertEquals(HttpStatus.OK, response.getStatusCode());
@@ -147,9 +160,8 @@ class JobExecutorServiceTest {
long customTimeout = 60000L;
// Use reflection to access the private executeWithTimeout method
java.lang.reflect.Method executeMethod =
JobExecutorService.class.getDeclaredMethod(
"executeWithTimeout", Supplier.class, long.class);
java.lang.reflect.Method executeMethod = JobExecutorService.class
.getDeclaredMethod("executeWithTimeout", Supplier.class, long.class);
executeMethod.setAccessible(true);
// Create a spy on the JobExecutorService to verify method calls
@@ -165,21 +177,19 @@ class JobExecutorServiceTest {
@Test
void shouldHandleTimeout() throws Exception {
// Given
Supplier<Object> work =
() -> {
try {
Thread.sleep(100); // Simulate long-running job
return "test-result";
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
throw new RuntimeException(e);
}
};
Supplier<Object> work = () -> {
try {
Thread.sleep(100); // Simulate long-running job
return "test-result";
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
throw new RuntimeException(e);
}
};
// Use reflection to access the private executeWithTimeout method
java.lang.reflect.Method executeMethod =
JobExecutorService.class.getDeclaredMethod(
"executeWithTimeout", Supplier.class, long.class);
java.lang.reflect.Method executeMethod = JobExecutorService.class
.getDeclaredMethod("executeWithTimeout", Supplier.class, long.class);
executeMethod.setAccessible(true);
// When/Then
@@ -1,8 +1,10 @@
package stirling.software.common.service;
import static org.junit.jupiter.api.Assertions.*;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.ArgumentMatchers.anyInt;
import static org.mockito.Mockito.lenient;
import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.when;
import java.util.Map;
@@ -15,6 +17,7 @@ import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.Mock;
import org.mockito.junit.jupiter.MockitoExtension;
import stirling.software.common.model.job.JobProgress;
import stirling.software.common.service.ResourceMonitor.ResourceStatus;
@ExtendWith(MockitoExtension.class)
@@ -22,17 +25,16 @@ class JobQueueTest {
private JobQueue jobQueue;
@Mock private ResourceMonitor resourceMonitor;
@Mock
private ResourceMonitor resourceMonitor;
private final AtomicReference<ResourceStatus> statusRef =
new AtomicReference<>(ResourceStatus.OK);
private final AtomicReference<ResourceStatus> statusRef = new AtomicReference<>(ResourceStatus.OK);
@BeforeEach
void setUp() {
// Mark stubbing as lenient to avoid UnnecessaryStubbingException
lenient()
.when(resourceMonitor.calculateDynamicQueueCapacity(anyInt(), anyInt()))
.thenReturn(10);
lenient().when(resourceMonitor.calculateDynamicQueueCapacity(anyInt(), anyInt())).thenReturn(10);
lenient().when(resourceMonitor.getCurrentStatus()).thenReturn(statusRef);
// Initialize JobQueue with mocked ResourceMonitor
@@ -48,6 +50,7 @@ class JobQueueTest {
jobQueue.queueJob(jobId, resourceWeight, work, timeoutMs);
assertTrue(jobQueue.isJobQueued(jobId));
assertEquals(1, jobQueue.getTotalQueuedJobs());
}
@@ -1,10 +1,14 @@
package stirling.software.common.service;
import static org.assertj.core.api.Assertions.assertThat;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertTrue;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.when;
import java.lang.management.MemoryMXBean;
import java.lang.management.MemoryUsage;
import java.lang.management.OperatingSystemMXBean;
import java.time.Instant;
import java.util.concurrent.atomic.AtomicReference;
@@ -26,19 +30,20 @@ import stirling.software.common.service.ResourceMonitor.ResourceStatus;
@ExtendWith(MockitoExtension.class)
class ResourceMonitorTest {
@InjectMocks private ResourceMonitor resourceMonitor;
@InjectMocks
private ResourceMonitor resourceMonitor;
@Mock private OperatingSystemMXBean osMXBean;
@Mock
private OperatingSystemMXBean osMXBean;
@Mock private MemoryMXBean memoryMXBean;
@Mock
private MemoryMXBean memoryMXBean;
@Spy
private AtomicReference<ResourceStatus> currentStatus =
new AtomicReference<>(ResourceStatus.OK);
private AtomicReference<ResourceStatus> currentStatus = new AtomicReference<>(ResourceStatus.OK);
@Spy
private AtomicReference<ResourceMetrics> latestMetrics =
new AtomicReference<>(new ResourceMetrics());
private AtomicReference<ResourceMetrics> latestMetrics = new AtomicReference<>(new ResourceMetrics());
@BeforeEach
void setUp() {
@@ -87,26 +92,23 @@ class ResourceMonitorTest {
assertEquals(3, capacity, "With CRITICAL status, capacity should be reduced to 30%");
// Test minimum capacity enforcement
assertEquals(
minCapacity,
resourceMonitor.calculateDynamicQueueCapacity(1, minCapacity),
assertEquals(minCapacity, resourceMonitor.calculateDynamicQueueCapacity(1, minCapacity),
"Should never go below minimum capacity");
}
@ParameterizedTest
@CsvSource({
"10, OK, false", // Light job, OK status
"10, OK, false", // Light job, OK status
"10, WARNING, false", // Light job, WARNING status
"10, CRITICAL, true", // Light job, CRITICAL status
"30, OK, false", // Medium job, OK status
"30, WARNING, true", // Medium job, WARNING status
"30, OK, false", // Medium job, OK status
"30, WARNING, true", // Medium job, WARNING status
"30, CRITICAL, true", // Medium job, CRITICAL status
"80, OK, true", // Heavy job, OK status
"80, WARNING, true", // Heavy job, WARNING status
"80, CRITICAL, true" // Heavy job, CRITICAL status
"80, OK, true", // Heavy job, OK status
"80, WARNING, true", // Heavy job, WARNING status
"80, CRITICAL, true" // Heavy job, CRITICAL status
})
void shouldQueueJobBasedOnWeightAndStatus(
int weight, ResourceStatus status, boolean shouldQueue) {
void shouldQueueJobBasedOnWeightAndStatus(int weight, ResourceStatus status, boolean shouldQueue) {
// Given
currentStatus.set(status);
@@ -114,11 +116,8 @@ class ResourceMonitorTest {
boolean result = resourceMonitor.shouldQueueJob(weight);
// Then
assertEquals(
shouldQueue,
result,
String.format(
"For weight %d and status %s, shouldQueue should be %s",
assertEquals(shouldQueue, result,
String.format("For weight %d and status %s, shouldQueue should be %s",
weight, status, shouldQueue));
}
@@ -132,9 +131,7 @@ class ResourceMonitorTest {
ResourceMetrics freshMetrics = new ResourceMetrics(0.5, 0.5, 1024, 2048, 4096, now);
// When/Then
assertTrue(
staleMetrics.isStale(5000),
"Metrics from 6 seconds ago should be stale with 5s threshold");
assertTrue(staleMetrics.isStale(5000), "Metrics from 6 seconds ago should be stale with 5s threshold");
assertFalse(freshMetrics.isStale(5000), "Fresh metrics should not be stale");
}
}
@@ -6,6 +6,7 @@ import static org.mockito.Mockito.*;
import java.time.LocalDateTime;
import java.util.Map;
import java.util.UUID;
import java.util.concurrent.TimeUnit;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
@@ -21,9 +22,11 @@ import stirling.software.common.model.job.ResultFile;
class TaskManagerTest {
@Mock private FileStorage fileStorage;
@Mock
private FileStorage fileStorage;
@InjectMocks private TaskManager taskManager;
@InjectMocks
private TaskManager taskManager;
private AutoCloseable closeable;
@@ -231,20 +234,18 @@ class TaskManagerTest {
ReflectionTestUtils.setField(oldJob, "complete", true);
// Create a ResultFile and set it using the new approach
ResultFile resultFile =
ResultFile.builder()
.fileId("file-id")
.fileName("test.pdf")
.contentType("application/pdf")
.fileSize(1024L)
.build();
ResultFile resultFile = ResultFile.builder()
.fileId("file-id")
.fileName("test.pdf")
.contentType("application/pdf")
.fileSize(1024L)
.build();
ReflectionTestUtils.setField(oldJob, "resultFiles", java.util.List.of(resultFile));
when(fileStorage.deleteFile("file-id")).thenReturn(true);
// Obtain access to the private jobResults map
Map<String, JobResult> jobResultsMap =
(Map<String, JobResult>) ReflectionTestUtils.getField(taskManager, "jobResults");
Map<String, JobResult> jobResultsMap = (Map<String, JobResult>) ReflectionTestUtils.getField(taskManager, "jobResults");
// 3. Create an active job
String activeJobId = "active-job";
@@ -12,6 +12,7 @@ import java.nio.file.Path;
import java.nio.file.attribute.FileTime;
import java.util.HashSet;
import java.util.Set;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.function.Consumer;
import java.util.stream.Stream;
@@ -29,22 +30,31 @@ import stirling.software.common.model.ApplicationProperties;
import stirling.software.common.util.TempFileManager;
import stirling.software.common.util.TempFileRegistry;
/** Tests for the TempFileCleanupService, focusing on its pattern-matching and cleanup logic. */
/**
* Tests for the TempFileCleanupService, focusing on its pattern-matching and cleanup logic.
*/
public class TempFileCleanupServiceTest {
@TempDir Path tempDir;
@TempDir
Path tempDir;
@Mock private TempFileRegistry registry;
@Mock
private TempFileRegistry registry;
@Mock private TempFileManager tempFileManager;
@Mock
private TempFileManager tempFileManager;
@Mock private ApplicationProperties applicationProperties;
@Mock
private ApplicationProperties applicationProperties;
@Mock private ApplicationProperties.System system;
@Mock
private ApplicationProperties.System system;
@Mock private ApplicationProperties.TempFileManagement tempFileManagement;
@Mock
private ApplicationProperties.TempFileManagement tempFileManagement;
@InjectMocks private TempFileCleanupService cleanupService;
@InjectMocks
private TempFileCleanupService cleanupService;
private Path systemTempDir;
private Path customTempDir;
@@ -114,8 +124,7 @@ public class TempFileCleanupServiceTest {
// Files that should be preserved
Path jettyFile1 = Files.createFile(systemTempDir.resolve("jetty-123.tmp"));
Path jettyFile2 =
Files.createFile(systemTempDir.resolve("something-with-jetty-inside.tmp"));
Path jettyFile2 = Files.createFile(systemTempDir.resolve("something-with-jetty-inside.tmp"));
Path regularFile = Files.createFile(systemTempDir.resolve("important.txt"));
// Create a nested directory with temp files
@@ -134,29 +143,19 @@ public class TempFileCleanupServiceTest {
// Use MockedStatic to mock Files operations
try (MockedStatic<Files> mockedFiles = mockStatic(Files.class)) {
// Mock Files.list for each directory we'll process
mockedFiles
.when(() -> Files.list(eq(systemTempDir)))
.thenReturn(
Stream.of(
ourTempFile1,
ourTempFile2,
oldTempFile,
sysTempFile1,
jettyFile1,
jettyFile2,
regularFile,
emptyFile,
nestedDir));
mockedFiles.when(() -> Files.list(eq(systemTempDir)))
.thenReturn(Stream.of(
ourTempFile1, ourTempFile2, oldTempFile, sysTempFile1,
jettyFile1, jettyFile2, regularFile, emptyFile, nestedDir));
mockedFiles
.when(() -> Files.list(eq(customTempDir)))
mockedFiles.when(() -> Files.list(eq(customTempDir)))
.thenReturn(Stream.of(ourTempFile3, ourTempFile4, sysTempFile2, sysTempFile3));
mockedFiles
.when(() -> Files.list(eq(libreOfficeTempDir)))
mockedFiles.when(() -> Files.list(eq(libreOfficeTempDir)))
.thenReturn(Stream.of(ourTempFile5));
mockedFiles.when(() -> Files.list(eq(nestedDir))).thenReturn(Stream.of(nestedTempFile));
mockedFiles.when(() -> Files.list(eq(nestedDir)))
.thenReturn(Stream.of(nestedTempFile));
// Configure Files.isDirectory for each path
mockedFiles.when(() -> Files.isDirectory(eq(nestedDir))).thenReturn(true);
@@ -166,59 +165,48 @@ public class TempFileCleanupServiceTest {
mockedFiles.when(() -> Files.exists(any(Path.class))).thenReturn(true);
// Configure Files.getLastModifiedTime to return different times based on file names
mockedFiles
.when(() -> Files.getLastModifiedTime(any(Path.class)))
.thenAnswer(
invocation -> {
Path path = invocation.getArgument(0);
String fileName = path.getFileName().toString();
mockedFiles.when(() -> Files.getLastModifiedTime(any(Path.class)))
.thenAnswer(invocation -> {
Path path = invocation.getArgument(0);
String fileName = path.getFileName().toString();
// For files with "old" in the name, return a timestamp older than
// maxAgeMillis
if (fileName.contains("old")) {
return FileTime.fromMillis(
System.currentTimeMillis() - 5000000);
}
// For empty.tmp file, return a timestamp older than 5 minutes (for
// empty file test)
else if (fileName.equals("empty.tmp")) {
return FileTime.fromMillis(
System.currentTimeMillis() - 6 * 60 * 1000);
}
// For all other files, return a recent timestamp
else {
return FileTime.fromMillis(
System.currentTimeMillis() - 60000); // 1 minute ago
}
});
// For files with "old" in the name, return a timestamp older than maxAgeMillis
if (fileName.contains("old")) {
return FileTime.fromMillis(System.currentTimeMillis() - 5000000);
}
// For empty.tmp file, return a timestamp older than 5 minutes (for empty file test)
else if (fileName.equals("empty.tmp")) {
return FileTime.fromMillis(System.currentTimeMillis() - 6 * 60 * 1000);
}
// For all other files, return a recent timestamp
else {
return FileTime.fromMillis(System.currentTimeMillis() - 60000); // 1 minute ago
}
});
// Configure Files.size to return different sizes based on file names
mockedFiles
.when(() -> Files.size(any(Path.class)))
.thenAnswer(
invocation -> {
Path path = invocation.getArgument(0);
String fileName = path.getFileName().toString();
mockedFiles.when(() -> Files.size(any(Path.class)))
.thenAnswer(invocation -> {
Path path = invocation.getArgument(0);
String fileName = path.getFileName().toString();
// Return 0 bytes for the empty file
if (fileName.equals("empty.tmp")) {
return 0L;
}
// Return normal size for all other files
else {
return 1024L; // 1 KB
}
});
// Return 0 bytes for the empty file
if (fileName.equals("empty.tmp")) {
return 0L;
}
// Return normal size for all other files
else {
return 1024L; // 1 KB
}
});
// For deleteIfExists, track which files would be deleted
mockedFiles
.when(() -> Files.deleteIfExists(any(Path.class)))
.thenAnswer(
invocation -> {
Path path = invocation.getArgument(0);
deletedFiles.add(path);
return true;
});
mockedFiles.when(() -> Files.deleteIfExists(any(Path.class)))
.thenAnswer(invocation -> {
Path path = invocation.getArgument(0);
deletedFiles.add(path);
return true;
});
// Act - set containerMode to false for this test
invokeCleanupDirectoryStreaming(systemTempDir, false, 0, 3600000);
@@ -230,33 +218,20 @@ public class TempFileCleanupServiceTest {
assertTrue(deletedFiles.contains(emptyFile), "Empty file should be deleted");
// Regular temp files should not be deleted because they're too new
assertFalse(
deletedFiles.contains(ourTempFile1), "Recent temp file should be preserved");
assertFalse(
deletedFiles.contains(ourTempFile2), "Recent temp file should be preserved");
assertFalse(
deletedFiles.contains(ourTempFile3), "Recent temp file should be preserved");
assertFalse(
deletedFiles.contains(ourTempFile4), "Recent temp file should be preserved");
assertFalse(
deletedFiles.contains(ourTempFile5), "Recent temp file should be preserved");
assertFalse(deletedFiles.contains(ourTempFile1), "Recent temp file should be preserved");
assertFalse(deletedFiles.contains(ourTempFile2), "Recent temp file should be preserved");
assertFalse(deletedFiles.contains(ourTempFile3), "Recent temp file should be preserved");
assertFalse(deletedFiles.contains(ourTempFile4), "Recent temp file should be preserved");
assertFalse(deletedFiles.contains(ourTempFile5), "Recent temp file should be preserved");
// System temp files should not be deleted in non-container mode
assertFalse(
deletedFiles.contains(sysTempFile1),
"System temp file should be preserved in non-container mode");
assertFalse(
deletedFiles.contains(sysTempFile2),
"System temp file should be preserved in non-container mode");
assertFalse(
deletedFiles.contains(sysTempFile3),
"System temp file should be preserved in non-container mode");
assertFalse(deletedFiles.contains(sysTempFile1), "System temp file should be preserved in non-container mode");
assertFalse(deletedFiles.contains(sysTempFile2), "System temp file should be preserved in non-container mode");
assertFalse(deletedFiles.contains(sysTempFile3), "System temp file should be preserved in non-container mode");
// Jetty files and regular files should never be deleted
assertFalse(deletedFiles.contains(jettyFile1), "Jetty file should be preserved");
assertFalse(
deletedFiles.contains(jettyFile2),
"File with jetty in name should be preserved");
assertFalse(deletedFiles.contains(jettyFile2), "File with jetty in name should be preserved");
assertFalse(deletedFiles.contains(regularFile), "Regular file should be preserved");
}
}
@@ -277,8 +252,7 @@ public class TempFileCleanupServiceTest {
// Use MockedStatic to mock Files operations
try (MockedStatic<Files> mockedFiles = mockStatic(Files.class)) {
// Mock Files.list for systemTempDir
mockedFiles
.when(() -> Files.list(eq(systemTempDir)))
mockedFiles.when(() -> Files.list(eq(systemTempDir)))
.thenReturn(Stream.of(ourTempFile, sysTempFile, regularFile));
// Configure Files.isDirectory
@@ -288,37 +262,28 @@ public class TempFileCleanupServiceTest {
mockedFiles.when(() -> Files.exists(any(Path.class))).thenReturn(true);
// Configure Files.getLastModifiedTime to return recent timestamps
mockedFiles
.when(() -> Files.getLastModifiedTime(any(Path.class)))
.thenReturn(
FileTime.fromMillis(
System.currentTimeMillis() - 60000)); // 1 minute ago
mockedFiles.when(() -> Files.getLastModifiedTime(any(Path.class)))
.thenReturn(FileTime.fromMillis(System.currentTimeMillis() - 60000)); // 1 minute ago
// Configure Files.size to return normal size
mockedFiles.when(() -> Files.size(any(Path.class))).thenReturn(1024L); // 1 KB
mockedFiles.when(() -> Files.size(any(Path.class)))
.thenReturn(1024L); // 1 KB
// For deleteIfExists, track which files would be deleted
mockedFiles
.when(() -> Files.deleteIfExists(any(Path.class)))
.thenAnswer(
invocation -> {
Path path = invocation.getArgument(0);
deletedFiles.add(path);
return true;
});
mockedFiles.when(() -> Files.deleteIfExists(any(Path.class)))
.thenAnswer(invocation -> {
Path path = invocation.getArgument(0);
deletedFiles.add(path);
return true;
});
// Act - set containerMode to true and maxAgeMillis to 0 for container startup cleanup
invokeCleanupDirectoryStreaming(systemTempDir, true, 0, 0);
// Assert - In container mode, both our temp files and system temp files should be
// deleted
// Assert - In container mode, both our temp files and system temp files should be deleted
// regardless of age (when maxAgeMillis is 0)
assertTrue(
deletedFiles.contains(ourTempFile),
"Our temp file should be deleted in container mode");
assertTrue(
deletedFiles.contains(sysTempFile),
"System temp file should be deleted in container mode");
assertTrue(deletedFiles.contains(ourTempFile), "Our temp file should be deleted in container mode");
assertTrue(deletedFiles.contains(sysTempFile), "System temp file should be deleted in container mode");
assertFalse(deletedFiles.contains(regularFile), "Regular file should be preserved");
}
}
@@ -338,8 +303,7 @@ public class TempFileCleanupServiceTest {
// Use MockedStatic to mock Files operations
try (MockedStatic<Files> mockedFiles = mockStatic(Files.class)) {
// Mock Files.list for systemTempDir
mockedFiles
.when(() -> Files.list(eq(systemTempDir)))
mockedFiles.when(() -> Files.list(eq(systemTempDir)))
.thenReturn(Stream.of(emptyFile, recentEmptyFile));
// Configure Files.isDirectory
@@ -349,46 +313,39 @@ public class TempFileCleanupServiceTest {
mockedFiles.when(() -> Files.exists(any(Path.class))).thenReturn(true);
// Configure Files.getLastModifiedTime to return different times based on file names
mockedFiles
.when(() -> Files.getLastModifiedTime(any(Path.class)))
.thenAnswer(
invocation -> {
Path path = invocation.getArgument(0);
String fileName = path.getFileName().toString();
mockedFiles.when(() -> Files.getLastModifiedTime(any(Path.class)))
.thenAnswer(invocation -> {
Path path = invocation.getArgument(0);
String fileName = path.getFileName().toString();
if (fileName.equals("empty.tmp")) {
// More than 5 minutes old
return FileTime.fromMillis(
System.currentTimeMillis() - 6 * 60 * 1000);
} else {
// Less than 5 minutes old
return FileTime.fromMillis(
System.currentTimeMillis() - 2 * 60 * 1000);
}
});
if (fileName.equals("empty.tmp")) {
// More than 5 minutes old
return FileTime.fromMillis(System.currentTimeMillis() - 6 * 60 * 1000);
} else {
// Less than 5 minutes old
return FileTime.fromMillis(System.currentTimeMillis() - 2 * 60 * 1000);
}
});
// Configure Files.size to return 0 for empty files
mockedFiles.when(() -> Files.size(any(Path.class))).thenReturn(0L);
mockedFiles.when(() -> Files.size(any(Path.class)))
.thenReturn(0L);
// For deleteIfExists, track which files would be deleted
mockedFiles
.when(() -> Files.deleteIfExists(any(Path.class)))
.thenAnswer(
invocation -> {
Path path = invocation.getArgument(0);
deletedFiles.add(path);
return true;
});
mockedFiles.when(() -> Files.deleteIfExists(any(Path.class)))
.thenAnswer(invocation -> {
Path path = invocation.getArgument(0);
deletedFiles.add(path);
return true;
});
// Act
invokeCleanupDirectoryStreaming(systemTempDir, false, 0, 3600000);
// Assert
assertTrue(
deletedFiles.contains(emptyFile),
assertTrue(deletedFiles.contains(emptyFile),
"Empty file older than 5 minutes should be deleted");
assertFalse(
deletedFiles.contains(recentEmptyFile),
assertFalse(deletedFiles.contains(recentEmptyFile),
"Empty file newer than 5 minutes should not be deleted");
}
}
@@ -413,13 +370,17 @@ public class TempFileCleanupServiceTest {
// Use MockedStatic to mock Files operations
try (MockedStatic<Files> mockedFiles = mockStatic(Files.class)) {
// Mock Files.list for each directory
mockedFiles.when(() -> Files.list(eq(systemTempDir))).thenReturn(Stream.of(dir1));
mockedFiles.when(() -> Files.list(eq(systemTempDir)))
.thenReturn(Stream.of(dir1));
mockedFiles.when(() -> Files.list(eq(dir1))).thenReturn(Stream.of(tempFile1, dir2));
mockedFiles.when(() -> Files.list(eq(dir1)))
.thenReturn(Stream.of(tempFile1, dir2));
mockedFiles.when(() -> Files.list(eq(dir2))).thenReturn(Stream.of(tempFile2, dir3));
mockedFiles.when(() -> Files.list(eq(dir2)))
.thenReturn(Stream.of(tempFile2, dir3));
mockedFiles.when(() -> Files.list(eq(dir3))).thenReturn(Stream.of(tempFile3));
mockedFiles.when(() -> Files.list(eq(dir3)))
.thenReturn(Stream.of(tempFile3));
// Configure Files.isDirectory for each path
mockedFiles.when(() -> Files.isDirectory(eq(dir1))).thenReturn(true);
@@ -433,35 +394,31 @@ public class TempFileCleanupServiceTest {
mockedFiles.when(() -> Files.exists(any(Path.class))).thenReturn(true);
// Configure Files.getLastModifiedTime to return different times based on file names
mockedFiles
.when(() -> Files.getLastModifiedTime(any(Path.class)))
.thenAnswer(
invocation -> {
Path path = invocation.getArgument(0);
String fileName = path.getFileName().toString();
mockedFiles.when(() -> Files.getLastModifiedTime(any(Path.class)))
.thenAnswer(invocation -> {
Path path = invocation.getArgument(0);
String fileName = path.getFileName().toString();
if (fileName.contains("old")) {
// Old file
return FileTime.fromMillis(
System.currentTimeMillis() - 5000000);
} else {
// Recent file
return FileTime.fromMillis(System.currentTimeMillis() - 60000);
}
});
if (fileName.contains("old")) {
// Old file
return FileTime.fromMillis(System.currentTimeMillis() - 5000000);
} else {
// Recent file
return FileTime.fromMillis(System.currentTimeMillis() - 60000);
}
});
// Configure Files.size to return normal size
mockedFiles.when(() -> Files.size(any(Path.class))).thenReturn(1024L);
mockedFiles.when(() -> Files.size(any(Path.class)))
.thenReturn(1024L);
// For deleteIfExists, track which files would be deleted
mockedFiles
.when(() -> Files.deleteIfExists(any(Path.class)))
.thenAnswer(
invocation -> {
Path path = invocation.getArgument(0);
deletedFiles.add(path);
return true;
});
mockedFiles.when(() -> Files.deleteIfExists(any(Path.class)))
.thenAnswer(invocation -> {
Path path = invocation.getArgument(0);
deletedFiles.add(path);
return true;
});
// Act
invokeCleanupDirectoryStreaming(systemTempDir, false, 0, 3600000);
@@ -473,15 +430,14 @@ public class TempFileCleanupServiceTest {
// Assert
assertFalse(deletedFiles.contains(tempFile1), "Recent temp file should be preserved");
assertFalse(deletedFiles.contains(tempFile2), "Recent temp file should be preserved");
assertTrue(
deletedFiles.contains(tempFile3),
"Old temp file in nested directory should be deleted");
assertTrue(deletedFiles.contains(tempFile3), "Old temp file in nested directory should be deleted");
}
}
/** Helper method to invoke the private cleanupDirectoryStreaming method using reflection */
private void invokeCleanupDirectoryStreaming(
Path directory, boolean containerMode, int depth, long maxAgeMillis)
/**
* Helper method to invoke the private cleanupDirectoryStreaming method using reflection
*/
private void invokeCleanupDirectoryStreaming(Path directory, boolean containerMode, int depth, long maxAgeMillis)
throws IOException {
try {
// Create a consumer that tracks deleted files
@@ -489,26 +445,13 @@ public class TempFileCleanupServiceTest {
Consumer<Path> deleteCallback = path -> deleteCount.incrementAndGet();
// Get the method with updated signature
var method =
TempFileCleanupService.class.getDeclaredMethod(
"cleanupDirectoryStreaming",
Path.class,
boolean.class,
int.class,
long.class,
boolean.class,
Consumer.class);
var method = TempFileCleanupService.class.getDeclaredMethod(
"cleanupDirectoryStreaming",
Path.class, boolean.class, int.class, long.class, boolean.class, Consumer.class);
method.setAccessible(true);
// Invoke the method with appropriate parameters
method.invoke(
cleanupService,
directory,
containerMode,
depth,
maxAgeMillis,
false,
deleteCallback);
method.invoke(cleanupService, directory, containerMode, depth, maxAgeMillis, false, deleteCallback);
} catch (Exception e) {
throw new RuntimeException("Error invoking cleanupDirectoryStreaming", e);
}
@@ -1,5 +1,14 @@
package stirling.software.common.util;
import java.io.IOException;
import java.lang.reflect.Field;
import java.util.Arrays;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.mockito.MockedStatic;
import org.mockito.Mockito;
import stirling.software.common.util.ProcessExecutor.ProcessExecutorResult;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertNull;
@@ -10,18 +19,6 @@ import static org.mockito.Mockito.times;
import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.when;
import java.io.IOException;
import java.lang.reflect.Field;
import java.util.Arrays;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.mockito.MockedStatic;
import org.mockito.Mockito;
import stirling.software.common.util.ProcessExecutor.ProcessExecutorResult;
class CheckProgramInstallTest {
private MockedStatic<ProcessExecutor> mockProcessExecutor;
@@ -3,46 +3,21 @@ package stirling.software.common.util;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertTrue;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.when;
import java.util.stream.Stream;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.Arguments;
import org.junit.jupiter.params.provider.MethodSource;
import stirling.software.common.service.SsrfProtectionService;
class CustomHtmlSanitizerTest {
private CustomHtmlSanitizer customHtmlSanitizer;
@BeforeEach
void setUp() {
SsrfProtectionService mockSsrfProtectionService = mock(SsrfProtectionService.class);
stirling.software.common.model.ApplicationProperties mockApplicationProperties =
mock(stirling.software.common.model.ApplicationProperties.class);
stirling.software.common.model.ApplicationProperties.System mockSystem =
mock(stirling.software.common.model.ApplicationProperties.System.class);
// Allow all URLs by default for basic tests
when(mockSsrfProtectionService.isUrlAllowed(org.mockito.ArgumentMatchers.anyString()))
.thenReturn(true);
when(mockApplicationProperties.getSystem()).thenReturn(mockSystem);
when(mockSystem.getDisableSanitize()).thenReturn(false); // Enable sanitization for tests
customHtmlSanitizer =
new CustomHtmlSanitizer(mockSsrfProtectionService, mockApplicationProperties);
}
@ParameterizedTest
@MethodSource("provideHtmlTestCases")
void testSanitizeHtml(String inputHtml, String[] expectedContainedTags) {
// Act
String sanitizedHtml = customHtmlSanitizer.sanitize(inputHtml);
String sanitizedHtml = CustomHtmlSanitizer.sanitize(inputHtml);
// Assert
for (String tag : expectedContainedTags) {
@@ -83,7 +58,7 @@ class CustomHtmlSanitizerTest {
"<p style=\"color: blue; font-size: 16px; margin-top: 10px;\">Styled text</p>";
// Act
String sanitizedHtml = customHtmlSanitizer.sanitize(htmlWithStyles);
String sanitizedHtml = CustomHtmlSanitizer.sanitize(htmlWithStyles);
// Assert
// The OWASP HTML Sanitizer might filter some specific styles, so we only check that
@@ -100,7 +75,7 @@ class CustomHtmlSanitizerTest {
"<a href=\"https://example.com\" title=\"Example Site\">Example Link</a>";
// Act
String sanitizedHtml = customHtmlSanitizer.sanitize(htmlWithLink);
String sanitizedHtml = CustomHtmlSanitizer.sanitize(htmlWithLink);
// Assert
// The most important aspect is that the link content is preserved
@@ -122,7 +97,7 @@ class CustomHtmlSanitizerTest {
String htmlWithJsLink = "<a href=\"javascript:alert('XSS')\">Malicious Link</a>";
// Act
String sanitizedHtml = customHtmlSanitizer.sanitize(htmlWithJsLink);
String sanitizedHtml = CustomHtmlSanitizer.sanitize(htmlWithJsLink);
// Assert
assertFalse(sanitizedHtml.contains("javascript:"), "JavaScript URLs should be removed");
@@ -141,7 +116,7 @@ class CustomHtmlSanitizerTest {
+ "</table>";
// Act
String sanitizedHtml = customHtmlSanitizer.sanitize(htmlWithTable);
String sanitizedHtml = CustomHtmlSanitizer.sanitize(htmlWithTable);
// Assert
assertTrue(sanitizedHtml.contains("<table"), "Table should be preserved");
@@ -168,7 +143,7 @@ class CustomHtmlSanitizerTest {
"<img src=\"image.jpg\" alt=\"An image\" width=\"100\" height=\"100\">";
// Act
String sanitizedHtml = customHtmlSanitizer.sanitize(htmlWithImage);
String sanitizedHtml = CustomHtmlSanitizer.sanitize(htmlWithImage);
// Assert
assertTrue(sanitizedHtml.contains("<img"), "Image tag should be preserved");
@@ -185,7 +160,7 @@ class CustomHtmlSanitizerTest {
"<img src=\"data:image/svg+xml;base64,PHN2ZyBvbmxvYWQ9ImFsZXJ0KDEpIj48L3N2Zz4=\" alt=\"SVG with XSS\">";
// Act
String sanitizedHtml = customHtmlSanitizer.sanitize(htmlWithDataUrlImage);
String sanitizedHtml = CustomHtmlSanitizer.sanitize(htmlWithDataUrlImage);
// Assert
assertFalse(
@@ -200,7 +175,7 @@ class CustomHtmlSanitizerTest {
"<a href=\"#\" onclick=\"alert('XSS')\" onmouseover=\"alert('XSS')\">Click me</a>";
// Act
String sanitizedHtml = customHtmlSanitizer.sanitize(htmlWithJsEvent);
String sanitizedHtml = CustomHtmlSanitizer.sanitize(htmlWithJsEvent);
// Assert
assertFalse(
@@ -217,7 +192,7 @@ class CustomHtmlSanitizerTest {
String htmlWithScript = "<p>Safe content</p><script>alert('XSS');</script>";
// Act
String sanitizedHtml = customHtmlSanitizer.sanitize(htmlWithScript);
String sanitizedHtml = CustomHtmlSanitizer.sanitize(htmlWithScript);
// Assert
assertFalse(sanitizedHtml.contains("<script>"), "Script tags should be removed");
@@ -231,7 +206,7 @@ class CustomHtmlSanitizerTest {
String htmlWithNoscript = "<p>Safe content</p><noscript>JavaScript is disabled</noscript>";
// Act
String sanitizedHtml = customHtmlSanitizer.sanitize(htmlWithNoscript);
String sanitizedHtml = CustomHtmlSanitizer.sanitize(htmlWithNoscript);
// Assert
assertFalse(sanitizedHtml.contains("<noscript>"), "Noscript tags should be removed");
@@ -245,7 +220,7 @@ class CustomHtmlSanitizerTest {
String htmlWithIframe = "<p>Safe content</p><iframe src=\"https://example.com\"></iframe>";
// Act
String sanitizedHtml = customHtmlSanitizer.sanitize(htmlWithIframe);
String sanitizedHtml = CustomHtmlSanitizer.sanitize(htmlWithIframe);
// Assert
assertFalse(sanitizedHtml.contains("<iframe"), "Iframe tags should be removed");
@@ -262,7 +237,7 @@ class CustomHtmlSanitizerTest {
+ "<embed src=\"embed.swf\" type=\"application/x-shockwave-flash\">";
// Act
String sanitizedHtml = customHtmlSanitizer.sanitize(htmlWithObjects);
String sanitizedHtml = CustomHtmlSanitizer.sanitize(htmlWithObjects);
// Assert
assertFalse(sanitizedHtml.contains("<object"), "Object tags should be removed");
@@ -281,7 +256,7 @@ class CustomHtmlSanitizerTest {
+ "<link rel=\"stylesheet\" href=\"evil.css\">";
// Act
String sanitizedHtml = customHtmlSanitizer.sanitize(htmlWithMetaTags);
String sanitizedHtml = CustomHtmlSanitizer.sanitize(htmlWithMetaTags);
// Assert
assertFalse(sanitizedHtml.contains("<meta"), "Meta tags should be removed");
@@ -308,7 +283,7 @@ class CustomHtmlSanitizerTest {
+ "</div>";
// Act
String sanitizedHtml = customHtmlSanitizer.sanitize(complexHtml);
String sanitizedHtml = CustomHtmlSanitizer.sanitize(complexHtml);
// Assert
assertTrue(sanitizedHtml.contains("<div"), "Div should be preserved");
@@ -339,7 +314,7 @@ class CustomHtmlSanitizerTest {
@Test
void testSanitizeHandlesEmpty() {
// Act
String sanitizedHtml = customHtmlSanitizer.sanitize("");
String sanitizedHtml = CustomHtmlSanitizer.sanitize("");
// Assert
assertEquals("", sanitizedHtml, "Empty input should result in empty string");
@@ -348,7 +323,7 @@ class CustomHtmlSanitizerTest {
@Test
void testSanitizeHandlesNull() {
// Act
String sanitizedHtml = customHtmlSanitizer.sanitize(null);
String sanitizedHtml = CustomHtmlSanitizer.sanitize(null);
// Assert
assertEquals("", sanitizedHtml, "Null input should result in empty string");
File diff suppressed because it is too large Load Diff
@@ -19,7 +19,6 @@ import org.junit.jupiter.api.io.TempDir;
import org.mockito.Mock;
import org.mockito.Mockito;
import org.mockito.junit.jupiter.MockitoExtension;
import stirling.software.common.configuration.RuntimePathConfig;
@ExtendWith(MockitoExtension.class)
@@ -3,40 +3,19 @@ package stirling.software.common.util;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.mockito.ArgumentMatchers.anyString;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.when;
import static org.mockito.ArgumentMatchers.anyString;
import java.io.File;
import java.io.IOException;
import java.nio.file.Files;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import stirling.software.common.model.api.converters.HTMLToPdfRequest;
import stirling.software.common.service.SsrfProtectionService;
public class FileToPdfTest {
private CustomHtmlSanitizer customHtmlSanitizer;
@BeforeEach
void setUp() {
SsrfProtectionService mockSsrfProtectionService = mock(SsrfProtectionService.class);
stirling.software.common.model.ApplicationProperties mockApplicationProperties =
mock(stirling.software.common.model.ApplicationProperties.class);
stirling.software.common.model.ApplicationProperties.System mockSystem =
mock(stirling.software.common.model.ApplicationProperties.System.class);
when(mockSsrfProtectionService.isUrlAllowed(org.mockito.ArgumentMatchers.anyString()))
.thenReturn(true);
when(mockApplicationProperties.getSystem()).thenReturn(mockSystem);
when(mockSystem.getDisableSanitize()).thenReturn(false);
customHtmlSanitizer =
new CustomHtmlSanitizer(mockSsrfProtectionService, mockApplicationProperties);
}
/**
* Test the HTML to PDF conversion. This test expects an IOException when an empty HTML input is
* provided.
@@ -46,13 +25,14 @@ public class FileToPdfTest {
HTMLToPdfRequest request = new HTMLToPdfRequest();
byte[] fileBytes = new byte[0]; // Sample file bytes (empty input)
String fileName = "test.html"; // Sample file name indicating an HTML file
boolean disableSanitize = false; // Flag to control sanitization
TempFileManager tempFileManager = mock(TempFileManager.class); // Mock TempFileManager
// Mock the temp file creation to return real temp files
try {
when(tempFileManager.createTempFile(anyString()))
.thenReturn(Files.createTempFile("test", ".pdf").toFile())
.thenReturn(Files.createTempFile("test", ".html").toFile());
.thenReturn(File.createTempFile("test", ".pdf"))
.thenReturn(File.createTempFile("test", ".html"));
} catch (IOException e) {
throw new RuntimeException(e);
}
@@ -63,12 +43,7 @@ public class FileToPdfTest {
Exception.class,
() ->
FileToPdf.convertHtmlToPdf(
"/path/",
request,
fileBytes,
fileName,
tempFileManager,
customHtmlSanitizer));
"/path/", request, fileBytes, fileName, disableSanitize, tempFileManager));
assertNotNull(thrown);
}
@@ -1,23 +1,21 @@
package stirling.software.common.util;
import static org.junit.jupiter.api.Assertions.*;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.when;
import java.util.List;
import java.util.stream.Stream;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.Arguments;
import org.junit.jupiter.params.provider.MethodSource;
import org.mockito.junit.jupiter.MockitoExtension;
import stirling.software.common.model.enumeration.UsernameAttribute;
import stirling.software.common.model.oauth2.GitHubProvider;
import stirling.software.common.model.oauth2.GoogleProvider;
import stirling.software.common.model.oauth2.Provider;
import static org.junit.jupiter.api.Assertions.*;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.when;
@ExtendWith(MockitoExtension.class)
class ProviderUtilsTest {
@@ -42,7 +40,7 @@ class ProviderUtilsTest {
public static Stream<Arguments> providerParams() {
Provider generic = null;
var google =
new GoogleProvider(null, "clientSecret", List.of("scope"), UsernameAttribute.EMAIL);
new GoogleProvider(null, "clientSecret", List.of("scope"), UsernameAttribute.EMAIL);
var github = new GitHubProvider("clientId", "", List.of("scope"), UsernameAttribute.LOGIN);
return Stream.of(Arguments.of(generic), Arguments.of(google), Arguments.of(github));
@@ -42,6 +42,7 @@ class SpringContextHolderTest {
verify(mockApplicationContext).getBean(TestBean.class);
}
@Test
void testGetBean_ApplicationContextNotSet() {
// Don't set application context
@@ -57,8 +58,7 @@ class SpringContextHolderTest {
void testGetBean_BeanNotFound() {
// Arrange
contextHolder.setApplicationContext(mockApplicationContext);
when(mockApplicationContext.getBean(TestBean.class))
.thenThrow(new org.springframework.beans.BeansException("Bean not found") {});
when(mockApplicationContext.getBean(TestBean.class)).thenThrow(new org.springframework.beans.BeansException("Bean not found") {});
// Act
TestBean result = SpringContextHolder.getBean(TestBean.class);
@@ -68,5 +68,6 @@ class SpringContextHolderTest {
}
// Simple test class
private static class TestBean {}
private static class TestBean {
}
}
@@ -1,12 +1,10 @@
package stirling.software.common.util.misc;
import static org.junit.jupiter.api.Assertions.assertArrayEquals;
import static org.junit.jupiter.api.Assertions.assertNull;
import org.junit.jupiter.api.Test;
import stirling.software.common.model.api.misc.HighContrastColorCombination;
import stirling.software.common.model.api.misc.ReplaceAndInvert;
import static org.junit.jupiter.api.Assertions.assertArrayEquals;
import static org.junit.jupiter.api.Assertions.assertNull;
class HighContrastColorReplaceDeciderTest {
@@ -26,7 +26,6 @@ import org.junit.jupiter.api.Test;
import org.springframework.core.io.InputStreamResource;
import org.springframework.mock.web.MockMultipartFile;
import org.springframework.web.multipart.MultipartFile;
import stirling.software.common.model.api.misc.ReplaceAndInvert;
class InvertFullColorStrategyTest {
@@ -9,7 +9,6 @@ import org.junit.jupiter.api.Test;
import org.springframework.core.io.InputStreamResource;
import org.springframework.mock.web.MockMultipartFile;
import org.springframework.web.multipart.MultipartFile;
import stirling.software.common.model.api.misc.ReplaceAndInvert;
class ReplaceAndInvertColorStrategyTest {
@@ -1,17 +1,14 @@
package stirling.software.common.util.propertyeditor;
import java.util.List;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import stirling.software.common.model.api.security.RedactionArea;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertTrue;
import java.util.List;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import stirling.software.common.model.api.security.RedactionArea;
class StringToArrayListPropertyEditorTest {
private StringToArrayListPropertyEditor editor;
+2 -1
View File
@@ -16,7 +16,8 @@ local.properties
version.properties
#### Stirling-PDF Files ###
pipeline/*
pipeline/watchedFolders/
pipeline/finishedFolders/
customFiles/
configs/
watchedFolders/
+3 -15
View File
@@ -14,7 +14,7 @@ configurations {
spotless {
java {
target 'src/**/java/**/*.java'
target sourceSets.main.allJava
googleJavaFormat(googleJavaFormatVersion).aosp().reorderImports(false)
importOrder("java", "javax", "org", "com", "net", "io", "jakarta", "lombok", "me", "stirling")
@@ -23,18 +23,6 @@ spotless {
leadingTabsToSpaces()
endWithNewline()
}
yaml {
target '**/*.yml', '**/*.yaml'
trimTrailingWhitespace()
leadingTabsToSpaces()
endWithNewline()
}
format 'gradle', {
target '**/gradle/*.gradle', '**/*.gradle'
trimTrailingWhitespace()
leadingTabsToSpaces()
endWithNewline()
}
}
dependencies {
@@ -55,7 +43,7 @@ dependencies {
implementation project(':common')
implementation 'org.springframework.boot:spring-boot-starter-jetty'
implementation 'com.posthog.java:posthog:1.2.0'
implementation 'commons-io:commons-io:2.20.0'
implementation 'commons-io:commons-io:2.19.0'
implementation "org.bouncycastle:bcprov-jdk18on:$bouncycastleVersion"
implementation "org.bouncycastle:bcpkix-jdk18on:$bouncycastleVersion"
implementation 'io.micrometer:micrometer-core:1.15.2'
@@ -74,7 +62,7 @@ dependencies {
exclude group: 'com.google.code.gson', module: 'gson'
}
implementation 'org.apache.pdfbox:jbig2-imageio:3.0.4'
implementation 'com.opencsv:opencsv:5.12.0' // https://mvnrepository.com/artifact/com.opencsv/opencsv
implementation 'com.opencsv:opencsv:5.11.2' // https://mvnrepository.com/artifact/com.opencsv/opencsv
// Batik
implementation 'org.apache.xmlgraphics:batik-all:1.19'
@@ -36,15 +36,14 @@ public class CleanUrlInterceptor implements HandlerInterceptor {
public boolean preHandle(
HttpServletRequest request, HttpServletResponse response, Object handler)
throws Exception {
String requestURI = request.getRequestURI();
// Skip URL cleaning for API endpoints - they need their own parameter handling
if (requestURI.contains("/api/")) {
return true;
}
String queryString = request.getQueryString();
if (queryString != null && !queryString.isEmpty()) {
String requestURI = request.getRequestURI();
if (requestURI.contains("/api/")) {
return true;
}
Map<String, String> allowedParameters = new HashMap<>();
// Keep only the allowed parameters
@@ -421,6 +421,7 @@ public class EndpointConfiguration {
// file-to-pdf has multiple implementations
addEndpointAlternative("file-to-pdf", "LibreOffice");
addEndpointAlternative("file-to-pdf", "Python");
addEndpointAlternative("file-to-pdf", "Unoconvert");
// pdf-to-html and pdf-to-markdown can use either LibreOffice or Pdftohtml
@@ -35,7 +35,6 @@ public class InitialSetup {
initEnableCSRFSecurity();
initLegalUrls();
initSetAppVersion();
GeneralUtils.extractPipeline();
}
public void initUUIDKey() throws IOException {
@@ -10,6 +10,7 @@ import io.swagger.v3.oas.models.info.Info;
import io.swagger.v3.oas.models.info.License;
import io.swagger.v3.oas.models.security.SecurityRequirement;
import io.swagger.v3.oas.models.security.SecurityScheme;
import io.swagger.v3.oas.models.servers.Server;
import lombok.RequiredArgsConstructor;
@@ -50,17 +51,26 @@ public class OpenApiConfig {
.url("https://www.stirlingpdf.com")
.email("contact@stirlingpdf.com"))
.description(DEFAULT_DESCRIPTION);
OpenAPI openAPI = new OpenAPI().info(info);
// Add server configuration from environment variable
String swaggerServerUrl = System.getenv("SWAGGER_SERVER_URL");
if (swaggerServerUrl != null && !swaggerServerUrl.trim().isEmpty()) {
Server server = new Server().url(swaggerServerUrl).description("API Server");
openAPI.addServersItem(server);
}
if (!applicationProperties.getSecurity().getEnableLogin()) {
return new OpenAPI().components(new Components()).info(info);
return openAPI.components(new Components());
} else {
SecurityScheme apiKeyScheme =
new SecurityScheme()
.type(SecurityScheme.Type.APIKEY)
.in(SecurityScheme.In.HEADER)
.name("X-API-KEY");
return new OpenAPI()
return openAPI
.components(new Components().addSecuritySchemes("apiKey", apiKeyScheme))
.info(info)
.addSecurityItem(new SecurityRequirement().addList("apiKey"));
}
}
@@ -1,5 +1,7 @@
package stirling.software.SPDF.controller.api;
import stirling.software.common.annotations.AutoJobPostMapping;
import java.io.IOException;
import java.util.*;
@@ -29,7 +31,7 @@ public class AnalysisController {
private final CustomPDFDocumentFactory pdfDocumentFactory;
@PostMapping(value = "/page-count", consumes = "multipart/form-data")
@AutoJobPostMapping(value = "/page-count", consumes = "multipart/form-data")
@Operation(
summary = "Get PDF page count",
description = "Returns total number of pages in PDF. Input:PDF Output:JSON Type:SISO")
@@ -39,7 +41,7 @@ public class AnalysisController {
}
}
@PostMapping(value = "/basic-info", consumes = "multipart/form-data")
@AutoJobPostMapping(value = "/basic-info", consumes = "multipart/form-data")
@Operation(
summary = "Get basic PDF information",
description = "Returns page count, version, file size. Input:PDF Output:JSON Type:SISO")
@@ -53,7 +55,7 @@ public class AnalysisController {
}
}
@PostMapping(value = "/document-properties", consumes = "multipart/form-data")
@AutoJobPostMapping(value = "/document-properties", consumes = "multipart/form-data")
@Operation(
summary = "Get PDF document properties",
description = "Returns title, author, subject, etc. Input:PDF Output:JSON Type:SISO")
@@ -76,7 +78,7 @@ public class AnalysisController {
}
}
@PostMapping(value = "/page-dimensions", consumes = "multipart/form-data")
@AutoJobPostMapping(value = "/page-dimensions", consumes = "multipart/form-data")
@Operation(
summary = "Get page dimensions for all pages",
description = "Returns width and height of each page. Input:PDF Output:JSON Type:SISO")
@@ -96,7 +98,7 @@ public class AnalysisController {
}
}
@PostMapping(value = "/form-fields", consumes = "multipart/form-data")
@AutoJobPostMapping(value = "/form-fields", consumes = "multipart/form-data")
@Operation(
summary = "Get form field information",
description =
@@ -119,7 +121,7 @@ public class AnalysisController {
}
}
@PostMapping(value = "/annotation-info", consumes = "multipart/form-data")
@AutoJobPostMapping(value = "/annotation-info", consumes = "multipart/form-data")
@Operation(
summary = "Get annotation information",
description = "Returns count and types of annotations. Input:PDF Output:JSON Type:SISO")
@@ -143,7 +145,7 @@ public class AnalysisController {
}
}
@PostMapping(value = "/font-info", consumes = "multipart/form-data")
@AutoJobPostMapping(value = "/font-info", consumes = "multipart/form-data")
@Operation(
summary = "Get font information",
description =
@@ -165,7 +167,7 @@ public class AnalysisController {
}
}
@PostMapping(value = "/security-info", consumes = "multipart/form-data")
@AutoJobPostMapping(value = "/security-info", consumes = "multipart/form-data")
@Operation(
summary = "Get security information",
description =
@@ -1,5 +1,7 @@
package stirling.software.SPDF.controller.api;
import stirling.software.common.annotations.AutoJobPostMapping;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
@@ -33,7 +35,7 @@ public class CropController {
private final CustomPDFDocumentFactory pdfDocumentFactory;
@PostMapping(value = "/crop", consumes = "multipart/form-data")
@AutoJobPostMapping(value = "/crop", consumes = "multipart/form-data")
@Operation(
summary = "Crops a PDF document",
description =
@@ -1,5 +1,7 @@
package stirling.software.SPDF.controller.api;
import stirling.software.common.annotations.AutoJobPostMapping;
import java.io.ByteArrayOutputStream;
import java.util.ArrayList;
import java.util.HashMap;
@@ -44,7 +46,7 @@ public class EditTableOfContentsController {
private final CustomPDFDocumentFactory pdfDocumentFactory;
private final ObjectMapper objectMapper;
@PostMapping(value = "/extract-bookmarks", consumes = "multipart/form-data")
@AutoJobPostMapping(value = "/extract-bookmarks", consumes = "multipart/form-data")
@Operation(
summary = "Extract PDF Bookmarks",
description = "Extracts bookmarks/table of contents from a PDF document as JSON.")
@@ -152,7 +154,7 @@ public class EditTableOfContentsController {
return bookmark;
}
@PostMapping(value = "/edit-table-of-contents", consumes = "multipart/form-data")
@AutoJobPostMapping(value = "/edit-table-of-contents", consumes = "multipart/form-data")
@Operation(
summary = "Edit Table of Contents",
description = "Add or edit bookmarks/table of contents in a PDF document.")
@@ -1,5 +1,7 @@
package stirling.software.SPDF.controller.api;
import stirling.software.common.annotations.AutoJobPostMapping;
import java.io.ByteArrayOutputStream;
import java.io.File;
import java.io.IOException;
@@ -154,7 +156,7 @@ public class MergeController {
}
}
@PostMapping(consumes = "multipart/form-data", value = "/merge-pdfs")
@AutoJobPostMapping(consumes = "multipart/form-data", value = "/merge-pdfs")
@Operation(
summary = "Merge multiple PDF files into one",
description =
@@ -1,5 +1,7 @@
package stirling.software.SPDF.controller.api;
import stirling.software.common.annotations.AutoJobPostMapping;
import java.awt.*;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
@@ -36,7 +38,7 @@ public class MultiPageLayoutController {
private final CustomPDFDocumentFactory pdfDocumentFactory;
@PostMapping(value = "/multi-page-layout", consumes = "multipart/form-data")
@AutoJobPostMapping(value = "/multi-page-layout", consumes = "multipart/form-data")
@Operation(
summary = "Merge multiple pages of a PDF document into a single page",
description =
@@ -1,5 +1,7 @@
package stirling.software.SPDF.controller.api;
import stirling.software.common.annotations.AutoJobPostMapping;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
@@ -46,7 +48,7 @@ public class PdfImageRemovalController {
* content type and filename.
* @throws IOException If an error occurs while processing the PDF file.
*/
@PostMapping(consumes = "multipart/form-data", value = "/remove-image-pdf")
@AutoJobPostMapping(consumes = "multipart/form-data", value = "/remove-image-pdf")
@Operation(
summary = "Remove images from file to reduce the file size.",
description =
@@ -1,5 +1,7 @@
package stirling.software.SPDF.controller.api;
import stirling.software.common.annotations.AutoJobPostMapping;
import java.io.ByteArrayOutputStream;
import java.io.File;
import java.io.IOException;
@@ -39,7 +41,7 @@ public class PdfOverlayController {
private final CustomPDFDocumentFactory pdfDocumentFactory;
@PostMapping(value = "/overlay-pdfs", consumes = "multipart/form-data")
@AutoJobPostMapping(value = "/overlay-pdfs", consumes = "multipart/form-data")
@Operation(
summary = "Overlay PDF files in various modes",
description =
@@ -1,5 +1,7 @@
package stirling.software.SPDF.controller.api;
import stirling.software.common.annotations.AutoJobPostMapping;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Collections;
@@ -38,7 +40,7 @@ public class RearrangePagesPDFController {
private final CustomPDFDocumentFactory pdfDocumentFactory;
@PostMapping(consumes = "multipart/form-data", value = "/remove-pages")
@AutoJobPostMapping(consumes = "multipart/form-data", value = "/remove-pages")
@Operation(
summary = "Remove pages from a PDF file",
description =
@@ -237,7 +239,7 @@ public class RearrangePagesPDFController {
}
}
@PostMapping(consumes = "multipart/form-data", value = "/rearrange-pages")
@AutoJobPostMapping(consumes = "multipart/form-data", value = "/rearrange-pages")
@Operation(
summary = "Rearrange pages in a PDF file",
description =
@@ -1,5 +1,7 @@
package stirling.software.SPDF.controller.api;
import stirling.software.common.annotations.AutoJobPostMapping;
import java.io.IOException;
import org.apache.pdfbox.pdmodel.PDDocument;
@@ -31,7 +33,7 @@ public class RotationController {
private final CustomPDFDocumentFactory pdfDocumentFactory;
@PostMapping(consumes = "multipart/form-data", value = "/rotate-pdf")
@AutoJobPostMapping(consumes = "multipart/form-data", value = "/rotate-pdf")
@Operation(
summary = "Rotate a PDF file",
description =
@@ -1,5 +1,7 @@
package stirling.software.SPDF.controller.api;
import stirling.software.common.annotations.AutoJobPostMapping;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.util.HashMap;
@@ -38,7 +40,7 @@ public class ScalePagesController {
private final CustomPDFDocumentFactory pdfDocumentFactory;
@PostMapping(value = "/scale-pages", consumes = "multipart/form-data")
@AutoJobPostMapping(value = "/scale-pages", consumes = "multipart/form-data")
@Operation(
summary = "Change the size of a PDF page/document",
description =
@@ -1,5 +1,7 @@
package stirling.software.SPDF.controller.api;
import stirling.software.common.annotations.AutoJobPostMapping;
import java.io.IOException;
import java.util.Map;
@@ -31,7 +33,7 @@ public class SettingsController {
private final ApplicationProperties applicationProperties;
private final EndpointConfiguration endpointConfiguration;
@PostMapping("/update-enable-analytics")
@AutoJobPostMapping("/update-enable-analytics")
@Hidden
public ResponseEntity<String> updateApiKey(@RequestBody Boolean enabled) throws IOException {
if (applicationProperties.getSystem().getEnableAnalytics() != null) {
@@ -1,5 +1,7 @@
package stirling.software.SPDF.controller.api;
import stirling.software.common.annotations.AutoJobPostMapping;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.nio.file.Files;
@@ -41,7 +43,7 @@ public class SplitPDFController {
private final CustomPDFDocumentFactory pdfDocumentFactory;
@PostMapping(consumes = "multipart/form-data", value = "/split-pages")
@AutoJobPostMapping(consumes = "multipart/form-data", value = "/split-pages")
@Operation(
summary = "Split a PDF file into separate documents",
description =
@@ -1,5 +1,7 @@
package stirling.software.SPDF.controller.api;
import stirling.software.common.annotations.AutoJobPostMapping;
import java.io.ByteArrayOutputStream;
import java.nio.file.Files;
import java.nio.file.Path;
@@ -117,7 +119,7 @@ public class SplitPdfByChaptersController {
return bookmarks;
}
@PostMapping(value = "/split-pdf-by-chapters", consumes = "multipart/form-data")
@AutoJobPostMapping(value = "/split-pdf-by-chapters", consumes = "multipart/form-data")
@Operation(
summary = "Split PDFs by Chapters",
description = "Splits a PDF into chapters and returns a ZIP file.")
@@ -1,5 +1,7 @@
package stirling.software.SPDF.controller.api;
import stirling.software.common.annotations.AutoJobPostMapping;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.nio.file.Files;
@@ -43,7 +45,7 @@ public class SplitPdfBySectionsController {
private final CustomPDFDocumentFactory pdfDocumentFactory;
@PostMapping(value = "/split-pdf-by-sections", consumes = "multipart/form-data")
@AutoJobPostMapping(value = "/split-pdf-by-sections", consumes = "multipart/form-data")
@Operation(
summary = "Split PDF pages into smaller sections",
description =
@@ -1,5 +1,7 @@
package stirling.software.SPDF.controller.api;
import stirling.software.common.annotations.AutoJobPostMapping;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.nio.file.Files;
@@ -39,7 +41,7 @@ public class SplitPdfBySizeController {
private final CustomPDFDocumentFactory pdfDocumentFactory;
@PostMapping(value = "/split-by-size-or-count", consumes = "multipart/form-data")
@AutoJobPostMapping(value = "/split-by-size-or-count", consumes = "multipart/form-data")
@Operation(
summary = "Auto split PDF pages into separate documents based on size or count",
description =
@@ -1,5 +1,7 @@
package stirling.software.SPDF.controller.api;
import stirling.software.common.annotations.AutoJobPostMapping;
import java.awt.geom.AffineTransform;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
@@ -33,7 +35,7 @@ public class ToSinglePageController {
private final CustomPDFDocumentFactory pdfDocumentFactory;
@PostMapping(consumes = "multipart/form-data", value = "/pdf-to-single-page")
@AutoJobPostMapping(consumes = "multipart/form-data", value = "/pdf-to-single-page")
@Operation(
summary = "Convert a multi-page PDF into a single long page PDF",
description =
@@ -0,0 +1,301 @@
package stirling.software.SPDF.controller.api;
import java.io.IOException;
import java.io.InputStream;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.*;
import java.util.stream.Stream;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.core.io.ClassPathResource;
import org.springframework.core.io.Resource;
import org.springframework.core.io.ResourceLoader;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import com.fasterxml.jackson.core.type.TypeReference;
import com.fasterxml.jackson.databind.ObjectMapper;
import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.tags.Tag;
import lombok.Data;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import stirling.software.SPDF.model.Dependency;
import stirling.software.SPDF.model.SignatureFile;
import stirling.software.SPDF.service.SignatureService;
import stirling.software.common.configuration.InstallationPathConfig;
import stirling.software.common.configuration.RuntimePathConfig;
import stirling.software.common.model.ApplicationProperties;
import stirling.software.common.service.UserServiceInterface;
import stirling.software.common.util.ExceptionUtils;
import stirling.software.common.util.GeneralUtils;
@Slf4j
@RestController
@RequestMapping("/api/v1/ui-data")
@Tag(name = "UI Data", description = "APIs for React UI data")
public class UIDataController {
private final ApplicationProperties applicationProperties;
private final SignatureService signatureService;
private final UserServiceInterface userService;
private final ResourceLoader resourceLoader;
private final RuntimePathConfig runtimePathConfig;
public UIDataController(
ApplicationProperties applicationProperties,
SignatureService signatureService,
@Autowired(required = false) UserServiceInterface userService,
ResourceLoader resourceLoader,
RuntimePathConfig runtimePathConfig) {
this.applicationProperties = applicationProperties;
this.signatureService = signatureService;
this.userService = userService;
this.resourceLoader = resourceLoader;
this.runtimePathConfig = runtimePathConfig;
}
@GetMapping("/home")
@Operation(summary = "Get home page data")
public ResponseEntity<HomeData> getHomeData() {
String showSurvey = System.getenv("SHOW_SURVEY");
boolean showSurveyValue = showSurvey == null || "true".equalsIgnoreCase(showSurvey);
HomeData data = new HomeData();
data.setShowSurveyFromDocker(showSurveyValue);
return ResponseEntity.ok(data);
}
@GetMapping("/licenses")
@Operation(summary = "Get third-party licenses data")
public ResponseEntity<LicensesData> getLicensesData() {
LicensesData data = new LicensesData();
Resource resource = new ClassPathResource("static/3rdPartyLicenses.json");
try {
InputStream is = resource.getInputStream();
String json = new String(is.readAllBytes(), StandardCharsets.UTF_8);
ObjectMapper mapper = new ObjectMapper();
Map<String, List<Dependency>> licenseData =
mapper.readValue(json, new TypeReference<>() {});
data.setDependencies(licenseData.get("dependencies"));
} catch (IOException e) {
log.error("Failed to load licenses data", e);
data.setDependencies(Collections.emptyList());
}
return ResponseEntity.ok(data);
}
@GetMapping("/pipeline")
@Operation(summary = "Get pipeline configuration data")
public ResponseEntity<PipelineData> getPipelineData() {
PipelineData data = new PipelineData();
List<String> pipelineConfigs = new ArrayList<>();
List<Map<String, String>> pipelineConfigsWithNames = new ArrayList<>();
if (new java.io.File(runtimePathConfig.getPipelineDefaultWebUiConfigs()).exists()) {
try (Stream<Path> paths =
Files.walk(Paths.get(runtimePathConfig.getPipelineDefaultWebUiConfigs()))) {
List<Path> jsonFiles =
paths.filter(Files::isRegularFile)
.filter(p -> p.toString().endsWith(".json"))
.toList();
for (Path jsonFile : jsonFiles) {
String content = Files.readString(jsonFile, StandardCharsets.UTF_8);
pipelineConfigs.add(content);
}
for (String config : pipelineConfigs) {
Map<String, Object> jsonContent =
new ObjectMapper()
.readValue(config, new TypeReference<Map<String, Object>>() {});
String name = (String) jsonContent.get("name");
if (name == null || name.length() < 1) {
String filename =
jsonFiles
.get(pipelineConfigs.indexOf(config))
.getFileName()
.toString();
name = filename.substring(0, filename.lastIndexOf('.'));
}
Map<String, String> configWithName = new HashMap<>();
configWithName.put("json", config);
configWithName.put("name", name);
pipelineConfigsWithNames.add(configWithName);
}
} catch (IOException e) {
log.error("Failed to load pipeline configs", e);
}
}
if (pipelineConfigsWithNames.isEmpty()) {
Map<String, String> configWithName = new HashMap<>();
configWithName.put("json", "");
configWithName.put("name", "No preloaded configs found");
pipelineConfigsWithNames.add(configWithName);
}
data.setPipelineConfigsWithNames(pipelineConfigsWithNames);
data.setPipelineConfigs(pipelineConfigs);
return ResponseEntity.ok(data);
}
@GetMapping("/sign")
@Operation(summary = "Get signature form data")
public ResponseEntity<SignData> getSignData() {
String username = "";
if (userService != null) {
username = userService.getCurrentUsername();
}
List<SignatureFile> signatures = signatureService.getAvailableSignatures(username);
List<FontResource> fonts = getFontNames();
SignData data = new SignData();
data.setSignatures(signatures);
data.setFonts(fonts);
return ResponseEntity.ok(data);
}
@GetMapping("/ocr-pdf")
@Operation(summary = "Get OCR PDF data")
public ResponseEntity<OcrData> getOcrPdfData() {
List<String> languages = getAvailableTesseractLanguages();
OcrData data = new OcrData();
data.setLanguages(languages);
return ResponseEntity.ok(data);
}
private List<String> getAvailableTesseractLanguages() {
String tessdataDir = applicationProperties.getSystem().getTessdataDir();
java.io.File[] files = new java.io.File(tessdataDir).listFiles();
if (files == null) {
return Collections.emptyList();
}
return Arrays.stream(files)
.filter(file -> file.getName().endsWith(".traineddata"))
.map(file -> file.getName().replace(".traineddata", ""))
.filter(lang -> !"osd".equalsIgnoreCase(lang))
.sorted()
.toList();
}
private List<FontResource> getFontNames() {
List<FontResource> fontNames = new ArrayList<>();
fontNames.addAll(getFontNamesFromLocation("classpath:static/fonts/*.woff2"));
fontNames.addAll(
getFontNamesFromLocation(
"file:"
+ InstallationPathConfig.getStaticPath()
+ "fonts"
+ java.io.File.separator
+ "*"));
return fontNames;
}
private List<FontResource> getFontNamesFromLocation(String locationPattern) {
try {
Resource[] resources =
GeneralUtils.getResourcesFromLocationPattern(locationPattern, resourceLoader);
return Arrays.stream(resources)
.map(
resource -> {
try {
String filename = resource.getFilename();
if (filename != null) {
int lastDotIndex = filename.lastIndexOf('.');
if (lastDotIndex != -1) {
String name = filename.substring(0, lastDotIndex);
String extension = filename.substring(lastDotIndex + 1);
return new FontResource(name, extension);
}
}
return null;
} catch (Exception e) {
throw ExceptionUtils.createRuntimeException(
"error.fontLoadingFailed",
"Error processing font file",
e);
}
})
.filter(Objects::nonNull)
.toList();
} catch (Exception e) {
throw ExceptionUtils.createRuntimeException(
"error.fontDirectoryReadFailed", "Failed to read font directory", e);
}
}
// Data classes
@Data
public static class HomeData {
private boolean showSurveyFromDocker;
}
@Data
public static class LicensesData {
private List<Dependency> dependencies;
}
@Data
public static class PipelineData {
private List<Map<String, String>> pipelineConfigsWithNames;
private List<String> pipelineConfigs;
}
@Data
public static class SignData {
private List<SignatureFile> signatures;
private List<FontResource> fonts;
}
@Data
public static class OcrData {
private List<String> languages;
}
@Data
public static class FontResource {
private String name;
private String extension;
private String type;
public FontResource(String name, String extension) {
this.name = name;
this.extension = extension;
this.type = getFormatFromExtension(extension);
}
private static String getFormatFromExtension(String extension) {
switch (extension) {
case "ttf":
return "truetype";
case "woff":
return "woff";
case "woff2":
return "woff2";
case "eot":
return "embedded-opentype";
case "svg":
return "svg";
default:
return "";
}
}
}
}
@@ -1,5 +1,7 @@
package stirling.software.SPDF.controller.api.converters;
import stirling.software.common.annotations.AutoJobPostMapping;
import java.io.IOException;
import java.nio.charset.StandardCharsets;
@@ -23,7 +25,6 @@ import lombok.extern.slf4j.Slf4j;
import stirling.software.common.configuration.RuntimePathConfig;
import stirling.software.common.model.api.converters.EmlToPdfRequest;
import stirling.software.common.service.CustomPDFDocumentFactory;
import stirling.software.common.util.CustomHtmlSanitizer;
import stirling.software.common.util.EmlToPdf;
import stirling.software.common.util.TempFileManager;
import stirling.software.common.util.WebResponseUtils;
@@ -38,9 +39,8 @@ public class ConvertEmlToPDF {
private final CustomPDFDocumentFactory pdfDocumentFactory;
private final RuntimePathConfig runtimePathConfig;
private final TempFileManager tempFileManager;
private final CustomHtmlSanitizer customHtmlSanitizer;
@PostMapping(consumes = "multipart/form-data", value = "/eml/pdf")
@AutoJobPostMapping(consumes = "multipart/form-data", value = "/eml/pdf")
@Operation(
summary = "Convert EML to PDF",
description =
@@ -105,9 +105,9 @@ public class ConvertEmlToPDF {
request,
fileBytes,
originalFilename,
false,
pdfDocumentFactory,
tempFileManager,
customHtmlSanitizer);
tempFileManager);
if (pdfBytes == null || pdfBytes.length == 0) {
log.error("PDF conversion failed - empty output for {}", originalFilename);
@@ -1,5 +1,7 @@
package stirling.software.SPDF.controller.api.converters;
import stirling.software.common.annotations.AutoJobPostMapping;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.ModelAttribute;
import org.springframework.web.bind.annotation.PostMapping;
@@ -14,9 +16,9 @@ import io.swagger.v3.oas.annotations.tags.Tag;
import lombok.RequiredArgsConstructor;
import stirling.software.common.configuration.RuntimePathConfig;
import stirling.software.common.model.ApplicationProperties;
import stirling.software.common.model.api.converters.HTMLToPdfRequest;
import stirling.software.common.service.CustomPDFDocumentFactory;
import stirling.software.common.util.CustomHtmlSanitizer;
import stirling.software.common.util.ExceptionUtils;
import stirling.software.common.util.FileToPdf;
import stirling.software.common.util.TempFileManager;
@@ -30,13 +32,13 @@ public class ConvertHtmlToPDF {
private final CustomPDFDocumentFactory pdfDocumentFactory;
private final ApplicationProperties applicationProperties;
private final RuntimePathConfig runtimePathConfig;
private final TempFileManager tempFileManager;
private final CustomHtmlSanitizer customHtmlSanitizer;
@PostMapping(consumes = "multipart/form-data", value = "/html/pdf")
@AutoJobPostMapping(consumes = "multipart/form-data", value = "/html/pdf")
@Operation(
summary = "Convert an HTML or ZIP (containing HTML and CSS) to PDF",
description =
@@ -57,14 +59,17 @@ public class ConvertHtmlToPDF {
"error.fileFormatRequired", "File must be in {0} format", ".html or .zip");
}
boolean disableSanitize =
Boolean.TRUE.equals(applicationProperties.getSystem().getDisableSanitize());
byte[] pdfBytes =
FileToPdf.convertHtmlToPdf(
runtimePathConfig.getWeasyPrintPath(),
request,
fileInput.getBytes(),
originalFilename,
tempFileManager,
customHtmlSanitizer);
disableSanitize,
tempFileManager);
pdfBytes = pdfDocumentFactory.createNewBytesBasedOnOldDocument(pdfBytes);
@@ -1,5 +1,7 @@
package stirling.software.SPDF.controller.api.converters;
import stirling.software.common.annotations.AutoJobPostMapping;
import java.io.ByteArrayOutputStream;
import java.io.File;
import java.io.FileOutputStream;
@@ -51,7 +53,7 @@ public class ConvertImgPDFController {
private final CustomPDFDocumentFactory pdfDocumentFactory;
@PostMapping(consumes = "multipart/form-data", value = "/pdf/img")
@AutoJobPostMapping(consumes = "multipart/form-data", value = "/pdf/img")
@Operation(
summary = "Convert PDF to image(s)",
description =
@@ -211,7 +213,7 @@ public class ConvertImgPDFController {
}
}
@PostMapping(consumes = "multipart/form-data", value = "/img/pdf")
@AutoJobPostMapping(consumes = "multipart/form-data", value = "/img/pdf")
@Operation(
summary = "Convert images to a PDF file",
description =
@@ -1,5 +1,7 @@
package stirling.software.SPDF.controller.api.converters;
import stirling.software.common.annotations.AutoJobPostMapping;
import java.util.List;
import java.util.Map;
@@ -24,9 +26,9 @@ import io.swagger.v3.oas.annotations.tags.Tag;
import lombok.RequiredArgsConstructor;
import stirling.software.common.configuration.RuntimePathConfig;
import stirling.software.common.model.ApplicationProperties;
import stirling.software.common.model.api.GeneralFile;
import stirling.software.common.service.CustomPDFDocumentFactory;
import stirling.software.common.util.CustomHtmlSanitizer;
import stirling.software.common.util.ExceptionUtils;
import stirling.software.common.util.FileToPdf;
import stirling.software.common.util.TempFileManager;
@@ -39,13 +41,13 @@ import stirling.software.common.util.WebResponseUtils;
public class ConvertMarkdownToPdf {
private final CustomPDFDocumentFactory pdfDocumentFactory;
private final ApplicationProperties applicationProperties;
private final RuntimePathConfig runtimePathConfig;
private final TempFileManager tempFileManager;
private final CustomHtmlSanitizer customHtmlSanitizer;
@PostMapping(consumes = "multipart/form-data", value = "/markdown/pdf")
@AutoJobPostMapping(consumes = "multipart/form-data", value = "/markdown/pdf")
@Operation(
summary = "Convert a Markdown file to PDF",
description =
@@ -79,14 +81,17 @@ public class ConvertMarkdownToPdf {
String htmlContent = renderer.render(document);
boolean disableSanitize =
Boolean.TRUE.equals(applicationProperties.getSystem().getDisableSanitize());
byte[] pdfBytes =
FileToPdf.convertHtmlToPdf(
runtimePathConfig.getWeasyPrintPath(),
null,
htmlContent.getBytes(),
"converted.html",
tempFileManager,
customHtmlSanitizer);
disableSanitize,
tempFileManager);
pdfBytes = pdfDocumentFactory.createNewBytesBasedOnOldDocument(pdfBytes);
String outputFilename =
originalFilename.replaceFirst("[.][^.]+$", "")
@@ -1,8 +1,9 @@
package stirling.software.SPDF.controller.api.converters;
import stirling.software.common.annotations.AutoJobPostMapping;
import java.io.File;
import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.ArrayList;
@@ -27,7 +28,6 @@ import lombok.RequiredArgsConstructor;
import stirling.software.common.configuration.RuntimePathConfig;
import stirling.software.common.model.api.GeneralFile;
import stirling.software.common.service.CustomPDFDocumentFactory;
import stirling.software.common.util.CustomHtmlSanitizer;
import stirling.software.common.util.ProcessExecutor;
import stirling.software.common.util.ProcessExecutor.ProcessExecutorResult;
import stirling.software.common.util.WebResponseUtils;
@@ -40,7 +40,6 @@ public class ConvertOfficeController {
private final CustomPDFDocumentFactory pdfDocumentFactory;
private final RuntimePathConfig runtimePathConfig;
private final CustomHtmlSanitizer customHtmlSanitizer;
public File convertToPdf(MultipartFile inputFile) throws IOException, InterruptedException {
// Check for valid file extension
@@ -53,17 +52,7 @@ public class ConvertOfficeController {
// Save the uploaded file to a temporary location
Path tempInputFile =
Files.createTempFile("input_", "." + FilenameUtils.getExtension(originalFilename));
// Check if the file is HTML and apply sanitization if needed
String fileExtension = FilenameUtils.getExtension(originalFilename).toLowerCase();
if ("html".equals(fileExtension) || "htm".equals(fileExtension)) {
// Read and sanitize HTML content
String htmlContent = new String(inputFile.getBytes(), StandardCharsets.UTF_8);
String sanitizedHtml = customHtmlSanitizer.sanitize(htmlContent);
Files.write(tempInputFile, sanitizedHtml.getBytes(StandardCharsets.UTF_8));
} else {
inputFile.transferTo(tempInputFile);
}
inputFile.transferTo(tempInputFile);
// Prepare the output file path
Path tempOutputFile = Files.createTempFile("output_", ".pdf");
@@ -97,7 +86,7 @@ public class ConvertOfficeController {
return fileExtension.matches(extensionPattern);
}
@PostMapping(consumes = "multipart/form-data", value = "/file/pdf")
@AutoJobPostMapping(consumes = "multipart/form-data", value = "/file/pdf")
@Operation(
summary = "Convert a file to a PDF using LibreOffice",
description =
@@ -1,5 +1,7 @@
package stirling.software.SPDF.controller.api.converters;
import stirling.software.common.annotations.AutoJobPostMapping;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.ModelAttribute;
import org.springframework.web.bind.annotation.PostMapping;
@@ -18,7 +20,7 @@ import stirling.software.common.util.PDFToFile;
@RequestMapping("/api/v1/convert")
public class ConvertPDFToHtml {
@PostMapping(consumes = "multipart/form-data", value = "/pdf/html")
@AutoJobPostMapping(consumes = "multipart/form-data", value = "/pdf/html")
@Operation(
summary = "Convert PDF to HTML",
description =
@@ -1,5 +1,7 @@
package stirling.software.SPDF.controller.api.converters;
import stirling.software.common.annotations.AutoJobPostMapping;
import java.io.IOException;
import org.apache.pdfbox.pdmodel.PDDocument;
@@ -34,7 +36,7 @@ public class ConvertPDFToOffice {
private final CustomPDFDocumentFactory pdfDocumentFactory;
@PostMapping(consumes = "multipart/form-data", value = "/pdf/presentation")
@AutoJobPostMapping(consumes = "multipart/form-data", value = "/pdf/presentation")
@Operation(
summary = "Convert PDF to Presentation format",
description =
@@ -49,7 +51,7 @@ public class ConvertPDFToOffice {
return pdfToFile.processPdfToOfficeFormat(inputFile, outputFormat, "impress_pdf_import");
}
@PostMapping(consumes = "multipart/form-data", value = "/pdf/text")
@AutoJobPostMapping(consumes = "multipart/form-data", value = "/pdf/text")
@Operation(
summary = "Convert PDF to Text or RTF format",
description =
@@ -77,7 +79,7 @@ public class ConvertPDFToOffice {
}
}
@PostMapping(consumes = "multipart/form-data", value = "/pdf/word")
@AutoJobPostMapping(consumes = "multipart/form-data", value = "/pdf/word")
@Operation(
summary = "Convert PDF to Word document",
description =
@@ -91,7 +93,7 @@ public class ConvertPDFToOffice {
return pdfToFile.processPdfToOfficeFormat(inputFile, outputFormat, "writer_pdf_import");
}
@PostMapping(consumes = "multipart/form-data", value = "/pdf/xml")
@AutoJobPostMapping(consumes = "multipart/form-data", value = "/pdf/xml")
@Operation(
summary = "Convert PDF to XML",
description =
@@ -1,5 +1,7 @@
package stirling.software.SPDF.controller.api.converters;
import stirling.software.common.annotations.AutoJobPostMapping;
import java.awt.Color;
import java.io.ByteArrayOutputStream;
import java.io.File;
@@ -78,7 +80,7 @@ import stirling.software.common.util.WebResponseUtils;
@Tag(name = "Convert", description = "Convert APIs")
public class ConvertPDFToPDFA {
@PostMapping(consumes = "multipart/form-data", value = "/pdf/pdfa")
@AutoJobPostMapping(consumes = "multipart/form-data", value = "/pdf/pdfa")
@Operation(
summary = "Convert a PDF to a PDF/A",
description =
@@ -1,5 +1,7 @@
package stirling.software.SPDF.controller.api.converters;
import stirling.software.common.annotations.AutoJobPostMapping;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
@@ -40,7 +42,7 @@ public class ConvertWebsiteToPDF {
private final RuntimePathConfig runtimePathConfig;
private final ApplicationProperties applicationProperties;
@PostMapping(consumes = "multipart/form-data", value = "/url/pdf")
@AutoJobPostMapping(consumes = "multipart/form-data", value = "/url/pdf")
@Operation(
summary = "Convert a URL to a PDF",
description =
@@ -1,5 +1,7 @@
package stirling.software.SPDF.controller.api.converters;
import stirling.software.common.annotations.AutoJobPostMapping;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.StringWriter;
@@ -46,7 +48,7 @@ public class ExtractCSVController {
private final CustomPDFDocumentFactory pdfDocumentFactory;
@PostMapping(value = "/pdf/csv", consumes = "multipart/form-data")
@AutoJobPostMapping(value = "/pdf/csv", consumes = "multipart/form-data")
@Operation(
summary = "Extracts a CSV document from a PDF",
description =
@@ -1,5 +1,7 @@
package stirling.software.SPDF.controller.api.filters;
import stirling.software.common.annotations.AutoJobPostMapping;
import java.io.IOException;
import org.apache.pdfbox.pdmodel.PDDocument;
@@ -37,7 +39,7 @@ public class FilterController {
private final CustomPDFDocumentFactory pdfDocumentFactory;
@PostMapping(consumes = "multipart/form-data", value = "/filter-contains-text")
@AutoJobPostMapping(consumes = "multipart/form-data", value = "/filter-contains-text")
@Operation(
summary = "Checks if a PDF contains set text, returns true if does",
description = "Input:PDF Output:Boolean Type:SISO")
@@ -55,7 +57,7 @@ public class FilterController {
}
// TODO
@PostMapping(consumes = "multipart/form-data", value = "/filter-contains-image")
@AutoJobPostMapping(consumes = "multipart/form-data", value = "/filter-contains-image")
@Operation(
summary = "Checks if a PDF contains an image",
description = "Input:PDF Output:Boolean Type:SISO")
@@ -71,7 +73,7 @@ public class FilterController {
return null;
}
@PostMapping(consumes = "multipart/form-data", value = "/filter-page-count")
@AutoJobPostMapping(consumes = "multipart/form-data", value = "/filter-page-count")
@Operation(
summary = "Checks if a PDF is greater, less or equal to a setPageCount",
description = "Input:PDF Output:Boolean Type:SISO")
@@ -104,7 +106,7 @@ public class FilterController {
return null;
}
@PostMapping(consumes = "multipart/form-data", value = "/filter-page-size")
@AutoJobPostMapping(consumes = "multipart/form-data", value = "/filter-page-size")
@Operation(
summary = "Checks if a PDF is of a certain size",
description = "Input:PDF Output:Boolean Type:SISO")
@@ -147,7 +149,7 @@ public class FilterController {
return null;
}
@PostMapping(consumes = "multipart/form-data", value = "/filter-file-size")
@AutoJobPostMapping(consumes = "multipart/form-data", value = "/filter-file-size")
@Operation(
summary = "Checks if a PDF is a set file size",
description = "Input:PDF Output:Boolean Type:SISO")
@@ -180,7 +182,7 @@ public class FilterController {
return null;
}
@PostMapping(consumes = "multipart/form-data", value = "/filter-page-rotation")
@AutoJobPostMapping(consumes = "multipart/form-data", value = "/filter-page-rotation")
@Operation(
summary = "Checks if a PDF is of a certain rotation",
description = "Input:PDF Output:Boolean Type:SISO")

Some files were not shown because too many files have changed in this diff Show More