c141a15215e4cb767ffe42ccf9dc27225acd2847
11
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
d23c2eaa30 |
feat: Auto-redact to support text removal on True PDFs/non-custom encoded PDFs, JUnit tests for RedactController, and TextFinder (#3936)
# Description of Changes ## Overview This enhancement adds **true PDF text removal** to RedactController. It changes auto-redaction from visual covering to actual text removal. The feature removes text from True PDFs completely while keeping compatibility with other PDF types. ## Features ### 1. True PDF Text Removal - Removes text from PDF structure instead of just hiding it - No impact to manual redaction or other types of PDFs (e.g.: to searchable PDFs or custom encoded PDFs) ### 2. Advanced Content Stream Processing #### How It Works (only high level overview) - Token Processing: Breaks PDF content into small pieces for exact text finding - Font Tracking: Keeps track of fonts and formatting - Text Operators: Finds PDF commands that show text (`Tj`, `TJ`, `'`, `"`) - Position Mapping: Maps text to exact locations for removal - Rebuilds PDF: Rebuilds PDFs without the text, while keeping formatting operators #### No change for other types PDFs - Because the iteration through the PDF for token/text removal and for box placing are two separate completely methods - This means when the there is custom encoded PDF the token/text removal won't find any text to remove (because there is no logic for decoding for, for now) but the box finding methods still reliably finds redacted words and puts a box onto them. So no change. ### 3. Enhanced TextFinder Integration #### Minor Improvements - Page Grouping: Groups found text by page for faster processing ### JUnit tests for both of files. - Added JUnit tests for both files. - Might need future improvement. ### TODOs - Support for additional PDF types besides true PDFs (currently a WIP), e.g.: searchable PDF/custom encoded PDF - Feature to be expected in few weeks (best case scenario, and only if I succeed), sadly that is significantly harder task so only true PDFs for now ### UI - No UI change for now ### Sample files: [Free_Test_Data_500KB_PDF_redacted.pdf](https://github.com/user-attachments/files/21195841/Free_Test_Data_500KB_PDF_redacted.pdf) [lorem-ipsum_redacted.pdf](https://github.com/user-attachments/files/21195842/lorem-ipsum_redacted.pdf) [true-pdf-sample-1_redacted.pdf](https://github.com/user-attachments/files/21195843/true-pdf-sample-1_redacted.pdf) [true-pdf-sample-2_redacted.pdf](https://github.com/user-attachments/files/21195844/true-pdf-sample-2_redacted.pdf) [true-pdf-sample-3_redacted.pdf](https://github.com/user-attachments/files/21195845/true-pdf-sample-3_redacted.pdf) Closes: does not actually close any issues, since it only works with true PDFs --- ## Checklist ### General - [x] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [x] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/DeveloperGuide.md) (if applicable) - [ ] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md) (if applicable) - [x] I have performed a self-review of my own code - [x] My changes generate no new warnings ### Documentation - [ ] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) (if functionality has heavily changed) - [ ] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags) (for new translation tags only) ### UI Changes (if applicable) - [ ] Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR) ### Testing (if applicable) - [x] I have tested my changes locally. Refer to the [Testing Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/DeveloperGuide.md#6-testing) for more details. --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Anthony Stirling <77850077+Frooodle@users.noreply.github.com> |
||
|
LudyandGitHub
|
299d52c517 |
refactor: move modules under app/ directory and update file paths (#3938)
# Description of Changes - **What was changed:** - Renamed top-level directories: `stirling-pdf` → `app/core`, `common` → `app/common`, `proprietary` → `app/proprietary`. - Updated all path references in `.gitattributes`, GitHub workflows (`.github/workflows/*`), scripts (`.github/scripts/*`), `.gitignore`, Dockerfiles, license files, and template settings to reflect the new structure. - Added a new CI job `check-generateOpenApiDocs` to generate and upload OpenAPI documentation. - Removed redundant `@Autowired` annotations from `TempFileShutdownHook` and `UnlockPDFFormsController`. - Minor formatting and comment adjustments in YAML templates and resource files. - **Why the change was made:** - To introduce a clear `app/` directory hierarchy for core, common, and proprietary modules, improving organization and maintainability. - To ensure continuous integration and Docker builds continue to work seamlessly with the reorganized structure. - To automate OpenAPI documentation generation as part of the CI pipeline. --- ## Checklist ### General - [x] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [x] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/DeveloperGuide.md) (if applicable) - [ ] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md) (if applicable) - [x] I have performed a self-review of my own code - [x] My changes generate no new warnings ### Documentation - [ ] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) (if functionality has heavily changed) - [ ] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/HowToAddNewLanguage.md#add-new-translation-tags) (for new translation tags only) ### UI Changes (if applicable) - [ ] Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR) ### Testing (if applicable) - [ ] I have tested my changes locally. Refer to the [Testing Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/devGuide/DeveloperGuide.md#6-testing) for more details. |
||
|
|
bbf5d5f6d4 |
Support multi-file async job results and ZIP extraction (#3922)
# Description of Changes
This PR introduces multi-file support for asynchronous jobs in the
Stirling PDF backend, enabling jobs to return and manage multiple result
files. Previously, job results were limited to a single file represented
by fileId, originalFileName, and contentType. This change replaces that
legacy structure with a new ResultFile abstraction and expands the
functionality throughout the core system.
ZIP File Support
If a job result is a ZIP file:
It is automatically unpacked using buffered streaming.
Each contained file is stored individually and recorded as a ResultFile.
The original ZIP is deleted after successful extraction.
If ZIP extraction fails, the job result is treated as a single file.
New and Updated API Endpoints
1. GET /api/v1/general/job/{jobId}/result
If the job has multiple files → returns a JSON metadata list.
If the job has a single file → streams the file directly.
Includes UTF-8-safe Content-Disposition headers for filename support.
2. GET /api/v1/general/job/{jobId}/result/files
New endpoint that returns:
```json
{
"jobId": "123",
"fileCount": 2,
"files": [
{
"fileId": "abc",
"fileName": "page1.pdf",
"contentType": "application/pdf",
"fileSize": 12345
},
...
]
}
```
3. GET /api/v1/general/files/{fileId}/metadata
Returns metadata for a specific file:
4. GET /api/v1/general/files/{fileId}
Downloads a file by fileId, using metadata to determine filename and
content type.
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: pixeebot[bot] <104101892+pixeebot[bot]@users.noreply.github.com>
|
||
|
|
dc6823d7ba |
exception handling and exception improvements (#3858)
# Description of Changes This pull request introduces several improvements to enhance error handling, internationalization, and documentation in the codebase. The key changes include the addition of `ExceptionUtils` and `I18nUtils` utility classes for consistent exception handling and internationalized messages, updates to documentation paths, and modifications to existing methods to integrate the new utilities. ### Error Handling Enhancements: * **Added `ExceptionUtils` utility class**: Provides standardized methods for creating and handling exceptions with internationalized error messages, including specific handling for PDF corruption, encryption issues, and other file-related errors. * **Integrated `ExceptionUtils` into `CustomPDFDocumentFactory`**: Updated `loadFromFile` and `loadFromBytes` methods to log and handle exceptions using `ExceptionUtils`, ensuring consistent error handling across PDF operations. [[1]](diffhunk://#diff-10208c1fc2e04631a8cf2a2a99b2a1160e532e75a7b840ad752f3b0130b89851R358-R363) [[2]](diffhunk://#diff-10208c1fc2e04631a8cf2a2a99b2a1160e532e75a7b840ad752f3b0130b89851R375-R381) * **Updated `FileToPdf` to use `ExceptionUtils`**: Replaced direct exception throwing with `ExceptionUtils.createHtmlFileRequiredException` for unsupported file formats. ### Internationalization Improvements: * **Added `I18nUtils` utility class**: Centralized access to Spring's `MessageSource` for retrieving localized messages, enabling consistent internationalization across the application. ### Documentation Updates: * **Updated documentation paths in `CONTRIBUTING.md` and `README.md`**: Changed paths to reference the new `devGuide` folder for developer documentation and translation guides. [[1]](diffhunk://#diff-eca12c0a30e25b4b46522ebf89465a03ba72a03f540796c979137931d8f92055L28-R28) [[2]](diffhunk://#diff-eca12c0a30e25b4b46522ebf89465a03ba72a03f540796c979137931d8f92055L40-R51) [[3]](diffhunk://#diff-b335630551682c19a781afebcf4d07bf978fb1f8ac04c6bf87428ed5106870f5L171-L174) --- ## Checklist ### General - [ ] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [ ] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [ ] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/HowToAddNewLanguage.md) (if applicable) - [ ] I have performed a self-review of my own code - [ ] My changes generate no new warnings ### Documentation - [ ] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) (if functionality has heavily changed) - [ ] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/HowToAddNewLanguage.md#add-new-translation-tags) (for new translation tags only) ### UI Changes (if applicable) - [ ] Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR) ### Testing (if applicable) - [ ] I have tested my changes locally. Refer to the [Testing Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#6-testing) for more details. --------- Co-authored-by: a <a> |
||
|
|
14d4fbb2a3 |
🤖 format everything with pre-commit by stirlingbot (#3804)
Auto-generated by [create-pull-request][1] with **stirlingbot** [1]: https://github.com/peter-evans/create-pull-request Signed-off-by: stirlingbot[bot] <stirlingbot[bot]@users.noreply.github.com> Co-authored-by: stirlingbot[bot] <195170888+stirlingbot[bot]@users.noreply.github.com> |
||
|
Dario Ghunney WareandGitHub
|
32aa568196 |
Add Attachments Feature (#3781)
# Description of Changes Added a new feature to add attachments to a PDF document. ### Added: - `AttachmentController`: Endpoint for adding attachments at `/add-attachments` with parameters `fileInput` for the PDF and `attachments` as a list of files to attach - `AttachmentServiceInterface` - `AttachmentService`: Handles the logic of adding attachments to the PDF - `AttachmentUtils`: to handle setting the catalog viewer preferences in the viewer - Add Attachments page - Tests for new feature ### Changes: - `EmlToPdf`: Moved setting of viewer preferences to `AttachmentUtils` - `EndpointConfiguration: Included '/add-attachments' - Updated language files with attachments copy - General clean up Closes #1259 --- ## Checklist ### General - [x] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [x] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [x] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/HowToAddNewLanguage.md) (if applicable) - [x] I have performed a self-review of my own code - [x] My changes generate no new warnings ### Documentation - [x] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) (if functionality has heavily changed) - [x] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/HowToAddNewLanguage.md#add-new-translation-tags) (for new translation tags only) ### UI Changes (if applicable)     - [x] Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR) ### Testing (if applicable) - [x] I have tested my changes locally. Refer to the [Testing Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#6-testing) for more details. |
||
|
|
7d7f1272e4 |
Async (#3773)
# Description of Changes This pull request introduces a job management system with enhanced capabilities for handling asynchronous tasks, file operations, and progress tracking. Key changes include the addition of new annotations and aspects for job execution, file management services, and models for job progress and results. ### Job Execution Enhancements: * [`common/src/main/java/stirling/software/common/annotations/AutoJobPostMapping.java`](diffhunk://#diff-570304f67b974d5bd30a28d05d34759b86bcb4a35148d779e2b46904e8dd2904R1-R47): Added a custom annotation to simplify job handling for POST requests, including support for retries, progress tracking, and resource management. * [`common/src/main/java/stirling/software/common/aop/AutoJobAspect.java`](diffhunk://#diff-5f725b1d99dbc47dfe9b1d07f37382ca7c81d587725dc35a62c644d1a25f9869R1-R231): Implemented an aspect to integrate job execution logic, handling retries, asynchronous processing, and file management seamlessly. ### File Management: * [`common/src/main/java/stirling/software/common/service/FileStorage.java`](diffhunk://#diff-f382e12c197ad6f7c5b01b1cea912e9b141a4b4e4ab7f12baafa1b69cb112962R1-R152): Added a service for storing, retrieving, and managing files using unique IDs, enabling persistent file handling for jobs. * [`common/src/main/java/stirling/software/common/service/FileOrUploadService.java`](diffhunk://#diff-e0637404eea2b1c1413cf5f3247208a9196b14388a90a896314d3e9c2949c893R1-R78): Added utility methods for converting files to `MultipartFile` and resolving file paths. ### Job Models: * [`common/src/main/java/stirling/software/common/model/job/JobProgress.java`](diffhunk://#diff-edc765f0e32ef4cb5a03dd3badafad450336a5248221ecc27976eb692280f003R1-R15): Introduced a model to represent job progress, including completion percentage and status messages. * [`common/src/main/java/stirling/software/common/model/job/JobResult.java`](diffhunk://#diff-b34316aa0ebfd849f41086339ae0323cb5cc2066b8200c38c6a39564e17b88f3R1-R94): Added a model to encapsulate job results, supporting both file-based and object-based outcomes. * [`common/src/main/java/stirling/software/common/model/job/JobResponse.java`](diffhunk://#diff-b02e9f86d44beda10ceb66650c79d1e032acd6f6a609887fb5f5596713048ab1R1-R14): Created a model for job responses, including async execution details and job IDs. * [`common/src/main/java/stirling/software/common/model/job/JobStats.java`](diffhunk://#diff-6067e6bd9e44d9dc40419d2435fa24d6753ec51e3baf7967dbcbc1a51e95e8afR1-R43): Added a model for tracking job statistics, such as total jobs, success rates, and average processing times. ### Other Changes: * [`common/src/main/java/stirling/software/common/model/api/PDFFile.java`](diffhunk://#diff-d2419d05a852acf8f8d0bd5c3673bbdd8e385b2d5cf1d80fbd8b66691ebd2cb2L17-R24): Updated the `PDFFile` model to include a `fileId` field for server-side file references, enhancing flexibility in file handling. * [`common/build.gradle`](diffhunk://#diff-824c1e8ad11e20caed0bec7162a99779b9a4bcf1178d99fae3e39f69889f8959R31): Added the `spring-boot-starter-aop` dependency to enable aspect-oriented programming. --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: a <a> |
||
|
|
a49eb3a629 |
🤖 format everything with pre-commit by stirlingbot (#3697)
Auto-generated by [create-pull-request][1] with **stirlingbot** [1]: https://github.com/peter-evans/create-pull-request Signed-off-by: stirlingbot[bot] <stirlingbot[bot]@users.noreply.github.com> Co-authored-by: stirlingbot[bot] <195170888+stirlingbot[bot]@users.noreply.github.com> |
||
|
LudyandGitHub
|
5393ae24cb |
Fix: Assign Internal API User to Internal Team and Eager-Load User’s Team Association (#3698)
# Description of Changes - **What was changed** - Added logic in `InitialSecuritySetup` to assign the `INTERNAL_API_USER` to a dedicated “internal” team both during initial creation and on subsequent startups. - Enhanced `assignUsersToDefaultTeamIfMissing()` to route the internal API user to the `internalTeam`, while all other users go to the default team. - Switched the JPA mapping of `User.team` from `LAZY` to `EAGER` fetch to ensure the team association is always loaded with the user. - Introduced a new `UserService.changeUserTeam(User, Team)` method to handle moving an existing user to a different team and persist the change. - Imported `java.util.Optional` to safely handle lookups of the internal API user. - **Why the change was made** - To guarantee that the special internal API user is always part of the “internal” team and never left on the default team, preventing permission and routing issues. - Eagerly loading the `team` association avoids lazy-init exceptions in contexts where the user’s team is needed immediately (e.g., security checks). --- ## Checklist ### General - [x] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [x] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [ ] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/HowToAddNewLanguage.md) (if applicable) - [x] I have performed a self-review of my own code - [x] My changes generate no new warnings ### Documentation - [ ] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) (if functionality has heavily changed) - [ ] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/HowToAddNewLanguage.md#add-new-translation-tags) (for new translation tags only) ### UI Changes (if applicable) - [ ] Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR) ### Testing (if applicable) - [ ] I have tested my changes locally. Refer to the [Testing Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#6-testing) for more details. |
||
|
|
1f2365f03c |
Ensure Pixel gets disabled, PDF ToC support (#3659)
# Description of Changes Please provide a summary of the changes, including: - What was changed - Why the change was made - Any challenges encountered Closes #(issue_number) --- ## Checklist ### General - [ ] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [ ] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [ ] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/HowToAddNewLanguage.md) (if applicable) - [ ] I have performed a self-review of my own code - [ ] My changes generate no new warnings ### Documentation - [ ] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) (if functionality has heavily changed) - [ ] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/HowToAddNewLanguage.md#add-new-translation-tags) (for new translation tags only) ### UI Changes (if applicable) - [ ] Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR) ### Testing (if applicable) - [ ] I have tested my changes locally. Refer to the [Testing Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#6-testing) for more details. --------- Co-authored-by: Dario Ghunney Ware <dariogware@gmail.com> Co-authored-by: Connor Yoh <con.yoh13@gmail.com> Co-authored-by: a <a> Co-authored-by: Reece <reecebrowne1995@gmail.com> |
||
|
Dario Ghunney WareandGitHub
|
c7d6a063d7 |
Multi module refactor (#3640)
# Description of Changes Migrated Stirling PDF to a multi-module structure: * Introduced new `:stirling-pdf` module * Moved all the core logic and features of Stirling PDF into `:stirling-pdf` * Updated paths of jobs and scripts --- ## Checklist ### General - [ ] I have read the [Contribution Guidelines](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/CONTRIBUTING.md) - [ ] I have read the [Stirling-PDF Developer Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md) (if applicable) - [ ] I have read the [How to add new languages to Stirling-PDF](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/HowToAddNewLanguage.md) (if applicable) - [ ] I have performed a self-review of my own code - [ ] My changes generate no new warnings ### Documentation - [ ] I have updated relevant docs on [Stirling-PDF's doc repo](https://github.com/Stirling-Tools/Stirling-Tools.github.io/blob/main/docs/) (if functionality has heavily changed) - [ ] I have read the section [Add New Translation Tags](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/HowToAddNewLanguage.md#add-new-translation-tags) (for new translation tags only) ### UI Changes (if applicable) - [ ] Screenshots or videos demonstrating the UI changes are attached (e.g., as comments or direct attachments in the PR) ### Testing (if applicable) - [ ] I have tested my changes locally. Refer to the [Testing Guide](https://github.com/Stirling-Tools/Stirling-PDF/blob/main/DeveloperGuide.md#6-testing) for more details. |